2024-09-21T11:23:29.637 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T11:23:29.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T11:23:29.746 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913860 branch: wip-guits-main-2024-09-21-0813 description: orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} email: gabrioux@ibm.com first_in_suite: false job_id: '7913860' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-main-2024-09-21-0813 ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms bind msgr1: false ms bind msgr2: true ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) log-only-match: - CEPHADM_ sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 workunit: branch: wip-guits-main-2024-09-21-0813 sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 3041 sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-main-2024-09-21-0813 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_5c1cdc3c470dae78e91a0ed2bcf13509a046d301/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbfcHIwVmKpDcnkUtpZJ428CjS7OYTPB7DjiSK+Ar9VHNF3Cw0O9529AiKxsjdN5JTxwjP/Ga8EHuD3MOkr5XE= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOWIrKA0fEM/x1LyZCVfp33gYAqez45n6a6ys7PuXU7mz0VLNF0xH4MIAySILiwhYpe8KNQcDK1WufRDyjmxiKo= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-21_10:07:29 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T11:23:29.746 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_5c1cdc3c470dae78e91a0ed2bcf13509a046d301/qa; will attempt to use it 2024-09-21T11:23:29.747 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_5c1cdc3c470dae78e91a0ed2bcf13509a046d301/qa/tasks 2024-09-21T11:23:29.747 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T11:23:29.748 INFO:teuthology.task.internal:Checking packages... 2024-09-21T11:23:29.767 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5c1cdc3c470dae78e91a0ed2bcf13509a046d301' 2024-09-21T11:23:29.767 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T11:23:29.768 INFO:teuthology.packaging:ref: None 2024-09-21T11:23:29.768 INFO:teuthology.packaging:tag: None 2024-09-21T11:23:29.768 INFO:teuthology.packaging:branch: wip-guits-main-2024-09-21-0813 2024-09-21T11:23:29.768 INFO:teuthology.packaging:sha1: 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:23:29.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-main-2024-09-21-0813 2024-09-21T11:23:29.907 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5103.g5c1cdc3c 2024-09-21T11:23:29.908 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T11:23:29.914 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T11:23:29.914 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T11:23:29.921 INFO:teuthology.task.internal:Saving configuration 2024-09-21T11:23:29.933 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T11:23:29.939 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T11:23:29.964 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913860', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 11:18:35.205003', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbfcHIwVmKpDcnkUtpZJ428CjS7OYTPB7DjiSK+Ar9VHNF3Cw0O9529AiKxsjdN5JTxwjP/Ga8EHuD3MOkr5XE='} 2024-09-21T11:23:29.987 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913860', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 11:18:35.206512', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOWIrKA0fEM/x1LyZCVfp33gYAqez45n6a6ys7PuXU7mz0VLNF0xH4MIAySILiwhYpe8KNQcDK1WufRDyjmxiKo='} 2024-09-21T11:23:29.987 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T11:23:29.992 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-21T11:23:29.992 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-21T11:23:29.992 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T11:23:30.050 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc3093c0280>, signals=[15]) 2024-09-21T11:23:30.050 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T11:23:30.055 INFO:teuthology.task.internal:Opening connections... 2024-09-21T11:23:30.055 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-09-21T11:23:30.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:23:30.129 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T11:23:30.129 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:23:30.202 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T11:23:30.208 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-09-21T11:23:30.226 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-09-21T11:23:30.226 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-09-21T11:23:30.283 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T11:23:30.284 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T11:23:30.285 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T11:23:30.285 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-09-21T11:23:30.307 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-09-21T11:23:30.325 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-09-21T11:23:30.325 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-09-21T11:23:30.386 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-09-21T11:23:30.387 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-09-21T11:23:30.387 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-09-21T11:23:30.387 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-09-21T11:23:30.387 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-09-21T11:23:30.387 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-09-21T11:23:30.387 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T11:23:30.387 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-09-21T11:23:30.387 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-09-21T11:23:30.387 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T11:23:30.387 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-09-21T11:23:30.388 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T11:23:30.388 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T11:23:30.388 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T11:23:30.388 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-09-21T11:23:30.407 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T11:23:30.418 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T11:23:30.435 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T11:23:30.436 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-09-21T11:23:30.438 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-09-21T11:23:30.455 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T11:23:30.470 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T11:23:30.470 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-09-21T11:23:30.495 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-09-21T11:23:30.523 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T11:23:30.578 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T11:23:30.594 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-21T11:23:30.594 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-21T11:23:30.595 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T11:23:30.595 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T11:23:30.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:23:30.612 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-21T11:23:30.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:23:30.613 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-21T11:23:30.669 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-503.el9.x86_64 2024-09-21T11:23:30.669 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-503.el9.x86_64 2024-09-21T11:23:30.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-09-21T11:23:30.671 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-503.el9.x86_64 2024-09-21T11:23:30.671 INFO:teuthology.task.kernel:Running kernel on smithi175: 5.14.0-503.el9.x86_64 2024-09-21T11:23:30.671 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum install -y kernel 2024-09-21T11:23:32.966 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-21T11:23:34.336 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.3 MB 00:01 2024-09-21T11:23:34.676 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-21T11:23:36.520 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-21T11:23:40.526 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-21T11:23:41.977 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 115 kB/s | 19 kB 00:00 2024-09-21T11:23:42.045 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-21T11:23:42.739 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-21T11:23:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-21T11:23:44.300 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-21T11:23:49.737 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 1.2 kB/s | 1.7 kB 00:01 2024-09-21T11:23:49.864 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 9.0 kB/s | 1.7 kB 00:00 2024-09-21T11:23:51.718 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T11:23:51.718 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T11:23:51.787 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T11:23:51.787 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T11:23:51.815 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:23:51.819 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:23:51.820 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-21T11:23:51.820 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:23:51.820 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:23:51.820 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T11:23:51.820 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-21T11:23:51.820 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T11:23:51.820 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T11:23:51.820 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T11:23:51.820 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:23:51.821 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:23:51.821 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:23:51.821 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-21T11:23:51.821 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:23:51.822 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 79 M 2024-09-21T11:23:51.822 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 126 M 2024-09-21T11:23:51.822 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:23:51.887 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:23:51.895 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:23:51.896 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-21T11:23:51.896 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:23:51.896 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:23:51.896 INFO:teuthology.orchestra.run.smithi175.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T11:23:51.896 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T11:23:51.896 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T11:23:51.896 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T11:23:51.896 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T11:23:51.896 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:23:51.897 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:23:51.897 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:23:51.897 INFO:teuthology.orchestra.run.smithi175.stdout:Install 4 Packages 2024-09-21T11:23:51.897 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:23:51.897 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 79 M 2024-09-21T11:23:51.898 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 126 M 2024-09-21T11:23:51.898 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:23:52.023 INFO:teuthology.orchestra.run.smithi175.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 451 kB/s | 36 kB 00:00 2024-09-21T11:23:52.407 INFO:teuthology.orchestra.run.smithi175.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-21T11:23:52.538 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 58 kB/s | 36 kB 00:00 2024-09-21T11:23:52.824 INFO:teuthology.orchestra.run.smithi175.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 39 MB/s | 35 MB 00:00 2024-09-21T11:23:53.174 INFO:teuthology.orchestra.run.smithi175.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-21T11:23:53.174 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:23:53.174 INFO:teuthology.orchestra.run.smithi175.stdout:Total 62 MB/s | 79 MB 00:01 2024-09-21T11:23:53.313 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-21T11:23:53.857 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:23:53.980 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-21T11:23:54.197 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:23:54.198 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:23:55.039 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 11 MB/s | 29 MB 00:02 2024-09-21T11:23:55.039 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:23:55.039 INFO:teuthology.orchestra.run.smithi043.stdout:Total 24 MB/s | 79 MB 00:03 2024-09-21T11:23:55.180 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:23:55.181 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:23:55.752 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:23:56.133 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:23:56.133 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:23:57.129 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:23:57.130 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:23:57.624 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:23:58.309 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T11:23:58.432 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:23:59.692 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:24:00.371 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T11:24:00.458 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:24:01.215 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:24:01.351 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:24:03.227 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:24:03.351 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:24:06.222 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:24:06.366 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:24:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:11.247 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:13.665 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:49.441 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:49.480 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:49.833 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:49.833 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T11:24:49.833 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:24:49.833 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:24:50.668 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:50.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:24:50.668 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:24:50.668 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T11:24:50.668 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T11:24:50.668 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T11:24:50.669 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T11:24:50.669 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:24:50.669 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:24:51.044 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T11:24:51.120 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T11:24:51.121 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T11:24:51.121 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T11:24:51.121 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T11:24:51.121 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T11:24:51.121 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-21T11:24:51.121 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T11:24:51.121 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T11:24:51.180 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T11:24:51.181 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T11:24:51.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2024-09-21T11:24:51.219 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2024-09-21T11:24:51.219 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2024-09-21T11:24:51.219 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 82.el9 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8274670 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T11:24:51.220 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2024-09-21T11:24:51.221 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2024-09-21T11:24:51.221 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:24:51.221 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T11:24:51.221 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T11:24:51.221 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T11:24:51.221 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2024-09-21T11:24:51.221 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:24:51.221 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T11:24:51.222 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T11:24:51.222 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 5.14.0-508.el9.x86_64 2024-09-21T11:24:51.222 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T11:24:51.921 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-09-21T11:24:53.300 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T11:24:53.331 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-09-21T11:24:53.334 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-09-21T11:24:53.349 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.RoeWkVTfOO 2024-09-21T11:24:53.349 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RoeWkVTfOO 2024-09-21T11:24:53.416 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.RoeWkVTfOO 2024-09-21T11:24:53.515 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:53.552 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.RoeWkVTfOO is 6KB 2024-09-21T11:24:53.557 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:53.600 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.RoeWkVTfOO 2024-09-21T11:24:53.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T11:24:53.678 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T11:24:53.678 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T11:24:53.678 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T11:24:53.678 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T11:24:53.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T11:24:53.843 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-09-21T11:24:54.031 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:54.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T11:24:54.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T11:24:54.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T11:24:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T11:24:54.909 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:24:54.910 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:24:54.910 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T11:24:54.910 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T11:24:54.910 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T11:24:54.910 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T11:24:54.910 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:24:54.910 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:24:55.265 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T11:24:55.362 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T11:24:55.362 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T11:24:55.362 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T11:24:55.362 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T11:24:55.362 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T11:24:55.363 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-21T11:24:55.363 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T11:24:55.363 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T11:24:55.422 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T11:24:55.422 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T11:24:55.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-09-21T11:24:55.459 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-09-21T11:24:55.459 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-09-21T11:24:55.459 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-09-21T11:24:55.459 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 82.el9 2024-09-21T11:24:55.459 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8274670 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-09-21T11:24:55.460 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:24:55.461 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T11:24:55.461 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T11:24:55.461 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T11:24:55.461 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-09-21T11:24:55.461 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:24:55.461 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T11:24:55.462 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T11:24:55.462 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 5.14.0-508.el9.x86_64 2024-09-21T11:24:55.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T11:24:56.196 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-21T11:24:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T11:24:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-21T11:24:57.491 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-21T11:24:57.506 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.MGSs9dRi1s 2024-09-21T11:24:57.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MGSs9dRi1s 2024-09-21T11:24:57.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.MGSs9dRi1s 2024-09-21T11:24:57.717 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.MGSs9dRi1s is 6KB 2024-09-21T11:24:57.765 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.MGSs9dRi1s 2024-09-21T11:24:57.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T11:24:57.849 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T11:24:57.849 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T11:24:57.849 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T11:24:57.849 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T11:24:57.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T11:24:58.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-09-21T11:25:23.870 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-21T11:25:23.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T11:25:23.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:25:28.006 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-21T11:25:28.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-21T11:25:28.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:25:42.259 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-09-21T11:25:46.355 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2024-09-21T11:25:51.265 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T11:25:51.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:25:54.319 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-09-21T11:25:55.356 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-21T11:25:55.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:26:06.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T11:26:06.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:26:38.155 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-21T11:26:38.710 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T11:26:38.710 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-21T11:26:38.711 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-21T11:26:38.736 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-508.el9.x86_64 2024-09-21T11:26:38.736 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T11:26:38.736 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T11:26:38.736 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T11:26:39.737 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-21T11:26:39.737 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T11:26:39.817 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2024-09-21T11:26:39.841 DEBUG:teuthology.parallel:result is None 2024-09-21T11:26:55.360 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T11:27:07.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-21T11:27:07.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:27:07.650 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-21T11:27:08.045 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-21T11:27:08.045 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-21T11:27:08.045 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-21T11:27:08.062 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-508.el9.x86_64 2024-09-21T11:27:08.062 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T11:27:08.063 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T11:27:08.063 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T11:27:09.064 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-21T11:27:09.064 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-21T11:27:09.145 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-09-21T11:27:09.169 DEBUG:teuthology.parallel:result is None 2024-09-21T11:27:09.169 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T11:27:09.175 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T11:27:09.176 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T11:27:09.178 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T11:27:09.197 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T11:27:09.210 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T11:27:09.235 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T11:27:09.235 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T11:27:09.237 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T11:27:09.304 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T11:27:09.328 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T11:27:09.328 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T11:27:09.330 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T11:27:09.365 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T11:27:09.372 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T11:27:09.377 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T11:27:09.384 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T11:27:09.385 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T11:27:09.391 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T11:27:09.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T11:27:09.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T11:27:09.454 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T11:27:09.462 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T11:27:09.462 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T11:27:09.489 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T11:27:09.511 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T11:27:09.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T11:27:09.638 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T11:27:09.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T11:27:09.764 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:27:09.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T11:27:09.831 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T11:27:09.873 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T11:27:09.952 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T11:27:09.997 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T11:27:10.041 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:27:10.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T11:27:10.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-09-21T11:27:10.110 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-09-21T11:27:10.155 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T11:27:10.186 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T11:27:10.667 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T11:27:10.673 INFO:teuthology.task.internal:Starting timer... 2024-09-21T11:27:10.673 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T11:27:10.695 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T11:27:10.704 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-21T11:27:10.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-09-21T11:27:10.772 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-09-21T11:27:10.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-09-21T11:27:10.827 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-09-21T11:27:10.829 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T11:27:10.829 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-09-21T11:27:10.855 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-09-21T11:27:10.855 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-09-21T11:27:10.879 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-09-21T11:27:10.880 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-09-21T11:27:10.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T11:27:10.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:27:10.924 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T11:27:10.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:27:10.953 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T11:27:10.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-09-21T11:27:10.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-09-21T11:27:11.029 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T11:27:11.041 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T11:27:11.044 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T11:27:11.151 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T11:27:11.161 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T11:27:11.162 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T11:31:01.297 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-09-21T11:31:01.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-21T11:31:01.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:31:01.373 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-21T11:31:01.454 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-21T11:31:01.455 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T11:31:01.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T11:31:01.534 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-21T11:31:01.614 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T11:31:01.614 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T11:31:01.628 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T11:31:01.628 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T11:31:01.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T11:31:01.632 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T11:31:01.632 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T11:31:01.662 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T11:31:01.678 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T11:31:01.683 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T11:31:01.697 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T11:31:01.704 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-09-21T11:31:01.716 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-09-21T11:31:01.721 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-09-21T11:31:01.732 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T11:31:01.732 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-09-21T11:31:01.746 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T11:31:01.746 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T11:31:01.760 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T11:31:01.805 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-21T11:31:01.809 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T11:31:01.809 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-21T11:31:01.809 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:31:01.809 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:31:01.809 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:31:01.809 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:31:01.814 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-09-21T11:31:01.819 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T11:31:01.819 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-09-21T11:31:01.819 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:31:01.819 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:31:01.819 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:31:01.819 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T11:31:01.819 INFO:teuthology.run_tasks:Running task pexec... 2024-09-21T11:31:01.829 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-21T11:31:01.829 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi043.front.sepia.ceph.com 2024-09-21T11:31:01.829 DEBUG:teuthology.orchestra.run.smithi043:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-21T11:31:01.830 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T11:31:01.830 DEBUG:teuthology.orchestra.run.smithi175:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-21T11:31:02.209 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout:Removing: 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout:Removing unused dependencies: 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout:Remove 2 Packages 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout:Freed space: 5.5 M 2024-09-21T11:31:02.211 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:31:02.215 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:31:02.215 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:31:02.218 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:31:02.234 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:02.234 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-21T11:31:02.234 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:02.234 INFO:teuthology.orchestra.run.smithi043.stdout:Removing: 2024-09-21T11:31:02.235 INFO:teuthology.orchestra.run.smithi043.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-21T11:31:02.235 INFO:teuthology.orchestra.run.smithi043.stdout:Removing unused dependencies: 2024-09-21T11:31:02.235 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-21T11:31:02.235 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:02.235 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:31:02.235 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:02.235 INFO:teuthology.orchestra.run.smithi043.stdout:Remove 2 Packages 2024-09-21T11:31:02.235 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:02.235 INFO:teuthology.orchestra.run.smithi043.stdout:Freed space: 5.5 M 2024-09-21T11:31:02.235 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:31:02.239 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:31:02.239 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:31:02.271 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:31:02.271 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:31:02.306 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:31:02.307 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:31:02.405 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:31:02.444 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:31:02.476 INFO:teuthology.orchestra.run.smithi175.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-21T11:31:02.531 INFO:teuthology.orchestra.run.smithi175.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-21T11:31:02.531 INFO:teuthology.orchestra.run.smithi043.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-21T11:31:02.587 INFO:teuthology.orchestra.run.smithi043.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-21T11:31:02.926 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-21T11:31:02.926 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-21T11:31:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-21T11:31:03.116 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-21T11:31:03.181 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-21T11:31:03.182 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:03.182 INFO:teuthology.orchestra.run.smithi175.stdout:Removed: 2024-09-21T11:31:03.182 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-21T11:31:03.182 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:03.182 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:31:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-21T11:31:03.394 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:03.394 INFO:teuthology.orchestra.run.smithi043.stdout:Removed: 2024-09-21T11:31:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-21T11:31:03.395 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:03.395 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:31:04.009 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:56 ago on Sat 21 Sep 2024 11:30:07 AM UTC. 2024-09-21T11:31:04.144 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:31:04.144 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:04.144 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-21T11:31:04.144 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:04.144 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout:Install 6 Packages 2024-09-21T11:31:04.145 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:04.151 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 2.0 M 2024-09-21T11:31:04.151 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 9.2 M 2024-09-21T11:31:04.151 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:31:04.222 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:50 ago on Sat 21 Sep 2024 11:30:13 AM UTC. 2024-09-21T11:31:04.358 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:04.359 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:31:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:04.360 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-09-21T11:31:04.360 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.0 M 2024-09-21T11:31:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 9.2 M 2024-09-21T11:31:04.365 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:31:04.380 INFO:teuthology.orchestra.run.smithi175.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 437 kB/s | 44 kB 00:00 2024-09-21T11:31:04.405 INFO:teuthology.orchestra.run.smithi175.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 836 kB/s | 106 kB 00:00 2024-09-21T11:31:04.439 INFO:teuthology.orchestra.run.smithi175.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 1.2 MB/s | 72 kB 00:00 2024-09-21T11:31:04.481 INFO:teuthology.orchestra.run.smithi175.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2024-09-21T11:31:04.514 INFO:teuthology.orchestra.run.smithi175.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.7 MB/s | 906 kB 00:00 2024-09-21T11:31:04.640 INFO:teuthology.orchestra.run.smithi175.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.1 MB/s | 837 kB 00:00 2024-09-21T11:31:04.640 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:31:04.640 INFO:teuthology.orchestra.run.smithi175.stdout:Total 4.1 MB/s | 2.0 MB 00:00 2024-09-21T11:31:04.713 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:31:04.726 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:31:04.726 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:31:04.732 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 536 kB/s | 44 kB 00:00 2024-09-21T11:31:04.757 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 981 kB/s | 106 kB 00:00 2024-09-21T11:31:04.782 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 1.4 MB/s | 72 kB 00:00 2024-09-21T11:31:04.816 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.4 MB/s | 84 kB 00:00 2024-09-21T11:31:04.858 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.2 MB/s | 906 kB 00:00 2024-09-21T11:31:04.900 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 7.0 MB/s | 837 kB 00:00 2024-09-21T11:31:04.900 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:31:04.900 INFO:teuthology.orchestra.run.smithi043.stdout:Total 3.7 MB/s | 2.0 MB 00:00 2024-09-21T11:31:04.978 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:31:04.993 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:31:04.993 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:31:05.019 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:31:05.020 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:31:05.268 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:31:05.268 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:31:05.445 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:31:05.533 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-21T11:31:05.577 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-21T11:31:05.605 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-21T11:31:05.712 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:31:05.803 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-21T11:31:05.811 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-21T11:31:05.845 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-21T11:31:05.853 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-21T11:31:05.874 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-21T11:31:06.097 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-21T11:31:06.130 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-21T11:31:06.290 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-21T11:31:06.322 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-21T11:31:06.609 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-21T11:31:06.641 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-21T11:31:07.289 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-21T11:31:07.289 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-21T11:31:07.289 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-21T11:31:07.289 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-21T11:31:07.289 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-21T11:31:07.289 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-21T11:31:07.618 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-21T11:31:07.618 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-21T11:31:07.618 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-21T11:31:07.618 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-21T11:31:07.619 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-21T11:31:07.619 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-21T11:31:07.628 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-21T11:31:07.628 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:07.628 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:31:07.628 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-21T11:31:07.628 INFO:teuthology.orchestra.run.smithi175.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-21T11:31:07.628 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-21T11:31:07.628 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:07.628 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:31:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-21T11:31:07.956 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:07.957 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:31:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-21T11:31:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-21T11:31:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-21T11:31:07.957 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:07.957 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:31:07.958 DEBUG:teuthology.parallel:result is None 2024-09-21T11:31:08.263 DEBUG:teuthology.parallel:result is None 2024-09-21T11:31:08.263 INFO:teuthology.run_tasks:Running task install... 2024-09-21T11:31:08.273 DEBUG:teuthology.task.install:project ceph 2024-09-21T11:31:08.273 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301'}} 2024-09-21T11:31:08.273 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301'} 2024-09-21T11:31:08.273 INFO:teuthology.task.install:Using flavor: default 2024-09-21T11:31:08.279 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T11:31:08.279 INFO:teuthology.task.install:extra packages: [] 2024-09-21T11:31:08.279 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301', 'tag': None, 'wait_for_package': False} 2024-09-21T11:31:08.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:31:08.281 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301', 'tag': None, 'wait_for_package': False} 2024-09-21T11:31:08.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:31:08.422 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/ 2024-09-21T11:31:08.423 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5103.g5c1cdc3c 2024-09-21T11:31:08.438 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/ 2024-09-21T11:31:08.438 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5103.g5c1cdc3c 2024-09-21T11:31:08.532 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T11:31:08.532 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:31:08.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T11:31:08.546 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T11:31:08.546 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:31:08.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T11:31:08.563 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T11:31:08.563 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T11:31:08.575 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-21T11:31:08.575 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T11:31:08.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T11:31:08.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T11:31:08.711 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T11:31:08.718 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T11:31:08.778 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-09-21T11:31:08.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-09-21T11:31:08.783 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-09-21T11:31:08.784 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-09-21T11:31:09.001 INFO:teuthology.orchestra.run.smithi175.stdout:45 files removed 2024-09-21T11:31:09.009 INFO:teuthology.orchestra.run.smithi043.stdout:45 files removed 2024-09-21T11:31:09.029 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw 2024-09-21T11:31:09.037 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw 2024-09-21T11:31:09.588 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 334 kB/s | 84 kB 00:00 2024-09-21T11:31:09.593 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 304 kB/s | 84 kB 00:00 2024-09-21T11:31:09.872 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 102 kB/s | 24 kB 00:00 2024-09-21T11:31:09.886 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T11:31:10.139 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-21T11:31:10.170 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-21T11:31:10.606 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.3 MB 00:00 2024-09-21T11:31:10.611 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.3 MB 00:00 2024-09-21T11:31:12.440 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-21T11:31:12.446 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-21T11:31:16.758 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-21T11:31:18.088 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-21T11:31:18.242 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 108 kB/s | 19 kB 00:00 2024-09-21T11:31:18.996 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-21T11:31:19.547 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 19 kB 00:00 2024-09-21T11:31:20.276 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 37 MB/s | 23 MB 00:00 2024-09-21T11:31:24.677 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T11:31:25.707 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-21T11:31:26.999 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:31:27.002 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-21T11:31:27.002 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:27.002 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:31:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 11 M 2024-09-21T11:31:27.002 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-09-21T11:31:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.5 M 2024-09-21T11:31:27.002 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.2 M 2024-09-21T11:31:27.002 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 5.5 M 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 22 M 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 25 k 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 755 k 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T11:31:27.003 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 509 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 5.5 M 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 45 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 148 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 161 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 321 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 306 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 99 k 2024-09-21T11:31:27.004 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout:Install 31 Packages 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 2 Packages 2024-09-21T11:31:27.005 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:27.006 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 62 M 2024-09-21T11:31:27.006 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:31:27.745 INFO:teuthology.orchestra.run.smithi043.stdout:(1/33): ceph-base-19.3.0-5103.g5c1cdc3c.el9.x86 10 MB/s | 5.5 MB 00:00 2024-09-21T11:31:27.812 INFO:teuthology.orchestra.run.smithi043.stdout:(2/33): ceph-selinux-19.3.0-5103.g5c1cdc3c.el9. 378 kB/s | 25 kB 00:00 2024-09-21T11:31:27.920 INFO:teuthology.orchestra.run.smithi043.stdout:(3/33): libcephfs2-19.3.0-5103.g5c1cdc3c.el9.x8 6.8 MB/s | 755 kB 00:00 2024-09-21T11:31:27.996 INFO:teuthology.orchestra.run.smithi043.stdout:(4/33): libradosstriper1-19.3.0-5103.g5c1cdc3c. 6.6 MB/s | 509 kB 00:00 2024-09-21T11:31:28.085 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:31:28.087 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:28.087 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T11:31:28.087 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:28.087 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 11 M 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.5 M 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.2 M 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 5.5 M 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 22 M 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 25 k 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 755 k 2024-09-21T11:31:28.088 INFO:teuthology.orchestra.run.smithi175.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 509 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 5.5 M 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 45 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 148 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 161 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T11:31:28.089 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 321 k 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 306 k 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 99 k 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout:Install 31 Packages 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 2 Packages 2024-09-21T11:31:28.090 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:28.091 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 62 M 2024-09-21T11:31:28.091 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:31:28.254 INFO:teuthology.orchestra.run.smithi043.stdout:(5/33): ceph-radosgw-19.3.0-5103.g5c1cdc3c.el9. 10 MB/s | 11 MB 00:01 2024-09-21T11:31:28.471 INFO:teuthology.orchestra.run.smithi043.stdout:(6/33): ceph-common-19.3.0-5103.g5c1cdc3c.el9.x 17 MB/s | 22 MB 00:01 2024-09-21T11:31:28.496 INFO:teuthology.orchestra.run.smithi043.stdout:(7/33): python3-ceph-argparse-19.3.0-5103.g5c1c 187 kB/s | 45 kB 00:00 2024-09-21T11:31:28.613 INFO:teuthology.orchestra.run.smithi043.stdout:(8/33): librgw2-19.3.0-5103.g5c1cdc3c.el9.x86_6 8.9 MB/s | 5.5 MB 00:00 2024-09-21T11:31:28.624 INFO:teuthology.orchestra.run.smithi175.stdout:(1/33): ceph-base-19.3.0-5103.g5c1cdc3c.el9.x86 13 MB/s | 5.5 MB 00:00 2024-09-21T11:31:28.639 INFO:teuthology.orchestra.run.smithi043.stdout:(9/33): python3-ceph-common-19.3.0-5103.g5c1cdc 889 kB/s | 148 kB 00:00 2024-09-21T11:31:28.664 INFO:teuthology.orchestra.run.smithi043.stdout:(10/33): python3-cephfs-19.3.0-5103.g5c1cdc3c.e 965 kB/s | 161 kB 00:00 2024-09-21T11:31:28.689 INFO:teuthology.orchestra.run.smithi043.stdout:(11/33): python3-rados-19.3.0-5103.g5c1cdc3c.el 4.2 MB/s | 321 kB 00:00 2024-09-21T11:31:28.691 INFO:teuthology.orchestra.run.smithi175.stdout:(2/33): ceph-selinux-19.3.0-5103.g5c1cdc3c.el9. 378 kB/s | 25 kB 00:00 2024-09-21T11:31:28.714 INFO:teuthology.orchestra.run.smithi043.stdout:(12/33): python3-rbd-19.3.0-5103.g5c1cdc3c.el9. 4.0 MB/s | 306 kB 00:00 2024-09-21T11:31:28.740 INFO:teuthology.orchestra.run.smithi043.stdout:(13/33): python3-rgw-19.3.0-5103.g5c1cdc3c.el9. 1.3 MB/s | 99 kB 00:00 2024-09-21T11:31:28.792 INFO:teuthology.orchestra.run.smithi175.stdout:(3/33): libcephfs2-19.3.0-5103.g5c1cdc3c.el9.x8 7.4 MB/s | 755 kB 00:00 2024-09-21T11:31:28.884 INFO:teuthology.orchestra.run.smithi175.stdout:(4/33): libradosstriper1-19.3.0-5103.g5c1cdc3c. 5.4 MB/s | 509 kB 00:00 2024-09-21T11:31:29.148 INFO:teuthology.orchestra.run.smithi043.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 233 kB/s | 107 kB 00:00 2024-09-21T11:31:29.184 INFO:teuthology.orchestra.run.smithi175.stdout:(5/33): librgw2-19.3.0-5103.g5c1cdc3c.el9.x86_6 18 MB/s | 5.5 MB 00:00 2024-09-21T11:31:29.242 INFO:teuthology.orchestra.run.smithi175.stdout:(6/33): python3-ceph-argparse-19.3.0-5103.g5c1c 778 kB/s | 45 kB 00:00 2024-09-21T11:31:29.265 INFO:teuthology.orchestra.run.smithi043.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-09-21T11:31:29.291 INFO:teuthology.orchestra.run.smithi043.stdout:(16/33): libnbd-1.20.2-2.el9.x86_64.rpm 296 kB/s | 171 kB 00:00 2024-09-21T11:31:29.316 INFO:teuthology.orchestra.run.smithi043.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 278 kB/s | 160 kB 00:00 2024-09-21T11:31:29.359 INFO:teuthology.orchestra.run.smithi175.stdout:(7/33): ceph-radosgw-19.3.0-5103.g5c1cdc3c.el9. 9.3 MB/s | 11 MB 00:01 2024-09-21T11:31:29.385 INFO:teuthology.orchestra.run.smithi175.stdout:(8/33): python3-ceph-common-19.3.0-5103.g5c1cdc 1.0 MB/s | 148 kB 00:00 2024-09-21T11:31:29.410 INFO:teuthology.orchestra.run.smithi175.stdout:(9/33): python3-cephfs-19.3.0-5103.g5c1cdc3c.el 3.2 MB/s | 161 kB 00:00 2024-09-21T11:31:29.416 INFO:teuthology.orchestra.run.smithi043.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 423 kB/s | 42 kB 00:00 2024-09-21T11:31:29.583 INFO:teuthology.orchestra.run.smithi043.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 292 kB 00:00 2024-09-21T11:31:29.616 INFO:teuthology.orchestra.run.smithi043.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-09-21T11:31:29.618 INFO:teuthology.orchestra.run.smithi175.stdout:(10/33): ceph-common-19.3.0-5103.g5c1cdc3c.el9. 15 MB/s | 22 MB 00:01 2024-09-21T11:31:29.644 INFO:teuthology.orchestra.run.smithi175.stdout:(11/33): python3-rados-19.3.0-5103.g5c1cdc3c.el 1.2 MB/s | 321 kB 00:00 2024-09-21T11:31:29.669 INFO:teuthology.orchestra.run.smithi175.stdout:(12/33): python3-rgw-19.3.0-5103.g5c1cdc3c.el9. 1.9 MB/s | 99 kB 00:00 2024-09-21T11:31:29.694 INFO:teuthology.orchestra.run.smithi175.stdout:(13/33): python3-rbd-19.3.0-5103.g5c1cdc3c.el9. 1.1 MB/s | 306 kB 00:00 2024-09-21T11:31:29.700 INFO:teuthology.orchestra.run.smithi043.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 307 kB/s | 36 kB 00:00 2024-09-21T11:31:29.734 INFO:teuthology.orchestra.run.smithi043.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-09-21T11:31:29.776 INFO:teuthology.orchestra.run.smithi043.stdout:(23/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.0 MB/s | 308 kB 00:00 2024-09-21T11:31:29.801 INFO:teuthology.orchestra.run.smithi043.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-21T11:31:29.835 INFO:teuthology.orchestra.run.smithi043.stdout:(25/33): librdkafka-1.6.1-102.el9.x86_64.rpm 1.1 MB/s | 662 kB 00:00 2024-09-21T11:31:29.844 INFO:teuthology.orchestra.run.smithi175.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 534 kB/s | 107 kB 00:00 2024-09-21T11:31:29.860 INFO:teuthology.orchestra.run.smithi043.stdout:(26/33): liboath-2.6.7-2.el9.x86_64.rpm 829 kB/s | 49 kB 00:00 2024-09-21T11:31:29.869 INFO:teuthology.orchestra.run.smithi175.stdout:(15/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 914 kB/s | 160 kB 00:00 2024-09-21T11:31:29.885 INFO:teuthology.orchestra.run.smithi043.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T11:31:29.895 INFO:teuthology.orchestra.run.smithi175.stdout:(16/33): libnbd-1.20.2-2.el9.x86_64.rpm 755 kB/s | 171 kB 00:00 2024-09-21T11:31:29.919 INFO:teuthology.orchestra.run.smithi043.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 7.3 MB/s | 191 kB 00:00 2024-09-21T11:31:29.920 INFO:teuthology.orchestra.run.smithi175.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 598 kB/s | 45 kB 00:00 2024-09-21T11:31:29.970 INFO:teuthology.orchestra.run.smithi175.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 848 kB/s | 42 kB 00:00 2024-09-21T11:31:29.994 INFO:teuthology.orchestra.run.smithi043.stdout:(29/33): thrift-0.15.0-2.el9.x86_64.rpm 22 MB/s | 1.6 MB 00:00 2024-09-21T11:31:29.995 INFO:teuthology.orchestra.run.smithi175.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-21T11:31:30.028 INFO:teuthology.orchestra.run.smithi043.stdout:(30/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-09-21T11:31:30.037 INFO:teuthology.orchestra.run.smithi175.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-21T11:31:30.063 INFO:teuthology.orchestra.run.smithi175.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 535 kB/s | 36 kB 00:00 2024-09-21T11:31:30.096 INFO:teuthology.orchestra.run.smithi175.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-21T11:31:30.111 INFO:teuthology.orchestra.run.smithi043.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-21T11:31:30.130 INFO:teuthology.orchestra.run.smithi175.stdout:(23/33): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-09-21T11:31:30.163 INFO:teuthology.orchestra.run.smithi175.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-21T11:31:30.189 INFO:teuthology.orchestra.run.smithi175.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-21T11:31:30.214 INFO:teuthology.orchestra.run.smithi175.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 317 kB/s | 26 kB 00:00 2024-09-21T11:31:30.239 INFO:teuthology.orchestra.run.smithi175.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T11:31:30.262 INFO:teuthology.orchestra.run.smithi043.stdout:(32/33): librbd1-19.3.0-5103.g5c1cdc3c.el9.x86_ 14 MB/s | 3.2 MB 00:00 2024-09-21T11:31:30.273 INFO:teuthology.orchestra.run.smithi175.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-21T11:31:30.307 INFO:teuthology.orchestra.run.smithi175.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-21T11:31:30.320 INFO:teuthology.orchestra.run.smithi043.stdout:(33/33): librados2-19.3.0-5103.g5c1cdc3c.el9.x8 11 MB/s | 3.5 MB 00:00 2024-09-21T11:31:30.325 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:31:30.325 INFO:teuthology.orchestra.run.smithi043.stdout:Total 19 MB/s | 62 MB 00:03 2024-09-21T11:31:30.390 INFO:teuthology.orchestra.run.smithi175.stdout:(30/33): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-21T11:31:30.432 INFO:teuthology.orchestra.run.smithi175.stdout:(31/33): thrift-0.15.0-2.el9.x86_64.rpm 9.9 MB/s | 1.6 MB 00:00 2024-09-21T11:31:30.447 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:31:30.488 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:31:30.489 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:31:30.532 INFO:teuthology.orchestra.run.smithi175.stdout:(32/33): librados2-19.3.0-5103.g5c1cdc3c.el9.x8 15 MB/s | 3.5 MB 00:00 2024-09-21T11:31:30.624 INFO:teuthology.orchestra.run.smithi175.stdout:(33/33): librbd1-19.3.0-5103.g5c1cdc3c.el9.x86_ 14 MB/s | 3.2 MB 00:00 2024-09-21T11:31:30.629 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:31:30.629 INFO:teuthology.orchestra.run.smithi175.stdout:Total 24 MB/s | 62 MB 00:02 2024-09-21T11:31:30.738 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:31:30.773 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:31:30.773 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:31:31.020 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:31:31.020 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:31:31.273 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:31:31.273 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:31:32.516 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:31:32.603 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-21T11:31:32.643 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-21T11:31:32.667 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-21T11:31:32.690 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-21T11:31:32.718 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-21T11:31:32.759 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:31:32.837 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-21T11:31:32.862 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-21T11:31:32.887 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-21T11:31:32.910 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-21T11:31:32.919 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-21T11:31:32.930 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-21T11:31:32.981 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-21T11:31:33.112 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-21T11:31:33.165 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-21T11:31:33.392 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-21T11:31:33.453 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:31:33.522 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-21T11:31:33.546 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:31:33.630 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:31:33.635 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 8/35 2024-09-21T11:31:33.671 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 9/35 2024-09-21T11:31:33.722 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:31:33.809 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 8/35 2024-09-21T11:31:33.831 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 9/35 2024-09-21T11:31:33.886 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 9/35 2024-09-21T11:31:33.909 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 9/35 2024-09-21T11:31:33.914 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-21T11:31:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-21T11:31:33.957 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9. 11/35 2024-09-21T11:31:33.998 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9. 11/35 2024-09-21T11:31:34.027 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 12/35 2024-09-21T11:31:34.050 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 13/35 2024-09-21T11:31:34.069 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 12/35 2024-09-21T11:31:34.098 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 13/35 2024-09-21T11:31:34.122 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 13/35 2024-09-21T11:31:34.146 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 13/35 2024-09-21T11:31:34.157 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-21T11:31:34.180 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-21T11:31:34.268 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-21T11:31:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-21T11:31:34.307 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-21T11:31:34.339 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-21T11:31:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-21T11:31:34.380 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-21T11:31:34.389 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-21T11:31:34.413 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-21T11:31:34.418 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-21T11:31:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-21T11:31:34.451 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-21T11:31:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-21T11:31:34.488 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-21T11:31:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-21T11:31:34.827 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-21T11:31:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-21T11:31:34.873 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 23/35 2024-09-21T11:31:34.905 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 23/35 2024-09-21T11:31:34.962 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 23/35 2024-09-21T11:31:34.995 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 24/35 2024-09-21T11:31:35.007 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 23/35 2024-09-21T11:31:35.040 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-21T11:31:35.043 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 24/35 2024-09-21T11:31:35.080 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-21T11:31:35.551 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-21T11:31:35.576 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 27/35 2024-09-21T11:31:35.596 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-21T11:31:35.647 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 27/35 2024-09-21T11:31:35.649 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 27/35 2024-09-21T11:31:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 27/35 2024-09-21T11:31:35.740 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 28/35 2024-09-21T11:31:35.778 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x8 29/35 2024-09-21T11:31:35.807 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 28/35 2024-09-21T11:31:35.835 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x8 29/35 2024-09-21T11:31:38.120 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:31:38.241 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:31:38.268 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:31:38.306 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:31:38.816 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:31:38.852 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 31/35 2024-09-21T11:31:38.922 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 30/35 2024-09-21T11:31:38.947 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 31/35 2024-09-21T11:31:38.948 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T11:31:38.948 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T11:31:38.948 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:38.959 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 31/35 2024-09-21T11:31:38.986 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:31:39.051 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 31/35 2024-09-21T11:31:39.051 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T11:31:39.051 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T11:31:39.051 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:39.086 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:31:48.953 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:31:48.953 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-09-21T11:31:48.953 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-09-21T11:31:48.953 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-09-21T11:31:48.953 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-09-21T11:31:48.953 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-09-21T11:31:48.953 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-09-21T11:31:48.954 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-09-21T11:31:48.954 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:49.741 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:31:49.742 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-09-21T11:31:49.742 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-09-21T11:31:49.742 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-09-21T11:31:49.742 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-09-21T11:31:49.742 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-09-21T11:31:49.742 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-09-21T11:31:49.742 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-09-21T11:31:49.742 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:49.977 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/35 2024-09-21T11:31:50.002 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/35 2024-09-21T11:31:50.002 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:31:50.002 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T11:31:50.002 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T11:31:50.002 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T11:31:50.002 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:50.004 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-21T11:31:50.087 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-21T11:31:50.087 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:31:50.785 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/35 2024-09-21T11:31:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/35 2024-09-21T11:31:50.812 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:31:50.813 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T11:31:50.813 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T11:31:50.813 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T11:31:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-21T11:31:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-21T11:31:50.913 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 5/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 6/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9. 8/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x8 9/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 10/35 2024-09-21T11:31:51.225 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 11/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 12/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 13/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-21T11:31:51.226 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-21T11:31:51.227 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-21T11:31:51.227 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-21T11:31:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-21T11:31:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-21T11:31:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-21T11:31:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-21T11:31:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:31:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-21T11:31:51.228 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/35 2024-09-21T11:31:52.076 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 5/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_6 6/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 7/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9. 8/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x8 9/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 10/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 11/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 12/35 2024-09-21T11:31:52.077 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 13/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-21T11:31:52.078 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-21T11:31:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-21T11:31:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-21T11:31:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-21T11:31:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-21T11:31:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 32/35 2024-09-21T11:31:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-21T11:31:52.079 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/35 2024-09-21T11:31:52.124 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:31:52.124 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:52.124 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-09-21T11:31:52.124 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.124 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T11:31:52.125 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.126 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.127 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T11:31:52.127 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T11:31:52.127 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T11:31:52.127 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T11:31:52.127 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T11:31:52.127 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:52.127 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:31:52.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-test 2024-09-21T11:31:52.881 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-21T11:31:52.881 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:52.881 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-09-21T11:31:52.881 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.881 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.881 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:31:52.881 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T11:31:52.881 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.882 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T11:31:52.883 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T11:31:52.884 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T11:31:52.884 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T11:31:52.884 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:52.884 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:31:53.092 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:28 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:31:53.225 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:31:53.225 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:53.225 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 46 M 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 161 k 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout:Install 4 Packages 2024-09-21T11:31:53.226 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:31:53.227 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 46 M 2024-09-21T11:31:53.227 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 201 M 2024-09-21T11:31:53.227 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:31:53.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-test 2024-09-21T11:31:53.560 INFO:teuthology.orchestra.run.smithi175.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-21T11:31:53.585 INFO:teuthology.orchestra.run.smithi175.stdout:(2/4): libcephsqlite-19.3.0-5103.g5c1cdc3c.el9. 700 kB/s | 161 kB 00:00 2024-09-21T11:31:53.728 INFO:teuthology.orchestra.run.smithi175.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 382 kB/s | 64 kB 00:00 2024-09-21T11:31:53.887 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:29 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:31:54.033 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 46 M 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 161 k 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:54.034 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:31:54.035 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:31:54.035 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-21T11:31:54.035 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:31:54.035 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 46 M 2024-09-21T11:31:54.035 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 201 M 2024-09-21T11:31:54.035 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:31:54.319 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.9 MB/s | 304 kB 00:00 2024-09-21T11:31:54.361 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-21T11:31:54.503 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): libcephsqlite-19.3.0-5103.g5c1cdc3c.el9. 472 kB/s | 161 kB 00:00 2024-09-21T11:31:54.811 INFO:teuthology.orchestra.run.smithi175.stdout:(4/4): ceph-test-19.3.0-5103.g5c1cdc3c.el9.x86_ 31 MB/s | 46 MB 00:01 2024-09-21T11:31:54.813 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:31:54.814 INFO:teuthology.orchestra.run.smithi175.stdout:Total 29 MB/s | 46 MB 00:01 2024-09-21T11:31:54.841 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:31:54.862 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:31:54.862 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:31:55.238 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:31:55.238 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:31:55.712 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): ceph-test-19.3.0-5103.g5c1cdc3c.el9.x86_ 29 MB/s | 46 MB 00:01 2024-09-21T11:31:55.714 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:31:55.714 INFO:teuthology.orchestra.run.smithi043.stdout:Total 27 MB/s | 46 MB 00:01 2024-09-21T11:31:55.742 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:31:55.755 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:31:55.765 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:31:55.765 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:31:55.821 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T11:31:55.860 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T11:31:55.887 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/4 2024-09-21T11:31:56.154 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:31:56.155 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:31:56.687 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:31:56.782 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-21T11:31:56.820 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-21T11:31:56.847 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/4 2024-09-21T11:32:00.247 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/4 2024-09-21T11:32:00.328 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/4 2024-09-21T11:32:01.477 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/4 2024-09-21T11:32:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/4 2024-09-21T11:32:02.711 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/4 2024-09-21T11:32:02.711 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/4 2024-09-21T11:32:02.711 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/4 2024-09-21T11:32:02.712 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T11:32:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T11:32:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:03.091 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:32:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T11:32:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T11:32:03.091 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:03.092 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:03.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph 2024-09-21T11:32:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/4 2024-09-21T11:32:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/4 2024-09-21T11:32:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/4 2024-09-21T11:32:03.940 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-21T11:32:03.963 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:38 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:04.103 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 6.6 k 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 2.4 M 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 1.5 M 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 292 k 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 4.8 M 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 17 M 2024-09-21T11:32:04.106 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-21T11:32:04.107 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T11:32:04.108 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T11:32:04.109 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T11:32:04.109 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T11:32:04.109 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T11:32:04.109 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T11:32:04.109 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T11:32:04.109 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:04.109 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:32:04.109 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:04.109 INFO:teuthology.orchestra.run.smithi175.stdout:Install 37 Packages 2024-09-21T11:32:04.109 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:04.110 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 31 M 2024-09-21T11:32:04.110 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 109 M 2024-09-21T11:32:04.110 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:32:04.344 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-21T11:32:04.345 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:04.345 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:32:04.345 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:04.345 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:04.345 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T11:32:04.345 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T11:32:04.345 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:04.345 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:32:04.421 INFO:teuthology.orchestra.run.smithi175.stdout:(1/37): ceph-19.3.0-5103.g5c1cdc3c.el9.x86_64.r 51 kB/s | 6.6 kB 00:00 2024-09-21T11:32:04.594 INFO:teuthology.orchestra.run.smithi175.stdout:(2/37): ceph-mgr-19.3.0-5103.g5c1cdc3c.el9.x86_ 5.0 MB/s | 1.5 MB 00:00 2024-09-21T11:32:04.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph 2024-09-21T11:32:04.653 INFO:teuthology.orchestra.run.smithi175.stdout:(3/37): ceph-mds-19.3.0-5103.g5c1cdc3c.el9.x86_ 6.6 MB/s | 2.4 MB 00:00 2024-09-21T11:32:04.736 INFO:teuthology.orchestra.run.smithi175.stdout:(4/37): ceph-mgr-modules-core-19.3.0-5103.g5c1c 3.4 MB/s | 292 kB 00:00 2024-09-21T11:32:04.803 INFO:teuthology.orchestra.run.smithi175.stdout:(5/37): ceph-mon-19.3.0-5103.g5c1cdc3c.el9.x86_ 12 MB/s | 4.8 MB 00:00 2024-09-21T11:32:04.920 INFO:teuthology.orchestra.run.smithi175.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-21T11:32:04.962 INFO:teuthology.orchestra.run.smithi175.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T11:32:05.013 INFO:teuthology.orchestra.run.smithi175.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 5.9 MB/s | 1.2 MB 00:00 2024-09-21T11:32:05.037 INFO:teuthology.orchestra.run.smithi175.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-09-21T11:32:05.063 INFO:teuthology.orchestra.run.smithi175.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-21T11:32:05.088 INFO:teuthology.orchestra.run.smithi175.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-21T11:32:05.228 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:41 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:32:05.247 INFO:teuthology.orchestra.run.smithi175.stdout:(12/37): ceph-osd-19.3.0-5103.g5c1cdc3c.el9.x86 26 MB/s | 17 MB 00:00 2024-09-21T11:32:05.368 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:32:05.371 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:05.371 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-21T11:32:05.371 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:05.371 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:32:05.371 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 6.6 k 2024-09-21T11:32:05.371 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-21T11:32:05.371 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 2.4 M 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 1.5 M 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 292 k 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 4.8 M 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 17 M 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T11:32:05.372 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T11:32:05.373 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T11:32:05.374 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T11:32:05.374 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T11:32:05.374 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T11:32:05.374 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T11:32:05.374 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T11:32:05.374 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T11:32:05.374 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T11:32:05.374 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T11:32:05.374 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T11:32:05.374 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:05.375 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:32:05.375 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:05.375 INFO:teuthology.orchestra.run.smithi043.stdout:Install 37 Packages 2024-09-21T11:32:05.375 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:05.376 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 31 M 2024-09-21T11:32:05.376 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 109 M 2024-09-21T11:32:05.376 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:32:05.522 INFO:teuthology.orchestra.run.smithi175.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 126 kB/s | 35 kB 00:00 2024-09-21T11:32:05.706 INFO:teuthology.orchestra.run.smithi175.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 279 kB/s | 172 kB 00:00 2024-09-21T11:32:05.739 INFO:teuthology.orchestra.run.smithi175.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 279 kB/s | 188 kB 00:00 2024-09-21T11:32:05.807 INFO:teuthology.orchestra.run.smithi175.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 271 kB/s | 77 kB 00:00 2024-09-21T11:32:05.832 INFO:teuthology.orchestra.run.smithi175.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 406 kB/s | 51 kB 00:00 2024-09-21T11:32:05.857 INFO:teuthology.orchestra.run.smithi175.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 355 kB/s | 42 kB 00:00 2024-09-21T11:32:05.872 INFO:teuthology.orchestra.run.smithi043.stdout:(1/37): ceph-19.3.0-5103.g5c1cdc3c.el9.x86_64.r 55 kB/s | 6.6 kB 00:00 2024-09-21T11:32:05.899 INFO:teuthology.orchestra.run.smithi175.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-21T11:32:05.924 INFO:teuthology.orchestra.run.smithi175.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-09-21T11:32:05.949 INFO:teuthology.orchestra.run.smithi175.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 472 kB/s | 43 kB 00:00 2024-09-21T11:32:05.983 INFO:teuthology.orchestra.run.smithi175.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-21T11:32:06.008 INFO:teuthology.orchestra.run.smithi175.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 208 kB/s | 11 kB 00:00 2024-09-21T11:32:06.033 INFO:teuthology.orchestra.run.smithi175.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 365 kB/s | 18 kB 00:00 2024-09-21T11:32:06.039 INFO:teuthology.orchestra.run.smithi043.stdout:(2/37): ceph-mgr-19.3.0-5103.g5c1cdc3c.el9.x86_ 5.3 MB/s | 1.5 MB 00:00 2024-09-21T11:32:06.058 INFO:teuthology.orchestra.run.smithi175.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-21T11:32:06.084 INFO:teuthology.orchestra.run.smithi175.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-21T11:32:06.106 INFO:teuthology.orchestra.run.smithi043.stdout:(3/37): ceph-mds-19.3.0-5103.g5c1cdc3c.el9.x86_ 6.7 MB/s | 2.4 MB 00:00 2024-09-21T11:32:06.109 INFO:teuthology.orchestra.run.smithi175.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-21T11:32:06.134 INFO:teuthology.orchestra.run.smithi175.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-21T11:32:06.160 INFO:teuthology.orchestra.run.smithi175.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-21T11:32:06.185 INFO:teuthology.orchestra.run.smithi175.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-21T11:32:06.207 INFO:teuthology.orchestra.run.smithi043.stdout:(4/37): ceph-mon-19.3.0-5103.g5c1cdc3c.el9.x86_ 14 MB/s | 4.8 MB 00:00 2024-09-21T11:32:06.210 INFO:teuthology.orchestra.run.smithi175.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-21T11:32:06.231 INFO:teuthology.orchestra.run.smithi043.stdout:(5/37): ceph-mgr-modules-core-19.3.0-5103.g5c1c 2.3 MB/s | 292 kB 00:00 2024-09-21T11:32:06.235 INFO:teuthology.orchestra.run.smithi175.stdout:(32/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-21T11:32:06.260 INFO:teuthology.orchestra.run.smithi175.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-21T11:32:06.286 INFO:teuthology.orchestra.run.smithi175.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-09-21T11:32:06.311 INFO:teuthology.orchestra.run.smithi175.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 809 kB/s | 20 kB 00:00 2024-09-21T11:32:06.336 INFO:teuthology.orchestra.run.smithi175.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T11:32:06.370 INFO:teuthology.orchestra.run.smithi175.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-21T11:32:06.374 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:06.374 INFO:teuthology.orchestra.run.smithi175.stdout:Total 14 MB/s | 31 MB 00:02 2024-09-21T11:32:06.523 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:32:06.563 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:32:06.563 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:32:06.624 INFO:teuthology.orchestra.run.smithi043.stdout:(6/37): ceph-osd-19.3.0-5103.g5c1cdc3c.el9.x86_ 29 MB/s | 17 MB 00:00 2024-09-21T11:32:06.732 INFO:teuthology.orchestra.run.smithi043.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 482 kB/s | 253 kB 00:00 2024-09-21T11:32:06.816 INFO:teuthology.orchestra.run.smithi043.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.6 MB/s | 135 kB 00:00 2024-09-21T11:32:06.841 INFO:teuthology.orchestra.run.smithi043.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 491 kB/s | 106 kB 00:00 2024-09-21T11:32:06.900 INFO:teuthology.orchestra.run.smithi043.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.5 MB/s | 125 kB 00:00 2024-09-21T11:32:06.933 INFO:teuthology.orchestra.run.smithi043.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 1.7 MB/s | 1.2 MB 00:00 2024-09-21T11:32:07.024 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:32:07.025 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:32:07.033 INFO:teuthology.orchestra.run.smithi043.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.1 MB/s | 215 kB 00:00 2024-09-21T11:32:07.142 INFO:teuthology.orchestra.run.smithi043.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 778 kB/s | 188 kB 00:00 2024-09-21T11:32:07.168 INFO:teuthology.orchestra.run.smithi043.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 736 kB/s | 172 kB 00:00 2024-09-21T11:32:07.193 INFO:teuthology.orchestra.run.smithi043.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 219 kB/s | 35 kB 00:00 2024-09-21T11:32:07.218 INFO:teuthology.orchestra.run.smithi043.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-21T11:32:07.243 INFO:teuthology.orchestra.run.smithi043.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T11:32:07.277 INFO:teuthology.orchestra.run.smithi043.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 498 kB/s | 42 kB 00:00 2024-09-21T11:32:07.327 INFO:teuthology.orchestra.run.smithi043.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-21T11:32:07.361 INFO:teuthology.orchestra.run.smithi043.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-09-21T11:32:07.386 INFO:teuthology.orchestra.run.smithi043.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 407 kB/s | 43 kB 00:00 2024-09-21T11:32:07.411 INFO:teuthology.orchestra.run.smithi043.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 428 kB/s | 11 kB 00:00 2024-09-21T11:32:07.436 INFO:teuthology.orchestra.run.smithi043.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 711 kB/s | 18 kB 00:00 2024-09-21T11:32:07.471 INFO:teuthology.orchestra.run.smithi043.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-21T11:32:07.495 INFO:teuthology.orchestra.run.smithi043.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 395 kB/s | 23 kB 00:00 2024-09-21T11:32:07.521 INFO:teuthology.orchestra.run.smithi043.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 390 kB/s | 19 kB 00:00 2024-09-21T11:32:07.546 INFO:teuthology.orchestra.run.smithi043.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-21T11:32:07.571 INFO:teuthology.orchestra.run.smithi043.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-21T11:32:07.597 INFO:teuthology.orchestra.run.smithi043.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.5 MB/s | 358 kB 00:00 2024-09-21T11:32:07.622 INFO:teuthology.orchestra.run.smithi043.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-21T11:32:07.639 INFO:teuthology.orchestra.run.smithi043.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 391 kB/s | 16 kB 00:00 2024-09-21T11:32:07.656 INFO:teuthology.orchestra.run.smithi043.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 2.1 MB/s | 36 kB 00:00 2024-09-21T11:32:07.681 INFO:teuthology.orchestra.run.smithi043.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-21T11:32:07.715 INFO:teuthology.orchestra.run.smithi043.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-21T11:32:07.740 INFO:teuthology.orchestra.run.smithi043.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-21T11:32:07.765 INFO:teuthology.orchestra.run.smithi043.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-09-21T11:32:07.791 INFO:teuthology.orchestra.run.smithi043.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-21T11:32:07.794 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:07.794 INFO:teuthology.orchestra.run.smithi043.stdout:Total 13 MB/s | 31 MB 00:02 2024-09-21T11:32:07.953 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:32:07.996 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:32:07.997 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:32:08.314 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:32:08.389 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-21T11:32:08.422 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-21T11:32:08.449 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-21T11:32:08.500 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:32:08.500 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:32:08.520 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-21T11:32:08.565 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-21T11:32:08.598 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-21T11:32:08.689 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-21T11:32:08.776 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-21T11:32:08.816 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-21T11:32:08.852 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-21T11:32:08.882 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-21T11:32:08.916 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-21T11:32:08.950 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-21T11:32:08.982 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-21T11:32:09.016 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-21T11:32:09.051 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-21T11:32:09.099 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-21T11:32:09.132 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-21T11:32:09.204 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-21T11:32:09.351 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-21T11:32:09.438 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-21T11:32:09.501 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-21T11:32:09.539 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-21T11:32:09.587 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-21T11:32:09.642 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-21T11:32:09.804 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:32:09.828 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-21T11:32:09.876 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-21T11:32:09.885 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-21T11:32:09.918 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-21T11:32:09.945 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-21T11:32:09.953 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-21T11:32:10.022 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-21T11:32:10.053 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-21T11:32:10.087 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-21T11:32:10.095 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-21T11:32:10.128 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-21T11:32:10.219 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-21T11:32:10.232 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-21T11:32:10.320 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-21T11:32:10.367 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-21T11:32:10.382 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9. 32/37 2024-09-21T11:32:10.409 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-21T11:32:10.410 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/37 2024-09-21T11:32:10.437 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/37 2024-09-21T11:32:10.437 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:10.437 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T11:32:10.437 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T11:32:10.437 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T11:32:10.437 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-21T11:32:10.481 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-21T11:32:10.515 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-21T11:32:10.566 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-21T11:32:10.607 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-21T11:32:10.641 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-21T11:32:10.691 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-21T11:32:10.734 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-21T11:32:10.815 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-21T11:32:10.968 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-21T11:32:11.058 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-21T11:32:11.118 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-21T11:32:11.155 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-21T11:32:11.214 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-21T11:32:11.286 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-21T11:32:11.484 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-21T11:32:11.534 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-21T11:32:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-21T11:32:11.713 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-21T11:32:11.777 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-21T11:32:11.919 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-21T11:32:12.036 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/37 2024-09-21T11:32:12.064 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/37 2024-09-21T11:32:12.064 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:12.064 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T11:32:12.064 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T11:32:12.064 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T11:32:12.065 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:12.087 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9. 32/37 2024-09-21T11:32:12.117 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/37 2024-09-21T11:32:12.146 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 33/37 2024-09-21T11:32:12.146 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:12.146 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T11:32:12.146 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T11:32:12.146 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T11:32:12.146 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:12.512 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 35/37 2024-09-21T11:32:12.536 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 35/37 2024-09-21T11:32:12.536 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:12.536 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T11:32:12.536 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T11:32:12.536 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T11:32:12.537 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:12.771 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 36/37 2024-09-21T11:32:12.795 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 36/37 2024-09-21T11:32:12.795 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:12.795 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T11:32:12.795 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T11:32:12.795 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T11:32:12.795 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:12.859 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 37/37 2024-09-21T11:32:13.693 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/37 2024-09-21T11:32:13.722 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 34/37 2024-09-21T11:32:13.722 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:13.722 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T11:32:13.722 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T11:32:13.723 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T11:32:13.723 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:14.194 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 35/37 2024-09-21T11:32:14.219 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 35/37 2024-09-21T11:32:14.219 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:14.219 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T11:32:14.219 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T11:32:14.219 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T11:32:14.219 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 37/37 2024-09-21T11:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/37 2024-09-21T11:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/37 2024-09-21T11:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/37 2024-09-21T11:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/37 2024-09-21T11:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 5/37 2024-09-21T11:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9. 6/37 2024-09-21T11:32:14.344 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-21T11:32:14.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-21T11:32:14.346 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-21T11:32:14.346 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-21T11:32:14.346 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-21T11:32:14.346 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-21T11:32:14.346 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-21T11:32:14.346 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-21T11:32:14.346 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-21T11:32:14.346 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-21T11:32:14.346 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-21T11:32:14.347 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-21T11:32:14.347 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-21T11:32:14.347 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-21T11:32:14.347 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-21T11:32:14.347 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-21T11:32:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 36/37 2024-09-21T11:32:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 36/37 2024-09-21T11:32:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T11:32:14.490 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T11:32:14.490 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T11:32:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:14.553 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 37/37 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T11:32:15.410 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T11:32:15.411 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T11:32:15.412 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T11:32:15.413 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:15.413 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:15.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-base 2024-09-21T11:32:16.353 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 37/37 2024-09-21T11:32:16.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/37 2024-09-21T11:32:16.353 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 3/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 4/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 5/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9. 6/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-21T11:32:16.354 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-21T11:32:16.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-21T11:32:16.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-21T11:32:16.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-21T11:32:16.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-21T11:32:16.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-21T11:32:16.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-21T11:32:16.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-21T11:32:16.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-21T11:32:16.355 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-21T11:32:16.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-21T11:32:16.356 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-21T11:32:16.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-21T11:32:16.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-21T11:32:16.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-21T11:32:16.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-21T11:32:16.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-21T11:32:16.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-21T11:32:16.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-21T11:32:16.357 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-21T11:32:16.358 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-21T11:32:16.406 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:51 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:16.496 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:32:16.539 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:16.540 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-21T11:32:16.540 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:16.597 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install cephadm 2024-09-21T11:32:17.178 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:52 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:17.190 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-21T11:32:17.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:17.190 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:32:17.190 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:17.190 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:17.190 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:17.190 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:17.190 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T11:32:17.191 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T11:32:17.192 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T11:32:17.193 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T11:32:17.193 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T11:32:17.193 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T11:32:17.193 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T11:32:17.193 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T11:32:17.193 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T11:32:17.193 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T11:32:17.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:17.193 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:32:17.309 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:17.310 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:17.310 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T11:32:17.310 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:17.310 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:32:17.311 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 784 k 2024-09-21T11:32:17.311 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:17.311 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:32:17.311 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:17.311 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-21T11:32:17.311 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:17.311 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 784 k 2024-09-21T11:32:17.311 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 791 k 2024-09-21T11:32:17.311 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:32:17.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-base 2024-09-21T11:32:17.571 INFO:teuthology.orchestra.run.smithi175.stdout:cephadm-19.3.0-5103.g5c1cdc3c.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-21T11:32:17.572 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:17.572 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.9 MB/s | 784 kB 00:00 2024-09-21T11:32:17.572 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:32:17.577 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:32:17.577 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:32:17.583 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:32:17.584 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:32:17.658 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:32:18.159 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:32:18.167 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:18.251 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-base-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:32:18.258 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:32:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-21T11:32:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:32:18.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install cephadm 2024-09-21T11:32:18.627 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:18.880 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:18.880 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:18.880 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:32:18.880 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:18.880 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:18.880 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:18.961 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:32:19.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T11:32:19.098 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 784 k 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 784 k 2024-09-21T11:32:19.099 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 791 k 2024-09-21T11:32:19.100 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:32:19.376 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm-19.3.0-5103.g5c1cdc3c.el9.noarch.rpm 2.8 MB/s | 784 kB 00:00 2024-09-21T11:32:19.377 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:19.377 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-21T11:32:19.377 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:32:19.383 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:32:19.383 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:32:19.389 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:32:19.389 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:32:19.466 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:32:19.672 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:54 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:19.809 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-21T11:32:19.810 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 145 k 2024-09-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 145 k 2024-09-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 432 k 2024-09-21T11:32:19.811 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:32:19.873 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:19.957 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:20.026 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-immutable-object-cache-19.3.0-5103.g5c1cdc 674 kB/s | 145 kB 00:00 2024-09-21T11:32:20.027 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:20.027 INFO:teuthology.orchestra.run.smithi175.stdout:Total 669 kB/s | 145 kB 00:00 2024-09-21T11:32:20.027 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:32:20.035 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:32:20.035 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:32:20.092 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:32:20.092 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:32:20.246 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:32:20.319 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:32:20.335 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:20.345 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:32:20.345 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:20.345 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T11:32:20.346 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:20.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:20.605 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:20.605 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:32:20.605 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:20.606 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:20.606 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:32:20.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-immutable-object-cache 2024-09-21T11:32:21.356 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:32:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:21.357 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:32:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:21.357 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:21.357 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:21.424 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:32:21.562 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:32:21.562 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:21.563 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-21T11:32:21.563 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:21.563 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:32:21.563 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 145 k 2024-09-21T11:32:21.563 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:21.563 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:32:21.563 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:21.563 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-21T11:32:21.563 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:21.564 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 145 k 2024-09-21T11:32:21.564 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 432 k 2024-09-21T11:32:21.564 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:32:21.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr 2024-09-21T11:32:21.790 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-immutable-object-cache-19.3.0-5103.g5c1cdc 641 kB/s | 145 kB 00:00 2024-09-21T11:32:21.791 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:21.791 INFO:teuthology.orchestra.run.smithi043.stdout:Total 636 kB/s | 145 kB 00:00 2024-09-21T11:32:21.791 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:32:21.799 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:32:21.799 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:32:21.861 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:32:21.861 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:32:22.015 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:32:22.100 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:32:22.127 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:32:22.127 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:22.127 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T11:32:22.127 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:22.173 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:22.265 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:32:22.310 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:22.311 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-21T11:32:22.311 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:22.373 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T11:32:22.955 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:23.086 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c. 1/1 2024-09-21T11:32:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:23.087 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:32:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:32:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:23.087 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:32:23.087 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 14 M 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 34 k 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 16 k 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T11:32:23.088 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T11:32:23.089 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T11:32:23.089 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T11:32:23.089 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T11:32:23.089 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T11:32:23.089 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:23.089 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:32:23.089 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:23.089 INFO:teuthology.orchestra.run.smithi175.stdout:Install 12 Packages 2024-09-21T11:32:23.089 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:23.090 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 19 M 2024-09-21T11:32:23.090 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 159 M 2024-09-21T11:32:23.090 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:32:23.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr 2024-09-21T11:32:23.596 INFO:teuthology.orchestra.run.smithi175.stdout:(1/12): ceph-grafana-dashboards-19.3.0-5103.g5c 235 kB/s | 34 kB 00:00 2024-09-21T11:32:23.621 INFO:teuthology.orchestra.run.smithi175.stdout:(2/12): ceph-prometheus-alerts-19.3.0-5103.g5c1 96 kB/s | 16 kB 00:00 2024-09-21T11:32:23.755 INFO:teuthology.orchestra.run.smithi175.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.0 MB/s | 269 kB 00:00 2024-09-21T11:32:23.788 INFO:teuthology.orchestra.run.smithi175.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-09-21T11:32:23.839 INFO:teuthology.orchestra.run.smithi175.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-21T11:32:23.864 INFO:teuthology.orchestra.run.smithi175.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 790 kB/s | 19 kB 00:00 2024-09-21T11:32:23.923 INFO:teuthology.orchestra.run.smithi175.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 35 MB/s | 2.0 MB 00:00 2024-09-21T11:32:23.928 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:59 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:32:23.948 INFO:teuthology.orchestra.run.smithi175.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-21T11:32:23.981 INFO:teuthology.orchestra.run.smithi175.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 921 kB/s | 31 kB 00:00 2024-09-21T11:32:24.007 INFO:teuthology.orchestra.run.smithi175.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-21T11:32:24.021 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-mgr-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:32:24.067 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:32:24.068 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-21T11:32:24.068 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:32:24.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-dashboard 2024-09-21T11:32:24.174 INFO:teuthology.orchestra.run.smithi175.stdout:(11/12): ceph-mgr-dashboard-19.3.0-5103.g5c1cdc 20 MB/s | 14 MB 00:00 2024-09-21T11:32:24.207 INFO:teuthology.orchestra.run.smithi175.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.9 MB/s | 863 kB 00:00 2024-09-21T11:32:24.210 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:24.210 INFO:teuthology.orchestra.run.smithi175.stdout:Total 17 MB/s | 19 MB 00:01 2024-09-21T11:32:24.259 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:32:24.278 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:32:24.278 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:32:24.553 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:32:24.553 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:32:24.726 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:00 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:32:24.864 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:32:24.865 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:24.865 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-21T11:32:24.865 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:24.865 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:32:24.865 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 14 M 2024-09-21T11:32:24.865 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-21T11:32:24.865 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T11:32:24.865 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 34 k 2024-09-21T11:32:24.865 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 16 k 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-09-21T11:32:24.866 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:24.867 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 19 M 2024-09-21T11:32:24.867 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 159 M 2024-09-21T11:32:24.867 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:32:25.151 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:32:25.225 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-21T11:32:25.260 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-21T11:32:25.306 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-21T11:32:25.337 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-21T11:32:25.402 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): ceph-prometheus-alerts-19.3.0-5103.g5c1 109 kB/s | 16 kB 00:00 2024-09-21T11:32:25.427 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): ceph-grafana-dashboards-19.3.0-5103.g5c 195 kB/s | 34 kB 00:00 2024-09-21T11:32:25.433 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-21T11:32:25.544 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.3 MB/s | 269 kB 00:00 2024-09-21T11:32:25.563 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-21T11:32:25.611 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-09-21T11:32:25.655 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-21T11:32:25.661 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 7.2 MB/s | 863 kB 00:00 2024-09-21T11:32:25.703 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 494 kB/s | 19 kB 00:00 2024-09-21T11:32:25.724 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-21T11:32:25.737 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.3 MB/s | 548 kB 00:00 2024-09-21T11:32:25.761 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-21T11:32:25.762 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.6 MB/s | 144 kB 00:00 2024-09-21T11:32:25.788 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-21T11:32:25.793 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9 10/12 2024-09-21T11:32:25.813 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-21T11:32:25.872 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2024-09-21T11:32:26.022 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): ceph-mgr-dashboard-19.3.0-5103.g5c1cdc 18 MB/s | 14 MB 00:00 2024-09-21T11:32:26.026 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:26.026 INFO:teuthology.orchestra.run.smithi043.stdout:Total 17 MB/s | 19 MB 00:01 2024-09-21T11:32:26.076 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:32:26.096 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:32:26.096 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:32:26.379 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:32:26.379 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:32:26.998 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:32:27.082 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-21T11:32:27.116 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-21T11:32:27.164 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-21T11:32:27.193 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-21T11:32:27.293 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-21T11:32:27.433 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-21T11:32:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-21T11:32:27.591 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-21T11:32:27.618 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-21T11:32:27.651 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9 10/12 2024-09-21T11:32:28.986 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el 11/12 2024-09-21T11:32:29.056 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 12/12 2024-09-21T11:32:31.053 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el 11/12 2024-09-21T11:32:31.104 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 12/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 12/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el 1/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 2/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9 3/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-21T11:32:31.153 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-21T11:32:31.454 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-21T11:32:31.454 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:31.454 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:32:31.454 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:31.455 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:31.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T11:32:32.271 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:07 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:32.405 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:32.406 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-21T11:32:32.406 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T11:32:32.406 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 7.4 M 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T11:32:32.407 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T11:32:32.408 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:32.408 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:32:32.408 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-21T11:32:32.408 INFO:teuthology.orchestra.run.smithi175.stdout:Install 12 Packages 2024-09-21T11:32:32.408 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:32.408 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 38 M 2024-09-21T11:32:32.408 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 212 M 2024-09-21T11:32:32.409 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:32:32.612 INFO:teuthology.orchestra.run.smithi175.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-21T11:32:32.646 INFO:teuthology.orchestra.run.smithi175.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.6 MB/s | 801 kB 00:00 2024-09-21T11:32:32.746 INFO:teuthology.orchestra.run.smithi175.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 245 kB/s | 33 kB 00:00 2024-09-21T11:32:32.796 INFO:teuthology.orchestra.run.smithi175.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 363 kB/s | 18 kB 00:00 2024-09-21T11:32:32.847 INFO:teuthology.orchestra.run.smithi175.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 772 kB/s | 38 kB 00:00 2024-09-21T11:32:32.956 INFO:teuthology.orchestra.run.smithi175.stdout:(6/12): ceph-mgr-diskprediction-local-19.3.0-51 15 MB/s | 7.4 MB 00:00 2024-09-21T11:32:33.072 INFO:teuthology.orchestra.run.smithi175.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.0 MB/s | 3.0 MB 00:00 2024-09-21T11:32:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 12/12 2024-09-21T11:32:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el 1/12 2024-09-21T11:32:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noa 2/12 2024-09-21T11:32:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9 3/12 2024-09-21T11:32:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-21T11:32:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-21T11:32:33.148 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-21T11:32:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-21T11:32:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-21T11:32:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-21T11:32:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-21T11:32:33.149 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-21T11:32:33.248 INFO:teuthology.orchestra.run.smithi175.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 12 MB/s | 4.9 MB 00:00 2024-09-21T11:32:33.273 INFO:teuthology.orchestra.run.smithi175.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 773 kB/s | 245 kB 00:00 2024-09-21T11:32:33.340 INFO:teuthology.orchestra.run.smithi175.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-09-21T11:32:33.365 INFO:teuthology.orchestra.run.smithi175.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.1 MB/s | 250 kB 00:00 2024-09-21T11:32:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-21T11:32:33.595 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:33.596 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:32:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T11:32:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T11:32:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T11:32:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T11:32:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T11:32:33.596 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T11:32:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T11:32:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T11:32:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T11:32:33.597 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:33.597 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:32:33.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-21T11:32:34.450 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:10 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:32:34.593 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:32:34.594 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-21T11:32:34.594 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-21T11:32:34.594 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-21T11:32:34.594 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:32:34.594 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 7.4 M 2024-09-21T11:32:34.594 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T11:32:34.595 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:34.596 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:32:34.596 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-09-21T11:32:34.596 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-09-21T11:32:34.596 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:34.596 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 38 M 2024-09-21T11:32:34.596 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 212 M 2024-09-21T11:32:34.596 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:32:34.782 INFO:teuthology.orchestra.run.smithi175.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-21T11:32:34.783 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:34.783 INFO:teuthology.orchestra.run.smithi175.stdout:Total 16 MB/s | 38 MB 00:02 2024-09-21T11:32:34.904 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-21T11:32:34.938 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.8 MB/s | 801 kB 00:00 2024-09-21T11:32:35.138 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:32:35.164 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:32:35.165 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:32:35.246 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-51 16 MB/s | 7.4 MB 00:00 2024-09-21T11:32:35.534 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-09-21T11:32:35.603 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:32:35.603 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:32:35.730 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 38 kB/s | 18 kB 00:00 2024-09-21T11:32:36.089 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-21T11:32:36.377 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:32:36.469 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T11:32:36.513 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T11:32:36.612 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T11:32:36.656 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T11:32:36.822 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.5 MB/s | 4.9 MB 00:01 2024-09-21T11:32:36.840 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T11:32:36.847 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 323 kB/s | 245 kB 00:00 2024-09-21T11:32:36.882 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T11:32:36.964 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T11:32:37.614 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T11:32:37.622 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 323 kB/s | 250 kB 00:00 2024-09-21T11:32:37.733 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T11:32:37.964 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-21T11:32:38.165 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 949 kB/s | 3.0 MB 00:03 2024-09-21T11:32:38.964 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T11:32:39.165 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-21T11:32:39.166 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:39.166 INFO:teuthology.orchestra.run.smithi043.stdout:Total 8.4 MB/s | 38 MB 00:04 2024-09-21T11:32:39.540 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:32:39.568 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:32:39.568 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:32:40.007 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T11:32:40.031 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:32:40.032 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:32:40.037 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 12/12 2024-09-21T11:32:40.820 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:32:40.887 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-21T11:32:40.938 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-21T11:32:41.047 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-21T11:32:41.090 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-21T11:32:41.273 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-21T11:32:41.305 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-21T11:32:41.383 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-21T11:32:42.097 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-21T11:32:42.232 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-21T11:32:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 12/12 2024-09-21T11:32:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 1/12 2024-09-21T11:32:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T11:32:42.658 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T11:32:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T11:32:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T11:32:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T11:32:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T11:32:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T11:32:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T11:32:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T11:32:42.659 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T11:32:43.097 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T11:32:43.098 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T11:32:43.099 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T11:32:43.099 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T11:32:43.099 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T11:32:43.099 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:43.099 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:43.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-rook 2024-09-21T11:32:43.588 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T11:32:44.045 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:19 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:44.179 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:44.180 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:44.180 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T11:32:44.180 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:44.180 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 49 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T11:32:44.181 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T11:32:44.182 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:44.182 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:32:44.182 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:44.182 INFO:teuthology.orchestra.run.smithi175.stdout:Install 13 Packages 2024-09-21T11:32:44.182 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:44.182 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 2.2 M 2024-09-21T11:32:44.183 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 26 M 2024-09-21T11:32:44.183 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:32:44.667 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-21T11:32:44.695 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 12/12 2024-09-21T11:32:44.712 INFO:teuthology.orchestra.run.smithi175.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 210 kB/s | 26 kB 00:00 2024-09-21T11:32:44.728 INFO:teuthology.orchestra.run.smithi175.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 138 kB/s | 19 kB 00:00 2024-09-21T11:32:44.745 INFO:teuthology.orchestra.run.smithi175.stdout:(3/13): ceph-mgr-rook-19.3.0-5103.g5c1cdc3c.el9 317 kB/s | 49 kB 00:00 2024-09-21T11:32:44.871 INFO:teuthology.orchestra.run.smithi175.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-21T11:32:44.896 INFO:teuthology.orchestra.run.smithi175.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-09-21T11:32:44.930 INFO:teuthology.orchestra.run.smithi175.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 910 kB/s | 54 kB 00:00 2024-09-21T11:32:44.963 INFO:teuthology.orchestra.run.smithi175.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 482 kB/s | 32 kB 00:00 2024-09-21T11:32:44.988 INFO:teuthology.orchestra.run.smithi175.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 249 kB/s | 14 kB 00:00 2024-09-21T11:32:45.014 INFO:teuthology.orchestra.run.smithi175.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.0 MB/s | 279 kB 00:00 2024-09-21T11:32:45.039 INFO:teuthology.orchestra.run.smithi175.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.9 MB/s | 220 kB 00:00 2024-09-21T11:32:45.073 INFO:teuthology.orchestra.run.smithi175.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.3 MB/s | 90 kB 00:00 2024-09-21T11:32:45.098 INFO:teuthology.orchestra.run.smithi175.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-09-21T11:32:45.131 INFO:teuthology.orchestra.run.smithi175.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-21T11:32:45.134 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:45.134 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.3 MB/s | 2.2 MB 00:00 2024-09-21T11:32:45.199 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:32:45.214 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:32:45.214 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:32:45.302 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:32:45.834 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:32:46.006 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T11:32:46.066 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T11:32:46.221 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T11:32:46.258 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T11:32:46.300 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T11:32:46.403 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T11:32:46.512 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T11:32:46.568 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T11:32:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 12/12 2024-09-21T11:32:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cd 1/12 2024-09-21T11:32:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-21T11:32:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-21T11:32:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-21T11:32:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-21T11:32:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-21T11:32:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-21T11:32:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-21T11:32:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-21T11:32:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-21T11:32:47.176 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-21T11:32:47.300 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T11:32:47.340 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T11:32:47.373 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T11:32:47.422 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T11:32:47.452 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 13/13 2024-09-21T11:32:47.613 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-21T11:32:47.613 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:47.613 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:32:47.613 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:47.613 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:47.614 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:32:47.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-rook 2024-09-21T11:32:48.115 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 13/13 2024-09-21T11:32:48.115 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/13 2024-09-21T11:32:48.115 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T11:32:48.115 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T11:32:48.115 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T11:32:48.115 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T11:32:48.116 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T11:32:48.116 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T11:32:48.116 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T11:32:48.116 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T11:32:48.116 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T11:32:48.117 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T11:32:48.117 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T11:32:48.451 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T11:32:48.452 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T11:32:48.453 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T11:32:48.453 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T11:32:48.453 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:48.453 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:48.561 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:24 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:32:48.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T11:32:48.700 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:32:48.701 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:48.701 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-21T11:32:48.701 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 49 k 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T11:32:48.702 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T11:32:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T11:32:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:48.703 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:32:48.703 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:48.703 INFO:teuthology.orchestra.run.smithi043.stdout:Install 13 Packages 2024-09-21T11:32:48.703 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:48.703 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.2 M 2024-09-21T11:32:48.703 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 26 M 2024-09-21T11:32:48.704 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:32:49.179 INFO:teuthology.orchestra.run.smithi043.stdout:(1/13): ceph-mgr-rook-19.3.0-5103.g5c1cdc3c.el9 296 kB/s | 49 kB 00:00 2024-09-21T11:32:49.284 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:24 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:49.418 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:49.419 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:49.419 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T11:32:49.419 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:49.419 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 154 k 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout:Install 6 Packages 2024-09-21T11:32:49.420 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:49.421 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 7.0 M 2024-09-21T11:32:49.421 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 33 M 2024-09-21T11:32:49.421 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:32:49.520 INFO:teuthology.orchestra.run.smithi043.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 38 kB/s | 19 kB 00:00 2024-09-21T11:32:49.571 INFO:teuthology.orchestra.run.smithi043.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 46 kB/s | 26 kB 00:00 2024-09-21T11:32:50.004 INFO:teuthology.orchestra.run.smithi043.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 269 kB/s | 222 kB 00:00 2024-09-21T11:32:50.015 INFO:teuthology.orchestra.run.smithi175.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-21T11:32:50.040 INFO:teuthology.orchestra.run.smithi175.stdout:(2/6): ceph-mgr-cephadm-19.3.0-5103.g5c1cdc3c.e 683 kB/s | 154 kB 00:00 2024-09-21T11:32:50.099 INFO:teuthology.orchestra.run.smithi175.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-21T11:32:50.124 INFO:teuthology.orchestra.run.smithi175.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 691 kB/s | 58 kB 00:00 2024-09-21T11:32:50.150 INFO:teuthology.orchestra.run.smithi175.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-21T11:32:50.229 INFO:teuthology.orchestra.run.smithi043.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 224 kB/s | 159 kB 00:00 2024-09-21T11:32:50.282 INFO:teuthology.orchestra.run.smithi043.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 775 kB/s | 32 kB 00:00 2024-09-21T11:32:50.305 INFO:teuthology.orchestra.run.smithi043.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 565 kB/s | 14 kB 00:00 2024-09-21T11:32:50.330 INFO:teuthology.orchestra.run.smithi043.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.6 MB/s | 220 kB 00:00 2024-09-21T11:32:50.381 INFO:teuthology.orchestra.run.smithi043.stdout:(9/13): python3-kubernetes-26.1.0-2.el9.noarch. 21 MB/s | 1.0 MB 00:00 2024-09-21T11:32:50.406 INFO:teuthology.orchestra.run.smithi043.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T11:32:50.431 INFO:teuthology.orchestra.run.smithi043.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 324 kB/s | 279 kB 00:00 2024-09-21T11:32:50.456 INFO:teuthology.orchestra.run.smithi043.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-21T11:32:50.648 INFO:teuthology.orchestra.run.smithi043.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 83 kB/s | 54 kB 00:00 2024-09-21T11:32:50.651 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:50.651 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.1 MB/s | 2.2 MB 00:01 2024-09-21T11:32:50.725 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:32:50.742 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:32:50.743 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:32:50.841 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:32:50.841 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:32:51.191 INFO:teuthology.orchestra.run.smithi175.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.3 MB/s | 6.0 MB 00:01 2024-09-21T11:32:51.194 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:51.194 INFO:teuthology.orchestra.run.smithi175.stdout:Total 4.0 MB/s | 7.0 MB 00:01 2024-09-21T11:32:51.263 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:32:51.273 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:32:51.273 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:32:51.381 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:32:51.383 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:32:51.383 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:32:51.562 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-21T11:32:51.626 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-21T11:32:51.682 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-21T11:32:51.695 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:32:51.717 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-21T11:32:51.759 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-21T11:32:51.863 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T11:32:51.865 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-21T11:32:51.909 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T11:32:51.984 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-21T11:32:52.037 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-21T11:32:52.425 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T11:32:52.512 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T11:32:52.589 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T11:32:52.634 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 6/6 2024-09-21T11:32:52.805 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-21T11:32:52.858 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-21T11:32:52.892 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-21T11:32:52.949 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-21T11:32:52.977 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 13/13 2024-09-21T11:32:53.440 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 6/6 2024-09-21T11:32:53.440 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/6 2024-09-21T11:32:53.441 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T11:32:53.441 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T11:32:53.441 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T11:32:53.441 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T11:32:53.708 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T11:32:53.708 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:53.708 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:32:53.708 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:53.709 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T11:32:53.709 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T11:32:53.709 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T11:32:53.709 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T11:32:53.709 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T11:32:53.709 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:53.709 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:53.764 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 13/13 2024-09-21T11:32:53.764 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/13 2024-09-21T11:32:53.764 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-21T11:32:53.764 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-21T11:32:53.764 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-21T11:32:53.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-21T11:32:53.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-21T11:32:53.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-21T11:32:53.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-21T11:32:53.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-21T11:32:53.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-21T11:32:53.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-21T11:32:53.765 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-21T11:32:53.953 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-fuse 2024-09-21T11:32:54.119 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T11:32:54.120 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T11:32:54.121 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T11:32:54.121 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T11:32:54.121 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T11:32:54.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:54.121 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:32:54.389 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-cephadm 2024-09-21T11:32:54.545 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:29 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:54.679 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 899 k 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout:Install 2 Packages 2024-09-21T11:32:54.680 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:54.681 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 979 k 2024-09-21T11:32:54.681 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 2.9 M 2024-09-21T11:32:54.681 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:32:54.926 INFO:teuthology.orchestra.run.smithi175.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 546 kB/s | 80 kB 00:00 2024-09-21T11:32:55.001 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:30 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:32:55.051 INFO:teuthology.orchestra.run.smithi175.stdout:(2/2): ceph-fuse-19.3.0-5103.g5c1cdc3c.el9.x86_ 3.2 MB/s | 899 kB 00:00 2024-09-21T11:32:55.051 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:55.052 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.6 MB/s | 979 kB 00:00 2024-09-21T11:32:55.063 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:32:55.085 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:32:55.086 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:32:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:32:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:55.141 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-21T11:32:55.142 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:55.142 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:32:55.142 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 154 k 2024-09-21T11:32:55.142 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-21T11:32:55.142 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T11:32:55.142 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T11:32:55.142 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T11:32:55.142 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T11:32:55.142 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T11:32:55.142 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:55.143 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:32:55.143 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:55.143 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-09-21T11:32:55.143 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:55.143 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 7.0 M 2024-09-21T11:32:55.143 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 33 M 2024-09-21T11:32:55.144 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:32:55.167 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:32:55.167 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:32:55.355 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:32:55.491 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-21T11:32:55.494 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T11:32:55.517 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): ceph-mgr-cephadm-19.3.0-5103.g5c1cdc3c.e 763 kB/s | 154 kB 00:00 2024-09-21T11:32:55.557 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/2 2024-09-21T11:32:55.567 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-21T11:32:55.584 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 862 kB/s | 58 kB 00:00 2024-09-21T11:32:55.609 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.7 MB/s | 74 kB 00:00 2024-09-21T11:32:55.735 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-21T11:32:55.737 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:55.737 INFO:teuthology.orchestra.run.smithi043.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-09-21T11:32:55.811 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:32:55.823 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:32:55.823 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:32:55.930 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:32:55.931 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:32:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/2 2024-09-21T11:32:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/2 2024-09-21T11:32:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:32:56.424 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-21T11:32:56.478 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T11:32:56.479 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:32:56.479 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T11:32:56.479 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:56.479 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:56.479 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-21T11:32:56.746 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-volume 2024-09-21T11:32:57.034 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-21T11:32:57.134 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-21T11:32:57.219 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-21T11:32:57.249 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 6/6 2024-09-21T11:32:57.348 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:32 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:32:57.485 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:32:57.485 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:57.485 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T11:32:57.486 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:57.486 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:32:57.486 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 287 k 2024-09-21T11:32:57.486 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:57.486 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:32:57.486 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:32:57.486 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-21T11:32:57.486 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:57.487 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 287 k 2024-09-21T11:32:57.487 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 1.4 M 2024-09-21T11:32:57.487 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:32:57.714 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-volume-19.3.0-5103.g5c1cdc3c.el9.noarch.rp 1.2 MB/s | 287 kB 00:00 2024-09-21T11:32:57.715 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:57.715 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.2 MB/s | 287 kB 00:00 2024-09-21T11:32:57.715 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:32:57.723 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:32:57.723 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:32:57.751 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:32:57.751 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:32:57.972 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:32:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 6/6 2024-09-21T11:32:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/6 2024-09-21T11:32:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-21T11:32:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-21T11:32:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-21T11:32:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-21T11:32:58.040 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:58.056 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:58.056 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:32:58.056 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T11:32:58.056 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:58.307 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-21T11:32:58.307 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:58.307 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:32:58.307 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:58.307 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T11:32:58.308 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T11:32:58.308 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T11:32:58.308 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T11:32:58.308 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T11:32:58.308 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:58.308 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:32:58.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-fuse 2024-09-21T11:32:59.015 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:32:59.015 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:59.015 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:32:59.015 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:32:59.015 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:32:59.015 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:32:59.199 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:35 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:32:59.301 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados-devel 2024-09-21T11:32:59.338 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:32:59.338 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 899 k 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout:Install 2 Packages 2024-09-21T11:32:59.339 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:32:59.340 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 979 k 2024-09-21T11:32:59.340 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 2.9 M 2024-09-21T11:32:59.340 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:32:59.628 INFO:teuthology.orchestra.run.smithi043.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-21T11:32:59.762 INFO:teuthology.orchestra.run.smithi043.stdout:(2/2): ceph-fuse-19.3.0-5103.g5c1cdc3c.el9.x86_ 3.0 MB/s | 899 kB 00:00 2024-09-21T11:32:59.762 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:32:59.763 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.3 MB/s | 979 kB 00:00 2024-09-21T11:32:59.775 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:32:59.799 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:32:59.799 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:32:59.887 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:32:59.887 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:32:59.903 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:34 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:00.038 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 126 k 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 126 k 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 456 k 2024-09-21T11:33:00.039 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:33:00.080 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:33:00.227 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-21T11:33:00.242 INFO:teuthology.orchestra.run.smithi175.stdout:librados-devel-19.3.0-5103.g5c1cdc3c.el9.x86_64 624 kB/s | 126 kB 00:00 2024-09-21T11:33:00.243 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:00.244 INFO:teuthology.orchestra.run.smithi175.stdout:Total 619 kB/s | 126 kB 00:00 2024-09-21T11:33:00.244 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:33:00.249 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:33:00.249 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:33:00.298 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/2 2024-09-21T11:33:00.313 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:33:00.314 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:33:00.463 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:33:00.575 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:00.951 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2/2 2024-09-21T11:33:00.951 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/2 2024-09-21T11:33:01.009 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-21T11:33:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:01.234 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:33:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-21T11:33:01.234 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:01.234 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:01.268 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:01.268 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:01.268 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:33:01.268 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:33:01.268 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:01.269 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:01.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-volume 2024-09-21T11:33:01.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs2 2024-09-21T11:33:02.127 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:38 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:02.127 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:37 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:02.221 INFO:teuthology.orchestra.run.smithi175.stdout:Package libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:02.265 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:02.266 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:02.266 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-21T11:33:02.266 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:02.266 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:33:02.266 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.3.0-5103.g5c1cdc3c.el9 ceph-noarch 287 k 2024-09-21T11:33:02.267 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:33:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-21T11:33:02.267 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 287 k 2024-09-21T11:33:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 1.4 M 2024-09-21T11:33:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-21T11:33:02.268 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:02.322 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs-devel 2024-09-21T11:33:02.499 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-volume-19.3.0-5103.g5c1cdc3c.el9.noarch.rp 1.2 MB/s | 287 kB 00:00 2024-09-21T11:33:02.500 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:02.500 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.2 MB/s | 287 kB 00:00 2024-09-21T11:33:02.500 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:33:02.510 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:33:02.510 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:33:02.543 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:33:02.543 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:33:02.776 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:33:02.860 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:33:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:33:02.877 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:33:02.877 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T11:33:02.877 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:02.908 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:37 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:03.042 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:03.043 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:03.043 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-21T11:33:03.043 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:03.043 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:33:03.043 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 32 k 2024-09-21T11:33:03.043 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:03.043 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:33:03.043 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:03.043 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-21T11:33:03.043 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:03.044 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 32 k 2024-09-21T11:33:03.044 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 142 k 2024-09-21T11:33:03.044 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:33:03.196 INFO:teuthology.orchestra.run.smithi175.stdout:libcephfs-devel-19.3.0-5103.g5c1cdc3c.el9.x86_6 209 kB/s | 32 kB 00:00 2024-09-21T11:33:03.196 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:03.197 INFO:teuthology.orchestra.run.smithi175.stdout:Total 207 kB/s | 32 kB 00:00 2024-09-21T11:33:03.197 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:33:03.200 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:33:03.200 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:33:03.220 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:33:03.220 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:33:03.301 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:33:03.434 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:03.830 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 1/1 2024-09-21T11:33:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:03.919 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:33:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.3.0-5103.g5c1cdc3c.el9.noarch 2024-09-21T11:33:03.919 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:03.919 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:04.072 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:04.072 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:04.072 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:33:04.072 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:33:04.073 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:04.073 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:04.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados-devel 2024-09-21T11:33:04.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados2 2024-09-21T11:33:04.829 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:39 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:04.855 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:04.920 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:04.963 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:04.964 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-21T11:33:04.965 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:04.995 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:04.995 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:04.995 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-21T11:33:04.995 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:04.995 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:33:04.995 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 126 k 2024-09-21T11:33:04.996 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:04.996 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:33:04.996 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:04.996 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-21T11:33:04.996 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:04.996 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 126 k 2024-09-21T11:33:04.996 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 456 k 2024-09-21T11:33:04.997 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:33:05.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librbd1 2024-09-21T11:33:05.213 INFO:teuthology.orchestra.run.smithi043.stdout:librados-devel-19.3.0-5103.g5c1cdc3c.el9.x86_64 586 kB/s | 126 kB 00:00 2024-09-21T11:33:05.213 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:05.214 INFO:teuthology.orchestra.run.smithi043.stdout:Total 582 kB/s | 126 kB 00:00 2024-09-21T11:33:05.214 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:33:05.220 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:33:05.220 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:33:05.288 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:33:05.288 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:33:05.456 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:33:05.564 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:05.603 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:40 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:05.692 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:05.735 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:05.736 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-21T11:33:05.736 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:05.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rados 2024-09-21T11:33:05.980 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:06.259 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:33:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:33:06.259 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:06.260 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:06.376 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:41 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:06.466 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:06.509 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:06.510 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-21T11:33:06.510 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:06.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs2 2024-09-21T11:33:06.569 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rgw 2024-09-21T11:33:07.153 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:42 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:07.163 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:07.245 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:07.257 INFO:teuthology.orchestra.run.smithi043.stdout:Package libcephfs2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:07.288 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:07.289 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-21T11:33:07.289 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:07.304 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:07.305 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-21T11:33:07.305 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:07.352 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-cephfs 2024-09-21T11:33:07.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs-devel 2024-09-21T11:33:07.933 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:42 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:07.985 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:08.022 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:08.066 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:08.067 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-21T11:33:08.067 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:08.121 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rbd 2024-09-21T11:33:08.128 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:08.128 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:08.128 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-21T11:33:08.129 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:08.129 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:33:08.129 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 32 k 2024-09-21T11:33:08.129 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:08.129 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:33:08.129 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:08.129 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-21T11:33:08.129 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:08.130 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 32 k 2024-09-21T11:33:08.130 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 142 k 2024-09-21T11:33:08.130 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:33:08.284 INFO:teuthology.orchestra.run.smithi043.stdout:libcephfs-devel-19.3.0-5103.g5c1cdc3c.el9.x86_6 206 kB/s | 32 kB 00:00 2024-09-21T11:33:08.284 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:08.284 INFO:teuthology.orchestra.run.smithi043.stdout:Total 204 kB/s | 32 kB 00:00 2024-09-21T11:33:08.285 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:33:08.288 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:33:08.288 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:33:08.310 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:33:08.310 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:33:08.404 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:33:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:08.707 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:43 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:08.796 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:08.841 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:08.842 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-21T11:33:08.843 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:08.902 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-fuse 2024-09-21T11:33:08.931 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:09.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:09.184 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:33:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:33:09.184 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:09.184 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:09.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados2 2024-09-21T11:33:09.493 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:44 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:09.626 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:09.627 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:09.627 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-21T11:33:09.627 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:09.627 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:33:09.627 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 85 k 2024-09-21T11:33:09.627 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:09.627 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:33:09.627 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:09.627 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-21T11:33:09.628 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:09.628 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 85 k 2024-09-21T11:33:09.628 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 231 k 2024-09-21T11:33:09.628 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:33:09.832 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-fuse-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 417 kB/s | 85 kB 00:00 2024-09-21T11:33:09.832 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:09.833 INFO:teuthology.orchestra.run.smithi175.stdout:Total 414 kB/s | 85 kB 00:00 2024-09-21T11:33:09.833 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:33:09.839 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:33:09.840 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:33:09.903 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:33:09.903 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:33:09.991 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:45 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:10.046 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:33:10.096 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:10.143 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:10.144 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-21T11:33:10.144 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:10.154 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:10.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librbd1 2024-09-21T11:33:10.530 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:10.808 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:10.808 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:10.808 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:33:10.808 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:33:10.808 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:10.808 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:10.824 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:46 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:10.917 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:10.963 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:10.964 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-21T11:33:10.964 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:11.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-mirror 2024-09-21T11:33:11.038 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rados 2024-09-21T11:33:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:46 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:11.666 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:47 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:11.759 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rados-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:11.770 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:11.771 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:11.771 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-21T11:33:11.771 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:11.771 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:33:11.771 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.2 M 2024-09-21T11:33:11.771 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:11.771 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:33:11.771 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:11.772 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-21T11:33:11.772 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:11.772 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 3.2 M 2024-09-21T11:33:11.772 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 14 M 2024-09-21T11:33:11.772 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:33:11.805 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:11.806 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-21T11:33:11.806 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:11.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rgw 2024-09-21T11:33:12.118 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-mirror-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 9.2 MB/s | 3.2 MB 00:00 2024-09-21T11:33:12.119 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:12.119 INFO:teuthology.orchestra.run.smithi175.stdout:Total 9.2 MB/s | 3.2 MB 00:00 2024-09-21T11:33:12.119 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:33:12.129 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:33:12.129 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:33:12.212 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:33:12.212 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:33:12.488 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:48 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:12.582 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rgw-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:12.628 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:12.629 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-21T11:33:12.629 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:12.684 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:33:12.702 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-cephfs 2024-09-21T11:33:12.744 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:12.770 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:12.770 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:33:12.770 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T11:33:12.770 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T11:33:12.770 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T11:33:12.771 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:13.310 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:49 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:13.403 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-cephfs-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:13.449 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:13.450 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-21T11:33:13.451 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:13.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rbd 2024-09-21T11:33:13.922 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:13.922 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:13.923 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:33:13.923 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:33:13.923 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:13.923 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:14.127 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:50 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:14.153 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-nbd 2024-09-21T11:33:14.220 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 is already installed. 2024-09-21T11:33:14.266 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:14.268 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-21T11:33:14.268 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:14.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-fuse 2024-09-21T11:33:14.746 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:49 ago on Sat 21 Sep 2024 11:31:25 AM UTC. 2024-09-21T11:33:14.881 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T11:33:14.882 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:14.882 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-21T11:33:14.882 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:14.882 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T11:33:14.882 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 170 k 2024-09-21T11:33:14.882 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:14.882 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T11:33:14.882 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T11:33:14.883 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-21T11:33:14.883 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:14.883 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 170 k 2024-09-21T11:33:14.883 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 490 k 2024-09-21T11:33:14.883 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T11:33:14.959 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:50 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:15.097 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-nbd-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 799 kB/s | 170 kB 00:00 2024-09-21T11:33:15.097 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:15.097 INFO:teuthology.orchestra.run.smithi175.stdout:Total 792 kB/s | 170 kB 00:00 2024-09-21T11:33:15.098 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T11:33:15.098 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:15.099 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:15.099 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-21T11:33:15.099 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:15.099 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:33:15.099 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 85 k 2024-09-21T11:33:15.099 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:15.099 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:33:15.099 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:15.099 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-21T11:33:15.099 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:15.100 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 85 k 2024-09-21T11:33:15.100 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 231 k 2024-09-21T11:33:15.100 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:33:15.105 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T11:33:15.105 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T11:33:15.170 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T11:33:15.170 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T11:33:15.287 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-fuse-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 456 kB/s | 85 kB 00:00 2024-09-21T11:33:15.287 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:15.287 INFO:teuthology.orchestra.run.smithi043.stdout:Total 453 kB/s | 85 kB 00:00 2024-09-21T11:33:15.288 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:33:15.295 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:33:15.295 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:33:15.327 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T11:33:15.363 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:33:15.363 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:33:15.427 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:15.520 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:33:15.629 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:15.896 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:16.016 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:16.163 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:16.164 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:16.164 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T11:33:16.164 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:33:16.164 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:33:16.164 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T11:33:16.290 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:16.290 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:16.290 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:33:16.290 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:33:16.291 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:16.291 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:16.399 DEBUG:teuthology.parallel:result is None 2024-09-21T11:33:16.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-mirror 2024-09-21T11:33:17.138 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:53 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:17.281 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-21T11:33:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:33:17.281 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 3.2 M 2024-09-21T11:33:17.282 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:33:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-21T11:33:17.282 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 3.2 M 2024-09-21T11:33:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 14 M 2024-09-21T11:33:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:33:17.615 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-mirror-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 9.6 MB/s | 3.2 MB 00:00 2024-09-21T11:33:17.615 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:17.616 INFO:teuthology.orchestra.run.smithi043.stdout:Total 9.5 MB/s | 3.2 MB 00:00 2024-09-21T11:33:17.616 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:33:17.626 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:33:17.626 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:33:17.715 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:33:17.715 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:33:18.250 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:33:18.315 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:18.343 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:18.343 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T11:33:18.343 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T11:33:18.343 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T11:33:18.343 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T11:33:18.343 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:19.456 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:19.457 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:33:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:33:19.457 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:19.457 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:19.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-nbd 2024-09-21T11:33:20.305 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:56 ago on Sat 21 Sep 2024 11:31:24 AM UTC. 2024-09-21T11:33:20.442 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-21T11:33:20.443 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-21T11:33:20.443 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:20.443 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-21T11:33:20.443 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.3.0-5103.g5c1cdc3c.el9 ceph 170 k 2024-09-21T11:33:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:20.444 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-21T11:33:20.444 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-21T11:33:20.444 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-21T11:33:20.444 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:20.444 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 170 k 2024-09-21T11:33:20.444 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 490 k 2024-09-21T11:33:20.444 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-21T11:33:20.652 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-nbd-19.3.0-5103.g5c1cdc3c.el9.x86_64.rpm 821 kB/s | 170 kB 00:00 2024-09-21T11:33:20.653 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:20.653 INFO:teuthology.orchestra.run.smithi043.stdout:Total 815 kB/s | 170 kB 00:00 2024-09-21T11:33:20.653 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-21T11:33:20.661 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-21T11:33:20.661 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-21T11:33:20.730 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-21T11:33:20.730 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-21T11:33:20.901 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-21T11:33:21.003 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:21.424 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:21.689 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 1/1 2024-09-21T11:33:21.689 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:21.689 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-21T11:33:21.689 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.3.0-5103.g5c1cdc3c.el9.x86_64 2024-09-21T11:33:21.689 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:33:21.689 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-21T11:33:21.950 DEBUG:teuthology.parallel:result is None 2024-09-21T11:33:21.951 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:33:22.092 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T11:33:22.114 INFO:teuthology.orchestra.run.smithi043.stdout:19.3.0-5103.g5c1cdc3c.el9 2024-09-21T11:33:22.114 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5103.g5c1cdc3c.el9 2024-09-21T11:33:22.115 INFO:teuthology.task.install:The correct ceph version 19.3.0-5103.g5c1cdc3c is installed. 2024-09-21T11:33:22.116 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:33:22.255 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T11:33:22.280 INFO:teuthology.orchestra.run.smithi175.stdout:19.3.0-5103.g5c1cdc3c.el9 2024-09-21T11:33:22.280 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5103.g5c1cdc3c.el9 2024-09-21T11:33:22.280 INFO:teuthology.task.install:The correct ceph version 19.3.0-5103.g5c1cdc3c is installed. 2024-09-21T11:33:22.282 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T11:33:22.283 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:33:22.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T11:33:22.310 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:33:22.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T11:33:22.348 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T11:33:22.348 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:33:22.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T11:33:22.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T11:33:22.443 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:33:22.443 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T11:33:22.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T11:33:22.537 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T11:33:22.537 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:33:22.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T11:33:22.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T11:33:22.628 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:33:22.628 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T11:33:22.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T11:33:22.720 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T11:33:22.720 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:33:22.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T11:33:22.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T11:33:22.812 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:33:22.812 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T11:33:22.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T11:33:22.903 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T11:33:22.998 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301', 'cephadm_mode': 'root'} 2024-09-21T11:33:22.998 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:33:22.999 INFO:tasks.cephadm:Cluster fsid is 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:33:22.999 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T11:33:22.999 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.43', 'mon.c': '[v2:172.21.15.43:3301,v1:172.21.15.43:6790]', 'mon.b': '172.21.15.175'} 2024-09-21T11:33:22.999 INFO:tasks.cephadm:First mon is mon.a on smithi043 2024-09-21T11:33:22.999 INFO:tasks.cephadm:First mgr is y 2024-09-21T11:33:22.999 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T11:33:22.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-09-21T11:33:23.026 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2024-09-21T11:33:23.051 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T11:33:23.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:33:23.194 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83200/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5103-g5c1cdc3c', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5103.g5c1cdc3c'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 10:00:04.055809', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-main-2024-09-21-0813', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-21T11:33:23.329 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-guits-main-2024-09-21-0813, sha1 5c1cdc3c470dae78e91a0ed2bcf13509a046d301 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:33:23.330 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/x86_64/flavors/default/cephadm 2024-09-21T11:33:23.330 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/x86_64/flavors/default/cephadm 2024-09-21T11:33:23.330 DEBUG:teuthology.orchestra.run.smithi043:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T11:33:23.708 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r--. 1 ubuntu ubuntu 804189 Sep 21 11:33 /home/ubuntu/cephtest/cephadm 2024-09-21T11:33:23.708 DEBUG:teuthology.orchestra.run.smithi175:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-21-0813/5c1cdc3c470dae78e91a0ed2bcf13509a046d301/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T11:33:24.071 INFO:teuthology.orchestra.run.smithi175.stdout:-rw-r--r--. 1 ubuntu ubuntu 804189 Sep 21 11:33 /home/ubuntu/cephtest/cephadm 2024-09-21T11:33:24.072 DEBUG:teuthology.orchestra.run.smithi043:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T11:33:24.092 DEBUG:teuthology.orchestra.run.smithi175:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T11:33:24.124 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 on all hosts... 2024-09-21T11:33:24.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 pull 2024-09-21T11:33:24.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 pull 2024-09-21T11:33:25.724 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301... 2024-09-21T11:33:25.767 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301... 2024-09-21T11:33:58.328 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:33:58.329 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev)", 2024-09-21T11:33:58.329 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063", 2024-09-21T11:33:58.329 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2024-09-21T11:33:58.329 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472" 2024-09-21T11:33:58.329 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:33:58.329 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:33:58.390 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:33:58.390 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev)", 2024-09-21T11:33:58.390 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063", 2024-09-21T11:33:58.390 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-09-21T11:33:58.390 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472" 2024-09-21T11:33:58.390 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:33:58.390 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:33:58.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-09-21T11:33:58.455 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2024-09-21T11:33:58.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-09-21T11:33:58.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2024-09-21T11:33:58.548 INFO:tasks.cephadm:Writing seed config... 2024-09-21T11:33:58.549 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T11:33:58.549 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T11:33:58.549 INFO:tasks.cephadm: override: [client.0] debug ms = 1 2024-09-21T11:33:58.549 INFO:tasks.cephadm: override: [client.0] debug objecter = 20 2024-09-21T11:33:58.549 INFO:tasks.cephadm: override: [client.0] debug rados = 20 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [global] ms bind msgr1 = False 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [global] ms bind msgr2 = True 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T11:33:58.550 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-09-21T11:33:58.551 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-09-21T11:33:58.551 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-09-21T11:33:58.551 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-09-21T11:33:58.551 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-09-21T11:33:58.551 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T11:33:58.551 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-09-21T11:33:58.552 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:33:58.552 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T11:33:58.578 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4f2c5252-780d-11ef-baf6-efdc52797490 ms bind msgr1 = False ms bind msgr2 = True ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.0] debug ms = 1 debug objecter = 20 debug rados = 20 2024-09-21T11:33:58.579 DEBUG:teuthology.orchestra.run.smithi043:mon.a> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.a.service 2024-09-21T11:33:58.621 DEBUG:teuthology.orchestra.run.smithi043:mgr.y> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.y.service 2024-09-21T11:33:58.664 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T11:33:58.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 -v bootstrap --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.43 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T11:33:58.875 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-21T11:33:58.875 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301', '-v', 'bootstrap', '--fsid', '4f2c5252-780d-11ef-baf6-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.43', '--skip-admin-label'] 2024-09-21T11:33:58.876 INFO:teuthology.orchestra.run.smithi043.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T11:33:58.876 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying podman|docker is present... 2024-09-21T11:33:58.900 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-21T11:33:58.901 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying lvm2 is present... 2024-09-21T11:33:58.901 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying time synchronization is in place... 2024-09-21T11:33:58.908 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T11:33:58.909 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T11:33:58.915 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T11:33:58.915 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-21T11:33:58.922 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-09-21T11:33:58.929 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-09-21T11:33:58.929 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-09-21T11:33:58.929 INFO:teuthology.orchestra.run.smithi043.stdout:Repeating the final host check... 2024-09-21T11:33:58.952 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 5.2.2 2024-09-21T11:33:58.952 INFO:teuthology.orchestra.run.smithi043.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T11:33:58.952 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl is present 2024-09-21T11:33:58.952 INFO:teuthology.orchestra.run.smithi043.stdout:lvcreate is present 2024-09-21T11:33:58.959 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T11:33:58.960 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T11:33:58.965 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T11:33:58.965 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-21T11:33:58.972 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-09-21T11:33:58.979 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-09-21T11:33:58.979 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-09-21T11:33:58.979 INFO:teuthology.orchestra.run.smithi043.stdout:Host looks OK 2024-09-21T11:33:58.979 INFO:teuthology.orchestra.run.smithi043.stdout:Cluster fsid: 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:33:58.979 INFO:teuthology.orchestra.run.smithi043.stdout:Acquiring lock 140206992406896 on /run/cephadm/4f2c5252-780d-11ef-baf6-efdc52797490.lock 2024-09-21T11:33:58.979 INFO:teuthology.orchestra.run.smithi043.stdout:Lock 140206992406896 acquired on /run/cephadm/4f2c5252-780d-11ef-baf6-efdc52797490.lock 2024-09-21T11:33:58.980 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 3300 ... 2024-09-21T11:33:58.980 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 6789 ... 2024-09-21T11:33:58.980 INFO:teuthology.orchestra.run.smithi043.stdout:Base mon IP(s) is [172.21.15.43:3300, 172.21.15.43:6789], mon addrv is [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-21T11:33:58.983 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.43 metric 100 2024-09-21T11:33:58.983 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.43 metric 100 2024-09-21T11:33:58.986 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T11:33:58.986 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T11:33:58.986 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-09-21T11:33:58.989 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T11:33:58.989 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T11:33:58.989 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T11:33:58.989 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T11:33:58.989 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-21T11:33:58.989 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T11:33:58.990 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-09-21T11:33:58.990 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-09-21T11:33:58.990 INFO:teuthology.orchestra.run.smithi043.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T11:33:58.990 INFO:teuthology.orchestra.run.smithi043.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T11:33:58.991 INFO:teuthology.orchestra.run.smithi043.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301... 2024-09-21T11:33:59.459 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 2024-09-21T11:33:59.459 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301... 2024-09-21T11:33:59.459 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T11:33:59.459 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:832cb702be6ace10bd8c779318aa247eb0f26450f5c31f0789905499e59c463e 2024-09-21T11:33:59.459 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T11:33:59.459 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying config sha256:8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 2024-09-21T11:33:59.459 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T11:34:00.347 INFO:teuthology.orchestra.run.smithi043.stdout:ceph: stdout ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev) 2024-09-21T11:34:00.347 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph version: ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev) 2024-09-21T11:34:00.347 INFO:teuthology.orchestra.run.smithi043.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T11:34:01.211 INFO:teuthology.orchestra.run.smithi043.stdout:stat: stdout 167 167 2024-09-21T11:34:01.211 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial keys... 2024-09-21T11:34:02.067 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQApr+5mKuFvHRAAIeEgsi3CntylQyeS78NJEA== 2024-09-21T11:34:02.947 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQAqr+5miR5WFxAAim071+k2LS/wA4I4cMq7xQ== 2024-09-21T11:34:03.862 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQArr+5mE5PHEBAAoU3MHIfM0DVHvHEe/RmlBA== 2024-09-21T11:34:03.863 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial monmap... 2024-09-21T11:34:04.820 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T11:34:04.820 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T11:34:04.820 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:04.820 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T11:34:04.820 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool for a [v2:172.21.15.43:3300,v1:172.21.15.43:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T11:34:04.820 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-09-21T11:34:04.820 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: set fsid to 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:04.820 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T11:34:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:34:04.820 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mon... 2024-09-21T11:34:05.766 INFO:teuthology.orchestra.run.smithi043.stdout:create mon.a on 2024-09-21T11:34:05.982 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T11:34:06.128 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T11:34:06.316 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4f2c5252-780d-11ef-baf6-efdc52797490.target → /etc/systemd/system/ceph-4f2c5252-780d-11ef-baf6-efdc52797490.target. 2024-09-21T11:34:06.316 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4f2c5252-780d-11ef-baf6-efdc52797490.target → /etc/systemd/system/ceph-4f2c5252-780d-11ef-baf6-efdc52797490.target. 2024-09-21T11:34:06.559 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.a 2024-09-21T11:34:06.559 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.a.service: Unit ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.a.service not loaded. 2024-09-21T11:34:06.738 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4f2c5252-780d-11ef-baf6-efdc52797490.target.wants/ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.a.service → /etc/systemd/system/ceph-4f2c5252-780d-11ef-baf6-efdc52797490@.service. 2024-09-21T11:34:07.284 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T11:34:07.284 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-21T11:34:07.292 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T11:34:07.292 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-21T11:34:07.292 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-21T11:34:07.292 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T11:34:07.293 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon to start... 2024-09-21T11:34:07.293 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon... 2024-09-21T11:34:07.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:07 smithi043 systemd[1]: Started Ceph mon.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:34:08.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T11:34:08.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: monmap epoch 1 2024-09-21T11:34:08.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:08.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: last_changed 2024-09-21T11:34:04.177212+0000 2024-09-21T11:34:08.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: created 2024-09-21T11:34:04.177212+0000 2024-09-21T11:34:08.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: min_mon_release 19 (squid) 2024-09-21T11:34:08.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: election_strategy: 1 2024-09-21T11:34:08.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-21T11:34:08.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: fsmap 2024-09-21T11:34:08.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T11:34:08.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: mgrmap e1: no daemons active 2024-09-21T11:34:08.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:08 smithi043 ceph-mon[29342]: from='client.? 172.21.15.43:0/99998239' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T11:34:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T11:34:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout id: 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T11:34:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout services: 2024-09-21T11:34:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.590013s) 2024-09-21T11:34:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T11:34:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T11:34:08.403 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout data: 2024-09-21T11:34:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T11:34:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T11:34:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T11:34:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T11:34:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:mon is available 2024-09-21T11:34:08.404 INFO:teuthology.orchestra.run.smithi043.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T11:34:09.384 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:09.384 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T11:34:09.385 INFO:teuthology.orchestra.run.smithi043.stdout:Generating new minimal ceph.conf... 2024-09-21T11:34:09.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:09 smithi043 ceph-mon[29342]: from='client.? 172.21.15.43:0/3470929136' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T11:34:09.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:09 smithi043 ceph-mon[29342]: from='client.? 172.21.15.43:0/3470929136' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T11:34:10.378 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:10 smithi043 ceph-mon[29342]: from='client.? 172.21.15.43:0/3501684735' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:34:10.381 INFO:teuthology.orchestra.run.smithi043.stdout:Restarting the monitor... 2024-09-21T11:34:10.657 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:10 smithi043 systemd[1]: Stopping Ceph mon.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:34:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:10 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a[29338]: 2024-09-21T11:34:10.654+0000 7fe337f36640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T11:34:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:10 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a[29338]: 2024-09-21T11:34:10.654+0000 7fe337f36640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T11:34:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:10 smithi043 podman[29585]: 2024-09-21 11:34:10.778698521 +0000 UTC m=+0.232294311 container died f309ce962d752ab2d24c1d0f2780a9fe081023501bce63e6aeabee1d2a54b693 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T11:34:11.407 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:11 smithi043 podman[29585]: 2024-09-21 11:34:11.146561454 +0000 UTC m=+0.600157239 container cleanup f309ce962d752ab2d24c1d0f2780a9fe081023501bce63e6aeabee1d2a54b693 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T11:34:11.407 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:11 smithi043 bash[29585]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a 2024-09-21T11:34:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:11 smithi043 podman[29598]: 2024-09-21 11:34:11.405816193 +0000 UTC m=+0.622626522 container remove f309ce962d752ab2d24c1d0f2780a9fe081023501bce63e6aeabee1d2a54b693 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T11:34:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:11 smithi043 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T11:34:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:11 smithi043 systemd[1]: Stopped Ceph mon.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:34:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:11 smithi043 systemd[1]: Starting Ceph mon.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:34:12.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:11 smithi043 podman[29692]: 2024-09-21 11:34:11.681321317 +0000 UTC m=+0.016018707 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:34:12.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:11 smithi043 podman[29692]: 2024-09-21 11:34:11.781606177 +0000 UTC m=+0.116303560 container create 4062dd7fcef735b53b9051ca2cfd3d11de41564243d23bcdfd68f51637eee38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T11:34:12.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:11 smithi043 podman[29692]: 2024-09-21 11:34:11.975760096 +0000 UTC m=+0.310457488 container init 4062dd7fcef735b53b9051ca2cfd3d11de41564243d23bcdfd68f51637eee38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T11:34:12.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:11 smithi043 podman[29692]: 2024-09-21 11:34:11.979137862 +0000 UTC m=+0.313835243 container start 4062dd7fcef735b53b9051ca2cfd3d11de41564243d23bcdfd68f51637eee38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T11:34:12.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T11:34:12.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev), process ceph-mon, pid 2 2024-09-21T11:34:12.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: pidfile_write: ignore empty --pid-file 2024-09-21T11:34:12.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: load: jerasure load: lrc 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: RocksDB version: 7.9.2 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Git sha 0 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Compile date 2024-09-21 08:36:07 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: DB SUMMARY 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: DB Session ID: FYGFZAJOGZWUJ1TS17C9 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: CURRENT file: CURRENT 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95884 ; 2024-09-21T11:34:12.045 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.error_if_exists: 0 2024-09-21T11:34:12.046 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.create_if_missing: 0 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.paranoid_checks: 1 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.env: 0x556229721120 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.info_log: 0x55622b681700 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.statistics: (nil) 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.use_fsync: 0 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_log_file_size: 0 2024-09-21T11:34:12.047 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T11:34:12.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T11:34:12.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T11:34:12.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T11:34:12.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.allow_fallocate: 1 2024-09-21T11:34:12.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T11:34:12.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T11:34:12.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.use_direct_reads: 0 2024-09-21T11:34:12.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T11:34:12.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T11:34:12.049 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.db_log_dir: 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.wal_dir: 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.write_buffer_manager: 0x55622b685b80 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T11:34:12.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T11:34:12.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T11:34:12.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T11:34:12.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T11:34:12.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T11:34:12.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.unordered_write: 0 2024-09-21T11:34:12.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T11:34:12.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T11:34:12.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T11:34:12.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.row_cache: None 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.wal_filter: None 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.two_write_queues: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.wal_compression: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.atomic_flush: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.log_readahead_size: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T11:34:12.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_background_jobs: 2 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_background_compactions: -1 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_subcompactions: 1 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T11:34:12.053 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_open_files: -1 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_background_flushes: -1 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Compression algorithms supported: 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: kZSTD supported: 0 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: kXpressCompression supported: 0 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: kBZip2Compression supported: 0 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: kLZ4Compression supported: 1 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: kZlibCompression supported: 1 2024-09-21T11:34:12.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: kSnappyCompression supported: 1 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.merge_operator: 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_filter: None 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55622b681320) 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-09-21T11:34:12.055 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_type: 0 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_index_type: 0 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_shortening: 1 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: checksum: 4 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: no_block_cache: 0 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache: 0x55622b675610 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_options: 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: capacity : 536870912 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_shard_bits : 4 2024-09-21T11:34:12.056 INFO:journalctl@ceph.mon.a.smithi043.stdout: strict_capacity_limit : 0 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_compressed: (nil) 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: persistent_cache: (nil) 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size: 4096 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size_deviation: 10 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_restart_interval: 16 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_block_restart_interval: 1 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: metadata_block_size: 4096 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: partition_filters: 0 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: use_delta_encoding: 1 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: filter_policy: bloomfilter 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: whole_key_filtering: 1 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: verify_compression: 0 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-09-21T11:34:12.057 INFO:journalctl@ceph.mon.a.smithi043.stdout: format_version: 5 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_index_compression: 1 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_align: 0 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout: max_auto_readahead_size: 262144 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout: prepopulate_block_cache: 0 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout: initial_auto_readahead_size: 8192 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compression: NoCompression 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T11:34:12.058 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.num_levels: 7 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T11:34:12.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T11:34:12.060 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T11:34:12.060 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T11:34:12.060 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T11:34:12.060 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T11:34:12.060 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T11:34:12.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T11:34:12.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.inplace_update_support: 0 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T11:34:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.bloom_locality: 0 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.max_successive_merges: 0 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.ttl: 2592000 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.enable_blob_files: false 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.min_blob_size: 0 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T11:34:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f2053b3c-4883-48a5-a4c6-a61bca149384 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726918452006170, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726918452007193, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91843, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 289, "table_properties": {"data_size": 89891, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11920, "raw_average_key_size": 48, "raw_value_size": 83291, "raw_average_value_size": 335, "num_data_blocks": 12, "num_entries": 248, "num_filter_entries": 248, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726918452, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f2053b3c-4883-48a5-a4c6-a61bca149384", "db_session_id": "FYGFZAJOGZWUJ1TS17C9", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726918452007299, "job": 1, "event": "recovery_finished"} 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55622b6a4e00 2024-09-21T11:34:12.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: DB pointer 0x55622b7a6000 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** DB Stats ** 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T11:34:12.066 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 91.55 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 91.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 134.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T11:34:12.067 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 27.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 27.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache BinnedLRUCache@0x55622b675610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: starting mon.a rank 0 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: mon.a@-1(???) e1 preinit fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: mon.a@-1(???).mds e1 new map 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: mon.a@-1(???).mds e1 print_map 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: e1 2024-09-21T11:34:12.068 INFO:journalctl@ceph.mon.a.smithi043.stdout: btime 2024-09-21T11:34:07:249039+0000 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout: legacy client fscid: -1 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout: No filesystems configured 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 bash[29692]: 4062dd7fcef735b53b9051ca2cfd3d11de41564243d23bcdfd68f51637eee38c 2024-09-21T11:34:12.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 systemd[1]: Started Ceph mon.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:34:12.069 INFO:teuthology.orchestra.run.smithi043.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T11:34:12.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T11:34:12.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: monmap epoch 1 2024-09-21T11:34:12.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:12.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: last_changed 2024-09-21T11:34:04.177212+0000 2024-09-21T11:34:12.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: created 2024-09-21T11:34:04.177212+0000 2024-09-21T11:34:12.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: min_mon_release 19 (squid) 2024-09-21T11:34:12.356 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: election_strategy: 1 2024-09-21T11:34:12.356 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-21T11:34:12.356 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: fsmap 2024-09-21T11:34:12.356 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T11:34:12.356 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:12 smithi043 ceph-mon[29710]: mgrmap e1: no daemons active 2024-09-21T11:34:13.104 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T11:34:13.125 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T11:34:13.125 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mgr... 2024-09-21T11:34:13.125 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T11:34:13.126 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T11:34:13.387 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.y 2024-09-21T11:34:13.387 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.y.service: Unit ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.y.service not loaded. 2024-09-21T11:34:13.543 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4f2c5252-780d-11ef-baf6-efdc52797490.target.wants/ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.y.service → /etc/systemd/system/ceph-4f2c5252-780d-11ef-baf6-efdc52797490@.service. 2024-09-21T11:34:13.776 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3714965727' entity='client.admin' 2024-09-21T11:34:13.777 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:13 smithi043 systemd[1]: Starting Ceph mgr.y for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:34:14.054 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:13 smithi043 podman[29923]: 2024-09-21 11:34:13.677916224 +0000 UTC m=+0.016026437 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 2024-09-21T11:34:14.054 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:13 smithi043 podman[29923]: 2024-09-21 11:34:13.783965589 +0000 UTC m=+0.122075792 container create 4ca0cb7bf57c6af598cb451e44cfb4ade33d19c1a72c59c587c25b56f6a773e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T11:34:14.055 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:13 smithi043 podman[29923]: 2024-09-21 11:34:13.978351599 +0000 UTC m=+0.316461810 container init 4ca0cb7bf57c6af598cb451e44cfb4ade33d19c1a72c59c587c25b56f6a773e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T11:34:14.055 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:13 smithi043 podman[29923]: 2024-09-21 11:34:13.98105779 +0000 UTC m=+0.319168004 container start 4ca0cb7bf57c6af598cb451e44cfb4ade33d19c1a72c59c587c25b56f6a773e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T11:34:14.055 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:14 smithi043 bash[29923]: 4ca0cb7bf57c6af598cb451e44cfb4ade33d19c1a72c59c587c25b56f6a773e4 2024-09-21T11:34:14.069 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T11:34:14.069 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-21T11:34:14.076 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T11:34:14.076 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-21T11:34:14.076 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-21T11:34:14.077 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T11:34:14.085 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T11:34:14.086 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-21T11:34:14.093 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T11:34:14.094 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-21T11:34:14.094 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-21T11:34:14.094 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T11:34:14.094 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr to start... 2024-09-21T11:34:14.094 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr... 2024-09-21T11:34:14.330 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:14 smithi043 systemd[1]: Started Ceph mgr.y for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:34:14.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/671270774' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T11:34:15.230 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "4f2c5252-780d-11ef-baf6-efdc52797490", 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T11:34:15.233 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T11:34:15.234 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T11:34:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T11:34:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T11:34:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T11:34:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T11:34:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T11:34:15.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T11:34:07:249039+0000", 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T11:34:15.238 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T11:34:07.249730+0000", 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T11:34:15.239 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:34:15.240 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (1/15)... 2024-09-21T11:34:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3630837240' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T11:34:18.388 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:18.388 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "4f2c5252-780d-11ef-baf6-efdc52797490", 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T11:34:18.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T11:34:18.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:18.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T11:34:18.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T11:34:18.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:18.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T11:34:18.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:18.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T11:34:18.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T11:34:18.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T11:34:18.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T11:34:18.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T11:34:18.393 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T11:34:07:249039+0000", 2024-09-21T11:34:18.394 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:18.395 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T11:34:07.249730+0000", 2024-09-21T11:34:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:34:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T11:34:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:34:18.396 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (2/15)... 2024-09-21T11:34:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: Activating manager daemon y 2024-09-21T11:34:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: mgrmap e2: y(active, starting, since 0.00333217s) 2024-09-21T11:34:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: from='mgr.14100 172.21.15.43:0/3665383511' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:34:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: from='mgr.14100 172.21.15.43:0/3665383511' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:34:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: from='mgr.14100 172.21.15.43:0/3665383511' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:34:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: from='mgr.14100 172.21.15.43:0/3665383511' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:34:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: from='mgr.14100 172.21.15.43:0/3665383511' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:34:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: Manager daemon y is now available 2024-09-21T11:34:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: from='mgr.14100 172.21.15.43:0/3665383511' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:34:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: from='mgr.14100 172.21.15.43:0/3665383511' entity='mgr.y' 2024-09-21T11:34:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: from='mgr.14100 172.21.15.43:0/3665383511' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:34:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: from='mgr.14100 172.21.15.43:0/3665383511' entity='mgr.y' 2024-09-21T11:34:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:19 smithi043 ceph-mon[29710]: from='mgr.14100 172.21.15.43:0/3665383511' entity='mgr.y' 2024-09-21T11:34:21.390 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:21 smithi043 ceph-mon[29710]: mgrmap e3: y(active, since 1.00782s) 2024-09-21T11:34:21.390 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/157626173' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "4f2c5252-780d-11ef-baf6-efdc52797490", 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T11:34:21.724 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T11:34:21.725 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:21.726 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T11:34:07:249039+0000", 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T11:34:21.727 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T11:34:21.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T11:34:07.249730+0000", 2024-09-21T11:34:21.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T11:34:21.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-09-21T11:34:21.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T11:34:21.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:34:21.728 INFO:teuthology.orchestra.run.smithi043.stdout:mgr is available 2024-09-21T11:34:22.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:22 smithi043 ceph-mon[29710]: mgrmap e4: y(active, since 2s) 2024-09-21T11:34:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/991558119' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T11:34:28.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:28.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T11:34:28.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:28.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T11:34:28.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-09-21T11:34:28.423 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T11:34:28.424 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling cephadm module... 2024-09-21T11:34:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3049634717' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T11:34:32.235 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:31 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ignoring --setuser ceph since I am not root 2024-09-21T11:34:32.235 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:31 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ignoring --setgroup ceph since I am not root 2024-09-21T11:34:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:32 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3049634717' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T11:34:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:32 smithi043 ceph-mon[29710]: mgrmap e5: y(active, since 12s) 2024-09-21T11:34:37.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:36 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/893563528' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T11:34:37.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:34:37.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T11:34:37.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T11:34:37.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-21T11:34:37.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T11:34:37.518 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:34:37.518 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-09-21T11:34:37.518 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 5... 2024-09-21T11:34:38.060 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:37 smithi043 ceph-mon[29710]: Active manager daemon y restarted 2024-09-21T11:34:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:37 smithi043 ceph-mon[29710]: Activating manager daemon y 2024-09-21T11:34:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:37 smithi043 ceph-mon[29710]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T11:34:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:37 smithi043 ceph-mon[29710]: mgrmap e6: y(active, starting, since 0.00321358s) 2024-09-21T11:34:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:37 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:34:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:37 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:34:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:37 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:34:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:37 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:34:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:37 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:34:38.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:37 smithi043 ceph-mon[29710]: Manager daemon y is now available 2024-09-21T11:34:38.591 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:34:38.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T11:34:38.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T11:34:38.592 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:34:38.592 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 5 is available 2024-09-21T11:34:38.592 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying orchestrator module is enabled... 2024-09-21T11:34:39.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-mon[29710]: mgrmap e7: y(active, since 1.00493s) 2024-09-21T11:34:39.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:39.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:39.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-mon[29710]: Found migration_current of "None". Setting to last migration. 2024-09-21T11:34:39.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:39.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:34:39.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:34:39.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: 2024-09-21T11:34:39.486+0000 7f87fa4ef640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:11:34:39] ENGINE Error in HTTPServer.serve 2024-09-21T11:34:39.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: Traceback (most recent call last): 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self._connections.run(self.expiration_interval) 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self._run(expiration_interval) 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: s = self.context.wrap_socket( 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: return self.sslsocket_class._create( 2024-09-21T11:34:39.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:34:39.743 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self.do_handshake() 2024-09-21T11:34:39.743 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:34:39.743 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self._sslobj.do_handshake() 2024-09-21T11:34:39.743 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:34:39.743 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:39 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: 2024-09-21T11:34:40.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:40 smithi043 ceph-mon[29710]: mgrmap e8: y(active, since 2s) 2024-09-21T11:34:40.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3692351434' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T11:34:40.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:40 smithi043 ceph-mon[29710]: [21/Sep/2024:11:34:39] ENGINE Bus STARTING 2024-09-21T11:34:40.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:40 smithi043 ceph-mon[29710]: [21/Sep/2024:11:34:39] ENGINE Serving on https://172.21.15.43:7150 2024-09-21T11:34:40.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:40 smithi043 ceph-mon[29710]: [21/Sep/2024:11:34:39] ENGINE Error in HTTPServer.serve 2024-09-21T11:34:40.345 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-21T11:34:40.345 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:34:40.346 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-21T11:34:40.347 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:34:40.347 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-21T11:34:40.347 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:34:40.347 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-21T11:34:40.347 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:34:40.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:40 smithi043 ceph-mon[29710]: [21/Sep/2024:11:34:39] ENGINE Serving on http://172.21.15.43:8765 2024-09-21T11:34:40.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:40 smithi043 ceph-mon[29710]: [21/Sep/2024:11:34:39] ENGINE Bus STARTED 2024-09-21T11:34:40.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:40 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:34:40.639 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T11:34:40.640 INFO:teuthology.orchestra.run.smithi043.stdout:Setting orchestrator backend to cephadm... 2024-09-21T11:34:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:41 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3692351434' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T11:34:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:41 smithi043 ceph-mon[29710]: mgrmap e9: y(active, since 3s) 2024-09-21T11:34:42.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:42 smithi043 ceph-mon[29710]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:34:42.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:42 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:42.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:42 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:34:43.261 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T11:34:43.261 INFO:teuthology.orchestra.run.smithi043.stdout:Generating ssh key... 2024-09-21T11:34:43.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:43 smithi043 ceph-mon[29710]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:34:44.447 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: Generating public/private rsa key pair. 2024-09-21T11:34:44.447 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: Your identification has been saved in /tmp/tmp61nsejby/key 2024-09-21T11:34:44.447 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: Your public key has been saved in /tmp/tmp61nsejby/key.pub 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: The key fingerprint is: 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: SHA256:XbrjAbieG75k8DbTMyUCchdfj9i91NdKj7G3dpikZko ceph-4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: The key's randomart image is: 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: +---[RSA 3072]----+ 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: | . . | 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: | o + + . .| 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: | . o . o o = + o| 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: | o o . . + o B | 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: | . o S + . = o| 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: | o + + . o +.| 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: | X + E + oo.| 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: | * = = * . .| 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: | *o o | 2024-09-21T11:34:44.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:44 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: +----[SHA256]-----+ 2024-09-21T11:34:45.344 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:45 smithi043 ceph-mon[29710]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:34:45.344 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:45 smithi043 ceph-mon[29710]: Generating ssh key... 2024-09-21T11:34:45.344 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:45 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:45.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:45 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:45.948 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCMoEwrMmd7VhhXPYOF5IEbKNH+kprwRwttLWXXKhNRfJb1NN99ElTvcxfpDFQJysl23t0COSW90xJOiEMjNyUgxEEqt1XJhewOUOptX5tbAHbKa9waAIBrCy+ayjpCCQBLsdauWCnq/crX6Ign0H2sOGuoOUKP++c5UzpDwq9hclkcjREsXL86+BKroOXyYu2naByzepXu352lnFTM2pinWyblMfBaxgaXK+JdpmeV8y8H6oq5Kr2hd7jX1hW+ndN7mKiaGeo2Ni3dsotvTopRR9VenyEr+oso0AL4bY1774d+ROfmTju9ZtIRig5CqXXJ8bX1QbJLL/9hoiAQcgo3wkRezCcnkE3UIgESnXYImTmVqR0vfiydzyTP+lxJ9NiGZoN0jXDcYO4xQx6OfPwFSMhUArodHnqpdqJFM/+lyxpDycB3rG+84MaOZu3cmW0aPIoT7m+JIpw88a5EmBFALTe7DSynJQnA9dHaslp8IeSe6LQ8MrPiSICUDblOOH8= ceph-4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:34:45.949 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T11:34:45.949 INFO:teuthology.orchestra.run.smithi043.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T11:34:45.955 INFO:teuthology.orchestra.run.smithi043.stdout:Adding host smithi043... 2024-09-21T11:34:46.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:46 smithi043 ceph-mon[29710]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:34:47.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:47 smithi043 ceph-mon[29710]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:34:48.444 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:48 smithi043 ceph-mon[29710]: Deploying cephadm binary to smithi043 2024-09-21T11:34:50.998 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Added host 'smithi043' with addr '172.21.15.43' 2024-09-21T11:34:50.998 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mon service... 2024-09-21T11:34:51.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:51 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:51.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:51 smithi043 ceph-mon[29710]: Added host smithi043 2024-09-21T11:34:51.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:51 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:34:52.450 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T11:34:52.450 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mgr service... 2024-09-21T11:34:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:52 smithi043 ceph-mon[29710]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:34:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:52 smithi043 ceph-mon[29710]: Saving service mon spec with placement count:5 2024-09-21T11:34:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:52 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:54.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:54 smithi043 ceph-mon[29710]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:34:54.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:54 smithi043 ceph-mon[29710]: Saving service mgr spec with placement count:2 2024-09-21T11:34:54.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:54 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:54.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T11:34:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:55 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:55 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:55 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3943925115' entity='client.admin' 2024-09-21T11:34:56.944 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling the dashboard module... 2024-09-21T11:34:57.536 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1573970735' entity='client.admin' 2024-09-21T11:34:58.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:58 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ignoring --setuser ceph since I am not root 2024-09-21T11:34:58.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:34:58 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ignoring --setgroup ceph since I am not root 2024-09-21T11:34:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:58 smithi043 ceph-mon[29710]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:34:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:58 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/248913443' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T11:34:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:58 smithi043 ceph-mon[29710]: from='mgr.14118 172.21.15.43:0/1769963174' entity='mgr.y' 2024-09-21T11:34:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:59 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/248913443' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T11:34:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:34:59 smithi043 ceph-mon[29710]: mgrmap e10: y(active, since 21s) 2024-09-21T11:35:00.176 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:35:00.177 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-21T11:35:00.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T11:35:00.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-21T11:35:00.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T11:35:00.178 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:35:00.178 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-09-21T11:35:00.178 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 10... 2024-09-21T11:35:00.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2301237335' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T11:35:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:03 smithi043 ceph-mon[29710]: Active manager daemon y restarted 2024-09-21T11:35:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:03 smithi043 ceph-mon[29710]: Activating manager daemon y 2024-09-21T11:35:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:03 smithi043 ceph-mon[29710]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T11:35:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:03 smithi043 ceph-mon[29710]: mgrmap e11: y(active, starting, since 0.00342141s) 2024-09-21T11:35:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:03 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:35:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:03 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:35:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:03 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:35:03.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:03 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:35:03.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:03 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:35:03.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:03 smithi043 ceph-mon[29710]: Manager daemon y is now available 2024-09-21T11:35:04.963 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:04 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:04.963 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:04 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:35:04.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:04 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:35:04.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:04 smithi043 ceph-mon[29710]: mgrmap e12: y(active, since 1.00675s) 2024-09-21T11:35:05.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-mon[29710]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T11:35:05.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-mon[29710]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T11:35:05.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:05.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-09-21T11:35:05.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-21T11:35:05.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T11:35:05.878 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-09-21T11:35:05.878 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 10 is available 2024-09-21T11:35:05.879 INFO:teuthology.orchestra.run.smithi043.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T11:35:06.195 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: 2024-09-21T11:35:05.891+0000 7f8b1a648640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:11:35:05] ENGINE Error in HTTPServer.serve 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: Traceback (most recent call last): 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self._connections.run(self.expiration_interval) 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self._run(expiration_interval) 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: s = self.context.wrap_socket( 2024-09-21T11:35:06.196 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:35:06.197 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: return self.sslsocket_class._create( 2024-09-21T11:35:06.197 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:35:06.197 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self.do_handshake() 2024-09-21T11:35:06.197 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:35:06.197 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self._sslobj.do_handshake() 2024-09-21T11:35:06.197 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:35:06.197 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:35:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:06 smithi043 ceph-mon[29710]: [21/Sep/2024:11:35:05] ENGINE Bus STARTING 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:06 smithi043 ceph-mon[29710]: [21/Sep/2024:11:35:05] ENGINE Serving on http://172.21.15.43:8765 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:06 smithi043 ceph-mon[29710]: [21/Sep/2024:11:35:05] ENGINE Serving on https://172.21.15.43:7150 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:06 smithi043 ceph-mon[29710]: [21/Sep/2024:11:35:05] ENGINE Bus STARTED 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:06 smithi043 ceph-mon[29710]: [21/Sep/2024:11:35:05] ENGINE Error in HTTPServer.serve 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-21T11:35:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:35:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:06 smithi043 ceph-mon[29710]: mgrmap e13: y(active, since 2s) 2024-09-21T11:35:08.076 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:07 smithi043 ceph-mon[29710]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:08.076 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:07 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:09.241 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUMnQPNhpswGJAYa1sJKKLs2cCQZMwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTExMzUwOFoXDTM0\nMDkyMjExMzUwOFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjQzMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAo8p2ThClaKK/gdDwua227qQ27c2ryBK/o/6D\nsfV/uO1GnsvkRpvdefkc595TIgD1pSadCkPEeJ8nZQodC0Vj4IBKzhKAvD3evOCZ\nhdQKamn4BFSlPX2PONTU+LffectUYWS+mU4TptKrVL9B8gpt/UQ1RlHcDgh/jbVo\ncRTrVqiPlNrNdZJ+Cwm23Ebdh/dOeKq91gcYProPP5Sct9JWLPdh9B2FU2c+DYQD\nJS6aReuzem3+riwvOxYXCjH8iqJpWeSZG5NWuya0D6IUHbfUrT77Laf/DZXRyEZM\nycD1VrBVeTiRMlZnLF/UG4O+xK/WtAiaE61Mxb9bcMNaNuj3kAo46BUypeueIkj4\nQJSTTozJgoTTiV8vdx3z/MkC2MZwUqiE4u+dY50UswfRWpzFIYYhu5sGbXpesFyZ\n432iv/ec/nJVX3BdLlk7HEF071JlLF69O5x1BNMoQehfkzkpgxcWXlai/73W4MuL\n0vz+bG8p4nHAINcqok8so7pl7bmCXZEFp0c0ouMIHekE8IDT1WdtJ6T7yhYOVvxs\nqka0nVi4N33vhXFGbqG8kvRsE68hzq5tUe59omqIevekDU4B9v+nCeIuo0xfc3Fe\n6ZDmkWuQE0fsvbLOJ7nthdmJdOyPyKfFgobB7nuxKy1gQEsD4atfeF9qwhN3NJ6b\nbnLy7v0CAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDQzghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPKzAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQCn\nndT4ZtOlPIfkMPEuSu9K33D71ENhHyT80gVzypNR3KkYw77nJ+oq8s0f58/bKBTi\nj68d34wrAUEA3Oz7MBG3Sfqe2cPpye994Q3qCfdmI1/VGiNx9+66tQrNlkwU1mgH\nEHkIpk6KquWSR4Kxsu2/lQ45gSdeavA2GaBeicwysmXDmp4fhOnetcWeRv/9V0an\nDQP4NoDHsjyBt6lf4LoxQd5iD0YdS6P9FWWC9SXUbORwArUdqk/AuKw0qDTAXgA1\nB+uZA8SZEiUNxuNHNfa25Nj/xoU9XriOwZsYo+buJmCz+CoyX8Jj4ip8SbliDt5p\nXfvvr/UZMDw3YIrr/y8ZN9KEhmY27wcqC26xx36A8xzPyhWEEtAZ4Yxbaw8b1nj7\n7fihP0mMFDEM1JiMhLO9f9ZfUd3h9ZjpWAC/kI4xy+Iuz81TrPh1VManCx0wGGze\nl6O2rsaI9eA6a9WtswL+eNdmLSWyNOWeB2aBYiD0TyDMggRWyHmRIjv34/hwm2tZ\nJR2BtOL+vCRkiu5Xy7kK5xi29P+3yhNzL3sP4jejr1y+lPhRumMYj9Psz0xiHReV\ndJdYJZq1UOhVDso0p/Uev2tqNk60RP2JUYFWdv9912cDtRjmCsLvVFuOpL2lzhG0\n5soqo+diAmX8uBXjf2qUCLbUaKjwl2HTPTBlzogQZQ==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAo8p2ThClaKK/gdDwua227qQ27c2ryBK/o/6DsfV/uO1Gnsvk\nRpvdefkc595TIgD1pSadCkPEeJ8nZQodC0Vj4IBKzhKAvD3evOCZhdQKamn4BFSl\nPX2PONTU+LffectUYWS+mU4TptKrVL9B8gpt/UQ1RlHcDgh/jbVocRTrVqiPlNrN\ndZJ+Cwm23Ebdh/dOeKq91gcYProPP5Sct9JWLPdh9B2FU2c+DYQDJS6aReuzem3+\nriwvOxYXCjH8iqJpWeSZG5NWuya0D6IUHbfUrT77Laf/DZXRyEZMycD1VrBVeTiR\nMlZnLF/UG4O+xK/WtAiaE61Mxb9bcMNaNuj3kAo46BUypeueIkj4QJSTTozJgoTT\niV8vdx3z/MkC2MZwUqiE4u+dY50UswfRWpzFIYYhu5sGbXpesFyZ432iv/ec/nJV\nX3BdLlk7HEF071JlLF69O5x1BNMoQehfkzkpgxcWXlai/73W4MuL0vz+bG8p4nHA\nINcqok8so7pl7bmCXZEFp0c0ouMIHekE8IDT1WdtJ6T7yhYOVvxsqka0nVi4N33v\nhXFGbqG8kvRsE68hzq5tUe59omqIevekDU4B9v+nCeIuo0xfc3Fe6ZDmkWuQE0fs\nvbLOJ7nthdmJdOyPyKfFgobB7nuxKy1gQEsD4atfeF9qwhN3NJ6bbnLy7v0CAwEA\nAQKCAgABaFo86OIgbOujZQEXOggE14TGPnwMYLBvvuE0D1CvOIMSvaqEbLMVNuQW\n+gDCnaZMtbalbCpkCdD8witfZ5DyOn3xCzN4DXdUScPmQz56f2efqXgtE8XUmmlN\nf1b017Bt0Nj5fa7RtIDP8axdk7oWMk/sVTCzv4UwJ7TBhdzGCerKwrGhGZ5iUg2E\n94bmtrprpUyc/c3QOPmhB4xx0EmijCTPAZZViabFHx4+dDOyUx1ts+A7oj+kZ0zA\nUzvYqEBtSxj2ftkAyROHfbEAgEAs3j74F2zgqk4NSzGigmBKyYfXN1kTJObMw9x1\nqEsDBCJTngGcuAyBb2MEeiZc+SmO4VutP2TBU5rBe91YNGvtf7cyFJTxjSnISjzA\n7GJCUdzK567zdWoMjncB5E6ZzPQoSRXfzUNFQa295qSvV9Xrn4WjlPC4DVckqOMi\n9zOJryNsIZz8NiXtlGnQttrxdyGRiP6lROr+JzwXW3NqIMVAWlKpaMKQixYt6IPi\ngbS2VC6v7YdBhqGFtAGFvB/vNZRx8i/YocbvC0dBwiYXNepg0DxirZGI5DLBQGn5\nShIPVpIF+PWwLdiMzOTb3bcvWciPo0iTZTuZYFa59AGWpA6zT623YKeWR/CLD5Db\nGWjYyNg0X/NVtcR0MO2re8Ccqe5RUj9stkguHIKsClRKubBkeQKCAQEA5Rn6/0cJ\nk0+O47QI9tAKw8goPkzb/B8eaOZ2RJ4vypRu/xfdjNSk5ku7OUlcwzbFdgzVokXt\nSVDP+rJdvCyFBR6S2syUXljAf7hKupqSz7+gKwmlMXSUMfEwWWe6Dzp24mmVdb/F\nVKt+WYkmBiS8iJmWPjMPxqbbmyrsq9LQSl33bUHpdXVYhCNiK+210Z8WnydJmSBR\n6GOpzOIIYNSLP3WHmZhVUKcF9PBts4sNpqFxpAxtm6kWq2pu1ilZ7kQxCxNLzDG3\nywHMS4FD6WSzl5GmWkheLcbbkvRAKpubnQiRILnBa8lOn23f7ONFIOrYXFYW4KjU\ncgZc1N/KuSjbNQKCAQEAtwV23Op/c+G6EbKOP/UNKFUA4A4PPlzSSwAzvIwuIOBM\nUlYqhnm7CXMQamLtwPOxhC8dQKMs3ThgsGipmWxRlLprY04u2we0Xu1MSDON4R1A\nR84/NmgFmjuH9pE3tI7ENth8lCbdhoc4dkEY3mkMOpWd5dl04c4jLeSgJXQJ2oVT\nQ1WpvkeYIXC3NygpWglPzdERdwfSfhHB5QETI/OM9et+EYxFkR/p0i2kYEjtrUE1\nsoxI/WEvDejdGSvC75z0eKg+P92liDnV1BFB60sd4H2WV/PGnrR3KKgiT/4yk6/T\nnX0xhMfcdZRIlbZacQlVphCT3joGahASHA7cjf91qQKCAQACuD48jZFViFo/5cE9\nfNbD4DpqEuuBoLTxtdqtkdsrYnu214evxk521WtRkMdjIeFSN3DJzl60whrV9Qwb\n0cN6fhRsOIWCYTdXzM5GTNN1V+cE28Z0QdgIh751MUAKMj17DpnRngcvrCGB2pjy\n9bivPnOEJYMIbq6otue8WnJSj68CQwgMduooF2tw7g98f5D7LrI2+SU/xv8bCYvj\n6mIEzNZhQmew32q4r6n9uabd1YjN07/xWx6Dxm2uaStCItkaMo5xzB1McacFNnGh\n8aoYIIiDBdzoWjX9Fp00ZEiCorlmVA4xw2AFkH5aB5CohWEwJ216GUb8W4r6k/WK\nISwJAoIBAEPFljo/62ox0AD1qY8tYu3obCaLjkF9hbGYJZBQKFSsOhbTKXQaZ5Qc\nM5vxfjDRDI7yE1b4FYjlc4hfeU4CukLktRdUwC0DMVtla7XgwUz20/LDAM1Yv5pZ\nKQO4ynPl2eqlR5xkt+s070d624agR5L6DDosyAdFmelIm7TCv1NXw3lPo0d+o2re\nnjmieY2JnS+Nl3YIQ33DfZUbTMcnTK9TwJ8m3ivbMW5O0GuHFdvuJcA0CmMpNY9l\nB1pmxgyMhkqXolVr8fv7zp50tYA3lJyoZJ+0+XuTP/yiJfKRIpFlrmrHDSsqBwzT\nF/24RM1Os5SG1vqXagCkatBRAILkceECggEBAJxmaZ9kbMCCOM+pGKP6ohsAC0Gd\nVZ0bsu6f+vt24sH05X8jvUOP5+QgKiv60a57IIENgENS5tw3eGpsiaD0VeS/yVC9\nUlNHCt2gOHtKPUss24rfT29WBCyFfI5vo9x5beS/vOjo03rpdcwIrmv26WMy0Xau\nkQgyNaC9rZJRo0h5305zEOLqGCWromO24bNDD+KsWdbAV2mV2cadt3wxzoFIZYsU\nkvzSmCmpL14h5Tyd5Av2C4VA5zE7ux/CtwreROtsoVycBPn1hiZaTvefeGW0nq5Z\n1g4MH/WyrvgwCcxdukG4bm5EYX1cSg/5QE52HUs7N0vSJQBOnUcngbiHqmQ=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T11:35:10.465 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T11:35:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:10 smithi043 ceph-mon[29710]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:10 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:11.772 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T11:35:11.772 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial admin user... 2024-09-21T11:35:12.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:12 smithi043 ceph-mon[29710]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:12.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:12 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Z1hbM5att0HpalxnTOXW9u7KjTLkVW.TKMvyHieYagx2LKueOzQLu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726918512, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T11:35:13.237 INFO:teuthology.orchestra.run.smithi043.stdout:Fetching dashboard port number... 2024-09-21T11:35:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:13 smithi043 ceph-mon[29710]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:13 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T11:35:14.513 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T11:35:14.513 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-09-21T11:35:14.520 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T11:35:14.520 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-09-21T11:35:14.520 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-09-21T11:35:14.520 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T11:35:14.522 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph Dashboard is now available at: 2024-09-21T11:35:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: URL: https://smithi043.front.sepia.ceph.com:8443/ 2024-09-21T11:35:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: User: admin 2024-09-21T11:35:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: Password: 7lhvkoxbtb 2024-09-21T11:35:14.522 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:14.522 INFO:teuthology.orchestra.run.smithi043.stdout:Saving cluster configuration to /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config directory 2024-09-21T11:35:14.808 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2646486226' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T11:35:15.984 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T11:35:15.984 INFO:teuthology.orchestra.run.smithi043.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T11:35:15.984 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout: ceph telemetry on 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout:For more information see: 2024-09-21T11:35:15.985 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:15.986 INFO:teuthology.orchestra.run.smithi043.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T11:35:15.986 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:15.986 INFO:teuthology.orchestra.run.smithi043.stdout:Bootstrap complete. 2024-09-21T11:35:16.029 INFO:tasks.cephadm:Fetching config... 2024-09-21T11:35:16.029 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:35:16.029 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T11:35:16.047 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T11:35:16.047 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:35:16.047 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T11:35:16.104 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T11:35:16.104 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:35:16.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T11:35:16.170 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T11:35:16.170 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:35:16.170 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T11:35:16.226 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T11:35:16.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCMoEwrMmd7VhhXPYOF5IEbKNH+kprwRwttLWXXKhNRfJb1NN99ElTvcxfpDFQJysl23t0COSW90xJOiEMjNyUgxEEqt1XJhewOUOptX5tbAHbKa9waAIBrCy+ayjpCCQBLsdauWCnq/crX6Ign0H2sOGuoOUKP++c5UzpDwq9hclkcjREsXL86+BKroOXyYu2naByzepXu352lnFTM2pinWyblMfBaxgaXK+JdpmeV8y8H6oq5Kr2hd7jX1hW+ndN7mKiaGeo2Ni3dsotvTopRR9VenyEr+oso0AL4bY1774d+ROfmTju9ZtIRig5CqXXJ8bX1QbJLL/9hoiAQcgo3wkRezCcnkE3UIgESnXYImTmVqR0vfiydzyTP+lxJ9NiGZoN0jXDcYO4xQx6OfPwFSMhUArodHnqpdqJFM/+lyxpDycB3rG+84MaOZu3cmW0aPIoT7m+JIpw88a5EmBFALTe7DSynJQnA9dHaslp8IeSe6LQ8MrPiSICUDblOOH8= ceph-4f2c5252-780d-11ef-baf6-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T11:35:16.302 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCMoEwrMmd7VhhXPYOF5IEbKNH+kprwRwttLWXXKhNRfJb1NN99ElTvcxfpDFQJysl23t0COSW90xJOiEMjNyUgxEEqt1XJhewOUOptX5tbAHbKa9waAIBrCy+ayjpCCQBLsdauWCnq/crX6Ign0H2sOGuoOUKP++c5UzpDwq9hclkcjREsXL86+BKroOXyYu2naByzepXu352lnFTM2pinWyblMfBaxgaXK+JdpmeV8y8H6oq5Kr2hd7jX1hW+ndN7mKiaGeo2Ni3dsotvTopRR9VenyEr+oso0AL4bY1774d+ROfmTju9ZtIRig5CqXXJ8bX1QbJLL/9hoiAQcgo3wkRezCcnkE3UIgESnXYImTmVqR0vfiydzyTP+lxJ9NiGZoN0jXDcYO4xQx6OfPwFSMhUArodHnqpdqJFM/+lyxpDycB3rG+84MaOZu3cmW0aPIoT7m+JIpw88a5EmBFALTe7DSynJQnA9dHaslp8IeSe6LQ8MrPiSICUDblOOH8= ceph-4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:16.324 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCMoEwrMmd7VhhXPYOF5IEbKNH+kprwRwttLWXXKhNRfJb1NN99ElTvcxfpDFQJysl23t0COSW90xJOiEMjNyUgxEEqt1XJhewOUOptX5tbAHbKa9waAIBrCy+ayjpCCQBLsdauWCnq/crX6Ign0H2sOGuoOUKP++c5UzpDwq9hclkcjREsXL86+BKroOXyYu2naByzepXu352lnFTM2pinWyblMfBaxgaXK+JdpmeV8y8H6oq5Kr2hd7jX1hW+ndN7mKiaGeo2Ni3dsotvTopRR9VenyEr+oso0AL4bY1774d+ROfmTju9ZtIRig5CqXXJ8bX1QbJLL/9hoiAQcgo3wkRezCcnkE3UIgESnXYImTmVqR0vfiydzyTP+lxJ9NiGZoN0jXDcYO4xQx6OfPwFSMhUArodHnqpdqJFM/+lyxpDycB3rG+84MaOZu3cmW0aPIoT7m+JIpw88a5EmBFALTe7DSynJQnA9dHaslp8IeSe6LQ8MrPiSICUDblOOH8= ceph-4f2c5252-780d-11ef-baf6-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T11:35:16.391 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCMoEwrMmd7VhhXPYOF5IEbKNH+kprwRwttLWXXKhNRfJb1NN99ElTvcxfpDFQJysl23t0COSW90xJOiEMjNyUgxEEqt1XJhewOUOptX5tbAHbKa9waAIBrCy+ayjpCCQBLsdauWCnq/crX6Ign0H2sOGuoOUKP++c5UzpDwq9hclkcjREsXL86+BKroOXyYu2naByzepXu352lnFTM2pinWyblMfBaxgaXK+JdpmeV8y8H6oq5Kr2hd7jX1hW+ndN7mKiaGeo2Ni3dsotvTopRR9VenyEr+oso0AL4bY1774d+ROfmTju9ZtIRig5CqXXJ8bX1QbJLL/9hoiAQcgo3wkRezCcnkE3UIgESnXYImTmVqR0vfiydzyTP+lxJ9NiGZoN0jXDcYO4xQx6OfPwFSMhUArodHnqpdqJFM/+lyxpDycB3rG+84MaOZu3cmW0aPIoT7m+JIpw88a5EmBFALTe7DSynJQnA9dHaslp8IeSe6LQ8MrPiSICUDblOOH8= ceph-4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:16.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T11:35:16.672 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:35:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:16 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2778599958' entity='client.admin' 2024-09-21T11:35:17.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:17 smithi043 ceph-mon[29710]: mgrmap e14: y(active, since 12s) 2024-09-21T11:35:18.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:18 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4041540045' entity='client.admin' 2024-09-21T11:35:18.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:18 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:18.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:18 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:18.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:18 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"} : dispatch 2024-09-21T11:35:18.672 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:18 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:18.672 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:18 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:18.672 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:18 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:18.704 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T11:35:18.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T11:35:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:35:20.325 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi175 2024-09-21T11:35:20.325 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:35:20.325 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.conf 2024-09-21T11:35:20.344 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:35:20.344 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:35:20.403 INFO:tasks.cephadm:Adding host smithi175 to orchestrator... 2024-09-21T11:35:20.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch host add smithi175 2024-09-21T11:35:20.651 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:35:20.850 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:20.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:21 smithi043 ceph-mon[29710]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:22 smithi043 ceph-mon[29710]: Deploying cephadm binary to smithi175 2024-09-21T11:35:24.965 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi175' with addr '172.21.15.175' 2024-09-21T11:35:24.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:24 smithi043 ceph-mon[29710]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:25.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T11:35:25.849 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:35:26.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:26.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:25 smithi043 ceph-mon[29710]: Added host smithi175 2024-09-21T11:35:26.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:26.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:26.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:26.532 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:26.532 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}] 2024-09-21T11:35:27.164 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T11:35:27.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd crush tunables default 2024-09-21T11:35:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:35:27.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:27 smithi043 ceph-mon[29710]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:27.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:28.279 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-09-21T11:35:28.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:28 smithi043 ceph-mon[29710]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:35:28.578 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:28 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/867363052' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T11:35:28.900 INFO:tasks.cephadm:Adding mon.a on smithi043 2024-09-21T11:35:28.900 INFO:tasks.cephadm:Adding mon.c on smithi043 2024-09-21T11:35:28.900 INFO:tasks.cephadm:Adding mon.b on smithi175 2024-09-21T11:35:28.900 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch apply mon '3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi175:172.21.15.175=b' 2024-09-21T11:35:29.149 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T11:35:29.195 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T11:35:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:29 smithi043 ceph-mon[29710]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/867363052' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T11:35:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:29 smithi043 ceph-mon[29710]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:35:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:29 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:29 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:29 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:29 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:29 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"} : dispatch 2024-09-21T11:35:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:29 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:29 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:29.889 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mon update... 2024-09-21T11:35:30.437 DEBUG:teuthology.orchestra.run.smithi043:mon.c> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.c.service 2024-09-21T11:35:30.440 DEBUG:teuthology.orchestra.run.smithi175:mon.b> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.b.service 2024-09-21T11:35:30.444 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T11:35:30.445 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph mon dump -f json 2024-09-21T11:35:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-21T11:35:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:35:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:35:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:31 smithi043 ceph-mon[29710]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:35:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:31 smithi043 ceph-mon[29710]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:31 smithi043 ceph-mon[29710]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi175:172.21.15.175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:31 smithi043 ceph-mon[29710]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi175:172.21.15.175=b;count:3 2024-09-21T11:35:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:31 smithi043 ceph-mon[29710]: Deploying daemon mon.b on smithi175 2024-09-21T11:35:32.363 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:35:33.186 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:35:33.186 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"4f2c5252-780d-11ef-baf6-efdc52797490","modified":"2024-09-21T11:34:04.177212Z","created":"2024-09-21T11:34:04.177212Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T11:35:33.186 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2024-09-21T11:35:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:33 smithi043 ceph-mon[29710]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:33 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/407571025' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:35:34.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:34 smithi175 systemd[1]: Started Ceph mon.b for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:35:34.814 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T11:35:34.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph mon dump -f json 2024-09-21T11:35:35.073 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:35:39.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:39.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:35:39.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: mon.a calling monitor election 2024-09-21T11:35:39.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:39.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: mon.b calling monitor election 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: monmap epoch 2 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: last_changed 2024-09-21T11:35:34.248610+0000 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: created 2024-09-21T11:34:04.177212+0000 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: min_mon_release 19 (squid) 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: election_strategy: 1 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-09-21T11:35:39.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: fsmap 2024-09-21T11:35:39.685 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:35:39.685 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: mgrmap e14: y(active, since 35s) 2024-09-21T11:35:39.685 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: overall HEALTH_OK 2024-09-21T11:35:39.685 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:39.685 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:39.685 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:39.685 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:39.685 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:35:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: mon.a calling monitor election 2024-09-21T11:35:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: mon.b calling monitor election 2024-09-21T11:35:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: monmap epoch 2 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: last_changed 2024-09-21T11:35:34.248610+0000 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: created 2024-09-21T11:34:04.177212+0000 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: min_mon_release 19 (squid) 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: election_strategy: 1 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: fsmap 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: mgrmap e14: y(active, since 35s) 2024-09-21T11:35:39.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: overall HEALTH_OK 2024-09-21T11:35:39.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:39.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:39.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:39.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:39.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:40.130 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:35:40.130 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":2,"fsid":"4f2c5252-780d-11ef-baf6-efdc52797490","modified":"2024-09-21T11:35:34.248610Z","created":"2024-09-21T11:34:04.177212Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T11:35:40.131 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 2 2024-09-21T11:35:40.465 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:40 smithi175 ceph-mon[32613]: Deploying daemon mon.c on smithi043 2024-09-21T11:35:40.465 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:40 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/1198174822' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:35:40.465 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:40 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:40.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:40 smithi043 ceph-mon[29710]: Deploying daemon mon.c on smithi043 2024-09-21T11:35:40.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/1198174822' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:35:40.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:40 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:41.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:41 smithi043 ceph-mon[29710]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:41.726 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T11:35:41.727 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph mon dump -f json 2024-09-21T11:35:41.754 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:41 smithi175 ceph-mon[32613]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:41.976 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:35:42.385 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 systemd[1]: Starting Ceph mon.c for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:35:42.650 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 podman[36435]: 2024-09-21 11:35:42.383508887 +0000 UTC m=+0.111227920 container create 56e77e4ebe6d8eb3b74d4cba1581d99ed0f2df68af9875b27161f0511b460730 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-c, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T11:35:42.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 podman[36435]: 2024-09-21 11:35:42.288814565 +0000 UTC m=+0.016533598 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T11:35:42.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 podman[36435]: 2024-09-21 11:35:42.59476038 +0000 UTC m=+0.322479413 container init 56e77e4ebe6d8eb3b74d4cba1581d99ed0f2df68af9875b27161f0511b460730 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T11:35:42.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 podman[36435]: 2024-09-21 11:35:42.598125696 +0000 UTC m=+0.325844730 container start 56e77e4ebe6d8eb3b74d4cba1581d99ed0f2df68af9875b27161f0511b460730 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T11:35:42.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T11:35:42.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev), process ceph-mon, pid 2 2024-09-21T11:35:42.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: pidfile_write: ignore empty --pid-file 2024-09-21T11:35:42.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: load: jerasure load: lrc 2024-09-21T11:35:42.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: RocksDB version: 7.9.2 2024-09-21T11:35:42.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Git sha 0 2024-09-21T11:35:42.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Compile date 2024-09-21 08:36:07 2024-09-21T11:35:42.652 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: DB SUMMARY 2024-09-21T11:35:42.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: DB Session ID: CLW0ES37QQ4Y7BMO8PGR 2024-09-21T11:35:42.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: CURRENT file: CURRENT 2024-09-21T11:35:42.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T11:35:42.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T11:35:42.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T11:35:42.653 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-21T11:35:42.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.error_if_exists: 0 2024-09-21T11:35:42.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.create_if_missing: 0 2024-09-21T11:35:42.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.paranoid_checks: 1 2024-09-21T11:35:42.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T11:35:42.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T11:35:42.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T11:35:42.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.env: 0x55eee8dba120 2024-09-21T11:35:42.654 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.info_log: 0x55eeeaed3700 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.statistics: (nil) 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.use_fsync: 0 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_log_file_size: 0 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.allow_fallocate: 1 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T11:35:42.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.use_direct_reads: 0 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.db_log_dir: 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.wal_dir: 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T11:35:42.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.write_buffer_manager: 0x55eeeaed7b80 2024-09-21T11:35:42.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T11:35:42.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T11:35:42.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T11:35:42.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.unordered_write: 0 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.row_cache: None 2024-09-21T11:35:42.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.wal_filter: None 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.two_write_queues: 0 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.wal_compression: 0 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.atomic_flush: 0 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.log_readahead_size: 0 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T11:35:42.660 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_background_jobs: 2 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_background_compactions: -1 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_subcompactions: 1 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T11:35:42.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_open_files: -1 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_background_flushes: -1 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Compression algorithms supported: 2024-09-21T11:35:42.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: kZSTD supported: 0 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: kXpressCompression supported: 0 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: kBZip2Compression supported: 0 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: kLZ4Compression supported: 1 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: kZlibCompression supported: 1 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: kSnappyCompression supported: 1 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T11:35:42.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.merge_operator: 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_filter: None 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55eeeaed3320) 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-09-21T11:35:42.664 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_type: 0 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_index_type: 0 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_shortening: 1 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: checksum: 4 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: no_block_cache: 0 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache: 0x55eeeaec7610 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_options: 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: capacity : 536870912 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_shard_bits : 4 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: strict_capacity_limit : 0 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-09-21T11:35:42.665 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_compressed: (nil) 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: persistent_cache: (nil) 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size: 4096 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size_deviation: 10 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_restart_interval: 16 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_block_restart_interval: 1 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: metadata_block_size: 4096 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: partition_filters: 0 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: use_delta_encoding: 1 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: filter_policy: bloomfilter 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: whole_key_filtering: 1 2024-09-21T11:35:42.666 INFO:journalctl@ceph.mon.c.smithi043.stdout: verify_compression: 0 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout: format_version: 5 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_index_compression: 1 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_align: 0 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout: max_auto_readahead_size: 262144 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout: prepopulate_block_cache: 0 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout: initial_auto_readahead_size: 8192 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compression: NoCompression 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T11:35:42.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.num_levels: 7 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T11:35:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T11:35:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T11:35:42.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T11:35:42.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.inplace_update_support: 0 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T11:35:42.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.bloom_locality: 0 2024-09-21T11:35:42.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.max_successive_merges: 0 2024-09-21T11:35:42.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T11:35:42.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T11:35:42.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T11:35:42.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T11:35:42.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.ttl: 2592000 2024-09-21T11:35:42.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T11:35:42.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T11:35:42.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T11:35:42.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.enable_blob_files: false 2024-09-21T11:35:42.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.min_blob_size: 0 2024-09-21T11:35:42.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T11:35:42.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T11:35:42.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T11:35:42.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T11:35:42.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T11:35:42.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T11:35:42.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T11:35:42.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T11:35:42.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 183425f6-0034-441d-8080-22acb99b710b 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726918542626329, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726918542626849, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726918542, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "183425f6-0034-441d-8080-22acb99b710b", "db_session_id": "CLW0ES37QQ4Y7BMO8PGR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726918542626988, "job": 1, "event": "recovery_finished"} 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55eeeaef6e00 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: DB pointer 0x55eeeaff8000 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T11:35:42.675 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** DB Stats ** 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T11:35:42.676 INFO:journalctl@ceph.mon.c.smithi043.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T11:35:42.677 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout: Block cache BinnedLRUCache@0x55eeeaec7610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T11:35:42.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: using public_addrv [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] 2024-09-21T11:35:42.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: starting mon.c rank -1 at public addrs [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] at bind addrs [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:42.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(???) e0 preinit fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:42.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T11:35:42.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T11:35:42.679 INFO:journalctl@ceph.mon.c.smithi043.stdout: e1 2024-09-21T11:35:42.679 INFO:journalctl@ceph.mon.c.smithi043.stdout: btime 2024-09-21T11:34:07:249039+0000 2024-09-21T11:35:42.679 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T11:35:42.679 INFO:journalctl@ceph.mon.c.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T11:35:42.680 INFO:journalctl@ceph.mon.c.smithi043.stdout: legacy client fscid: -1 2024-09-21T11:35:42.680 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-09-21T11:35:42.680 INFO:journalctl@ceph.mon.c.smithi043.stdout: No filesystems configured 2024-09-21T11:35:42.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T11:35:42.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T11:35:42.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T11:35:42.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T11:35:42.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T11:35:42.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T11:35:42.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T11:35:42.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:35:42.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:35:42.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T11:35:42.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:42.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:35:42.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/867363052' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T11:35:42.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:42.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/867363052' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T11:35:42.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:35:42.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"} : dispatch 2024-09-21T11:35:42.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:42.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:42.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-21T11:35:42.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:35:42.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:42.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:35:42.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:42.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi175:172.21.15.175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:42.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi175:172.21.15.175=b;count:3 2024-09-21T11:35:42.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: Deploying daemon mon.b on smithi175 2024-09-21T11:35:42.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:42.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/407571025' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:35:42.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:42.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:35:42.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.a calling monitor election 2024-09-21T11:35:42.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:42.685 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:42.685 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:42.685 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:42.685 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.b calling monitor election 2024-09-21T11:35:42.685 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:42.685 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:42.685 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:42.685 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:42.685 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-21T11:35:42.686 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: monmap epoch 2 2024-09-21T11:35:42.686 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:42.686 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: last_changed 2024-09-21T11:35:34.248610+0000 2024-09-21T11:35:42.686 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: created 2024-09-21T11:34:04.177212+0000 2024-09-21T11:35:42.686 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: min_mon_release 19 (squid) 2024-09-21T11:35:42.686 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: election_strategy: 1 2024-09-21T11:35:42.686 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-21T11:35:42.686 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-09-21T11:35:42.686 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: fsmap 2024-09-21T11:35:42.686 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:35:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mgrmap e14: y(active, since 35s) 2024-09-21T11:35:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: overall HEALTH_OK 2024-09-21T11:35:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: Deploying daemon mon.c on smithi043 2024-09-21T11:35:42.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/1198174822' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:35:42.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:42.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:42.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 ceph-mon[36454]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T11:35:42.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 bash[36435]: 56e77e4ebe6d8eb3b74d4cba1581d99ed0f2df68af9875b27161f0511b460730 2024-09-21T11:35:42.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:42 smithi043 systemd[1]: Started Ceph mon.c for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:35:47.668 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:35:47.669 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":3,"fsid":"4f2c5252-780d-11ef-baf6-efdc52797490","modified":"2024-09-21T11:35:42.654039Z","created":"2024-09-21T11:34:04.177212Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:3300","nonce":0},{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T11:35:47.669 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 3 2024-09-21T11:35:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:35:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: mon.a calling monitor election 2024-09-21T11:35:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: mon.b calling monitor election 2024-09-21T11:35:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/1303323353' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:35:47.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: mon.c calling monitor election 2024-09-21T11:35:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-21T11:35:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: monmap epoch 3 2024-09-21T11:35:47.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:47.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: last_changed 2024-09-21T11:35:42.654039+0000 2024-09-21T11:35:47.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: created 2024-09-21T11:34:04.177212+0000 2024-09-21T11:35:47.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: min_mon_release 19 (squid) 2024-09-21T11:35:47.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: election_strategy: 1 2024-09-21T11:35:47.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-21T11:35:47.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-09-21T11:35:47.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: 2: [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon.c 2024-09-21T11:35:47.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: fsmap 2024-09-21T11:35:47.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:35:47.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: mgrmap e14: y(active, since 44s) 2024-09-21T11:35:47.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: overall HEALTH_OK 2024-09-21T11:35:47.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:47.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:47.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:47.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: mon.a calling monitor election 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: mon.b calling monitor election 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/1303323353' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: mon.c calling monitor election 2024-09-21T11:35:47.996 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-21T11:35:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: monmap epoch 3 2024-09-21T11:35:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: last_changed 2024-09-21T11:35:42.654039+0000 2024-09-21T11:35:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: created 2024-09-21T11:34:04.177212+0000 2024-09-21T11:35:47.997 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: min_mon_release 19 (squid) 2024-09-21T11:35:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: election_strategy: 1 2024-09-21T11:35:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-21T11:35:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-09-21T11:35:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: 2: [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon.c 2024-09-21T11:35:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: fsmap 2024-09-21T11:35:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:35:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: mgrmap e14: y(active, since 44s) 2024-09-21T11:35:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: overall HEALTH_OK 2024-09-21T11:35:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:47.998 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:47.999 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:47.999 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:47.999 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:35:47.999 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: mon.a calling monitor election 2024-09-21T11:35:47.999 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:35:47.999 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:47.999 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: mon.b calling monitor election 2024-09-21T11:35:47.999 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/1303323353' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: mon.c calling monitor election 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: monmap epoch 3 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: last_changed 2024-09-21T11:35:42.654039+0000 2024-09-21T11:35:48.000 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: created 2024-09-21T11:34:04.177212+0000 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: min_mon_release 19 (squid) 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: election_strategy: 1 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: 0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: 1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: 2: [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon.c 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: fsmap 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: mgrmap e14: y(active, since 44s) 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: overall HEALTH_OK 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:48.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:48.002 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:48.316 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T11:35:48.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T11:35:48.812 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:35:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:48 smithi043 ceph-mon[29710]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:48 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:48 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:48.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:48 smithi043 ceph-mon[36454]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:48.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:48 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:48.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:48 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:48 smithi175 ceph-mon[32613]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:48 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:48 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:35:49.519 INFO:teuthology.orchestra.run.smithi043.stdout:# minimal ceph.conf for 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:49.519 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-09-21T11:35:49.519 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:35:49.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon_host = [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] 2024-09-21T11:35:50.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:50.167 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:35:50.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3740410790' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:35:50.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:50.170 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3740410790' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:50.194 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T11:35:50.195 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:35:50.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T11:35:50.222 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:35:50.222 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:35:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:50.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:50.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:35:50.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:50.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3740410790' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:50.288 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:35:50.289 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T11:35:50.315 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:35:50.316 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:35:50.381 INFO:tasks.cephadm:Adding mgr.y on smithi043 2024-09-21T11:35:50.381 INFO:tasks.cephadm:Adding mgr.x on smithi175 2024-09-21T11:35:50.381 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch apply mgr '2;smithi043=y;smithi175=x' 2024-09-21T11:35:50.657 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:35:51.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:50 smithi175 ceph-mon[32613]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T11:35:51.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:50 smithi175 ceph-mon[32613]: Reconfiguring daemon mon.a on smithi043 2024-09-21T11:35:51.082 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:50 smithi175 ceph-mon[32613]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:51.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:50 smithi043 ceph-mon[36454]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T11:35:51.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:50 smithi043 ceph-mon[36454]: Reconfiguring daemon mon.a on smithi043 2024-09-21T11:35:51.117 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:50 smithi043 ceph-mon[36454]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:51.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:50 smithi043 ceph-mon[29710]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T11:35:51.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:50 smithi043 ceph-mon[29710]: Reconfiguring daemon mon.a on smithi043 2024-09-21T11:35:51.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:50 smithi043 ceph-mon[29710]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:51.283 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mgr update... 2024-09-21T11:35:51.826 DEBUG:teuthology.orchestra.run.smithi175:mgr.x> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.x.service 2024-09-21T11:35:51.829 DEBUG:tasks.cephadm:set 0 configs 2024-09-21T11:35:51.829 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T11:35:51.829 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:35:51.829 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T11:35:51.846 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T11:35:51.846 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-09-21T11:35:51.902 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T11:35:51.902 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:35:51.902 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 945 Links: 1 2024-09-21T11:35:51.902 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:35:51.902 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:35:51.902 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-21 11:35:42.111509511 +0000 2024-09-21T11:35:51.902 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-21 11:31:06.719861311 +0000 2024-09-21T11:35:51.902 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-21 11:31:06.719861311 +0000 2024-09-21T11:35:51.902 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-21 11:31:06.719861311 +0000 2024-09-21T11:35:51.903 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T11:35:51.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-21T11:35:51.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-21T11:35:51.967 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 9.2815e-05 s, 5.5 MB/s 2024-09-21T11:35:51.968 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T11:35:52.026 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-09-21T11:35:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T11:35:52.083 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:35:52.083 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 956 Links: 1 2024-09-21T11:35:52.083 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:35:52.083 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:35:52.083 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-21 11:35:42.111509511 +0000 2024-09-21T11:35:52.083 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-21 11:31:06.720861245 +0000 2024-09-21T11:35:52.084 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-21 11:31:06.720861245 +0000 2024-09-21T11:35:52.084 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-21 11:31:06.720861245 +0000 2024-09-21T11:35:52.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T11:35:52.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-21T11:35:52.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-21T11:35:52.147 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000105414 s, 4.9 MB/s 2024-09-21T11:35:52.148 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T11:35:52.206 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-09-21T11:35:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[29710]: Reconfiguring mon.c (monmap changed)... 2024-09-21T11:35:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:35:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[29710]: Reconfiguring daemon mon.c on smithi043 2024-09-21T11:35:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[29710]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi175=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[29710]: Saving service mgr spec with placement smithi043=y;smithi175=x;count:2 2024-09-21T11:35:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[36454]: Reconfiguring mon.c (monmap changed)... 2024-09-21T11:35:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:35:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[36454]: Reconfiguring daemon mon.c on smithi043 2024-09-21T11:35:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[36454]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi175=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[36454]: Saving service mgr spec with placement smithi043=y;smithi175=x;count:2 2024-09-21T11:35:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:52.257 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T11:35:52.258 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:35:52.258 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-21T11:35:52.258 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:35:52.258 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:35:52.258 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-21 11:35:42.111509511 +0000 2024-09-21T11:35:52.258 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-21 11:31:06.721861179 +0000 2024-09-21T11:35:52.258 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-21 11:31:06.721861179 +0000 2024-09-21T11:35:52.258 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-21 11:31:06.721861179 +0000 2024-09-21T11:35:52.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T11:35:52.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-21T11:35:52.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-21T11:35:52.323 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000104634 s, 4.9 MB/s 2024-09-21T11:35:52.324 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T11:35:52.381 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-09-21T11:35:52.437 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T11:35:52.437 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:35:52.437 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 943 Links: 1 2024-09-21T11:35:52.437 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:35:52.437 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:35:52.437 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-21 11:35:42.112509499 +0000 2024-09-21T11:35:52.437 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-21 11:31:06.719861311 +0000 2024-09-21T11:35:52.437 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-21 11:31:06.719861311 +0000 2024-09-21T11:35:52.437 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-21 11:31:06.719861311 +0000 2024-09-21T11:35:52.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T11:35:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-21T11:35:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-21T11:35:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000109426 s, 4.7 MB/s 2024-09-21T11:35:52.504 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T11:35:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:52 smithi175 ceph-mon[32613]: Reconfiguring mon.c (monmap changed)... 2024-09-21T11:35:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:35:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:52 smithi175 ceph-mon[32613]: Reconfiguring daemon mon.c on smithi043 2024-09-21T11:35:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:52 smithi175 ceph-mon[32613]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi175=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:52 smithi175 ceph-mon[32613]: Saving service mgr spec with placement smithi043=y;smithi175=x;count:2 2024-09-21T11:35:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:52.561 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:35:52.561 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T11:35:52.577 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T11:35:52.577 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-09-21T11:35:52.632 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T11:35:52.632 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:35:52.632 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-21T11:35:52.632 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:35:52.632 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:35:52.633 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 11:35:33.265787390 +0000 2024-09-21T11:35:52.633 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 11:31:06.397863650 +0000 2024-09-21T11:35:52.633 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 11:31:06.397863650 +0000 2024-09-21T11:35:52.633 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-21 11:31:06.397863650 +0000 2024-09-21T11:35:52.633 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T11:35:52.697 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T11:35:52.697 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T11:35:52.697 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000235387 s, 2.2 MB/s 2024-09-21T11:35:52.698 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T11:35:52.755 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-09-21T11:35:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T11:35:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:35:52.812 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-21T11:35:52.812 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:35:52.812 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:35:52.812 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 11:35:33.266787375 +0000 2024-09-21T11:35:52.812 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 11:31:06.397863650 +0000 2024-09-21T11:35:52.812 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 11:31:06.397863650 +0000 2024-09-21T11:35:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-21 11:31:06.397863650 +0000 2024-09-21T11:35:52.812 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T11:35:52.874 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T11:35:52.875 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T11:35:52.875 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000173224 s, 3.0 MB/s 2024-09-21T11:35:52.876 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T11:35:52.933 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-09-21T11:35:52.988 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T11:35:52.988 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:35:52.988 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 965 Links: 1 2024-09-21T11:35:52.989 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:35:52.989 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:35:52.989 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 11:35:33.266787375 +0000 2024-09-21T11:35:52.989 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 11:31:06.396863665 +0000 2024-09-21T11:35:52.989 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 11:31:06.396863665 +0000 2024-09-21T11:35:52.989 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-21 11:31:06.396863665 +0000 2024-09-21T11:35:52.989 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T11:35:53.053 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T11:35:53.053 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T11:35:53.053 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000177624 s, 2.9 MB/s 2024-09-21T11:35:53.054 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T11:35:53.111 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-09-21T11:35:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T11:35:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T11:35:53.168 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-21T11:35:53.168 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T11:35:53.168 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T11:35:53.168 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 11:35:33.266787375 +0000 2024-09-21T11:35:53.168 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 11:31:06.396863665 +0000 2024-09-21T11:35:53.168 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 11:31:06.396863665 +0000 2024-09-21T11:35:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-21 11:31:06.396863665 +0000 2024-09-21T11:35:53.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T11:35:53.231 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T11:35:53.231 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T11:35:53.231 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000192124 s, 2.7 MB/s 2024-09-21T11:35:53.232 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T11:35:53.288 INFO:tasks.cephadm:Deploying osd.0 on smithi043 with /dev/vg_nvme/lv_4... 2024-09-21T11:35:53.289 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T11:35:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:35:53.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[36454]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:53.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:53.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:53.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:53.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:35:53.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:53.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[29710]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:53.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:53.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:53.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:53.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:35:53.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:53 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:53 smithi175 ceph-mon[32613]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:53 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:53 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:53 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T11:35:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:53 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T11:35:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:53 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: Reconfiguring mon.b (monmap changed)... 2024-09-21T11:35:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: Reconfiguring daemon mon.b on smithi175 2024-09-21T11:35:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:35:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T11:35:54.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:35:54.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:54.536 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: Reconfiguring mon.b (monmap changed)... 2024-09-21T11:35:54.536 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: Reconfiguring daemon mon.b on smithi175 2024-09-21T11:35:54.536 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: Reconfiguring mon.b (monmap changed)... 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: Reconfiguring daemon mon.b on smithi175 2024-09-21T11:35:54.537 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:54.538 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:54.538 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:54.538 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:54.538 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:54.538 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:54.538 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:35:54.538 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T11:35:54.538 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:35:54.538 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:55.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:55 smithi175 ceph-mon[32613]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:55.422 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:55 smithi175 ceph-mon[32613]: Deploying daemon mgr.x on smithi175 2024-09-21T11:35:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:55 smithi043 ceph-mon[29710]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:55 smithi043 ceph-mon[29710]: Deploying daemon mgr.x on smithi175 2024-09-21T11:35:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:55 smithi043 ceph-mon[36454]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:55 smithi043 ceph-mon[36454]: Deploying daemon mgr.x on smithi175 2024-09-21T11:35:55.761 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:35:55 smithi175 systemd[1]: Starting Ceph mgr.x for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:35:56.122 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:35:56.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-09-21T11:35:56.261 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:35:55 smithi175 podman[33985]: 2024-09-21 11:35:55.804082461 +0000 UTC m=+0.101107483 container create fec51c1f8ac423528b1d77933b2baf80ef741bb7b97bd044f117a4070c8c0596 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T11:35:56.261 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:35:55 smithi175 podman[33985]: 2024-09-21 11:35:55.719450496 +0000 UTC m=+0.016475493 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T11:35:56.261 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:35:55 smithi175 podman[33985]: 2024-09-21 11:35:55.981280881 +0000 UTC m=+0.278305870 container init fec51c1f8ac423528b1d77933b2baf80ef741bb7b97bd044f117a4070c8c0596 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T11:35:56.261 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:35:55 smithi175 podman[33985]: 2024-09-21 11:35:55.983954379 +0000 UTC m=+0.280979359 container start fec51c1f8ac423528b1d77933b2baf80ef741bb7b97bd044f117a4070c8c0596 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T11:35:56.261 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:35:56 smithi175 ceph-mgr[34003]: -- 172.21.15.175:0/1212768730 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55683f2c70e0 con 0x55683f2a4800 2024-09-21T11:35:56.261 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:35:56 smithi175 bash[33985]: fec51c1f8ac423528b1d77933b2baf80ef741bb7b97bd044f117a4070c8c0596 2024-09-21T11:35:56.261 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:35:56 smithi175 systemd[1]: Started Ceph mgr.x for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:35:56.408 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:35:57.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[29710]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:57.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[36454]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:57.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:35:57.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:35:57.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:57 smithi175 ceph-mon[32613]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:35:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:35:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:35:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:58.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: Reconfiguring mgr.y (unknown last config time)... 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[29710]: Reconfiguring daemon mgr.y on smithi043 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: Reconfiguring mgr.y (unknown last config time)... 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:35:58.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:35:58.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:58.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:58 smithi043 ceph-mon[36454]: Reconfiguring daemon mgr.y on smithi043 2024-09-21T11:35:58.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:35:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:35:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: Reconfiguring mgr.y (unknown last config time)... 2024-09-21T11:35:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T11:35:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T11:35:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:35:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:58 smithi175 ceph-mon[32613]: Reconfiguring daemon mgr.y on smithi043 2024-09-21T11:35:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:59 smithi043 ceph-mon[29710]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:35:59 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:59 smithi043 ceph-mon[36454]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:35:59 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:35:59.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:59 smithi175 ceph-mon[32613]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:35:59.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:35:59 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1301539431' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4"} : dispatch 2024-09-21T11:36:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4"} : dispatch 2024-09-21T11:36:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4"}]': finished 2024-09-21T11:36:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[36454]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T11:36:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3971046185' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:36:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1301539431' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4"} : dispatch 2024-09-21T11:36:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4"} : dispatch 2024-09-21T11:36:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4"}]': finished 2024-09-21T11:36:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[29710]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T11:36:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:00.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3971046185' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:36:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1301539431' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4"} : dispatch 2024-09-21T11:36:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:00 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4"} : dispatch 2024-09-21T11:36:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:00 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4"}]': finished 2024-09-21T11:36:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:00 smithi175 ceph-mon[32613]: osdmap e5: 1 total, 0 up, 1 in 2024-09-21T11:36:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3971046185' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:36:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[36454]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[36454]: Standby manager daemon x started 2024-09-21T11:36:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[36454]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:36:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[36454]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:36:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[36454]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:36:01.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[36454]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:36:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[29710]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[29710]: Standby manager daemon x started 2024-09-21T11:36:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[29710]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:36:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[29710]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:36:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[29710]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:36:01.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:01 smithi043 ceph-mon[29710]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:36:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:01 smithi175 ceph-mon[32613]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:01 smithi175 ceph-mon[32613]: Standby manager daemon x started 2024-09-21T11:36:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:01 smithi175 ceph-mon[32613]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:36:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:01 smithi175 ceph-mon[32613]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:36:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:01 smithi175 ceph-mon[32613]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:36:01.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:01 smithi175 ceph-mon[32613]: from='mgr.? 172.21.15.175:0/336815785' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:36:02.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:02 smithi043 ceph-mon[29710]: mgrmap e15: y(active, since 57s), standbys: x 2024-09-21T11:36:02.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:02 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:36:02.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:02 smithi043 ceph-mon[29710]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:02 smithi043 ceph-mon[36454]: mgrmap e15: y(active, since 57s), standbys: x 2024-09-21T11:36:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:02 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:36:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:02 smithi043 ceph-mon[36454]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:02 smithi175 ceph-mon[32613]: mgrmap e15: y(active, since 57s), standbys: x 2024-09-21T11:36:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:02 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:36:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:02 smithi175 ceph-mon[32613]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:04.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:04 smithi043 ceph-mon[29710]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:04.919 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:04 smithi043 ceph-mon[36454]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:05.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:04 smithi175 ceph-mon[32613]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:06 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:06 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:06 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:06 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:06 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:06 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:06.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:06.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:06.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:06.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:06.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:06.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:06.551 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:06.551 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:06.551 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:06.551 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:06.552 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:06.552 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:06 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:07.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:07 smithi043 ceph-mon[29710]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:07.466 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:07 smithi043 ceph-mon[36454]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:07.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:07 smithi175 ceph-mon[32613]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:08 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T11:36:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:08 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:08.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:08 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T11:36:08.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:08 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:08.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:08 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T11:36:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:08 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:09.407 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:09 smithi043 ceph-mon[29710]: Deploying daemon osd.0 on smithi043 2024-09-21T11:36:09.407 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:09 smithi043 ceph-mon[29710]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:09.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:09 smithi043 ceph-mon[36454]: Deploying daemon osd.0 on smithi043 2024-09-21T11:36:09.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:09 smithi043 ceph-mon[36454]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:09.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:09 smithi175 ceph-mon[32613]: Deploying daemon osd.0 on smithi043 2024-09-21T11:36:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:09 smithi175 ceph-mon[32613]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:11.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:11 smithi043 ceph-mon[29710]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:11 smithi043 ceph-mon[36454]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:11 smithi175 ceph-mon[32613]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:13 smithi043 ceph-mon[29710]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:13 smithi043 ceph-mon[36454]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:13.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:13 smithi175 ceph-mon[32613]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:15.444 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:15 smithi043 ceph-mon[29710]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:15.445 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:15 smithi043 ceph-mon[36454]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:15.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:15 smithi175 ceph-mon[32613]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:16.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:16 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:16.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:16 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:16.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:16 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:16.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:16 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:16.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:16 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:16.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:16 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:16 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:16 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:16 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:17.541 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:17 smithi043 ceph-mon[29710]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:17.542 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:17 smithi043 ceph-mon[36454]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:17.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:17 smithi175 ceph-mon[32613]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[29710]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[36454]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:19.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:19 smithi175 ceph-mon[32613]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:21.286 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 0 on host 'smithi043' 2024-09-21T11:36:21.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[36454]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:21.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[36454]: from='osd.0 v2:172.21.15.43:6801/817279193' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:36:21.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[36454]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:36:21.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:21.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:21.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[29710]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[29710]: from='osd.0 v2:172.21.15.43:6801/817279193' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[29710]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:21.719 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:21 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:21 smithi175 ceph-mon[32613]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:21 smithi175 ceph-mon[32613]: from='osd.0 v2:172.21.15.43:6801/817279193' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:36:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:21 smithi175 ceph-mon[32613]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T11:36:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:21 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:21 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:21 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:22.065 DEBUG:teuthology.orchestra.run.smithi043:osd.0> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.0.service 2024-09-21T11:36:22.067 INFO:tasks.cephadm:Deploying osd.1 on smithi043 with /dev/vg_nvme/lv_3... 2024-09-21T11:36:22.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T11:36:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[29710]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T11:36:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[29710]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T11:36:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[29710]: from='osd.0 v2:172.21.15.43:6801/817279193' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:36:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[29710]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:36:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[29710]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:36:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[36454]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T11:36:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[36454]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T11:36:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[36454]: from='osd.0 v2:172.21.15.43:6801/817279193' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:36:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[36454]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:36:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-mon[36454]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:36:22.742 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0[39801]: 2024-09-21T11:36:22.586+0000 7fdea6872640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T11:36:22.742 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 11:36:22 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0[39801]: 2024-09-21T11:36:22.589+0000 7fde9e691640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:36:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:22 smithi175 ceph-mon[32613]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T11:36:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:22 smithi175 ceph-mon[32613]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T11:36:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:22 smithi175 ceph-mon[32613]: from='osd.0 v2:172.21.15.43:6801/817279193' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:36:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:22 smithi175 ceph-mon[32613]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:36:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:22 smithi175 ceph-mon[32613]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:36:23.179 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:36:23.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[29710]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:23.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[29710]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T11:36:23.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:23.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:23.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[29710]: from='osd.0 ' entity='osd.0' 2024-09-21T11:36:23.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:23.412 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[36454]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:23.412 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[36454]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T11:36:23.412 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:23.412 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:23.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[36454]: from='osd.0 ' entity='osd.0' 2024-09-21T11:36:23.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:23 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:23 smithi175 ceph-mon[32613]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:23 smithi175 ceph-mon[32613]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T11:36:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:23 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:23 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:23 smithi175 ceph-mon[32613]: from='osd.0 ' entity='osd.0' 2024-09-21T11:36:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:23 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:36:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:36:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[29710]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[29710]: osd.0 v2:172.21.15.43:6801/817279193 boot 2024-09-21T11:36:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[29710]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T11:36:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:36:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:36:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[36454]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[36454]: osd.0 v2:172.21.15.43:6801/817279193 boot 2024-09-21T11:36:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[36454]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T11:36:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:24 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:36:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:24 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:36:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:24 smithi175 ceph-mon[32613]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T11:36:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:24 smithi175 ceph-mon[32613]: osd.0 v2:172.21.15.43:6801/817279193 boot 2024-09-21T11:36:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:24 smithi175 ceph-mon[32613]: osdmap e8: 1 total, 1 up, 1 in 2024-09-21T11:36:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:36:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[36454]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T11:36:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:36:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[29710]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T11:36:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:36:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:25 smithi175 ceph-mon[32613]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T11:36:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:25 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:25 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:25 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:36:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:25 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:25 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:25 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:25 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:26 smithi043 ceph-mon[36454]: Detected new or changed devices on smithi043 2024-09-21T11:36:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:26 smithi043 ceph-mon[36454]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:26 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi043 to 16068M 2024-09-21T11:36:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:26 smithi043 ceph-mon[29710]: Detected new or changed devices on smithi043 2024-09-21T11:36:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:26 smithi043 ceph-mon[29710]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:26 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi043 to 16068M 2024-09-21T11:36:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:26 smithi175 ceph-mon[32613]: Detected new or changed devices on smithi043 2024-09-21T11:36:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:26 smithi175 ceph-mon[32613]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:26 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi043 to 16068M 2024-09-21T11:36:27.363 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:36:27.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-09-21T11:36:27.651 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:36:28.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:28 smithi043 ceph-mon[29710]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:28.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:28 smithi043 ceph-mon[29710]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:36:28.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:36:28.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:36:28.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:28.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:28 smithi043 ceph-mon[36454]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:28.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:28 smithi043 ceph-mon[36454]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:36:28.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:36:28.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:36:28.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:28 smithi175 ceph-mon[32613]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:28 smithi175 ceph-mon[32613]: from='client.14247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:36:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:36:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:36:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:30.956 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:30 smithi043 ceph-mon[29710]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:30.957 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:30 smithi043 ceph-mon[36454]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:31.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:30 smithi175 ceph-mon[32613]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/571170565' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607"} : dispatch 2024-09-21T11:36:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607"} : dispatch 2024-09-21T11:36:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607"}]': finished 2024-09-21T11:36:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[29710]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T11:36:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2214859397' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:36:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/571170565' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607"} : dispatch 2024-09-21T11:36:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607"} : dispatch 2024-09-21T11:36:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607"}]': finished 2024-09-21T11:36:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[36454]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T11:36:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2214859397' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:36:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/571170565' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607"} : dispatch 2024-09-21T11:36:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:31 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607"} : dispatch 2024-09-21T11:36:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:31 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607"}]': finished 2024-09-21T11:36:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:31 smithi175 ceph-mon[32613]: osdmap e10: 2 total, 1 up, 2 in 2024-09-21T11:36:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:31 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2214859397' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:36:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:32 smithi043 ceph-mon[29710]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:32 smithi043 ceph-mon[36454]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:33.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:32 smithi175 ceph-mon[32613]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:34.862 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:34 smithi043 ceph-mon[36454]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:34.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:34 smithi043 ceph-mon[29710]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:35.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:34 smithi175 ceph-mon[32613]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:36.809 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:36 smithi043 ceph-mon[29710]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:36.809 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:36 smithi043 ceph-mon[36454]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:37.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:36 smithi175 ceph-mon[32613]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:38.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:38 smithi043 ceph-mon[36454]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:38.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:38 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T11:36:38.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:38 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:38.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:38 smithi043 ceph-mon[36454]: Deploying daemon osd.1 on smithi043 2024-09-21T11:36:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:38 smithi043 ceph-mon[29710]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:38 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T11:36:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:38 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:38 smithi043 ceph-mon[29710]: Deploying daemon osd.1 on smithi043 2024-09-21T11:36:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:38 smithi175 ceph-mon[32613]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:38 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T11:36:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:38 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:38 smithi175 ceph-mon[32613]: Deploying daemon osd.1 on smithi043 2024-09-21T11:36:40.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:40 smithi043 ceph-mon[29710]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:40 smithi043 ceph-mon[36454]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:41.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:40 smithi175 ceph-mon[32613]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:43 smithi043 ceph-mon[29710]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:43 smithi043 ceph-mon[36454]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:43 smithi175 ceph-mon[32613]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:45.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:45 smithi043 ceph-mon[29710]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:45.372 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:45 smithi043 ceph-mon[36454]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:45 smithi175 ceph-mon[32613]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:47.314 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:47 smithi175 ceph-mon[32613]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:47.314 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:47.314 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:47.314 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:47.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:47 smithi043 ceph-mon[29710]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:47.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:47.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:47.399 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:47.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:47 smithi043 ceph-mon[36454]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:47.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:47.399 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:47.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[36454]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[29710]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:49.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:49 smithi175 ceph-mon[32613]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:49.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:49.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:51.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:51 smithi043 ceph-mon[36454]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:51.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:51 smithi043 ceph-mon[36454]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T11:36:51.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:51 smithi043 ceph-mon[29710]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:51.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:51 smithi043 ceph-mon[29710]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T11:36:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:51 smithi175 ceph-mon[32613]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:51 smithi175 ceph-mon[32613]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T11:36:51.939 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 1 on host 'smithi043' 2024-09-21T11:36:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[29710]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T11:36:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[29710]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T11:36:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[29710]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:36:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[29710]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[29710]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[36454]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[36454]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[36454]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[36454]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:36:52.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[36454]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T11:36:52.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:52 smithi175 ceph-mon[32613]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T11:36:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:52 smithi175 ceph-mon[32613]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T11:36:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:52 smithi175 ceph-mon[32613]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:36:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:36:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:52 smithi175 ceph-mon[32613]: from='osd.1 v2:172.21.15.43:6805/263756171' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:36:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:52 smithi175 ceph-mon[32613]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T11:36:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:52.634 DEBUG:teuthology.orchestra.run.smithi043:osd.1> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.1.service 2024-09-21T11:36:52.636 INFO:tasks.cephadm:Deploying osd.2 on smithi043 with /dev/vg_nvme/lv_2... 2024-09-21T11:36:52.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T11:36:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:53 smithi043 ceph-mon[29710]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:53 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:53 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:53.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:53 smithi043 ceph-mon[36454]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:53 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:53 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:53.318 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:36:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:53 smithi175 ceph-mon[32613]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:53 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:53 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:54.374 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:54 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:36:54.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:54 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:36:54.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:54 smithi043 ceph-mon[36454]: osd.1 v2:172.21.15.43:6805/263756171 boot 2024-09-21T11:36:54.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:54 smithi043 ceph-mon[36454]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T11:36:54.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:54.375 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:54 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:36:54.375 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:54 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:36:54.375 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:54 smithi043 ceph-mon[29710]: osd.1 v2:172.21.15.43:6805/263756171 boot 2024-09-21T11:36:54.375 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:54 smithi043 ceph-mon[29710]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T11:36:54.375 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:54 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:36:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:54 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:36:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:54 smithi175 ceph-mon[32613]: osd.1 v2:172.21.15.43:6805/263756171 boot 2024-09-21T11:36:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:54 smithi175 ceph-mon[32613]: osdmap e13: 2 total, 2 up, 2 in 2024-09-21T11:36:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:36:55.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:55 smithi043 ceph-mon[36454]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:55.195 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:55 smithi043 ceph-mon[36454]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T11:36:55.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:55 smithi043 ceph-mon[29710]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:55.196 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:55 smithi043 ceph-mon[29710]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T11:36:55.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:55 smithi175 ceph-mon[32613]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T11:36:55.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:55 smithi175 ceph-mon[32613]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T11:36:57.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[29710]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:36:57.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[36454]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:36:57.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:57.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:57.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:57 smithi175 ceph-mon[32613]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:36:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:36:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:36:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:36:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:36:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:36:58.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:58 smithi043 ceph-mon[29710]: Detected new or changed devices on smithi043 2024-09-21T11:36:58.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:58 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi043 to 8034M 2024-09-21T11:36:58.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:58 smithi043 ceph-mon[36454]: Detected new or changed devices on smithi043 2024-09-21T11:36:58.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:58 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi043 to 8034M 2024-09-21T11:36:58.187 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:36:58.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-09-21T11:36:58.471 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:36:58.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:58 smithi175 ceph-mon[32613]: Detected new or changed devices on smithi043 2024-09-21T11:36:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:58 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi043 to 8034M 2024-09-21T11:36:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:36:59 smithi043 ceph-mon[29710]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:36:59.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:36:59 smithi043 ceph-mon[36454]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:36:59.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:36:59 smithi175 ceph-mon[32613]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:00.469 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:00 smithi043 ceph-mon[36454]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:37:00.469 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:37:00.469 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:37:00.469 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:00.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:00 smithi043 ceph-mon[29710]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:37:00.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:37:00.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:37:00.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:00.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:00 smithi175 ceph-mon[32613]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:37:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:37:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:37:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:01.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:01 smithi043 ceph-mon[29710]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:01 smithi043 ceph-mon[36454]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:01.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:01 smithi175 ceph-mon[32613]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/80965709' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce"} : dispatch 2024-09-21T11:37:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:02 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce"} : dispatch 2024-09-21T11:37:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:02 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce"}]': finished 2024-09-21T11:37:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:02 smithi043 ceph-mon[29710]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T11:37:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:02 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/80965709' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce"} : dispatch 2024-09-21T11:37:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:02 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce"} : dispatch 2024-09-21T11:37:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:02 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce"}]': finished 2024-09-21T11:37:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:02 smithi043 ceph-mon[36454]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T11:37:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:02 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:02.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/80965709' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce"} : dispatch 2024-09-21T11:37:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:02 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce"} : dispatch 2024-09-21T11:37:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:02 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce"}]': finished 2024-09-21T11:37:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:02 smithi175 ceph-mon[32613]: osdmap e15: 3 total, 2 up, 3 in 2024-09-21T11:37:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:02 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:03.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:03 smithi043 ceph-mon[29710]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:03.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:03 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1982359285' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:37:03.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:03 smithi043 ceph-mon[36454]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:03.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:03 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1982359285' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:37:03.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:03 smithi175 ceph-mon[32613]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:03.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:03 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1982359285' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:37:05.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:05 smithi043 ceph-mon[36454]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:05.185 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:05 smithi043 ceph-mon[29710]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:05.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:05 smithi175 ceph-mon[32613]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:07.218 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:07 smithi043 ceph-mon[29710]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:07.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:07 smithi043 ceph-mon[36454]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:07 smithi175 ceph-mon[32613]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:09 smithi043 ceph-mon[36454]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:09 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T11:37:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:09 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:09.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:09 smithi043 ceph-mon[29710]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:09.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:09 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T11:37:09.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:09 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:09 smithi175 ceph-mon[32613]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:09 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T11:37:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:09 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:10.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:10 smithi043 ceph-mon[36454]: Deploying daemon osd.2 on smithi043 2024-09-21T11:37:10.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:10 smithi043 ceph-mon[29710]: Deploying daemon osd.2 on smithi043 2024-09-21T11:37:10.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:10 smithi175 ceph-mon[32613]: Deploying daemon osd.2 on smithi043 2024-09-21T11:37:11.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:11 smithi043 ceph-mon[36454]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:11.206 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:11 smithi043 ceph-mon[29710]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:11.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:11 smithi175 ceph-mon[32613]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:13 smithi043 ceph-mon[29710]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:13.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:13 smithi043 ceph-mon[36454]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:13.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:13 smithi175 ceph-mon[32613]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:15.389 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:15 smithi043 ceph-mon[29710]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:15.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:15 smithi043 ceph-mon[36454]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:15.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:15 smithi175 ceph-mon[32613]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:17.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:17 smithi043 ceph-mon[29710]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:17.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:17 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:17.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:17 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:17.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:17 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:17.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:17 smithi043 ceph-mon[36454]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:17.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:17 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:17.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:17 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:17.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:17 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:17.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:17 smithi175 ceph-mon[32613]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:17.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:17 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:17.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:17 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:17.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:17 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:19.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:19 smithi043 ceph-mon[29710]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:19.373 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:19 smithi043 ceph-mon[36454]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:19.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:19 smithi175 ceph-mon[32613]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[36454]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[29710]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:21.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:20 smithi175 ceph-mon[32613]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:20 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:20 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:20 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:20 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:20 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:22.510 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 2 on host 'smithi043' 2024-09-21T11:37:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:22 smithi043 ceph-mon[29710]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:22 smithi043 ceph-mon[29710]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T11:37:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:22 smithi043 ceph-mon[36454]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:22 smithi043 ceph-mon[36454]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T11:37:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:22 smithi175 ceph-mon[32613]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:22 smithi175 ceph-mon[32613]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T11:37:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:23.151 DEBUG:teuthology.orchestra.run.smithi043:osd.2> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.2.service 2024-09-21T11:37:23.153 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_1... 2024-09-21T11:37:23.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T11:37:23.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:23 smithi043 ceph-mon[36454]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T11:37:23.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:23 smithi043 ceph-mon[36454]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T11:37:23.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:23 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:23.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:23 smithi043 ceph-mon[36454]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:37:23.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:23 smithi043 ceph-mon[29710]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T11:37:23.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:23 smithi043 ceph-mon[29710]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T11:37:23.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:23 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:23.684 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:23 smithi043 ceph-mon[29710]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:37:23.684 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 11:37:23 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2[50799]: 2024-09-21T11:37:23.605+0000 7f52af81e640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T11:37:23.684 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 11:37:23 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2[50799]: 2024-09-21T11:37:23.608+0000 7f52ab645640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:37:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:37:24.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:23 smithi175 ceph-mon[32613]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T11:37:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:23 smithi175 ceph-mon[32613]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T11:37:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:23 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:23 smithi175 ceph-mon[32613]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:37:24.665 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:24 smithi043 ceph-mon[29710]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:24.665 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:24 smithi043 ceph-mon[29710]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:37:24.665 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:24 smithi043 ceph-mon[29710]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T11:37:24.665 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:24.665 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:24.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:24 smithi043 ceph-mon[36454]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:24.665 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:24 smithi043 ceph-mon[36454]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:37:24.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:24 smithi043 ceph-mon[36454]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T11:37:24.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:24.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:24 smithi175 ceph-mon[32613]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:24 smithi175 ceph-mon[32613]: from='osd.2 v2:172.21.15.43:6809/687577863' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:37:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:24 smithi175 ceph-mon[32613]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T11:37:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:25 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:37:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:25 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:37:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:25 smithi043 ceph-mon[29710]: osd.2 v2:172.21.15.43:6809/687577863 boot 2024-09-21T11:37:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:25 smithi043 ceph-mon[29710]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T11:37:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:25 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:37:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:25 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:37:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:25 smithi043 ceph-mon[36454]: osd.2 v2:172.21.15.43:6809/687577863 boot 2024-09-21T11:37:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:25 smithi043 ceph-mon[36454]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T11:37:25.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:25 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:26.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:25 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:37:26.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:25 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:37:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:25 smithi175 ceph-mon[32613]: osd.2 v2:172.21.15.43:6809/687577863 boot 2024-09-21T11:37:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:25 smithi175 ceph-mon[32613]: osdmap e18: 3 total, 3 up, 3 in 2024-09-21T11:37:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:25 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:37:26.950 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:26 smithi043 ceph-mon[36454]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:26.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:26 smithi043 ceph-mon[36454]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T11:37:26.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:26 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:37:26.951 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:26 smithi043 ceph-mon[29710]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:26.951 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:26 smithi043 ceph-mon[29710]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T11:37:26.951 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:26 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:37:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:26 smithi175 ceph-mon[32613]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T11:37:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:26 smithi175 ceph-mon[32613]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T11:37:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:26 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:37:27.882 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T11:37:27.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T11:37:27.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:37:27.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: Detected new or changed devices on smithi043 2024-09-21T11:37:27.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.883 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi043 to 5356M 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 sudo[34443]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 sudo[34443]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 sudo[34443]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:27.884 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 sudo[34443]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:27.929 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54392]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:37:27.929 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54392]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:27.929 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54392]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:27.929 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54392]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:27.929 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54395]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:37:27.929 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54395]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:27.929 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54395]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:27.929 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54395]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:27.930 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54292]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:37:27.930 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54292]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:27.930 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54292]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:27.930 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54292]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:27.930 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54297]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:37:27.930 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54297]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:27.930 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54297]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:27.930 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54297]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: Detected new or changed devices on smithi043 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi043 to 5356M 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:27.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.932 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54382]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:37:27.932 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54382]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:27.932 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54382]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:27.932 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54382]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:27.932 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54389]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:37:27.932 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54389]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:27.932 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54389]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:27.932 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54389]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:27.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: Detected new or changed devices on smithi043 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi043 to 5356M 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54398]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:37:27.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54398]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:27.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54398]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:27.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54398]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:27.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54401]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:37:27.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54401]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:27.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54401]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:27.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54401]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54404]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T11:37:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54404]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54404]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54404]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54407]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:37:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54407]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54407]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:27 smithi043 sudo[54407]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 sudo[34446]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T11:37:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 sudo[34446]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T11:37:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 sudo[34446]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T11:37:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:27 smithi175 sudo[34446]: pam_unix(sudo:session): session closed for user root 2024-09-21T11:37:28.721 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:37:28.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-09-21T11:37:28.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:28.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:37:28.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T11:37:28.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:28.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:37:28.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:37:28.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:37:28.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[29710]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:37:28.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:37:28.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:37:28.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:28.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:37:28.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:37:28.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:37:28.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:37:28.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:37:28.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:28 smithi043 ceph-mon[36454]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:37:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:37:29.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:37:29.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:37:29.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:29.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T11:37:29.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:37:29.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:37:29.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:37:29.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T11:37:29.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:28 smithi175 ceph-mon[32613]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:37:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:37:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:29 smithi043 ceph-mon[29710]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T11:37:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:29 smithi043 ceph-mon[36454]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T11:37:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:29 smithi175 ceph-mon[32613]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T11:37:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[29710]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[29710]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:37:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:37:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[29710]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[29710]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[36454]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[36454]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[36454]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[36454]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:37:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:30 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:31.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:30 smithi175 ceph-mon[32613]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:30 smithi175 ceph-mon[32613]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:37:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:30 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:37:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:30 smithi175 ceph-mon[32613]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-21T11:37:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:30 smithi175 ceph-mon[32613]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:37:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:30 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:37:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:30 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:37:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:30 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:32 smithi043 ceph-mon[29710]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:32 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3969462587' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d"} : dispatch 2024-09-21T11:37:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:32 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3969462587' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d"}]': finished 2024-09-21T11:37:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:32 smithi043 ceph-mon[29710]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T11:37:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:32 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:32 smithi043 ceph-mon[36454]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:32 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3969462587' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d"} : dispatch 2024-09-21T11:37:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:32 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3969462587' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d"}]': finished 2024-09-21T11:37:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:32 smithi043 ceph-mon[36454]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T11:37:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:32 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:33.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:32 smithi175 ceph-mon[32613]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:33.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:32 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3969462587' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d"} : dispatch 2024-09-21T11:37:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:32 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3969462587' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d"}]': finished 2024-09-21T11:37:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:32 smithi175 ceph-mon[32613]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T11:37:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:32 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:33 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/736553135' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:37:33.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:33 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/736553135' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:37:34.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:33 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/736553135' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:37:34.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:34 smithi043 ceph-mon[29710]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:34.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:34 smithi043 ceph-mon[36454]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:35.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:34 smithi175 ceph-mon[32613]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:36.810 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:36 smithi043 ceph-mon[36454]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:36.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:36 smithi043 ceph-mon[29710]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:37.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:36 smithi175 ceph-mon[32613]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:38 smithi043 ceph-mon[29710]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:38.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:38 smithi043 ceph-mon[36454]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:38 smithi175 ceph-mon[32613]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:39 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T11:37:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:39 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T11:37:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:39 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T11:37:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:39 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:40.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:40 smithi043 ceph-mon[36454]: Deploying daemon osd.3 on smithi043 2024-09-21T11:37:40.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:40 smithi043 ceph-mon[36454]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:40.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:40 smithi043 ceph-mon[29710]: Deploying daemon osd.3 on smithi043 2024-09-21T11:37:40.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:40 smithi043 ceph-mon[29710]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:40 smithi175 ceph-mon[32613]: Deploying daemon osd.3 on smithi043 2024-09-21T11:37:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:40 smithi175 ceph-mon[32613]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:42.933 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:42 smithi043 ceph-mon[29710]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:42.933 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:42 smithi043 ceph-mon[36454]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:42 smithi175 ceph-mon[32613]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:44.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:44 smithi043 ceph-mon[29710]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:44 smithi043 ceph-mon[36454]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:45.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:44 smithi175 ceph-mon[32613]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:46.954 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:46 smithi043 ceph-mon[29710]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:46.955 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:46 smithi043 ceph-mon[36454]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:46 smithi175 ceph-mon[32613]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:48.378 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:48 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:48.378 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:48 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:48.378 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:48 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:48.379 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:48 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:48.379 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:48 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:48.379 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:48 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:48 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:48 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:48 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:49.321 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:49 smithi043 ceph-mon[29710]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:49.322 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:49 smithi043 ceph-mon[36454]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:49 smithi175 ceph-mon[32613]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:51.390 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[29710]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:51.390 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:51.390 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:51.390 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:51.390 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:51.391 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:51.391 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[36454]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:51.391 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:51.391 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:51.391 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:51.391 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:51.391 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:51 smithi175 ceph-mon[32613]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:52.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:52 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:37:52.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:52 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:37:52.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:52 smithi043 ceph-mon[36454]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T11:37:52.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:52.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:52 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:37:52.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:52 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:37:52.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:52 smithi043 ceph-mon[29710]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T11:37:52.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:52 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:37:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:52 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:37:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:52 smithi175 ceph-mon[32613]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T11:37:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:53.097 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-09-21T11:37:53.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:53 smithi043 ceph-mon[36454]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:53.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:53 smithi043 ceph-mon[36454]: from='osd.3 v2:172.21.15.43:6813/129493158' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:37:53.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:53 smithi043 ceph-mon[36454]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:37:53.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:53 smithi043 ceph-mon[29710]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:53.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:53 smithi043 ceph-mon[29710]: from='osd.3 v2:172.21.15.43:6813/129493158' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:37:53.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:53 smithi043 ceph-mon[29710]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:37:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:53 smithi175 ceph-mon[32613]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:53 smithi175 ceph-mon[32613]: from='osd.3 v2:172.21.15.43:6813/129493158' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:37:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:53 smithi175 ceph-mon[32613]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T11:37:53.829 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.3.service 2024-09-21T11:37:53.831 INFO:tasks.cephadm:Deploying osd.4 on smithi175 with /dev/vg_nvme/lv_4... 2024-09-21T11:37:53.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T11:37:54.083 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:37:54.204 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:54 smithi175 ceph-mon[32613]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T11:37:54.204 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:54 smithi175 ceph-mon[32613]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T11:37:54.204 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:54.204 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:54 smithi175 ceph-mon[32613]: from='osd.3 v2:172.21.15.43:6813/129493158' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:37:54.204 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:54 smithi175 ceph-mon[32613]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:37:54.204 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:54.204 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:54.204 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:54.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[29710]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T11:37:54.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[29710]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T11:37:54.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:54.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[29710]: from='osd.3 v2:172.21.15.43:6813/129493158' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:37:54.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[29710]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:37:54.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:54.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:54.346 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:54.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[36454]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T11:37:54.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[36454]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T11:37:54.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:54.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[36454]: from='osd.3 v2:172.21.15.43:6813/129493158' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:37:54.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[36454]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]} : dispatch 2024-09-21T11:37:54.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:37:54.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:54.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:54.741 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3[56403]: 2024-09-21T11:37:54.342+0000 7f28ce705640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T11:37:54.741 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 11:37:54 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3[56403]: 2024-09-21T11:37:54.345+0000 7f28c5d10640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:37:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[29710]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[29710]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:37:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[29710]: osdmap e26: 4 total, 3 up, 4 in 2024-09-21T11:37:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:55.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[29710]: from='osd.3 ' entity='osd.3' 2024-09-21T11:37:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[36454]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[36454]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:37:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[36454]: osdmap e26: 4 total, 3 up, 4 in 2024-09-21T11:37:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:55 smithi043 ceph-mon[36454]: from='osd.3 ' entity='osd.3' 2024-09-21T11:37:55.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:55 smithi175 ceph-mon[32613]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:55.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:55 smithi175 ceph-mon[32613]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-21T11:37:55.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:55 smithi175 ceph-mon[32613]: osdmap e26: 4 total, 3 up, 4 in 2024-09-21T11:37:55.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:55 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:55.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:55 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:55.340 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:55 smithi175 ceph-mon[32613]: from='osd.3 ' entity='osd.3' 2024-09-21T11:37:56.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:37:56.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:37:56.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:56.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[36454]: osd.3 v2:172.21.15.43:6813/129493158 boot 2024-09-21T11:37:56.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[36454]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T11:37:56.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:56.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:37:56.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:37:56.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:56.309 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[29710]: osd.3 v2:172.21.15.43:6813/129493158 boot 2024-09-21T11:37:56.309 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[29710]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T11:37:56.309 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:56 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:56.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:56 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:37:56.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:56 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:37:56.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:56 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:56.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:56 smithi175 ceph-mon[32613]: osd.3 v2:172.21.15.43:6813/129493158 boot 2024-09-21T11:37:56.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:56 smithi175 ceph-mon[32613]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T11:37:56.379 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:56 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:37:56.588 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:37:56.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi175:vg_nvme/lv_4 2024-09-21T11:37:56.870 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:37:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:57.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:57.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:57 smithi175 ceph-mon[32613]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T11:37:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[29710]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:37:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:37:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:37:57.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:57 smithi043 ceph-mon[36454]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T11:37:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:58 smithi175 ceph-mon[32613]: Detected new or changed devices on smithi043 2024-09-21T11:37:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:58 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi043 to 4017M 2024-09-21T11:37:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:37:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:37:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:58 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:58 smithi043 ceph-mon[29710]: Detected new or changed devices on smithi043 2024-09-21T11:37:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:58 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi043 to 4017M 2024-09-21T11:37:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:37:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:37:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:58 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:58 smithi043 ceph-mon[36454]: Detected new or changed devices on smithi043 2024-09-21T11:37:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:58 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi043 to 4017M 2024-09-21T11:37:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:37:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:37:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:58 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:37:59.490 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:59 smithi175 ceph-mon[32613]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:37:59.490 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:37:59 smithi175 ceph-mon[32613]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:37:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:59 smithi043 ceph-mon[36454]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:37:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:37:59 smithi043 ceph-mon[36454]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:37:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:59 smithi043 ceph-mon[29710]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:37:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:37:59 smithi043 ceph-mon[29710]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/2848205255' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c"} : dispatch 2024-09-21T11:38:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:00 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c"} : dispatch 2024-09-21T11:38:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:00 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c"}]': finished 2024-09-21T11:38:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:00 smithi043 ceph-mon[29710]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T11:38:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/2848205255' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c"} : dispatch 2024-09-21T11:38:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:00 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c"} : dispatch 2024-09-21T11:38:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:00 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c"}]': finished 2024-09-21T11:38:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:00 smithi043 ceph-mon[36454]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T11:38:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/2848205255' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c"} : dispatch 2024-09-21T11:38:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:00 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c"} : dispatch 2024-09-21T11:38:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:00 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c"}]': finished 2024-09-21T11:38:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:00 smithi175 ceph-mon[32613]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T11:38:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:01.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:01 smithi043 ceph-mon[29710]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:01.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/3713259973' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:38:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:01 smithi043 ceph-mon[36454]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/3713259973' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:38:01.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:01 smithi175 ceph-mon[32613]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:01.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/3713259973' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:38:03.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:03 smithi175 ceph-mon[32613]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:03 smithi043 ceph-mon[29710]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:03 smithi043 ceph-mon[36454]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:05.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:05 smithi175 ceph-mon[32613]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:05.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:05 smithi043 ceph-mon[29710]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:05.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:05 smithi043 ceph-mon[36454]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:07.447 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:07 smithi175 ceph-mon[32613]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:07.447 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:07 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T11:38:07.447 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:07 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:07.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:07 smithi043 ceph-mon[29710]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:07.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:07 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T11:38:07.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:07 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:07 smithi043 ceph-mon[36454]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:07 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T11:38:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:07 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:08.496 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:08 smithi175 ceph-mon[32613]: Deploying daemon osd.4 on smithi175 2024-09-21T11:38:08.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:08 smithi043 ceph-mon[36454]: Deploying daemon osd.4 on smithi175 2024-09-21T11:38:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:08 smithi043 ceph-mon[29710]: Deploying daemon osd.4 on smithi175 2024-09-21T11:38:09.642 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:09 smithi043 ceph-mon[29710]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:09.642 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:09 smithi043 ceph-mon[36454]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:09 smithi175 ceph-mon[32613]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:10.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:10 smithi043 ceph-mon[29710]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:10.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:10 smithi043 ceph-mon[36454]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:10.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:10 smithi175 ceph-mon[32613]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:12 smithi043 ceph-mon[29710]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:12.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:12 smithi043 ceph-mon[36454]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:12 smithi175 ceph-mon[32613]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:14.856 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:14 smithi175 ceph-mon[32613]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:14.856 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:14 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:14.856 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:14 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:14.856 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:14 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:14 smithi043 ceph-mon[29710]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:14 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:14 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:14 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:14.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:14 smithi043 ceph-mon[36454]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:14 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:14 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:14.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:14 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[29710]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[36454]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:16 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:16 smithi175 ceph-mon[32613]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:16 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:16 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:16 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:16 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:16 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:17 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:38:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:17 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:38:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:17 smithi043 ceph-mon[29710]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T11:38:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:17 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:17 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:38:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:17 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:38:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:17 smithi043 ceph-mon[36454]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T11:38:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:17 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:18.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:17 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:38:18.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:17 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:38:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:17 smithi175 ceph-mon[32613]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T11:38:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:17 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:18.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:18 smithi175 ceph-mon[32613]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:18 smithi043 ceph-mon[29710]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:18 smithi043 ceph-mon[36454]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:19.653 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 4 on host 'smithi175' 2024-09-21T11:38:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:19 smithi043 ceph-mon[29710]: from='osd.4 v2:172.21.15.175:6800/1323992640' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:38:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:19 smithi043 ceph-mon[29710]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:38:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:19.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:19 smithi043 ceph-mon[36454]: from='osd.4 v2:172.21.15.175:6800/1323992640' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:38:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:19 smithi043 ceph-mon[36454]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:38:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:19.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:19 smithi175 ceph-mon[32613]: from='osd.4 v2:172.21.15.175:6800/1323992640' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:38:19.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:19 smithi175 ceph-mon[32613]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T11:38:19.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:19.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:19.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:20.298 DEBUG:teuthology.orchestra.run.smithi175:osd.4> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.4.service 2024-09-21T11:38:20.300 INFO:tasks.cephadm:Deploying osd.5 on smithi175 with /dev/vg_nvme/lv_3... 2024-09-21T11:38:20.300 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T11:38:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[36454]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[36454]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T11:38:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[36454]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T11:38:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[36454]: from='osd.4 v2:172.21.15.175:6800/1323992640' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[36454]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[29710]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[29710]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T11:38:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[29710]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T11:38:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[29710]: from='osd.4 v2:172.21.15.175:6800/1323992640' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:20 smithi043 ceph-mon[29710]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:20 smithi175 ceph-mon[32613]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:20 smithi175 ceph-mon[32613]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T11:38:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:20 smithi175 ceph-mon[32613]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T11:38:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:20 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:20 smithi175 ceph-mon[32613]: from='osd.4 v2:172.21.15.175:6800/1323992640' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:20 smithi175 ceph-mon[32613]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:21.469 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:38:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[36714]: 2024-09-21T11:38:21.108+0000 7f713282c640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T11:38:21.470 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:38:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[36714]: 2024-09-21T11:38:21.114+0000 7f712a64b640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:38:21.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:21 smithi175 ceph-mon[32613]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:38:21.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:21 smithi175 ceph-mon[32613]: osdmap e32: 5 total, 4 up, 5 in 2024-09-21T11:38:21.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:21 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:21.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:21 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:21.738 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:21 smithi175 ceph-mon[32613]: from='osd.4 ' entity='osd.4' 2024-09-21T11:38:21.805 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:38:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:21 smithi043 ceph-mon[29710]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:38:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:21 smithi043 ceph-mon[29710]: osdmap e32: 5 total, 4 up, 5 in 2024-09-21T11:38:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:21 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:21 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:21 smithi043 ceph-mon[29710]: from='osd.4 ' entity='osd.4' 2024-09-21T11:38:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:21 smithi043 ceph-mon[36454]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:38:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:21 smithi043 ceph-mon[36454]: osdmap e32: 5 total, 4 up, 5 in 2024-09-21T11:38:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:21 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:21 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:21.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:21 smithi043 ceph-mon[36454]: from='osd.4 ' entity='osd.4' 2024-09-21T11:38:22.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:38:22.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:38:22.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[36454]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:22.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[36454]: osd.4 v2:172.21.15.175:6800/1323992640 boot 2024-09-21T11:38:22.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[36454]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T11:38:22.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:22.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[36454]: osdmap e34: 5 total, 5 up, 5 in 2024-09-21T11:38:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:38:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:38:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[29710]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[29710]: osd.4 v2:172.21.15.175:6800/1323992640 boot 2024-09-21T11:38:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[29710]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T11:38:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:22 smithi043 ceph-mon[29710]: osdmap e34: 5 total, 5 up, 5 in 2024-09-21T11:38:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:22 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:38:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:22 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:38:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:22 smithi175 ceph-mon[32613]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T11:38:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:22 smithi175 ceph-mon[32613]: osd.4 v2:172.21.15.175:6800/1323992640 boot 2024-09-21T11:38:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:22 smithi175 ceph-mon[32613]: osdmap e33: 5 total, 5 up, 5 in 2024-09-21T11:38:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:38:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:22 smithi175 ceph-mon[32613]: osdmap e34: 5 total, 5 up, 5 in 2024-09-21T11:38:24.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:24 smithi175 ceph-mon[32613]: osdmap e35: 5 total, 5 up, 5 in 2024-09-21T11:38:24.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:38:24.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:24.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:24.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[29710]: osdmap e35: 5 total, 5 up, 5 in 2024-09-21T11:38:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:38:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:24.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[36454]: osdmap e35: 5 total, 5 up, 5 in 2024-09-21T11:38:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:38:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:24.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:38:25.023 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi175:vg_nvme/lv_3 2024-09-21T11:38:25.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:25 smithi175 ceph-mon[32613]: Detected new or changed devices on smithi175 2024-09-21T11:38:25.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:25 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi175 to 17088M 2024-09-21T11:38:25.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:25 smithi175 ceph-mon[32613]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 479 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:25.267 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:38:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:25 smithi043 ceph-mon[29710]: Detected new or changed devices on smithi175 2024-09-21T11:38:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:25 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi175 to 17088M 2024-09-21T11:38:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:25 smithi043 ceph-mon[29710]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 479 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:25 smithi043 ceph-mon[36454]: Detected new or changed devices on smithi175 2024-09-21T11:38:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:25 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi175 to 17088M 2024-09-21T11:38:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:25 smithi043 ceph-mon[36454]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 479 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:26.396 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:26 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:38:26.396 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:26 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:38:26.396 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:26 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:26.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:26 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:38:26.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:26 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:38:26.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:26 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:26.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:26 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:38:26.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:26 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:38:26.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:26 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:27.371 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:27 smithi175 ceph-mon[32613]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 479 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T11:38:27.371 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:27 smithi175 ceph-mon[32613]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:38:27.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:27 smithi043 ceph-mon[29710]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 479 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T11:38:27.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:27 smithi043 ceph-mon[29710]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:38:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:27 smithi043 ceph-mon[36454]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 479 MiB used, 447 GiB / 447 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T11:38:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:27 smithi043 ceph-mon[36454]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:38:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[29710]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T11:38:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/3009836391' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4d647321-71a1-412b-bbf5-921441923a65"} : dispatch 2024-09-21T11:38:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4d647321-71a1-412b-bbf5-921441923a65"} : dispatch 2024-09-21T11:38:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d647321-71a1-412b-bbf5-921441923a65"}]': finished 2024-09-21T11:38:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[29710]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T11:38:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/400790505' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:38:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[36454]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T11:38:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/3009836391' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4d647321-71a1-412b-bbf5-921441923a65"} : dispatch 2024-09-21T11:38:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4d647321-71a1-412b-bbf5-921441923a65"} : dispatch 2024-09-21T11:38:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d647321-71a1-412b-bbf5-921441923a65"}]': finished 2024-09-21T11:38:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[36454]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T11:38:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:29.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/400790505' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:38:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:29 smithi175 ceph-mon[32613]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T11:38:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/3009836391' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4d647321-71a1-412b-bbf5-921441923a65"} : dispatch 2024-09-21T11:38:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:29 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4d647321-71a1-412b-bbf5-921441923a65"} : dispatch 2024-09-21T11:38:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:29 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d647321-71a1-412b-bbf5-921441923a65"}]': finished 2024-09-21T11:38:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:29 smithi175 ceph-mon[32613]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T11:38:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:29 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/400790505' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:38:31.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:31 smithi043 ceph-mon[29710]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T11:38:31.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:31 smithi043 ceph-mon[36454]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T11:38:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:31 smithi175 ceph-mon[32613]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-21T11:38:33.305 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:33 smithi175 ceph-mon[32613]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T11:38:33.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:33 smithi043 ceph-mon[36454]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T11:38:33.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:33 smithi043 ceph-mon[29710]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-21T11:38:35.476 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:35 smithi175 ceph-mon[32613]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:38:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:35 smithi043 ceph-mon[29710]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:38:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:35 smithi043 ceph-mon[36454]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:38:36.370 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:36 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T11:38:36.370 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:36 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:36.370 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:36 smithi175 ceph-mon[32613]: Deploying daemon osd.5 on smithi175 2024-09-21T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:36 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:36 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:36.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:36 smithi043 ceph-mon[29710]: Deploying daemon osd.5 on smithi175 2024-09-21T11:38:36.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:36 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T11:38:36.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:36 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:36.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:36 smithi043 ceph-mon[36454]: Deploying daemon osd.5 on smithi175 2024-09-21T11:38:37.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:37 smithi175 ceph-mon[32613]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:37 smithi043 ceph-mon[29710]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:37.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:37 smithi043 ceph-mon[36454]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:39.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:39 smithi043 ceph-mon[29710]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:39.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:39 smithi043 ceph-mon[36454]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:39.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:39 smithi175 ceph-mon[32613]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:40.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:40 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:38:40.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:40 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:38:40.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:40 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:38:40.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:40 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:38:40.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:40 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:38:40.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:40 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:38:41.374 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:41 smithi175 ceph-mon[32613]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:41.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:41 smithi043 ceph-mon[29710]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:41 smithi043 ceph-mon[36454]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:43.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:43 smithi175 ceph-mon[32613]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:43.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:43 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:43.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:43 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:43.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:43 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:43 smithi043 ceph-mon[29710]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:43 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:43 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:43 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:43 smithi043 ceph-mon[36454]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:43 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:43 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:43.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:43 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:45 smithi043 ceph-mon[29710]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:45.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:45 smithi043 ceph-mon[36454]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:45.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:45 smithi175 ceph-mon[32613]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:46 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:46 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:46 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:46 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:46.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:46 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:46 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:46 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:46 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:46 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:46.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:46 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:46.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:46 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:46.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:46 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:46.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:46 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:46.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:46 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:46.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:46 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:47.471 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:47 smithi175 ceph-mon[32613]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:47.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:47 smithi043 ceph-mon[29710]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:47.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:47 smithi043 ceph-mon[36454]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:48.182 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 5 on host 'smithi175' 2024-09-21T11:38:48.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:48 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:38:48.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:48 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:38:48.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:48 smithi175 ceph-mon[32613]: from='osd.5 v2:172.21.15.175:6804/1163208458' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:38:48.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:48 smithi175 ceph-mon[32613]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:38:48.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:48 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:48.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:48 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:48.487 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:48 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:38:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:38:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[29710]: from='osd.5 v2:172.21.15.175:6804/1163208458' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:38:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[29710]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:38:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:38:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:38:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[36454]: from='osd.5 v2:172.21.15.175:6804/1163208458' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:38:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[36454]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T11:38:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:38:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:48 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:48.790 DEBUG:teuthology.orchestra.run.smithi175:osd.5> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.5.service 2024-09-21T11:38:48.792 INFO:tasks.cephadm:Deploying osd.6 on smithi175 with /dev/vg_nvme/lv_2... 2024-09-21T11:38:48.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T11:38:49.373 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:49 smithi175 ceph-mon[32613]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:49.373 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:49 smithi175 ceph-mon[32613]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T11:38:49.373 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:49 smithi175 ceph-mon[32613]: from='osd.5 v2:172.21.15.175:6804/1163208458' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:49.374 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:49 smithi175 ceph-mon[32613]: osdmap e37: 6 total, 5 up, 6 in 2024-09-21T11:38:49.374 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:49.374 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:49 smithi175 ceph-mon[32613]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[29710]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[29710]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T11:38:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[29710]: from='osd.5 v2:172.21.15.175:6804/1163208458' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[29710]: osdmap e37: 6 total, 5 up, 6 in 2024-09-21T11:38:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[29710]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[36454]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[36454]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T11:38:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[36454]: from='osd.5 v2:172.21.15.175:6804/1163208458' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[36454]: osdmap e37: 6 total, 5 up, 6 in 2024-09-21T11:38:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:49.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:49 smithi043 ceph-mon[36454]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:38:49.991 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 11:38:49 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T11:38:49.624+0000 7fd297fe1640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T11:38:49.991 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 11:38:49 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T11:38:49.630+0000 7fd28fe00640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:38:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:50 smithi175 ceph-mon[32613]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:38:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:50 smithi175 ceph-mon[32613]: osdmap e38: 6 total, 5 up, 6 in 2024-09-21T11:38:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:50 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:50 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:50.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:50 smithi175 ceph-mon[32613]: from='osd.5 ' entity='osd.5' 2024-09-21T11:38:50.401 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:38:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:50 smithi043 ceph-mon[29710]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:38:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:50 smithi043 ceph-mon[29710]: osdmap e38: 6 total, 5 up, 6 in 2024-09-21T11:38:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:50 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:50 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:50 smithi043 ceph-mon[29710]: from='osd.5 ' entity='osd.5' 2024-09-21T11:38:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:50 smithi043 ceph-mon[36454]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:38:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:50 smithi043 ceph-mon[36454]: osdmap e38: 6 total, 5 up, 6 in 2024-09-21T11:38:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:50 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:50 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:50 smithi043 ceph-mon[36454]: from='osd.5 ' entity='osd.5' 2024-09-21T11:38:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:38:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:38:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[29710]: pgmap v141: 1 pgs: 1 unknown; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[29710]: osd.5 v2:172.21.15.175:6804/1163208458 boot 2024-09-21T11:38:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[29710]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T11:38:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[29710]: osdmap e40: 6 total, 6 up, 6 in 2024-09-21T11:38:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:38:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:38:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[36454]: pgmap v141: 1 pgs: 1 unknown; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[36454]: osd.5 v2:172.21.15.175:6804/1163208458 boot 2024-09-21T11:38:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[36454]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T11:38:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:51 smithi043 ceph-mon[36454]: osdmap e40: 6 total, 6 up, 6 in 2024-09-21T11:38:51.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:51 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:38:51.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:51 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:38:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:51 smithi175 ceph-mon[32613]: pgmap v141: 1 pgs: 1 unknown; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-21T11:38:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:51 smithi175 ceph-mon[32613]: osd.5 v2:172.21.15.175:6804/1163208458 boot 2024-09-21T11:38:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:51 smithi175 ceph-mon[32613]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T11:38:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:38:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:51 smithi175 ceph-mon[32613]: osdmap e40: 6 total, 6 up, 6 in 2024-09-21T11:38:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: pgmap v144: 1 pgs: 1 unknown; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:38:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: Detected new or changed devices on smithi175 2024-09-21T11:38:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi175 to 8544M 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[29710]: osdmap e41: 6 total, 6 up, 6 in 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: pgmap v144: 1 pgs: 1 unknown; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: Detected new or changed devices on smithi175 2024-09-21T11:38:53.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:38:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:38:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi175 to 8544M 2024-09-21T11:38:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:52 smithi043 ceph-mon[36454]: osdmap e41: 6 total, 6 up, 6 in 2024-09-21T11:38:53.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: pgmap v144: 1 pgs: 1 unknown; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:38:53.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: Detected new or changed devices on smithi175 2024-09-21T11:38:53.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:38:53.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:38:53.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi175 to 8544M 2024-09-21T11:38:53.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:53.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:38:53.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:38:53.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:52 smithi175 ceph-mon[32613]: osdmap e41: 6 total, 6 up, 6 in 2024-09-21T11:38:53.549 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:38:53.576 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi175:vg_nvme/lv_2 2024-09-21T11:38:53.820 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:38:55.068 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:54 smithi175 ceph-mon[32613]: pgmap v146: 1 pgs: 1 unknown; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:38:55.068 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:38:55.068 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:38:55.068 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:54 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:54 smithi043 ceph-mon[29710]: pgmap v146: 1 pgs: 1 unknown; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:38:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:38:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:38:55.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:54 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:55.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:54 smithi043 ceph-mon[36454]: pgmap v146: 1 pgs: 1 unknown; 577 KiB data, 570 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:38:55.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:38:55.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:38:55.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:54 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:38:56.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:55 smithi043 ceph-mon[29710]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:38:56.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:55 smithi043 ceph-mon[36454]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:38:56.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:55 smithi175 ceph-mon[32613]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:38:57.229 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:56 smithi175 ceph-mon[32613]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T11:38:57.230 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:56 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/940980378' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354"} : dispatch 2024-09-21T11:38:57.230 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:56 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354"} : dispatch 2024-09-21T11:38:57.230 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:56 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354"}]': finished 2024-09-21T11:38:57.230 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:56 smithi175 ceph-mon[32613]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T11:38:57.230 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:56 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:38:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[29710]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T11:38:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/940980378' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354"} : dispatch 2024-09-21T11:38:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354"} : dispatch 2024-09-21T11:38:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354"}]': finished 2024-09-21T11:38:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[29710]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T11:38:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:38:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[36454]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T11:38:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/940980378' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354"} : dispatch 2024-09-21T11:38:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354"} : dispatch 2024-09-21T11:38:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354"}]': finished 2024-09-21T11:38:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[36454]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T11:38:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:56 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:38:58.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/1573989625' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:38:58.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:57 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/1573989625' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:38:58.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:57 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/1573989625' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:38:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:38:58 smithi043 ceph-mon[29710]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-21T11:38:59.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:38:58 smithi043 ceph-mon[36454]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-21T11:38:59.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:38:58 smithi175 ceph-mon[32613]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-21T11:39:01.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:00 smithi175 ceph-mon[32613]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:39:01.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:00 smithi043 ceph-mon[29710]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:39:01.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:00 smithi043 ceph-mon[36454]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:39:03.096 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:02 smithi175 ceph-mon[32613]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T11:39:03.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:02 smithi043 ceph-mon[29710]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T11:39:03.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:02 smithi043 ceph-mon[36454]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T11:39:05.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:04 smithi043 ceph-mon[29710]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:39:05.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:04 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T11:39:05.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:04 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:05.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:04 smithi043 ceph-mon[29710]: Deploying daemon osd.6 on smithi175 2024-09-21T11:39:05.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:04 smithi043 ceph-mon[36454]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:39:05.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:04 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T11:39:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:04 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:05.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:04 smithi043 ceph-mon[36454]: Deploying daemon osd.6 on smithi175 2024-09-21T11:39:05.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:04 smithi175 ceph-mon[32613]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:39:05.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:04 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T11:39:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:04 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:04 smithi175 ceph-mon[32613]: Deploying daemon osd.6 on smithi175 2024-09-21T11:39:07.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:06 smithi043 ceph-mon[29710]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:07.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:06 smithi043 ceph-mon[36454]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:07.246 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:06 smithi175 ceph-mon[32613]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:09.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:09 smithi043 ceph-mon[36454]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:09.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:09 smithi043 ceph-mon[29710]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:09.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:09 smithi175 ceph-mon[32613]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:10.344 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:10 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:39:10.345 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:10 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:39:10.345 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:10 smithi175 ceph-mon[32613]: osdmap e43: 7 total, 6 up, 7 in 2024-09-21T11:39:10.345 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:10 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:10 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:39:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:10 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:39:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:10 smithi043 ceph-mon[29710]: osdmap e43: 7 total, 6 up, 7 in 2024-09-21T11:39:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:10 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:10 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:39:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:10 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:39:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:10 smithi043 ceph-mon[36454]: osdmap e43: 7 total, 6 up, 7 in 2024-09-21T11:39:10.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:10 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:11.452 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:11 smithi175 ceph-mon[32613]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:11.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:11 smithi043 ceph-mon[29710]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:11 smithi043 ceph-mon[36454]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:12.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:12 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:12.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:12 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:12.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:12 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:12.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:12 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:12.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:12 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:12.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:12 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:12.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:12 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:12.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:12 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:12.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:12 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:13 smithi175 ceph-mon[32613]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:13 smithi043 ceph-mon[29710]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:13.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:13 smithi043 ceph-mon[36454]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:15.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[36454]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:15.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:15.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:15.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:15.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:15.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:15.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[29710]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:15.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:15.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:15.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:15.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:15.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:15 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:15.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:15 smithi175 ceph-mon[32613]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:15 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:15 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:15 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:15 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:15 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:16.895 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:16 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:39:16.895 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:16 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:39:16.895 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:16 smithi175 ceph-mon[32613]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:16.896 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:16 smithi175 ceph-mon[32613]: osdmap e44: 7 total, 6 up, 7 in 2024-09-21T11:39:16.896 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:16 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:16.896 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:16 smithi175 ceph-mon[32613]: from='osd.6 v2:172.21.15.175:6808/932395625' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:39:16.896 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:16 smithi175 ceph-mon[32613]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:39:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:39:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:39:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[29710]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[29710]: osdmap e44: 7 total, 6 up, 7 in 2024-09-21T11:39:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[29710]: from='osd.6 v2:172.21.15.175:6808/932395625' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:39:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[29710]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:39:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:39:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:39:16.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[36454]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:16.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[36454]: osdmap e44: 7 total, 6 up, 7 in 2024-09-21T11:39:16.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:16.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[36454]: from='osd.6 v2:172.21.15.175:6808/932395625' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:39:16.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:16 smithi043 ceph-mon[36454]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T11:39:17.205 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 6 on host 'smithi175' 2024-09-21T11:39:17.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:17 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:17.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:17 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:17.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:17 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:17.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:17 smithi175 ceph-mon[32613]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T11:39:17.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:17 smithi175 ceph-mon[32613]: osdmap e45: 7 total, 6 up, 7 in 2024-09-21T11:39:17.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:17 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:17.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:17 smithi175 ceph-mon[32613]: from='osd.6 v2:172.21.15.175:6808/932395625' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:17.866 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:17 smithi175 ceph-mon[32613]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:17.892 DEBUG:teuthology.orchestra.run.smithi175:osd.6> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.6.service 2024-09-21T11:39:17.895 INFO:tasks.cephadm:Deploying osd.7 on smithi175 with /dev/vg_nvme/lv_1... 2024-09-21T11:39:17.895 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T11:39:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[36454]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T11:39:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[36454]: osdmap e45: 7 total, 6 up, 7 in 2024-09-21T11:39:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[36454]: from='osd.6 v2:172.21.15.175:6808/932395625' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[36454]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[29710]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T11:39:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[29710]: osdmap e45: 7 total, 6 up, 7 in 2024-09-21T11:39:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[29710]: from='osd.6 v2:172.21.15.175:6808/932395625' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:17 smithi043 ceph-mon[29710]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:19.009 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:39:19.010 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 11:39:18 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T11:39:18.733+0000 7efef02b1640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T11:39:19.010 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 11:39:18 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T11:39:18.739+0000 7efee80d0640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:39:19.366 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:19 smithi175 ceph-mon[32613]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:19.366 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:19 smithi175 ceph-mon[32613]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:39:19.366 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:19 smithi175 ceph-mon[32613]: osdmap e46: 7 total, 6 up, 7 in 2024-09-21T11:39:19.366 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:19.366 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:19.366 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:19 smithi175 ceph-mon[32613]: from='osd.6 ' entity='osd.6' 2024-09-21T11:39:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[36454]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[36454]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:39:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[36454]: osdmap e46: 7 total, 6 up, 7 in 2024-09-21T11:39:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[36454]: from='osd.6 ' entity='osd.6' 2024-09-21T11:39:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[29710]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-21T11:39:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[29710]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:39:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[29710]: osdmap e46: 7 total, 6 up, 7 in 2024-09-21T11:39:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:19 smithi043 ceph-mon[29710]: from='osd.6 ' entity='osd.6' 2024-09-21T11:39:20.330 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:20 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:39:20.331 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:20 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:39:20.331 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:20 smithi175 ceph-mon[32613]: osd.6 v2:172.21.15.175:6808/932395625 boot 2024-09-21T11:39:20.331 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:20 smithi175 ceph-mon[32613]: osdmap e47: 7 total, 7 up, 7 in 2024-09-21T11:39:20.331 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:20 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:20 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:39:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:20 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:39:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:20 smithi043 ceph-mon[29710]: osd.6 v2:172.21.15.175:6808/932395625 boot 2024-09-21T11:39:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:20 smithi043 ceph-mon[29710]: osdmap e47: 7 total, 7 up, 7 in 2024-09-21T11:39:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:20 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:20 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:39:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:20 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:39:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:20 smithi043 ceph-mon[36454]: osd.6 v2:172.21.15.175:6808/932395625 boot 2024-09-21T11:39:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:20 smithi043 ceph-mon[36454]: osdmap e47: 7 total, 7 up, 7 in 2024-09-21T11:39:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:20 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:39:21.692 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:21 smithi175 ceph-mon[32613]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 533 MiB used, 625 GiB / 626 GiB avail 2024-09-21T11:39:21.692 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:21 smithi175 ceph-mon[32613]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T11:39:21.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:21 smithi175 ceph-mon[32613]: osdmap e48: 7 total, 7 up, 7 in 2024-09-21T11:39:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:21 smithi043 ceph-mon[29710]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 533 MiB used, 625 GiB / 626 GiB avail 2024-09-21T11:39:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:21 smithi043 ceph-mon[29710]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T11:39:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:21 smithi043 ceph-mon[29710]: osdmap e48: 7 total, 7 up, 7 in 2024-09-21T11:39:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:21 smithi043 ceph-mon[36454]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 533 MiB used, 625 GiB / 626 GiB avail 2024-09-21T11:39:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:21 smithi043 ceph-mon[36454]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-21T11:39:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:21 smithi043 ceph-mon[36454]: osdmap e48: 7 total, 7 up, 7 in 2024-09-21T11:39:22.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:22 smithi175 ceph-mon[32613]: osdmap e49: 7 total, 7 up, 7 in 2024-09-21T11:39:22.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:22.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:22.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:22.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:22.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:22.554 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:22 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[29710]: osdmap e49: 7 total, 7 up, 7 in 2024-09-21T11:39:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[36454]: osdmap e49: 7 total, 7 up, 7 in 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:22.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:22.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:22 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:23.202 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T11:39:23.231 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch daemon add osd smithi175:vg_nvme/lv_1 2024-09-21T11:39:23.482 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:39:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:23 smithi175 ceph-mon[32613]: Detected new or changed devices on smithi175 2024-09-21T11:39:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:23 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi175 to 5696M 2024-09-21T11:39:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:23 smithi175 ceph-mon[32613]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 533 MiB used, 625 GiB / 626 GiB avail 2024-09-21T11:39:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:23 smithi043 ceph-mon[29710]: Detected new or changed devices on smithi175 2024-09-21T11:39:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:23 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi175 to 5696M 2024-09-21T11:39:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:23 smithi043 ceph-mon[29710]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 533 MiB used, 625 GiB / 626 GiB avail 2024-09-21T11:39:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:23 smithi043 ceph-mon[36454]: Detected new or changed devices on smithi175 2024-09-21T11:39:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:23 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi175 to 5696M 2024-09-21T11:39:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:23 smithi043 ceph-mon[36454]: pgmap v168: 1 pgs: 1 remapped+peering; 577 KiB data, 533 MiB used, 625 GiB / 626 GiB avail 2024-09-21T11:39:24.636 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:39:24.636 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:39:24.636 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:39:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:39:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:24.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T11:39:24.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T11:39:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:25.482 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:25 smithi175 ceph-mon[32613]: pgmap v169: 1 pgs: 1 remapped+peering; 577 KiB data, 533 MiB used, 625 GiB / 626 GiB avail 2024-09-21T11:39:25.482 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:25 smithi175 ceph-mon[32613]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:39:25.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:25 smithi043 ceph-mon[29710]: pgmap v169: 1 pgs: 1 remapped+peering; 577 KiB data, 533 MiB used, 625 GiB / 626 GiB avail 2024-09-21T11:39:25.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:25 smithi043 ceph-mon[29710]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:39:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:25 smithi043 ceph-mon[36454]: pgmap v169: 1 pgs: 1 remapped+peering; 577 KiB data, 533 MiB used, 625 GiB / 626 GiB avail 2024-09-21T11:39:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:25 smithi043 ceph-mon[36454]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:39:26.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:26 smithi175 ceph-mon[32613]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T11:39:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:26 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:39:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:26 smithi043 ceph-mon[29710]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T11:39:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:26 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:39:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:26 smithi043 ceph-mon[36454]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-21T11:39:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:26 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:39:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[36454]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/2829274315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22"} : dispatch 2024-09-21T11:39:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22"} : dispatch 2024-09-21T11:39:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[36454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22"}]': finished 2024-09-21T11:39:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[36454]: osdmap e50: 8 total, 7 up, 8 in 2024-09-21T11:39:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/1663771807' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:39:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[29710]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/2829274315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22"} : dispatch 2024-09-21T11:39:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22"} : dispatch 2024-09-21T11:39:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[29710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22"}]': finished 2024-09-21T11:39:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[29710]: osdmap e50: 8 total, 7 up, 8 in 2024-09-21T11:39:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:27.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:27 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/1663771807' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:39:27.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:27 smithi175 ceph-mon[32613]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:27 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/2829274315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22"} : dispatch 2024-09-21T11:39:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:27 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22"} : dispatch 2024-09-21T11:39:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:27 smithi175 ceph-mon[32613]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22"}]': finished 2024-09-21T11:39:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:27 smithi175 ceph-mon[32613]: osdmap e50: 8 total, 7 up, 8 in 2024-09-21T11:39:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:27 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:27 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/1663771807' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T11:39:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:29 smithi043 ceph-mon[29710]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T11:39:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:29 smithi043 ceph-mon[36454]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T11:39:29.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:29 smithi175 ceph-mon[32613]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T11:39:31.634 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:31 smithi175 ceph-mon[32613]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T11:39:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:31 smithi043 ceph-mon[36454]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T11:39:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:31 smithi043 ceph-mon[29710]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T11:39:33.494 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:33 smithi175 ceph-mon[32613]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:39:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:33 smithi043 ceph-mon[36454]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:39:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:33 smithi043 ceph-mon[29710]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:39:34.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:34 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T11:39:34.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:34 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:34.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:34 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T11:39:34.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:34 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:34.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:34 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T11:39:34.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:34 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:35.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:35 smithi175 ceph-mon[32613]: Deploying daemon osd.7 on smithi175 2024-09-21T11:39:35.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:35 smithi175 ceph-mon[32613]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:39:35.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:35 smithi043 ceph-mon[36454]: Deploying daemon osd.7 on smithi175 2024-09-21T11:39:35.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:35 smithi043 ceph-mon[36454]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:39:35.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:35 smithi043 ceph-mon[29710]: Deploying daemon osd.7 on smithi175 2024-09-21T11:39:35.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:35 smithi043 ceph-mon[29710]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T11:39:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:37 smithi043 ceph-mon[36454]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:37 smithi043 ceph-mon[29710]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:37 smithi175 ceph-mon[32613]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:39 smithi043 ceph-mon[36454]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:39 smithi043 ceph-mon[29710]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:39.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:39 smithi175 ceph-mon[32613]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:41 smithi043 ceph-mon[29710]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:41.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:41 smithi043 ceph-mon[36454]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:41.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:41 smithi175 ceph-mon[32613]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:42.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:42 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:42.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:42 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:42.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:42 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:42 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:42 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:42 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:42.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:42.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:42 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:43.539 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:43 smithi175 ceph-mon[32613]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:43 smithi043 ceph-mon[29710]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:43 smithi043 ceph-mon[36454]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:45.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[36454]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:45.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:39:45.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:39:45.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:45.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:45.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[36454]: osdmap e51: 8 total, 7 up, 8 in 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[29710]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:45.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[29710]: osdmap e51: 8 total, 7 up, 8 in 2024-09-21T11:39:45.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:45 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:45.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:45 smithi175 ceph-mon[32613]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:45.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:45 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:39:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:45 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:39:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:45 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:45 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:45 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:45 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:45 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:45 smithi175 ceph-mon[32613]: osdmap e51: 8 total, 7 up, 8 in 2024-09-21T11:39:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:45 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:47.067 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 7 on host 'smithi175' 2024-09-21T11:39:47.393 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:47 smithi175 ceph-mon[32613]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:47.393 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:47 smithi175 ceph-mon[32613]: from='osd.7 v2:172.21.15.175:6812/3676713275' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:39:47.394 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:47 smithi175 ceph-mon[32613]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:39:47.394 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:47.394 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:47.394 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:47 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:47.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[29710]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:47.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[29710]: from='osd.7 v2:172.21.15.175:6812/3676713275' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:39:47.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[29710]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:39:47.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:47.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:47.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:47.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[36454]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:47.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[36454]: from='osd.7 v2:172.21.15.175:6812/3676713275' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:39:47.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[36454]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T11:39:47.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:47.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:47.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:47 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:47.695 DEBUG:teuthology.orchestra.run.smithi175:osd.7> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.7.service 2024-09-21T11:39:47.697 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-21T11:39:47.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd stat -f json 2024-09-21T11:39:47.957 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:39:48.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:48 smithi043 ceph-mon[36454]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T11:39:48.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:48 smithi043 ceph-mon[36454]: osdmap e52: 8 total, 7 up, 8 in 2024-09-21T11:39:48.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:48 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:48.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:48 smithi043 ceph-mon[36454]: from='osd.7 v2:172.21.15.175:6812/3676713275' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:48.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:48 smithi043 ceph-mon[36454]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:48.329 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:48 smithi043 ceph-mon[29710]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T11:39:48.329 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:48 smithi043 ceph-mon[29710]: osdmap e52: 8 total, 7 up, 8 in 2024-09-21T11:39:48.329 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:48 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:48.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:48 smithi043 ceph-mon[29710]: from='osd.7 v2:172.21.15.175:6812/3676713275' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:48.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:48 smithi043 ceph-mon[29710]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:48 smithi175 ceph-mon[32613]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T11:39:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:48 smithi175 ceph-mon[32613]: osdmap e52: 8 total, 7 up, 8 in 2024-09-21T11:39:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:48 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:48 smithi175 ceph-mon[32613]: from='osd.7 v2:172.21.15.175:6812/3676713275' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:48 smithi175 ceph-mon[32613]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T11:39:48.608 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:39:48.842 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 11:39:48 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T11:39:48.655+0000 7efcb87cb640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T11:39:48.842 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 11:39:48 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T11:39:48.660+0000 7efcb05ea640 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T11:39:49.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:49 smithi175 ceph-mon[32613]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:49.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:49 smithi175 ceph-mon[32613]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:39:49.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:49 smithi175 ceph-mon[32613]: osdmap e53: 8 total, 7 up, 8 in 2024-09-21T11:39:49.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:49.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:49 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:49.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1794805720' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:39:49.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:49 smithi175 ceph-mon[32613]: from='osd.7 ' entity='osd.7' 2024-09-21T11:39:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[36454]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[36454]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:39:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[36454]: osdmap e53: 8 total, 7 up, 8 in 2024-09-21T11:39:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1794805720' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:39:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[36454]: from='osd.7 ' entity='osd.7' 2024-09-21T11:39:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[29710]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-21T11:39:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[29710]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-21T11:39:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[29710]: osdmap e53: 8 total, 7 up, 8 in 2024-09-21T11:39:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1794805720' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:39:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:49 smithi043 ceph-mon[29710]: from='osd.7 ' entity='osd.7' 2024-09-21T11:39:49.554 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":53,"num_osds":8,"num_up_osds":7,"osd_up_since":1726918759,"num_in_osds":8,"osd_in_since":1726918766,"num_remapped_pgs":0} 2024-09-21T11:39:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:39:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:39:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[29710]: osd.7 v2:172.21.15.175:6812/3676713275 boot 2024-09-21T11:39:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[29710]: osdmap e54: 8 total, 8 up, 8 in 2024-09-21T11:39:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:39:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:39:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:39:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:39:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[36454]: osd.7 v2:172.21.15.175:6812/3676713275 boot 2024-09-21T11:39:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[36454]: osdmap e54: 8 total, 8 up, 8 in 2024-09-21T11:39:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:39:50.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:50 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:39:50.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:50 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:39:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:50 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:39:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:50 smithi175 ceph-mon[32613]: osd.7 v2:172.21.15.175:6812/3676713275 boot 2024-09-21T11:39:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:50 smithi175 ceph-mon[32613]: osdmap e54: 8 total, 8 up, 8 in 2024-09-21T11:39:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:50 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:39:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:50 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:39:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:50 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:39:50.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd stat -f json 2024-09-21T11:39:50.831 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:39:51.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 356 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:51.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: osdmap e55: 8 total, 8 up, 8 in 2024-09-21T11:39:51.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-21T11:39:51.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 356 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: osdmap e55: 8 total, 8 up, 8 in 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-21T11:39:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:51.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:51.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:51 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 356 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: osdmap e55: 8 total, 8 up, 8 in 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:51 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:51.624 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:39:52.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:52 smithi043 ceph-mon[29710]: Detected new or changed devices on smithi175 2024-09-21T11:39:52.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:52 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi175 to 4272M 2024-09-21T11:39:52.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:52 smithi043 ceph-mon[29710]: osdmap e56: 8 total, 8 up, 8 in 2024-09-21T11:39:52.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:52 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3255279671' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:39:52.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:52 smithi043 ceph-mon[36454]: Detected new or changed devices on smithi175 2024-09-21T11:39:52.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:52 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi175 to 4272M 2024-09-21T11:39:52.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:52 smithi043 ceph-mon[36454]: osdmap e56: 8 total, 8 up, 8 in 2024-09-21T11:39:52.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:52 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3255279671' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:39:52.434 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1726918789,"num_in_osds":8,"osd_in_since":1726918766,"num_remapped_pgs":0} 2024-09-21T11:39:52.434 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump --format=json 2024-09-21T11:39:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:52 smithi175 ceph-mon[32613]: Detected new or changed devices on smithi175 2024-09-21T11:39:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:52 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi175 to 4272M 2024-09-21T11:39:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:52 smithi175 ceph-mon[32613]: osdmap e56: 8 total, 8 up, 8 in 2024-09-21T11:39:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:52 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3255279671' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T11:39:52.696 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:39:53.402 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:39:53.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:53 smithi043 ceph-mon[29710]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 356 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:53.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:53 smithi043 ceph-mon[36454]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 356 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:53.404 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":56,"fsid":"4f2c5252-780d-11ef-baf6-efdc52797490","created":"2024-09-21T11:34:07.249195+0000","modified":"2024-09-21T11:39:51.216564+0000","last_up_change":"2024-09-21T11:39:49.206441+0000","last_in_change":"2024-09-21T11:39:26.485131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T11:37:26.023430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5229029d-4fd6-46b6-8325-a3e004df09b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":817279193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":817279193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":817279193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6803","nonce":817279193}]},"public_addr":"172.21.15.43:6801/817279193","cluster_addr":"172.21.15.43:6802/817279193","heartbeat_back_addr":"172.21.15.43:6804/817279193","heartbeat_front_addr":"172.21.15.43:6803/817279193","state":["exists","up"]},{"osd":1,"uuid":"a2a7b58b-ba99-4383-a6d3-3b19388b2607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":263756171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":263756171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":263756171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6807","nonce":263756171}]},"public_addr":"172.21.15.43:6805/263756171","cluster_addr":"172.21.15.43:6806/263756171","heartbeat_back_addr":"172.21.15.43:6808/263756171","heartbeat_front_addr":"172.21.15.43:6807/263756171","state":["exists","up"]},{"osd":2,"uuid":"c24a60b5-5713-492b-82ee-576ea7a743ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6809","nonce":687577863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":687577863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":687577863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":687577863}]},"public_addr":"172.21.15.43:6809/687577863","cluster_addr":"172.21.15.43:6810/687577863","heartbeat_back_addr":"172.21.15.43:6812/687577863","heartbeat_front_addr":"172.21.15.43:6811/687577863","state":["exists","up"]},{"osd":3,"uuid":"c5d43cf8-1c72-4d7d-ae41-e8136b47177d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":129493158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":129493158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":129493158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":129493158}]},"public_addr":"172.21.15.43:6813/129493158","cluster_addr":"172.21.15.43:6814/129493158","heartbeat_back_addr":"172.21.15.43:6816/129493158","heartbeat_front_addr":"172.21.15.43:6815/129493158","state":["exists","up"]},{"osd":4,"uuid":"79ee85c5-afdc-4d68-a253-89b7b0eb652c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1323992640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1323992640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1323992640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1323992640}]},"public_addr":"172.21.15.175:6800/1323992640","cluster_addr":"172.21.15.175:6801/1323992640","heartbeat_back_addr":"172.21.15.175:6803/1323992640","heartbeat_front_addr":"172.21.15.175:6802/1323992640","state":["exists","up"]},{"osd":5,"uuid":"4d647321-71a1-412b-bbf5-921441923a65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1163208458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":1163208458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":1163208458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1163208458}]},"public_addr":"172.21.15.175:6804/1163208458","cluster_addr":"172.21.15.175:6805/1163208458","heartbeat_back_addr":"172.21.15.175:6807/1163208458","heartbeat_front_addr":"172.21.15.175:6806/1163208458","state":["exists","up"]},{"osd":6,"uuid":"3c33bea7-94b5-46c1-9905-81b4612a2354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":932395625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":932395625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":932395625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":932395625}]},"public_addr":"172.21.15.175:6808/932395625","cluster_addr":"172.21.15.175:6809/932395625","heartbeat_back_addr":"172.21.15.175:6811/932395625","heartbeat_front_addr":"172.21.15.175:6810/932395625","state":["exists","up"]},{"osd":7,"uuid":"41bc9817-d9d1-4c11-94ff-7de332af6d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3676713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":3676713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":3676713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3676713275}]},"public_addr":"172.21.15.175:6812/3676713275","cluster_addr":"172.21.15.175:6813/3676713275","heartbeat_back_addr":"172.21.15.175:6815/3676713275","heartbeat_front_addr":"172.21.15.175:6814/3676713275","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:14.937500+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:44.194175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:38:47.503019+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:08.234118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:49.521781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:38:48.570111+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:17.543517+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:47.269592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/3345448895":"2024-09-22T11:35:03.554546+0000","172.21.15.43:6800/1513342881":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/4225358170":"2024-09-22T11:35:03.554546+0000","172.21.15.43:6800/2583179759":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/911630798":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/3115176321":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/2518306973":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/3249046048":"2024-09-22T11:34:37.015137+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T11:39:53.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:53 smithi175 ceph-mon[32613]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 356 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:54.115 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T11:37:26.023430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T11:39:54.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T11:39:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:39:54.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:54 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3016605424' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:39:54.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:54 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3016605424' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:39:54.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:54 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3016605424' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:39:55.044 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-09-21T11:39:55.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:55 smithi043 ceph-mon[29710]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 356 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:55.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:55 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/468624088' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T11:39:55.406 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:55 smithi043 ceph-mon[36454]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 356 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:55.406 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:55 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/468624088' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T11:39:55.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:55 smithi175 ceph-mon[32613]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 356 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:55.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:55 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/468624088' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T11:39:55.850 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi043 2024-09-21T11:39:55.850 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch apply rgw foo.a --placement '1;smithi043=foo.a' 2024-09-21T11:39:56.110 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:39:56.476 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:56 smithi175 ceph-mon[32613]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-21T11:39:56.476 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:56 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:39:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:56 smithi043 ceph-mon[29710]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-21T11:39:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:56 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:39:56.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:56 smithi043 ceph-mon[36454]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-21T11:39:56.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:56 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:39:56.765 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled rgw.foo.a update... 2024-09-21T11:39:57.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:57 smithi175 ceph-mon[32613]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:57.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:57.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:57.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:57.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:57.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:57.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-21T11:39:57.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-21T11:39:57.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:57.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:57 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:57.437 DEBUG:teuthology.orchestra.run.smithi043:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@rgw.foo.a.service 2024-09-21T11:39:57.439 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi175 2024-09-21T11:39:57.439 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool create datapool 3 3 replicated 2024-09-21T11:39:57.460 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[36454]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:57.460 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:57.460 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:57.460 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:57.460 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:57.461 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:57.461 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-21T11:39:57.461 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-21T11:39:57.461 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:57.461 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:57.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[29710]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:39:57.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:57.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:39:57.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:57.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:39:57.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:57.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-21T11:39:57.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-21T11:39:57.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:39:57.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:57 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:39:57.684 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:39:58.303 INFO:teuthology.orchestra.run.smithi175.stderr:pool 'datapool' created 2024-09-21T11:39:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:58 smithi043 ceph-mon[29710]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:39:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:58 smithi043 ceph-mon[29710]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-21T11:39:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:58 smithi043 ceph-mon[29710]: Deploying daemon rgw.foo.a on smithi043 2024-09-21T11:39:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:58 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/1148033648' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:39:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:58 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:39:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:58 smithi043 ceph-mon[36454]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:39:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:58 smithi043 ceph-mon[36454]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-21T11:39:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:58 smithi043 ceph-mon[36454]: Deploying daemon rgw.foo.a on smithi043 2024-09-21T11:39:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:58 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/1148033648' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:39:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:58 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:39:58.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:58 smithi175 ceph-mon[32613]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:39:58.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:58 smithi175 ceph-mon[32613]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-21T11:39:58.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:58 smithi175 ceph-mon[32613]: Deploying daemon rgw.foo.a on smithi043 2024-09-21T11:39:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:58 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/1148033648' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:39:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:58 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-21T11:39:58.934 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 11:39:58 smithi043 systemd[1]: Starting Ceph rgw.foo.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:39:58.951 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- rbd pool init datapool 2024-09-21T11:39:59.186 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 11:39:58 smithi043 podman[60705]: 2024-09-21 11:39:58.932483479 +0000 UTC m=+0.112958860 container create 0c9e2ed7972df881964939d2d448a10f70b08382ff520e50a1dd87ccd51f1b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-rgw-foo-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T11:39:59.186 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 11:39:58 smithi043 podman[60705]: 2024-09-21 11:39:58.835922788 +0000 UTC m=+0.016398140 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T11:39:59.186 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 11:39:59 smithi043 podman[60705]: 2024-09-21 11:39:59.185045691 +0000 UTC m=+0.365521038 container init 0c9e2ed7972df881964939d2d448a10f70b08382ff520e50a1dd87ccd51f1b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-rgw-foo-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T11:39:59.197 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:39:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:59 smithi043 ceph-mon[36454]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:39:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:59 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-21T11:39:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:39:59 smithi043 ceph-mon[36454]: osdmap e57: 8 total, 8 up, 8 in 2024-09-21T11:39:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:59 smithi043 ceph-mon[29710]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:39:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:59 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-21T11:39:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:39:59 smithi043 ceph-mon[29710]: osdmap e57: 8 total, 8 up, 8 in 2024-09-21T11:39:59.492 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 11:39:59 smithi043 podman[60705]: 2024-09-21 11:39:59.18815492 +0000 UTC m=+0.368630264 container start 0c9e2ed7972df881964939d2d448a10f70b08382ff520e50a1dd87ccd51f1b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T11:39:59.492 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 11:39:59 smithi043 bash[60705]: 0c9e2ed7972df881964939d2d448a10f70b08382ff520e50a1dd87ccd51f1b5f 2024-09-21T11:39:59.492 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 11:39:59 smithi043 systemd[1]: Started Ceph rgw.foo.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:39:59.590 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:59 smithi175 ceph-mon[32613]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:39:59.590 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:59 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-21T11:39:59.590 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:39:59 smithi175 ceph-mon[32613]: osdmap e57: 8 total, 8 up, 8 in 2024-09-21T11:40:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: osdmap e58: 8 total, 8 up, 8 in 2024-09-21T11:40:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1706821443' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-21T11:40:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/279999937' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: overall HEALTH_OK 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1706821443' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[29710]: osdmap e59: 8 total, 8 up, 8 in 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: osdmap e58: 8 total, 8 up, 8 in 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1706821443' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-21T11:40:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/279999937' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: overall HEALTH_OK 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1706821443' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-21T11:40:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:00 smithi043 ceph-mon[36454]: osdmap e59: 8 total, 8 up, 8 in 2024-09-21T11:40:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: osdmap e58: 8 total, 8 up, 8 in 2024-09-21T11:40:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1706821443' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-21T11:40:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-09-21T11:40:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:40:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/279999937' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:40:00.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-21T11:40:00.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: overall HEALTH_OK 2024-09-21T11:40:00.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1706821443' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-21T11:40:00.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-21T11:40:00.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:00 smithi175 ceph-mon[32613]: osdmap e59: 8 total, 8 up, 8 in 2024-09-21T11:40:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[36454]: pgmap v196: 36 pgs: 9 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:40:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[36454]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:40:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:40:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:40:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[36454]: osdmap e60: 8 total, 8 up, 8 in 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[29710]: pgmap v196: 36 pgs: 9 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[29710]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[29710]: osdmap e60: 8 total, 8 up, 8 in 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:40:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:40:01.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:01 smithi175 ceph-mon[32613]: pgmap v196: 36 pgs: 9 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T11:40:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:01 smithi175 ceph-mon[32613]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:40:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:01 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:01 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:01 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:40:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:01 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:40:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:01 smithi175 ceph-mon[32613]: osdmap e60: 8 total, 8 up, 8 in 2024-09-21T11:40:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:40:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-21T11:40:02.642 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:02 smithi175 ceph-mon[32613]: Checking dashboard <-> RGW credentials 2024-09-21T11:40:02.642 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:40:02.642 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:40:02.642 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:02 smithi175 ceph-mon[32613]: osdmap e61: 8 total, 8 up, 8 in 2024-09-21T11:40:02.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:02 smithi043 ceph-mon[29710]: Checking dashboard <-> RGW credentials 2024-09-21T11:40:02.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:40:02.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:40:02.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:02 smithi043 ceph-mon[29710]: osdmap e61: 8 total, 8 up, 8 in 2024-09-21T11:40:02.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:02 smithi043 ceph-mon[36454]: Checking dashboard <-> RGW credentials 2024-09-21T11:40:02.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:40:02.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-21T11:40:02.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:02 smithi043 ceph-mon[36454]: osdmap e61: 8 total, 8 up, 8 in 2024-09-21T11:40:02.945 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.175 --placement '1;smithi175=iscsi.a' 2024-09-21T11:40:03.196 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:40:03.627 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:03 smithi175 ceph-mon[32613]: pgmap v199: 68 pgs: 14 creating+peering, 9 active+clean, 45 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:03.627 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:03 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:40:03.627 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:03 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:40:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:03 smithi043 ceph-mon[29710]: pgmap v199: 68 pgs: 14 creating+peering, 9 active+clean, 45 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:03 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:40:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:03 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:40:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:03 smithi043 ceph-mon[36454]: pgmap v199: 68 pgs: 14 creating+peering, 9 active+clean, 45 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:03 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:40:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:03 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:40:03.852 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled iscsi.datapool update... 2024-09-21T11:40:04.456 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:04 smithi175 ceph-mon[32613]: osdmap e62: 8 total, 8 up, 8 in 2024-09-21T11:40:04.456 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:04 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:40:04.456 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:04 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:40:04.456 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:04 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:04.456 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:04 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:04.456 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:04 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:40:04.457 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:04 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:40:04.457 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:04 smithi175 ceph-mon[32613]: osdmap e63: 8 total, 8 up, 8 in 2024-09-21T11:40:04.483 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-21T11:40:04.484 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:40:04.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-21T11:40:04.514 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:40:04.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-21T11:40:04.541 DEBUG:teuthology.orchestra.run.smithi175:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@iscsi.iscsi.a.service 2024-09-21T11:40:04.583 INFO:tasks.cephadm:Adding prometheus.a on smithi175 2024-09-21T11:40:04.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch apply prometheus '1;smithi175=a' 2024-09-21T11:40:04.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[29710]: osdmap e62: 8 total, 8 up, 8 in 2024-09-21T11:40:04.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:40:04.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:40:04.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[29710]: osdmap e63: 8 total, 8 up, 8 in 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[36454]: osdmap e62: 8 total, 8 up, 8 in 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:40:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-21T11:40:04.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:04 smithi043 ceph-mon[36454]: osdmap e63: 8 total, 8 up, 8 in 2024-09-21T11:40:04.867 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:40:05.518 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled prometheus update... 2024-09-21T11:40:05.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:05 smithi043 ceph-mon[29710]: pgmap v202: 100 pgs: 23 creating+peering, 13 active+clean, 64 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:40:05.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:05 smithi043 ceph-mon[29710]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.175", "placement": "1;smithi175=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:05.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:05 smithi043 ceph-mon[29710]: Saving service iscsi.datapool spec with placement smithi175=iscsi.a;count:1 2024-09-21T11:40:05.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:05 smithi043 ceph-mon[29710]: osdmap e64: 8 total, 8 up, 8 in 2024-09-21T11:40:05.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:05 smithi043 ceph-mon[36454]: pgmap v202: 100 pgs: 23 creating+peering, 13 active+clean, 64 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:40:05.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:05 smithi043 ceph-mon[36454]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.175", "placement": "1;smithi175=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:05.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:05 smithi043 ceph-mon[36454]: Saving service iscsi.datapool spec with placement smithi175=iscsi.a;count:1 2024-09-21T11:40:05.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:05 smithi043 ceph-mon[36454]: osdmap e64: 8 total, 8 up, 8 in 2024-09-21T11:40:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:05 smithi175 ceph-mon[32613]: pgmap v202: 100 pgs: 23 creating+peering, 13 active+clean, 64 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-21T11:40:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:05 smithi175 ceph-mon[32613]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.175", "placement": "1;smithi175=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:05 smithi175 ceph-mon[32613]: Saving service iscsi.datapool spec with placement smithi175=iscsi.a;count:1 2024-09-21T11:40:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:05 smithi175 ceph-mon[32613]: osdmap e64: 8 total, 8 up, 8 in 2024-09-21T11:40:06.124 DEBUG:teuthology.orchestra.run.smithi175:prometheus.a> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@prometheus.a.service 2024-09-21T11:40:06.127 INFO:tasks.cephadm:Adding node-exporter.a on smithi043 2024-09-21T11:40:06.127 INFO:tasks.cephadm:Adding node-exporter.b on smithi175 2024-09-21T11:40:06.127 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch apply node-exporter '2;smithi043=a;smithi175=b' 2024-09-21T11:40:06.414 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:40:06.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:06 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:40:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:06 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:40:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:06 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:06 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:40:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:06 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:40:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:06 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:06 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:40:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:06 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-21T11:40:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:06 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:07.089 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled node-exporter update... 2024-09-21T11:40:07.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:07 smithi175 ceph-mon[32613]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:07.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:07 smithi175 ceph-mon[32613]: Saving service prometheus spec with placement smithi175=a;count:1 2024-09-21T11:40:07.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:07 smithi175 ceph-mon[32613]: pgmap v205: 132 pgs: 20 creating+peering, 59 active+clean, 53 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-21T11:40:07.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:40:07.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:40:07.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:07 smithi175 ceph-mon[32613]: osdmap e65: 8 total, 8 up, 8 in 2024-09-21T11:40:07.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:40:07.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:40:07.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:07 smithi175 ceph-mon[32613]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:40:07.721 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:07 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[36454]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[36454]: Saving service prometheus spec with placement smithi175=a;count:1 2024-09-21T11:40:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[36454]: pgmap v205: 132 pgs: 20 creating+peering, 59 active+clean, 53 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-21T11:40:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:40:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:40:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[36454]: osdmap e65: 8 total, 8 up, 8 in 2024-09-21T11:40:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:40:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:40:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[36454]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:40:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[29710]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[29710]: Saving service prometheus spec with placement smithi175=a;count:1 2024-09-21T11:40:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[29710]: pgmap v205: 132 pgs: 20 creating+peering, 59 active+clean, 53 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-21T11:40:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:40:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-21T11:40:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[29710]: osdmap e65: 8 total, 8 up, 8 in 2024-09-21T11:40:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:40:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-21T11:40:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[29710]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:40:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:07 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:07.748 DEBUG:teuthology.orchestra.run.smithi043:node-exporter.a> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@node-exporter.a.service 2024-09-21T11:40:07.750 DEBUG:teuthology.orchestra.run.smithi175:node-exporter.b> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@node-exporter.b.service 2024-09-21T11:40:07.753 INFO:tasks.cephadm:Adding alertmanager.a on smithi043 2024-09-21T11:40:07.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch apply alertmanager '1;smithi043=a' 2024-09-21T11:40:08.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:08.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: Saving service node-exporter spec with placement smithi043=a;smithi175=b;count:2 2024-09-21T11:40:08.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:40:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:40:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: osdmap e66: 8 total, 8 up, 8 in 2024-09-21T11:40:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-21T11:40:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-21T11:40:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:40:08.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: Saving service node-exporter spec with placement smithi043=a;smithi175=b;count:2 2024-09-21T11:40:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:40:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:40:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: osdmap e66: 8 total, 8 up, 8 in 2024-09-21T11:40:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-21T11:40:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-21T11:40:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:08 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: Saving service node-exporter spec with placement smithi043=a;smithi175=b;count:2 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2311158182' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1749088654' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: osdmap e66: 8 total, 8 up, 8 in 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-21T11:40:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:08 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:40:09.135 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:40:09.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:09 smithi175 ceph-mon[32613]: Deploying daemon iscsi.iscsi.a on smithi175 2024-09-21T11:40:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:09 smithi175 ceph-mon[32613]: pgmap v208: 132 pgs: 20 creating+peering, 94 active+clean, 18 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-21T11:40:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:09 smithi175 ceph-mon[32613]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:40:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:09 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:40:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:09 smithi043 ceph-mon[36454]: Deploying daemon iscsi.iscsi.a on smithi175 2024-09-21T11:40:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:09 smithi043 ceph-mon[36454]: pgmap v208: 132 pgs: 20 creating+peering, 94 active+clean, 18 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-21T11:40:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:09 smithi043 ceph-mon[36454]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:40:09.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:09 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:40:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:09 smithi043 ceph-mon[29710]: Deploying daemon iscsi.iscsi.a on smithi175 2024-09-21T11:40:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:09 smithi043 ceph-mon[29710]: pgmap v208: 132 pgs: 20 creating+peering, 94 active+clean, 18 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-21T11:40:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:09 smithi043 ceph-mon[29710]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:40:09.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:09 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:40:09.906 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled alertmanager update... 2024-09-21T11:40:10.913 DEBUG:teuthology.orchestra.run.smithi043:alertmanager.a> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@alertmanager.a.service 2024-09-21T11:40:10.916 INFO:tasks.cephadm:Adding grafana.a on smithi175 2024-09-21T11:40:10.916 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph orch apply grafana '1;smithi175=a' 2024-09-21T11:40:10.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:10 smithi175 ceph-mon[32613]: pgmap v209: 132 pgs: 9 creating+peering, 119 active+clean, 4 unknown; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 4.2 KiB/s wr, 98 op/s 2024-09-21T11:40:10.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:10 smithi175 ceph-mon[32613]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:10.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:10 smithi175 ceph-mon[32613]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-09-21T11:40:10.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:10 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:10.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:10 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:10.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:10 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:10.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:10 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:10.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:10 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[36454]: pgmap v209: 132 pgs: 9 creating+peering, 119 active+clean, 4 unknown; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 4.2 KiB/s wr, 98 op/s 2024-09-21T11:40:11.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[36454]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:11.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[36454]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-09-21T11:40:11.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[29710]: pgmap v209: 132 pgs: 9 creating+peering, 119 active+clean, 4 unknown; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 4.2 KiB/s wr, 98 op/s 2024-09-21T11:40:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[29710]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[29710]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-09-21T11:40:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:10 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:11.533 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:40:12.211 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled grafana update... 2024-09-21T11:40:12.212 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:11 smithi175 ceph-mon[32613]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-21T11:40:12.212 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:11 smithi175 ceph-mon[32613]: Deploying daemon prometheus.a on smithi175 2024-09-21T11:40:12.212 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:11 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/2360273124' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T11:40:12.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:11 smithi043 ceph-mon[36454]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-21T11:40:12.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:11 smithi043 ceph-mon[36454]: Deploying daemon prometheus.a on smithi175 2024-09-21T11:40:12.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:11 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/2360273124' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T11:40:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:11 smithi043 ceph-mon[29710]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-21T11:40:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:11 smithi043 ceph-mon[29710]: Deploying daemon prometheus.a on smithi175 2024-09-21T11:40:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:11 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/2360273124' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T11:40:13.014 DEBUG:teuthology.orchestra.run.smithi175:grafana.a> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@grafana.a.service 2024-09-21T11:40:13.056 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T11:40:13.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T11:40:13.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[36454]: pgmap v210: 132 pgs: 132 active+clean; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 KiB/s wr, 134 op/s 2024-09-21T11:40:13.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[36454]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-21T11:40:13.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[36454]: osdmap e67: 8 total, 8 up, 8 in 2024-09-21T11:40:13.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[36454]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:13.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[36454]: Saving service grafana spec with placement smithi175=a;count:1 2024-09-21T11:40:13.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:13.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[29710]: pgmap v210: 132 pgs: 132 active+clean; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 KiB/s wr, 134 op/s 2024-09-21T11:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[29710]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-21T11:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[29710]: osdmap e67: 8 total, 8 up, 8 in 2024-09-21T11:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[29710]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[29710]: Saving service grafana spec with placement smithi175=a;count:1 2024-09-21T11:40:13.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:12 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:13.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:12 smithi175 ceph-mon[32613]: pgmap v210: 132 pgs: 132 active+clean; 581 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 4.5 KiB/s wr, 134 op/s 2024-09-21T11:40:13.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:12 smithi175 ceph-mon[32613]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-21T11:40:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:12 smithi175 ceph-mon[32613]: osdmap e67: 8 total, 8 up, 8 in 2024-09-21T11:40:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:12 smithi175 ceph-mon[32613]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:40:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:12 smithi175 ceph-mon[32613]: Saving service grafana spec with placement smithi175=a;count:1 2024-09-21T11:40:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:12 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:13.326 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:13 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:40:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:13 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:40:13.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:13 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:40:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:13 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:40:14.079 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-21T11:40:14.079 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQCesO5m/aSPBBAAkORO9HH1wM3+xvGtTxYtsQ== 2024-09-21T11:40:14.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:13 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:40:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:13 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:40:14.810 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-21T11:40:14.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T11:40:14.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T11:40:14.853 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T11:40:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:15 smithi043 ceph-mon[36454]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 4.9 KiB/s wr, 154 op/s 2024-09-21T11:40:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:15 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1845204506' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:40:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1845204506' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:40:15.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:15 smithi043 ceph-mon[29710]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 4.9 KiB/s wr, 154 op/s 2024-09-21T11:40:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:15 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1845204506' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:40:15.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1845204506' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:40:15.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:15 smithi175 ceph-mon[32613]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 4.9 KiB/s wr, 154 op/s 2024-09-21T11:40:15.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:15 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:15.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1845204506' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:40:15.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1845204506' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:40:15.379 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:40:16.189 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2024-09-21T11:40:16.189 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCgsO5mM431ChAA2eGD4/KBailg3evl5gl1Gg== 2024-09-21T11:40:16.985 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T11:40:16.985 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-21T11:40:16.985 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T11:40:17.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T11:40:17.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:17 smithi175 ceph-mon[32613]: pgmap v213: 132 pgs: 132 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 138 op/s 2024-09-21T11:40:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:17 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:40:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:17 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:40:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:17 smithi175 ceph-mon[32613]: osdmap e68: 8 total, 8 up, 8 in 2024-09-21T11:40:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:17 smithi175 ceph-mon[32613]: from='client.? 172.21.15.175:0/59866302' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:40:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:17 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:40:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:17 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:40:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:17 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:40:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:17 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:40:17.294 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:17.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[29710]: pgmap v213: 132 pgs: 132 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 138 op/s 2024-09-21T11:40:17.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:40:17.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:40:17.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[29710]: osdmap e68: 8 total, 8 up, 8 in 2024-09-21T11:40:17.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.175:0/59866302' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:40:17.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:40:17.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[36454]: pgmap v213: 132 pgs: 132 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.5 KiB/s wr, 138 op/s 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[36454]: osdmap e68: 8 total, 8 up, 8 in 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[36454]: from='client.? 172.21.15.175:0/59866302' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:40:17.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:17 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:40:18.008 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:40:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:18 smithi175 ceph-mon[32613]: osdmap e69: 8 total, 8 up, 8 in 2024-09-21T11:40:18.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:18 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1662412486' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T11:40:18.436 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:18 smithi043 ceph-mon[36454]: osdmap e69: 8 total, 8 up, 8 in 2024-09-21T11:40:18.436 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:18 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1662412486' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T11:40:18.436 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:18 smithi043 ceph-mon[29710]: osdmap e69: 8 total, 8 up, 8 in 2024-09-21T11:40:18.436 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:18 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1662412486' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T11:40:18.740 INFO:teuthology.orchestra.run.smithi043.stdout:[{"version":26,"timestamp":"2024-09-21T11:40:07.577569+0000","name":"","changes":[{"name":"mgr/mgr/dashboard/RGW_API_SECRET_KEY","new_value":"P7maGWDQXgLKeeNrp2ICWLP2ds12NYJNePViPYuk"}]}] 2024-09-21T11:40:18.741 INFO:tasks.ceph_manager:config epoch is 26 2024-09-21T11:40:18.741 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T11:40:18.741 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T11:40:18.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T11:40:19.001 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:19.360 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:19 smithi043 ceph-mon[36454]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 1.5 KiB/s wr, 49 op/s 2024-09-21T11:40:19.360 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:19 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:19.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:19 smithi043 ceph-mon[29710]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 1.5 KiB/s wr, 49 op/s 2024-09-21T11:40:19.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:19 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:19.405 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:19 smithi175 ceph-mon[32613]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 1.5 KiB/s wr, 49 op/s 2024-09-21T11:40:19.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:19 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:19.694 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:40:20.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:20 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:40:20.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:20 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:40:20.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:20 smithi043 ceph-mon[36454]: osdmap e70: 8 total, 8 up, 8 in 2024-09-21T11:40:20.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1905773038' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T11:40:20.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:20 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:40:20.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:20 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:40:20.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:20 smithi043 ceph-mon[29710]: osdmap e70: 8 total, 8 up, 8 in 2024-09-21T11:40:20.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1905773038' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T11:40:20.462 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":4145025121}]},"active_addr":"172.21.15.43:6800/4145025121","active_change":"2024-09-21T11:35:03.554649+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.43:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2419845767}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":347827280}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2454875216}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1113684867}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":983028036}]}]} 2024-09-21T11:40:20.467 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T11:40:20.467 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T11:40:20.467 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump --format=json 2024-09-21T11:40:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:20 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:40:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:20 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:40:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:20 smithi175 ceph-mon[32613]: osdmap e70: 8 total, 8 up, 8 in 2024-09-21T11:40:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1905773038' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T11:40:20.734 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:21.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:21 smithi043 ceph-mon[29710]: pgmap v218: 132 pgs: 132 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T11:40:21.135 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:21 smithi043 ceph-mon[36454]: pgmap v218: 132 pgs: 132 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T11:40:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:21 smithi175 ceph-mon[32613]: pgmap v218: 132 pgs: 132 active+clean; 583 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-21T11:40:21.421 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:40:21.421 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":70,"fsid":"4f2c5252-780d-11ef-baf6-efdc52797490","created":"2024-09-21T11:34:07.249195+0000","modified":"2024-09-21T11:40:19.085687+0000","last_up_change":"2024-09-21T11:39:49.206441+0000","last_in_change":"2024-09-21T11:39:26.485131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T11:37:26.023430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-21T11:39:58.262982+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-21T11:39:59.229530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-21T11:40:00.380059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-21T11:40:02.330041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-21T11:40:04.395413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5229029d-4fd6-46b6-8325-a3e004df09b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":817279193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":817279193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":817279193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6803","nonce":817279193}]},"public_addr":"172.21.15.43:6801/817279193","cluster_addr":"172.21.15.43:6802/817279193","heartbeat_back_addr":"172.21.15.43:6804/817279193","heartbeat_front_addr":"172.21.15.43:6803/817279193","state":["exists","up"]},{"osd":1,"uuid":"a2a7b58b-ba99-4383-a6d3-3b19388b2607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":263756171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":263756171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":263756171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6807","nonce":263756171}]},"public_addr":"172.21.15.43:6805/263756171","cluster_addr":"172.21.15.43:6806/263756171","heartbeat_back_addr":"172.21.15.43:6808/263756171","heartbeat_front_addr":"172.21.15.43:6807/263756171","state":["exists","up"]},{"osd":2,"uuid":"c24a60b5-5713-492b-82ee-576ea7a743ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6809","nonce":687577863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":687577863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":687577863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":687577863}]},"public_addr":"172.21.15.43:6809/687577863","cluster_addr":"172.21.15.43:6810/687577863","heartbeat_back_addr":"172.21.15.43:6812/687577863","heartbeat_front_addr":"172.21.15.43:6811/687577863","state":["exists","up"]},{"osd":3,"uuid":"c5d43cf8-1c72-4d7d-ae41-e8136b47177d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":129493158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":129493158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":129493158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":129493158}]},"public_addr":"172.21.15.43:6813/129493158","cluster_addr":"172.21.15.43:6814/129493158","heartbeat_back_addr":"172.21.15.43:6816/129493158","heartbeat_front_addr":"172.21.15.43:6815/129493158","state":["exists","up"]},{"osd":4,"uuid":"79ee85c5-afdc-4d68-a253-89b7b0eb652c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1323992640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1323992640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1323992640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1323992640}]},"public_addr":"172.21.15.175:6800/1323992640","cluster_addr":"172.21.15.175:6801/1323992640","heartbeat_back_addr":"172.21.15.175:6803/1323992640","heartbeat_front_addr":"172.21.15.175:6802/1323992640","state":["exists","up"]},{"osd":5,"uuid":"4d647321-71a1-412b-bbf5-921441923a65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1163208458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":1163208458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":1163208458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1163208458}]},"public_addr":"172.21.15.175:6804/1163208458","cluster_addr":"172.21.15.175:6805/1163208458","heartbeat_back_addr":"172.21.15.175:6807/1163208458","heartbeat_front_addr":"172.21.15.175:6806/1163208458","state":["exists","up"]},{"osd":6,"uuid":"3c33bea7-94b5-46c1-9905-81b4612a2354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":932395625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":932395625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":932395625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":932395625}]},"public_addr":"172.21.15.175:6808/932395625","cluster_addr":"172.21.15.175:6809/932395625","heartbeat_back_addr":"172.21.15.175:6811/932395625","heartbeat_front_addr":"172.21.15.175:6810/932395625","state":["exists","up"]},{"osd":7,"uuid":"41bc9817-d9d1-4c11-94ff-7de332af6d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3676713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":3676713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":3676713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3676713275}]},"public_addr":"172.21.15.175:6812/3676713275","cluster_addr":"172.21.15.175:6813/3676713275","heartbeat_back_addr":"172.21.15.175:6815/3676713275","heartbeat_front_addr":"172.21.15.175:6814/3676713275","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:40:16.631651+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:44.194175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:40:15.876008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:08.234118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:49.521781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:40:01.898137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:40:19.007842+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:47.269592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/3345448895":"2024-09-22T11:35:03.554546+0000","172.21.15.43:6800/1513342881":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/4225358170":"2024-09-22T11:35:03.554546+0000","172.21.15.43:6800/2583179759":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/911630798":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/3115176321":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/2518306973":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/3249046048":"2024-09-22T11:34:37.015137+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T11:40:22.046 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T11:40:22.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump --format=json 2024-09-21T11:40:22.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:22.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2463119360' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:40:22.151 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:22 smithi175 ceph-mon[32613]: osdmap e71: 8 total, 8 up, 8 in 2024-09-21T11:40:22.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:22.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2463119360' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:40:22.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:22 smithi043 ceph-mon[36454]: osdmap e71: 8 total, 8 up, 8 in 2024-09-21T11:40:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:22.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2463119360' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:40:22.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:22 smithi043 ceph-mon[29710]: osdmap e71: 8 total, 8 up, 8 in 2024-09-21T11:40:22.311 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:22.904 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:22 smithi175 systemd[1]: Starting Ceph prometheus.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:40:22.991 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:40:22.992 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":71,"fsid":"4f2c5252-780d-11ef-baf6-efdc52797490","created":"2024-09-21T11:34:07.249195+0000","modified":"2024-09-21T11:40:22.081314+0000","last_up_change":"2024-09-21T11:39:49.206441+0000","last_in_change":"2024-09-21T11:39:26.485131+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T11:37:26.023430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-21T11:39:58.262982+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-21T11:39:59.229530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-21T11:40:00.380059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-21T11:40:02.330041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-21T11:40:04.395413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5229029d-4fd6-46b6-8325-a3e004df09b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":817279193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":817279193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":817279193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6803","nonce":817279193}]},"public_addr":"172.21.15.43:6801/817279193","cluster_addr":"172.21.15.43:6802/817279193","heartbeat_back_addr":"172.21.15.43:6804/817279193","heartbeat_front_addr":"172.21.15.43:6803/817279193","state":["exists","up"]},{"osd":1,"uuid":"a2a7b58b-ba99-4383-a6d3-3b19388b2607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":263756171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":263756171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":263756171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6807","nonce":263756171}]},"public_addr":"172.21.15.43:6805/263756171","cluster_addr":"172.21.15.43:6806/263756171","heartbeat_back_addr":"172.21.15.43:6808/263756171","heartbeat_front_addr":"172.21.15.43:6807/263756171","state":["exists","up"]},{"osd":2,"uuid":"c24a60b5-5713-492b-82ee-576ea7a743ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6809","nonce":687577863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":687577863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":687577863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":687577863}]},"public_addr":"172.21.15.43:6809/687577863","cluster_addr":"172.21.15.43:6810/687577863","heartbeat_back_addr":"172.21.15.43:6812/687577863","heartbeat_front_addr":"172.21.15.43:6811/687577863","state":["exists","up"]},{"osd":3,"uuid":"c5d43cf8-1c72-4d7d-ae41-e8136b47177d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":129493158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":129493158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":129493158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":129493158}]},"public_addr":"172.21.15.43:6813/129493158","cluster_addr":"172.21.15.43:6814/129493158","heartbeat_back_addr":"172.21.15.43:6816/129493158","heartbeat_front_addr":"172.21.15.43:6815/129493158","state":["exists","up"]},{"osd":4,"uuid":"79ee85c5-afdc-4d68-a253-89b7b0eb652c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1323992640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1323992640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1323992640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1323992640}]},"public_addr":"172.21.15.175:6800/1323992640","cluster_addr":"172.21.15.175:6801/1323992640","heartbeat_back_addr":"172.21.15.175:6803/1323992640","heartbeat_front_addr":"172.21.15.175:6802/1323992640","state":["exists","up"]},{"osd":5,"uuid":"4d647321-71a1-412b-bbf5-921441923a65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1163208458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":1163208458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":1163208458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1163208458}]},"public_addr":"172.21.15.175:6804/1163208458","cluster_addr":"172.21.15.175:6805/1163208458","heartbeat_back_addr":"172.21.15.175:6807/1163208458","heartbeat_front_addr":"172.21.15.175:6806/1163208458","state":["exists","up"]},{"osd":6,"uuid":"3c33bea7-94b5-46c1-9905-81b4612a2354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":932395625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":932395625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":932395625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":932395625}]},"public_addr":"172.21.15.175:6808/932395625","cluster_addr":"172.21.15.175:6809/932395625","heartbeat_back_addr":"172.21.15.175:6811/932395625","heartbeat_front_addr":"172.21.15.175:6810/932395625","state":["exists","up"]},{"osd":7,"uuid":"41bc9817-d9d1-4c11-94ff-7de332af6d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3676713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":3676713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":3676713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3676713275}]},"public_addr":"172.21.15.175:6812/3676713275","cluster_addr":"172.21.15.175:6813/3676713275","heartbeat_back_addr":"172.21.15.175:6815/3676713275","heartbeat_front_addr":"172.21.15.175:6814/3676713275","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:40:16.631651+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:44.194175+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:40:15.876008+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:08.234118+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:49.521781+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:40:01.898137+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:40:19.007842+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T11:39:47.269592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/3345448895":"2024-09-22T11:35:03.554546+0000","172.21.15.43:6800/1513342881":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/4225358170":"2024-09-22T11:35:03.554546+0000","172.21.15.43:6800/2583179759":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/911630798":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/3115176321":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/2518306973":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/3249046048":"2024-09-22T11:34:37.015137+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T11:40:23.156 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-mon[32613]: pgmap v219: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:40:23.156 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1360886023' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:40:23.156 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:22 smithi175 podman[59605]: 2024-09-21 11:40:22.806140581 +0000 UTC m=+0.016437800 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-21T11:40:23.156 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:22 smithi175 podman[59605]: 2024-09-21 11:40:22.926770499 +0000 UTC m=+0.137067703 container create 0082b47ebe2bedd2f3f5989a604a6cf98346871d8600a15bf87f35515e349a9f (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:40:23.156 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 podman[59605]: 2024-09-21 11:40:23.128744948 +0000 UTC m=+0.339042162 container init 0082b47ebe2bedd2f3f5989a604a6cf98346871d8600a15bf87f35515e349a9f (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:40:23.157 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 podman[59605]: 2024-09-21 11:40:23.131537884 +0000 UTC m=+0.341835083 container start 0082b47ebe2bedd2f3f5989a604a6cf98346871d8600a15bf87f35515e349a9f (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:40:23.157 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.151Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-21T11:40:23.157 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.151Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-21T11:40:23.157 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.151Z caller=main.go:623 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi175 (none))" 2024-09-21T11:40:23.157 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.151Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-21T11:40:23.157 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.151Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-21T11:40:23.157 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.152Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-21T11:40:23.157 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.153Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-21T11:40:23.157 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.154Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-21T11:40:23.157 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.154Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-21T11:40:23.371 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:23 smithi043 ceph-mon[36454]: pgmap v219: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:40:23.371 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1360886023' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:40:23.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:23 smithi043 ceph-mon[29710]: pgmap v219: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:40:23.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1360886023' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T11:40:23.512 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.156Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-21T11:40:23.584 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.156Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.949µs 2024-09-21T11:40:23.584 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.156Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-21T11:40:23.584 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.156Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-21T11:40:23.584 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.156Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=21.166µs wal_replay_duration=158.922µs wbl_replay_duration=94ns total_replay_duration=198.051µs 2024-09-21T11:40:23.584 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.158Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-21T11:40:23.584 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.158Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-21T11:40:23.585 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.158Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-21T11:40:23.585 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.173Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.662315ms db_storage=1.129µs remote_storage=1.03µs web_handler=357ns query_engine=785ns scrape=366.02µs scrape_sd=92.46µs notify=900ns notify_sd=831ns rules=14.90021ms tracing=9.388µs 2024-09-21T11:40:23.585 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.173Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-21T11:40:23.585 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:40:23.173Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-21T11:40:23.585 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 bash[59605]: 0082b47ebe2bedd2f3f5989a604a6cf98346871d8600a15bf87f35515e349a9f 2024-09-21T11:40:23.585 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:40:23 smithi175 systemd[1]: Started Ceph prometheus.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:40:23.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T11:40:23.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T11:40:23.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T11:40:23.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T11:40:23.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T11:40:23.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T11:40:23.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T11:40:23.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T11:40:24.201 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:24.202 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:24.209 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:24.210 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:24.221 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:24.568 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ignoring --setuser ceph since I am not root 2024-09-21T11:40:24.568 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ignoring --setgroup ceph since I am not root 2024-09-21T11:40:24.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-21T11:40:24.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-21T11:40:24.569 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:24 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-21T11:40:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:24 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' 2024-09-21T11:40:24.761 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:40:24 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x[33999]: ignoring --setuser ceph since I am not root 2024-09-21T11:40:24.761 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:40:24 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x[33999]: ignoring --setgroup ceph since I am not root 2024-09-21T11:40:24.761 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:40:24 smithi175 ceph-mgr[34003]: -- 172.21.15.175:0/1837137163 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642b4a6f0e0 con 0x5642b4a4c800 2024-09-21T11:40:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:25 smithi043 ceph-mon[29710]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-21T11:40:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:25 smithi043 ceph-mon[29710]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-21T11:40:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:25 smithi043 ceph-mon[36454]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-21T11:40:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:25 smithi043 ceph-mon[36454]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-21T11:40:25.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:25 smithi175 ceph-mon[32613]: from='mgr.14152 172.21.15.43:0/2330573588' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-21T11:40:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:25 smithi175 ceph-mon[32613]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-21T11:40:26.414 INFO:teuthology.orchestra.run.smithi043.stdout:231928233994 2024-09-21T11:40:26.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T11:40:28.024 INFO:teuthology.orchestra.run.smithi043.stdout:141733920794 2024-09-21T11:40:28.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T11:40:28.422 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:28 smithi043 ceph-mon[36454]: osdmap e72: 8 total, 8 up, 8 in 2024-09-21T11:40:28.422 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:28 smithi043 ceph-mon[29710]: osdmap e72: 8 total, 8 up, 8 in 2024-09-21T11:40:28.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:28 smithi175 ceph-mon[32613]: osdmap e72: 8 total, 8 up, 8 in 2024-09-21T11:40:29.486 INFO:teuthology.orchestra.run.smithi043.stdout:55834574893 2024-09-21T11:40:29.487 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T11:40:29.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-mon[32613]: Standby manager daemon x restarted 2024-09-21T11:40:29.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-mon[32613]: Standby manager daemon x started 2024-09-21T11:40:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-mon[32613]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:40:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-mon[32613]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:40:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-mon[32613]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:40:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-mon[32613]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:40:29.511 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x[33999]: [21/Sep/2024:11:40:29] ENGINE Bus STARTING 2024-09-21T11:40:29.511 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x[33999]: CherryPy Checker: 2024-09-21T11:40:29.511 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x[33999]: The Application mounted at '' has an empty config. 2024-09-21T11:40:29.511 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x[33999]: 2024-09-21T11:40:29.511 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x[33999]: [21/Sep/2024:11:40:29] ENGINE Serving on http://:::9283 2024-09-21T11:40:29.511 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 11:40:29 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x[33999]: [21/Sep/2024:11:40:29] ENGINE Bus STARTED 2024-09-21T11:40:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[29710]: Standby manager daemon x restarted 2024-09-21T11:40:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[29710]: Standby manager daemon x started 2024-09-21T11:40:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[29710]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:40:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[29710]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:40:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[29710]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:40:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[29710]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:40:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[36454]: Standby manager daemon x restarted 2024-09-21T11:40:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[36454]: Standby manager daemon x started 2024-09-21T11:40:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[36454]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-21T11:40:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[36454]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T11:40:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[36454]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-21T11:40:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:29 smithi043 ceph-mon[36454]: from='mgr.? 172.21.15.175:0/3122169864' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T11:40:29.927 INFO:teuthology.orchestra.run.smithi043.stdout:77309411366 2024-09-21T11:40:29.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T11:40:30.305 INFO:teuthology.orchestra.run.smithi043.stdout:201863462927 2024-09-21T11:40:30.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T11:40:30.524 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-21T11:40:30.524 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: Active manager daemon y restarted 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: Activating manager daemon y 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: osdmap e73: 8 total, 8 up, 8 in 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: mgrmap e20: y(active, starting, since 0.0116237s), standbys: x 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:40:30.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: Manager daemon y is now available 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:40:30.526 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: Active manager daemon y restarted 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: Activating manager daemon y 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: osdmap e73: 8 total, 8 up, 8 in 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: mgrmap e20: y(active, starting, since 0.0116237s), standbys: x 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:40:30.527 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: Manager daemon y is now available 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:40:30.528 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:40:30.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-21T11:40:30.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: Active manager daemon y restarted 2024-09-21T11:40:30.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: Activating manager daemon y 2024-09-21T11:40:30.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: osdmap e73: 8 total, 8 up, 8 in 2024-09-21T11:40:30.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: mgrmap e20: y(active, starting, since 0.0116237s), standbys: x 2024-09-21T11:40:30.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T11:40:30.714 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: Manager daemon y is now available 2024-09-21T11:40:30.715 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:40:30.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:30.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:40:30.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:40:30.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:40:30.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-21T11:40:30.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:40:30.716 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-21T11:40:30.872 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:30] ENGINE Bus STARTING 2024-09-21T11:40:30.872 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: CherryPy Checker: 2024-09-21T11:40:30.872 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: The Application mounted at '' has an empty config. 2024-09-21T11:40:30.872 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: 2024-09-21T11:40:30.872 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:30] ENGINE Serving on http://:::9283 2024-09-21T11:40:30.872 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:30 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:30] ENGINE Bus STARTED 2024-09-21T11:40:30.986 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:40:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:40:31.976 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:31 smithi175 ceph-mon[32613]: mgrmap e21: y(active, since 1.02108s), standbys: x 2024-09-21T11:40:31.977 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:31 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:40:31.977 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:31 smithi175 ceph-mon[32613]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:31.977 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:31 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:31 smithi043 ceph-mon[36454]: mgrmap e21: y(active, since 1.02108s), standbys: x 2024-09-21T11:40:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:31 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:40:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:31 smithi043 ceph-mon[36454]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:31 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:31 smithi043 ceph-mon[29710]: mgrmap e21: y(active, since 1.02108s), standbys: x 2024-09-21T11:40:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:31 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:40:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:31 smithi043 ceph-mon[29710]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:31 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:32.023 INFO:teuthology.orchestra.run.smithi043.stdout:115964117024 2024-09-21T11:40:32.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T11:40:32.691 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: 2024-09-21T11:40:32.303+0000 7fe1a3832640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:11:40:32] ENGINE Error in HTTPServer.serve 2024-09-21T11:40:32.691 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: Traceback (most recent call last): 2024-09-21T11:40:32.691 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:40:32.691 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self._connections.run(self.expiration_interval) 2024-09-21T11:40:32.691 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:40:32.691 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self._run(expiration_interval) 2024-09-21T11:40:32.691 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:40:32.691 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:40:32.691 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:40:32.691 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:40:32.692 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:40:32.692 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: s = self.context.wrap_socket( 2024-09-21T11:40:32.692 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:40:32.692 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: return self.sslsocket_class._create( 2024-09-21T11:40:32.692 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:40:32.692 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self.do_handshake() 2024-09-21T11:40:32.692 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:40:32.692 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: self._sslobj.do_handshake() 2024-09-21T11:40:32.692 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:40:32.692 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: 2024-09-21T11:40:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[36454]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-21T11:40:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[36454]: [21/Sep/2024:11:40:32] ENGINE Bus STARTING 2024-09-21T11:40:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[36454]: osdmap e74: 8 total, 8 up, 8 in 2024-09-21T11:40:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[36454]: [21/Sep/2024:11:40:32] ENGINE Serving on http://172.21.15.43:8765 2024-09-21T11:40:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[36454]: [21/Sep/2024:11:40:32] ENGINE Serving on https://172.21.15.43:7150 2024-09-21T11:40:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[36454]: [21/Sep/2024:11:40:32] ENGINE Bus STARTED 2024-09-21T11:40:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[36454]: [21/Sep/2024:11:40:32] ENGINE Error in HTTPServer.serve 2024-09-21T11:40:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout: Traceback (most recent call last): 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: self._run(expiration_interval) 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: s = self.context.wrap_socket( 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: return self.sslsocket_class._create( 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: self.do_handshake() 2024-09-21T11:40:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.c.smithi043.stdout: self._sslobj.do_handshake() 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.c.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[29710]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[29710]: [21/Sep/2024:11:40:32] ENGINE Bus STARTING 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[29710]: osdmap e74: 8 total, 8 up, 8 in 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[29710]: [21/Sep/2024:11:40:32] ENGINE Serving on http://172.21.15.43:8765 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[29710]: [21/Sep/2024:11:40:32] ENGINE Serving on https://172.21.15.43:7150 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[29710]: [21/Sep/2024:11:40:32] ENGINE Bus STARTED 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:32 smithi043 ceph-mon[29710]: [21/Sep/2024:11:40:32] ENGINE Error in HTTPServer.serve 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout: Traceback (most recent call last): 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-21T11:40:32.993 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._run(expiration_interval) 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: s = self.context.wrap_socket( 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: return self.sslsocket_class._create( 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: self.do_handshake() 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: self._sslobj.do_handshake() 2024-09-21T11:40:32.994 INFO:journalctl@ceph.mon.a.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:32 smithi175 ceph-mon[32613]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:32 smithi175 ceph-mon[32613]: [21/Sep/2024:11:40:32] ENGINE Bus STARTING 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:32 smithi175 ceph-mon[32613]: osdmap e74: 8 total, 8 up, 8 in 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:32 smithi175 ceph-mon[32613]: [21/Sep/2024:11:40:32] ENGINE Serving on http://172.21.15.43:8765 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:32 smithi175 ceph-mon[32613]: [21/Sep/2024:11:40:32] ENGINE Serving on https://172.21.15.43:7150 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:32 smithi175 ceph-mon[32613]: [21/Sep/2024:11:40:32] ENGINE Bus STARTED 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:32 smithi175 ceph-mon[32613]: [21/Sep/2024:11:40:32] ENGINE Error in HTTPServer.serve 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout: Traceback (most recent call last): 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout: self._connections.run(self.expiration_interval) 2024-09-21T11:40:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: self._run(expiration_interval) 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: s = self.context.wrap_socket( 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: return self.sslsocket_class._create( 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: self.do_handshake() 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T11:40:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout: self._sslobj.do_handshake() 2024-09-21T11:40:33.013 INFO:journalctl@ceph.mon.b.smithi175.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T11:40:33.137 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:33.174 INFO:teuthology.orchestra.run.smithi043.stdout:34359738420 2024-09-21T11:40:33.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T11:40:33.541 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:33.580 INFO:teuthology.orchestra.run.smithi043.stdout:167503724565 2024-09-21T11:40:33.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T11:40:33.823 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:33.823 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:33.827 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:33.995 INFO:teuthology.orchestra.run.smithi043.stdout:231928233995 2024-09-21T11:40:34.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.240 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:34.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:34.728 INFO:teuthology.orchestra.run.smithi043.stdout:115964117025 2024-09-21T11:40:34.994 INFO:teuthology.orchestra.run.smithi043.stdout:201863462928 2024-09-21T11:40:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:34 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi175 to 3760M 2024-09-21T11:40:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:34 smithi043 ceph-mon[29710]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3587594515' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T11:40:35.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:34 smithi043 ceph-mon[29710]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-21T11:40:35.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3532824242' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T11:40:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:34 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi175 to 3760M 2024-09-21T11:40:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:34 smithi043 ceph-mon[36454]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3587594515' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T11:40:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:34 smithi043 ceph-mon[36454]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-21T11:40:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3532824242' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T11:40:35.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:34 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi175 to 3760M 2024-09-21T11:40:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:34 smithi175 ceph-mon[32613]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3587594515' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T11:40:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:34 smithi175 ceph-mon[32613]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-21T11:40:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3532824242' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T11:40:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:40:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:40:35.943 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928233994 got 231928233995 for osd.7 2024-09-21T11:40:35.943 DEBUG:teuthology.parallel:result is None 2024-09-21T11:40:36.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:35 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/134039225' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T11:40:36.240 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:35 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/134039225' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T11:40:36.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:35 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/134039225' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T11:40:37.013 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117024 got 115964117025 for osd.3 2024-09-21T11:40:37.013 DEBUG:teuthology.parallel:result is None 2024-09-21T11:40:37.106 INFO:teuthology.orchestra.run.smithi043.stdout:34359738420 2024-09-21T11:40:37.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:36 smithi043 ceph-mon[36454]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:37.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:36 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/644774879' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T11:40:37.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:36 smithi043 ceph-mon[29710]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:37.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:36 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/644774879' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T11:40:37.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:36 smithi175 ceph-mon[32613]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T11:40:37.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:36 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/644774879' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T11:40:37.647 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:37.647 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:37.780 INFO:teuthology.orchestra.run.smithi043.stdout:141733920795 2024-09-21T11:40:38.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:37 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:40:38.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:37 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:40:38.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:37 smithi043 ceph-mon[36454]: osdmap e75: 8 total, 8 up, 8 in 2024-09-21T11:40:38.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:37 smithi043 ceph-mon[36454]: osdmap e76: 8 total, 8 up, 8 in 2024-09-21T11:40:38.006 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:37 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2483736964' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T11:40:38.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:37 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:40:38.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:37 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:40:38.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:37 smithi043 ceph-mon[29710]: osdmap e75: 8 total, 8 up, 8 in 2024-09-21T11:40:38.006 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:37 smithi043 ceph-mon[29710]: osdmap e76: 8 total, 8 up, 8 in 2024-09-21T11:40:38.007 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:37 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2483736964' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T11:40:38.021 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:38.036 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462927 got 201863462928 for osd.6 2024-09-21T11:40:38.036 DEBUG:teuthology.parallel:result is None 2024-09-21T11:40:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:37 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:40:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:37 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:40:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:37 smithi175 ceph-mon[32613]: osdmap e75: 8 total, 8 up, 8 in 2024-09-21T11:40:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:37 smithi175 ceph-mon[32613]: osdmap e76: 8 total, 8 up, 8 in 2024-09-21T11:40:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:37 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2483736964' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T11:40:38.700 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920794 got 141733920795 for osd.4 2024-09-21T11:40:38.700 DEBUG:teuthology.parallel:result is None 2024-09-21T11:40:39.390 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:39 smithi043 ceph-mon[36454]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T11:40:39.391 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:39 smithi043 ceph-mon[29710]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T11:40:39.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:39 smithi175 ceph-mon[32613]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-21T11:40:39.909 INFO:teuthology.orchestra.run.smithi043.stdout:167503724566 2024-09-21T11:40:40.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1578101892' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T11:40:40.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:40 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1578101892' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T11:40:40.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:40 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1578101892' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T11:40:40.945 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738420 got 34359738420 for osd.0 2024-09-21T11:40:40.946 DEBUG:teuthology.parallel:result is None 2024-09-21T11:40:41.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:40:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:40:41.319 INFO:teuthology.orchestra.run.smithi043.stdout:55834574895 2024-09-21T11:40:41.394 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:41 smithi043 ceph-mon[36454]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T11:40:41.394 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:41 smithi043 ceph-mon[29710]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T11:40:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:41 smithi175 ceph-mon[32613]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T11:40:41.543 INFO:teuthology.orchestra.run.smithi043.stdout:77309411368 2024-09-21T11:40:42.039 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724565 got 167503724566 for osd.5 2024-09-21T11:40:42.039 DEBUG:teuthology.parallel:result is None 2024-09-21T11:40:42.410 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:42.410 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1570480787' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T11:40:42.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2830078851' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T11:40:42.411 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:42.411 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1570480787' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T11:40:42.411 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2830078851' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T11:40:42.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:42.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1570480787' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T11:40:42.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2830078851' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T11:40:42.772 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411366 got 77309411368 for osd.2 2024-09-21T11:40:42.773 DEBUG:teuthology.parallel:result is None 2024-09-21T11:40:43.353 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574895 for osd.1 2024-09-21T11:40:43.353 DEBUG:teuthology.parallel:result is None 2024-09-21T11:40:43.353 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T11:40:43.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:40:43.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:43 smithi043 ceph-mon[29710]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T11:40:43.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:43 smithi043 ceph-mon[36454]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T11:40:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:43 smithi175 ceph-mon[32613]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T11:40:43.658 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:40:44.373 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:40:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T11:40:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi043 to 3761M 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[29710]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-21T11:40:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi043 to 3761M 2024-09-21T11:40:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:44.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:40:44.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:40:44.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-21T11:40:44.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:44 smithi043 ceph-mon[36454]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-21T11:40:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-21T11:40:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi043 to 3761M 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-21T11:40:45.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:44 smithi175 ceph-mon[32613]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-21T11:40:45.178 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-21T11:40:43.690820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":883,"num_read_kb":740,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244872,"kb_used_data":7836,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716344,"statfs":{"total":767960285184,"available":767709536256,"internally_reserved":0,"allocated":8024064,"data_stored":4238393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":186,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001718"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116965+0000","last_change":"2024-09-21T11:40:00.313497+0000","last_active":"2024-09-21T11:40:38.116965+0000","last_peered":"2024-09-21T11:40:38.116965+0000","last_clean":"2024-09-21T11:40:38.116965+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:40:38.116965+0000","last_undegraded":"2024-09-21T11:40:38.116965+0000","last_fullsized":"2024-09-21T11:40:38.116965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.636264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116915+0000","last_change":"2024-09-21T11:40:02.319143+0000","last_active":"2024-09-21T11:40:38.116915+0000","last_peered":"2024-09-21T11:40:38.116915+0000","last_clean":"2024-09-21T11:40:38.116915+0000","last_became_active":"2024-09-21T11:40:02.318639+0000","last_became_peered":"2024-09-21T11:40:02.318639+0000","last_unstale":"2024-09-21T11:40:38.116915+0000","last_undegraded":"2024-09-21T11:40:38.116915+0000","last_fullsized":"2024-09-21T11:40:38.116915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.717901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096596+0000","last_change":"2024-09-21T11:40:04.331203+0000","last_active":"2024-09-21T11:40:37.096596+0000","last_peered":"2024-09-21T11:40:37.096596+0000","last_clean":"2024-09-21T11:40:37.096596+0000","last_became_active":"2024-09-21T11:40:04.330879+0000","last_became_peered":"2024-09-21T11:40:04.330879+0000","last_unstale":"2024-09-21T11:40:37.096596+0000","last_undegraded":"2024-09-21T11:40:37.096596+0000","last_fullsized":"2024-09-21T11:40:37.096596+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.860431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117023+0000","last_change":"2024-09-21T11:40:06.346317+0000","last_active":"2024-09-21T11:40:38.117023+0000","last_peered":"2024-09-21T11:40:38.117023+0000","last_clean":"2024-09-21T11:40:38.117023+0000","last_became_active":"2024-09-21T11:40:06.345995+0000","last_became_peered":"2024-09-21T11:40:06.345995+0000","last_unstale":"2024-09-21T11:40:38.117023+0000","last_undegraded":"2024-09-21T11:40:38.117023+0000","last_fullsized":"2024-09-21T11:40:38.117023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.419437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090252+0000","last_change":"2024-09-21T11:40:06.489876+0000","last_active":"2024-09-21T11:40:37.090252+0000","last_peered":"2024-09-21T11:40:37.090252+0000","last_clean":"2024-09-21T11:40:37.090252+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:40:37.090252+0000","last_undegraded":"2024-09-21T11:40:37.090252+0000","last_fullsized":"2024-09-21T11:40:37.090252+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.939900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090828+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:40:37.090828+0000","last_peered":"2024-09-21T11:40:37.090828+0000","last_clean":"2024-09-21T11:40:37.090828+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:40:37.090828+0000","last_undegraded":"2024-09-21T11:40:37.090828+0000","last_fullsized":"2024-09-21T11:40:37.090828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089985+0000","last_change":"2024-09-21T11:40:02.315669+0000","last_active":"2024-09-21T11:40:37.089985+0000","last_peered":"2024-09-21T11:40:37.089985+0000","last_clean":"2024-09-21T11:40:37.089985+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:40:37.089985+0000","last_undegraded":"2024-09-21T11:40:37.089985+0000","last_fullsized":"2024-09-21T11:40:37.089985+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.511141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116814+0000","last_change":"2024-09-21T11:40:04.326924+0000","last_active":"2024-09-21T11:40:38.116814+0000","last_peered":"2024-09-21T11:40:38.116814+0000","last_clean":"2024-09-21T11:40:38.116814+0000","last_became_active":"2024-09-21T11:40:04.326663+0000","last_became_peered":"2024-09-21T11:40:04.326663+0000","last_unstale":"2024-09-21T11:40:38.116814+0000","last_undegraded":"2024-09-21T11:40:38.116814+0000","last_fullsized":"2024-09-21T11:40:38.116814+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:04.814146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119541+0000","last_change":"2024-09-21T11:40:00.314157+0000","last_active":"2024-09-21T11:40:38.119541+0000","last_peered":"2024-09-21T11:40:38.119541+0000","last_clean":"2024-09-21T11:40:38.119541+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:40:38.119541+0000","last_undegraded":"2024-09-21T11:40:38.119541+0000","last_fullsized":"2024-09-21T11:40:38.119541+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.659277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118645+0000","last_change":"2024-09-21T11:40:02.314170+0000","last_active":"2024-09-21T11:40:38.118645+0000","last_peered":"2024-09-21T11:40:38.118645+0000","last_clean":"2024-09-21T11:40:38.118645+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:40:38.118645+0000","last_undegraded":"2024-09-21T11:40:38.118645+0000","last_fullsized":"2024-09-21T11:40:38.118645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.670327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117406+0000","last_change":"2024-09-21T11:40:04.339616+0000","last_active":"2024-09-21T11:40:38.117406+0000","last_peered":"2024-09-21T11:40:38.117406+0000","last_clean":"2024-09-21T11:40:38.117406+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:40:38.117406+0000","last_undegraded":"2024-09-21T11:40:38.117406+0000","last_fullsized":"2024-09-21T11:40:38.117406+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.214530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116334+0000","last_change":"2024-09-21T11:40:06.344622+0000","last_active":"2024-09-21T11:40:38.116334+0000","last_peered":"2024-09-21T11:40:38.116334+0000","last_clean":"2024-09-21T11:40:38.116334+0000","last_became_active":"2024-09-21T11:40:06.344291+0000","last_became_peered":"2024-09-21T11:40:06.344291+0000","last_unstale":"2024-09-21T11:40:38.116334+0000","last_undegraded":"2024-09-21T11:40:38.116334+0000","last_fullsized":"2024-09-21T11:40:38.116334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.653671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119780+0000","last_change":"2024-09-21T11:40:00.315137+0000","last_active":"2024-09-21T11:40:38.119780+0000","last_peered":"2024-09-21T11:40:38.119780+0000","last_clean":"2024-09-21T11:40:38.119780+0000","last_became_active":"2024-09-21T11:40:00.314838+0000","last_became_peered":"2024-09-21T11:40:00.314838+0000","last_unstale":"2024-09-21T11:40:38.119780+0000","last_undegraded":"2024-09-21T11:40:38.119780+0000","last_fullsized":"2024-09-21T11:40:38.119780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:40:59.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118997+0000","last_change":"2024-09-21T11:40:02.320524+0000","last_active":"2024-09-21T11:40:38.118997+0000","last_peered":"2024-09-21T11:40:38.118997+0000","last_clean":"2024-09-21T11:40:38.118997+0000","last_became_active":"2024-09-21T11:40:02.320184+0000","last_became_peered":"2024-09-21T11:40:02.320184+0000","last_unstale":"2024-09-21T11:40:38.118997+0000","last_undegraded":"2024-09-21T11:40:38.118997+0000","last_fullsized":"2024-09-21T11:40:38.118997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.631864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117300+0000","last_change":"2024-09-21T11:40:04.333696+0000","last_active":"2024-09-21T11:40:38.117300+0000","last_peered":"2024-09-21T11:40:38.117300+0000","last_clean":"2024-09-21T11:40:38.117300+0000","last_became_active":"2024-09-21T11:40:04.333406+0000","last_became_peered":"2024-09-21T11:40:04.333406+0000","last_unstale":"2024-09-21T11:40:38.117300+0000","last_undegraded":"2024-09-21T11:40:38.117300+0000","last_fullsized":"2024-09-21T11:40:38.117300+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.055519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117137+0000","last_change":"2024-09-21T11:40:06.349798+0000","last_active":"2024-09-21T11:40:38.117137+0000","last_peered":"2024-09-21T11:40:38.117137+0000","last_clean":"2024-09-21T11:40:38.117137+0000","last_became_active":"2024-09-21T11:40:06.349538+0000","last_became_peered":"2024-09-21T11:40:06.349538+0000","last_unstale":"2024-09-21T11:40:38.117137+0000","last_undegraded":"2024-09-21T11:40:38.117137+0000","last_fullsized":"2024-09-21T11:40:38.117137+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:34.219537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118957+0000","last_change":"2024-09-21T11:40:06.487919+0000","last_active":"2024-09-21T11:40:38.118957+0000","last_peered":"2024-09-21T11:40:38.118957+0000","last_clean":"2024-09-21T11:40:38.118957+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:40:38.118957+0000","last_undegraded":"2024-09-21T11:40:38.118957+0000","last_fullsized":"2024-09-21T11:40:38.118957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:18.602419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117461+0000","last_change":"2024-09-21T11:40:00.312124+0000","last_active":"2024-09-21T11:40:38.117461+0000","last_peered":"2024-09-21T11:40:38.117461+0000","last_clean":"2024-09-21T11:40:38.117461+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:40:38.117461+0000","last_undegraded":"2024-09-21T11:40:38.117461+0000","last_fullsized":"2024-09-21T11:40:38.117461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.023370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089986+0000","last_change":"2024-09-21T11:40:02.319850+0000","last_active":"2024-09-21T11:40:37.089986+0000","last_peered":"2024-09-21T11:40:37.089986+0000","last_clean":"2024-09-21T11:40:37.089986+0000","last_became_active":"2024-09-21T11:40:02.319153+0000","last_became_peered":"2024-09-21T11:40:02.319153+0000","last_unstale":"2024-09-21T11:40:37.089986+0000","last_undegraded":"2024-09-21T11:40:37.089986+0000","last_fullsized":"2024-09-21T11:40:37.089986+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.765972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097393+0000","last_change":"2024-09-21T11:40:04.339562+0000","last_active":"2024-09-21T11:40:37.097393+0000","last_peered":"2024-09-21T11:40:37.097393+0000","last_clean":"2024-09-21T11:40:37.097393+0000","last_became_active":"2024-09-21T11:40:04.339222+0000","last_became_peered":"2024-09-21T11:40:04.339222+0000","last_unstale":"2024-09-21T11:40:37.097393+0000","last_undegraded":"2024-09-21T11:40:37.097393+0000","last_fullsized":"2024-09-21T11:40:37.097393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.022703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090832+0000","last_change":"2024-09-21T11:40:06.487933+0000","last_active":"2024-09-21T11:40:37.090832+0000","last_peered":"2024-09-21T11:40:37.090832+0000","last_clean":"2024-09-21T11:40:37.090832+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:40:37.090832+0000","last_undegraded":"2024-09-21T11:40:37.090832+0000","last_fullsized":"2024-09-21T11:40:37.090832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.209907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119180+0000","last_change":"2024-09-21T11:40:00.312534+0000","last_active":"2024-09-21T11:40:38.119180+0000","last_peered":"2024-09-21T11:40:38.119180+0000","last_clean":"2024-09-21T11:40:38.119180+0000","last_became_active":"2024-09-21T11:40:00.312239+0000","last_became_peered":"2024-09-21T11:40:00.312239+0000","last_unstale":"2024-09-21T11:40:38.119180+0000","last_undegraded":"2024-09-21T11:40:38.119180+0000","last_fullsized":"2024-09-21T11:40:38.119180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.443761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090511+0000","last_change":"2024-09-21T11:40:02.324039+0000","last_active":"2024-09-21T11:40:37.090511+0000","last_peered":"2024-09-21T11:40:37.090511+0000","last_clean":"2024-09-21T11:40:37.090511+0000","last_became_active":"2024-09-21T11:40:02.323784+0000","last_became_peered":"2024-09-21T11:40:02.323784+0000","last_unstale":"2024-09-21T11:40:37.090511+0000","last_undegraded":"2024-09-21T11:40:37.090511+0000","last_fullsized":"2024-09-21T11:40:37.090511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.707671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118544+0000","last_change":"2024-09-21T11:40:04.328865+0000","last_active":"2024-09-21T11:40:38.118544+0000","last_peered":"2024-09-21T11:40:38.118544+0000","last_clean":"2024-09-21T11:40:38.118544+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:40:38.118544+0000","last_undegraded":"2024-09-21T11:40:38.118544+0000","last_fullsized":"2024-09-21T11:40:38.118544+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.407835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118453+0000","last_change":"2024-09-21T11:40:06.340320+0000","last_active":"2024-09-21T11:40:38.118453+0000","last_peered":"2024-09-21T11:40:38.118453+0000","last_clean":"2024-09-21T11:40:38.118453+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:40:38.118453+0000","last_undegraded":"2024-09-21T11:40:38.118453+0000","last_fullsized":"2024-09-21T11:40:38.118453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.731856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096791+0000","last_change":"2024-09-21T11:40:00.308535+0000","last_active":"2024-09-21T11:40:37.096791+0000","last_peered":"2024-09-21T11:40:37.096791+0000","last_clean":"2024-09-21T11:40:37.096791+0000","last_became_active":"2024-09-21T11:40:00.308219+0000","last_became_peered":"2024-09-21T11:40:00.308219+0000","last_unstale":"2024-09-21T11:40:37.096791+0000","last_undegraded":"2024-09-21T11:40:37.096791+0000","last_fullsized":"2024-09-21T11:40:37.096791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.036879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116981+0000","last_change":"2024-09-21T11:40:02.400244+0000","last_active":"2024-09-21T11:40:38.116981+0000","last_peered":"2024-09-21T11:40:38.116981+0000","last_clean":"2024-09-21T11:40:38.116981+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:40:38.116981+0000","last_undegraded":"2024-09-21T11:40:38.116981+0000","last_fullsized":"2024-09-21T11:40:38.116981+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.785487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092593+0000","last_change":"2024-09-21T11:40:04.328515+0000","last_active":"2024-09-21T11:40:37.092593+0000","last_peered":"2024-09-21T11:40:37.092593+0000","last_clean":"2024-09-21T11:40:37.092593+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:40:37.092593+0000","last_undegraded":"2024-09-21T11:40:37.092593+0000","last_fullsized":"2024-09-21T11:40:37.092593+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:03.806494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":66,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097724+0000","last_change":"2024-09-21T11:40:02.323539+0000","last_active":"2024-09-21T11:40:37.097724+0000","last_peered":"2024-09-21T11:40:37.097724+0000","last_clean":"2024-09-21T11:40:37.097724+0000","last_became_active":"2024-09-21T11:40:02.323204+0000","last_became_peered":"2024-09-21T11:40:02.323204+0000","last_unstale":"2024-09-21T11:40:37.097724+0000","last_undegraded":"2024-09-21T11:40:37.097724+0000","last_fullsized":"2024-09-21T11:40:37.097724+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089484+0000","last_change":"2024-09-21T11:40:00.315307+0000","last_active":"2024-09-21T11:40:37.089484+0000","last_peered":"2024-09-21T11:40:37.089484+0000","last_clean":"2024-09-21T11:40:37.089484+0000","last_became_active":"2024-09-21T11:40:00.315015+0000","last_became_peered":"2024-09-21T11:40:00.315015+0000","last_unstale":"2024-09-21T11:40:37.089484+0000","last_undegraded":"2024-09-21T11:40:37.089484+0000","last_fullsized":"2024-09-21T11:40:37.089484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.959497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119140+0000","last_change":"2024-09-21T11:40:04.342645+0000","last_active":"2024-09-21T11:40:38.119140+0000","last_peered":"2024-09-21T11:40:38.119140+0000","last_clean":"2024-09-21T11:40:38.119140+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:40:38.119140+0000","last_undegraded":"2024-09-21T11:40:38.119140+0000","last_fullsized":"2024-09-21T11:40:38.119140+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.259518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118232+0000","last_change":"2024-09-21T11:40:06.345062+0000","last_active":"2024-09-21T11:40:38.118232+0000","last_peered":"2024-09-21T11:40:38.118232+0000","last_clean":"2024-09-21T11:40:38.118232+0000","last_became_active":"2024-09-21T11:40:06.344796+0000","last_became_peered":"2024-09-21T11:40:06.344796+0000","last_unstale":"2024-09-21T11:40:38.118232+0000","last_undegraded":"2024-09-21T11:40:38.118232+0000","last_fullsized":"2024-09-21T11:40:38.118232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.403408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":74,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089287+0000","last_change":"2024-09-21T11:40:02.399334+0000","last_active":"2024-09-21T11:40:37.089287+0000","last_peered":"2024-09-21T11:40:37.089287+0000","last_clean":"2024-09-21T11:40:37.089287+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:40:37.089287+0000","last_undegraded":"2024-09-21T11:40:37.089287+0000","last_fullsized":"2024-09-21T11:40:37.089287+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.386531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091103+0000","last_change":"2024-09-21T11:40:00.310330+0000","last_active":"2024-09-21T11:40:37.091103+0000","last_peered":"2024-09-21T11:40:37.091103+0000","last_clean":"2024-09-21T11:40:37.091103+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:40:37.091103+0000","last_undegraded":"2024-09-21T11:40:37.091103+0000","last_fullsized":"2024-09-21T11:40:37.091103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.371930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117521+0000","last_change":"2024-09-21T11:40:04.330180+0000","last_active":"2024-09-21T11:40:38.117521+0000","last_peered":"2024-09-21T11:40:38.117521+0000","last_clean":"2024-09-21T11:40:38.117521+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:40:38.117521+0000","last_undegraded":"2024-09-21T11:40:38.117521+0000","last_fullsized":"2024-09-21T11:40:38.117521+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.570842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118790+0000","last_change":"2024-09-21T11:40:06.343742+0000","last_active":"2024-09-21T11:40:38.118790+0000","last_peered":"2024-09-21T11:40:38.118790+0000","last_clean":"2024-09-21T11:40:38.118790+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:40:38.118790+0000","last_undegraded":"2024-09-21T11:40:38.118790+0000","last_fullsized":"2024-09-21T11:40:38.118790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.217075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":63,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117749+0000","last_change":"2024-09-21T11:40:02.312574+0000","last_active":"2024-09-21T11:40:38.117749+0000","last_peered":"2024-09-21T11:40:38.117749+0000","last_clean":"2024-09-21T11:40:38.117749+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:40:38.117749+0000","last_undegraded":"2024-09-21T11:40:38.117749+0000","last_fullsized":"2024-09-21T11:40:38.117749+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.497994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117503+0000","last_change":"2024-09-21T11:40:00.312084+0000","last_active":"2024-09-21T11:40:38.117503+0000","last_peered":"2024-09-21T11:40:38.117503+0000","last_clean":"2024-09-21T11:40:38.117503+0000","last_became_active":"2024-09-21T11:40:00.311813+0000","last_became_peered":"2024-09-21T11:40:00.311813+0000","last_unstale":"2024-09-21T11:40:38.117503+0000","last_undegraded":"2024-09-21T11:40:38.117503+0000","last_fullsized":"2024-09-21T11:40:38.117503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.090287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089147+0000","last_change":"2024-09-21T11:40:04.324572+0000","last_active":"2024-09-21T11:40:37.089147+0000","last_peered":"2024-09-21T11:40:37.089147+0000","last_clean":"2024-09-21T11:40:37.089147+0000","last_became_active":"2024-09-21T11:40:04.324272+0000","last_became_peered":"2024-09-21T11:40:04.324272+0000","last_unstale":"2024-09-21T11:40:37.089147+0000","last_undegraded":"2024-09-21T11:40:37.089147+0000","last_fullsized":"2024-09-21T11:40:37.089147+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.830113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117696+0000","last_change":"2024-09-21T11:40:06.489239+0000","last_active":"2024-09-21T11:40:38.117696+0000","last_peered":"2024-09-21T11:40:38.117696+0000","last_clean":"2024-09-21T11:40:38.117696+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:40:38.117696+0000","last_undegraded":"2024-09-21T11:40:38.117696+0000","last_fullsized":"2024-09-21T11:40:38.117696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.580118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097096+0000","last_change":"2024-09-21T11:40:02.318985+0000","last_active":"2024-09-21T11:40:37.097096+0000","last_peered":"2024-09-21T11:40:37.097096+0000","last_clean":"2024-09-21T11:40:37.097096+0000","last_became_active":"2024-09-21T11:40:02.318270+0000","last_became_peered":"2024-09-21T11:40:02.318270+0000","last_unstale":"2024-09-21T11:40:37.097096+0000","last_undegraded":"2024-09-21T11:40:37.097096+0000","last_fullsized":"2024-09-21T11:40:37.097096+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.457674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119649+0000","last_change":"2024-09-21T11:40:00.318794+0000","last_active":"2024-09-21T11:40:38.119649+0000","last_peered":"2024-09-21T11:40:38.119649+0000","last_clean":"2024-09-21T11:40:38.119649+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:40:38.119649+0000","last_undegraded":"2024-09-21T11:40:38.119649+0000","last_fullsized":"2024-09-21T11:40:38.119649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.580077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116819+0000","last_change":"2024-09-21T11:40:04.335582+0000","last_active":"2024-09-21T11:40:38.116819+0000","last_peered":"2024-09-21T11:40:38.116819+0000","last_clean":"2024-09-21T11:40:38.116819+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:40:38.116819+0000","last_undegraded":"2024-09-21T11:40:38.116819+0000","last_fullsized":"2024-09-21T11:40:38.116819+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:03.386490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116484+0000","last_change":"2024-09-21T11:40:06.352201+0000","last_active":"2024-09-21T11:40:38.116484+0000","last_peered":"2024-09-21T11:40:38.116484+0000","last_clean":"2024-09-21T11:40:38.116484+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:40:38.116484+0000","last_undegraded":"2024-09-21T11:40:38.116484+0000","last_fullsized":"2024-09-21T11:40:38.116484+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.165004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":77,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116928+0000","last_change":"2024-09-21T11:40:02.399117+0000","last_active":"2024-09-21T11:40:38.116928+0000","last_peered":"2024-09-21T11:40:38.116928+0000","last_clean":"2024-09-21T11:40:38.116928+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:40:38.116928+0000","last_undegraded":"2024-09-21T11:40:38.116928+0000","last_fullsized":"2024-09-21T11:40:38.116928+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.772060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097142+0000","last_change":"2024-09-21T11:40:00.311519+0000","last_active":"2024-09-21T11:40:37.097142+0000","last_peered":"2024-09-21T11:40:37.097142+0000","last_clean":"2024-09-21T11:40:37.097142+0000","last_became_active":"2024-09-21T11:40:00.311211+0000","last_became_peered":"2024-09-21T11:40:00.311211+0000","last_unstale":"2024-09-21T11:40:37.097142+0000","last_undegraded":"2024-09-21T11:40:37.097142+0000","last_fullsized":"2024-09-21T11:40:37.097142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.982610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091780+0000","last_change":"2024-09-21T11:40:04.339202+0000","last_active":"2024-09-21T11:40:37.091780+0000","last_peered":"2024-09-21T11:40:37.091780+0000","last_clean":"2024-09-21T11:40:37.091780+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:40:37.091780+0000","last_undegraded":"2024-09-21T11:40:37.091780+0000","last_fullsized":"2024-09-21T11:40:37.091780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.313979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096533+0000","last_change":"2024-09-21T11:40:06.487976+0000","last_active":"2024-09-21T11:40:37.096533+0000","last_peered":"2024-09-21T11:40:37.096533+0000","last_clean":"2024-09-21T11:40:37.096533+0000","last_became_active":"2024-09-21T11:40:06.487644+0000","last_became_peered":"2024-09-21T11:40:06.487644+0000","last_unstale":"2024-09-21T11:40:37.096533+0000","last_undegraded":"2024-09-21T11:40:37.096533+0000","last_fullsized":"2024-09-21T11:40:37.096533+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.318979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":98,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097110+0000","last_change":"2024-09-21T11:40:02.320105+0000","last_active":"2024-09-21T11:40:37.097110+0000","last_peered":"2024-09-21T11:40:37.097110+0000","last_clean":"2024-09-21T11:40:37.097110+0000","last_became_active":"2024-09-21T11:40:02.319770+0000","last_became_peered":"2024-09-21T11:40:02.319770+0000","last_unstale":"2024-09-21T11:40:37.097110+0000","last_undegraded":"2024-09-21T11:40:37.097110+0000","last_fullsized":"2024-09-21T11:40:37.097110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.672594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119401+0000","last_change":"2024-09-21T11:40:00.314186+0000","last_active":"2024-09-21T11:40:38.119401+0000","last_peered":"2024-09-21T11:40:38.119401+0000","last_clean":"2024-09-21T11:40:38.119401+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:40:38.119401+0000","last_undegraded":"2024-09-21T11:40:38.119401+0000","last_fullsized":"2024-09-21T11:40:38.119401+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:14.587338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117801+0000","last_change":"2024-09-21T11:40:02.311375+0000","last_active":"2024-09-21T11:40:38.117801+0000","last_peered":"2024-09-21T11:40:38.117801+0000","last_clean":"2024-09-21T11:40:38.117801+0000","last_became_active":"2024-09-21T11:39:59.303309+0000","last_became_peered":"2024-09-21T11:39:59.303309+0000","last_unstale":"2024-09-21T11:40:38.117801+0000","last_undegraded":"2024-09-21T11:40:38.117801+0000","last_fullsized":"2024-09-21T11:40:38.117801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.322560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116887+0000","last_change":"2024-09-21T11:40:04.337767+0000","last_active":"2024-09-21T11:40:38.116887+0000","last_peered":"2024-09-21T11:40:38.116887+0000","last_clean":"2024-09-21T11:40:38.116887+0000","last_became_active":"2024-09-21T11:40:04.337422+0000","last_became_peered":"2024-09-21T11:40:04.337422+0000","last_unstale":"2024-09-21T11:40:38.116887+0000","last_undegraded":"2024-09-21T11:40:38.116887+0000","last_fullsized":"2024-09-21T11:40:38.116887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.195058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090431+0000","last_change":"2024-09-21T11:40:06.350784+0000","last_active":"2024-09-21T11:40:37.090431+0000","last_peered":"2024-09-21T11:40:37.090431+0000","last_clean":"2024-09-21T11:40:37.090431+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:40:37.090431+0000","last_undegraded":"2024-09-21T11:40:37.090431+0000","last_fullsized":"2024-09-21T11:40:37.090431+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.561650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097771+0000","last_change":"2024-09-21T11:40:02.324944+0000","last_active":"2024-09-21T11:40:37.097771+0000","last_peered":"2024-09-21T11:40:37.097771+0000","last_clean":"2024-09-21T11:40:37.097771+0000","last_became_active":"2024-09-21T11:40:02.324667+0000","last_became_peered":"2024-09-21T11:40:02.324667+0000","last_unstale":"2024-09-21T11:40:37.097771+0000","last_undegraded":"2024-09-21T11:40:37.097771+0000","last_fullsized":"2024-09-21T11:40:37.097771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.447522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096780+0000","last_change":"2024-09-21T11:40:00.313632+0000","last_active":"2024-09-21T11:40:37.096780+0000","last_peered":"2024-09-21T11:40:37.096780+0000","last_clean":"2024-09-21T11:40:37.096780+0000","last_became_active":"2024-09-21T11:40:00.313231+0000","last_became_peered":"2024-09-21T11:40:00.313231+0000","last_unstale":"2024-09-21T11:40:37.096780+0000","last_undegraded":"2024-09-21T11:40:37.096780+0000","last_fullsized":"2024-09-21T11:40:37.096780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.689078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089293+0000","last_change":"2024-09-21T11:40:02.305143+0000","last_active":"2024-09-21T11:40:37.089293+0000","last_peered":"2024-09-21T11:40:37.089293+0000","last_clean":"2024-09-21T11:40:37.089293+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:40:37.089293+0000","last_undegraded":"2024-09-21T11:40:37.089293+0000","last_fullsized":"2024-09-21T11:40:37.089293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.513240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089829+0000","last_change":"2024-09-21T11:40:04.332073+0000","last_active":"2024-09-21T11:40:37.089829+0000","last_peered":"2024-09-21T11:40:37.089829+0000","last_clean":"2024-09-21T11:40:37.089829+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:40:37.089829+0000","last_undegraded":"2024-09-21T11:40:37.089829+0000","last_fullsized":"2024-09-21T11:40:37.089829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.887948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119154+0000","last_change":"2024-09-21T11:40:06.487757+0000","last_active":"2024-09-21T11:40:38.119154+0000","last_peered":"2024-09-21T11:40:38.119154+0000","last_clean":"2024-09-21T11:40:38.119154+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:40:38.119154+0000","last_undegraded":"2024-09-21T11:40:38.119154+0000","last_fullsized":"2024-09-21T11:40:38.119154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.207444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117442+0000","last_change":"2024-09-21T11:40:02.320851+0000","last_active":"2024-09-21T11:40:38.117442+0000","last_peered":"2024-09-21T11:40:38.117442+0000","last_clean":"2024-09-21T11:40:38.117442+0000","last_became_active":"2024-09-21T11:40:02.320522+0000","last_became_peered":"2024-09-21T11:40:02.320522+0000","last_unstale":"2024-09-21T11:40:38.117442+0000","last_undegraded":"2024-09-21T11:40:38.117442+0000","last_fullsized":"2024-09-21T11:40:38.117442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:30.451592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116473+0000","last_change":"2024-09-21T11:40:00.312222+0000","last_active":"2024-09-21T11:40:38.116473+0000","last_peered":"2024-09-21T11:40:38.116473+0000","last_clean":"2024-09-21T11:40:38.116473+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:40:38.116473+0000","last_undegraded":"2024-09-21T11:40:38.116473+0000","last_fullsized":"2024-09-21T11:40:38.116473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.238210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":91,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.735310+0000","last_change":"2024-09-21T11:40:02.398538+0000","last_active":"2024-09-21T11:40:38.735310+0000","last_peered":"2024-09-21T11:40:38.735310+0000","last_clean":"2024-09-21T11:40:38.735310+0000","last_became_active":"2024-09-21T11:39:59.306320+0000","last_became_peered":"2024-09-21T11:39:59.306320+0000","last_unstale":"2024-09-21T11:40:38.735310+0000","last_undegraded":"2024-09-21T11:40:38.735310+0000","last_fullsized":"2024-09-21T11:40:38.735310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:18.630069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119601+0000","last_change":"2024-09-21T11:40:04.339615+0000","last_active":"2024-09-21T11:40:38.119601+0000","last_peered":"2024-09-21T11:40:38.119601+0000","last_clean":"2024-09-21T11:40:38.119601+0000","last_became_active":"2024-09-21T11:40:04.339331+0000","last_became_peered":"2024-09-21T11:40:04.339331+0000","last_unstale":"2024-09-21T11:40:38.119601+0000","last_undegraded":"2024-09-21T11:40:38.119601+0000","last_fullsized":"2024-09-21T11:40:38.119601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:33.310635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096709+0000","last_change":"2024-09-21T11:40:06.352155+0000","last_active":"2024-09-21T11:40:37.096709+0000","last_peered":"2024-09-21T11:40:37.096709+0000","last_clean":"2024-09-21T11:40:37.096709+0000","last_became_active":"2024-09-21T11:40:06.351777+0000","last_became_peered":"2024-09-21T11:40:06.351777+0000","last_unstale":"2024-09-21T11:40:37.096709+0000","last_undegraded":"2024-09-21T11:40:37.096709+0000","last_fullsized":"2024-09-21T11:40:37.096709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.594495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":72,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092570+0000","last_change":"2024-09-21T11:40:02.399268+0000","last_active":"2024-09-21T11:40:37.092570+0000","last_peered":"2024-09-21T11:40:37.092570+0000","last_clean":"2024-09-21T11:40:37.092570+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:40:37.092570+0000","last_undegraded":"2024-09-21T11:40:37.092570+0000","last_fullsized":"2024-09-21T11:40:37.092570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.058592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090286+0000","last_change":"2024-09-21T11:40:00.309464+0000","last_active":"2024-09-21T11:40:37.090286+0000","last_peered":"2024-09-21T11:40:37.090286+0000","last_clean":"2024-09-21T11:40:37.090286+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:40:37.090286+0000","last_undegraded":"2024-09-21T11:40:37.090286+0000","last_fullsized":"2024-09-21T11:40:37.090286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.615608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":90,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118342+0000","last_change":"2024-09-21T11:40:12.500662+0000","last_active":"2024-09-21T11:40:38.118342+0000","last_peered":"2024-09-21T11:40:38.118342+0000","last_clean":"2024-09-21T11:40:38.118342+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:40:38.118342+0000","last_undegraded":"2024-09-21T11:40:38.118342+0000","last_fullsized":"2024-09-21T11:40:38.118342+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'72","last_scrub_stamp":"2024-09-21T11:40:12.500606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:40:12.500606+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.061712+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118696+0000","last_change":"2024-09-21T11:40:04.333889+0000","last_active":"2024-09-21T11:40:38.118696+0000","last_peered":"2024-09-21T11:40:38.118696+0000","last_clean":"2024-09-21T11:40:38.118696+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:40:38.118696+0000","last_undegraded":"2024-09-21T11:40:38.118696+0000","last_fullsized":"2024-09-21T11:40:38.118696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.202161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119447+0000","last_change":"2024-09-21T11:40:06.349857+0000","last_active":"2024-09-21T11:40:38.119447+0000","last_peered":"2024-09-21T11:40:38.119447+0000","last_clean":"2024-09-21T11:40:38.119447+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:40:38.119447+0000","last_undegraded":"2024-09-21T11:40:38.119447+0000","last_fullsized":"2024-09-21T11:40:38.119447+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.149814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119188+0000","last_change":"2024-09-21T11:40:02.319709+0000","last_active":"2024-09-21T11:40:38.119188+0000","last_peered":"2024-09-21T11:40:38.119188+0000","last_clean":"2024-09-21T11:40:38.119188+0000","last_became_active":"2024-09-21T11:40:02.319422+0000","last_became_peered":"2024-09-21T11:40:02.319422+0000","last_unstale":"2024-09-21T11:40:38.119188+0000","last_undegraded":"2024-09-21T11:40:38.119188+0000","last_fullsized":"2024-09-21T11:40:38.119188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.852025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117859+0000","last_change":"2024-09-21T11:40:00.312057+0000","last_active":"2024-09-21T11:40:38.117859+0000","last_peered":"2024-09-21T11:40:38.117859+0000","last_clean":"2024-09-21T11:40:38.117859+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:40:38.117859+0000","last_undegraded":"2024-09-21T11:40:38.117859+0000","last_fullsized":"2024-09-21T11:40:38.117859+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.891862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119217+0000","last_change":"2024-09-21T11:40:04.329480+0000","last_active":"2024-09-21T11:40:38.119217+0000","last_peered":"2024-09-21T11:40:38.119217+0000","last_clean":"2024-09-21T11:40:38.119217+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:40:38.119217+0000","last_undegraded":"2024-09-21T11:40:38.119217+0000","last_fullsized":"2024-09-21T11:40:38.119217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090387+0000","last_change":"2024-09-21T11:40:06.487868+0000","last_active":"2024-09-21T11:40:37.090387+0000","last_peered":"2024-09-21T11:40:37.090387+0000","last_clean":"2024-09-21T11:40:37.090387+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:40:37.090387+0000","last_undegraded":"2024-09-21T11:40:37.090387+0000","last_fullsized":"2024-09-21T11:40:37.090387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.512063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":70,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118993+0000","last_change":"2024-09-21T11:40:02.319361+0000","last_active":"2024-09-21T11:40:38.118993+0000","last_peered":"2024-09-21T11:40:38.118993+0000","last_clean":"2024-09-21T11:40:38.118993+0000","last_became_active":"2024-09-21T11:40:02.318902+0000","last_became_peered":"2024-09-21T11:40:02.318902+0000","last_unstale":"2024-09-21T11:40:38.118993+0000","last_undegraded":"2024-09-21T11:40:38.118993+0000","last_fullsized":"2024-09-21T11:40:38.118993+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.422023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092089+0000","last_change":"2024-09-21T11:40:00.313702+0000","last_active":"2024-09-21T11:40:37.092089+0000","last_peered":"2024-09-21T11:40:37.092089+0000","last_clean":"2024-09-21T11:40:37.092089+0000","last_became_active":"2024-09-21T11:40:00.313432+0000","last_became_peered":"2024-09-21T11:40:00.313432+0000","last_unstale":"2024-09-21T11:40:37.092089+0000","last_undegraded":"2024-09-21T11:40:37.092089+0000","last_fullsized":"2024-09-21T11:40:37.092089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:21.740308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096816+0000","last_change":"2024-09-21T11:40:04.342081+0000","last_active":"2024-09-21T11:40:37.096816+0000","last_peered":"2024-09-21T11:40:37.096816+0000","last_clean":"2024-09-21T11:40:37.096816+0000","last_became_active":"2024-09-21T11:40:04.341743+0000","last_became_peered":"2024-09-21T11:40:04.341743+0000","last_unstale":"2024-09-21T11:40:37.096816+0000","last_undegraded":"2024-09-21T11:40:37.096816+0000","last_fullsized":"2024-09-21T11:40:37.096816+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.025793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089044+0000","last_change":"2024-09-21T11:40:06.349130+0000","last_active":"2024-09-21T11:40:37.089044+0000","last_peered":"2024-09-21T11:40:37.089044+0000","last_clean":"2024-09-21T11:40:37.089044+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:40:37.089044+0000","last_undegraded":"2024-09-21T11:40:37.089044+0000","last_fullsized":"2024-09-21T11:40:37.089044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:14.687896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118670+0000","last_change":"2024-09-21T11:40:02.317262+0000","last_active":"2024-09-21T11:40:38.118670+0000","last_peered":"2024-09-21T11:40:38.118670+0000","last_clean":"2024-09-21T11:40:38.118670+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:40:38.118670+0000","last_undegraded":"2024-09-21T11:40:38.118670+0000","last_fullsized":"2024-09-21T11:40:38.118670+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.898962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090545+0000","last_change":"2024-09-21T11:40:00.307721+0000","last_active":"2024-09-21T11:40:37.090545+0000","last_peered":"2024-09-21T11:40:37.090545+0000","last_clean":"2024-09-21T11:40:37.090545+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:40:37.090545+0000","last_undegraded":"2024-09-21T11:40:37.090545+0000","last_fullsized":"2024-09-21T11:40:37.090545+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.478046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089485+0000","last_change":"2024-09-21T11:40:04.332939+0000","last_active":"2024-09-21T11:40:37.089485+0000","last_peered":"2024-09-21T11:40:37.089485+0000","last_clean":"2024-09-21T11:40:37.089485+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:40:37.089485+0000","last_undegraded":"2024-09-21T11:40:37.089485+0000","last_fullsized":"2024-09-21T11:40:37.089485+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.250509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118805+0000","last_change":"2024-09-21T11:40:06.346865+0000","last_active":"2024-09-21T11:40:38.118805+0000","last_peered":"2024-09-21T11:40:38.118805+0000","last_clean":"2024-09-21T11:40:38.118805+0000","last_became_active":"2024-09-21T11:40:06.346602+0000","last_became_peered":"2024-09-21T11:40:06.346602+0000","last_unstale":"2024-09-21T11:40:38.118805+0000","last_undegraded":"2024-09-21T11:40:38.118805+0000","last_fullsized":"2024-09-21T11:40:38.118805+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.724328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116816+0000","last_change":"2024-09-21T11:40:02.319787+0000","last_active":"2024-09-21T11:40:38.116816+0000","last_peered":"2024-09-21T11:40:38.116816+0000","last_clean":"2024-09-21T11:40:38.116816+0000","last_became_active":"2024-09-21T11:40:02.319502+0000","last_became_peered":"2024-09-21T11:40:02.319502+0000","last_unstale":"2024-09-21T11:40:38.116816+0000","last_undegraded":"2024-09-21T11:40:38.116816+0000","last_fullsized":"2024-09-21T11:40:38.116816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.504218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118415+0000","last_change":"2024-09-21T11:40:00.318087+0000","last_active":"2024-09-21T11:40:38.118415+0000","last_peered":"2024-09-21T11:40:38.118415+0000","last_clean":"2024-09-21T11:40:38.118415+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:40:38.118415+0000","last_undegraded":"2024-09-21T11:40:38.118415+0000","last_fullsized":"2024-09-21T11:40:38.118415+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.865238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089043+0000","last_change":"2024-09-21T11:40:04.330158+0000","last_active":"2024-09-21T11:40:37.089043+0000","last_peered":"2024-09-21T11:40:37.089043+0000","last_clean":"2024-09-21T11:40:37.089043+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:40:37.089043+0000","last_undegraded":"2024-09-21T11:40:37.089043+0000","last_fullsized":"2024-09-21T11:40:37.089043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.222022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116330+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:40:38.116330+0000","last_peered":"2024-09-21T11:40:38.116330+0000","last_clean":"2024-09-21T11:40:38.116330+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:40:38.116330+0000","last_undegraded":"2024-09-21T11:40:38.116330+0000","last_fullsized":"2024-09-21T11:40:38.116330+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:32.858006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":72,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091924+0000","last_change":"2024-09-21T11:40:02.399340+0000","last_active":"2024-09-21T11:40:37.091924+0000","last_peered":"2024-09-21T11:40:37.091924+0000","last_clean":"2024-09-21T11:40:37.091924+0000","last_became_active":"2024-09-21T11:40:02.398876+0000","last_became_peered":"2024-09-21T11:40:02.398876+0000","last_unstale":"2024-09-21T11:40:37.091924+0000","last_undegraded":"2024-09-21T11:40:37.091924+0000","last_fullsized":"2024-09-21T11:40:37.091924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.043119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119634+0000","last_change":"2024-09-21T11:40:00.314507+0000","last_active":"2024-09-21T11:40:38.119634+0000","last_peered":"2024-09-21T11:40:38.119634+0000","last_clean":"2024-09-21T11:40:38.119634+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:40:38.119634+0000","last_undegraded":"2024-09-21T11:40:38.119634+0000","last_fullsized":"2024-09-21T11:40:38.119634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.755846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089466+0000","last_change":"2024-09-21T11:40:04.341835+0000","last_active":"2024-09-21T11:40:37.089466+0000","last_peered":"2024-09-21T11:40:37.089466+0000","last_clean":"2024-09-21T11:40:37.089466+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:40:37.089466+0000","last_undegraded":"2024-09-21T11:40:37.089466+0000","last_fullsized":"2024-09-21T11:40:37.089466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:21.943269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118201+0000","last_change":"2024-09-21T11:40:06.348279+0000","last_active":"2024-09-21T11:40:38.118201+0000","last_peered":"2024-09-21T11:40:38.118201+0000","last_clean":"2024-09-21T11:40:38.118201+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:40:38.118201+0000","last_undegraded":"2024-09-21T11:40:38.118201+0000","last_fullsized":"2024-09-21T11:40:38.118201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.108944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117650+0000","last_change":"2024-09-21T11:40:02.320045+0000","last_active":"2024-09-21T11:40:38.117650+0000","last_peered":"2024-09-21T11:40:38.117650+0000","last_clean":"2024-09-21T11:40:38.117650+0000","last_became_active":"2024-09-21T11:40:02.319752+0000","last_became_peered":"2024-09-21T11:40:02.319752+0000","last_unstale":"2024-09-21T11:40:38.117650+0000","last_undegraded":"2024-09-21T11:40:38.117650+0000","last_fullsized":"2024-09-21T11:40:38.117650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.042672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.120082+0000","last_change":"2024-09-21T11:40:00.311267+0000","last_active":"2024-09-21T11:40:38.120082+0000","last_peered":"2024-09-21T11:40:38.120082+0000","last_clean":"2024-09-21T11:40:38.120082+0000","last_became_active":"2024-09-21T11:40:00.310457+0000","last_became_peered":"2024-09-21T11:40:00.310457+0000","last_unstale":"2024-09-21T11:40:38.120082+0000","last_undegraded":"2024-09-21T11:40:38.120082+0000","last_fullsized":"2024-09-21T11:40:38.120082+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.362207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.088844+0000","last_change":"2024-09-21T11:40:04.334536+0000","last_active":"2024-09-21T11:40:37.088844+0000","last_peered":"2024-09-21T11:40:37.088844+0000","last_clean":"2024-09-21T11:40:37.088844+0000","last_became_active":"2024-09-21T11:40:04.334205+0000","last_became_peered":"2024-09-21T11:40:04.334205+0000","last_unstale":"2024-09-21T11:40:37.088844+0000","last_undegraded":"2024-09-21T11:40:37.088844+0000","last_fullsized":"2024-09-21T11:40:37.088844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.125493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090268+0000","last_change":"2024-09-21T11:40:06.352508+0000","last_active":"2024-09-21T11:40:37.090268+0000","last_peered":"2024-09-21T11:40:37.090268+0000","last_clean":"2024-09-21T11:40:37.090268+0000","last_became_active":"2024-09-21T11:40:06.352210+0000","last_became_peered":"2024-09-21T11:40:06.352210+0000","last_unstale":"2024-09-21T11:40:37.090268+0000","last_undegraded":"2024-09-21T11:40:37.090268+0000","last_fullsized":"2024-09-21T11:40:37.090268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.957978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":66,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116907+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:40:38.116907+0000","last_peered":"2024-09-21T11:40:38.116907+0000","last_clean":"2024-09-21T11:40:38.116907+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:40:38.116907+0000","last_undegraded":"2024-09-21T11:40:38.116907+0000","last_fullsized":"2024-09-21T11:40:38.116907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.513015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.120072+0000","last_change":"2024-09-21T11:40:00.311525+0000","last_active":"2024-09-21T11:40:38.120072+0000","last_peered":"2024-09-21T11:40:38.120072+0000","last_clean":"2024-09-21T11:40:38.120072+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:40:38.120072+0000","last_undegraded":"2024-09-21T11:40:38.120072+0000","last_fullsized":"2024-09-21T11:40:38.120072+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.568697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118359+0000","last_change":"2024-09-21T11:40:04.331990+0000","last_active":"2024-09-21T11:40:38.118359+0000","last_peered":"2024-09-21T11:40:38.118359+0000","last_clean":"2024-09-21T11:40:38.118359+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:40:38.118359+0000","last_undegraded":"2024-09-21T11:40:38.118359+0000","last_fullsized":"2024-09-21T11:40:38.118359+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.347865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117350+0000","last_change":"2024-09-21T11:40:06.487630+0000","last_active":"2024-09-21T11:40:38.117350+0000","last_peered":"2024-09-21T11:40:38.117350+0000","last_clean":"2024-09-21T11:40:38.117350+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:40:38.117350+0000","last_undegraded":"2024-09-21T11:40:38.117350+0000","last_fullsized":"2024-09-21T11:40:38.117350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.388794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092001+0000","last_change":"2024-09-21T11:40:00.310655+0000","last_active":"2024-09-21T11:40:37.092001+0000","last_peered":"2024-09-21T11:40:37.092001+0000","last_clean":"2024-09-21T11:40:37.092001+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:40:37.092001+0000","last_undegraded":"2024-09-21T11:40:37.092001+0000","last_fullsized":"2024-09-21T11:40:37.092001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.882425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091044+0000","last_change":"2024-09-21T11:40:02.399093+0000","last_active":"2024-09-21T11:40:37.091044+0000","last_peered":"2024-09-21T11:40:37.091044+0000","last_clean":"2024-09-21T11:40:37.091044+0000","last_became_active":"2024-09-21T11:40:02.398803+0000","last_became_peered":"2024-09-21T11:40:02.398803+0000","last_unstale":"2024-09-21T11:40:37.091044+0000","last_undegraded":"2024-09-21T11:40:37.091044+0000","last_fullsized":"2024-09-21T11:40:37.091044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.504556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118790+0000","last_change":"2024-09-21T11:40:04.328297+0000","last_active":"2024-09-21T11:40:38.118790+0000","last_peered":"2024-09-21T11:40:38.118790+0000","last_clean":"2024-09-21T11:40:38.118790+0000","last_became_active":"2024-09-21T11:40:04.327422+0000","last_became_peered":"2024-09-21T11:40:04.327422+0000","last_unstale":"2024-09-21T11:40:38.118790+0000","last_undegraded":"2024-09-21T11:40:38.118790+0000","last_fullsized":"2024-09-21T11:40:38.118790+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.663596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119034+0000","last_change":"2024-09-21T11:40:06.488399+0000","last_active":"2024-09-21T11:40:38.119034+0000","last_peered":"2024-09-21T11:40:38.119034+0000","last_clean":"2024-09-21T11:40:38.119034+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:40:38.119034+0000","last_undegraded":"2024-09-21T11:40:38.119034+0000","last_fullsized":"2024-09-21T11:40:38.119034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.617144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117426+0000","last_change":"2024-09-21T11:40:02.399564+0000","last_active":"2024-09-21T11:40:38.117426+0000","last_peered":"2024-09-21T11:40:38.117426+0000","last_clean":"2024-09-21T11:40:38.117426+0000","last_became_active":"2024-09-21T11:40:02.399267+0000","last_became_peered":"2024-09-21T11:40:02.399267+0000","last_unstale":"2024-09-21T11:40:38.117426+0000","last_undegraded":"2024-09-21T11:40:38.117426+0000","last_fullsized":"2024-09-21T11:40:38.117426+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.891469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118903+0000","last_change":"2024-09-21T11:40:00.308959+0000","last_active":"2024-09-21T11:40:38.118903+0000","last_peered":"2024-09-21T11:40:38.118903+0000","last_clean":"2024-09-21T11:40:38.118903+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:40:38.118903+0000","last_undegraded":"2024-09-21T11:40:38.118903+0000","last_fullsized":"2024-09-21T11:40:38.118903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.467036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090927+0000","last_change":"2024-09-21T11:40:04.326964+0000","last_active":"2024-09-21T11:40:37.090927+0000","last_peered":"2024-09-21T11:40:37.090927+0000","last_clean":"2024-09-21T11:40:37.090927+0000","last_became_active":"2024-09-21T11:40:04.326625+0000","last_became_peered":"2024-09-21T11:40:04.326625+0000","last_unstale":"2024-09-21T11:40:37.090927+0000","last_undegraded":"2024-09-21T11:40:37.090927+0000","last_fullsized":"2024-09-21T11:40:37.090927+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089630+0000","last_change":"2024-09-21T11:40:06.343913+0000","last_active":"2024-09-21T11:40:37.089630+0000","last_peered":"2024-09-21T11:40:37.089630+0000","last_clean":"2024-09-21T11:40:37.089630+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:40:37.089630+0000","last_undegraded":"2024-09-21T11:40:37.089630+0000","last_fullsized":"2024-09-21T11:40:37.089630+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.682920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119505+0000","last_change":"2024-09-21T11:40:02.400725+0000","last_active":"2024-09-21T11:40:38.119505+0000","last_peered":"2024-09-21T11:40:38.119505+0000","last_clean":"2024-09-21T11:40:38.119505+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:40:38.119505+0000","last_undegraded":"2024-09-21T11:40:38.119505+0000","last_fullsized":"2024-09-21T11:40:38.119505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.300079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116459+0000","last_change":"2024-09-21T11:40:00.311525+0000","last_active":"2024-09-21T11:40:38.116459+0000","last_peered":"2024-09-21T11:40:38.116459+0000","last_clean":"2024-09-21T11:40:38.116459+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:40:38.116459+0000","last_undegraded":"2024-09-21T11:40:38.116459+0000","last_fullsized":"2024-09-21T11:40:38.116459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.269082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090572+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:40:37.090572+0000","last_peered":"2024-09-21T11:40:37.090572+0000","last_clean":"2024-09-21T11:40:37.090572+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:40:37.090572+0000","last_undegraded":"2024-09-21T11:40:37.090572+0000","last_fullsized":"2024-09-21T11:40:37.090572+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.404995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118969+0000","last_change":"2024-09-21T11:40:06.346686+0000","last_active":"2024-09-21T11:40:38.118969+0000","last_peered":"2024-09-21T11:40:38.118969+0000","last_clean":"2024-09-21T11:40:38.118969+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:40:38.118969+0000","last_undegraded":"2024-09-21T11:40:38.118969+0000","last_fullsized":"2024-09-21T11:40:38.118969+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.014386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090690+0000","last_change":"2024-09-21T11:40:02.399404+0000","last_active":"2024-09-21T11:40:37.090690+0000","last_peered":"2024-09-21T11:40:37.090690+0000","last_clean":"2024-09-21T11:40:37.090690+0000","last_became_active":"2024-09-21T11:40:02.398771+0000","last_became_peered":"2024-09-21T11:40:02.398771+0000","last_unstale":"2024-09-21T11:40:37.090690+0000","last_undegraded":"2024-09-21T11:40:37.090690+0000","last_fullsized":"2024-09-21T11:40:37.090690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.091232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118174+0000","last_change":"2024-09-21T11:40:00.314627+0000","last_active":"2024-09-21T11:40:38.118174+0000","last_peered":"2024-09-21T11:40:38.118174+0000","last_clean":"2024-09-21T11:40:38.118174+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:40:38.118174+0000","last_undegraded":"2024-09-21T11:40:38.118174+0000","last_fullsized":"2024-09-21T11:40:38.118174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.535972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119238+0000","last_change":"2024-09-21T11:40:04.340164+0000","last_active":"2024-09-21T11:40:38.119238+0000","last_peered":"2024-09-21T11:40:38.119238+0000","last_clean":"2024-09-21T11:40:38.119238+0000","last_became_active":"2024-09-21T11:40:04.339253+0000","last_became_peered":"2024-09-21T11:40:04.339253+0000","last_unstale":"2024-09-21T11:40:38.119238+0000","last_undegraded":"2024-09-21T11:40:38.119238+0000","last_fullsized":"2024-09-21T11:40:38.119238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:30.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116791+0000","last_change":"2024-09-21T11:40:06.351973+0000","last_active":"2024-09-21T11:40:38.116791+0000","last_peered":"2024-09-21T11:40:38.116791+0000","last_clean":"2024-09-21T11:40:38.116791+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:40:38.116791+0000","last_undegraded":"2024-09-21T11:40:38.116791+0000","last_fullsized":"2024-09-21T11:40:38.116791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.555091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119008+0000","last_change":"2024-09-21T11:40:02.318645+0000","last_active":"2024-09-21T11:40:38.119008+0000","last_peered":"2024-09-21T11:40:38.119008+0000","last_clean":"2024-09-21T11:40:38.119008+0000","last_became_active":"2024-09-21T11:40:02.318352+0000","last_became_peered":"2024-09-21T11:40:02.318352+0000","last_unstale":"2024-09-21T11:40:38.119008+0000","last_undegraded":"2024-09-21T11:40:38.119008+0000","last_fullsized":"2024-09-21T11:40:38.119008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.071118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119237+0000","last_change":"2024-09-21T11:40:00.317768+0000","last_active":"2024-09-21T11:40:38.119237+0000","last_peered":"2024-09-21T11:40:38.119237+0000","last_clean":"2024-09-21T11:40:38.119237+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:40:38.119237+0000","last_undegraded":"2024-09-21T11:40:38.119237+0000","last_fullsized":"2024-09-21T11:40:38.119237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.919244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096986+0000","last_change":"2024-09-21T11:40:04.328561+0000","last_active":"2024-09-21T11:40:37.096986+0000","last_peered":"2024-09-21T11:40:37.096986+0000","last_clean":"2024-09-21T11:40:37.096986+0000","last_became_active":"2024-09-21T11:40:04.328216+0000","last_became_peered":"2024-09-21T11:40:04.328216+0000","last_unstale":"2024-09-21T11:40:37.096986+0000","last_undegraded":"2024-09-21T11:40:37.096986+0000","last_fullsized":"2024-09-21T11:40:37.096986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.146311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089641+0000","last_change":"2024-09-21T11:40:06.353577+0000","last_active":"2024-09-21T11:40:37.089641+0000","last_peered":"2024-09-21T11:40:37.089641+0000","last_clean":"2024-09-21T11:40:37.089641+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:40:37.089641+0000","last_undegraded":"2024-09-21T11:40:37.089641+0000","last_fullsized":"2024-09-21T11:40:37.089641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.269429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097098+0000","last_change":"2024-09-21T11:40:02.319305+0000","last_active":"2024-09-21T11:40:37.097098+0000","last_peered":"2024-09-21T11:40:37.097098+0000","last_clean":"2024-09-21T11:40:37.097098+0000","last_became_active":"2024-09-21T11:40:02.318699+0000","last_became_peered":"2024-09-21T11:40:02.318699+0000","last_unstale":"2024-09-21T11:40:37.097098+0000","last_undegraded":"2024-09-21T11:40:37.097098+0000","last_fullsized":"2024-09-21T11:40:37.097098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.659293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119439+0000","last_change":"2024-09-21T11:40:00.309690+0000","last_active":"2024-09-21T11:40:38.119439+0000","last_peered":"2024-09-21T11:40:38.119439+0000","last_clean":"2024-09-21T11:40:38.119439+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:40:38.119439+0000","last_undegraded":"2024-09-21T11:40:38.119439+0000","last_fullsized":"2024-09-21T11:40:38.119439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.952393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090841+0000","last_change":"2024-09-21T11:40:04.336820+0000","last_active":"2024-09-21T11:40:37.090841+0000","last_peered":"2024-09-21T11:40:37.090841+0000","last_clean":"2024-09-21T11:40:37.090841+0000","last_became_active":"2024-09-21T11:40:04.336469+0000","last_became_peered":"2024-09-21T11:40:04.336469+0000","last_unstale":"2024-09-21T11:40:37.090841+0000","last_undegraded":"2024-09-21T11:40:37.090841+0000","last_fullsized":"2024-09-21T11:40:37.090841+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.369123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116254+0000","last_change":"2024-09-21T11:40:06.346455+0000","last_active":"2024-09-21T11:40:38.116254+0000","last_peered":"2024-09-21T11:40:38.116254+0000","last_clean":"2024-09-21T11:40:38.116254+0000","last_became_active":"2024-09-21T11:40:06.346090+0000","last_became_peered":"2024-09-21T11:40:06.346090+0000","last_unstale":"2024-09-21T11:40:38.116254+0000","last_undegraded":"2024-09-21T11:40:38.116254+0000","last_fullsized":"2024-09-21T11:40:38.116254+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.873349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089766+0000","last_change":"2024-09-21T11:40:02.399706+0000","last_active":"2024-09-21T11:40:37.089766+0000","last_peered":"2024-09-21T11:40:37.089766+0000","last_clean":"2024-09-21T11:40:37.089766+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:40:37.089766+0000","last_undegraded":"2024-09-21T11:40:37.089766+0000","last_fullsized":"2024-09-21T11:40:37.089766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.937696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118907+0000","last_change":"2024-09-21T11:40:00.317057+0000","last_active":"2024-09-21T11:40:38.118907+0000","last_peered":"2024-09-21T11:40:38.118907+0000","last_clean":"2024-09-21T11:40:38.118907+0000","last_became_active":"2024-09-21T11:40:00.316756+0000","last_became_peered":"2024-09-21T11:40:00.316756+0000","last_unstale":"2024-09-21T11:40:38.118907+0000","last_undegraded":"2024-09-21T11:40:38.118907+0000","last_fullsized":"2024-09-21T11:40:38.118907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.129760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116741+0000","last_change":"2024-09-21T11:40:04.331954+0000","last_active":"2024-09-21T11:40:38.116741+0000","last_peered":"2024-09-21T11:40:38.116741+0000","last_clean":"2024-09-21T11:40:38.116741+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:40:38.116741+0000","last_undegraded":"2024-09-21T11:40:38.116741+0000","last_fullsized":"2024-09-21T11:40:38.116741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.778314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090438+0000","last_change":"2024-09-21T11:40:06.488252+0000","last_active":"2024-09-21T11:40:37.090438+0000","last_peered":"2024-09-21T11:40:37.090438+0000","last_clean":"2024-09-21T11:40:37.090438+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:40:37.090438+0000","last_undegraded":"2024-09-21T11:40:37.090438+0000","last_fullsized":"2024-09-21T11:40:37.090438+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.158855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":48,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089194+0000","last_change":"2024-09-21T11:40:02.319060+0000","last_active":"2024-09-21T11:40:37.089194+0000","last_peered":"2024-09-21T11:40:37.089194+0000","last_clean":"2024-09-21T11:40:37.089194+0000","last_became_active":"2024-09-21T11:40:02.318727+0000","last_became_peered":"2024-09-21T11:40:02.318727+0000","last_unstale":"2024-09-21T11:40:37.089194+0000","last_undegraded":"2024-09-21T11:40:37.089194+0000","last_fullsized":"2024-09-21T11:40:37.089194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.877093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116996+0000","last_change":"2024-09-21T11:40:00.314387+0000","last_active":"2024-09-21T11:40:38.116996+0000","last_peered":"2024-09-21T11:40:38.116996+0000","last_clean":"2024-09-21T11:40:38.116996+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:40:38.116996+0000","last_undegraded":"2024-09-21T11:40:38.116996+0000","last_fullsized":"2024-09-21T11:40:38.116996+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:00.574148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091722+0000","last_change":"2024-09-21T11:40:04.332492+0000","last_active":"2024-09-21T11:40:37.091722+0000","last_peered":"2024-09-21T11:40:37.091722+0000","last_clean":"2024-09-21T11:40:37.091722+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:40:37.091722+0000","last_undegraded":"2024-09-21T11:40:37.091722+0000","last_fullsized":"2024-09-21T11:40:37.091722+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.592217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117167+0000","last_change":"2024-09-21T11:40:06.342962+0000","last_active":"2024-09-21T11:40:38.117167+0000","last_peered":"2024-09-21T11:40:38.117167+0000","last_clean":"2024-09-21T11:40:38.117167+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:40:38.117167+0000","last_undegraded":"2024-09-21T11:40:38.117167+0000","last_fullsized":"2024-09-21T11:40:38.117167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.430215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097021+0000","last_change":"2024-09-21T11:40:06.349089+0000","last_active":"2024-09-21T11:40:37.097021+0000","last_peered":"2024-09-21T11:40:37.097021+0000","last_clean":"2024-09-21T11:40:37.097021+0000","last_became_active":"2024-09-21T11:40:06.348796+0000","last_became_peered":"2024-09-21T11:40:06.348796+0000","last_unstale":"2024-09-21T11:40:37.097021+0000","last_undegraded":"2024-09-21T11:40:37.097021+0000","last_fullsized":"2024-09-21T11:40:37.097021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.850881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089414+0000","last_change":"2024-09-21T11:40:00.309865+0000","last_active":"2024-09-21T11:40:37.089414+0000","last_peered":"2024-09-21T11:40:37.089414+0000","last_clean":"2024-09-21T11:40:37.089414+0000","last_became_active":"2024-09-21T11:40:00.309598+0000","last_became_peered":"2024-09-21T11:40:00.309598+0000","last_unstale":"2024-09-21T11:40:37.089414+0000","last_undegraded":"2024-09-21T11:40:37.089414+0000","last_fullsized":"2024-09-21T11:40:37.089414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.604856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":60,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092528+0000","last_change":"2024-09-21T11:40:02.318498+0000","last_active":"2024-09-21T11:40:37.092528+0000","last_peered":"2024-09-21T11:40:37.092528+0000","last_clean":"2024-09-21T11:40:37.092528+0000","last_became_active":"2024-09-21T11:40:02.318160+0000","last_became_peered":"2024-09-21T11:40:02.318160+0000","last_unstale":"2024-09-21T11:40:37.092528+0000","last_undegraded":"2024-09-21T11:40:37.092528+0000","last_fullsized":"2024-09-21T11:40:37.092528+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.280806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116971+0000","last_change":"2024-09-21T11:40:04.340406+0000","last_active":"2024-09-21T11:40:38.116971+0000","last_peered":"2024-09-21T11:40:38.116971+0000","last_clean":"2024-09-21T11:40:38.116971+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:40:38.116971+0000","last_undegraded":"2024-09-21T11:40:38.116971+0000","last_fullsized":"2024-09-21T11:40:38.116971+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.851842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928233997,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28364,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716788,"statfs":{"total":95995035648,"available":95965990912,"internally_reserved":0,"allocated":1368064,"data_stored":898615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462930,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712640,"statfs":{"total":95995035648,"available":95961743360,"internally_reserved":0,"allocated":1355776,"data_stored":898959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724567,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31932,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713220,"statfs":{"total":95995035648,"available":95962337280,"internally_reserved":0,"allocated":761856,"data_stored":307534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920797,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":786432,"data_stored":308760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117027,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":790528,"data_stored":308053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411369,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27768,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717384,"statfs":{"total":95995035648,"available":95966601216,"internally_reserved":0,"allocated":753664,"data_stored":307746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":827392,"data_stored":309396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32536,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712616,"statfs":{"total":95995035648,"available":95961718784,"internally_reserved":0,"allocated":1380352,"data_stored":899330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:40:45.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:40:45.452 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:45.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:40:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:40:46.158 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:40:46.158 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: Deploying daemon alertmanager.a on smithi043 2024-09-21T11:40:46.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: Deploying daemon alertmanager.a on smithi043 2024-09-21T11:40:46.161 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:45 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:40:46.163 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:40:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:40:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.conf 2024-09-21T11:40:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:40:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-21T11:40:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: Updating smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:40:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: Updating smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T11:40:46.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:46.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: Deploying daemon alertmanager.a on smithi043 2024-09-21T11:40:46.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:45 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:40:47.175 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:46 smithi043 ceph-mon[36454]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 588 B/s rd, 0 op/s 2024-09-21T11:40:47.175 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:46 smithi043 ceph-mon[36454]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:47.176 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:46 smithi043 ceph-mon[29710]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 588 B/s rd, 0 op/s 2024-09-21T11:40:47.176 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:46 smithi043 ceph-mon[29710]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:47.204 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-21T11:40:45.691272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":883,"num_read_kb":740,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244872,"kb_used_data":7836,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716344,"statfs":{"total":767960285184,"available":767709536256,"internally_reserved":0,"allocated":8024064,"data_stored":4238393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.695807"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116965+0000","last_change":"2024-09-21T11:40:00.313497+0000","last_active":"2024-09-21T11:40:38.116965+0000","last_peered":"2024-09-21T11:40:38.116965+0000","last_clean":"2024-09-21T11:40:38.116965+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:40:38.116965+0000","last_undegraded":"2024-09-21T11:40:38.116965+0000","last_fullsized":"2024-09-21T11:40:38.116965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.636264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116915+0000","last_change":"2024-09-21T11:40:02.319143+0000","last_active":"2024-09-21T11:40:38.116915+0000","last_peered":"2024-09-21T11:40:38.116915+0000","last_clean":"2024-09-21T11:40:38.116915+0000","last_became_active":"2024-09-21T11:40:02.318639+0000","last_became_peered":"2024-09-21T11:40:02.318639+0000","last_unstale":"2024-09-21T11:40:38.116915+0000","last_undegraded":"2024-09-21T11:40:38.116915+0000","last_fullsized":"2024-09-21T11:40:38.116915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.717901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096596+0000","last_change":"2024-09-21T11:40:04.331203+0000","last_active":"2024-09-21T11:40:37.096596+0000","last_peered":"2024-09-21T11:40:37.096596+0000","last_clean":"2024-09-21T11:40:37.096596+0000","last_became_active":"2024-09-21T11:40:04.330879+0000","last_became_peered":"2024-09-21T11:40:04.330879+0000","last_unstale":"2024-09-21T11:40:37.096596+0000","last_undegraded":"2024-09-21T11:40:37.096596+0000","last_fullsized":"2024-09-21T11:40:37.096596+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.860431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117023+0000","last_change":"2024-09-21T11:40:06.346317+0000","last_active":"2024-09-21T11:40:38.117023+0000","last_peered":"2024-09-21T11:40:38.117023+0000","last_clean":"2024-09-21T11:40:38.117023+0000","last_became_active":"2024-09-21T11:40:06.345995+0000","last_became_peered":"2024-09-21T11:40:06.345995+0000","last_unstale":"2024-09-21T11:40:38.117023+0000","last_undegraded":"2024-09-21T11:40:38.117023+0000","last_fullsized":"2024-09-21T11:40:38.117023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.419437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090252+0000","last_change":"2024-09-21T11:40:06.489876+0000","last_active":"2024-09-21T11:40:37.090252+0000","last_peered":"2024-09-21T11:40:37.090252+0000","last_clean":"2024-09-21T11:40:37.090252+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:40:37.090252+0000","last_undegraded":"2024-09-21T11:40:37.090252+0000","last_fullsized":"2024-09-21T11:40:37.090252+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.939900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090828+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:40:37.090828+0000","last_peered":"2024-09-21T11:40:37.090828+0000","last_clean":"2024-09-21T11:40:37.090828+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:40:37.090828+0000","last_undegraded":"2024-09-21T11:40:37.090828+0000","last_fullsized":"2024-09-21T11:40:37.090828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089985+0000","last_change":"2024-09-21T11:40:02.315669+0000","last_active":"2024-09-21T11:40:37.089985+0000","last_peered":"2024-09-21T11:40:37.089985+0000","last_clean":"2024-09-21T11:40:37.089985+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:40:37.089985+0000","last_undegraded":"2024-09-21T11:40:37.089985+0000","last_fullsized":"2024-09-21T11:40:37.089985+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.511141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116814+0000","last_change":"2024-09-21T11:40:04.326924+0000","last_active":"2024-09-21T11:40:38.116814+0000","last_peered":"2024-09-21T11:40:38.116814+0000","last_clean":"2024-09-21T11:40:38.116814+0000","last_became_active":"2024-09-21T11:40:04.326663+0000","last_became_peered":"2024-09-21T11:40:04.326663+0000","last_unstale":"2024-09-21T11:40:38.116814+0000","last_undegraded":"2024-09-21T11:40:38.116814+0000","last_fullsized":"2024-09-21T11:40:38.116814+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:04.814146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119541+0000","last_change":"2024-09-21T11:40:00.314157+0000","last_active":"2024-09-21T11:40:38.119541+0000","last_peered":"2024-09-21T11:40:38.119541+0000","last_clean":"2024-09-21T11:40:38.119541+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:40:38.119541+0000","last_undegraded":"2024-09-21T11:40:38.119541+0000","last_fullsized":"2024-09-21T11:40:38.119541+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.659277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118645+0000","last_change":"2024-09-21T11:40:02.314170+0000","last_active":"2024-09-21T11:40:38.118645+0000","last_peered":"2024-09-21T11:40:38.118645+0000","last_clean":"2024-09-21T11:40:38.118645+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:40:38.118645+0000","last_undegraded":"2024-09-21T11:40:38.118645+0000","last_fullsized":"2024-09-21T11:40:38.118645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.670327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117406+0000","last_change":"2024-09-21T11:40:04.339616+0000","last_active":"2024-09-21T11:40:38.117406+0000","last_peered":"2024-09-21T11:40:38.117406+0000","last_clean":"2024-09-21T11:40:38.117406+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:40:38.117406+0000","last_undegraded":"2024-09-21T11:40:38.117406+0000","last_fullsized":"2024-09-21T11:40:38.117406+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.214530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116334+0000","last_change":"2024-09-21T11:40:06.344622+0000","last_active":"2024-09-21T11:40:38.116334+0000","last_peered":"2024-09-21T11:40:38.116334+0000","last_clean":"2024-09-21T11:40:38.116334+0000","last_became_active":"2024-09-21T11:40:06.344291+0000","last_became_peered":"2024-09-21T11:40:06.344291+0000","last_unstale":"2024-09-21T11:40:38.116334+0000","last_undegraded":"2024-09-21T11:40:38.116334+0000","last_fullsized":"2024-09-21T11:40:38.116334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.653671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119780+0000","last_change":"2024-09-21T11:40:00.315137+0000","last_active":"2024-09-21T11:40:38.119780+0000","last_peered":"2024-09-21T11:40:38.119780+0000","last_clean":"2024-09-21T11:40:38.119780+0000","last_became_active":"2024-09-21T11:40:00.314838+0000","last_became_peered":"2024-09-21T11:40:00.314838+0000","last_unstale":"2024-09-21T11:40:38.119780+0000","last_undegraded":"2024-09-21T11:40:38.119780+0000","last_fullsized":"2024-09-21T11:40:38.119780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:40:59.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118997+0000","last_change":"2024-09-21T11:40:02.320524+0000","last_active":"2024-09-21T11:40:38.118997+0000","last_peered":"2024-09-21T11:40:38.118997+0000","last_clean":"2024-09-21T11:40:38.118997+0000","last_became_active":"2024-09-21T11:40:02.320184+0000","last_became_peered":"2024-09-21T11:40:02.320184+0000","last_unstale":"2024-09-21T11:40:38.118997+0000","last_undegraded":"2024-09-21T11:40:38.118997+0000","last_fullsized":"2024-09-21T11:40:38.118997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.631864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117300+0000","last_change":"2024-09-21T11:40:04.333696+0000","last_active":"2024-09-21T11:40:38.117300+0000","last_peered":"2024-09-21T11:40:38.117300+0000","last_clean":"2024-09-21T11:40:38.117300+0000","last_became_active":"2024-09-21T11:40:04.333406+0000","last_became_peered":"2024-09-21T11:40:04.333406+0000","last_unstale":"2024-09-21T11:40:38.117300+0000","last_undegraded":"2024-09-21T11:40:38.117300+0000","last_fullsized":"2024-09-21T11:40:38.117300+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.055519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117137+0000","last_change":"2024-09-21T11:40:06.349798+0000","last_active":"2024-09-21T11:40:38.117137+0000","last_peered":"2024-09-21T11:40:38.117137+0000","last_clean":"2024-09-21T11:40:38.117137+0000","last_became_active":"2024-09-21T11:40:06.349538+0000","last_became_peered":"2024-09-21T11:40:06.349538+0000","last_unstale":"2024-09-21T11:40:38.117137+0000","last_undegraded":"2024-09-21T11:40:38.117137+0000","last_fullsized":"2024-09-21T11:40:38.117137+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:34.219537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118957+0000","last_change":"2024-09-21T11:40:06.487919+0000","last_active":"2024-09-21T11:40:38.118957+0000","last_peered":"2024-09-21T11:40:38.118957+0000","last_clean":"2024-09-21T11:40:38.118957+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:40:38.118957+0000","last_undegraded":"2024-09-21T11:40:38.118957+0000","last_fullsized":"2024-09-21T11:40:38.118957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:18.602419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117461+0000","last_change":"2024-09-21T11:40:00.312124+0000","last_active":"2024-09-21T11:40:38.117461+0000","last_peered":"2024-09-21T11:40:38.117461+0000","last_clean":"2024-09-21T11:40:38.117461+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:40:38.117461+0000","last_undegraded":"2024-09-21T11:40:38.117461+0000","last_fullsized":"2024-09-21T11:40:38.117461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.023370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089986+0000","last_change":"2024-09-21T11:40:02.319850+0000","last_active":"2024-09-21T11:40:37.089986+0000","last_peered":"2024-09-21T11:40:37.089986+0000","last_clean":"2024-09-21T11:40:37.089986+0000","last_became_active":"2024-09-21T11:40:02.319153+0000","last_became_peered":"2024-09-21T11:40:02.319153+0000","last_unstale":"2024-09-21T11:40:37.089986+0000","last_undegraded":"2024-09-21T11:40:37.089986+0000","last_fullsized":"2024-09-21T11:40:37.089986+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.765972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097393+0000","last_change":"2024-09-21T11:40:04.339562+0000","last_active":"2024-09-21T11:40:37.097393+0000","last_peered":"2024-09-21T11:40:37.097393+0000","last_clean":"2024-09-21T11:40:37.097393+0000","last_became_active":"2024-09-21T11:40:04.339222+0000","last_became_peered":"2024-09-21T11:40:04.339222+0000","last_unstale":"2024-09-21T11:40:37.097393+0000","last_undegraded":"2024-09-21T11:40:37.097393+0000","last_fullsized":"2024-09-21T11:40:37.097393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.022703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090832+0000","last_change":"2024-09-21T11:40:06.487933+0000","last_active":"2024-09-21T11:40:37.090832+0000","last_peered":"2024-09-21T11:40:37.090832+0000","last_clean":"2024-09-21T11:40:37.090832+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:40:37.090832+0000","last_undegraded":"2024-09-21T11:40:37.090832+0000","last_fullsized":"2024-09-21T11:40:37.090832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.209907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119180+0000","last_change":"2024-09-21T11:40:00.312534+0000","last_active":"2024-09-21T11:40:38.119180+0000","last_peered":"2024-09-21T11:40:38.119180+0000","last_clean":"2024-09-21T11:40:38.119180+0000","last_became_active":"2024-09-21T11:40:00.312239+0000","last_became_peered":"2024-09-21T11:40:00.312239+0000","last_unstale":"2024-09-21T11:40:38.119180+0000","last_undegraded":"2024-09-21T11:40:38.119180+0000","last_fullsized":"2024-09-21T11:40:38.119180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.443761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090511+0000","last_change":"2024-09-21T11:40:02.324039+0000","last_active":"2024-09-21T11:40:37.090511+0000","last_peered":"2024-09-21T11:40:37.090511+0000","last_clean":"2024-09-21T11:40:37.090511+0000","last_became_active":"2024-09-21T11:40:02.323784+0000","last_became_peered":"2024-09-21T11:40:02.323784+0000","last_unstale":"2024-09-21T11:40:37.090511+0000","last_undegraded":"2024-09-21T11:40:37.090511+0000","last_fullsized":"2024-09-21T11:40:37.090511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.707671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118544+0000","last_change":"2024-09-21T11:40:04.328865+0000","last_active":"2024-09-21T11:40:38.118544+0000","last_peered":"2024-09-21T11:40:38.118544+0000","last_clean":"2024-09-21T11:40:38.118544+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:40:38.118544+0000","last_undegraded":"2024-09-21T11:40:38.118544+0000","last_fullsized":"2024-09-21T11:40:38.118544+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.407835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118453+0000","last_change":"2024-09-21T11:40:06.340320+0000","last_active":"2024-09-21T11:40:38.118453+0000","last_peered":"2024-09-21T11:40:38.118453+0000","last_clean":"2024-09-21T11:40:38.118453+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:40:38.118453+0000","last_undegraded":"2024-09-21T11:40:38.118453+0000","last_fullsized":"2024-09-21T11:40:38.118453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.731856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096791+0000","last_change":"2024-09-21T11:40:00.308535+0000","last_active":"2024-09-21T11:40:37.096791+0000","last_peered":"2024-09-21T11:40:37.096791+0000","last_clean":"2024-09-21T11:40:37.096791+0000","last_became_active":"2024-09-21T11:40:00.308219+0000","last_became_peered":"2024-09-21T11:40:00.308219+0000","last_unstale":"2024-09-21T11:40:37.096791+0000","last_undegraded":"2024-09-21T11:40:37.096791+0000","last_fullsized":"2024-09-21T11:40:37.096791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.036879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116981+0000","last_change":"2024-09-21T11:40:02.400244+0000","last_active":"2024-09-21T11:40:38.116981+0000","last_peered":"2024-09-21T11:40:38.116981+0000","last_clean":"2024-09-21T11:40:38.116981+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:40:38.116981+0000","last_undegraded":"2024-09-21T11:40:38.116981+0000","last_fullsized":"2024-09-21T11:40:38.116981+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.785487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092593+0000","last_change":"2024-09-21T11:40:04.328515+0000","last_active":"2024-09-21T11:40:37.092593+0000","last_peered":"2024-09-21T11:40:37.092593+0000","last_clean":"2024-09-21T11:40:37.092593+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:40:37.092593+0000","last_undegraded":"2024-09-21T11:40:37.092593+0000","last_fullsized":"2024-09-21T11:40:37.092593+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:03.806494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":66,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097724+0000","last_change":"2024-09-21T11:40:02.323539+0000","last_active":"2024-09-21T11:40:37.097724+0000","last_peered":"2024-09-21T11:40:37.097724+0000","last_clean":"2024-09-21T11:40:37.097724+0000","last_became_active":"2024-09-21T11:40:02.323204+0000","last_became_peered":"2024-09-21T11:40:02.323204+0000","last_unstale":"2024-09-21T11:40:37.097724+0000","last_undegraded":"2024-09-21T11:40:37.097724+0000","last_fullsized":"2024-09-21T11:40:37.097724+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089484+0000","last_change":"2024-09-21T11:40:00.315307+0000","last_active":"2024-09-21T11:40:37.089484+0000","last_peered":"2024-09-21T11:40:37.089484+0000","last_clean":"2024-09-21T11:40:37.089484+0000","last_became_active":"2024-09-21T11:40:00.315015+0000","last_became_peered":"2024-09-21T11:40:00.315015+0000","last_unstale":"2024-09-21T11:40:37.089484+0000","last_undegraded":"2024-09-21T11:40:37.089484+0000","last_fullsized":"2024-09-21T11:40:37.089484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.959497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119140+0000","last_change":"2024-09-21T11:40:04.342645+0000","last_active":"2024-09-21T11:40:38.119140+0000","last_peered":"2024-09-21T11:40:38.119140+0000","last_clean":"2024-09-21T11:40:38.119140+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:40:38.119140+0000","last_undegraded":"2024-09-21T11:40:38.119140+0000","last_fullsized":"2024-09-21T11:40:38.119140+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.259518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118232+0000","last_change":"2024-09-21T11:40:06.345062+0000","last_active":"2024-09-21T11:40:38.118232+0000","last_peered":"2024-09-21T11:40:38.118232+0000","last_clean":"2024-09-21T11:40:38.118232+0000","last_became_active":"2024-09-21T11:40:06.344796+0000","last_became_peered":"2024-09-21T11:40:06.344796+0000","last_unstale":"2024-09-21T11:40:38.118232+0000","last_undegraded":"2024-09-21T11:40:38.118232+0000","last_fullsized":"2024-09-21T11:40:38.118232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.403408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":74,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089287+0000","last_change":"2024-09-21T11:40:02.399334+0000","last_active":"2024-09-21T11:40:37.089287+0000","last_peered":"2024-09-21T11:40:37.089287+0000","last_clean":"2024-09-21T11:40:37.089287+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:40:37.089287+0000","last_undegraded":"2024-09-21T11:40:37.089287+0000","last_fullsized":"2024-09-21T11:40:37.089287+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.386531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091103+0000","last_change":"2024-09-21T11:40:00.310330+0000","last_active":"2024-09-21T11:40:37.091103+0000","last_peered":"2024-09-21T11:40:37.091103+0000","last_clean":"2024-09-21T11:40:37.091103+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:40:37.091103+0000","last_undegraded":"2024-09-21T11:40:37.091103+0000","last_fullsized":"2024-09-21T11:40:37.091103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.371930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117521+0000","last_change":"2024-09-21T11:40:04.330180+0000","last_active":"2024-09-21T11:40:38.117521+0000","last_peered":"2024-09-21T11:40:38.117521+0000","last_clean":"2024-09-21T11:40:38.117521+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:40:38.117521+0000","last_undegraded":"2024-09-21T11:40:38.117521+0000","last_fullsized":"2024-09-21T11:40:38.117521+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.570842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118790+0000","last_change":"2024-09-21T11:40:06.343742+0000","last_active":"2024-09-21T11:40:38.118790+0000","last_peered":"2024-09-21T11:40:38.118790+0000","last_clean":"2024-09-21T11:40:38.118790+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:40:38.118790+0000","last_undegraded":"2024-09-21T11:40:38.118790+0000","last_fullsized":"2024-09-21T11:40:38.118790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.217075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":63,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117749+0000","last_change":"2024-09-21T11:40:02.312574+0000","last_active":"2024-09-21T11:40:38.117749+0000","last_peered":"2024-09-21T11:40:38.117749+0000","last_clean":"2024-09-21T11:40:38.117749+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:40:38.117749+0000","last_undegraded":"2024-09-21T11:40:38.117749+0000","last_fullsized":"2024-09-21T11:40:38.117749+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.497994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117503+0000","last_change":"2024-09-21T11:40:00.312084+0000","last_active":"2024-09-21T11:40:38.117503+0000","last_peered":"2024-09-21T11:40:38.117503+0000","last_clean":"2024-09-21T11:40:38.117503+0000","last_became_active":"2024-09-21T11:40:00.311813+0000","last_became_peered":"2024-09-21T11:40:00.311813+0000","last_unstale":"2024-09-21T11:40:38.117503+0000","last_undegraded":"2024-09-21T11:40:38.117503+0000","last_fullsized":"2024-09-21T11:40:38.117503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.090287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089147+0000","last_change":"2024-09-21T11:40:04.324572+0000","last_active":"2024-09-21T11:40:37.089147+0000","last_peered":"2024-09-21T11:40:37.089147+0000","last_clean":"2024-09-21T11:40:37.089147+0000","last_became_active":"2024-09-21T11:40:04.324272+0000","last_became_peered":"2024-09-21T11:40:04.324272+0000","last_unstale":"2024-09-21T11:40:37.089147+0000","last_undegraded":"2024-09-21T11:40:37.089147+0000","last_fullsized":"2024-09-21T11:40:37.089147+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.830113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117696+0000","last_change":"2024-09-21T11:40:06.489239+0000","last_active":"2024-09-21T11:40:38.117696+0000","last_peered":"2024-09-21T11:40:38.117696+0000","last_clean":"2024-09-21T11:40:38.117696+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:40:38.117696+0000","last_undegraded":"2024-09-21T11:40:38.117696+0000","last_fullsized":"2024-09-21T11:40:38.117696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.580118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097096+0000","last_change":"2024-09-21T11:40:02.318985+0000","last_active":"2024-09-21T11:40:37.097096+0000","last_peered":"2024-09-21T11:40:37.097096+0000","last_clean":"2024-09-21T11:40:37.097096+0000","last_became_active":"2024-09-21T11:40:02.318270+0000","last_became_peered":"2024-09-21T11:40:02.318270+0000","last_unstale":"2024-09-21T11:40:37.097096+0000","last_undegraded":"2024-09-21T11:40:37.097096+0000","last_fullsized":"2024-09-21T11:40:37.097096+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.457674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119649+0000","last_change":"2024-09-21T11:40:00.318794+0000","last_active":"2024-09-21T11:40:38.119649+0000","last_peered":"2024-09-21T11:40:38.119649+0000","last_clean":"2024-09-21T11:40:38.119649+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:40:38.119649+0000","last_undegraded":"2024-09-21T11:40:38.119649+0000","last_fullsized":"2024-09-21T11:40:38.119649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.580077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116819+0000","last_change":"2024-09-21T11:40:04.335582+0000","last_active":"2024-09-21T11:40:38.116819+0000","last_peered":"2024-09-21T11:40:38.116819+0000","last_clean":"2024-09-21T11:40:38.116819+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:40:38.116819+0000","last_undegraded":"2024-09-21T11:40:38.116819+0000","last_fullsized":"2024-09-21T11:40:38.116819+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:03.386490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116484+0000","last_change":"2024-09-21T11:40:06.352201+0000","last_active":"2024-09-21T11:40:38.116484+0000","last_peered":"2024-09-21T11:40:38.116484+0000","last_clean":"2024-09-21T11:40:38.116484+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:40:38.116484+0000","last_undegraded":"2024-09-21T11:40:38.116484+0000","last_fullsized":"2024-09-21T11:40:38.116484+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.165004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":77,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116928+0000","last_change":"2024-09-21T11:40:02.399117+0000","last_active":"2024-09-21T11:40:38.116928+0000","last_peered":"2024-09-21T11:40:38.116928+0000","last_clean":"2024-09-21T11:40:38.116928+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:40:38.116928+0000","last_undegraded":"2024-09-21T11:40:38.116928+0000","last_fullsized":"2024-09-21T11:40:38.116928+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.772060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097142+0000","last_change":"2024-09-21T11:40:00.311519+0000","last_active":"2024-09-21T11:40:37.097142+0000","last_peered":"2024-09-21T11:40:37.097142+0000","last_clean":"2024-09-21T11:40:37.097142+0000","last_became_active":"2024-09-21T11:40:00.311211+0000","last_became_peered":"2024-09-21T11:40:00.311211+0000","last_unstale":"2024-09-21T11:40:37.097142+0000","last_undegraded":"2024-09-21T11:40:37.097142+0000","last_fullsized":"2024-09-21T11:40:37.097142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.982610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091780+0000","last_change":"2024-09-21T11:40:04.339202+0000","last_active":"2024-09-21T11:40:37.091780+0000","last_peered":"2024-09-21T11:40:37.091780+0000","last_clean":"2024-09-21T11:40:37.091780+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:40:37.091780+0000","last_undegraded":"2024-09-21T11:40:37.091780+0000","last_fullsized":"2024-09-21T11:40:37.091780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.313979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096533+0000","last_change":"2024-09-21T11:40:06.487976+0000","last_active":"2024-09-21T11:40:37.096533+0000","last_peered":"2024-09-21T11:40:37.096533+0000","last_clean":"2024-09-21T11:40:37.096533+0000","last_became_active":"2024-09-21T11:40:06.487644+0000","last_became_peered":"2024-09-21T11:40:06.487644+0000","last_unstale":"2024-09-21T11:40:37.096533+0000","last_undegraded":"2024-09-21T11:40:37.096533+0000","last_fullsized":"2024-09-21T11:40:37.096533+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.318979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":99,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:40.365734+0000","last_change":"2024-09-21T11:40:02.320105+0000","last_active":"2024-09-21T11:40:40.365734+0000","last_peered":"2024-09-21T11:40:40.365734+0000","last_clean":"2024-09-21T11:40:40.365734+0000","last_became_active":"2024-09-21T11:40:02.319770+0000","last_became_peered":"2024-09-21T11:40:02.319770+0000","last_unstale":"2024-09-21T11:40:40.365734+0000","last_undegraded":"2024-09-21T11:40:40.365734+0000","last_fullsized":"2024-09-21T11:40:40.365734+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.672594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119401+0000","last_change":"2024-09-21T11:40:00.314186+0000","last_active":"2024-09-21T11:40:38.119401+0000","last_peered":"2024-09-21T11:40:38.119401+0000","last_clean":"2024-09-21T11:40:38.119401+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:40:38.119401+0000","last_undegraded":"2024-09-21T11:40:38.119401+0000","last_fullsized":"2024-09-21T11:40:38.119401+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:14.587338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117801+0000","last_change":"2024-09-21T11:40:02.311375+0000","last_active":"2024-09-21T11:40:38.117801+0000","last_peered":"2024-09-21T11:40:38.117801+0000","last_clean":"2024-09-21T11:40:38.117801+0000","last_became_active":"2024-09-21T11:39:59.303309+0000","last_became_peered":"2024-09-21T11:39:59.303309+0000","last_unstale":"2024-09-21T11:40:38.117801+0000","last_undegraded":"2024-09-21T11:40:38.117801+0000","last_fullsized":"2024-09-21T11:40:38.117801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.322560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116887+0000","last_change":"2024-09-21T11:40:04.337767+0000","last_active":"2024-09-21T11:40:38.116887+0000","last_peered":"2024-09-21T11:40:38.116887+0000","last_clean":"2024-09-21T11:40:38.116887+0000","last_became_active":"2024-09-21T11:40:04.337422+0000","last_became_peered":"2024-09-21T11:40:04.337422+0000","last_unstale":"2024-09-21T11:40:38.116887+0000","last_undegraded":"2024-09-21T11:40:38.116887+0000","last_fullsized":"2024-09-21T11:40:38.116887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.195058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090431+0000","last_change":"2024-09-21T11:40:06.350784+0000","last_active":"2024-09-21T11:40:37.090431+0000","last_peered":"2024-09-21T11:40:37.090431+0000","last_clean":"2024-09-21T11:40:37.090431+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:40:37.090431+0000","last_undegraded":"2024-09-21T11:40:37.090431+0000","last_fullsized":"2024-09-21T11:40:37.090431+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.561650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097771+0000","last_change":"2024-09-21T11:40:02.324944+0000","last_active":"2024-09-21T11:40:37.097771+0000","last_peered":"2024-09-21T11:40:37.097771+0000","last_clean":"2024-09-21T11:40:37.097771+0000","last_became_active":"2024-09-21T11:40:02.324667+0000","last_became_peered":"2024-09-21T11:40:02.324667+0000","last_unstale":"2024-09-21T11:40:37.097771+0000","last_undegraded":"2024-09-21T11:40:37.097771+0000","last_fullsized":"2024-09-21T11:40:37.097771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.447522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096780+0000","last_change":"2024-09-21T11:40:00.313632+0000","last_active":"2024-09-21T11:40:37.096780+0000","last_peered":"2024-09-21T11:40:37.096780+0000","last_clean":"2024-09-21T11:40:37.096780+0000","last_became_active":"2024-09-21T11:40:00.313231+0000","last_became_peered":"2024-09-21T11:40:00.313231+0000","last_unstale":"2024-09-21T11:40:37.096780+0000","last_undegraded":"2024-09-21T11:40:37.096780+0000","last_fullsized":"2024-09-21T11:40:37.096780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.689078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089293+0000","last_change":"2024-09-21T11:40:02.305143+0000","last_active":"2024-09-21T11:40:37.089293+0000","last_peered":"2024-09-21T11:40:37.089293+0000","last_clean":"2024-09-21T11:40:37.089293+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:40:37.089293+0000","last_undegraded":"2024-09-21T11:40:37.089293+0000","last_fullsized":"2024-09-21T11:40:37.089293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.513240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089829+0000","last_change":"2024-09-21T11:40:04.332073+0000","last_active":"2024-09-21T11:40:37.089829+0000","last_peered":"2024-09-21T11:40:37.089829+0000","last_clean":"2024-09-21T11:40:37.089829+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:40:37.089829+0000","last_undegraded":"2024-09-21T11:40:37.089829+0000","last_fullsized":"2024-09-21T11:40:37.089829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.887948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119154+0000","last_change":"2024-09-21T11:40:06.487757+0000","last_active":"2024-09-21T11:40:38.119154+0000","last_peered":"2024-09-21T11:40:38.119154+0000","last_clean":"2024-09-21T11:40:38.119154+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:40:38.119154+0000","last_undegraded":"2024-09-21T11:40:38.119154+0000","last_fullsized":"2024-09-21T11:40:38.119154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.207444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117442+0000","last_change":"2024-09-21T11:40:02.320851+0000","last_active":"2024-09-21T11:40:38.117442+0000","last_peered":"2024-09-21T11:40:38.117442+0000","last_clean":"2024-09-21T11:40:38.117442+0000","last_became_active":"2024-09-21T11:40:02.320522+0000","last_became_peered":"2024-09-21T11:40:02.320522+0000","last_unstale":"2024-09-21T11:40:38.117442+0000","last_undegraded":"2024-09-21T11:40:38.117442+0000","last_fullsized":"2024-09-21T11:40:38.117442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:30.451592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116473+0000","last_change":"2024-09-21T11:40:00.312222+0000","last_active":"2024-09-21T11:40:38.116473+0000","last_peered":"2024-09-21T11:40:38.116473+0000","last_clean":"2024-09-21T11:40:38.116473+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:40:38.116473+0000","last_undegraded":"2024-09-21T11:40:38.116473+0000","last_fullsized":"2024-09-21T11:40:38.116473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.238210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":91,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.735310+0000","last_change":"2024-09-21T11:40:02.398538+0000","last_active":"2024-09-21T11:40:38.735310+0000","last_peered":"2024-09-21T11:40:38.735310+0000","last_clean":"2024-09-21T11:40:38.735310+0000","last_became_active":"2024-09-21T11:39:59.306320+0000","last_became_peered":"2024-09-21T11:39:59.306320+0000","last_unstale":"2024-09-21T11:40:38.735310+0000","last_undegraded":"2024-09-21T11:40:38.735310+0000","last_fullsized":"2024-09-21T11:40:38.735310+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:18.630069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119601+0000","last_change":"2024-09-21T11:40:04.339615+0000","last_active":"2024-09-21T11:40:38.119601+0000","last_peered":"2024-09-21T11:40:38.119601+0000","last_clean":"2024-09-21T11:40:38.119601+0000","last_became_active":"2024-09-21T11:40:04.339331+0000","last_became_peered":"2024-09-21T11:40:04.339331+0000","last_unstale":"2024-09-21T11:40:38.119601+0000","last_undegraded":"2024-09-21T11:40:38.119601+0000","last_fullsized":"2024-09-21T11:40:38.119601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:33.310635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096709+0000","last_change":"2024-09-21T11:40:06.352155+0000","last_active":"2024-09-21T11:40:37.096709+0000","last_peered":"2024-09-21T11:40:37.096709+0000","last_clean":"2024-09-21T11:40:37.096709+0000","last_became_active":"2024-09-21T11:40:06.351777+0000","last_became_peered":"2024-09-21T11:40:06.351777+0000","last_unstale":"2024-09-21T11:40:37.096709+0000","last_undegraded":"2024-09-21T11:40:37.096709+0000","last_fullsized":"2024-09-21T11:40:37.096709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.594495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":72,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092570+0000","last_change":"2024-09-21T11:40:02.399268+0000","last_active":"2024-09-21T11:40:37.092570+0000","last_peered":"2024-09-21T11:40:37.092570+0000","last_clean":"2024-09-21T11:40:37.092570+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:40:37.092570+0000","last_undegraded":"2024-09-21T11:40:37.092570+0000","last_fullsized":"2024-09-21T11:40:37.092570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.058592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090286+0000","last_change":"2024-09-21T11:40:00.309464+0000","last_active":"2024-09-21T11:40:37.090286+0000","last_peered":"2024-09-21T11:40:37.090286+0000","last_clean":"2024-09-21T11:40:37.090286+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:40:37.090286+0000","last_undegraded":"2024-09-21T11:40:37.090286+0000","last_fullsized":"2024-09-21T11:40:37.090286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.615608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":90,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118342+0000","last_change":"2024-09-21T11:40:12.500662+0000","last_active":"2024-09-21T11:40:38.118342+0000","last_peered":"2024-09-21T11:40:38.118342+0000","last_clean":"2024-09-21T11:40:38.118342+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:40:38.118342+0000","last_undegraded":"2024-09-21T11:40:38.118342+0000","last_fullsized":"2024-09-21T11:40:38.118342+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'72","last_scrub_stamp":"2024-09-21T11:40:12.500606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:40:12.500606+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.061712+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118696+0000","last_change":"2024-09-21T11:40:04.333889+0000","last_active":"2024-09-21T11:40:38.118696+0000","last_peered":"2024-09-21T11:40:38.118696+0000","last_clean":"2024-09-21T11:40:38.118696+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:40:38.118696+0000","last_undegraded":"2024-09-21T11:40:38.118696+0000","last_fullsized":"2024-09-21T11:40:38.118696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.202161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119447+0000","last_change":"2024-09-21T11:40:06.349857+0000","last_active":"2024-09-21T11:40:38.119447+0000","last_peered":"2024-09-21T11:40:38.119447+0000","last_clean":"2024-09-21T11:40:38.119447+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:40:38.119447+0000","last_undegraded":"2024-09-21T11:40:38.119447+0000","last_fullsized":"2024-09-21T11:40:38.119447+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.149814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119188+0000","last_change":"2024-09-21T11:40:02.319709+0000","last_active":"2024-09-21T11:40:38.119188+0000","last_peered":"2024-09-21T11:40:38.119188+0000","last_clean":"2024-09-21T11:40:38.119188+0000","last_became_active":"2024-09-21T11:40:02.319422+0000","last_became_peered":"2024-09-21T11:40:02.319422+0000","last_unstale":"2024-09-21T11:40:38.119188+0000","last_undegraded":"2024-09-21T11:40:38.119188+0000","last_fullsized":"2024-09-21T11:40:38.119188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.852025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117859+0000","last_change":"2024-09-21T11:40:00.312057+0000","last_active":"2024-09-21T11:40:38.117859+0000","last_peered":"2024-09-21T11:40:38.117859+0000","last_clean":"2024-09-21T11:40:38.117859+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:40:38.117859+0000","last_undegraded":"2024-09-21T11:40:38.117859+0000","last_fullsized":"2024-09-21T11:40:38.117859+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.891862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119217+0000","last_change":"2024-09-21T11:40:04.329480+0000","last_active":"2024-09-21T11:40:38.119217+0000","last_peered":"2024-09-21T11:40:38.119217+0000","last_clean":"2024-09-21T11:40:38.119217+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:40:38.119217+0000","last_undegraded":"2024-09-21T11:40:38.119217+0000","last_fullsized":"2024-09-21T11:40:38.119217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090387+0000","last_change":"2024-09-21T11:40:06.487868+0000","last_active":"2024-09-21T11:40:37.090387+0000","last_peered":"2024-09-21T11:40:37.090387+0000","last_clean":"2024-09-21T11:40:37.090387+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:40:37.090387+0000","last_undegraded":"2024-09-21T11:40:37.090387+0000","last_fullsized":"2024-09-21T11:40:37.090387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.512063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":70,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118993+0000","last_change":"2024-09-21T11:40:02.319361+0000","last_active":"2024-09-21T11:40:38.118993+0000","last_peered":"2024-09-21T11:40:38.118993+0000","last_clean":"2024-09-21T11:40:38.118993+0000","last_became_active":"2024-09-21T11:40:02.318902+0000","last_became_peered":"2024-09-21T11:40:02.318902+0000","last_unstale":"2024-09-21T11:40:38.118993+0000","last_undegraded":"2024-09-21T11:40:38.118993+0000","last_fullsized":"2024-09-21T11:40:38.118993+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.422023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092089+0000","last_change":"2024-09-21T11:40:00.313702+0000","last_active":"2024-09-21T11:40:37.092089+0000","last_peered":"2024-09-21T11:40:37.092089+0000","last_clean":"2024-09-21T11:40:37.092089+0000","last_became_active":"2024-09-21T11:40:00.313432+0000","last_became_peered":"2024-09-21T11:40:00.313432+0000","last_unstale":"2024-09-21T11:40:37.092089+0000","last_undegraded":"2024-09-21T11:40:37.092089+0000","last_fullsized":"2024-09-21T11:40:37.092089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:21.740308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096816+0000","last_change":"2024-09-21T11:40:04.342081+0000","last_active":"2024-09-21T11:40:37.096816+0000","last_peered":"2024-09-21T11:40:37.096816+0000","last_clean":"2024-09-21T11:40:37.096816+0000","last_became_active":"2024-09-21T11:40:04.341743+0000","last_became_peered":"2024-09-21T11:40:04.341743+0000","last_unstale":"2024-09-21T11:40:37.096816+0000","last_undegraded":"2024-09-21T11:40:37.096816+0000","last_fullsized":"2024-09-21T11:40:37.096816+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.025793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089044+0000","last_change":"2024-09-21T11:40:06.349130+0000","last_active":"2024-09-21T11:40:37.089044+0000","last_peered":"2024-09-21T11:40:37.089044+0000","last_clean":"2024-09-21T11:40:37.089044+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:40:37.089044+0000","last_undegraded":"2024-09-21T11:40:37.089044+0000","last_fullsized":"2024-09-21T11:40:37.089044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:14.687896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118670+0000","last_change":"2024-09-21T11:40:02.317262+0000","last_active":"2024-09-21T11:40:38.118670+0000","last_peered":"2024-09-21T11:40:38.118670+0000","last_clean":"2024-09-21T11:40:38.118670+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:40:38.118670+0000","last_undegraded":"2024-09-21T11:40:38.118670+0000","last_fullsized":"2024-09-21T11:40:38.118670+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.898962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090545+0000","last_change":"2024-09-21T11:40:00.307721+0000","last_active":"2024-09-21T11:40:37.090545+0000","last_peered":"2024-09-21T11:40:37.090545+0000","last_clean":"2024-09-21T11:40:37.090545+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:40:37.090545+0000","last_undegraded":"2024-09-21T11:40:37.090545+0000","last_fullsized":"2024-09-21T11:40:37.090545+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.478046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:40.365773+0000","last_change":"2024-09-21T11:40:04.332939+0000","last_active":"2024-09-21T11:40:40.365773+0000","last_peered":"2024-09-21T11:40:40.365773+0000","last_clean":"2024-09-21T11:40:40.365773+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:40:40.365773+0000","last_undegraded":"2024-09-21T11:40:40.365773+0000","last_fullsized":"2024-09-21T11:40:40.365773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.250509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118805+0000","last_change":"2024-09-21T11:40:06.346865+0000","last_active":"2024-09-21T11:40:38.118805+0000","last_peered":"2024-09-21T11:40:38.118805+0000","last_clean":"2024-09-21T11:40:38.118805+0000","last_became_active":"2024-09-21T11:40:06.346602+0000","last_became_peered":"2024-09-21T11:40:06.346602+0000","last_unstale":"2024-09-21T11:40:38.118805+0000","last_undegraded":"2024-09-21T11:40:38.118805+0000","last_fullsized":"2024-09-21T11:40:38.118805+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.724328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116816+0000","last_change":"2024-09-21T11:40:02.319787+0000","last_active":"2024-09-21T11:40:38.116816+0000","last_peered":"2024-09-21T11:40:38.116816+0000","last_clean":"2024-09-21T11:40:38.116816+0000","last_became_active":"2024-09-21T11:40:02.319502+0000","last_became_peered":"2024-09-21T11:40:02.319502+0000","last_unstale":"2024-09-21T11:40:38.116816+0000","last_undegraded":"2024-09-21T11:40:38.116816+0000","last_fullsized":"2024-09-21T11:40:38.116816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.504218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118415+0000","last_change":"2024-09-21T11:40:00.318087+0000","last_active":"2024-09-21T11:40:38.118415+0000","last_peered":"2024-09-21T11:40:38.118415+0000","last_clean":"2024-09-21T11:40:38.118415+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:40:38.118415+0000","last_undegraded":"2024-09-21T11:40:38.118415+0000","last_fullsized":"2024-09-21T11:40:38.118415+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.865238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089043+0000","last_change":"2024-09-21T11:40:04.330158+0000","last_active":"2024-09-21T11:40:37.089043+0000","last_peered":"2024-09-21T11:40:37.089043+0000","last_clean":"2024-09-21T11:40:37.089043+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:40:37.089043+0000","last_undegraded":"2024-09-21T11:40:37.089043+0000","last_fullsized":"2024-09-21T11:40:37.089043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.222022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116330+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:40:38.116330+0000","last_peered":"2024-09-21T11:40:38.116330+0000","last_clean":"2024-09-21T11:40:38.116330+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:40:38.116330+0000","last_undegraded":"2024-09-21T11:40:38.116330+0000","last_fullsized":"2024-09-21T11:40:38.116330+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:32.858006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":72,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091924+0000","last_change":"2024-09-21T11:40:02.399340+0000","last_active":"2024-09-21T11:40:37.091924+0000","last_peered":"2024-09-21T11:40:37.091924+0000","last_clean":"2024-09-21T11:40:37.091924+0000","last_became_active":"2024-09-21T11:40:02.398876+0000","last_became_peered":"2024-09-21T11:40:02.398876+0000","last_unstale":"2024-09-21T11:40:37.091924+0000","last_undegraded":"2024-09-21T11:40:37.091924+0000","last_fullsized":"2024-09-21T11:40:37.091924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.043119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119634+0000","last_change":"2024-09-21T11:40:00.314507+0000","last_active":"2024-09-21T11:40:38.119634+0000","last_peered":"2024-09-21T11:40:38.119634+0000","last_clean":"2024-09-21T11:40:38.119634+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:40:38.119634+0000","last_undegraded":"2024-09-21T11:40:38.119634+0000","last_fullsized":"2024-09-21T11:40:38.119634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.755846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089466+0000","last_change":"2024-09-21T11:40:04.341835+0000","last_active":"2024-09-21T11:40:37.089466+0000","last_peered":"2024-09-21T11:40:37.089466+0000","last_clean":"2024-09-21T11:40:37.089466+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:40:37.089466+0000","last_undegraded":"2024-09-21T11:40:37.089466+0000","last_fullsized":"2024-09-21T11:40:37.089466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:21.943269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118201+0000","last_change":"2024-09-21T11:40:06.348279+0000","last_active":"2024-09-21T11:40:38.118201+0000","last_peered":"2024-09-21T11:40:38.118201+0000","last_clean":"2024-09-21T11:40:38.118201+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:40:38.118201+0000","last_undegraded":"2024-09-21T11:40:38.118201+0000","last_fullsized":"2024-09-21T11:40:38.118201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.108944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117650+0000","last_change":"2024-09-21T11:40:02.320045+0000","last_active":"2024-09-21T11:40:38.117650+0000","last_peered":"2024-09-21T11:40:38.117650+0000","last_clean":"2024-09-21T11:40:38.117650+0000","last_became_active":"2024-09-21T11:40:02.319752+0000","last_became_peered":"2024-09-21T11:40:02.319752+0000","last_unstale":"2024-09-21T11:40:38.117650+0000","last_undegraded":"2024-09-21T11:40:38.117650+0000","last_fullsized":"2024-09-21T11:40:38.117650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.042672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.120082+0000","last_change":"2024-09-21T11:40:00.311267+0000","last_active":"2024-09-21T11:40:38.120082+0000","last_peered":"2024-09-21T11:40:38.120082+0000","last_clean":"2024-09-21T11:40:38.120082+0000","last_became_active":"2024-09-21T11:40:00.310457+0000","last_became_peered":"2024-09-21T11:40:00.310457+0000","last_unstale":"2024-09-21T11:40:38.120082+0000","last_undegraded":"2024-09-21T11:40:38.120082+0000","last_fullsized":"2024-09-21T11:40:38.120082+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.362207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.088844+0000","last_change":"2024-09-21T11:40:04.334536+0000","last_active":"2024-09-21T11:40:37.088844+0000","last_peered":"2024-09-21T11:40:37.088844+0000","last_clean":"2024-09-21T11:40:37.088844+0000","last_became_active":"2024-09-21T11:40:04.334205+0000","last_became_peered":"2024-09-21T11:40:04.334205+0000","last_unstale":"2024-09-21T11:40:37.088844+0000","last_undegraded":"2024-09-21T11:40:37.088844+0000","last_fullsized":"2024-09-21T11:40:37.088844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.125493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090268+0000","last_change":"2024-09-21T11:40:06.352508+0000","last_active":"2024-09-21T11:40:37.090268+0000","last_peered":"2024-09-21T11:40:37.090268+0000","last_clean":"2024-09-21T11:40:37.090268+0000","last_became_active":"2024-09-21T11:40:06.352210+0000","last_became_peered":"2024-09-21T11:40:06.352210+0000","last_unstale":"2024-09-21T11:40:37.090268+0000","last_undegraded":"2024-09-21T11:40:37.090268+0000","last_fullsized":"2024-09-21T11:40:37.090268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.957978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":66,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116907+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:40:38.116907+0000","last_peered":"2024-09-21T11:40:38.116907+0000","last_clean":"2024-09-21T11:40:38.116907+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:40:38.116907+0000","last_undegraded":"2024-09-21T11:40:38.116907+0000","last_fullsized":"2024-09-21T11:40:38.116907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.513015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.120072+0000","last_change":"2024-09-21T11:40:00.311525+0000","last_active":"2024-09-21T11:40:38.120072+0000","last_peered":"2024-09-21T11:40:38.120072+0000","last_clean":"2024-09-21T11:40:38.120072+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:40:38.120072+0000","last_undegraded":"2024-09-21T11:40:38.120072+0000","last_fullsized":"2024-09-21T11:40:38.120072+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.568697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118359+0000","last_change":"2024-09-21T11:40:04.331990+0000","last_active":"2024-09-21T11:40:38.118359+0000","last_peered":"2024-09-21T11:40:38.118359+0000","last_clean":"2024-09-21T11:40:38.118359+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:40:38.118359+0000","last_undegraded":"2024-09-21T11:40:38.118359+0000","last_fullsized":"2024-09-21T11:40:38.118359+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.347865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117350+0000","last_change":"2024-09-21T11:40:06.487630+0000","last_active":"2024-09-21T11:40:38.117350+0000","last_peered":"2024-09-21T11:40:38.117350+0000","last_clean":"2024-09-21T11:40:38.117350+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:40:38.117350+0000","last_undegraded":"2024-09-21T11:40:38.117350+0000","last_fullsized":"2024-09-21T11:40:38.117350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.388794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092001+0000","last_change":"2024-09-21T11:40:00.310655+0000","last_active":"2024-09-21T11:40:37.092001+0000","last_peered":"2024-09-21T11:40:37.092001+0000","last_clean":"2024-09-21T11:40:37.092001+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:40:37.092001+0000","last_undegraded":"2024-09-21T11:40:37.092001+0000","last_fullsized":"2024-09-21T11:40:37.092001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.882425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091044+0000","last_change":"2024-09-21T11:40:02.399093+0000","last_active":"2024-09-21T11:40:37.091044+0000","last_peered":"2024-09-21T11:40:37.091044+0000","last_clean":"2024-09-21T11:40:37.091044+0000","last_became_active":"2024-09-21T11:40:02.398803+0000","last_became_peered":"2024-09-21T11:40:02.398803+0000","last_unstale":"2024-09-21T11:40:37.091044+0000","last_undegraded":"2024-09-21T11:40:37.091044+0000","last_fullsized":"2024-09-21T11:40:37.091044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.504556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118790+0000","last_change":"2024-09-21T11:40:04.328297+0000","last_active":"2024-09-21T11:40:38.118790+0000","last_peered":"2024-09-21T11:40:38.118790+0000","last_clean":"2024-09-21T11:40:38.118790+0000","last_became_active":"2024-09-21T11:40:04.327422+0000","last_became_peered":"2024-09-21T11:40:04.327422+0000","last_unstale":"2024-09-21T11:40:38.118790+0000","last_undegraded":"2024-09-21T11:40:38.118790+0000","last_fullsized":"2024-09-21T11:40:38.118790+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.663596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119034+0000","last_change":"2024-09-21T11:40:06.488399+0000","last_active":"2024-09-21T11:40:38.119034+0000","last_peered":"2024-09-21T11:40:38.119034+0000","last_clean":"2024-09-21T11:40:38.119034+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:40:38.119034+0000","last_undegraded":"2024-09-21T11:40:38.119034+0000","last_fullsized":"2024-09-21T11:40:38.119034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.617144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117426+0000","last_change":"2024-09-21T11:40:02.399564+0000","last_active":"2024-09-21T11:40:38.117426+0000","last_peered":"2024-09-21T11:40:38.117426+0000","last_clean":"2024-09-21T11:40:38.117426+0000","last_became_active":"2024-09-21T11:40:02.399267+0000","last_became_peered":"2024-09-21T11:40:02.399267+0000","last_unstale":"2024-09-21T11:40:38.117426+0000","last_undegraded":"2024-09-21T11:40:38.117426+0000","last_fullsized":"2024-09-21T11:40:38.117426+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.891469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118903+0000","last_change":"2024-09-21T11:40:00.308959+0000","last_active":"2024-09-21T11:40:38.118903+0000","last_peered":"2024-09-21T11:40:38.118903+0000","last_clean":"2024-09-21T11:40:38.118903+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:40:38.118903+0000","last_undegraded":"2024-09-21T11:40:38.118903+0000","last_fullsized":"2024-09-21T11:40:38.118903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.467036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090927+0000","last_change":"2024-09-21T11:40:04.326964+0000","last_active":"2024-09-21T11:40:37.090927+0000","last_peered":"2024-09-21T11:40:37.090927+0000","last_clean":"2024-09-21T11:40:37.090927+0000","last_became_active":"2024-09-21T11:40:04.326625+0000","last_became_peered":"2024-09-21T11:40:04.326625+0000","last_unstale":"2024-09-21T11:40:37.090927+0000","last_undegraded":"2024-09-21T11:40:37.090927+0000","last_fullsized":"2024-09-21T11:40:37.090927+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089630+0000","last_change":"2024-09-21T11:40:06.343913+0000","last_active":"2024-09-21T11:40:37.089630+0000","last_peered":"2024-09-21T11:40:37.089630+0000","last_clean":"2024-09-21T11:40:37.089630+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:40:37.089630+0000","last_undegraded":"2024-09-21T11:40:37.089630+0000","last_fullsized":"2024-09-21T11:40:37.089630+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.682920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119505+0000","last_change":"2024-09-21T11:40:02.400725+0000","last_active":"2024-09-21T11:40:38.119505+0000","last_peered":"2024-09-21T11:40:38.119505+0000","last_clean":"2024-09-21T11:40:38.119505+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:40:38.119505+0000","last_undegraded":"2024-09-21T11:40:38.119505+0000","last_fullsized":"2024-09-21T11:40:38.119505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.300079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116459+0000","last_change":"2024-09-21T11:40:00.311525+0000","last_active":"2024-09-21T11:40:38.116459+0000","last_peered":"2024-09-21T11:40:38.116459+0000","last_clean":"2024-09-21T11:40:38.116459+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:40:38.116459+0000","last_undegraded":"2024-09-21T11:40:38.116459+0000","last_fullsized":"2024-09-21T11:40:38.116459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.269082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090572+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:40:37.090572+0000","last_peered":"2024-09-21T11:40:37.090572+0000","last_clean":"2024-09-21T11:40:37.090572+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:40:37.090572+0000","last_undegraded":"2024-09-21T11:40:37.090572+0000","last_fullsized":"2024-09-21T11:40:37.090572+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.404995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118969+0000","last_change":"2024-09-21T11:40:06.346686+0000","last_active":"2024-09-21T11:40:38.118969+0000","last_peered":"2024-09-21T11:40:38.118969+0000","last_clean":"2024-09-21T11:40:38.118969+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:40:38.118969+0000","last_undegraded":"2024-09-21T11:40:38.118969+0000","last_fullsized":"2024-09-21T11:40:38.118969+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.014386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090690+0000","last_change":"2024-09-21T11:40:02.399404+0000","last_active":"2024-09-21T11:40:37.090690+0000","last_peered":"2024-09-21T11:40:37.090690+0000","last_clean":"2024-09-21T11:40:37.090690+0000","last_became_active":"2024-09-21T11:40:02.398771+0000","last_became_peered":"2024-09-21T11:40:02.398771+0000","last_unstale":"2024-09-21T11:40:37.090690+0000","last_undegraded":"2024-09-21T11:40:37.090690+0000","last_fullsized":"2024-09-21T11:40:37.090690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.091232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118174+0000","last_change":"2024-09-21T11:40:00.314627+0000","last_active":"2024-09-21T11:40:38.118174+0000","last_peered":"2024-09-21T11:40:38.118174+0000","last_clean":"2024-09-21T11:40:38.118174+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:40:38.118174+0000","last_undegraded":"2024-09-21T11:40:38.118174+0000","last_fullsized":"2024-09-21T11:40:38.118174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.535972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119238+0000","last_change":"2024-09-21T11:40:04.340164+0000","last_active":"2024-09-21T11:40:38.119238+0000","last_peered":"2024-09-21T11:40:38.119238+0000","last_clean":"2024-09-21T11:40:38.119238+0000","last_became_active":"2024-09-21T11:40:04.339253+0000","last_became_peered":"2024-09-21T11:40:04.339253+0000","last_unstale":"2024-09-21T11:40:38.119238+0000","last_undegraded":"2024-09-21T11:40:38.119238+0000","last_fullsized":"2024-09-21T11:40:38.119238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:30.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116791+0000","last_change":"2024-09-21T11:40:06.351973+0000","last_active":"2024-09-21T11:40:38.116791+0000","last_peered":"2024-09-21T11:40:38.116791+0000","last_clean":"2024-09-21T11:40:38.116791+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:40:38.116791+0000","last_undegraded":"2024-09-21T11:40:38.116791+0000","last_fullsized":"2024-09-21T11:40:38.116791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.555091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119008+0000","last_change":"2024-09-21T11:40:02.318645+0000","last_active":"2024-09-21T11:40:38.119008+0000","last_peered":"2024-09-21T11:40:38.119008+0000","last_clean":"2024-09-21T11:40:38.119008+0000","last_became_active":"2024-09-21T11:40:02.318352+0000","last_became_peered":"2024-09-21T11:40:02.318352+0000","last_unstale":"2024-09-21T11:40:38.119008+0000","last_undegraded":"2024-09-21T11:40:38.119008+0000","last_fullsized":"2024-09-21T11:40:38.119008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.071118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119237+0000","last_change":"2024-09-21T11:40:00.317768+0000","last_active":"2024-09-21T11:40:38.119237+0000","last_peered":"2024-09-21T11:40:38.119237+0000","last_clean":"2024-09-21T11:40:38.119237+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:40:38.119237+0000","last_undegraded":"2024-09-21T11:40:38.119237+0000","last_fullsized":"2024-09-21T11:40:38.119237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.919244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096986+0000","last_change":"2024-09-21T11:40:04.328561+0000","last_active":"2024-09-21T11:40:37.096986+0000","last_peered":"2024-09-21T11:40:37.096986+0000","last_clean":"2024-09-21T11:40:37.096986+0000","last_became_active":"2024-09-21T11:40:04.328216+0000","last_became_peered":"2024-09-21T11:40:04.328216+0000","last_unstale":"2024-09-21T11:40:37.096986+0000","last_undegraded":"2024-09-21T11:40:37.096986+0000","last_fullsized":"2024-09-21T11:40:37.096986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.146311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089641+0000","last_change":"2024-09-21T11:40:06.353577+0000","last_active":"2024-09-21T11:40:37.089641+0000","last_peered":"2024-09-21T11:40:37.089641+0000","last_clean":"2024-09-21T11:40:37.089641+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:40:37.089641+0000","last_undegraded":"2024-09-21T11:40:37.089641+0000","last_fullsized":"2024-09-21T11:40:37.089641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.269429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097098+0000","last_change":"2024-09-21T11:40:02.319305+0000","last_active":"2024-09-21T11:40:37.097098+0000","last_peered":"2024-09-21T11:40:37.097098+0000","last_clean":"2024-09-21T11:40:37.097098+0000","last_became_active":"2024-09-21T11:40:02.318699+0000","last_became_peered":"2024-09-21T11:40:02.318699+0000","last_unstale":"2024-09-21T11:40:37.097098+0000","last_undegraded":"2024-09-21T11:40:37.097098+0000","last_fullsized":"2024-09-21T11:40:37.097098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.659293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119439+0000","last_change":"2024-09-21T11:40:00.309690+0000","last_active":"2024-09-21T11:40:38.119439+0000","last_peered":"2024-09-21T11:40:38.119439+0000","last_clean":"2024-09-21T11:40:38.119439+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:40:38.119439+0000","last_undegraded":"2024-09-21T11:40:38.119439+0000","last_fullsized":"2024-09-21T11:40:38.119439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.952393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090841+0000","last_change":"2024-09-21T11:40:04.336820+0000","last_active":"2024-09-21T11:40:37.090841+0000","last_peered":"2024-09-21T11:40:37.090841+0000","last_clean":"2024-09-21T11:40:37.090841+0000","last_became_active":"2024-09-21T11:40:04.336469+0000","last_became_peered":"2024-09-21T11:40:04.336469+0000","last_unstale":"2024-09-21T11:40:37.090841+0000","last_undegraded":"2024-09-21T11:40:37.090841+0000","last_fullsized":"2024-09-21T11:40:37.090841+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.369123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116254+0000","last_change":"2024-09-21T11:40:06.346455+0000","last_active":"2024-09-21T11:40:38.116254+0000","last_peered":"2024-09-21T11:40:38.116254+0000","last_clean":"2024-09-21T11:40:38.116254+0000","last_became_active":"2024-09-21T11:40:06.346090+0000","last_became_peered":"2024-09-21T11:40:06.346090+0000","last_unstale":"2024-09-21T11:40:38.116254+0000","last_undegraded":"2024-09-21T11:40:38.116254+0000","last_fullsized":"2024-09-21T11:40:38.116254+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.873349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089766+0000","last_change":"2024-09-21T11:40:02.399706+0000","last_active":"2024-09-21T11:40:37.089766+0000","last_peered":"2024-09-21T11:40:37.089766+0000","last_clean":"2024-09-21T11:40:37.089766+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:40:37.089766+0000","last_undegraded":"2024-09-21T11:40:37.089766+0000","last_fullsized":"2024-09-21T11:40:37.089766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.937696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118907+0000","last_change":"2024-09-21T11:40:00.317057+0000","last_active":"2024-09-21T11:40:38.118907+0000","last_peered":"2024-09-21T11:40:38.118907+0000","last_clean":"2024-09-21T11:40:38.118907+0000","last_became_active":"2024-09-21T11:40:00.316756+0000","last_became_peered":"2024-09-21T11:40:00.316756+0000","last_unstale":"2024-09-21T11:40:38.118907+0000","last_undegraded":"2024-09-21T11:40:38.118907+0000","last_fullsized":"2024-09-21T11:40:38.118907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.129760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116741+0000","last_change":"2024-09-21T11:40:04.331954+0000","last_active":"2024-09-21T11:40:38.116741+0000","last_peered":"2024-09-21T11:40:38.116741+0000","last_clean":"2024-09-21T11:40:38.116741+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:40:38.116741+0000","last_undegraded":"2024-09-21T11:40:38.116741+0000","last_fullsized":"2024-09-21T11:40:38.116741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.778314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090438+0000","last_change":"2024-09-21T11:40:06.488252+0000","last_active":"2024-09-21T11:40:37.090438+0000","last_peered":"2024-09-21T11:40:37.090438+0000","last_clean":"2024-09-21T11:40:37.090438+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:40:37.090438+0000","last_undegraded":"2024-09-21T11:40:37.090438+0000","last_fullsized":"2024-09-21T11:40:37.090438+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.158855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":48,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089194+0000","last_change":"2024-09-21T11:40:02.319060+0000","last_active":"2024-09-21T11:40:37.089194+0000","last_peered":"2024-09-21T11:40:37.089194+0000","last_clean":"2024-09-21T11:40:37.089194+0000","last_became_active":"2024-09-21T11:40:02.318727+0000","last_became_peered":"2024-09-21T11:40:02.318727+0000","last_unstale":"2024-09-21T11:40:37.089194+0000","last_undegraded":"2024-09-21T11:40:37.089194+0000","last_fullsized":"2024-09-21T11:40:37.089194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.877093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116996+0000","last_change":"2024-09-21T11:40:00.314387+0000","last_active":"2024-09-21T11:40:38.116996+0000","last_peered":"2024-09-21T11:40:38.116996+0000","last_clean":"2024-09-21T11:40:38.116996+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:40:38.116996+0000","last_undegraded":"2024-09-21T11:40:38.116996+0000","last_fullsized":"2024-09-21T11:40:38.116996+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:00.574148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091722+0000","last_change":"2024-09-21T11:40:04.332492+0000","last_active":"2024-09-21T11:40:37.091722+0000","last_peered":"2024-09-21T11:40:37.091722+0000","last_clean":"2024-09-21T11:40:37.091722+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:40:37.091722+0000","last_undegraded":"2024-09-21T11:40:37.091722+0000","last_fullsized":"2024-09-21T11:40:37.091722+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.592217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117167+0000","last_change":"2024-09-21T11:40:06.342962+0000","last_active":"2024-09-21T11:40:38.117167+0000","last_peered":"2024-09-21T11:40:38.117167+0000","last_clean":"2024-09-21T11:40:38.117167+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:40:38.117167+0000","last_undegraded":"2024-09-21T11:40:38.117167+0000","last_fullsized":"2024-09-21T11:40:38.117167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.430215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097021+0000","last_change":"2024-09-21T11:40:06.349089+0000","last_active":"2024-09-21T11:40:37.097021+0000","last_peered":"2024-09-21T11:40:37.097021+0000","last_clean":"2024-09-21T11:40:37.097021+0000","last_became_active":"2024-09-21T11:40:06.348796+0000","last_became_peered":"2024-09-21T11:40:06.348796+0000","last_unstale":"2024-09-21T11:40:37.097021+0000","last_undegraded":"2024-09-21T11:40:37.097021+0000","last_fullsized":"2024-09-21T11:40:37.097021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.850881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089414+0000","last_change":"2024-09-21T11:40:00.309865+0000","last_active":"2024-09-21T11:40:37.089414+0000","last_peered":"2024-09-21T11:40:37.089414+0000","last_clean":"2024-09-21T11:40:37.089414+0000","last_became_active":"2024-09-21T11:40:00.309598+0000","last_became_peered":"2024-09-21T11:40:00.309598+0000","last_unstale":"2024-09-21T11:40:37.089414+0000","last_undegraded":"2024-09-21T11:40:37.089414+0000","last_fullsized":"2024-09-21T11:40:37.089414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.604856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":60,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092528+0000","last_change":"2024-09-21T11:40:02.318498+0000","last_active":"2024-09-21T11:40:37.092528+0000","last_peered":"2024-09-21T11:40:37.092528+0000","last_clean":"2024-09-21T11:40:37.092528+0000","last_became_active":"2024-09-21T11:40:02.318160+0000","last_became_peered":"2024-09-21T11:40:02.318160+0000","last_unstale":"2024-09-21T11:40:37.092528+0000","last_undegraded":"2024-09-21T11:40:37.092528+0000","last_fullsized":"2024-09-21T11:40:37.092528+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.280806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116971+0000","last_change":"2024-09-21T11:40:04.340406+0000","last_active":"2024-09-21T11:40:38.116971+0000","last_peered":"2024-09-21T11:40:38.116971+0000","last_clean":"2024-09-21T11:40:38.116971+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:40:38.116971+0000","last_undegraded":"2024-09-21T11:40:38.116971+0000","last_fullsized":"2024-09-21T11:40:38.116971+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.851842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928233997,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28364,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716788,"statfs":{"total":95995035648,"available":95965990912,"internally_reserved":0,"allocated":1368064,"data_stored":898615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462930,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712640,"statfs":{"total":95995035648,"available":95961743360,"internally_reserved":0,"allocated":1355776,"data_stored":898959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724567,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31932,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713220,"statfs":{"total":95995035648,"available":95962337280,"internally_reserved":0,"allocated":761856,"data_stored":307534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920797,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":786432,"data_stored":308760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117027,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":790528,"data_stored":308053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411370,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27768,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717384,"statfs":{"total":95995035648,"available":95966601216,"internally_reserved":0,"allocated":753664,"data_stored":307746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":827392,"data_stored":309396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32536,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712616,"statfs":{"total":95995035648,"available":95961718784,"internally_reserved":0,"allocated":1380352,"data_stored":899330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:40:47.209 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T11:40:47.209 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T11:40:47.209 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T11:40:47.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph health --format=json 2024-09-21T11:40:47.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:46 smithi175 ceph-mon[32613]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 588 B/s rd, 0 op/s 2024-09-21T11:40:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:46 smithi175 ceph-mon[32613]: from='client.24596 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:47.474 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:48.403 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:40:48.403 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T11:40:49.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:49 smithi043 ceph-mon[29710]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-09-21T11:40:49.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2028620499' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T11:40:49.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:49 smithi043 ceph-mon[36454]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-09-21T11:40:49.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2028620499' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T11:40:49.495 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T11:40:49.495 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T11:40:49.495 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-21T11:40:49.506 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-21T11:40:49.506 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-guits-main-2024-09-21-0813'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms bind msgr1': False, 'ms bind msgr2': True, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'workunit': {'branch': 'wip-guits-main-2024-09-21-0813', 'sha1': '5c1cdc3c470dae78e91a0ed2bcf13509a046d301'}} 2024-09-21T11:40:49.506 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-21T11:40:49.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-21T11:40:49.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:49 smithi175 ceph-mon[32613]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-09-21T11:40:49.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2028620499' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T11:40:49.789 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:50.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:50 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:50] ENGINE Bus STOPPING 2024-09-21T11:40:50.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:50 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:50] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T11:40:50.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:50 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:50] ENGINE Bus STOPPED 2024-09-21T11:40:50.742 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:50 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:50] ENGINE Bus STARTING 2024-09-21T11:40:51.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:40:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:40:51.096 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:50 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:50] ENGINE Serving on http://:::9283 2024-09-21T11:40:51.097 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:50 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:50] ENGINE Bus STARTED 2024-09-21T11:40:51.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:51 smithi043 ceph-mon[29710]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:40:51.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:51 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3281087173' entity='client.admin' 2024-09-21T11:40:51.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:51 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:40:51.481 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:51 smithi043 ceph-mon[36454]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:40:51.481 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:51 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3281087173' entity='client.admin' 2024-09-21T11:40:51.481 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:51 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:40:51.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:51 smithi175 ceph-mon[32613]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:40:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:51 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3281087173' entity='client.admin' 2024-09-21T11:40:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:51 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:40:51.960 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-21T11:40:51.960 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T11:40:51.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:40:52.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:52.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:52 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:40:52.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:52.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:52 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:40:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:40:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:52 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:40:53.370 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:53 smithi043 ceph-mon[29710]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:40:53.370 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:53 smithi043 ceph-mon[36454]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:40:53.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:53 smithi175 ceph-mon[32613]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:40:54.174 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:54.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:54 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:40:54.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:54 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:40:54.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:54 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:40:54.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:54 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:40:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:54 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:40:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:54 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:40:54.886 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:40:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:40:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:55 smithi043 ceph-mon[36454]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:40:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:55 smithi043 ceph-mon[36454]: osdmap e77: 8 total, 8 up, 8 in 2024-09-21T11:40:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:55 smithi043 ceph-mon[29710]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:40:55.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:55 smithi043 ceph-mon[29710]: osdmap e77: 8 total, 8 up, 8 in 2024-09-21T11:40:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:40:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:40:55.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:55 smithi175 ceph-mon[32613]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:40:55.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:55 smithi175 ceph-mon[32613]: osdmap e77: 8 total, 8 up, 8 in 2024-09-21T11:40:56.374 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:56 smithi043 ceph-mon[36454]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:56.374 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:56 smithi043 ceph-mon[29710]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:56.390 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-21T11:40:54.108129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":893,"num_read_kb":750,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244872,"kb_used_data":7836,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716344,"statfs":{"total":767960285184,"available":767709536256,"internally_reserved":0,"allocated":8024064,"data_stored":4238393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.417308"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116965+0000","last_change":"2024-09-21T11:40:00.313497+0000","last_active":"2024-09-21T11:40:38.116965+0000","last_peered":"2024-09-21T11:40:38.116965+0000","last_clean":"2024-09-21T11:40:38.116965+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:40:38.116965+0000","last_undegraded":"2024-09-21T11:40:38.116965+0000","last_fullsized":"2024-09-21T11:40:38.116965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.636264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116915+0000","last_change":"2024-09-21T11:40:02.319143+0000","last_active":"2024-09-21T11:40:38.116915+0000","last_peered":"2024-09-21T11:40:38.116915+0000","last_clean":"2024-09-21T11:40:38.116915+0000","last_became_active":"2024-09-21T11:40:02.318639+0000","last_became_peered":"2024-09-21T11:40:02.318639+0000","last_unstale":"2024-09-21T11:40:38.116915+0000","last_undegraded":"2024-09-21T11:40:38.116915+0000","last_fullsized":"2024-09-21T11:40:38.116915+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.717901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096596+0000","last_change":"2024-09-21T11:40:04.331203+0000","last_active":"2024-09-21T11:40:37.096596+0000","last_peered":"2024-09-21T11:40:37.096596+0000","last_clean":"2024-09-21T11:40:37.096596+0000","last_became_active":"2024-09-21T11:40:04.330879+0000","last_became_peered":"2024-09-21T11:40:04.330879+0000","last_unstale":"2024-09-21T11:40:37.096596+0000","last_undegraded":"2024-09-21T11:40:37.096596+0000","last_fullsized":"2024-09-21T11:40:37.096596+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.860431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117023+0000","last_change":"2024-09-21T11:40:06.346317+0000","last_active":"2024-09-21T11:40:38.117023+0000","last_peered":"2024-09-21T11:40:38.117023+0000","last_clean":"2024-09-21T11:40:38.117023+0000","last_became_active":"2024-09-21T11:40:06.345995+0000","last_became_peered":"2024-09-21T11:40:06.345995+0000","last_unstale":"2024-09-21T11:40:38.117023+0000","last_undegraded":"2024-09-21T11:40:38.117023+0000","last_fullsized":"2024-09-21T11:40:38.117023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.419437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090252+0000","last_change":"2024-09-21T11:40:06.489876+0000","last_active":"2024-09-21T11:40:37.090252+0000","last_peered":"2024-09-21T11:40:37.090252+0000","last_clean":"2024-09-21T11:40:37.090252+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:40:37.090252+0000","last_undegraded":"2024-09-21T11:40:37.090252+0000","last_fullsized":"2024-09-21T11:40:37.090252+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.939900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090828+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:40:37.090828+0000","last_peered":"2024-09-21T11:40:37.090828+0000","last_clean":"2024-09-21T11:40:37.090828+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:40:37.090828+0000","last_undegraded":"2024-09-21T11:40:37.090828+0000","last_fullsized":"2024-09-21T11:40:37.090828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089985+0000","last_change":"2024-09-21T11:40:02.315669+0000","last_active":"2024-09-21T11:40:37.089985+0000","last_peered":"2024-09-21T11:40:37.089985+0000","last_clean":"2024-09-21T11:40:37.089985+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:40:37.089985+0000","last_undegraded":"2024-09-21T11:40:37.089985+0000","last_fullsized":"2024-09-21T11:40:37.089985+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.511141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116814+0000","last_change":"2024-09-21T11:40:04.326924+0000","last_active":"2024-09-21T11:40:38.116814+0000","last_peered":"2024-09-21T11:40:38.116814+0000","last_clean":"2024-09-21T11:40:38.116814+0000","last_became_active":"2024-09-21T11:40:04.326663+0000","last_became_peered":"2024-09-21T11:40:04.326663+0000","last_unstale":"2024-09-21T11:40:38.116814+0000","last_undegraded":"2024-09-21T11:40:38.116814+0000","last_fullsized":"2024-09-21T11:40:38.116814+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:04.814146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119541+0000","last_change":"2024-09-21T11:40:00.314157+0000","last_active":"2024-09-21T11:40:38.119541+0000","last_peered":"2024-09-21T11:40:38.119541+0000","last_clean":"2024-09-21T11:40:38.119541+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:40:38.119541+0000","last_undegraded":"2024-09-21T11:40:38.119541+0000","last_fullsized":"2024-09-21T11:40:38.119541+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.659277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118645+0000","last_change":"2024-09-21T11:40:02.314170+0000","last_active":"2024-09-21T11:40:38.118645+0000","last_peered":"2024-09-21T11:40:38.118645+0000","last_clean":"2024-09-21T11:40:38.118645+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:40:38.118645+0000","last_undegraded":"2024-09-21T11:40:38.118645+0000","last_fullsized":"2024-09-21T11:40:38.118645+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.670327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117406+0000","last_change":"2024-09-21T11:40:04.339616+0000","last_active":"2024-09-21T11:40:38.117406+0000","last_peered":"2024-09-21T11:40:38.117406+0000","last_clean":"2024-09-21T11:40:38.117406+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:40:38.117406+0000","last_undegraded":"2024-09-21T11:40:38.117406+0000","last_fullsized":"2024-09-21T11:40:38.117406+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.214530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116334+0000","last_change":"2024-09-21T11:40:06.344622+0000","last_active":"2024-09-21T11:40:38.116334+0000","last_peered":"2024-09-21T11:40:38.116334+0000","last_clean":"2024-09-21T11:40:38.116334+0000","last_became_active":"2024-09-21T11:40:06.344291+0000","last_became_peered":"2024-09-21T11:40:06.344291+0000","last_unstale":"2024-09-21T11:40:38.116334+0000","last_undegraded":"2024-09-21T11:40:38.116334+0000","last_fullsized":"2024-09-21T11:40:38.116334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.653671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119780+0000","last_change":"2024-09-21T11:40:00.315137+0000","last_active":"2024-09-21T11:40:38.119780+0000","last_peered":"2024-09-21T11:40:38.119780+0000","last_clean":"2024-09-21T11:40:38.119780+0000","last_became_active":"2024-09-21T11:40:00.314838+0000","last_became_peered":"2024-09-21T11:40:00.314838+0000","last_unstale":"2024-09-21T11:40:38.119780+0000","last_undegraded":"2024-09-21T11:40:38.119780+0000","last_fullsized":"2024-09-21T11:40:38.119780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:40:59.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118997+0000","last_change":"2024-09-21T11:40:02.320524+0000","last_active":"2024-09-21T11:40:38.118997+0000","last_peered":"2024-09-21T11:40:38.118997+0000","last_clean":"2024-09-21T11:40:38.118997+0000","last_became_active":"2024-09-21T11:40:02.320184+0000","last_became_peered":"2024-09-21T11:40:02.320184+0000","last_unstale":"2024-09-21T11:40:38.118997+0000","last_undegraded":"2024-09-21T11:40:38.118997+0000","last_fullsized":"2024-09-21T11:40:38.118997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.631864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117300+0000","last_change":"2024-09-21T11:40:04.333696+0000","last_active":"2024-09-21T11:40:38.117300+0000","last_peered":"2024-09-21T11:40:38.117300+0000","last_clean":"2024-09-21T11:40:38.117300+0000","last_became_active":"2024-09-21T11:40:04.333406+0000","last_became_peered":"2024-09-21T11:40:04.333406+0000","last_unstale":"2024-09-21T11:40:38.117300+0000","last_undegraded":"2024-09-21T11:40:38.117300+0000","last_fullsized":"2024-09-21T11:40:38.117300+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.055519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117137+0000","last_change":"2024-09-21T11:40:06.349798+0000","last_active":"2024-09-21T11:40:38.117137+0000","last_peered":"2024-09-21T11:40:38.117137+0000","last_clean":"2024-09-21T11:40:38.117137+0000","last_became_active":"2024-09-21T11:40:06.349538+0000","last_became_peered":"2024-09-21T11:40:06.349538+0000","last_unstale":"2024-09-21T11:40:38.117137+0000","last_undegraded":"2024-09-21T11:40:38.117137+0000","last_fullsized":"2024-09-21T11:40:38.117137+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:34.219537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118957+0000","last_change":"2024-09-21T11:40:06.487919+0000","last_active":"2024-09-21T11:40:38.118957+0000","last_peered":"2024-09-21T11:40:38.118957+0000","last_clean":"2024-09-21T11:40:38.118957+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:40:38.118957+0000","last_undegraded":"2024-09-21T11:40:38.118957+0000","last_fullsized":"2024-09-21T11:40:38.118957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:18.602419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117461+0000","last_change":"2024-09-21T11:40:00.312124+0000","last_active":"2024-09-21T11:40:38.117461+0000","last_peered":"2024-09-21T11:40:38.117461+0000","last_clean":"2024-09-21T11:40:38.117461+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:40:38.117461+0000","last_undegraded":"2024-09-21T11:40:38.117461+0000","last_fullsized":"2024-09-21T11:40:38.117461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.023370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089986+0000","last_change":"2024-09-21T11:40:02.319850+0000","last_active":"2024-09-21T11:40:37.089986+0000","last_peered":"2024-09-21T11:40:37.089986+0000","last_clean":"2024-09-21T11:40:37.089986+0000","last_became_active":"2024-09-21T11:40:02.319153+0000","last_became_peered":"2024-09-21T11:40:02.319153+0000","last_unstale":"2024-09-21T11:40:37.089986+0000","last_undegraded":"2024-09-21T11:40:37.089986+0000","last_fullsized":"2024-09-21T11:40:37.089986+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.765972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097393+0000","last_change":"2024-09-21T11:40:04.339562+0000","last_active":"2024-09-21T11:40:37.097393+0000","last_peered":"2024-09-21T11:40:37.097393+0000","last_clean":"2024-09-21T11:40:37.097393+0000","last_became_active":"2024-09-21T11:40:04.339222+0000","last_became_peered":"2024-09-21T11:40:04.339222+0000","last_unstale":"2024-09-21T11:40:37.097393+0000","last_undegraded":"2024-09-21T11:40:37.097393+0000","last_fullsized":"2024-09-21T11:40:37.097393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.022703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090832+0000","last_change":"2024-09-21T11:40:06.487933+0000","last_active":"2024-09-21T11:40:37.090832+0000","last_peered":"2024-09-21T11:40:37.090832+0000","last_clean":"2024-09-21T11:40:37.090832+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:40:37.090832+0000","last_undegraded":"2024-09-21T11:40:37.090832+0000","last_fullsized":"2024-09-21T11:40:37.090832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.209907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119180+0000","last_change":"2024-09-21T11:40:00.312534+0000","last_active":"2024-09-21T11:40:38.119180+0000","last_peered":"2024-09-21T11:40:38.119180+0000","last_clean":"2024-09-21T11:40:38.119180+0000","last_became_active":"2024-09-21T11:40:00.312239+0000","last_became_peered":"2024-09-21T11:40:00.312239+0000","last_unstale":"2024-09-21T11:40:38.119180+0000","last_undegraded":"2024-09-21T11:40:38.119180+0000","last_fullsized":"2024-09-21T11:40:38.119180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.443761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090511+0000","last_change":"2024-09-21T11:40:02.324039+0000","last_active":"2024-09-21T11:40:37.090511+0000","last_peered":"2024-09-21T11:40:37.090511+0000","last_clean":"2024-09-21T11:40:37.090511+0000","last_became_active":"2024-09-21T11:40:02.323784+0000","last_became_peered":"2024-09-21T11:40:02.323784+0000","last_unstale":"2024-09-21T11:40:37.090511+0000","last_undegraded":"2024-09-21T11:40:37.090511+0000","last_fullsized":"2024-09-21T11:40:37.090511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.707671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118544+0000","last_change":"2024-09-21T11:40:04.328865+0000","last_active":"2024-09-21T11:40:38.118544+0000","last_peered":"2024-09-21T11:40:38.118544+0000","last_clean":"2024-09-21T11:40:38.118544+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:40:38.118544+0000","last_undegraded":"2024-09-21T11:40:38.118544+0000","last_fullsized":"2024-09-21T11:40:38.118544+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.407835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118453+0000","last_change":"2024-09-21T11:40:06.340320+0000","last_active":"2024-09-21T11:40:38.118453+0000","last_peered":"2024-09-21T11:40:38.118453+0000","last_clean":"2024-09-21T11:40:38.118453+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:40:38.118453+0000","last_undegraded":"2024-09-21T11:40:38.118453+0000","last_fullsized":"2024-09-21T11:40:38.118453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.731856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096791+0000","last_change":"2024-09-21T11:40:00.308535+0000","last_active":"2024-09-21T11:40:37.096791+0000","last_peered":"2024-09-21T11:40:37.096791+0000","last_clean":"2024-09-21T11:40:37.096791+0000","last_became_active":"2024-09-21T11:40:00.308219+0000","last_became_peered":"2024-09-21T11:40:00.308219+0000","last_unstale":"2024-09-21T11:40:37.096791+0000","last_undegraded":"2024-09-21T11:40:37.096791+0000","last_fullsized":"2024-09-21T11:40:37.096791+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.036879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116981+0000","last_change":"2024-09-21T11:40:02.400244+0000","last_active":"2024-09-21T11:40:38.116981+0000","last_peered":"2024-09-21T11:40:38.116981+0000","last_clean":"2024-09-21T11:40:38.116981+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:40:38.116981+0000","last_undegraded":"2024-09-21T11:40:38.116981+0000","last_fullsized":"2024-09-21T11:40:38.116981+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.785487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:45.366909+0000","last_change":"2024-09-21T11:40:04.328515+0000","last_active":"2024-09-21T11:40:45.366909+0000","last_peered":"2024-09-21T11:40:45.366909+0000","last_clean":"2024-09-21T11:40:45.366909+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:40:45.366909+0000","last_undegraded":"2024-09-21T11:40:45.366909+0000","last_fullsized":"2024-09-21T11:40:45.366909+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:03.806494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":66,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097724+0000","last_change":"2024-09-21T11:40:02.323539+0000","last_active":"2024-09-21T11:40:37.097724+0000","last_peered":"2024-09-21T11:40:37.097724+0000","last_clean":"2024-09-21T11:40:37.097724+0000","last_became_active":"2024-09-21T11:40:02.323204+0000","last_became_peered":"2024-09-21T11:40:02.323204+0000","last_unstale":"2024-09-21T11:40:37.097724+0000","last_undegraded":"2024-09-21T11:40:37.097724+0000","last_fullsized":"2024-09-21T11:40:37.097724+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089484+0000","last_change":"2024-09-21T11:40:00.315307+0000","last_active":"2024-09-21T11:40:37.089484+0000","last_peered":"2024-09-21T11:40:37.089484+0000","last_clean":"2024-09-21T11:40:37.089484+0000","last_became_active":"2024-09-21T11:40:00.315015+0000","last_became_peered":"2024-09-21T11:40:00.315015+0000","last_unstale":"2024-09-21T11:40:37.089484+0000","last_undegraded":"2024-09-21T11:40:37.089484+0000","last_fullsized":"2024-09-21T11:40:37.089484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.959497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:45.366927+0000","last_change":"2024-09-21T11:40:04.342645+0000","last_active":"2024-09-21T11:40:45.366927+0000","last_peered":"2024-09-21T11:40:45.366927+0000","last_clean":"2024-09-21T11:40:45.366927+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:40:45.366927+0000","last_undegraded":"2024-09-21T11:40:45.366927+0000","last_fullsized":"2024-09-21T11:40:45.366927+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.259518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118232+0000","last_change":"2024-09-21T11:40:06.345062+0000","last_active":"2024-09-21T11:40:38.118232+0000","last_peered":"2024-09-21T11:40:38.118232+0000","last_clean":"2024-09-21T11:40:38.118232+0000","last_became_active":"2024-09-21T11:40:06.344796+0000","last_became_peered":"2024-09-21T11:40:06.344796+0000","last_unstale":"2024-09-21T11:40:38.118232+0000","last_undegraded":"2024-09-21T11:40:38.118232+0000","last_fullsized":"2024-09-21T11:40:38.118232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.403408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":74,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089287+0000","last_change":"2024-09-21T11:40:02.399334+0000","last_active":"2024-09-21T11:40:37.089287+0000","last_peered":"2024-09-21T11:40:37.089287+0000","last_clean":"2024-09-21T11:40:37.089287+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:40:37.089287+0000","last_undegraded":"2024-09-21T11:40:37.089287+0000","last_fullsized":"2024-09-21T11:40:37.089287+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.386531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091103+0000","last_change":"2024-09-21T11:40:00.310330+0000","last_active":"2024-09-21T11:40:37.091103+0000","last_peered":"2024-09-21T11:40:37.091103+0000","last_clean":"2024-09-21T11:40:37.091103+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:40:37.091103+0000","last_undegraded":"2024-09-21T11:40:37.091103+0000","last_fullsized":"2024-09-21T11:40:37.091103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.371930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117521+0000","last_change":"2024-09-21T11:40:04.330180+0000","last_active":"2024-09-21T11:40:38.117521+0000","last_peered":"2024-09-21T11:40:38.117521+0000","last_clean":"2024-09-21T11:40:38.117521+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:40:38.117521+0000","last_undegraded":"2024-09-21T11:40:38.117521+0000","last_fullsized":"2024-09-21T11:40:38.117521+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.570842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118790+0000","last_change":"2024-09-21T11:40:06.343742+0000","last_active":"2024-09-21T11:40:38.118790+0000","last_peered":"2024-09-21T11:40:38.118790+0000","last_clean":"2024-09-21T11:40:38.118790+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:40:38.118790+0000","last_undegraded":"2024-09-21T11:40:38.118790+0000","last_fullsized":"2024-09-21T11:40:38.118790+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.217075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":63,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117749+0000","last_change":"2024-09-21T11:40:02.312574+0000","last_active":"2024-09-21T11:40:38.117749+0000","last_peered":"2024-09-21T11:40:38.117749+0000","last_clean":"2024-09-21T11:40:38.117749+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:40:38.117749+0000","last_undegraded":"2024-09-21T11:40:38.117749+0000","last_fullsized":"2024-09-21T11:40:38.117749+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.497994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117503+0000","last_change":"2024-09-21T11:40:00.312084+0000","last_active":"2024-09-21T11:40:38.117503+0000","last_peered":"2024-09-21T11:40:38.117503+0000","last_clean":"2024-09-21T11:40:38.117503+0000","last_became_active":"2024-09-21T11:40:00.311813+0000","last_became_peered":"2024-09-21T11:40:00.311813+0000","last_unstale":"2024-09-21T11:40:38.117503+0000","last_undegraded":"2024-09-21T11:40:38.117503+0000","last_fullsized":"2024-09-21T11:40:38.117503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.090287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:45.366757+0000","last_change":"2024-09-21T11:40:04.324572+0000","last_active":"2024-09-21T11:40:45.366757+0000","last_peered":"2024-09-21T11:40:45.366757+0000","last_clean":"2024-09-21T11:40:45.366757+0000","last_became_active":"2024-09-21T11:40:04.324272+0000","last_became_peered":"2024-09-21T11:40:04.324272+0000","last_unstale":"2024-09-21T11:40:45.366757+0000","last_undegraded":"2024-09-21T11:40:45.366757+0000","last_fullsized":"2024-09-21T11:40:45.366757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.830113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117696+0000","last_change":"2024-09-21T11:40:06.489239+0000","last_active":"2024-09-21T11:40:38.117696+0000","last_peered":"2024-09-21T11:40:38.117696+0000","last_clean":"2024-09-21T11:40:38.117696+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:40:38.117696+0000","last_undegraded":"2024-09-21T11:40:38.117696+0000","last_fullsized":"2024-09-21T11:40:38.117696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.580118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097096+0000","last_change":"2024-09-21T11:40:02.318985+0000","last_active":"2024-09-21T11:40:37.097096+0000","last_peered":"2024-09-21T11:40:37.097096+0000","last_clean":"2024-09-21T11:40:37.097096+0000","last_became_active":"2024-09-21T11:40:02.318270+0000","last_became_peered":"2024-09-21T11:40:02.318270+0000","last_unstale":"2024-09-21T11:40:37.097096+0000","last_undegraded":"2024-09-21T11:40:37.097096+0000","last_fullsized":"2024-09-21T11:40:37.097096+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.457674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119649+0000","last_change":"2024-09-21T11:40:00.318794+0000","last_active":"2024-09-21T11:40:38.119649+0000","last_peered":"2024-09-21T11:40:38.119649+0000","last_clean":"2024-09-21T11:40:38.119649+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:40:38.119649+0000","last_undegraded":"2024-09-21T11:40:38.119649+0000","last_fullsized":"2024-09-21T11:40:38.119649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.580077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:45.366824+0000","last_change":"2024-09-21T11:40:04.335582+0000","last_active":"2024-09-21T11:40:45.366824+0000","last_peered":"2024-09-21T11:40:45.366824+0000","last_clean":"2024-09-21T11:40:45.366824+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:40:45.366824+0000","last_undegraded":"2024-09-21T11:40:45.366824+0000","last_fullsized":"2024-09-21T11:40:45.366824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:03.386490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116484+0000","last_change":"2024-09-21T11:40:06.352201+0000","last_active":"2024-09-21T11:40:38.116484+0000","last_peered":"2024-09-21T11:40:38.116484+0000","last_clean":"2024-09-21T11:40:38.116484+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:40:38.116484+0000","last_undegraded":"2024-09-21T11:40:38.116484+0000","last_fullsized":"2024-09-21T11:40:38.116484+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.165004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":77,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116928+0000","last_change":"2024-09-21T11:40:02.399117+0000","last_active":"2024-09-21T11:40:38.116928+0000","last_peered":"2024-09-21T11:40:38.116928+0000","last_clean":"2024-09-21T11:40:38.116928+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:40:38.116928+0000","last_undegraded":"2024-09-21T11:40:38.116928+0000","last_fullsized":"2024-09-21T11:40:38.116928+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.772060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097142+0000","last_change":"2024-09-21T11:40:00.311519+0000","last_active":"2024-09-21T11:40:37.097142+0000","last_peered":"2024-09-21T11:40:37.097142+0000","last_clean":"2024-09-21T11:40:37.097142+0000","last_became_active":"2024-09-21T11:40:00.311211+0000","last_became_peered":"2024-09-21T11:40:00.311211+0000","last_unstale":"2024-09-21T11:40:37.097142+0000","last_undegraded":"2024-09-21T11:40:37.097142+0000","last_fullsized":"2024-09-21T11:40:37.097142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.982610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091780+0000","last_change":"2024-09-21T11:40:04.339202+0000","last_active":"2024-09-21T11:40:37.091780+0000","last_peered":"2024-09-21T11:40:37.091780+0000","last_clean":"2024-09-21T11:40:37.091780+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:40:37.091780+0000","last_undegraded":"2024-09-21T11:40:37.091780+0000","last_fullsized":"2024-09-21T11:40:37.091780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.313979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096533+0000","last_change":"2024-09-21T11:40:06.487976+0000","last_active":"2024-09-21T11:40:37.096533+0000","last_peered":"2024-09-21T11:40:37.096533+0000","last_clean":"2024-09-21T11:40:37.096533+0000","last_became_active":"2024-09-21T11:40:06.487644+0000","last_became_peered":"2024-09-21T11:40:06.487644+0000","last_unstale":"2024-09-21T11:40:37.096533+0000","last_undegraded":"2024-09-21T11:40:37.096533+0000","last_fullsized":"2024-09-21T11:40:37.096533+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.318979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":100,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:45.366802+0000","last_change":"2024-09-21T11:40:02.320105+0000","last_active":"2024-09-21T11:40:45.366802+0000","last_peered":"2024-09-21T11:40:45.366802+0000","last_clean":"2024-09-21T11:40:45.366802+0000","last_became_active":"2024-09-21T11:40:02.319770+0000","last_became_peered":"2024-09-21T11:40:02.319770+0000","last_unstale":"2024-09-21T11:40:45.366802+0000","last_undegraded":"2024-09-21T11:40:45.366802+0000","last_fullsized":"2024-09-21T11:40:45.366802+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.672594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119401+0000","last_change":"2024-09-21T11:40:00.314186+0000","last_active":"2024-09-21T11:40:38.119401+0000","last_peered":"2024-09-21T11:40:38.119401+0000","last_clean":"2024-09-21T11:40:38.119401+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:40:38.119401+0000","last_undegraded":"2024-09-21T11:40:38.119401+0000","last_fullsized":"2024-09-21T11:40:38.119401+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:14.587338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117801+0000","last_change":"2024-09-21T11:40:02.311375+0000","last_active":"2024-09-21T11:40:38.117801+0000","last_peered":"2024-09-21T11:40:38.117801+0000","last_clean":"2024-09-21T11:40:38.117801+0000","last_became_active":"2024-09-21T11:39:59.303309+0000","last_became_peered":"2024-09-21T11:39:59.303309+0000","last_unstale":"2024-09-21T11:40:38.117801+0000","last_undegraded":"2024-09-21T11:40:38.117801+0000","last_fullsized":"2024-09-21T11:40:38.117801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.322560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116887+0000","last_change":"2024-09-21T11:40:04.337767+0000","last_active":"2024-09-21T11:40:38.116887+0000","last_peered":"2024-09-21T11:40:38.116887+0000","last_clean":"2024-09-21T11:40:38.116887+0000","last_became_active":"2024-09-21T11:40:04.337422+0000","last_became_peered":"2024-09-21T11:40:04.337422+0000","last_unstale":"2024-09-21T11:40:38.116887+0000","last_undegraded":"2024-09-21T11:40:38.116887+0000","last_fullsized":"2024-09-21T11:40:38.116887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.195058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090431+0000","last_change":"2024-09-21T11:40:06.350784+0000","last_active":"2024-09-21T11:40:37.090431+0000","last_peered":"2024-09-21T11:40:37.090431+0000","last_clean":"2024-09-21T11:40:37.090431+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:40:37.090431+0000","last_undegraded":"2024-09-21T11:40:37.090431+0000","last_fullsized":"2024-09-21T11:40:37.090431+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.561650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097771+0000","last_change":"2024-09-21T11:40:02.324944+0000","last_active":"2024-09-21T11:40:37.097771+0000","last_peered":"2024-09-21T11:40:37.097771+0000","last_clean":"2024-09-21T11:40:37.097771+0000","last_became_active":"2024-09-21T11:40:02.324667+0000","last_became_peered":"2024-09-21T11:40:02.324667+0000","last_unstale":"2024-09-21T11:40:37.097771+0000","last_undegraded":"2024-09-21T11:40:37.097771+0000","last_fullsized":"2024-09-21T11:40:37.097771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.447522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096780+0000","last_change":"2024-09-21T11:40:00.313632+0000","last_active":"2024-09-21T11:40:37.096780+0000","last_peered":"2024-09-21T11:40:37.096780+0000","last_clean":"2024-09-21T11:40:37.096780+0000","last_became_active":"2024-09-21T11:40:00.313231+0000","last_became_peered":"2024-09-21T11:40:00.313231+0000","last_unstale":"2024-09-21T11:40:37.096780+0000","last_undegraded":"2024-09-21T11:40:37.096780+0000","last_fullsized":"2024-09-21T11:40:37.096780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.689078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089293+0000","last_change":"2024-09-21T11:40:02.305143+0000","last_active":"2024-09-21T11:40:37.089293+0000","last_peered":"2024-09-21T11:40:37.089293+0000","last_clean":"2024-09-21T11:40:37.089293+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:40:37.089293+0000","last_undegraded":"2024-09-21T11:40:37.089293+0000","last_fullsized":"2024-09-21T11:40:37.089293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.513240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089829+0000","last_change":"2024-09-21T11:40:04.332073+0000","last_active":"2024-09-21T11:40:37.089829+0000","last_peered":"2024-09-21T11:40:37.089829+0000","last_clean":"2024-09-21T11:40:37.089829+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:40:37.089829+0000","last_undegraded":"2024-09-21T11:40:37.089829+0000","last_fullsized":"2024-09-21T11:40:37.089829+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.887948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119154+0000","last_change":"2024-09-21T11:40:06.487757+0000","last_active":"2024-09-21T11:40:38.119154+0000","last_peered":"2024-09-21T11:40:38.119154+0000","last_clean":"2024-09-21T11:40:38.119154+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:40:38.119154+0000","last_undegraded":"2024-09-21T11:40:38.119154+0000","last_fullsized":"2024-09-21T11:40:38.119154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.207444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117442+0000","last_change":"2024-09-21T11:40:02.320851+0000","last_active":"2024-09-21T11:40:38.117442+0000","last_peered":"2024-09-21T11:40:38.117442+0000","last_clean":"2024-09-21T11:40:38.117442+0000","last_became_active":"2024-09-21T11:40:02.320522+0000","last_became_peered":"2024-09-21T11:40:02.320522+0000","last_unstale":"2024-09-21T11:40:38.117442+0000","last_undegraded":"2024-09-21T11:40:38.117442+0000","last_fullsized":"2024-09-21T11:40:38.117442+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:30.451592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116473+0000","last_change":"2024-09-21T11:40:00.312222+0000","last_active":"2024-09-21T11:40:38.116473+0000","last_peered":"2024-09-21T11:40:38.116473+0000","last_clean":"2024-09-21T11:40:38.116473+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:40:38.116473+0000","last_undegraded":"2024-09-21T11:40:38.116473+0000","last_fullsized":"2024-09-21T11:40:38.116473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.238210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":101,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:48.750736+0000","last_change":"2024-09-21T11:40:02.398538+0000","last_active":"2024-09-21T11:40:48.750736+0000","last_peered":"2024-09-21T11:40:48.750736+0000","last_clean":"2024-09-21T11:40:48.750736+0000","last_became_active":"2024-09-21T11:39:59.306320+0000","last_became_peered":"2024-09-21T11:39:59.306320+0000","last_unstale":"2024-09-21T11:40:48.750736+0000","last_undegraded":"2024-09-21T11:40:48.750736+0000","last_fullsized":"2024-09-21T11:40:48.750736+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:18.630069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119601+0000","last_change":"2024-09-21T11:40:04.339615+0000","last_active":"2024-09-21T11:40:38.119601+0000","last_peered":"2024-09-21T11:40:38.119601+0000","last_clean":"2024-09-21T11:40:38.119601+0000","last_became_active":"2024-09-21T11:40:04.339331+0000","last_became_peered":"2024-09-21T11:40:04.339331+0000","last_unstale":"2024-09-21T11:40:38.119601+0000","last_undegraded":"2024-09-21T11:40:38.119601+0000","last_fullsized":"2024-09-21T11:40:38.119601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:33.310635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096709+0000","last_change":"2024-09-21T11:40:06.352155+0000","last_active":"2024-09-21T11:40:37.096709+0000","last_peered":"2024-09-21T11:40:37.096709+0000","last_clean":"2024-09-21T11:40:37.096709+0000","last_became_active":"2024-09-21T11:40:06.351777+0000","last_became_peered":"2024-09-21T11:40:06.351777+0000","last_unstale":"2024-09-21T11:40:37.096709+0000","last_undegraded":"2024-09-21T11:40:37.096709+0000","last_fullsized":"2024-09-21T11:40:37.096709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.594495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":74,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:45.366978+0000","last_change":"2024-09-21T11:40:02.399268+0000","last_active":"2024-09-21T11:40:45.366978+0000","last_peered":"2024-09-21T11:40:45.366978+0000","last_clean":"2024-09-21T11:40:45.366978+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:40:45.366978+0000","last_undegraded":"2024-09-21T11:40:45.366978+0000","last_fullsized":"2024-09-21T11:40:45.366978+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.058592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090286+0000","last_change":"2024-09-21T11:40:00.309464+0000","last_active":"2024-09-21T11:40:37.090286+0000","last_peered":"2024-09-21T11:40:37.090286+0000","last_clean":"2024-09-21T11:40:37.090286+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:40:37.090286+0000","last_undegraded":"2024-09-21T11:40:37.090286+0000","last_fullsized":"2024-09-21T11:40:37.090286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.615608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":90,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118342+0000","last_change":"2024-09-21T11:40:12.500662+0000","last_active":"2024-09-21T11:40:38.118342+0000","last_peered":"2024-09-21T11:40:38.118342+0000","last_clean":"2024-09-21T11:40:38.118342+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:40:38.118342+0000","last_undegraded":"2024-09-21T11:40:38.118342+0000","last_fullsized":"2024-09-21T11:40:38.118342+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'72","last_scrub_stamp":"2024-09-21T11:40:12.500606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:40:12.500606+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.061712+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118696+0000","last_change":"2024-09-21T11:40:04.333889+0000","last_active":"2024-09-21T11:40:38.118696+0000","last_peered":"2024-09-21T11:40:38.118696+0000","last_clean":"2024-09-21T11:40:38.118696+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:40:38.118696+0000","last_undegraded":"2024-09-21T11:40:38.118696+0000","last_fullsized":"2024-09-21T11:40:38.118696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.202161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119447+0000","last_change":"2024-09-21T11:40:06.349857+0000","last_active":"2024-09-21T11:40:38.119447+0000","last_peered":"2024-09-21T11:40:38.119447+0000","last_clean":"2024-09-21T11:40:38.119447+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:40:38.119447+0000","last_undegraded":"2024-09-21T11:40:38.119447+0000","last_fullsized":"2024-09-21T11:40:38.119447+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.149814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119188+0000","last_change":"2024-09-21T11:40:02.319709+0000","last_active":"2024-09-21T11:40:38.119188+0000","last_peered":"2024-09-21T11:40:38.119188+0000","last_clean":"2024-09-21T11:40:38.119188+0000","last_became_active":"2024-09-21T11:40:02.319422+0000","last_became_peered":"2024-09-21T11:40:02.319422+0000","last_unstale":"2024-09-21T11:40:38.119188+0000","last_undegraded":"2024-09-21T11:40:38.119188+0000","last_fullsized":"2024-09-21T11:40:38.119188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.852025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117859+0000","last_change":"2024-09-21T11:40:00.312057+0000","last_active":"2024-09-21T11:40:38.117859+0000","last_peered":"2024-09-21T11:40:38.117859+0000","last_clean":"2024-09-21T11:40:38.117859+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:40:38.117859+0000","last_undegraded":"2024-09-21T11:40:38.117859+0000","last_fullsized":"2024-09-21T11:40:38.117859+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.891862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119217+0000","last_change":"2024-09-21T11:40:04.329480+0000","last_active":"2024-09-21T11:40:38.119217+0000","last_peered":"2024-09-21T11:40:38.119217+0000","last_clean":"2024-09-21T11:40:38.119217+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:40:38.119217+0000","last_undegraded":"2024-09-21T11:40:38.119217+0000","last_fullsized":"2024-09-21T11:40:38.119217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090387+0000","last_change":"2024-09-21T11:40:06.487868+0000","last_active":"2024-09-21T11:40:37.090387+0000","last_peered":"2024-09-21T11:40:37.090387+0000","last_clean":"2024-09-21T11:40:37.090387+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:40:37.090387+0000","last_undegraded":"2024-09-21T11:40:37.090387+0000","last_fullsized":"2024-09-21T11:40:37.090387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.512063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":70,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118993+0000","last_change":"2024-09-21T11:40:02.319361+0000","last_active":"2024-09-21T11:40:38.118993+0000","last_peered":"2024-09-21T11:40:38.118993+0000","last_clean":"2024-09-21T11:40:38.118993+0000","last_became_active":"2024-09-21T11:40:02.318902+0000","last_became_peered":"2024-09-21T11:40:02.318902+0000","last_unstale":"2024-09-21T11:40:38.118993+0000","last_undegraded":"2024-09-21T11:40:38.118993+0000","last_fullsized":"2024-09-21T11:40:38.118993+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.422023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092089+0000","last_change":"2024-09-21T11:40:00.313702+0000","last_active":"2024-09-21T11:40:37.092089+0000","last_peered":"2024-09-21T11:40:37.092089+0000","last_clean":"2024-09-21T11:40:37.092089+0000","last_became_active":"2024-09-21T11:40:00.313432+0000","last_became_peered":"2024-09-21T11:40:00.313432+0000","last_unstale":"2024-09-21T11:40:37.092089+0000","last_undegraded":"2024-09-21T11:40:37.092089+0000","last_fullsized":"2024-09-21T11:40:37.092089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:21.740308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096816+0000","last_change":"2024-09-21T11:40:04.342081+0000","last_active":"2024-09-21T11:40:37.096816+0000","last_peered":"2024-09-21T11:40:37.096816+0000","last_clean":"2024-09-21T11:40:37.096816+0000","last_became_active":"2024-09-21T11:40:04.341743+0000","last_became_peered":"2024-09-21T11:40:04.341743+0000","last_unstale":"2024-09-21T11:40:37.096816+0000","last_undegraded":"2024-09-21T11:40:37.096816+0000","last_fullsized":"2024-09-21T11:40:37.096816+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.025793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089044+0000","last_change":"2024-09-21T11:40:06.349130+0000","last_active":"2024-09-21T11:40:37.089044+0000","last_peered":"2024-09-21T11:40:37.089044+0000","last_clean":"2024-09-21T11:40:37.089044+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:40:37.089044+0000","last_undegraded":"2024-09-21T11:40:37.089044+0000","last_fullsized":"2024-09-21T11:40:37.089044+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:14.687896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118670+0000","last_change":"2024-09-21T11:40:02.317262+0000","last_active":"2024-09-21T11:40:38.118670+0000","last_peered":"2024-09-21T11:40:38.118670+0000","last_clean":"2024-09-21T11:40:38.118670+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:40:38.118670+0000","last_undegraded":"2024-09-21T11:40:38.118670+0000","last_fullsized":"2024-09-21T11:40:38.118670+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.898962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090545+0000","last_change":"2024-09-21T11:40:00.307721+0000","last_active":"2024-09-21T11:40:37.090545+0000","last_peered":"2024-09-21T11:40:37.090545+0000","last_clean":"2024-09-21T11:40:37.090545+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:40:37.090545+0000","last_undegraded":"2024-09-21T11:40:37.090545+0000","last_fullsized":"2024-09-21T11:40:37.090545+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.478046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:45.366764+0000","last_change":"2024-09-21T11:40:04.332939+0000","last_active":"2024-09-21T11:40:45.366764+0000","last_peered":"2024-09-21T11:40:45.366764+0000","last_clean":"2024-09-21T11:40:45.366764+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:40:45.366764+0000","last_undegraded":"2024-09-21T11:40:45.366764+0000","last_fullsized":"2024-09-21T11:40:45.366764+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.250509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118805+0000","last_change":"2024-09-21T11:40:06.346865+0000","last_active":"2024-09-21T11:40:38.118805+0000","last_peered":"2024-09-21T11:40:38.118805+0000","last_clean":"2024-09-21T11:40:38.118805+0000","last_became_active":"2024-09-21T11:40:06.346602+0000","last_became_peered":"2024-09-21T11:40:06.346602+0000","last_unstale":"2024-09-21T11:40:38.118805+0000","last_undegraded":"2024-09-21T11:40:38.118805+0000","last_fullsized":"2024-09-21T11:40:38.118805+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.724328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116816+0000","last_change":"2024-09-21T11:40:02.319787+0000","last_active":"2024-09-21T11:40:38.116816+0000","last_peered":"2024-09-21T11:40:38.116816+0000","last_clean":"2024-09-21T11:40:38.116816+0000","last_became_active":"2024-09-21T11:40:02.319502+0000","last_became_peered":"2024-09-21T11:40:02.319502+0000","last_unstale":"2024-09-21T11:40:38.116816+0000","last_undegraded":"2024-09-21T11:40:38.116816+0000","last_fullsized":"2024-09-21T11:40:38.116816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.504218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118415+0000","last_change":"2024-09-21T11:40:00.318087+0000","last_active":"2024-09-21T11:40:38.118415+0000","last_peered":"2024-09-21T11:40:38.118415+0000","last_clean":"2024-09-21T11:40:38.118415+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:40:38.118415+0000","last_undegraded":"2024-09-21T11:40:38.118415+0000","last_fullsized":"2024-09-21T11:40:38.118415+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.865238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089043+0000","last_change":"2024-09-21T11:40:04.330158+0000","last_active":"2024-09-21T11:40:37.089043+0000","last_peered":"2024-09-21T11:40:37.089043+0000","last_clean":"2024-09-21T11:40:37.089043+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:40:37.089043+0000","last_undegraded":"2024-09-21T11:40:37.089043+0000","last_fullsized":"2024-09-21T11:40:37.089043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.222022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116330+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:40:38.116330+0000","last_peered":"2024-09-21T11:40:38.116330+0000","last_clean":"2024-09-21T11:40:38.116330+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:40:38.116330+0000","last_undegraded":"2024-09-21T11:40:38.116330+0000","last_fullsized":"2024-09-21T11:40:38.116330+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:32.858006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":72,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091924+0000","last_change":"2024-09-21T11:40:02.399340+0000","last_active":"2024-09-21T11:40:37.091924+0000","last_peered":"2024-09-21T11:40:37.091924+0000","last_clean":"2024-09-21T11:40:37.091924+0000","last_became_active":"2024-09-21T11:40:02.398876+0000","last_became_peered":"2024-09-21T11:40:02.398876+0000","last_unstale":"2024-09-21T11:40:37.091924+0000","last_undegraded":"2024-09-21T11:40:37.091924+0000","last_fullsized":"2024-09-21T11:40:37.091924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.043119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119634+0000","last_change":"2024-09-21T11:40:00.314507+0000","last_active":"2024-09-21T11:40:38.119634+0000","last_peered":"2024-09-21T11:40:38.119634+0000","last_clean":"2024-09-21T11:40:38.119634+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:40:38.119634+0000","last_undegraded":"2024-09-21T11:40:38.119634+0000","last_fullsized":"2024-09-21T11:40:38.119634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.755846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089466+0000","last_change":"2024-09-21T11:40:04.341835+0000","last_active":"2024-09-21T11:40:37.089466+0000","last_peered":"2024-09-21T11:40:37.089466+0000","last_clean":"2024-09-21T11:40:37.089466+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:40:37.089466+0000","last_undegraded":"2024-09-21T11:40:37.089466+0000","last_fullsized":"2024-09-21T11:40:37.089466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:21.943269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118201+0000","last_change":"2024-09-21T11:40:06.348279+0000","last_active":"2024-09-21T11:40:38.118201+0000","last_peered":"2024-09-21T11:40:38.118201+0000","last_clean":"2024-09-21T11:40:38.118201+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:40:38.118201+0000","last_undegraded":"2024-09-21T11:40:38.118201+0000","last_fullsized":"2024-09-21T11:40:38.118201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.108944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117650+0000","last_change":"2024-09-21T11:40:02.320045+0000","last_active":"2024-09-21T11:40:38.117650+0000","last_peered":"2024-09-21T11:40:38.117650+0000","last_clean":"2024-09-21T11:40:38.117650+0000","last_became_active":"2024-09-21T11:40:02.319752+0000","last_became_peered":"2024-09-21T11:40:02.319752+0000","last_unstale":"2024-09-21T11:40:38.117650+0000","last_undegraded":"2024-09-21T11:40:38.117650+0000","last_fullsized":"2024-09-21T11:40:38.117650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.042672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.120082+0000","last_change":"2024-09-21T11:40:00.311267+0000","last_active":"2024-09-21T11:40:38.120082+0000","last_peered":"2024-09-21T11:40:38.120082+0000","last_clean":"2024-09-21T11:40:38.120082+0000","last_became_active":"2024-09-21T11:40:00.310457+0000","last_became_peered":"2024-09-21T11:40:00.310457+0000","last_unstale":"2024-09-21T11:40:38.120082+0000","last_undegraded":"2024-09-21T11:40:38.120082+0000","last_fullsized":"2024-09-21T11:40:38.120082+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.362207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.088844+0000","last_change":"2024-09-21T11:40:04.334536+0000","last_active":"2024-09-21T11:40:37.088844+0000","last_peered":"2024-09-21T11:40:37.088844+0000","last_clean":"2024-09-21T11:40:37.088844+0000","last_became_active":"2024-09-21T11:40:04.334205+0000","last_became_peered":"2024-09-21T11:40:04.334205+0000","last_unstale":"2024-09-21T11:40:37.088844+0000","last_undegraded":"2024-09-21T11:40:37.088844+0000","last_fullsized":"2024-09-21T11:40:37.088844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.125493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090268+0000","last_change":"2024-09-21T11:40:06.352508+0000","last_active":"2024-09-21T11:40:37.090268+0000","last_peered":"2024-09-21T11:40:37.090268+0000","last_clean":"2024-09-21T11:40:37.090268+0000","last_became_active":"2024-09-21T11:40:06.352210+0000","last_became_peered":"2024-09-21T11:40:06.352210+0000","last_unstale":"2024-09-21T11:40:37.090268+0000","last_undegraded":"2024-09-21T11:40:37.090268+0000","last_fullsized":"2024-09-21T11:40:37.090268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.957978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":66,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116907+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:40:38.116907+0000","last_peered":"2024-09-21T11:40:38.116907+0000","last_clean":"2024-09-21T11:40:38.116907+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:40:38.116907+0000","last_undegraded":"2024-09-21T11:40:38.116907+0000","last_fullsized":"2024-09-21T11:40:38.116907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.513015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.120072+0000","last_change":"2024-09-21T11:40:00.311525+0000","last_active":"2024-09-21T11:40:38.120072+0000","last_peered":"2024-09-21T11:40:38.120072+0000","last_clean":"2024-09-21T11:40:38.120072+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:40:38.120072+0000","last_undegraded":"2024-09-21T11:40:38.120072+0000","last_fullsized":"2024-09-21T11:40:38.120072+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.568697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:45.366972+0000","last_change":"2024-09-21T11:40:04.331990+0000","last_active":"2024-09-21T11:40:45.366972+0000","last_peered":"2024-09-21T11:40:45.366972+0000","last_clean":"2024-09-21T11:40:45.366972+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:40:45.366972+0000","last_undegraded":"2024-09-21T11:40:45.366972+0000","last_fullsized":"2024-09-21T11:40:45.366972+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.347865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117350+0000","last_change":"2024-09-21T11:40:06.487630+0000","last_active":"2024-09-21T11:40:38.117350+0000","last_peered":"2024-09-21T11:40:38.117350+0000","last_clean":"2024-09-21T11:40:38.117350+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:40:38.117350+0000","last_undegraded":"2024-09-21T11:40:38.117350+0000","last_fullsized":"2024-09-21T11:40:38.117350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.388794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092001+0000","last_change":"2024-09-21T11:40:00.310655+0000","last_active":"2024-09-21T11:40:37.092001+0000","last_peered":"2024-09-21T11:40:37.092001+0000","last_clean":"2024-09-21T11:40:37.092001+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:40:37.092001+0000","last_undegraded":"2024-09-21T11:40:37.092001+0000","last_fullsized":"2024-09-21T11:40:37.092001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.882425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091044+0000","last_change":"2024-09-21T11:40:02.399093+0000","last_active":"2024-09-21T11:40:37.091044+0000","last_peered":"2024-09-21T11:40:37.091044+0000","last_clean":"2024-09-21T11:40:37.091044+0000","last_became_active":"2024-09-21T11:40:02.398803+0000","last_became_peered":"2024-09-21T11:40:02.398803+0000","last_unstale":"2024-09-21T11:40:37.091044+0000","last_undegraded":"2024-09-21T11:40:37.091044+0000","last_fullsized":"2024-09-21T11:40:37.091044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.504556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118790+0000","last_change":"2024-09-21T11:40:04.328297+0000","last_active":"2024-09-21T11:40:38.118790+0000","last_peered":"2024-09-21T11:40:38.118790+0000","last_clean":"2024-09-21T11:40:38.118790+0000","last_became_active":"2024-09-21T11:40:04.327422+0000","last_became_peered":"2024-09-21T11:40:04.327422+0000","last_unstale":"2024-09-21T11:40:38.118790+0000","last_undegraded":"2024-09-21T11:40:38.118790+0000","last_fullsized":"2024-09-21T11:40:38.118790+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.663596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119034+0000","last_change":"2024-09-21T11:40:06.488399+0000","last_active":"2024-09-21T11:40:38.119034+0000","last_peered":"2024-09-21T11:40:38.119034+0000","last_clean":"2024-09-21T11:40:38.119034+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:40:38.119034+0000","last_undegraded":"2024-09-21T11:40:38.119034+0000","last_fullsized":"2024-09-21T11:40:38.119034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.617144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117426+0000","last_change":"2024-09-21T11:40:02.399564+0000","last_active":"2024-09-21T11:40:38.117426+0000","last_peered":"2024-09-21T11:40:38.117426+0000","last_clean":"2024-09-21T11:40:38.117426+0000","last_became_active":"2024-09-21T11:40:02.399267+0000","last_became_peered":"2024-09-21T11:40:02.399267+0000","last_unstale":"2024-09-21T11:40:38.117426+0000","last_undegraded":"2024-09-21T11:40:38.117426+0000","last_fullsized":"2024-09-21T11:40:38.117426+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.891469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118903+0000","last_change":"2024-09-21T11:40:00.308959+0000","last_active":"2024-09-21T11:40:38.118903+0000","last_peered":"2024-09-21T11:40:38.118903+0000","last_clean":"2024-09-21T11:40:38.118903+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:40:38.118903+0000","last_undegraded":"2024-09-21T11:40:38.118903+0000","last_fullsized":"2024-09-21T11:40:38.118903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.467036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090927+0000","last_change":"2024-09-21T11:40:04.326964+0000","last_active":"2024-09-21T11:40:37.090927+0000","last_peered":"2024-09-21T11:40:37.090927+0000","last_clean":"2024-09-21T11:40:37.090927+0000","last_became_active":"2024-09-21T11:40:04.326625+0000","last_became_peered":"2024-09-21T11:40:04.326625+0000","last_unstale":"2024-09-21T11:40:37.090927+0000","last_undegraded":"2024-09-21T11:40:37.090927+0000","last_fullsized":"2024-09-21T11:40:37.090927+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089630+0000","last_change":"2024-09-21T11:40:06.343913+0000","last_active":"2024-09-21T11:40:37.089630+0000","last_peered":"2024-09-21T11:40:37.089630+0000","last_clean":"2024-09-21T11:40:37.089630+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:40:37.089630+0000","last_undegraded":"2024-09-21T11:40:37.089630+0000","last_fullsized":"2024-09-21T11:40:37.089630+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.682920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119505+0000","last_change":"2024-09-21T11:40:02.400725+0000","last_active":"2024-09-21T11:40:38.119505+0000","last_peered":"2024-09-21T11:40:38.119505+0000","last_clean":"2024-09-21T11:40:38.119505+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:40:38.119505+0000","last_undegraded":"2024-09-21T11:40:38.119505+0000","last_fullsized":"2024-09-21T11:40:38.119505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.300079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116459+0000","last_change":"2024-09-21T11:40:00.311525+0000","last_active":"2024-09-21T11:40:38.116459+0000","last_peered":"2024-09-21T11:40:38.116459+0000","last_clean":"2024-09-21T11:40:38.116459+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:40:38.116459+0000","last_undegraded":"2024-09-21T11:40:38.116459+0000","last_fullsized":"2024-09-21T11:40:38.116459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.269082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:45.367124+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:40:45.367124+0000","last_peered":"2024-09-21T11:40:45.367124+0000","last_clean":"2024-09-21T11:40:45.367124+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:40:45.367124+0000","last_undegraded":"2024-09-21T11:40:45.367124+0000","last_fullsized":"2024-09-21T11:40:45.367124+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.404995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118969+0000","last_change":"2024-09-21T11:40:06.346686+0000","last_active":"2024-09-21T11:40:38.118969+0000","last_peered":"2024-09-21T11:40:38.118969+0000","last_clean":"2024-09-21T11:40:38.118969+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:40:38.118969+0000","last_undegraded":"2024-09-21T11:40:38.118969+0000","last_fullsized":"2024-09-21T11:40:38.118969+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.014386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090690+0000","last_change":"2024-09-21T11:40:02.399404+0000","last_active":"2024-09-21T11:40:37.090690+0000","last_peered":"2024-09-21T11:40:37.090690+0000","last_clean":"2024-09-21T11:40:37.090690+0000","last_became_active":"2024-09-21T11:40:02.398771+0000","last_became_peered":"2024-09-21T11:40:02.398771+0000","last_unstale":"2024-09-21T11:40:37.090690+0000","last_undegraded":"2024-09-21T11:40:37.090690+0000","last_fullsized":"2024-09-21T11:40:37.090690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.091232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118174+0000","last_change":"2024-09-21T11:40:00.314627+0000","last_active":"2024-09-21T11:40:38.118174+0000","last_peered":"2024-09-21T11:40:38.118174+0000","last_clean":"2024-09-21T11:40:38.118174+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:40:38.118174+0000","last_undegraded":"2024-09-21T11:40:38.118174+0000","last_fullsized":"2024-09-21T11:40:38.118174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.535972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:45.366927+0000","last_change":"2024-09-21T11:40:04.340164+0000","last_active":"2024-09-21T11:40:45.366927+0000","last_peered":"2024-09-21T11:40:45.366927+0000","last_clean":"2024-09-21T11:40:45.366927+0000","last_became_active":"2024-09-21T11:40:04.339253+0000","last_became_peered":"2024-09-21T11:40:04.339253+0000","last_unstale":"2024-09-21T11:40:45.366927+0000","last_undegraded":"2024-09-21T11:40:45.366927+0000","last_fullsized":"2024-09-21T11:40:45.366927+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:30.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116791+0000","last_change":"2024-09-21T11:40:06.351973+0000","last_active":"2024-09-21T11:40:38.116791+0000","last_peered":"2024-09-21T11:40:38.116791+0000","last_clean":"2024-09-21T11:40:38.116791+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:40:38.116791+0000","last_undegraded":"2024-09-21T11:40:38.116791+0000","last_fullsized":"2024-09-21T11:40:38.116791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.555091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119008+0000","last_change":"2024-09-21T11:40:02.318645+0000","last_active":"2024-09-21T11:40:38.119008+0000","last_peered":"2024-09-21T11:40:38.119008+0000","last_clean":"2024-09-21T11:40:38.119008+0000","last_became_active":"2024-09-21T11:40:02.318352+0000","last_became_peered":"2024-09-21T11:40:02.318352+0000","last_unstale":"2024-09-21T11:40:38.119008+0000","last_undegraded":"2024-09-21T11:40:38.119008+0000","last_fullsized":"2024-09-21T11:40:38.119008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.071118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119237+0000","last_change":"2024-09-21T11:40:00.317768+0000","last_active":"2024-09-21T11:40:38.119237+0000","last_peered":"2024-09-21T11:40:38.119237+0000","last_clean":"2024-09-21T11:40:38.119237+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:40:38.119237+0000","last_undegraded":"2024-09-21T11:40:38.119237+0000","last_fullsized":"2024-09-21T11:40:38.119237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.919244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.096986+0000","last_change":"2024-09-21T11:40:04.328561+0000","last_active":"2024-09-21T11:40:37.096986+0000","last_peered":"2024-09-21T11:40:37.096986+0000","last_clean":"2024-09-21T11:40:37.096986+0000","last_became_active":"2024-09-21T11:40:04.328216+0000","last_became_peered":"2024-09-21T11:40:04.328216+0000","last_unstale":"2024-09-21T11:40:37.096986+0000","last_undegraded":"2024-09-21T11:40:37.096986+0000","last_fullsized":"2024-09-21T11:40:37.096986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.146311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089641+0000","last_change":"2024-09-21T11:40:06.353577+0000","last_active":"2024-09-21T11:40:37.089641+0000","last_peered":"2024-09-21T11:40:37.089641+0000","last_clean":"2024-09-21T11:40:37.089641+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:40:37.089641+0000","last_undegraded":"2024-09-21T11:40:37.089641+0000","last_fullsized":"2024-09-21T11:40:37.089641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.269429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097098+0000","last_change":"2024-09-21T11:40:02.319305+0000","last_active":"2024-09-21T11:40:37.097098+0000","last_peered":"2024-09-21T11:40:37.097098+0000","last_clean":"2024-09-21T11:40:37.097098+0000","last_became_active":"2024-09-21T11:40:02.318699+0000","last_became_peered":"2024-09-21T11:40:02.318699+0000","last_unstale":"2024-09-21T11:40:37.097098+0000","last_undegraded":"2024-09-21T11:40:37.097098+0000","last_fullsized":"2024-09-21T11:40:37.097098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.659293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.119439+0000","last_change":"2024-09-21T11:40:00.309690+0000","last_active":"2024-09-21T11:40:38.119439+0000","last_peered":"2024-09-21T11:40:38.119439+0000","last_clean":"2024-09-21T11:40:38.119439+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:40:38.119439+0000","last_undegraded":"2024-09-21T11:40:38.119439+0000","last_fullsized":"2024-09-21T11:40:38.119439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.952393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090841+0000","last_change":"2024-09-21T11:40:04.336820+0000","last_active":"2024-09-21T11:40:37.090841+0000","last_peered":"2024-09-21T11:40:37.090841+0000","last_clean":"2024-09-21T11:40:37.090841+0000","last_became_active":"2024-09-21T11:40:04.336469+0000","last_became_peered":"2024-09-21T11:40:04.336469+0000","last_unstale":"2024-09-21T11:40:37.090841+0000","last_undegraded":"2024-09-21T11:40:37.090841+0000","last_fullsized":"2024-09-21T11:40:37.090841+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.369123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116254+0000","last_change":"2024-09-21T11:40:06.346455+0000","last_active":"2024-09-21T11:40:38.116254+0000","last_peered":"2024-09-21T11:40:38.116254+0000","last_clean":"2024-09-21T11:40:38.116254+0000","last_became_active":"2024-09-21T11:40:06.346090+0000","last_became_peered":"2024-09-21T11:40:06.346090+0000","last_unstale":"2024-09-21T11:40:38.116254+0000","last_undegraded":"2024-09-21T11:40:38.116254+0000","last_fullsized":"2024-09-21T11:40:38.116254+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.873349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089766+0000","last_change":"2024-09-21T11:40:02.399706+0000","last_active":"2024-09-21T11:40:37.089766+0000","last_peered":"2024-09-21T11:40:37.089766+0000","last_clean":"2024-09-21T11:40:37.089766+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:40:37.089766+0000","last_undegraded":"2024-09-21T11:40:37.089766+0000","last_fullsized":"2024-09-21T11:40:37.089766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.937696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.118907+0000","last_change":"2024-09-21T11:40:00.317057+0000","last_active":"2024-09-21T11:40:38.118907+0000","last_peered":"2024-09-21T11:40:38.118907+0000","last_clean":"2024-09-21T11:40:38.118907+0000","last_became_active":"2024-09-21T11:40:00.316756+0000","last_became_peered":"2024-09-21T11:40:00.316756+0000","last_unstale":"2024-09-21T11:40:38.118907+0000","last_undegraded":"2024-09-21T11:40:38.118907+0000","last_fullsized":"2024-09-21T11:40:38.118907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.129760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116741+0000","last_change":"2024-09-21T11:40:04.331954+0000","last_active":"2024-09-21T11:40:38.116741+0000","last_peered":"2024-09-21T11:40:38.116741+0000","last_clean":"2024-09-21T11:40:38.116741+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:40:38.116741+0000","last_undegraded":"2024-09-21T11:40:38.116741+0000","last_fullsized":"2024-09-21T11:40:38.116741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.778314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.090438+0000","last_change":"2024-09-21T11:40:06.488252+0000","last_active":"2024-09-21T11:40:37.090438+0000","last_peered":"2024-09-21T11:40:37.090438+0000","last_clean":"2024-09-21T11:40:37.090438+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:40:37.090438+0000","last_undegraded":"2024-09-21T11:40:37.090438+0000","last_fullsized":"2024-09-21T11:40:37.090438+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.158855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":48,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089194+0000","last_change":"2024-09-21T11:40:02.319060+0000","last_active":"2024-09-21T11:40:37.089194+0000","last_peered":"2024-09-21T11:40:37.089194+0000","last_clean":"2024-09-21T11:40:37.089194+0000","last_became_active":"2024-09-21T11:40:02.318727+0000","last_became_peered":"2024-09-21T11:40:02.318727+0000","last_unstale":"2024-09-21T11:40:37.089194+0000","last_undegraded":"2024-09-21T11:40:37.089194+0000","last_fullsized":"2024-09-21T11:40:37.089194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.877093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116996+0000","last_change":"2024-09-21T11:40:00.314387+0000","last_active":"2024-09-21T11:40:38.116996+0000","last_peered":"2024-09-21T11:40:38.116996+0000","last_clean":"2024-09-21T11:40:38.116996+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:40:38.116996+0000","last_undegraded":"2024-09-21T11:40:38.116996+0000","last_fullsized":"2024-09-21T11:40:38.116996+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:00.574148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.091722+0000","last_change":"2024-09-21T11:40:04.332492+0000","last_active":"2024-09-21T11:40:37.091722+0000","last_peered":"2024-09-21T11:40:37.091722+0000","last_clean":"2024-09-21T11:40:37.091722+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:40:37.091722+0000","last_undegraded":"2024-09-21T11:40:37.091722+0000","last_fullsized":"2024-09-21T11:40:37.091722+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.592217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.117167+0000","last_change":"2024-09-21T11:40:06.342962+0000","last_active":"2024-09-21T11:40:38.117167+0000","last_peered":"2024-09-21T11:40:38.117167+0000","last_clean":"2024-09-21T11:40:38.117167+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:40:38.117167+0000","last_undegraded":"2024-09-21T11:40:38.117167+0000","last_fullsized":"2024-09-21T11:40:38.117167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.430215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.097021+0000","last_change":"2024-09-21T11:40:06.349089+0000","last_active":"2024-09-21T11:40:37.097021+0000","last_peered":"2024-09-21T11:40:37.097021+0000","last_clean":"2024-09-21T11:40:37.097021+0000","last_became_active":"2024-09-21T11:40:06.348796+0000","last_became_peered":"2024-09-21T11:40:06.348796+0000","last_unstale":"2024-09-21T11:40:37.097021+0000","last_undegraded":"2024-09-21T11:40:37.097021+0000","last_fullsized":"2024-09-21T11:40:37.097021+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.850881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.089414+0000","last_change":"2024-09-21T11:40:00.309865+0000","last_active":"2024-09-21T11:40:37.089414+0000","last_peered":"2024-09-21T11:40:37.089414+0000","last_clean":"2024-09-21T11:40:37.089414+0000","last_became_active":"2024-09-21T11:40:00.309598+0000","last_became_peered":"2024-09-21T11:40:00.309598+0000","last_unstale":"2024-09-21T11:40:37.089414+0000","last_undegraded":"2024-09-21T11:40:37.089414+0000","last_fullsized":"2024-09-21T11:40:37.089414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.604856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":60,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:37.092528+0000","last_change":"2024-09-21T11:40:02.318498+0000","last_active":"2024-09-21T11:40:37.092528+0000","last_peered":"2024-09-21T11:40:37.092528+0000","last_clean":"2024-09-21T11:40:37.092528+0000","last_became_active":"2024-09-21T11:40:02.318160+0000","last_became_peered":"2024-09-21T11:40:02.318160+0000","last_unstale":"2024-09-21T11:40:37.092528+0000","last_undegraded":"2024-09-21T11:40:37.092528+0000","last_fullsized":"2024-09-21T11:40:37.092528+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.280806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":37,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T11:40:38.116971+0000","last_change":"2024-09-21T11:40:04.340406+0000","last_active":"2024-09-21T11:40:38.116971+0000","last_peered":"2024-09-21T11:40:38.116971+0000","last_clean":"2024-09-21T11:40:38.116971+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:40:38.116971+0000","last_undegraded":"2024-09-21T11:40:38.116971+0000","last_fullsized":"2024-09-21T11:40:38.116971+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.851842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928233999,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28364,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716788,"statfs":{"total":95995035648,"available":95965990912,"internally_reserved":0,"allocated":1368064,"data_stored":898615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462932,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712640,"statfs":{"total":95995035648,"available":95961743360,"internally_reserved":0,"allocated":1355776,"data_stored":898959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724569,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31932,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713220,"statfs":{"total":95995035648,"available":95962337280,"internally_reserved":0,"allocated":761856,"data_stored":307534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920799,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":786432,"data_stored":308760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117029,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":790528,"data_stored":308053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411371,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27768,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717384,"statfs":{"total":95995035648,"available":95966601216,"internally_reserved":0,"allocated":753664,"data_stored":307746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":827392,"data_stored":309396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32536,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712616,"statfs":{"total":95995035648,"available":95961718784,"internally_reserved":0,"allocated":1380352,"data_stored":899330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:40:56.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:40:56.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:56 smithi175 ceph-mon[32613]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:56.712 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:56 smithi043 podman[67867]: 2024-09-21 11:40:56.618386905 +0000 UTC m=+0.137016334 volume create b000671e77d16f4ea924ab2d7e39ec8530f66e690278175041fd009df8a4c966 2024-09-21T11:40:56.991 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:56 smithi043 podman[67867]: 2024-09-21 11:40:56.710437472 +0000 UTC m=+0.229066945 container create f988a64964725524b3a25c9126c8195df2911e33fb686b30209662ed3f6757f3 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:40:57.222 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:57.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-mon[29710]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T11:40:57.341 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-mon[29710]: osdmap e78: 8 total, 8 up, 8 in 2024-09-21T11:40:57.342 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-mon[36454]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T11:40:57.342 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-mon[36454]: osdmap e78: 8 total, 8 up, 8 in 2024-09-21T11:40:57.342 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 podman[67867]: 2024-09-21 11:40:57.047103532 +0000 UTC m=+0.565732979 container init f988a64964725524b3a25c9126c8195df2911e33fb686b30209662ed3f6757f3 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:40:57.342 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 podman[67867]: 2024-09-21 11:40:57.050140031 +0000 UTC m=+0.568769471 container start f988a64964725524b3a25c9126c8195df2911e33fb686b30209662ed3f6757f3 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:40:57.342 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:40:57.062Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-21T11:40:57.342 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:40:57.062Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-21T11:40:57.342 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:40:57.063Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.43 port=9094 2024-09-21T11:40:57.342 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:40:57.064Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-21T11:40:57.342 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:40:57.085Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-21T11:40:57.342 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:40:57.086Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-21T11:40:57.342 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:40:57.087Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-21T11:40:57.343 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:40:57.087Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-21T11:40:57.343 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 bash[67867]: f988a64964725524b3a25c9126c8195df2911e33fb686b30209662ed3f6757f3 2024-09-21T11:40:57.343 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:57 smithi043 systemd[1]: Started Ceph alertmanager.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:40:57.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:57 smithi175 ceph-mon[32613]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-21T11:40:57.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:57 smithi175 ceph-mon[32613]: osdmap e78: 8 total, 8 up, 8 in 2024-09-21T11:40:57.893 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:40:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:40:58.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[36454]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-21T11:40:58.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-21T11:40:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:58] ENGINE Bus STOPPING 2024-09-21T11:40:58.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:58] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T11:40:58.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:58] ENGINE Bus STOPPED 2024-09-21T11:40:58.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:58] ENGINE Bus STARTING 2024-09-21T11:40:58.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:58] ENGINE Serving on http://:::9283 2024-09-21T11:40:58.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:40:58] ENGINE Bus STARTED 2024-09-21T11:40:58.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[29710]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-21T11:40:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-21T11:40:58.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:58 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:58 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:58 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:58 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:58 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:58 smithi175 ceph-mon[32613]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-21T11:40:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:58 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:58 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:58 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-21T11:40:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:58 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:40:58.651 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-21T11:40:57.694159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":898,"num_read_kb":755,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244944,"kb_used_data":7920,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716272,"statfs":{"total":767960285184,"available":767709462528,"internally_reserved":0,"allocated":8110080,"data_stored":4283347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001860"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111222+0000","last_change":"2024-09-21T11:40:00.313497+0000","last_active":"2024-09-21T11:40:54.111222+0000","last_peered":"2024-09-21T11:40:54.111222+0000","last_clean":"2024-09-21T11:40:54.111222+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:40:54.111222+0000","last_undegraded":"2024-09-21T11:40:54.111222+0000","last_fullsized":"2024-09-21T11:40:54.111222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.636264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109318+0000","last_change":"2024-09-21T11:40:02.319143+0000","last_active":"2024-09-21T11:40:54.109318+0000","last_peered":"2024-09-21T11:40:54.109318+0000","last_clean":"2024-09-21T11:40:54.109318+0000","last_became_active":"2024-09-21T11:40:02.318639+0000","last_became_peered":"2024-09-21T11:40:02.318639+0000","last_unstale":"2024-09-21T11:40:54.109318+0000","last_undegraded":"2024-09-21T11:40:54.109318+0000","last_fullsized":"2024-09-21T11:40:54.109318+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.717901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109354+0000","last_change":"2024-09-21T11:40:04.331203+0000","last_active":"2024-09-21T11:40:54.109354+0000","last_peered":"2024-09-21T11:40:54.109354+0000","last_clean":"2024-09-21T11:40:54.109354+0000","last_became_active":"2024-09-21T11:40:04.330879+0000","last_became_peered":"2024-09-21T11:40:04.330879+0000","last_unstale":"2024-09-21T11:40:54.109354+0000","last_undegraded":"2024-09-21T11:40:54.109354+0000","last_fullsized":"2024-09-21T11:40:54.109354+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.860431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109506+0000","last_change":"2024-09-21T11:40:06.346317+0000","last_active":"2024-09-21T11:40:54.109506+0000","last_peered":"2024-09-21T11:40:54.109506+0000","last_clean":"2024-09-21T11:40:54.109506+0000","last_became_active":"2024-09-21T11:40:06.345995+0000","last_became_peered":"2024-09-21T11:40:06.345995+0000","last_unstale":"2024-09-21T11:40:54.109506+0000","last_undegraded":"2024-09-21T11:40:54.109506+0000","last_fullsized":"2024-09-21T11:40:54.109506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.419437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391466+0000","last_change":"2024-09-21T11:40:06.489876+0000","last_active":"2024-09-21T11:40:54.391466+0000","last_peered":"2024-09-21T11:40:54.391466+0000","last_clean":"2024-09-21T11:40:54.391466+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:40:54.391466+0000","last_undegraded":"2024-09-21T11:40:54.391466+0000","last_fullsized":"2024-09-21T11:40:54.391466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.939900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391989+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:40:54.391989+0000","last_peered":"2024-09-21T11:40:54.391989+0000","last_clean":"2024-09-21T11:40:54.391989+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:40:54.391989+0000","last_undegraded":"2024-09-21T11:40:54.391989+0000","last_fullsized":"2024-09-21T11:40:54.391989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.921258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":69,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391461+0000","last_change":"2024-09-21T11:40:02.315669+0000","last_active":"2024-09-21T11:40:54.391461+0000","last_peered":"2024-09-21T11:40:54.391461+0000","last_clean":"2024-09-21T11:40:54.391461+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:40:54.391461+0000","last_undegraded":"2024-09-21T11:40:54.391461+0000","last_fullsized":"2024-09-21T11:40:54.391461+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.511141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109299+0000","last_change":"2024-09-21T11:40:04.326924+0000","last_active":"2024-09-21T11:40:54.109299+0000","last_peered":"2024-09-21T11:40:54.109299+0000","last_clean":"2024-09-21T11:40:54.109299+0000","last_became_active":"2024-09-21T11:40:04.326663+0000","last_became_peered":"2024-09-21T11:40:04.326663+0000","last_unstale":"2024-09-21T11:40:54.109299+0000","last_undegraded":"2024-09-21T11:40:54.109299+0000","last_fullsized":"2024-09-21T11:40:54.109299+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:04.814146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.651237+0000","last_change":"2024-09-21T11:40:00.314157+0000","last_active":"2024-09-21T11:40:54.651237+0000","last_peered":"2024-09-21T11:40:54.651237+0000","last_clean":"2024-09-21T11:40:54.651237+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:40:54.651237+0000","last_undegraded":"2024-09-21T11:40:54.651237+0000","last_fullsized":"2024-09-21T11:40:54.651237+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.659277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109689+0000","last_change":"2024-09-21T11:40:02.314170+0000","last_active":"2024-09-21T11:40:54.109689+0000","last_peered":"2024-09-21T11:40:54.109689+0000","last_clean":"2024-09-21T11:40:54.109689+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:40:54.109689+0000","last_undegraded":"2024-09-21T11:40:54.109689+0000","last_fullsized":"2024-09-21T11:40:54.109689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.670327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.650877+0000","last_change":"2024-09-21T11:40:04.339616+0000","last_active":"2024-09-21T11:40:54.650877+0000","last_peered":"2024-09-21T11:40:54.650877+0000","last_clean":"2024-09-21T11:40:54.650877+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:40:54.650877+0000","last_undegraded":"2024-09-21T11:40:54.650877+0000","last_fullsized":"2024-09-21T11:40:54.650877+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.214530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110126+0000","last_change":"2024-09-21T11:40:06.344622+0000","last_active":"2024-09-21T11:40:54.110126+0000","last_peered":"2024-09-21T11:40:54.110126+0000","last_clean":"2024-09-21T11:40:54.110126+0000","last_became_active":"2024-09-21T11:40:06.344291+0000","last_became_peered":"2024-09-21T11:40:06.344291+0000","last_unstale":"2024-09-21T11:40:54.110126+0000","last_undegraded":"2024-09-21T11:40:54.110126+0000","last_fullsized":"2024-09-21T11:40:54.110126+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.653671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.651034+0000","last_change":"2024-09-21T11:40:00.315137+0000","last_active":"2024-09-21T11:40:54.651034+0000","last_peered":"2024-09-21T11:40:54.651034+0000","last_clean":"2024-09-21T11:40:54.651034+0000","last_became_active":"2024-09-21T11:40:00.314838+0000","last_became_peered":"2024-09-21T11:40:00.314838+0000","last_unstale":"2024-09-21T11:40:54.651034+0000","last_undegraded":"2024-09-21T11:40:54.651034+0000","last_fullsized":"2024-09-21T11:40:54.651034+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:40:59.501107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109608+0000","last_change":"2024-09-21T11:40:02.320524+0000","last_active":"2024-09-21T11:40:54.109608+0000","last_peered":"2024-09-21T11:40:54.109608+0000","last_clean":"2024-09-21T11:40:54.109608+0000","last_became_active":"2024-09-21T11:40:02.320184+0000","last_became_peered":"2024-09-21T11:40:02.320184+0000","last_unstale":"2024-09-21T11:40:54.109608+0000","last_undegraded":"2024-09-21T11:40:54.109608+0000","last_fullsized":"2024-09-21T11:40:54.109608+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.631864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766827+0000","last_change":"2024-09-21T11:40:04.333696+0000","last_active":"2024-09-21T11:40:54.766827+0000","last_peered":"2024-09-21T11:40:54.766827+0000","last_clean":"2024-09-21T11:40:54.766827+0000","last_became_active":"2024-09-21T11:40:04.333406+0000","last_became_peered":"2024-09-21T11:40:04.333406+0000","last_unstale":"2024-09-21T11:40:54.766827+0000","last_undegraded":"2024-09-21T11:40:54.766827+0000","last_fullsized":"2024-09-21T11:40:54.766827+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.055519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.650087+0000","last_change":"2024-09-21T11:40:06.349798+0000","last_active":"2024-09-21T11:40:54.650087+0000","last_peered":"2024-09-21T11:40:54.650087+0000","last_clean":"2024-09-21T11:40:54.650087+0000","last_became_active":"2024-09-21T11:40:06.349538+0000","last_became_peered":"2024-09-21T11:40:06.349538+0000","last_unstale":"2024-09-21T11:40:54.650087+0000","last_undegraded":"2024-09-21T11:40:54.650087+0000","last_fullsized":"2024-09-21T11:40:54.650087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:34.219537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110066+0000","last_change":"2024-09-21T11:40:06.487919+0000","last_active":"2024-09-21T11:40:54.110066+0000","last_peered":"2024-09-21T11:40:54.110066+0000","last_clean":"2024-09-21T11:40:54.110066+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:40:54.110066+0000","last_undegraded":"2024-09-21T11:40:54.110066+0000","last_fullsized":"2024-09-21T11:40:54.110066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:18.602419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111294+0000","last_change":"2024-09-21T11:40:00.312124+0000","last_active":"2024-09-21T11:40:54.111294+0000","last_peered":"2024-09-21T11:40:54.111294+0000","last_clean":"2024-09-21T11:40:54.111294+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:40:54.111294+0000","last_undegraded":"2024-09-21T11:40:54.111294+0000","last_fullsized":"2024-09-21T11:40:54.111294+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.023370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":69,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.841175+0000","last_change":"2024-09-21T11:40:02.319850+0000","last_active":"2024-09-21T11:40:54.841175+0000","last_peered":"2024-09-21T11:40:54.841175+0000","last_clean":"2024-09-21T11:40:54.841175+0000","last_became_active":"2024-09-21T11:40:02.319153+0000","last_became_peered":"2024-09-21T11:40:02.319153+0000","last_unstale":"2024-09-21T11:40:54.841175+0000","last_undegraded":"2024-09-21T11:40:54.841175+0000","last_fullsized":"2024-09-21T11:40:54.841175+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.765972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110146+0000","last_change":"2024-09-21T11:40:04.339562+0000","last_active":"2024-09-21T11:40:54.110146+0000","last_peered":"2024-09-21T11:40:54.110146+0000","last_clean":"2024-09-21T11:40:54.110146+0000","last_became_active":"2024-09-21T11:40:04.339222+0000","last_became_peered":"2024-09-21T11:40:04.339222+0000","last_unstale":"2024-09-21T11:40:54.110146+0000","last_undegraded":"2024-09-21T11:40:54.110146+0000","last_fullsized":"2024-09-21T11:40:54.110146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.022703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.392055+0000","last_change":"2024-09-21T11:40:06.487933+0000","last_active":"2024-09-21T11:40:54.392055+0000","last_peered":"2024-09-21T11:40:54.392055+0000","last_clean":"2024-09-21T11:40:54.392055+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:40:54.392055+0000","last_undegraded":"2024-09-21T11:40:54.392055+0000","last_fullsized":"2024-09-21T11:40:54.392055+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.209907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110049+0000","last_change":"2024-09-21T11:40:00.312534+0000","last_active":"2024-09-21T11:40:54.110049+0000","last_peered":"2024-09-21T11:40:54.110049+0000","last_clean":"2024-09-21T11:40:54.110049+0000","last_became_active":"2024-09-21T11:40:00.312239+0000","last_became_peered":"2024-09-21T11:40:00.312239+0000","last_unstale":"2024-09-21T11:40:54.110049+0000","last_undegraded":"2024-09-21T11:40:54.110049+0000","last_fullsized":"2024-09-21T11:40:54.110049+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.443761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":67,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391828+0000","last_change":"2024-09-21T11:40:02.324039+0000","last_active":"2024-09-21T11:40:54.391828+0000","last_peered":"2024-09-21T11:40:54.391828+0000","last_clean":"2024-09-21T11:40:54.391828+0000","last_became_active":"2024-09-21T11:40:02.323784+0000","last_became_peered":"2024-09-21T11:40:02.323784+0000","last_unstale":"2024-09-21T11:40:54.391828+0000","last_undegraded":"2024-09-21T11:40:54.391828+0000","last_fullsized":"2024-09-21T11:40:54.391828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.707671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109610+0000","last_change":"2024-09-21T11:40:04.328865+0000","last_active":"2024-09-21T11:40:54.109610+0000","last_peered":"2024-09-21T11:40:54.109610+0000","last_clean":"2024-09-21T11:40:54.109610+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:40:54.109610+0000","last_undegraded":"2024-09-21T11:40:54.109610+0000","last_fullsized":"2024-09-21T11:40:54.109610+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.407835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766990+0000","last_change":"2024-09-21T11:40:06.340320+0000","last_active":"2024-09-21T11:40:54.766990+0000","last_peered":"2024-09-21T11:40:54.766990+0000","last_clean":"2024-09-21T11:40:54.766990+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:40:54.766990+0000","last_undegraded":"2024-09-21T11:40:54.766990+0000","last_fullsized":"2024-09-21T11:40:54.766990+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.731856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109576+0000","last_change":"2024-09-21T11:40:00.308535+0000","last_active":"2024-09-21T11:40:54.109576+0000","last_peered":"2024-09-21T11:40:54.109576+0000","last_clean":"2024-09-21T11:40:54.109576+0000","last_became_active":"2024-09-21T11:40:00.308219+0000","last_became_peered":"2024-09-21T11:40:00.308219+0000","last_unstale":"2024-09-21T11:40:54.109576+0000","last_undegraded":"2024-09-21T11:40:54.109576+0000","last_fullsized":"2024-09-21T11:40:54.109576+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.036879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111487+0000","last_change":"2024-09-21T11:40:02.400244+0000","last_active":"2024-09-21T11:40:54.111487+0000","last_peered":"2024-09-21T11:40:54.111487+0000","last_clean":"2024-09-21T11:40:54.111487+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:40:54.111487+0000","last_undegraded":"2024-09-21T11:40:54.111487+0000","last_fullsized":"2024-09-21T11:40:54.111487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.785487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.603420+0000","last_change":"2024-09-21T11:40:04.328515+0000","last_active":"2024-09-21T11:40:54.603420+0000","last_peered":"2024-09-21T11:40:54.603420+0000","last_clean":"2024-09-21T11:40:54.603420+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:40:54.603420+0000","last_undegraded":"2024-09-21T11:40:54.603420+0000","last_fullsized":"2024-09-21T11:40:54.603420+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:03.806494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":68,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111348+0000","last_change":"2024-09-21T11:40:02.323539+0000","last_active":"2024-09-21T11:40:54.111348+0000","last_peered":"2024-09-21T11:40:54.111348+0000","last_clean":"2024-09-21T11:40:54.111348+0000","last_became_active":"2024-09-21T11:40:02.323204+0000","last_became_peered":"2024-09-21T11:40:02.323204+0000","last_unstale":"2024-09-21T11:40:54.111348+0000","last_undegraded":"2024-09-21T11:40:54.111348+0000","last_fullsized":"2024-09-21T11:40:54.111348+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.577998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391163+0000","last_change":"2024-09-21T11:40:00.315307+0000","last_active":"2024-09-21T11:40:54.391163+0000","last_peered":"2024-09-21T11:40:54.391163+0000","last_clean":"2024-09-21T11:40:54.391163+0000","last_became_active":"2024-09-21T11:40:00.315015+0000","last_became_peered":"2024-09-21T11:40:00.315015+0000","last_unstale":"2024-09-21T11:40:54.391163+0000","last_undegraded":"2024-09-21T11:40:54.391163+0000","last_fullsized":"2024-09-21T11:40:54.391163+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.959497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110262+0000","last_change":"2024-09-21T11:40:04.342645+0000","last_active":"2024-09-21T11:40:54.110262+0000","last_peered":"2024-09-21T11:40:54.110262+0000","last_clean":"2024-09-21T11:40:54.110262+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:40:54.110262+0000","last_undegraded":"2024-09-21T11:40:54.110262+0000","last_fullsized":"2024-09-21T11:40:54.110262+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.259518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.650273+0000","last_change":"2024-09-21T11:40:06.345062+0000","last_active":"2024-09-21T11:40:54.650273+0000","last_peered":"2024-09-21T11:40:54.650273+0000","last_clean":"2024-09-21T11:40:54.650273+0000","last_became_active":"2024-09-21T11:40:06.344796+0000","last_became_peered":"2024-09-21T11:40:06.344796+0000","last_unstale":"2024-09-21T11:40:54.650273+0000","last_undegraded":"2024-09-21T11:40:54.650273+0000","last_fullsized":"2024-09-21T11:40:54.650273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.403408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":76,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391041+0000","last_change":"2024-09-21T11:40:02.399334+0000","last_active":"2024-09-21T11:40:54.391041+0000","last_peered":"2024-09-21T11:40:54.391041+0000","last_clean":"2024-09-21T11:40:54.391041+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:40:54.391041+0000","last_undegraded":"2024-09-21T11:40:54.391041+0000","last_fullsized":"2024-09-21T11:40:54.391041+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.386531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.392279+0000","last_change":"2024-09-21T11:40:00.310330+0000","last_active":"2024-09-21T11:40:54.392279+0000","last_peered":"2024-09-21T11:40:54.392279+0000","last_clean":"2024-09-21T11:40:54.392279+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:40:54.392279+0000","last_undegraded":"2024-09-21T11:40:54.392279+0000","last_fullsized":"2024-09-21T11:40:54.392279+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.371930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109373+0000","last_change":"2024-09-21T11:40:04.330180+0000","last_active":"2024-09-21T11:40:54.109373+0000","last_peered":"2024-09-21T11:40:54.109373+0000","last_clean":"2024-09-21T11:40:54.109373+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:40:54.109373+0000","last_undegraded":"2024-09-21T11:40:54.109373+0000","last_fullsized":"2024-09-21T11:40:54.109373+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.570842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109775+0000","last_change":"2024-09-21T11:40:06.343742+0000","last_active":"2024-09-21T11:40:54.109775+0000","last_peered":"2024-09-21T11:40:54.109775+0000","last_clean":"2024-09-21T11:40:54.109775+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:40:54.109775+0000","last_undegraded":"2024-09-21T11:40:54.109775+0000","last_fullsized":"2024-09-21T11:40:54.109775+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.217075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109601+0000","last_change":"2024-09-21T11:40:02.312574+0000","last_active":"2024-09-21T11:40:54.109601+0000","last_peered":"2024-09-21T11:40:54.109601+0000","last_clean":"2024-09-21T11:40:54.109601+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:40:54.109601+0000","last_undegraded":"2024-09-21T11:40:54.109601+0000","last_fullsized":"2024-09-21T11:40:54.109601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.497994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111635+0000","last_change":"2024-09-21T11:40:00.312084+0000","last_active":"2024-09-21T11:40:54.111635+0000","last_peered":"2024-09-21T11:40:54.111635+0000","last_clean":"2024-09-21T11:40:54.111635+0000","last_became_active":"2024-09-21T11:40:00.311813+0000","last_became_peered":"2024-09-21T11:40:00.311813+0000","last_unstale":"2024-09-21T11:40:54.111635+0000","last_undegraded":"2024-09-21T11:40:54.111635+0000","last_fullsized":"2024-09-21T11:40:54.111635+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.090287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.390956+0000","last_change":"2024-09-21T11:40:04.324572+0000","last_active":"2024-09-21T11:40:54.390956+0000","last_peered":"2024-09-21T11:40:54.390956+0000","last_clean":"2024-09-21T11:40:54.390956+0000","last_became_active":"2024-09-21T11:40:04.324272+0000","last_became_peered":"2024-09-21T11:40:04.324272+0000","last_unstale":"2024-09-21T11:40:54.390956+0000","last_undegraded":"2024-09-21T11:40:54.390956+0000","last_fullsized":"2024-09-21T11:40:54.390956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.830113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766892+0000","last_change":"2024-09-21T11:40:06.489239+0000","last_active":"2024-09-21T11:40:54.766892+0000","last_peered":"2024-09-21T11:40:54.766892+0000","last_clean":"2024-09-21T11:40:54.766892+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:40:54.766892+0000","last_undegraded":"2024-09-21T11:40:54.766892+0000","last_fullsized":"2024-09-21T11:40:54.766892+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.580118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110122+0000","last_change":"2024-09-21T11:40:02.318985+0000","last_active":"2024-09-21T11:40:54.110122+0000","last_peered":"2024-09-21T11:40:54.110122+0000","last_clean":"2024-09-21T11:40:54.110122+0000","last_became_active":"2024-09-21T11:40:02.318270+0000","last_became_peered":"2024-09-21T11:40:02.318270+0000","last_unstale":"2024-09-21T11:40:54.110122+0000","last_undegraded":"2024-09-21T11:40:54.110122+0000","last_fullsized":"2024-09-21T11:40:54.110122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.457674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.651301+0000","last_change":"2024-09-21T11:40:00.318794+0000","last_active":"2024-09-21T11:40:54.651301+0000","last_peered":"2024-09-21T11:40:54.651301+0000","last_clean":"2024-09-21T11:40:54.651301+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:40:54.651301+0000","last_undegraded":"2024-09-21T11:40:54.651301+0000","last_fullsized":"2024-09-21T11:40:54.651301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.580077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111717+0000","last_change":"2024-09-21T11:40:04.335582+0000","last_active":"2024-09-21T11:40:54.111717+0000","last_peered":"2024-09-21T11:40:54.111717+0000","last_clean":"2024-09-21T11:40:54.111717+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:40:54.111717+0000","last_undegraded":"2024-09-21T11:40:54.111717+0000","last_fullsized":"2024-09-21T11:40:54.111717+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:03.386490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109793+0000","last_change":"2024-09-21T11:40:06.352201+0000","last_active":"2024-09-21T11:40:54.109793+0000","last_peered":"2024-09-21T11:40:54.109793+0000","last_clean":"2024-09-21T11:40:54.109793+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:40:54.109793+0000","last_undegraded":"2024-09-21T11:40:54.109793+0000","last_fullsized":"2024-09-21T11:40:54.109793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.165004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":79,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110980+0000","last_change":"2024-09-21T11:40:02.399117+0000","last_active":"2024-09-21T11:40:54.110980+0000","last_peered":"2024-09-21T11:40:54.110980+0000","last_clean":"2024-09-21T11:40:54.110980+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:40:54.110980+0000","last_undegraded":"2024-09-21T11:40:54.110980+0000","last_fullsized":"2024-09-21T11:40:54.110980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.772060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111609+0000","last_change":"2024-09-21T11:40:00.311519+0000","last_active":"2024-09-21T11:40:54.111609+0000","last_peered":"2024-09-21T11:40:54.111609+0000","last_clean":"2024-09-21T11:40:54.111609+0000","last_became_active":"2024-09-21T11:40:00.311211+0000","last_became_peered":"2024-09-21T11:40:00.311211+0000","last_unstale":"2024-09-21T11:40:54.111609+0000","last_undegraded":"2024-09-21T11:40:54.111609+0000","last_fullsized":"2024-09-21T11:40:54.111609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.982610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.602669+0000","last_change":"2024-09-21T11:40:04.339202+0000","last_active":"2024-09-21T11:40:54.602669+0000","last_peered":"2024-09-21T11:40:54.602669+0000","last_clean":"2024-09-21T11:40:54.602669+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:40:54.602669+0000","last_undegraded":"2024-09-21T11:40:54.602669+0000","last_fullsized":"2024-09-21T11:40:54.602669+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:05.313979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109293+0000","last_change":"2024-09-21T11:40:06.487976+0000","last_active":"2024-09-21T11:40:54.109293+0000","last_peered":"2024-09-21T11:40:54.109293+0000","last_clean":"2024-09-21T11:40:54.109293+0000","last_became_active":"2024-09-21T11:40:06.487644+0000","last_became_peered":"2024-09-21T11:40:06.487644+0000","last_unstale":"2024-09-21T11:40:54.109293+0000","last_undegraded":"2024-09-21T11:40:54.109293+0000","last_fullsized":"2024-09-21T11:40:54.109293+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.318979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":103,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111545+0000","last_change":"2024-09-21T11:40:02.320105+0000","last_active":"2024-09-21T11:40:54.111545+0000","last_peered":"2024-09-21T11:40:54.111545+0000","last_clean":"2024-09-21T11:40:54.111545+0000","last_became_active":"2024-09-21T11:40:02.319770+0000","last_became_peered":"2024-09-21T11:40:02.319770+0000","last_unstale":"2024-09-21T11:40:54.111545+0000","last_undegraded":"2024-09-21T11:40:54.111545+0000","last_fullsized":"2024-09-21T11:40:54.111545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.672594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109996+0000","last_change":"2024-09-21T11:40:00.314186+0000","last_active":"2024-09-21T11:40:54.109996+0000","last_peered":"2024-09-21T11:40:54.109996+0000","last_clean":"2024-09-21T11:40:54.109996+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:40:54.109996+0000","last_undegraded":"2024-09-21T11:40:54.109996+0000","last_fullsized":"2024-09-21T11:40:54.109996+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:14.587338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.650771+0000","last_change":"2024-09-21T11:40:02.311375+0000","last_active":"2024-09-21T11:40:54.650771+0000","last_peered":"2024-09-21T11:40:54.650771+0000","last_clean":"2024-09-21T11:40:54.650771+0000","last_became_active":"2024-09-21T11:39:59.303309+0000","last_became_peered":"2024-09-21T11:39:59.303309+0000","last_unstale":"2024-09-21T11:40:54.650771+0000","last_undegraded":"2024-09-21T11:40:54.650771+0000","last_fullsized":"2024-09-21T11:40:54.650771+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.322560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110807+0000","last_change":"2024-09-21T11:40:04.337767+0000","last_active":"2024-09-21T11:40:54.110807+0000","last_peered":"2024-09-21T11:40:54.110807+0000","last_clean":"2024-09-21T11:40:54.110807+0000","last_became_active":"2024-09-21T11:40:04.337422+0000","last_became_peered":"2024-09-21T11:40:04.337422+0000","last_unstale":"2024-09-21T11:40:54.110807+0000","last_undegraded":"2024-09-21T11:40:54.110807+0000","last_fullsized":"2024-09-21T11:40:54.110807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.195058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.390727+0000","last_change":"2024-09-21T11:40:06.350784+0000","last_active":"2024-09-21T11:40:54.390727+0000","last_peered":"2024-09-21T11:40:54.390727+0000","last_clean":"2024-09-21T11:40:54.390727+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:40:54.390727+0000","last_undegraded":"2024-09-21T11:40:54.390727+0000","last_fullsized":"2024-09-21T11:40:54.390727+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.561650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":70,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111420+0000","last_change":"2024-09-21T11:40:02.324944+0000","last_active":"2024-09-21T11:40:54.111420+0000","last_peered":"2024-09-21T11:40:54.111420+0000","last_clean":"2024-09-21T11:40:54.111420+0000","last_became_active":"2024-09-21T11:40:02.324667+0000","last_became_peered":"2024-09-21T11:40:02.324667+0000","last_unstale":"2024-09-21T11:40:54.111420+0000","last_undegraded":"2024-09-21T11:40:54.111420+0000","last_fullsized":"2024-09-21T11:40:54.111420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.447522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":53,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109668+0000","last_change":"2024-09-21T11:40:00.313632+0000","last_active":"2024-09-21T11:40:54.109668+0000","last_peered":"2024-09-21T11:40:54.109668+0000","last_clean":"2024-09-21T11:40:54.109668+0000","last_became_active":"2024-09-21T11:40:00.313231+0000","last_became_peered":"2024-09-21T11:40:00.313231+0000","last_unstale":"2024-09-21T11:40:54.109668+0000","last_undegraded":"2024-09-21T11:40:54.109668+0000","last_fullsized":"2024-09-21T11:40:54.109668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.689078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.840675+0000","last_change":"2024-09-21T11:40:02.305143+0000","last_active":"2024-09-21T11:40:54.840675+0000","last_peered":"2024-09-21T11:40:54.840675+0000","last_clean":"2024-09-21T11:40:54.840675+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:40:54.840675+0000","last_undegraded":"2024-09-21T11:40:54.840675+0000","last_fullsized":"2024-09-21T11:40:54.840675+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.513240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.841248+0000","last_change":"2024-09-21T11:40:04.332073+0000","last_active":"2024-09-21T11:40:54.841248+0000","last_peered":"2024-09-21T11:40:54.841248+0000","last_clean":"2024-09-21T11:40:54.841248+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:40:54.841248+0000","last_undegraded":"2024-09-21T11:40:54.841248+0000","last_fullsized":"2024-09-21T11:40:54.841248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.887948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766386+0000","last_change":"2024-09-21T11:40:06.487757+0000","last_active":"2024-09-21T11:40:54.766386+0000","last_peered":"2024-09-21T11:40:54.766386+0000","last_clean":"2024-09-21T11:40:54.766386+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:40:54.766386+0000","last_undegraded":"2024-09-21T11:40:54.766386+0000","last_fullsized":"2024-09-21T11:40:54.766386+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.207444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":61,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111535+0000","last_change":"2024-09-21T11:40:02.320851+0000","last_active":"2024-09-21T11:40:54.111535+0000","last_peered":"2024-09-21T11:40:54.111535+0000","last_clean":"2024-09-21T11:40:54.111535+0000","last_became_active":"2024-09-21T11:40:02.320522+0000","last_became_peered":"2024-09-21T11:40:02.320522+0000","last_unstale":"2024-09-21T11:40:54.111535+0000","last_undegraded":"2024-09-21T11:40:54.111535+0000","last_fullsized":"2024-09-21T11:40:54.111535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:30.451592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109525+0000","last_change":"2024-09-21T11:40:00.312222+0000","last_active":"2024-09-21T11:40:54.109525+0000","last_peered":"2024-09-21T11:40:54.109525+0000","last_clean":"2024-09-21T11:40:54.109525+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:40:54.109525+0000","last_undegraded":"2024-09-21T11:40:54.109525+0000","last_fullsized":"2024-09-21T11:40:54.109525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.238210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":108,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766660+0000","last_change":"2024-09-21T11:40:02.398538+0000","last_active":"2024-09-21T11:40:54.766660+0000","last_peered":"2024-09-21T11:40:54.766660+0000","last_clean":"2024-09-21T11:40:54.766660+0000","last_became_active":"2024-09-21T11:39:59.306320+0000","last_became_peered":"2024-09-21T11:39:59.306320+0000","last_unstale":"2024-09-21T11:40:54.766660+0000","last_undegraded":"2024-09-21T11:40:54.766660+0000","last_fullsized":"2024-09-21T11:40:54.766660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:39:58.289765+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:18.630069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.651401+0000","last_change":"2024-09-21T11:40:04.339615+0000","last_active":"2024-09-21T11:40:54.651401+0000","last_peered":"2024-09-21T11:40:54.651401+0000","last_clean":"2024-09-21T11:40:54.651401+0000","last_became_active":"2024-09-21T11:40:04.339331+0000","last_became_peered":"2024-09-21T11:40:04.339331+0000","last_unstale":"2024-09-21T11:40:54.651401+0000","last_undegraded":"2024-09-21T11:40:54.651401+0000","last_fullsized":"2024-09-21T11:40:54.651401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:33.310635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110037+0000","last_change":"2024-09-21T11:40:06.352155+0000","last_active":"2024-09-21T11:40:54.110037+0000","last_peered":"2024-09-21T11:40:54.110037+0000","last_clean":"2024-09-21T11:40:54.110037+0000","last_became_active":"2024-09-21T11:40:06.351777+0000","last_became_peered":"2024-09-21T11:40:06.351777+0000","last_unstale":"2024-09-21T11:40:54.110037+0000","last_undegraded":"2024-09-21T11:40:54.110037+0000","last_fullsized":"2024-09-21T11:40:54.110037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.594495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":77,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.603300+0000","last_change":"2024-09-21T11:40:02.399268+0000","last_active":"2024-09-21T11:40:54.603300+0000","last_peered":"2024-09-21T11:40:54.603300+0000","last_clean":"2024-09-21T11:40:54.603300+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:40:54.603300+0000","last_undegraded":"2024-09-21T11:40:54.603300+0000","last_fullsized":"2024-09-21T11:40:54.603300+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.058592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391471+0000","last_change":"2024-09-21T11:40:00.309464+0000","last_active":"2024-09-21T11:40:54.391471+0000","last_peered":"2024-09-21T11:40:54.391471+0000","last_clean":"2024-09-21T11:40:54.391471+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:40:54.391471+0000","last_undegraded":"2024-09-21T11:40:54.391471+0000","last_fullsized":"2024-09-21T11:40:54.391471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.615608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":92,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766773+0000","last_change":"2024-09-21T11:40:12.500662+0000","last_active":"2024-09-21T11:40:54.766773+0000","last_peered":"2024-09-21T11:40:54.766773+0000","last_clean":"2024-09-21T11:40:54.766773+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:40:54.766773+0000","last_undegraded":"2024-09-21T11:40:54.766773+0000","last_fullsized":"2024-09-21T11:40:54.766773+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"21'72","last_scrub_stamp":"2024-09-21T11:40:12.500606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:40:12.500606+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.061712+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.767111+0000","last_change":"2024-09-21T11:40:04.333889+0000","last_active":"2024-09-21T11:40:54.767111+0000","last_peered":"2024-09-21T11:40:54.767111+0000","last_clean":"2024-09-21T11:40:54.767111+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:40:54.767111+0000","last_undegraded":"2024-09-21T11:40:54.767111+0000","last_fullsized":"2024-09-21T11:40:54.767111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:31.202161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.650356+0000","last_change":"2024-09-21T11:40:06.349857+0000","last_active":"2024-09-21T11:40:54.650356+0000","last_peered":"2024-09-21T11:40:54.650356+0000","last_clean":"2024-09-21T11:40:54.650356+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:40:54.650356+0000","last_undegraded":"2024-09-21T11:40:54.650356+0000","last_fullsized":"2024-09-21T11:40:54.650356+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.149814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110255+0000","last_change":"2024-09-21T11:40:02.319709+0000","last_active":"2024-09-21T11:40:54.110255+0000","last_peered":"2024-09-21T11:40:54.110255+0000","last_clean":"2024-09-21T11:40:54.110255+0000","last_became_active":"2024-09-21T11:40:02.319422+0000","last_became_peered":"2024-09-21T11:40:02.319422+0000","last_unstale":"2024-09-21T11:40:54.110255+0000","last_undegraded":"2024-09-21T11:40:54.110255+0000","last_fullsized":"2024-09-21T11:40:54.110255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.852025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109675+0000","last_change":"2024-09-21T11:40:00.312057+0000","last_active":"2024-09-21T11:40:54.109675+0000","last_peered":"2024-09-21T11:40:54.109675+0000","last_clean":"2024-09-21T11:40:54.109675+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:40:54.109675+0000","last_undegraded":"2024-09-21T11:40:54.109675+0000","last_fullsized":"2024-09-21T11:40:54.109675+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.891862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.651279+0000","last_change":"2024-09-21T11:40:04.329480+0000","last_active":"2024-09-21T11:40:54.651279+0000","last_peered":"2024-09-21T11:40:54.651279+0000","last_clean":"2024-09-21T11:40:54.651279+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:40:54.651279+0000","last_undegraded":"2024-09-21T11:40:54.651279+0000","last_fullsized":"2024-09-21T11:40:54.651279+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.295503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391592+0000","last_change":"2024-09-21T11:40:06.487868+0000","last_active":"2024-09-21T11:40:54.391592+0000","last_peered":"2024-09-21T11:40:54.391592+0000","last_clean":"2024-09-21T11:40:54.391592+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:40:54.391592+0000","last_undegraded":"2024-09-21T11:40:54.391592+0000","last_fullsized":"2024-09-21T11:40:54.391592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.512063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":72,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110252+0000","last_change":"2024-09-21T11:40:02.319361+0000","last_active":"2024-09-21T11:40:54.110252+0000","last_peered":"2024-09-21T11:40:54.110252+0000","last_clean":"2024-09-21T11:40:54.110252+0000","last_became_active":"2024-09-21T11:40:02.318902+0000","last_became_peered":"2024-09-21T11:40:02.318902+0000","last_unstale":"2024-09-21T11:40:54.110252+0000","last_undegraded":"2024-09-21T11:40:54.110252+0000","last_fullsized":"2024-09-21T11:40:54.110252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.422023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.603028+0000","last_change":"2024-09-21T11:40:00.313702+0000","last_active":"2024-09-21T11:40:54.603028+0000","last_peered":"2024-09-21T11:40:54.603028+0000","last_clean":"2024-09-21T11:40:54.603028+0000","last_became_active":"2024-09-21T11:40:00.313432+0000","last_became_peered":"2024-09-21T11:40:00.313432+0000","last_unstale":"2024-09-21T11:40:54.603028+0000","last_undegraded":"2024-09-21T11:40:54.603028+0000","last_fullsized":"2024-09-21T11:40:54.603028+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:21.740308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110796+0000","last_change":"2024-09-21T11:40:04.342081+0000","last_active":"2024-09-21T11:40:54.110796+0000","last_peered":"2024-09-21T11:40:54.110796+0000","last_clean":"2024-09-21T11:40:54.110796+0000","last_became_active":"2024-09-21T11:40:04.341743+0000","last_became_peered":"2024-09-21T11:40:04.341743+0000","last_unstale":"2024-09-21T11:40:54.110796+0000","last_undegraded":"2024-09-21T11:40:54.110796+0000","last_fullsized":"2024-09-21T11:40:54.110796+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.025793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.840322+0000","last_change":"2024-09-21T11:40:06.349130+0000","last_active":"2024-09-21T11:40:54.840322+0000","last_peered":"2024-09-21T11:40:54.840322+0000","last_clean":"2024-09-21T11:40:54.840322+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:40:54.840322+0000","last_undegraded":"2024-09-21T11:40:54.840322+0000","last_fullsized":"2024-09-21T11:40:54.840322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:14.687896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109805+0000","last_change":"2024-09-21T11:40:02.317262+0000","last_active":"2024-09-21T11:40:54.109805+0000","last_peered":"2024-09-21T11:40:54.109805+0000","last_clean":"2024-09-21T11:40:54.109805+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:40:54.109805+0000","last_undegraded":"2024-09-21T11:40:54.109805+0000","last_fullsized":"2024-09-21T11:40:54.109805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.898962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391674+0000","last_change":"2024-09-21T11:40:00.307721+0000","last_active":"2024-09-21T11:40:54.391674+0000","last_peered":"2024-09-21T11:40:54.391674+0000","last_clean":"2024-09-21T11:40:54.391674+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:40:54.391674+0000","last_undegraded":"2024-09-21T11:40:54.391674+0000","last_fullsized":"2024-09-21T11:40:54.391674+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.478046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.841038+0000","last_change":"2024-09-21T11:40:04.332939+0000","last_active":"2024-09-21T11:40:54.841038+0000","last_peered":"2024-09-21T11:40:54.841038+0000","last_clean":"2024-09-21T11:40:54.841038+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:40:54.841038+0000","last_undegraded":"2024-09-21T11:40:54.841038+0000","last_fullsized":"2024-09-21T11:40:54.841038+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.250509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109893+0000","last_change":"2024-09-21T11:40:06.346865+0000","last_active":"2024-09-21T11:40:54.109893+0000","last_peered":"2024-09-21T11:40:54.109893+0000","last_clean":"2024-09-21T11:40:54.109893+0000","last_became_active":"2024-09-21T11:40:06.346602+0000","last_became_peered":"2024-09-21T11:40:06.346602+0000","last_unstale":"2024-09-21T11:40:54.109893+0000","last_undegraded":"2024-09-21T11:40:54.109893+0000","last_fullsized":"2024-09-21T11:40:54.109893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.724328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110872+0000","last_change":"2024-09-21T11:40:02.319787+0000","last_active":"2024-09-21T11:40:54.110872+0000","last_peered":"2024-09-21T11:40:54.110872+0000","last_clean":"2024-09-21T11:40:54.110872+0000","last_became_active":"2024-09-21T11:40:02.319502+0000","last_became_peered":"2024-09-21T11:40:02.319502+0000","last_unstale":"2024-09-21T11:40:54.110872+0000","last_undegraded":"2024-09-21T11:40:54.110872+0000","last_fullsized":"2024-09-21T11:40:54.110872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:02.504218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.651081+0000","last_change":"2024-09-21T11:40:00.318087+0000","last_active":"2024-09-21T11:40:54.651081+0000","last_peered":"2024-09-21T11:40:54.651081+0000","last_clean":"2024-09-21T11:40:54.651081+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:40:54.651081+0000","last_undegraded":"2024-09-21T11:40:54.651081+0000","last_fullsized":"2024-09-21T11:40:54.651081+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.865238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.840826+0000","last_change":"2024-09-21T11:40:04.330158+0000","last_active":"2024-09-21T11:40:54.840826+0000","last_peered":"2024-09-21T11:40:54.840826+0000","last_clean":"2024-09-21T11:40:54.840826+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:40:54.840826+0000","last_undegraded":"2024-09-21T11:40:54.840826+0000","last_fullsized":"2024-09-21T11:40:54.840826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.222022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109516+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:40:54.109516+0000","last_peered":"2024-09-21T11:40:54.109516+0000","last_clean":"2024-09-21T11:40:54.109516+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:40:54.109516+0000","last_undegraded":"2024-09-21T11:40:54.109516+0000","last_fullsized":"2024-09-21T11:40:54.109516+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:32.858006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":74,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.602865+0000","last_change":"2024-09-21T11:40:02.399340+0000","last_active":"2024-09-21T11:40:54.602865+0000","last_peered":"2024-09-21T11:40:54.602865+0000","last_clean":"2024-09-21T11:40:54.602865+0000","last_became_active":"2024-09-21T11:40:02.398876+0000","last_became_peered":"2024-09-21T11:40:02.398876+0000","last_unstale":"2024-09-21T11:40:54.602865+0000","last_undegraded":"2024-09-21T11:40:54.602865+0000","last_fullsized":"2024-09-21T11:40:54.602865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.043119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.767334+0000","last_change":"2024-09-21T11:40:00.314507+0000","last_active":"2024-09-21T11:40:54.767334+0000","last_peered":"2024-09-21T11:40:54.767334+0000","last_clean":"2024-09-21T11:40:54.767334+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:40:54.767334+0000","last_undegraded":"2024-09-21T11:40:54.767334+0000","last_fullsized":"2024-09-21T11:40:54.767334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.755846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.840934+0000","last_change":"2024-09-21T11:40:04.341835+0000","last_active":"2024-09-21T11:40:54.840934+0000","last_peered":"2024-09-21T11:40:54.840934+0000","last_clean":"2024-09-21T11:40:54.840934+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:40:54.840934+0000","last_undegraded":"2024-09-21T11:40:54.840934+0000","last_fullsized":"2024-09-21T11:40:54.840934+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:21.943269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766736+0000","last_change":"2024-09-21T11:40:06.348279+0000","last_active":"2024-09-21T11:40:54.766736+0000","last_peered":"2024-09-21T11:40:54.766736+0000","last_clean":"2024-09-21T11:40:54.766736+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:40:54.766736+0000","last_undegraded":"2024-09-21T11:40:54.766736+0000","last_fullsized":"2024-09-21T11:40:54.766736+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:29.108944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":67,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109504+0000","last_change":"2024-09-21T11:40:02.320045+0000","last_active":"2024-09-21T11:40:54.109504+0000","last_peered":"2024-09-21T11:40:54.109504+0000","last_clean":"2024-09-21T11:40:54.109504+0000","last_became_active":"2024-09-21T11:40:02.319752+0000","last_became_peered":"2024-09-21T11:40:02.319752+0000","last_unstale":"2024-09-21T11:40:54.109504+0000","last_undegraded":"2024-09-21T11:40:54.109504+0000","last_fullsized":"2024-09-21T11:40:54.109504+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:26.042672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.767476+0000","last_change":"2024-09-21T11:40:00.311267+0000","last_active":"2024-09-21T11:40:54.767476+0000","last_peered":"2024-09-21T11:40:54.767476+0000","last_clean":"2024-09-21T11:40:54.767476+0000","last_became_active":"2024-09-21T11:40:00.310457+0000","last_became_peered":"2024-09-21T11:40:00.310457+0000","last_unstale":"2024-09-21T11:40:54.767476+0000","last_undegraded":"2024-09-21T11:40:54.767476+0000","last_fullsized":"2024-09-21T11:40:54.767476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:17.362207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.840448+0000","last_change":"2024-09-21T11:40:04.334536+0000","last_active":"2024-09-21T11:40:54.840448+0000","last_peered":"2024-09-21T11:40:54.840448+0000","last_clean":"2024-09-21T11:40:54.840448+0000","last_became_active":"2024-09-21T11:40:04.334205+0000","last_became_peered":"2024-09-21T11:40:04.334205+0000","last_unstale":"2024-09-21T11:40:54.840448+0000","last_undegraded":"2024-09-21T11:40:54.840448+0000","last_fullsized":"2024-09-21T11:40:54.840448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.125493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.390637+0000","last_change":"2024-09-21T11:40:06.352508+0000","last_active":"2024-09-21T11:40:54.390637+0000","last_peered":"2024-09-21T11:40:54.390637+0000","last_clean":"2024-09-21T11:40:54.390637+0000","last_became_active":"2024-09-21T11:40:06.352210+0000","last_became_peered":"2024-09-21T11:40:06.352210+0000","last_unstale":"2024-09-21T11:40:54.390637+0000","last_undegraded":"2024-09-21T11:40:54.390637+0000","last_fullsized":"2024-09-21T11:40:54.390637+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.957978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":68,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.650374+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:40:54.650374+0000","last_peered":"2024-09-21T11:40:54.650374+0000","last_clean":"2024-09-21T11:40:54.650374+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:40:54.650374+0000","last_undegraded":"2024-09-21T11:40:54.650374+0000","last_fullsized":"2024-09-21T11:40:54.650374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.513015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.767521+0000","last_change":"2024-09-21T11:40:00.311525+0000","last_active":"2024-09-21T11:40:54.767521+0000","last_peered":"2024-09-21T11:40:54.767521+0000","last_clean":"2024-09-21T11:40:54.767521+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:40:54.767521+0000","last_undegraded":"2024-09-21T11:40:54.767521+0000","last_fullsized":"2024-09-21T11:40:54.767521+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.568697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766799+0000","last_change":"2024-09-21T11:40:04.331990+0000","last_active":"2024-09-21T11:40:54.766799+0000","last_peered":"2024-09-21T11:40:54.766799+0000","last_clean":"2024-09-21T11:40:54.766799+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:40:54.766799+0000","last_undegraded":"2024-09-21T11:40:54.766799+0000","last_fullsized":"2024-09-21T11:40:54.766799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.347865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.650102+0000","last_change":"2024-09-21T11:40:06.487630+0000","last_active":"2024-09-21T11:40:54.650102+0000","last_peered":"2024-09-21T11:40:54.650102+0000","last_clean":"2024-09-21T11:40:54.650102+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:40:54.650102+0000","last_undegraded":"2024-09-21T11:40:54.650102+0000","last_fullsized":"2024-09-21T11:40:54.650102+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.388794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.602752+0000","last_change":"2024-09-21T11:40:00.310655+0000","last_active":"2024-09-21T11:40:54.602752+0000","last_peered":"2024-09-21T11:40:54.602752+0000","last_clean":"2024-09-21T11:40:54.602752+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:40:54.602752+0000","last_undegraded":"2024-09-21T11:40:54.602752+0000","last_fullsized":"2024-09-21T11:40:54.602752+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.882425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":53,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.392203+0000","last_change":"2024-09-21T11:40:02.399093+0000","last_active":"2024-09-21T11:40:54.392203+0000","last_peered":"2024-09-21T11:40:54.392203+0000","last_clean":"2024-09-21T11:40:54.392203+0000","last_became_active":"2024-09-21T11:40:02.398803+0000","last_became_peered":"2024-09-21T11:40:02.398803+0000","last_unstale":"2024-09-21T11:40:54.392203+0000","last_undegraded":"2024-09-21T11:40:54.392203+0000","last_fullsized":"2024-09-21T11:40:54.392203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.504556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.651241+0000","last_change":"2024-09-21T11:40:04.328297+0000","last_active":"2024-09-21T11:40:54.651241+0000","last_peered":"2024-09-21T11:40:54.651241+0000","last_clean":"2024-09-21T11:40:54.651241+0000","last_became_active":"2024-09-21T11:40:04.327422+0000","last_became_peered":"2024-09-21T11:40:04.327422+0000","last_unstale":"2024-09-21T11:40:54.651241+0000","last_undegraded":"2024-09-21T11:40:54.651241+0000","last_fullsized":"2024-09-21T11:40:54.651241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:08.663596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766392+0000","last_change":"2024-09-21T11:40:06.488399+0000","last_active":"2024-09-21T11:40:54.766392+0000","last_peered":"2024-09-21T11:40:54.766392+0000","last_clean":"2024-09-21T11:40:54.766392+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:40:54.766392+0000","last_undegraded":"2024-09-21T11:40:54.766392+0000","last_fullsized":"2024-09-21T11:40:54.766392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:23.617144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111587+0000","last_change":"2024-09-21T11:40:02.399564+0000","last_active":"2024-09-21T11:40:54.111587+0000","last_peered":"2024-09-21T11:40:54.111587+0000","last_clean":"2024-09-21T11:40:54.111587+0000","last_became_active":"2024-09-21T11:40:02.399267+0000","last_became_peered":"2024-09-21T11:40:02.399267+0000","last_unstale":"2024-09-21T11:40:54.111587+0000","last_undegraded":"2024-09-21T11:40:54.111587+0000","last_fullsized":"2024-09-21T11:40:54.111587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:01.891469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766937+0000","last_change":"2024-09-21T11:40:00.308959+0000","last_active":"2024-09-21T11:40:54.766937+0000","last_peered":"2024-09-21T11:40:54.766937+0000","last_clean":"2024-09-21T11:40:54.766937+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:40:54.766937+0000","last_undegraded":"2024-09-21T11:40:54.766937+0000","last_fullsized":"2024-09-21T11:40:54.766937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:12.467036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.392111+0000","last_change":"2024-09-21T11:40:04.326964+0000","last_active":"2024-09-21T11:40:54.392111+0000","last_peered":"2024-09-21T11:40:54.392111+0000","last_clean":"2024-09-21T11:40:54.392111+0000","last_became_active":"2024-09-21T11:40:04.326625+0000","last_became_peered":"2024-09-21T11:40:04.326625+0000","last_unstale":"2024-09-21T11:40:54.392111+0000","last_undegraded":"2024-09-21T11:40:54.392111+0000","last_fullsized":"2024-09-21T11:40:54.392111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.322427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.840299+0000","last_change":"2024-09-21T11:40:06.343913+0000","last_active":"2024-09-21T11:40:54.840299+0000","last_peered":"2024-09-21T11:40:54.840299+0000","last_clean":"2024-09-21T11:40:54.840299+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:40:54.840299+0000","last_undegraded":"2024-09-21T11:40:54.840299+0000","last_fullsized":"2024-09-21T11:40:54.840299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:20.682920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.651168+0000","last_change":"2024-09-21T11:40:02.400725+0000","last_active":"2024-09-21T11:40:54.651168+0000","last_peered":"2024-09-21T11:40:54.651168+0000","last_clean":"2024-09-21T11:40:54.651168+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:40:54.651168+0000","last_undegraded":"2024-09-21T11:40:54.651168+0000","last_fullsized":"2024-09-21T11:40:54.651168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:22.300079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110384+0000","last_change":"2024-09-21T11:40:00.311525+0000","last_active":"2024-09-21T11:40:54.110384+0000","last_peered":"2024-09-21T11:40:54.110384+0000","last_clean":"2024-09-21T11:40:54.110384+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:40:54.110384+0000","last_undegraded":"2024-09-21T11:40:54.110384+0000","last_fullsized":"2024-09-21T11:40:54.110384+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:06.269082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391792+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:40:54.391792+0000","last_peered":"2024-09-21T11:40:54.391792+0000","last_clean":"2024-09-21T11:40:54.391792+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:40:54.391792+0000","last_undegraded":"2024-09-21T11:40:54.391792+0000","last_fullsized":"2024-09-21T11:40:54.391792+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.404995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110358+0000","last_change":"2024-09-21T11:40:06.346686+0000","last_active":"2024-09-21T11:40:54.110358+0000","last_peered":"2024-09-21T11:40:54.110358+0000","last_clean":"2024-09-21T11:40:54.110358+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:40:54.110358+0000","last_undegraded":"2024-09-21T11:40:54.110358+0000","last_fullsized":"2024-09-21T11:40:54.110358+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.014386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391932+0000","last_change":"2024-09-21T11:40:02.399404+0000","last_active":"2024-09-21T11:40:54.391932+0000","last_peered":"2024-09-21T11:40:54.391932+0000","last_clean":"2024-09-21T11:40:54.391932+0000","last_became_active":"2024-09-21T11:40:02.398771+0000","last_became_peered":"2024-09-21T11:40:02.398771+0000","last_unstale":"2024-09-21T11:40:54.391932+0000","last_undegraded":"2024-09-21T11:40:54.391932+0000","last_fullsized":"2024-09-21T11:40:54.391932+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:11.091232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.767200+0000","last_change":"2024-09-21T11:40:00.314627+0000","last_active":"2024-09-21T11:40:54.767200+0000","last_peered":"2024-09-21T11:40:54.767200+0000","last_clean":"2024-09-21T11:40:54.767200+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:40:54.767200+0000","last_undegraded":"2024-09-21T11:40:54.767200+0000","last_fullsized":"2024-09-21T11:40:54.767200+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.535972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.651104+0000","last_change":"2024-09-21T11:40:04.340164+0000","last_active":"2024-09-21T11:40:54.651104+0000","last_peered":"2024-09-21T11:40:54.651104+0000","last_clean":"2024-09-21T11:40:54.651104+0000","last_became_active":"2024-09-21T11:40:04.339253+0000","last_became_peered":"2024-09-21T11:40:04.339253+0000","last_unstale":"2024-09-21T11:40:54.651104+0000","last_undegraded":"2024-09-21T11:40:54.651104+0000","last_fullsized":"2024-09-21T11:40:54.651104+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:30.031018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110987+0000","last_change":"2024-09-21T11:40:06.351973+0000","last_active":"2024-09-21T11:40:54.110987+0000","last_peered":"2024-09-21T11:40:54.110987+0000","last_clean":"2024-09-21T11:40:54.110987+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:40:54.110987+0000","last_undegraded":"2024-09-21T11:40:54.110987+0000","last_fullsized":"2024-09-21T11:40:54.110987+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.555091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109697+0000","last_change":"2024-09-21T11:40:02.318645+0000","last_active":"2024-09-21T11:40:54.109697+0000","last_peered":"2024-09-21T11:40:54.109697+0000","last_clean":"2024-09-21T11:40:54.109697+0000","last_became_active":"2024-09-21T11:40:02.318352+0000","last_became_peered":"2024-09-21T11:40:02.318352+0000","last_unstale":"2024-09-21T11:40:54.109697+0000","last_undegraded":"2024-09-21T11:40:54.109697+0000","last_fullsized":"2024-09-21T11:40:54.109697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:27.071118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110097+0000","last_change":"2024-09-21T11:40:00.317768+0000","last_active":"2024-09-21T11:40:54.110097+0000","last_peered":"2024-09-21T11:40:54.110097+0000","last_clean":"2024-09-21T11:40:54.110097+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:40:54.110097+0000","last_undegraded":"2024-09-21T11:40:54.110097+0000","last_fullsized":"2024-09-21T11:40:54.110097+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.919244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109887+0000","last_change":"2024-09-21T11:40:04.328561+0000","last_active":"2024-09-21T11:40:54.109887+0000","last_peered":"2024-09-21T11:40:54.109887+0000","last_clean":"2024-09-21T11:40:54.109887+0000","last_became_active":"2024-09-21T11:40:04.328216+0000","last_became_peered":"2024-09-21T11:40:04.328216+0000","last_unstale":"2024-09-21T11:40:54.109887+0000","last_undegraded":"2024-09-21T11:40:54.109887+0000","last_fullsized":"2024-09-21T11:40:54.109887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:10.146311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.390625+0000","last_change":"2024-09-21T11:40:06.353577+0000","last_active":"2024-09-21T11:40:54.390625+0000","last_peered":"2024-09-21T11:40:54.390625+0000","last_clean":"2024-09-21T11:40:54.390625+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:40:54.390625+0000","last_undegraded":"2024-09-21T11:40:54.390625+0000","last_fullsized":"2024-09-21T11:40:54.390625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:25.269429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.110893+0000","last_change":"2024-09-21T11:40:02.319305+0000","last_active":"2024-09-21T11:40:54.110893+0000","last_peered":"2024-09-21T11:40:54.110893+0000","last_clean":"2024-09-21T11:40:54.110893+0000","last_became_active":"2024-09-21T11:40:02.318699+0000","last_became_peered":"2024-09-21T11:40:02.318699+0000","last_unstale":"2024-09-21T11:40:54.110893+0000","last_undegraded":"2024-09-21T11:40:54.110893+0000","last_fullsized":"2024-09-21T11:40:54.110893+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:13.659293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.767268+0000","last_change":"2024-09-21T11:40:00.309690+0000","last_active":"2024-09-21T11:40:54.767268+0000","last_peered":"2024-09-21T11:40:54.767268+0000","last_clean":"2024-09-21T11:40:54.767268+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:40:54.767268+0000","last_undegraded":"2024-09-21T11:40:54.767268+0000","last_fullsized":"2024-09-21T11:40:54.767268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.952393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391516+0000","last_change":"2024-09-21T11:40:04.336820+0000","last_active":"2024-09-21T11:40:54.391516+0000","last_peered":"2024-09-21T11:40:54.391516+0000","last_clean":"2024-09-21T11:40:54.391516+0000","last_became_active":"2024-09-21T11:40:04.336469+0000","last_became_peered":"2024-09-21T11:40:04.336469+0000","last_unstale":"2024-09-21T11:40:54.391516+0000","last_undegraded":"2024-09-21T11:40:54.391516+0000","last_fullsized":"2024-09-21T11:40:54.391516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:09.369123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109535+0000","last_change":"2024-09-21T11:40:06.346455+0000","last_active":"2024-09-21T11:40:54.109535+0000","last_peered":"2024-09-21T11:40:54.109535+0000","last_clean":"2024-09-21T11:40:54.109535+0000","last_became_active":"2024-09-21T11:40:06.346090+0000","last_became_peered":"2024-09-21T11:40:06.346090+0000","last_unstale":"2024-09-21T11:40:54.109535+0000","last_undegraded":"2024-09-21T11:40:54.109535+0000","last_fullsized":"2024-09-21T11:40:54.109535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:16.873349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391215+0000","last_change":"2024-09-21T11:40:02.399706+0000","last_active":"2024-09-21T11:40:54.391215+0000","last_peered":"2024-09-21T11:40:54.391215+0000","last_clean":"2024-09-21T11:40:54.391215+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:40:54.391215+0000","last_undegraded":"2024-09-21T11:40:54.391215+0000","last_fullsized":"2024-09-21T11:40:54.391215+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.937696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.651258+0000","last_change":"2024-09-21T11:40:00.317057+0000","last_active":"2024-09-21T11:40:54.651258+0000","last_peered":"2024-09-21T11:40:54.651258+0000","last_clean":"2024-09-21T11:40:54.651258+0000","last_became_active":"2024-09-21T11:40:00.316756+0000","last_became_peered":"2024-09-21T11:40:00.316756+0000","last_unstale":"2024-09-21T11:40:54.651258+0000","last_undegraded":"2024-09-21T11:40:54.651258+0000","last_fullsized":"2024-09-21T11:40:54.651258+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.129760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766528+0000","last_change":"2024-09-21T11:40:04.331954+0000","last_active":"2024-09-21T11:40:54.766528+0000","last_peered":"2024-09-21T11:40:54.766528+0000","last_clean":"2024-09-21T11:40:54.766528+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:40:54.766528+0000","last_undegraded":"2024-09-21T11:40:54.766528+0000","last_fullsized":"2024-09-21T11:40:54.766528+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.778314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391660+0000","last_change":"2024-09-21T11:40:06.488252+0000","last_active":"2024-09-21T11:40:54.391660+0000","last_peered":"2024-09-21T11:40:54.391660+0000","last_clean":"2024-09-21T11:40:54.391660+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:40:54.391660+0000","last_undegraded":"2024-09-21T11:40:54.391660+0000","last_fullsized":"2024-09-21T11:40:54.391660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:28.158855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.390926+0000","last_change":"2024-09-21T11:40:02.319060+0000","last_active":"2024-09-21T11:40:54.390926+0000","last_peered":"2024-09-21T11:40:54.390926+0000","last_clean":"2024-09-21T11:40:54.390926+0000","last_became_active":"2024-09-21T11:40:02.318727+0000","last_became_peered":"2024-09-21T11:40:02.318727+0000","last_unstale":"2024-09-21T11:40:54.390926+0000","last_undegraded":"2024-09-21T11:40:54.390926+0000","last_fullsized":"2024-09-21T11:40:54.390926+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:07.877093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111327+0000","last_change":"2024-09-21T11:40:00.314387+0000","last_active":"2024-09-21T11:40:54.111327+0000","last_peered":"2024-09-21T11:40:54.111327+0000","last_clean":"2024-09-21T11:40:54.111327+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:40:54.111327+0000","last_undegraded":"2024-09-21T11:40:54.111327+0000","last_fullsized":"2024-09-21T11:40:54.111327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:00.574148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.602619+0000","last_change":"2024-09-21T11:40:04.332492+0000","last_active":"2024-09-21T11:40:54.602619+0000","last_peered":"2024-09-21T11:40:54.602619+0000","last_clean":"2024-09-21T11:40:54.602619+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:40:54.602619+0000","last_undegraded":"2024-09-21T11:40:54.602619+0000","last_fullsized":"2024-09-21T11:40:54.602619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.592217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.766353+0000","last_change":"2024-09-21T11:40:06.342962+0000","last_active":"2024-09-21T11:40:54.766353+0000","last_peered":"2024-09-21T11:40:54.766353+0000","last_clean":"2024-09-21T11:40:54.766353+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:40:54.766353+0000","last_undegraded":"2024-09-21T11:40:54.766353+0000","last_fullsized":"2024-09-21T11:40:54.766353+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.430215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.109882+0000","last_change":"2024-09-21T11:40:06.349089+0000","last_active":"2024-09-21T11:40:54.109882+0000","last_peered":"2024-09-21T11:40:54.109882+0000","last_clean":"2024-09-21T11:40:54.109882+0000","last_became_active":"2024-09-21T11:40:06.348796+0000","last_became_peered":"2024-09-21T11:40:06.348796+0000","last_unstale":"2024-09-21T11:40:54.109882+0000","last_undegraded":"2024-09-21T11:40:54.109882+0000","last_fullsized":"2024-09-21T11:40:54.109882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.850881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.391127+0000","last_change":"2024-09-21T11:40:00.309865+0000","last_active":"2024-09-21T11:40:54.391127+0000","last_peered":"2024-09-21T11:40:54.391127+0000","last_clean":"2024-09-21T11:40:54.391127+0000","last_became_active":"2024-09-21T11:40:00.309598+0000","last_became_peered":"2024-09-21T11:40:00.309598+0000","last_unstale":"2024-09-21T11:40:54.391127+0000","last_undegraded":"2024-09-21T11:40:54.391127+0000","last_fullsized":"2024-09-21T11:40:54.391127+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:24.604856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.603364+0000","last_change":"2024-09-21T11:40:02.318498+0000","last_active":"2024-09-21T11:40:54.603364+0000","last_peered":"2024-09-21T11:40:54.603364+0000","last_clean":"2024-09-21T11:40:54.603364+0000","last_became_active":"2024-09-21T11:40:02.318160+0000","last_became_peered":"2024-09-21T11:40:02.318160+0000","last_unstale":"2024-09-21T11:40:54.603364+0000","last_undegraded":"2024-09-21T11:40:54.603364+0000","last_fullsized":"2024-09-21T11:40:54.603364+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:15.280806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T11:40:54.111437+0000","last_change":"2024-09-21T11:40:04.340406+0000","last_active":"2024-09-21T11:40:54.111437+0000","last_peered":"2024-09-21T11:40:54.111437+0000","last_clean":"2024-09-21T11:40:54.111437+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:40:54.111437+0000","last_undegraded":"2024-09-21T11:40:54.111437+0000","last_fullsized":"2024-09-21T11:40:54.111437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:41:19.851842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":45,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234000,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28364,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716788,"statfs":{"total":95995035648,"available":95965990912,"internally_reserved":0,"allocated":1368064,"data_stored":898615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462933,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32524,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712628,"statfs":{"total":95995035648,"available":95961731072,"internally_reserved":0,"allocated":1368064,"data_stored":905381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724570,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713208,"statfs":{"total":95995035648,"available":95962324992,"internally_reserved":0,"allocated":774144,"data_stored":313956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920800,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31968,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713184,"statfs":{"total":95995035648,"available":95962300416,"internally_reserved":0,"allocated":798720,"data_stored":315182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117030,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":802816,"data_stored":314475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27776,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717376,"statfs":{"total":95995035648,"available":95966593024,"internally_reserved":0,"allocated":765952,"data_stored":314168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":315818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32548,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712604,"statfs":{"total":95995035648,"available":95961706496,"internally_reserved":0,"allocated":1392640,"data_stored":905752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:40:58.656 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T11:40:58.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump 2024-09-21T11:40:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:40:59.344 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:40:59.065Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00097317s 2024-09-21T11:40:59.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[29710]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:40:59.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[29710]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:59.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T11:40:59.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[29710]: Deploying daemon grafana.a on smithi175 2024-09-21T11:40:59.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:40:59.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:40:59.345 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[36454]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:40:59.345 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[36454]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:59.345 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T11:40:59.345 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[36454]: Deploying daemon grafana.a on smithi175 2024-09-21T11:40:59.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:40:59.346 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:40:59 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:40:59.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:59 smithi175 ceph-mon[32613]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:40:59.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:59 smithi175 ceph-mon[32613]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:40:59.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:59 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-21T11:40:59.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:59 smithi175 ceph-mon[32613]: Deploying daemon grafana.a on smithi175 2024-09-21T11:40:59.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:59 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:40:59.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:40:59 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:40:59.543 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 78 2024-09-21T11:40:59.543 INFO:teuthology.orchestra.run.smithi043.stdout:fsid 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T11:40:59.543 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-09-21T11:34:07.249195+0000 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:modified 2024-09-21T11:40:57.092145+0000 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:crush_version 18 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:full_ratio 0.95 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:backfillfull_ratio 0.9 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:nearfull_ratio 0.85 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:require_min_compat_client luminous 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:min_compat_client jewel 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:require_osd_release squid 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:stretch_mode_enabled false 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 7.89 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:pool 2 'datapool' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 61 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.65 2024-09-21T11:40:59.544 INFO:teuthology.orchestra.run.smithi043.stdout:pool 3 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 60 flags hashpspool stripe_width 0 application rgw read_balance_score 1.50 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 62 flags hashpspool stripe_width 0 application rgw read_balance_score 2.25 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 64 flags hashpspool stripe_width 0 application rgw read_balance_score 1.25 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 66 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw read_balance_score 1.75 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:max_osd 8 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 up in weight 1 up_from 8 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.43:6801/817279193 v2:172.21.15.43:6802/817279193 exists,up 5229029d-4fd6-46b6-8325-a3e004df09b4 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1 up in weight 1 up_from 13 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.43:6805/263756171 v2:172.21.15.43:6806/263756171 exists,up a2a7b58b-ba99-4383-a6d3-3b19388b2607 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2 up in weight 1 up_from 18 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.43:6809/687577863 v2:172.21.15.43:6810/687577863 exists,up c24a60b5-5713-492b-82ee-576ea7a743ce 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3 up in weight 1 up_from 27 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.43:6813/129493158 v2:172.21.15.43:6814/129493158 exists,up c5d43cf8-1c72-4d7d-ae41-e8136b47177d 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 up in weight 1 up_from 33 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.175:6800/1323992640 v2:172.21.15.175:6801/1323992640 exists,up 79ee85c5-afdc-4d68-a253-89b7b0eb652c 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5 up in weight 1 up_from 39 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.175:6804/1163208458 v2:172.21.15.175:6805/1163208458 exists,up 4d647321-71a1-412b-bbf5-921441923a65 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 up in weight 1 up_from 47 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.175:6808/932395625 v2:172.21.15.175:6809/932395625 exists,up 3c33bea7-94b5-46c1-9905-81b4612a2354 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7 up in weight 1 up_from 54 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.175:6812/3676713275 v2:172.21.15.175:6813/3676713275 exists,up 41bc9817-d9d1-4c11-94ff-7de332af6d22 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/2419845767 expires 2024-09-22T11:40:29.670859+0000 2024-09-21T11:40:59.545 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/3249046048 expires 2024-09-22T11:34:37.015137+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/2518306973 expires 2024-09-22T11:34:37.015137+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/3115176321 expires 2024-09-22T11:35:03.554546+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/2583179759 expires 2024-09-22T11:34:37.015137+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/4225358170 expires 2024-09-22T11:35:03.554546+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/347827280 expires 2024-09-22T11:40:29.670859+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/1513342881 expires 2024-09-22T11:35:03.554546+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/1113684867 expires 2024-09-22T11:40:29.670859+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/3345448895 expires 2024-09-22T11:35:03.554546+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/2454875216 expires 2024-09-22T11:40:29.670859+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/911630798 expires 2024-09-22T11:34:37.015137+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:0/983028036 expires 2024-09-22T11:40:29.670859+0000 2024-09-21T11:40:59.546 INFO:teuthology.orchestra.run.smithi043.stdout:blocklist 172.21.15.43:6800/4145025121 expires 2024-09-22T11:40:29.670859+0000 2024-09-21T11:41:00.324 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2974989431' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T11:41:00.324 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:00 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:00.325 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:00 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:00.325 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2974989431' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T11:41:00.325 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:00 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:00.325 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:00 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:00.360 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.43:6801/817279193 v2:172.21.15.43:6802/817279193 exists,up 5229029d-4fd6-46b6-8325-a3e004df09b4', 'osd.1 up in weight 1 up_from 13 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.43:6805/263756171 v2:172.21.15.43:6806/263756171 exists,up a2a7b58b-ba99-4383-a6d3-3b19388b2607', 'osd.2 up in weight 1 up_from 18 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.43:6809/687577863 v2:172.21.15.43:6810/687577863 exists,up c24a60b5-5713-492b-82ee-576ea7a743ce', 'osd.3 up in weight 1 up_from 27 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.43:6813/129493158 v2:172.21.15.43:6814/129493158 exists,up c5d43cf8-1c72-4d7d-ae41-e8136b47177d', 'osd.4 up in weight 1 up_from 33 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.175:6800/1323992640 v2:172.21.15.175:6801/1323992640 exists,up 79ee85c5-afdc-4d68-a253-89b7b0eb652c', 'osd.5 up in weight 1 up_from 39 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.175:6804/1163208458 v2:172.21.15.175:6805/1163208458 exists,up 4d647321-71a1-412b-bbf5-921441923a65', 'osd.6 up in weight 1 up_from 47 up_thru 62 down_at 0 last_clean_interval [0,0) v2:172.21.15.175:6808/932395625 v2:172.21.15.175:6809/932395625 exists,up 3c33bea7-94b5-46c1-9905-81b4612a2354', 'osd.7 up in weight 1 up_from 54 up_thru 64 down_at 0 last_clean_interval [0,0) v2:172.21.15.175:6812/3676713275 v2:172.21.15.175:6813/3676713275 exists,up 41bc9817-d9d1-4c11-94ff-7de332af6d22'] 2024-09-21T11:41:00.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon mon.a config show 2024-09-21T11:41:00.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2974989431' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T11:41:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:00 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:00 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:01.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:41:00 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:41:01.082 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "name": "mon.a", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster": "ceph", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "admin_socket_mode": "", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_client_required": "cephx, none", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_cluster_required": "cephx", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_debug": "true", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_service_required": "cephx", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-21T11:41:01.083 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_supported": "", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_aio": "true", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_aio_poll_ms": "250", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_aio_reap_max": "16", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_async_discard_threads": "0", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_block_size": "4096", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_debug_aio": "false", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_debug_inflight_ios": "false", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_enable_discard": "false", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_flock_retry": "3", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-21T11:41:01.084 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_inject_crash": "0", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_ioring": "false", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_ioring_hipri": "false", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bdev_type": "", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_alloc_size": "1048576", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_allocator": "hybrid", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_buffered_io": "true", 2024-09-21T11:41:01.085 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_check_for_zeros": "false", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_compact_log_sync": "false", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_max_log_runway": "4194304", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_max_prefetch": "1048576", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_min_flush_size": "524288", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_min_log_runway": "1048576", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_replay_recovery": "false", 2024-09-21T11:41:01.086 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_sync_write": "false", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_allocation_from_file": "true", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_allocator": "hybrid", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_async_db_compaction": "true", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-21T11:41:01.087 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_bdev_label_multi": "true", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_block_create": "true", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_block_db_create": "false", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_block_db_path": "", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_block_db_size": "0", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_block_path": "", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_block_preallocate_file": "false", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_block_size": "107374182400", 2024-09-21T11:41:01.088 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_block_wal_create": "false", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_block_wal_path": "", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_block_wal_size": "100663296", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_bluefs": "true", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_autotune": "true", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-21T11:41:01.089 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_size": "0", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_type": "2q", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_clone_cow": "true", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-21T11:41:01.090 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_compression_mode": "none", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_csum_type": "crc32c", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_freelist": "false", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-21T11:41:01.091 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_small_allocations": "0", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_default_buffered_read": "true", 2024-09-21T11:41:01.092 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_default_buffered_write": "false", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fail_eio": "false", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-21T11:41:01.093 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_on_mount": "false", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_on_umount": "false", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-21T11:41:01.094 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_ignore_data_csum": "false", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_log_op_age": "5.000000", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_max_alloc_size": "0", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_max_blob_size": "0", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_max_deferred_txc": "32", 2024-09-21T11:41:01.095 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_min_alloc_size": "0", 2024-09-21T11:41:01.096 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-21T11:41:01.096 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-21T11:41:01.096 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_nid_prealloc": "1024", 2024-09-21T11:41:01.096 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-21T11:41:01.096 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-21T11:41:01.096 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-21T11:41:01.096 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_retry_disk_reads": "3", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_rocksdb_cf": "true", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-21T11:41:01.097 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_spdk_mem": "512", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_tracing": "false", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-21T11:41:01.098 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_write_v2": "false", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_write_v2_random": "false", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_zero_block_detection": "false", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-21T11:41:01.099 INFO:teuthology.orchestra.run.smithi043.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephx_cluster_require_signatures": "false", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephx_cluster_require_version": "2", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephx_require_signatures": "false", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephx_require_version": "2", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephx_service_require_signatures": "false", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephx_service_require_version": "2", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "cephx_sign_messages": "true", 2024-09-21T11:41:01.100 INFO:teuthology.orchestra.run.smithi043.stdout: "chdir": "", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_acl_type": "", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_asio_thread_count": "2", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_cache_mid": "0.750000", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_cache_size": "16384", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps_release_delay": "5", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_check_pool_perm": "true", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_debug_force_sync_read": "false", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_debug_getattr_caps": "false", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_debug_inject_features": "", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_debug_inject_tick_delay": "0", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_die_on_failed_remount": "false", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_dirsize_rbytes": "true", 2024-09-21T11:41:01.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_force_lazyio": "false", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_fs": "", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_inject_release_failure": "false", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_max_inline_size": "4096", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_mds_namespace": "", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_metadata": "", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_mount_gid": "-1", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_mount_timeout": "300", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_mount_uid": "-1", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_mountpoint": "/", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_notify_timeout": "10", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_oc": "true", 2024-09-21T11:41:01.102 INFO:teuthology.orchestra.run.smithi043.stdout: "client_oc_max_dirty": "104857600", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_oc_max_objects": "1000", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_oc_size": "209715200", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_oc_target_dirty": "8388608", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_permissions": "true", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_quota": "true", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_quota_df": "true", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_readahead_max_bytes": "0", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_readahead_max_periods": "4", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_readahead_min": "131072", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_reconnect_stale": "false", 2024-09-21T11:41:01.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_shutdown_timeout": "30", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_snapdir": ".snap", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_tick_interval": "1", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_trace": "", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_try_dentry_invalidate": "false", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_use_faked_inos": "false", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_use_random_mds": "false", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "clog_to_graylog": "false", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "clog_to_graylog_port": "12201", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "clog_to_monitors": "default=true", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "clog_to_syslog": "false", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "clog_to_syslog_level": "info", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "-", 2024-09-21T11:41:01.104 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_network": "", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_network_interface": "", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "compressor_zlib_isal": "false", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "compressor_zlib_level": "5", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "compressor_zlib_winsize": "-15", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "compressor_zstd_level": "1", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "crimson_alien_op_num_threads": "6", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "crimson_osd_stat_interval": "0", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "crimson_seastar_cpu_cores": "", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "crimson_seastar_num_threads": "0", 2024-09-21T11:41:01.105 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_location": "", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_location_hook": "", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_location_hook_timeout": "10", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "daemonize": "false", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "daos_pool": "tank", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_alienstore": "0/5", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_allow_any_pool_priority": "false", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_asok": "1/5", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_asok_assert_abort": "false", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_asserts_on_shutdown": "true", 2024-09-21T11:41:01.106 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_auth": "1/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_bdev": "1/3", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_bluefs": "1/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_bluestore": "1/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_buffer": "0/1", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_ceph_dedup": "0/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_ceph_exporter": "1/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_cephfs_mirror": "0/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_cephsqlite": "0/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_client": "0/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_compressor": "1/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_context": "0/1", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_crimson_interrupt": "0/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_crush": "1/1", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_crypto": "1/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_cyanstore": "0/5", 2024-09-21T11:41:01.107 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_deliberately_leak_memory": "false", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_disable_randomized_ping": "false", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_dpdk": "1/5", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_eventtrace": "1/5", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_filer": "0/1", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_filestore": "1/3", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_finisher": "1/1", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_fuse": "1/5", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_heartbeat_testing_span": "0", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_heartbeatmap": "1/5", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_javaclient": "1/5", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_journal": "1/3", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_journaler": "0/5", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_kstore": "1/5", 2024-09-21T11:41:01.108 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_lockdep": "0/1", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mclock": "1/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mds": "1/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mds_balancer": "1/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mds_locker": "1/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mds_log": "1/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mds_log_expire": "1/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mds_migrator": "1/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mds_quiesce": "3/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_memstore": "1/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mgr": "2/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mgrc": "1/5", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_mon": "20/20", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_monc": "0/10", 2024-09-21T11:41:01.109 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_ms": "1/1", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_none": "0/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_objclass": "0/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_objectcacher": "0/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_objecter": "0/1", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_optracker": "0/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_osd": "1/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_paxos": "20/20", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_perfcounter": "1/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_prioritycache": "1/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rados": "0/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rbd": "0/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rbd_mirror": "0/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rbd_pwl": "0/5", 2024-09-21T11:41:01.110 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rbd_replay": "0/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_refs": "0/0", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_reserver": "1/1", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rgw": "1/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rgw_access": "1/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rgw_datacache": "1/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rgw_dbstore": "1/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rgw_flight": "1/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rgw_notification": "1/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rgw_sync": "1/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_rocksdb": "4/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore": "0/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_backref": "0/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_cache": "0/5", 2024-09-21T11:41:01.111 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_cleaner": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_device": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_epm": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_journal": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_lba": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_odata": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_omap": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_onode": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_t": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_seastore_tm": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_striper": "0/1", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_test": "0/5", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_throttle": "1/1", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_timer": "0/1", 2024-09-21T11:41:01.112 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_tp": "0/5", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "debug_trace": "1/5", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "device_failure_prediction_mode": "none", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "err_to_graylog": "false", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "err_to_journald": "false", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "err_to_stderr": "true", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "err_to_syslog": "false", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event_tracing": "false", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "exporter_addr": "0.0.0.0", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "exporter_cert_file": "", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "exporter_http_port": "9926", 2024-09-21T11:41:01.113 INFO:teuthology.orchestra.run.smithi043.stdout: "exporter_key_file": "", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "exporter_prio_limit": "5", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "exporter_sort_metrics": "true", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "exporter_stats_period": "5", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "fake_statfs_for_testing": "0", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "fatal_signal_handlers": "true", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "filer_max_purge_ops": "10", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "filer_max_truncate_ops": "128", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_apply_finisher_threads": "1", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_blackhole": "false", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_btrfs_clone_range": "true", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_btrfs_snap": "true", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_caller_concurrency": "10", 2024-09-21T11:41:01.114 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_collect_device_partition_information": "true", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_commit_timeout": "600.000000", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_debug_inject_read_err": "false", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_debug_omap_check": "false", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_debug_verify_split": "false", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_dump_file": "", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_fadvise": "true", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_fail_eio": "true", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_fd_cache_shards": "16", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_fd_cache_size": "128", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_fiemap": "false", 2024-09-21T11:41:01.115 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_fiemap_threshold": "4096", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_inject_stall": "0", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_journal_parallel": "false", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_journal_trailing": "false", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_journal_writeahead": "false", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_kill_at": "0", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_inline_xattrs": "0", 2024-09-21T11:41:01.116 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_xattr_value_size": "0", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_merge_threshold": "-10", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_odsync_write": "false", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_omap_backend": "rocksdb", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_omap_backend_path": "", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-21T11:41:01.117 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_op_thread_timeout": "60", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_op_threads": "2", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_punch_hole": "false", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-21T11:41:01.118 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_queue_max_ops": "50", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_seek_data_hole": "false", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_sloppy_crc": "false", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_splice": "false", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_split_multiple": "2", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_split_rand_factor": "20", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_update_to": "1000", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-21T11:41:01.119 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_enable": "true", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_xfs_extsize": "false", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "filestore_zfs_snap": "false", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "fio_dir": "/tmp/fio", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "fsid": "4f2c5252-780d-11ef-baf6-efdc52797490", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_allow_other": "true", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_atomic_o_trunc": "true", 2024-09-21T11:41:01.120 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_big_writes": "true", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_debug": "false", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_default_permissions": "false", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_disable_pagecache": "false", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_max_write": "0", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_multithreaded": "true", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_require_active_mds": "true", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_set_user_groups": "true", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_splice_move": "true", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_splice_read": "true", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_splice_write": "true", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "fuse_use_invalidate_cb": "true", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "gss_target_name": "ceph", 2024-09-21T11:41:01.121 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_file": "", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_inject_failure": "0", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_interval": "5", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "host": "smithi043", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-21T11:41:01.122 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "inject_early_sigterm": "false", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "jaeger_agent_port": "6799", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "jaeger_tracing_enable": "false", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_aio": "true", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_align_min_size": "65536", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_block_align": "true", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_block_size": "4096", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_dio": "true", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_discard": "false", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_force_aio": "false", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_ignore_corruption": "false", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_max_write_bytes": "10485760", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_max_write_entries": "100", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_replay_from": "0", 2024-09-21T11:41:01.123 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_write_header_frequency": "0", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "journal_zero_on_create": "false", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "journaler_prefetch_periods": "10", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "journaler_prezero_periods": "5", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "journaler_write_head_interval": "15", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "keyfile": "", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_backend": "rocksdb", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_default_stripe_size": "65536", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_fsck_on_mount": "false", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-21T11:41:01.124 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_max_bytes": "67108864", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_max_ops": "512", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_nid_prealloc": "1024", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_onode_map_size": "1024", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_sync_submit_transaction": "false", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "kstore_sync_transaction": "false", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "librados_thread_count": "2", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "lockdep": "false", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "lockdep_force_backtrace": "false", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "log_coarse_timestamps": "true", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "log_flush_on_exit": "false", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "log_graylog_host": "127.0.0.1", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "log_graylog_port": "12201", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "log_max_new": "1000", 2024-09-21T11:41:01.125 INFO:teuthology.orchestra.run.smithi043.stdout: "log_max_recent": "10000", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "log_stderr_prefix": "", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "log_stop_at_utilization": "0.970000", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_file": "true", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_graylog": "false", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_journald": "false", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_stderr": "false", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "log_to_syslog": "false", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "max_rotating_auth_attempts": "10", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_action_on_write_error": "1", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_alternate_name_max": "8192", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_asio_thread_count": "2", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_export_pin": "true", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_fragment_dirs": "true", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-21T11:41:01.126 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_fragment_interval": "5", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_interval": "10", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_max": "-1", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_max_until": "-1", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_merge_size": "50", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_midchunk": "0.300000", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_min_start": "0.200000", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_minchunk": "0.001000", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_mode": "0", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_need_max": "1.200000", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_need_min": "0.800000", 2024-09-21T11:41:01.127 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_overload_epochs": "2", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_split_bits": "3", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_split_size": "10000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_target_decay": "10.000000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_beacon_grace": "15.000000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_beacon_interval": "4.000000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_mid": "0.700000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_quiesce_delay": "0", 2024-09-21T11:41:01.128 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_release_free_interval": "10", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_reservation": "0.050000", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_trim_interval": "1", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache_trim_threshold": "262144", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_client_prealloc_inos": "1000", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_connect_bootstrapping": "false", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_damage_table_max_entries": "10000", 2024-09-21T11:41:01.129 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_debug_auth_pins": "false", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_debug_frag": "false", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_debug_scatterstat": "false", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_debug_subtrees": "false", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_decay_halflife": "5.000000", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_default_dir_hash": "2", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_defer_session_stale": "true", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_deny_all_reconnect": "false", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_dir_keys_per_op": "16384", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_dir_max_commit_size": "10", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_dir_max_entries": "0", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_dir_prefetch": "true", 2024-09-21T11:41:01.130 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_dump_cache_on_map": "false", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_early_reply": "true", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_enable_op_tracker": "true", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_enforce_unique_name": "true", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_export_ephemeral_random": "true", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-21T11:41:01.131 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_health_summarize_threshold": "10", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_inject_health_dummy": "false", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_inject_migrator_session_race": "false", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_join_fs": "", 2024-09-21T11:41:01.132 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_journal_format": "1", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_create_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_dirfrag_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_export_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_import_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_journal_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_journal_expire_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_journal_replay_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_link_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_mdstable_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_openc_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_rename_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_kill_shutdown_at": "0", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_event_large_threshold": "524288", 2024-09-21T11:41:01.133 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_events_per_segment": "1024", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_max_events": "-1", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_max_segments": "128", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_pause": "false", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_segment_size": "0", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_trim_threshold": "128", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log_warn_factor": "2.000000", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_max_caps_per_client": "1048576", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_max_completed_flushes": "100000", 2024-09-21T11:41:01.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_max_completed_requests": "100000", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_max_export_size": "20971520", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_max_file_recover": "32", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_max_purge_files": "64", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_max_purge_ops": "8192", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_max_snaps_per_dir": "100", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_metrics_update_interval": "2", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_min_caps_per_client": "100", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_min_caps_working_set": "10000", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_numa_node": "-1", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_op_complaint_time": "30.000000", 2024-09-21T11:41:01.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_op_history_duration": "600", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_op_history_size": "20", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_op_history_slow_op_size": "20", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_op_log_threshold": "5", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_ping_grace": "15", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_ping_interval": "5", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_recall_max_caps": "30000", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_recall_warning_threshold": "262144", 2024-09-21T11:41:01.136 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_replay_interval": "1.000000", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_root_ino_gid": "0", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_root_ino_uid": "0", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_scrub_stats_review_period": "1", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-21T11:41:01.137 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-21T11:41:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-21T11:41:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-21T11:41:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_shutdown_check": "0", 2024-09-21T11:41:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_skip_ino": "0", 2024-09-21T11:41:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-21T11:41:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_snap_max_uid": "4294967294", 2024-09-21T11:41:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_snap_min_uid": "0", 2024-09-21T11:41:01.138 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_snap_rstat": "false", 2024-09-21T11:41:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_standby_replay_damaged": "false", 2024-09-21T11:41:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_symlink_recovery": "true", 2024-09-21T11:41:01.139 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_thrash_exports": "0", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_thrash_fragments": "0", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_tick_interval": "5.000000", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_valgrind_exit": "false", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_verify_backtrace": "1", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_verify_scatter": "false", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_wipe_sessions": "false", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool_debug": "false", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "memstore_device_bytes": "1073741824", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "memstore_page_set": "false", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "memstore_page_size": "65536", 2024-09-21T11:41:01.140 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_client_bytes": "134217728", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_client_messages": "512", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_disabled_modules": "", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_enable_op_tracker": "true", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_max_pg_creating": "1024", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_max_pg_num_change": "128", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_mds_bytes": "134217728", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_mds_messages": "128", 2024-09-21T11:41:01.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_mon_bytes": "134217728", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_mon_messages": "128", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_op_history_duration": "600", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_op_history_size": "20", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_op_log_threshold": "5", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_osd_bytes": "536870912", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_osd_messages": "8192", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_pool": "true", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-21T11:41:01.142 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_standby_modules": "true", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_stats_period": "5", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_stats_threshold": "5", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_tick_period": "2", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_allow_pool_delete": "true", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_allow_pool_size_one": "false", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_auth_validate_all_caps": "true", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_bytes": "104857600", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_directed_command_retry": "2", 2024-09-21T11:41:01.143 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_hunt_parallel": "3", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_log_interval": "1.000000", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_ping_interval": "10.000000", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_client_target_rank": "-1", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cluster_log_level": "debug", 2024-09-21T11:41:01.144 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cluster_log_to_file": "true", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cluster_log_to_journald": "true", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_compact_on_bootstrap": "false", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_compact_on_start": "false", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_compact_on_trim": "true", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-21T11:41:01.145 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_cpu_threads": "4", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_crush_min_required_version": "hammer", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_daemon_bytes": "419430400", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_data_avail_crit": "5", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_data_avail_warn": "5", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_data_size_warn": "16106127360", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_dump_json": "false", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_dump_transactions": "false", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_extra_checks": "false", 2024-09-21T11:41:01.146 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_no_require_reef": "false", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_no_require_squid": "false", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_down_mkfs_grace": "60", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_down_uptime_grace": "60", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_election_default_strategy": "1", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_election_timeout": "5.000000", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_elector_ping_divisor": "2", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_enable_op_tracker": "true", 2024-09-21T11:41:01.147 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_fake_pool_delete": "false", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_force_quorum_join": "false", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_globalid_prealloc": "10000", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_health_detail_to_clog": "true", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_health_log_update_period": "5", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_health_max_detail": "50", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_health_to_clog": "true", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_health_to_clog_interval": "600", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_host": "[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0]", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_host_override": "", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_initial_members": "", 2024-09-21T11:41:01.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_lease": "5.000000", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_log_full_interval": "50", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_log_max": "10000", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_log_max_summary": "50", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_max_log_epochs": "500", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-21T11:41:01.149 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_max_osd": "10000", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_max_pg_per_osd": "10000", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_max_pool_pg_num": "65536", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_mds_force_trim_to": "0", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_mds_skip_sanity": "false", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_memory_autotune": "true", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_memory_target": "2147483648", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_mgr_beacon_grace": "30", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_mgr_digest_period": "5", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_mgr_inactive_grace": "60", 2024-09-21T11:41:01.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_min_osdmap_epochs": "50", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_op_complaint_time": "30", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_op_history_duration": "600", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_op_history_size": "20", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_op_history_slow_op_size": "20", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_op_log_threshold": "5", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_auto_mark_in": "false", 2024-09-21T11:41:01.151 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_cache_size": "500", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_down_out_interval": "600", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_force_trim_to": "0", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-21T11:41:01.152 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_min_down_reporters": "2", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_report_timeout": "900", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-21T11:41:01.153 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_pg_stuck_threshold": "60", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_probe_timeout": "2.000000", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_reweight_max_change": "0.050000", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_reweight_max_osds": "4", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-21T11:41:01.154 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_scrub_interval": "86400", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_scrub_max_keys": "100", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_scrub_timeout": "300", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_session_timeout": "300", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_smart_report_timeout": "5", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_stat_smooth_intervals": "6", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_stretch_pool_min_size": "2", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_stretch_pool_size": "4", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_sync_debug": "false", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-21T11:41:01.155 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_sync_provider_kill_at": "0", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_sync_requester_kill_at": "0", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_sync_timeout": "60.000000", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_target_pg_per_osd": "100", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_tick_interval": "5", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_timecheck_interval": "300.000000", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_use_min_delay_socket": "false", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_older_version_delay": "604800", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_filestore_osds": "true", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-21T11:41:01.156 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_misplaced": "false", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_older_version": "true", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "monmap": "", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-21T11:41:01.157 INFO:teuthology.orchestra.run.smithi043.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "motr_tracing_enabled": "false", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_op_threads": "3", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_cm": "false", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_device_name": "", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_dscp": "96", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_local_gid": "", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-21T11:41:01.158 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_port_num": "1", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_sl": "3", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_support_srq": "true", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_rdma_type": "ib", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_async_reap_threshold": "5", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_bind_before_connect": "false", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_bind_ipv4": "true", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_bind_ipv6": "false", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_bind_msgr1": "false", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_bind_msgr2": "true", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_bind_port_max": "7568", 2024-09-21T11:41:01.159 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_bind_port_min": "6800", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_bind_retry_count": "3", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_bind_retry_delay": "5", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_blackhole_client": "false", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_blackhole_mds": "false", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_blackhole_mgr": "false", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_blackhole_mon": "false", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_blackhole_osd": "false", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_client_mode": "crc secure", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_cluster_mode": "crc secure", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_cluster_type": "", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_compress_secure": "false", 2024-09-21T11:41:01.160 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_connection_idle_timeout": "900", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_connection_ready_timeout": "10", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_crc_data": "true", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_crc_header": "true", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_die_on_bad_msg": "false", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_die_on_bug": "true", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_die_on_old_message": "true", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_die_on_skipped_message": "false", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_coremask": "0xF", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_enable_tso": "true", 2024-09-21T11:41:01.161 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_hugepages": "", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_lro": "true", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_memory_channel": "4", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_pmd": "", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_port_id": "0", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_dump_on_send": "false", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_initial_backoff": "0.200000", 2024-09-21T11:41:01.162 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_inject_delay_max": "1.000000", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_inject_delay_type": "", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_inject_network_congestion": "0", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_inject_socket_failures": "0", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_learn_addr_from_peer": "true", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_max_accept_failures": "4", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_max_backoff": "15.000000", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_mon_client_mode": "secure crc", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_mon_service_mode": "secure crc", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_osd_compress_min_size": "1024", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_osd_compress_mode": "none", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-21T11:41:01.163 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_pq_min_cost": "65536", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_public_type": "", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_service_mode": "crc secure", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_tcp_listen_backlog": "512", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_tcp_nodelay": "true", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_tcp_rcvbuf": "0", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "ms_type": "async+posix", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "no_config_file": "false", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter_completion_locks_per_session": "32", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-21T11:41:01.164 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter_inflight_ops": "1024", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter_tick_interval": "5.000000", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter_timeout": "10.000000", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "objectstore_blackhole": "false", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "openssl_engine_opts": "", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_agent_delay_time": "5.000000", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_agent_hist_halflife": "1000", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_agent_max_low_ops": "2", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_agent_max_ops": "4", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-21T11:41:01.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_agent_slop": "0.020000", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_asio_thread_count": "2", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_async_recovery_min_cost": "100", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_backfill_scan_max": "512", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_backfill_scan_min": "64", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_backoff_on_degraded": "false", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_backoff_on_peering": "false", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_backoff_on_unfound": "true", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_beacon_report_interval": "300", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bench_duration": "30", 2024-09-21T11:41:01.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bench_max_block_size": "67108864", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_blkin_trace_all": "false", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_check_for_log_corruption": "false", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_class_update_on_start": "true", 2024-09-21T11:41:01.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_client_message_cap": "256", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_client_message_size_cap": "524288000", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_client_op_priority": "63", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_client_watch_timeout": "30", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_command_max_records": "256", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_compact_on_start": "false", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_crush_chooseleaf_type": "1", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_crush_update_on_start": "true", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_crush_update_weight_set": "true", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-21T11:41:01.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_feed_pullee": "-1", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_misdirected_ops": "false", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_no_acting_change": "false", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_no_purge_strays": "false", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_op_order": "false", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_shutdown": "false", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-21T11:41:01.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_trim_objects": "false", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_verify_snaps": "false", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_deep_scrub_keys": "1024", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_deep_scrub_stride": "524288", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_default_data_pool_replay_window": "45", 2024-09-21T11:41:01.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_default_notify_timeout": "30", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_delete_sleep": "0.000000", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_discard_disconnected_ops": "true", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_ec_partial_reads": "true", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_enable_op_tracker": "true", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_fast_info": "true", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_fast_shutdown": "true", 2024-09-21T11:41:01.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_function_tracing": "false", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_heartbeat_grace": "20", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_heartbeat_interval": "6", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_heartbeat_min_peers": "10", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_heartbeat_min_size": "2000", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_heartbeat_stale": "600", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_hit_set_max_size": "100000", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_hit_set_min_size": "1000", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-21T11:41:01.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_journal_size": "5120", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_kill_backfill_at": "0", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_map_cache_size": "50", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_map_dedup": "true", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_map_message_max": "40", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_map_share_max_epochs": "40", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_attr_name_len": "100", 2024-09-21T11:41:01.173 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_attr_size": "0", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_backfills": "1", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_markdown_count": "5", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_markdown_period": "600", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_object_name_len": "2048", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_object_namespace_len": "256", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_object_size": "134217728", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_pg_blocked_by": "16", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_pg_log_entries": "10000", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_pgls": "1024", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_push_cost": "8388608", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_push_objects": "10", 2024-09-21T11:41:01.174 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_scrubs": "3", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_trimming_pgs": "2", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_write_op_reply_len": "64", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_max_write_size": "90", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_profile": "balanced", 2024-09-21T11:41:01.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_memory_base": "805306368", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_memory_cache_min": "134217728", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-21T11:41:01.176 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_memory_target": "4294967296", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_memory_target_autotune": "false", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_min_pg_log_entries": "250", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_min_recovery_priority": "0", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mon_report_interval": "5", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_num_cache_shards": "32", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_num_op_tracker_shard": "32", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_numa_auto_affinity": "true", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_numa_node": "-1", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_numa_prefer_iface": "true", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-21T11:41:01.177 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_objecter_finishers": "1", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_objectstore": "bluestore", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_objectstore_fuse": "false", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_objectstore_tracing": "false", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_complaint_time": "30.000000", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_history_duration": "600", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_history_size": "20", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_history_slow_op_size": "20", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_log_threshold": "5", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_num_shards": "0", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_num_shards_hdd": "5", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_num_shards_ssd": "8", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-21T11:41:01.178 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_pq_min_cost": "65536", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_queue_cut_off": "high", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_op_thread_timeout": "15", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_open_classes_on_start": "true", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_os_flags": "0", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_peering_op_priority": "255", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_delete_cost": "1048576", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_delete_priority": "5", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_log_trim_max": "10000", 2024-09-21T11:41:01.179 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_log_trim_min": "100", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_crimson": "false", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-21T11:41:01.180 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_flags": "0", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_min_size": "0", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_pg_num": "32", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_pgp_num": "0", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_read_ratio": "70", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_size": "3", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_default_type": "replicated", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-21T11:41:01.181 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_push_per_object_cost": "1000", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recover_clone_overlap": "true", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_cost": "20971520", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_max_active": "0", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_max_single_start": "1", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_op_priority": "3", 2024-09-21T11:41:01.182 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_priority": "5", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_sleep": "0.000000", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_repair_during_recovery": "false", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_requested_scrub_priority": "5", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_auto_repair": "false", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_begin_hour": "0", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_begin_week_day": "0", 2024-09-21T11:41:01.183 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_chunk_max": "15", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_chunk_min": "5", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_cost": "52428800", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_during_recovery": "false", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_end_hour": "0", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_end_week_day": "0", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_event_cost": "4096", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_invalid_stats": "true", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_max_preemptions": "5", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_priority": "5", 2024-09-21T11:41:01.184 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_retry_delay": "30", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_retry_trimming": "10", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_scrub_sleep": "0.000000", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_skip_data_digest": "false", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_smart_report_timeout": "5", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_snap_trim_cost": "1048576", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_snap_trim_priority": "5", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-21T11:41:01.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_target_transaction_size": "30", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-21T11:41:01.186 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_tracing": "false", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_use_stale_snap": "false", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "osdc_blkin_trace_all": "false", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_kill_at": "0", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_max_join_drift": "10", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_min": "500", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_min_wait": "0.050000", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_propose_interval": "1.000000", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_service_trim_max": "500", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-21T11:41:01.187 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_service_trim_min": "10", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_stash_full_interval": "25", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_trim_max": "500", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "paxos_trim_min": "250", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "perf": "true", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "pid_file": "", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "-", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrv": "", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "public_bind_addr": "-", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "public_network": "", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "public_network_interface": "", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "qat_compressor_busy_polling": "false", 2024-09-21T11:41:01.188 INFO:teuthology.orchestra.run.smithi043.stdout: "qat_compressor_enabled": "false", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "qat_compressor_session_max_number": "256", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rados_mon_op_timeout": "0", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rados_osd_op_timeout": "0", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rados_tracing": "false", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_atime_update_interval": "60", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_balance_parent_reads": "false", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_balance_snap_reads": "false", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_blkin_trace_all": "false", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_cache": "true", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-21T11:41:01.189 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_cache_policy": "writearound", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_cache_size": "33554432", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_clone_copy_on_read": "false", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_compression_hint": "none", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_concurrent_management_ops": "10", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_default_clone_format": "auto", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_default_data_pool": "", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_default_features": "61", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_default_format": "2", 2024-09-21T11:41:01.190 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_default_map_options": "", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_default_order": "22", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_default_pool": "rbd", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_default_stripe_count": "0", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_default_stripe_unit": "0", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_enable_alloc_hint": "true", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_io_scheduler": "simple", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-21T11:41:01.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_order": "24", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_pool": "", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_journal_splay_width": "4", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_localize_parent_reads": "false", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_localize_snap_reads": "false", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-21T11:41:01.192 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-21T11:41:01.193 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_mtime_update_interval": "60", 2024-09-21T11:41:01.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_non_blocking_aio": "true", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_op_thread_timeout": "60", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_op_threads": "1", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_parent_cache_enabled": "false", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_plugins": "", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_bps_burst": "0", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_bps_limit": "0", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_exclude_ops": "0", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_iops_burst": "0", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_iops_limit": "0", 2024-09-21T11:41:01.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_read_from_replica_policy": "default", 2024-09-21T11:41:01.196 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_skip_partial_discard": "true", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_tracing": "false", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_validate_names": "true", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd_validate_pool": "true", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "restapi_base_url": "", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "restapi_log_level": "", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_acl_grants_max_num": "100", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_admin_entry": "admin", 2024-09-21T11:41:01.197 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_asio_assert_yielding": "false", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_backend_store": "rados", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_barbican_url": "", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_beast_enable_async": "true", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_bucket_counters_cache": "false", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-21T11:41:01.198 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_cache_enabled": "true", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_cache_expiry_interval": "900", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_cache_lru_size": "25000", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_config_store": "rados", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_content_length_compat": "false", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_copy_obj_progress": "true", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_cors_rules_max_num": "100", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_cross_domain_policy": "", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_kmip_addr": "", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-21T11:41:01.199 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_kmip_password": "", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_kmip_username": "", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_require_ssl": "true", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-21T11:41:01.200 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_vault_addr": "", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_vault_auth": "token", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_vault_namespace": "", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_vault_prefix": "", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-21T11:41:01.201 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_vault_token_file": "", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_curl_buffersize": "524288", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_curl_low_speed_time": "300", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-21T11:41:01.202 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_data_log_changes_size": "1000", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_data_log_num_shards": "128", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_data_log_window": "30", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-21T11:41:01.203 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_disable_s3select": "false", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-21T11:41:01.204 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dns_name": "", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dns_s3website_name": "", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dynamic_resharding": "true", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-21T11:41:01.205 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_enable_gc_threads": "true", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_enable_lc_threads": "true", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_enable_mdsearch": "true", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_enable_ops_log": "false", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_enable_quota_threads": "true", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_enable_static_website": "false", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_enable_usage_log": "false", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_enforce_swift_acls": "true", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_exit_timeout_secs": "120", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_expose_bucket": "false", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_extended_http_attrs": "", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_filter": "none", 2024-09-21T11:41:01.206 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_frontends": "beast port=7480", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_gc_max_deferred": "50", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_gc_max_objs": "32", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_gc_processor_period": "3600", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-21T11:41:01.207 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_init_timeout": "300", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_kafka_connection_idle": "300", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_admin_domain": "", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_admin_password": "", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_admin_password_path": "", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_admin_project": "", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_admin_tenant": "", 2024-09-21T11:41:01.208 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_admin_token": "", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_admin_token_path": "", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_admin_user": "", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_barbican_domain": "", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_barbican_password": "", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_barbican_project": "", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_barbican_user": "", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-21T11:41:01.209 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_url": "", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_lc_debug_interval": "-1", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_lc_lock_max_time": "90", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_lc_max_objs": "32", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_lc_max_rules": "1000", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_lc_max_worker": "3", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_lc_thread_delay": "0", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ldap_dnattr": "uid", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ldap_searchfilter": "", 2024-09-21T11:41:01.210 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ldap_uri": "", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_log_http_headers": "", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_log_object_name_utc": "false", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_attr_name_len": "0", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_attr_size": "0", 2024-09-21T11:41:01.211 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_chunk_size": "4194304", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_listing_results": "5000", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_notify_retries": "10", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_put_param_size": "1048576", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_put_size": "5368709120", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_max_slo_entries": "1000", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_md_log_max_shards": "64", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-21T11:41:01.212 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_mp_lock_max_time": "600", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_max_gc": "300", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-21T11:41:01.213 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_num_async_rados_threads": "32", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_num_control_oids": "8", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_numa_node": "-1", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_objexp_chunk_size": "100", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_objexp_gc_interval": "600", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-21T11:41:01.214 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_op_thread_timeout": "600", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_op_tracing": "false", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_opa_token": "", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_opa_url": "", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_opa_verify_ssl": "true", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ops_log_rados": "false", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_ops_log_socket_path": "", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_period_push_interval": "2.000000", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-21T11:41:01.215 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_posix_cache_lanes": "3", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_posix_cache_partitions": "3", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_print_continue": "true", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_rados_tracing": "false", 2024-09-21T11:41:01.216 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_realm": "", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_realm_id": "", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_region": "", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_request_uri": "", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_reshard_batch_size": "64", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_reshard_max_aio": "128", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_reshard_num_logs": "16", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-21T11:41:01.217 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_reshard_thread_interval": "600", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_resolve_cname": "false", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_run_sync_thread": "true", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-21T11:41:01.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_scheduler_type": "throttler", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_script_uri": "", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_service_provider_name": "", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sts_client_id": "", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sts_client_secret": "", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sts_entry": "sts", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sts_key": "sts", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sts_min_session_duration": "900", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sts_token_introspection_url": "", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_account_in_url": "false", 2024-09-21T11:41:01.219 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_auth_entry": "auth", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_auth_url": "", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_custom_header": "", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_need_stats": "true", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_tenant_name": "", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_token_expiration": "86400", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_url": "", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_url_prefix": "swift", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_lease_period": "120", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-21T11:41:01.220 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_thread_pool_size": "512", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_torrent_comment": "", 2024-09-21T11:41:01.221 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_torrent_createby": "", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_torrent_encoding": "", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_torrent_flag": "false", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_torrent_origin": "", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_torrent_tracker": "", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_trust_forwarded_https": "false", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_usage_max_shards": "32", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_usage_max_user_shards": "1", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_use_opa_authz": "false", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_counters_cache": "false", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-21T11:41:01.222 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_max_buckets": "1000", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_policies_max_num": "100", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_user_unique_email": "true", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_verify_ssl": "true", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_zone": "", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_zone_id": "", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_zonegroup": "", 2024-09-21T11:41:01.223 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_zonegroup_id": "", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rgwlc_auto_session_clear": "true", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_block_size": "4096", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_cache_size": "536870912", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-21T11:41:01.224 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_index_type": "binary_search", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_partition_filters": "false", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_perf": "false", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "run_dir": "/var/run/ceph", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_block_create": "true", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_cache_lru_size": "67108864", 2024-09-21T11:41:01.225 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_default_max_object_size": "16777216", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_device_size": "53687091200", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_full_integrity_check": "false", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_journal_batch_capacity": "16", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_main_device_type": "SSD", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-21T11:41:01.226 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "seastore_segment_size": "67108864", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "set_keepcaps": "false", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "setgroup": "ceph", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "setuser": "ceph", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "setuser_match_path": "", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "thp": "false", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "threadpool_default_timeout": "60", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "throttler_perf_counter": "true", 2024-09-21T11:41:01.227 INFO:teuthology.orchestra.run.smithi043.stdout: "tmp_dir": "/tmp", 2024-09-21T11:41:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-21T11:41:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "uadk_compressor_enabled": "false", 2024-09-21T11:41:01.228 INFO:teuthology.orchestra.run.smithi043.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-21T11:41:01.228 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:41:01.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:01 smithi043 ceph-mon[36454]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:41:01.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:01 smithi043 ceph-mon[36454]: 3.1c scrub starts 2024-09-21T11:41:01.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:01 smithi043 ceph-mon[36454]: 3.1c scrub ok 2024-09-21T11:41:01.449 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:01 smithi043 ceph-mon[29710]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:41:01.449 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:01 smithi043 ceph-mon[29710]: 3.1c scrub starts 2024-09-21T11:41:01.449 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:01 smithi043 ceph-mon[29710]: 3.1c scrub ok 2024-09-21T11:41:01.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:01 smithi175 ceph-mon[32613]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:41:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:01 smithi175 ceph-mon[32613]: 3.1c scrub starts 2024-09-21T11:41:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:01 smithi175 ceph-mon[32613]: 3.1c scrub ok 2024-09-21T11:41:01.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-21T11:41:02.122 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:02.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:02 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:41:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:02 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:41:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:02 smithi175 ceph-mon[32613]: osdmap e79: 8 total, 8 up, 8 in 2024-09-21T11:41:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:02 smithi175 ceph-mon[32613]: osdmap e80: 8 total, 8 up, 8 in 2024-09-21T11:41:02.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:02.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:02 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:41:02.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:02 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:41:02.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:02 smithi043 ceph-mon[29710]: osdmap e79: 8 total, 8 up, 8 in 2024-09-21T11:41:02.575 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:02 smithi043 ceph-mon[29710]: osdmap e80: 8 total, 8 up, 8 in 2024-09-21T11:41:02.575 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:02.575 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:02 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:41:02.575 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:02 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:41:02.575 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:02 smithi043 ceph-mon[36454]: osdmap e79: 8 total, 8 up, 8 in 2024-09-21T11:41:02.575 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:02 smithi043 ceph-mon[36454]: osdmap e80: 8 total, 8 up, 8 in 2024-09-21T11:41:02.671 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} 2024-09-21T11:41:02.672 INFO:teuthology.orchestra.run.smithi043.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T11:41:02.678 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} 2024-09-21T11:41:02.678 INFO:teuthology.orchestra.run.smithi043.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T11:41:02.686 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2024-09-21T11:41:02.686 INFO:teuthology.orchestra.run.smithi043.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T11:41:03.387 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-21T11:41:03.401 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-21T11:41:05.599 INFO:tasks.radosbench:Beginning radosbench... 2024-09-21T11:41:05.599 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-21T11:41:05.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool create unique_pool_0 16 2024-09-21T11:41:05.600 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-21T11:41:05.600 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:41:05.600 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:41:05.600 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-21T11:41:05.600 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T11:41:05.600 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T11:41:05.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-09-21T11:41:05.601 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-21T11:41:05.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:05.601 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-21T11:41:05.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:41:05.602 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-21T11:41:05.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:41:05.603 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-21T11:41:05.603 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:41:05.604 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:03 smithi043 ceph-mon[36454]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T11:41:05.604 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:03 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4186318384' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:41:05.604 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:03 smithi043 ceph-mon[36454]: 4.c scrub starts 2024-09-21T11:41:05.604 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:03 smithi043 ceph-mon[36454]: 4.c scrub ok 2024-09-21T11:41:05.604 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:03 smithi175 ceph-mon[32613]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T11:41:05.604 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:03 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4186318384' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:41:05.605 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:03 smithi175 ceph-mon[32613]: 4.c scrub starts 2024-09-21T11:41:05.605 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:03 smithi175 ceph-mon[32613]: 4.c scrub ok 2024-09-21T11:41:05.605 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:05 smithi175 ceph-mon[32613]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 775 B/s rd, 0 op/s 2024-09-21T11:41:05.605 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:05 smithi175 ceph-mon[32613]: 5.18 scrub starts 2024-09-21T11:41:05.605 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:05 smithi175 ceph-mon[32613]: 5.18 scrub ok 2024-09-21T11:41:05.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:03 smithi043 ceph-mon[29710]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-21T11:41:05.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:03 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4186318384' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T11:41:05.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:03 smithi043 ceph-mon[29710]: 4.c scrub starts 2024-09-21T11:41:05.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:03 smithi043 ceph-mon[29710]: 4.c scrub ok 2024-09-21T11:41:05.606 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:05 smithi043 ceph-mon[36454]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 775 B/s rd, 0 op/s 2024-09-21T11:41:05.606 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:05 smithi043 ceph-mon[36454]: 5.18 scrub starts 2024-09-21T11:41:05.606 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:05 smithi043 ceph-mon[36454]: 5.18 scrub ok 2024-09-21T11:41:05.606 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:05 smithi043 ceph-mon[29710]: pgmap v27: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 775 B/s rd, 0 op/s 2024-09-21T11:41:05.606 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:05 smithi043 ceph-mon[29710]: 5.18 scrub starts 2024-09-21T11:41:05.606 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:05 smithi043 ceph-mon[29710]: 5.18 scrub ok 2024-09-21T11:41:05.607 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:41:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:41:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:41:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:05.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:05.642 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:05.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:05.776 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:05.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:05.778 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:05.878 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:05.908 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:05.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:05.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:05.937 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:05.941 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:05.947 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:06.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:06.031 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:06.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:06.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:06.133 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:06.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:06.155 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:06.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:06.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:06.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:06.280 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:06.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:06.330 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:06 smithi043 ceph-mon[36454]: 3.17 scrub starts 2024-09-21T11:41:06.330 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:06 smithi043 ceph-mon[36454]: 3.17 scrub ok 2024-09-21T11:41:06.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:06 smithi043 ceph-mon[29710]: 3.17 scrub starts 2024-09-21T11:41:06.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:06 smithi043 ceph-mon[29710]: 3.17 scrub ok 2024-09-21T11:41:06.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:06.404 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:06.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:06.405 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:06.500 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:41:06.506 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:06.529 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:06.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:06.530 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:06.611 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:06 smithi175 ceph-mon[32613]: 3.17 scrub starts 2024-09-21T11:41:06.611 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:06 smithi175 ceph-mon[32613]: 3.17 scrub ok 2024-09-21T11:41:06.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:06.652 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:06.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:06.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:06.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:06.775 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:06.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:06.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:06.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:06.900 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:06.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:06.901 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:06.914 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:41:06.914 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:41:06.914 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-21T11:41:06.914 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:41:07.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:07.024 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:07.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:07.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:07.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:07.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:41:07 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:41:07.066Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002201898s 2024-09-21T11:41:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:07.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:07.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:07.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:07.266 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:41:07.274 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:07.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:07.275 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:07.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:07.399 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:07.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:07.400 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:07.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:07 smithi043 ceph-mon[36454]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:41:07.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:07 smithi043 ceph-mon[36454]: 5.1f scrub starts 2024-09-21T11:41:07.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:07 smithi043 ceph-mon[36454]: 5.1f scrub ok 2024-09-21T11:41:07.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3971580455' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:41:07.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:07 smithi043 ceph-mon[29710]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:41:07.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:07 smithi043 ceph-mon[29710]: 5.1f scrub starts 2024-09-21T11:41:07.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:07 smithi043 ceph-mon[29710]: 5.1f scrub ok 2024-09-21T11:41:07.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3971580455' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:41:07.501 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:07.522 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:07.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:07.523 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:07.527 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:07 smithi175 ceph-mon[32613]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:41:07.527 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:07 smithi175 ceph-mon[32613]: 5.1f scrub starts 2024-09-21T11:41:07.527 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:07 smithi175 ceph-mon[32613]: 5.1f scrub ok 2024-09-21T11:41:07.527 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3971580455' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:41:07.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:07.626 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:41:07.626 INFO:teuthology.orchestra.run.smithi175.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T11:41:07.626 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:41:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:07.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:07.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:07.748 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:07.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:07.771 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:07.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:07.873 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:07.894 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:07.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:07.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:07.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:08.017 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:08.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:08.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:08.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:08.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:08.140 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:08.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:08.241 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:08.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:08.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:08.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:08.389 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:08.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:08.390 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:08.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[29710]: 6.1c scrub starts 2024-09-21T11:41:08.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[29710]: 6.1c scrub ok 2024-09-21T11:41:08.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[29710]: 4.3 scrub starts 2024-09-21T11:41:08.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[29710]: 4.3 scrub ok 2024-09-21T11:41:08.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[29710]: 4.7 scrub starts 2024-09-21T11:41:08.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[29710]: 4.7 scrub ok 2024-09-21T11:41:08.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:08.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3971580455' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:41:08.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[29710]: osdmap e81: 8 total, 8 up, 8 in 2024-09-21T11:41:08.466 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[36454]: 6.1c scrub starts 2024-09-21T11:41:08.466 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[36454]: 6.1c scrub ok 2024-09-21T11:41:08.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[36454]: 4.3 scrub starts 2024-09-21T11:41:08.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[36454]: 4.3 scrub ok 2024-09-21T11:41:08.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[36454]: 4.7 scrub starts 2024-09-21T11:41:08.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[36454]: 4.7 scrub ok 2024-09-21T11:41:08.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:08.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3971580455' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:41:08.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:08 smithi043 ceph-mon[36454]: osdmap e81: 8 total, 8 up, 8 in 2024-09-21T11:41:08.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:08.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:08 smithi175 ceph-mon[32613]: 6.1c scrub starts 2024-09-21T11:41:08.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:08 smithi175 ceph-mon[32613]: 6.1c scrub ok 2024-09-21T11:41:08.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:08 smithi175 ceph-mon[32613]: 4.3 scrub starts 2024-09-21T11:41:08.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:08 smithi175 ceph-mon[32613]: 4.3 scrub ok 2024-09-21T11:41:08.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:08 smithi175 ceph-mon[32613]: 4.7 scrub starts 2024-09-21T11:41:08.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:08 smithi175 ceph-mon[32613]: 4.7 scrub ok 2024-09-21T11:41:08.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:08 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:08.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:08 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3971580455' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:41:08.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:08 smithi175 ceph-mon[32613]: osdmap e81: 8 total, 8 up, 8 in 2024-09-21T11:41:08.514 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:08.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:08.515 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:08.568 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:41:08.568 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:41:08.585 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:41:08.585 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:41:08.601 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:41:08.601 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:41:08.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:08.618 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:41:08.618 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:41:08.634 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:41:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:41:08.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:08.639 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:08.639 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:08.652 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:41:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:41:08.669 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:41:08.669 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:41:08.686 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:41:08.686 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:41:08.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:08.763 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:08.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:08.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:08.770 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:08 smithi175 systemd[1]: Starting Ceph grafana.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:41:08.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:08.887 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:08.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:08.888 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:08.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:09.010 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:09.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:09.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:09.030 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:08 smithi175 podman[61364]: 2024-09-21 11:41:08.961298989 +0000 UTC m=+0.093562109 container create c65f2f3d05b7dbb998b820af5e2cf5a42589cc943cf4e28eb9042cc3fe917d94 (image=quay.io/ceph/grafana:10.4.8, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T11:41:09.030 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:08 smithi175 podman[61364]: 2024-09-21 11:41:08.88483678 +0000 UTC m=+0.017099865 image pull 5aad1d7cf004c80a4acb170e1c80fb85781b022f00f08c6bcd22608882b4c748 quay.io/ceph/grafana:10.4.8 2024-09-21T11:41:09.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:09.136 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:09.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:09.137 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:09.238 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:09.261 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:09.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:09.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:09.281 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 podman[61364]: 2024-09-21 11:41:09.171740517 +0000 UTC m=+0.304003599 container init c65f2f3d05b7dbb998b820af5e2cf5a42589cc943cf4e28eb9042cc3fe917d94 (image=quay.io/ceph/grafana:10.4.8, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T11:41:09.282 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 podman[61364]: 2024-09-21 11:41:09.174855503 +0000 UTC m=+0.307118574 container start c65f2f3d05b7dbb998b820af5e2cf5a42589cc943cf4e28eb9042cc3fe917d94 (image=quay.io/ceph/grafana:10.4.8, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T11:41:09.282 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 bash[61364]: c65f2f3d05b7dbb998b820af5e2cf5a42589cc943cf4e28eb9042cc3fe917d94 2024-09-21T11:41:09.282 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 systemd[1]: Started Ceph grafana.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:41:09.291 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'unique_pool_0' created 2024-09-21T11:41:09.363 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:09.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:09.385 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:09.385 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:09.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:09 smithi043 ceph-mon[29710]: 6.3 scrub starts 2024-09-21T11:41:09.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:09 smithi043 ceph-mon[29710]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 791 B/s rd, 0 op/s 2024-09-21T11:41:09.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4102123598' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-21T11:41:09.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:09 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-21T11:41:09.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2247163045' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:09.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:09 smithi043 ceph-mon[36454]: 6.3 scrub starts 2024-09-21T11:41:09.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:09 smithi043 ceph-mon[36454]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 791 B/s rd, 0 op/s 2024-09-21T11:41:09.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4102123598' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-21T11:41:09.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:09 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-21T11:41:09.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2247163045' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:09.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:41:09.486 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:09.508 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:09.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:09.509 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:09.533 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-mon[32613]: 6.3 scrub starts 2024-09-21T11:41:09.533 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-mon[32613]: pgmap v30: 132 pgs: 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 791 B/s rd, 0 op/s 2024-09-21T11:41:09.533 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4102123598' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-21T11:41:09.533 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-21T11:41:09.533 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2247163045' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310546504Z level=info msg="Starting Grafana" version=10.4.8 commit=8c384b71bde142a25ddc01ed5a183ca572b2b7f7 branch=v10.4.x compiled=2024-09-21T11:41:09Z 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310910746Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310922337Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310933618Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310941498Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310946736Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310951732Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310956857Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310962395Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310967415Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.31097202Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-21T11:41:09.534 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.31097642Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-21T11:41:09.535 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.31098066Z level=info msg=Target target=[all] 2024-09-21T11:41:09.535 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310992905Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-21T11:41:09.535 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.310997306Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-21T11:41:09.535 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.311001372Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-21T11:41:09.535 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.311005465Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-21T11:41:09.535 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.311009461Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-21T11:41:09.536 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=settings t=2024-09-21T11:41:09.311013377Z level=info msg="App mode production" 2024-09-21T11:41:09.536 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore t=2024-09-21T11:41:09.311388356Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-21T11:41:09.536 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore t=2024-09-21T11:41:09.311423339Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-21T11:41:09.536 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.312124443Z level=info msg="Starting DB migrations" 2024-09-21T11:41:09.536 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.313369511Z level=info msg="Executing migration" id="create migration_log table" 2024-09-21T11:41:09.536 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.314547802Z level=info msg="Migration successfully executed" id="create migration_log table" duration=1.175921ms 2024-09-21T11:41:09.536 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.315184989Z level=info msg="Executing migration" id="create user table" 2024-09-21T11:41:09.536 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.315925954Z level=info msg="Migration successfully executed" id="create user table" duration=739.811µs 2024-09-21T11:41:09.536 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.316415481Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.31695694Z level=info msg="Migration successfully executed" id="add unique index user.login" duration=541.616µs 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.317380636Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.317993865Z level=info msg="Migration successfully executed" id="add unique index user.email" duration=612.901µs 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.318434077Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.318910359Z level=info msg="Migration successfully executed" id="drop index UQE_user_login - v1" duration=476.008µs 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.319375021Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.319818268Z level=info msg="Migration successfully executed" id="drop index UQE_user_email - v1" duration=443.037µs 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.32016969Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.321756039Z level=info msg="Migration successfully executed" id="Rename table user to user_v1 - v1" duration=1.586374ms 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.322143524Z level=info msg="Executing migration" id="create user table v2" 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.322681608Z level=info msg="Migration successfully executed" id="create user table v2" duration=537.945µs 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.323059876Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.323513473Z level=info msg="Migration successfully executed" id="create index UQE_user_login - v2" duration=453.427µs 2024-09-21T11:41:09.537 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.323835428Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.324269525Z level=info msg="Migration successfully executed" id="create index UQE_user_email - v2" duration=433.882µs 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.324643426Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.324914871Z level=info msg="Migration successfully executed" id="copy data_source v1 to v2" duration=271.675µs 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.325260559Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.325650128Z level=info msg="Migration successfully executed" id="Drop old table user_v1" duration=389.367µs 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.32595555Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.326698579Z level=info msg="Migration successfully executed" id="Add column help_flags1 to user table" duration=742.814µs 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.327068752Z level=info msg="Executing migration" id="Update user table charset" 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.327084557Z level=info msg="Migration successfully executed" id="Update user table charset" duration=16.415µs 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.327517021Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.328255163Z level=info msg="Migration successfully executed" id="Add last_seen_at column to user" duration=739.278µs 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.328623086Z level=info msg="Executing migration" id="Add missing user data" 2024-09-21T11:41:09.538 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.328784737Z level=info msg="Migration successfully executed" id="Add missing user data" duration=161.542µs 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.329226416Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.329955208Z level=info msg="Migration successfully executed" id="Add is_disabled column to user" duration=728.59µs 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.330320193Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.330791028Z level=info msg="Migration successfully executed" id="Add index user.login/user.email" duration=470.572µs 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.331172958Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.331918638Z level=info msg="Migration successfully executed" id="Add is_service_account column to user" duration=743.131µs 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.332253546Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.337494349Z level=info msg="Migration successfully executed" id="Update is_service_account column to nullable" duration=5.239161ms 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.337940918Z level=info msg="Executing migration" id="Add uid column to user" 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.338717675Z level=info msg="Migration successfully executed" id="Add uid column to user" duration=789.171µs 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.33914988Z level=info msg="Executing migration" id="Update uid column values for users" 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.339312467Z level=info msg="Migration successfully executed" id="Update uid column values for users" duration=162.568µs 2024-09-21T11:41:09.539 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.339716037Z level=info msg="Executing migration" id="Add unique index user_uid" 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.340217314Z level=info msg="Migration successfully executed" id="Add unique index user_uid" duration=501.124µs 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.34064906Z level=info msg="Executing migration" id="update login field with orgid to allow for multiple service accounts with same name across orgs" 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.340848528Z level=info msg="Migration successfully executed" id="update login field with orgid to allow for multiple service accounts with same name across orgs" duration=199.491µs 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.34128708Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.341835191Z level=info msg="Migration successfully executed" id="create temp user table v1-7" duration=548.455µs 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.342251636Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.342702231Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v1-7" duration=450.328µs 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.343052443Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.343536755Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v1-7" duration=483.974µs 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.343948638Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.344390557Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v1-7" duration=441.946µs 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.344727431Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-21T11:41:09.540 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.345173514Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v1-7" duration=445.512µs 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.345498121Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.345512816Z level=info msg="Migration successfully executed" id="Update temp_user table charset" duration=15.203µs 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.345904134Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.346357707Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_email - v1" duration=453.258µs 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.346681162Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.347127851Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_org_id - v1" duration=446.473µs 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.349334364Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.349774338Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_code - v1" duration=439.848µs 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.352044383Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.35254705Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_status - v1" duration=502.42µs 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.354859545Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.356814652Z level=info msg="Migration successfully executed" id="Rename table temp_user to temp_user_tmp_qwerty - v1" duration=1.954706ms 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.357201456Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.357716365Z level=info msg="Migration successfully executed" id="create temp_user v2" duration=514.573µs 2024-09-21T11:41:09.541 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.360009412Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.36045444Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v2" duration=444.746µs 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.362738253Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.363216768Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v2" duration=478.282µs 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.365465325Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.365962686Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v2" duration=497.143µs 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.366296967Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.36675255Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v2" duration=457.033µs 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.369083193Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.369380422Z level=info msg="Migration successfully executed" id="copy temp_user v1 to v2" duration=298.131µs 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.37176038Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.372185841Z level=info msg="Migration successfully executed" id="drop temp_user_tmp_qwerty" duration=425.37µs 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.374590621Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.374951889Z level=info msg="Migration successfully executed" id="Set created for temp users that will otherwise prematurely expire" duration=361.274µs 2024-09-21T11:41:09.542 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.377431915Z level=info msg="Executing migration" id="create star table" 2024-09-21T11:41:09.544 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.377925868Z level=info msg="Migration successfully executed" id="create star table" duration=497.022µs 2024-09-21T11:41:09.544 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.380010341Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-21T11:41:09.544 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.380495669Z level=info msg="Migration successfully executed" id="add unique index star.user_id_dashboard_id" duration=485.355µs 2024-09-21T11:41:09.544 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.382725886Z level=info msg="Executing migration" id="create org table v1" 2024-09-21T11:41:09.544 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.383227063Z level=info msg="Migration successfully executed" id="create org table v1" duration=500.931µs 2024-09-21T11:41:09.544 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.38540557Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-21T11:41:09.544 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.385879018Z level=info msg="Migration successfully executed" id="create index UQE_org_name - v1" duration=473.201µs 2024-09-21T11:41:09.544 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.388194376Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-21T11:41:09.544 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.388661361Z level=info msg="Migration successfully executed" id="create org_user table v1" duration=466.851µs 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.390886958Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.391379986Z level=info msg="Migration successfully executed" id="create index IDX_org_user_org_id - v1" duration=514.25µs 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.393340321Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.393831133Z level=info msg="Migration successfully executed" id="create index UQE_org_user_org_id_user_id - v1" duration=490.443µs 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.39598802Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.396460962Z level=info msg="Migration successfully executed" id="create index IDX_org_user_user_id - v1" duration=472.709µs 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.398569787Z level=info msg="Executing migration" id="Update org table charset" 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.398608026Z level=info msg="Migration successfully executed" id="Update org table charset" duration=38.234µs 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.40126152Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.40129871Z level=info msg="Migration successfully executed" id="Update org_user table charset" duration=34.9µs 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.403916187Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.404056975Z level=info msg="Migration successfully executed" id="Migrate all Read Only Viewers to Viewers" duration=140.452µs 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.404441854Z level=info msg="Executing migration" id="create dashboard table" 2024-09-21T11:41:09.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.404956059Z level=info msg="Migration successfully executed" id="create dashboard table" duration=513.394µs 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.407122538Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.40764523Z level=info msg="Migration successfully executed" id="add index dashboard.account_id" duration=522.562µs 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.409756241Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.410304548Z level=info msg="Migration successfully executed" id="add unique index dashboard_account_id_slug" duration=548.995µs 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.412272729Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.412720081Z level=info msg="Migration successfully executed" id="create dashboard_tag table" duration=447.096µs 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.413142486Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.413688497Z level=info msg="Migration successfully executed" id="add unique index dashboard_tag.dasboard_id_term" duration=546.182µs 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.415760598Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.416240052Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" duration=480.247µs 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.418386248Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.421808059Z level=info msg="Migration successfully executed" id="Rename table dashboard to dashboard_v1 - v1" duration=3.420952ms 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.422329921Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-21T11:41:09.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.422844851Z level=info msg="Migration successfully executed" id="create dashboard v2" duration=514.891µs 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.423199982Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.423655989Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_org_id - v2" duration=455.846µs 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.42401957Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.424505659Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_org_id_slug - v2" duration=485.862µs 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.424852394Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.42509332Z level=info msg="Migration successfully executed" id="copy dashboard v1 to v2" duration=240.674µs 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.425677706Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.426279608Z level=info msg="Migration successfully executed" id="drop table dashboard_v1" duration=601.644µs 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.426644522Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.426681917Z level=info msg="Migration successfully executed" id="alter dashboard.data to mediumtext v1" duration=37.804µs 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.427431373Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.42852904Z level=info msg="Migration successfully executed" id="Add column updated_by in dashboard - v2" duration=1.097383ms 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.428887154Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-21T11:41:09.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.429971569Z level=info msg="Migration successfully executed" id="Add column created_by in dashboard - v2" duration=1.083479ms 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.430336665Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.4314529Z level=info msg="Migration successfully executed" id="Add column gnetId in dashboard" duration=1.114208ms 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.431900208Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.432435831Z level=info msg="Migration successfully executed" id="Add index for gnetId in dashboard" duration=535.545µs 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.433305652Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.43515306Z level=info msg="Migration successfully executed" id="Add column plugin_id in dashboard" duration=1.843803ms 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.435734185Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.4365329Z level=info msg="Migration successfully executed" id="Add index for plugin_id in dashboard" duration=798.329µs 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.437017642Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.437899902Z level=info msg="Migration successfully executed" id="Add index for dashboard_id in dashboard_tag" duration=879.042µs 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.438633948Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.438659587Z level=info msg="Migration successfully executed" id="Update dashboard table charset" duration=26.712µs 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.439505007Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-21T11:41:09.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.439532636Z level=info msg="Migration successfully executed" id="Update dashboard_tag table charset" duration=28.779µs 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.440191868Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.44150126Z level=info msg="Migration successfully executed" id="Add column folder_id in dashboard" duration=1.309567ms 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.441858941Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.443112977Z level=info msg="Migration successfully executed" id="Add column isFolder in dashboard" duration=1.254085ms 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.443475537Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.444665117Z level=info msg="Migration successfully executed" id="Add column has_acl in dashboard" duration=1.188771ms 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.445086433Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.446324179Z level=info msg="Migration successfully executed" id="Add column uid in dashboard" duration=1.235522ms 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.446777092Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.446946979Z level=info msg="Migration successfully executed" id="Update uid column values in dashboard" duration=169.868µs 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.447605022Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.448145271Z level=info msg="Migration successfully executed" id="Add unique index dashboard_org_id_uid" duration=539.795µs 2024-09-21T11:41:09.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.448508146Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.449005336Z level=info msg="Migration successfully executed" id="Remove unique index org_id_slug" duration=496.924µs 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.449377086Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.449391619Z level=info msg="Migration successfully executed" id="Update dashboard title length" duration=15.358µs 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.45020374Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.450726822Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_title_folder_id" duration=523.267µs 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.451105208Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.451562088Z level=info msg="Migration successfully executed" id="create dashboard_provisioning" duration=455.442µs 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.451912777Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.455006383Z level=info msg="Migration successfully executed" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" duration=3.092683ms 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.455375804Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.455819188Z level=info msg="Migration successfully executed" id="create dashboard_provisioning v2" duration=441.804µs 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.456197423Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-21T11:41:09.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.456673823Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id - v2" duration=476.282µs 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.457057811Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.457528043Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" duration=469.996µs 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.457860048Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.458070006Z level=info msg="Migration successfully executed" id="copy dashboard_provisioning v1 to v2" duration=209.791µs 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.458696485Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.459053936Z level=info msg="Migration successfully executed" id="drop dashboard_provisioning_tmp_qwerty" duration=357.141µs 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.459471916Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.460672126Z level=info msg="Migration successfully executed" id="Add check_sum column" duration=1.199862ms 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.461042516Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.461553732Z level=info msg="Migration successfully executed" id="Add index for dashboard_title" duration=511.426µs 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.461916299Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-21T11:41:09.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.462047277Z level=info msg="Migration successfully executed" id="delete tags for deleted dashboards" duration=129.279µs 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.462741157Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.462856693Z level=info msg="Migration successfully executed" id="delete stars for deleted dashboards" duration=115.585µs 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.463594513Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.464095552Z level=info msg="Migration successfully executed" id="Add index for dashboard_is_folder" duration=500.77µs 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.466304944Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.467612883Z level=info msg="Migration successfully executed" id="Add isPublic for dashboard" duration=1.307654ms 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.46799713Z level=info msg="Executing migration" id="create data_source table" 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.468609451Z level=info msg="Migration successfully executed" id="create data_source table" duration=612.2µs 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.470672147Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.471200388Z level=info msg="Migration successfully executed" id="add index data_source.account_id" duration=528.087µs 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.473633561Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.474147946Z level=info msg="Migration successfully executed" id="add unique index data_source.account_id_name" duration=514.045µs 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.476472838Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-21T11:41:09.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.476971485Z level=info msg="Migration successfully executed" id="drop index IDX_data_source_account_id - v1" duration=498.523µs 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.477316976Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.47779895Z level=info msg="Migration successfully executed" id="drop index UQE_data_source_account_id_name - v1" duration=481.739µs 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.480024592Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.48350528Z level=info msg="Migration successfully executed" id="Rename table data_source to data_source_v1 - v1" duration=3.479648ms 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.483961883Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.484578568Z level=info msg="Migration successfully executed" id="create data_source table v2" duration=616.49µs 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.486707224Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.487294789Z level=info msg="Migration successfully executed" id="create index IDX_data_source_org_id - v2" duration=587.341µs 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.487658467Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.488217365Z level=info msg="Migration successfully executed" id="create index UQE_data_source_org_id_name - v2" duration=558.72µs 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.490506173Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.490970397Z level=info msg="Migration successfully executed" id="Drop old table data_source_v1 #2" duration=465.403µs 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.493113868Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-21T11:41:09.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.49467031Z level=info msg="Migration successfully executed" id="Add column with_credentials" duration=1.553544ms 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.495844306Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.497341691Z level=info msg="Migration successfully executed" id="Add secure json data column" duration=1.497632ms 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.497757808Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.497778748Z level=info msg="Migration successfully executed" id="Update data_source table charset" duration=21.748µs 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.500417609Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.500588593Z level=info msg="Migration successfully executed" id="Update initial version to 1" duration=171.889µs 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.503246986Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.50491262Z level=info msg="Migration successfully executed" id="Add read_only data column" duration=1.666279ms 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.505905733Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.506067997Z level=info msg="Migration successfully executed" id="Migrate logging ds to loki ds" duration=162.121µs 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.506466616Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.506590556Z level=info msg="Migration successfully executed" id="Update json_data with nulls" duration=122.739µs 2024-09-21T11:41:09.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.509178462Z level=info msg="Executing migration" id="Add uid column" 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.51096414Z level=info msg="Migration successfully executed" id="Add uid column" duration=1.78605ms 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.511406003Z level=info msg="Executing migration" id="Update uid value" 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.511564455Z level=info msg="Migration successfully executed" id="Update uid value" duration=158.435µs 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.51198951Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.512711028Z level=info msg="Migration successfully executed" id="Add unique index datasource_org_id_uid" duration=721.142µs 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.514611399Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.515135579Z level=info msg="Migration successfully executed" id="add unique index datasource_org_id_is_default" duration=524.017µs 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.515514612Z level=info msg="Executing migration" id="create api_key table" 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.516063215Z level=info msg="Migration successfully executed" id="create api_key table" duration=548.181µs 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.518171104Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.518669816Z level=info msg="Migration successfully executed" id="add index api_key.account_id" duration=498.58µs 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.520781576Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.521294571Z level=info msg="Migration successfully executed" id="add index api_key.key" duration=512.841µs 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.523451152Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-21T11:41:09.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.524007989Z level=info msg="Migration successfully executed" id="add index api_key.account_id_name" duration=556.579µs 2024-09-21T11:41:09.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.524390944Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-21T11:41:09.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.524886448Z level=info msg="Migration successfully executed" id="drop index IDX_api_key_account_id - v1" duration=495.363µs 2024-09-21T11:41:09.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.527074683Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-21T11:41:09.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.527560079Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_key - v1" duration=485.187µs 2024-09-21T11:41:09.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.529669957Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-21T11:41:09.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.530179629Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_account_id_name - v1" duration=509.72µs 2024-09-21T11:41:09.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.532333095Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-21T11:41:09.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.536519421Z level=info msg="Migration successfully executed" id="Rename table api_key to api_key_v1 - v1" duration=4.184415ms 2024-09-21T11:41:09.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.536981661Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-21T11:41:09.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.537571487Z level=info msg="Migration successfully executed" id="create api_key table v2" duration=590.393µs 2024-09-21T11:41:09.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:09.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:09.633 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:09.633 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:09.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:09.755 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:09.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:09.756 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:09.784 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.539514869Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.540052012Z level=info msg="Migration successfully executed" id="create index IDX_api_key_org_id - v2" duration=537.149µs 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.542247282Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.542749984Z level=info msg="Migration successfully executed" id="create index UQE_api_key_key - v2" duration=502.452µs 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.544958881Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.545476811Z level=info msg="Migration successfully executed" id="create index UQE_api_key_org_id_name - v2" duration=517.472µs 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.547664641Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.547916672Z level=info msg="Migration successfully executed" id="copy api_key v1 to v2" duration=251.868µs 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.548468855Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.548867021Z level=info msg="Migration successfully executed" id="Drop old table api_key_v1" duration=395.878µs 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.549267783Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.549286363Z level=info msg="Migration successfully executed" id="Update api_key table charset" duration=18.486µs 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.550077691Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-21T11:41:09.785 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.551578994Z level=info msg="Migration successfully executed" id="Add expires to api_key table" duration=1.501165ms 2024-09-21T11:41:09.786 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.551948656Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-21T11:41:09.786 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.553451052Z level=info msg="Migration successfully executed" id="Add service account foreign key" duration=1.501969ms 2024-09-21T11:41:09.786 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.553797909Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-21T11:41:09.786 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.553910856Z level=info msg="Migration successfully executed" id="set service account foreign key to nil if 0" duration=112.892µs 2024-09-21T11:41:09.786 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.554624273Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.55632004Z level=info msg="Migration successfully executed" id="Add last_used_at to api_key table" duration=1.695481ms 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.556685445Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.558195421Z level=info msg="Migration successfully executed" id="Add is_revoked column to api_key table" duration=1.509643ms 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.5585545Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.559097704Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v4" duration=542.918µs 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.559446456Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.559820887Z level=info msg="Migration successfully executed" id="drop table dashboard_snapshot_v4 #1" duration=374.8µs 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.560279341Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.560841857Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v5 #2" duration=562.785µs 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.561224821Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-21T11:41:09.787 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.561767861Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_key - v5" duration=542.771µs 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.562124464Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.56268861Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_delete_key - v5" duration=563.827µs 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.563048292Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.563547034Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_snapshot_user_id - v5" duration=498.541µs 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.563893467Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.563950318Z level=info msg="Migration successfully executed" id="alter dashboard_snapshot to mediumtext v2" duration=58.125µs 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.564662531Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.564681134Z level=info msg="Migration successfully executed" id="Update dashboard_snapshot table charset" duration=18.569µs 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.56544978Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.567134639Z level=info msg="Migration successfully executed" id="Add column external_delete_url to dashboard_snapshots table" duration=1.684352ms 2024-09-21T11:41:09.788 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.567495949Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.569145875Z level=info msg="Migration successfully executed" id="Add encrypted dashboard json column" duration=1.648179ms 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.569521468Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.569569925Z level=info msg="Migration successfully executed" id="Change dashboard_encrypted column to MEDIUMBLOB" duration=50.424µs 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.57040141Z level=info msg="Executing migration" id="create quota table v1" 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.570879963Z level=info msg="Migration successfully executed" id="create quota table v1" duration=478.344µs 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.571258069Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.57175825Z level=info msg="Migration successfully executed" id="create index UQE_quota_org_id_user_id_target - v1" duration=500.06µs 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.572111187Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.572130604Z level=info msg="Migration successfully executed" id="Update quota table charset" duration=19.417µs 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.572999783Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.573527277Z level=info msg="Migration successfully executed" id="create plugin_setting table" duration=527.179µs 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.573871865Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-21T11:41:09.789 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.574396701Z level=info msg="Migration successfully executed" id="create index UQE_plugin_setting_org_id_plugin_id - v1" duration=524.725µs 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.574737646Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.576493801Z level=info msg="Migration successfully executed" id="Add column plugin_version to plugin_settings" duration=1.756084ms 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.576854379Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.576874066Z level=info msg="Migration successfully executed" id="Update plugin_setting table charset" duration=19.62µs 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.577710617Z level=info msg="Executing migration" id="create session table" 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.578306501Z level=info msg="Migration successfully executed" id="create session table" duration=595.739µs 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.578689577Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.578760258Z level=info msg="Migration successfully executed" id="Drop old table playlist table" duration=70.731µs 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.581436015Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.581523839Z level=info msg="Migration successfully executed" id="Drop old table playlist_item table" duration=88.118µs 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.581946843Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.582436218Z level=info msg="Migration successfully executed" id="create playlist table v2" duration=489.07µs 2024-09-21T11:41:09.790 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.582813722Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.583320855Z level=info msg="Migration successfully executed" id="create playlist item table v2" duration=505.954µs 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.583684468Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.58370304Z level=info msg="Migration successfully executed" id="Update playlist table charset" duration=19.514µs 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.584119248Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.584138553Z level=info msg="Migration successfully executed" id="Update playlist_item table charset" duration=19.273µs 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.58454606Z level=info msg="Executing migration" id="Add playlist column created_at" 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.586442444Z level=info msg="Migration successfully executed" id="Add playlist column created_at" duration=1.896018ms 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.586804551Z level=info msg="Executing migration" id="Add playlist column updated_at" 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.588609482Z level=info msg="Migration successfully executed" id="Add playlist column updated_at" duration=1.80284ms 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.588978459Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.589049597Z level=info msg="Migration successfully executed" id="drop preferences table v2" duration=72.464µs 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.589424092Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.589489541Z level=info msg="Migration successfully executed" id="drop preferences table v3" duration=65.436µs 2024-09-21T11:41:09.791 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.589842929Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-21T11:41:09.792 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.590373147Z level=info msg="Migration successfully executed" id="create preferences table v3" duration=530.508µs 2024-09-21T11:41:09.792 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.59071634Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-21T11:41:09.792 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.590736036Z level=info msg="Migration successfully executed" id="Update preferences table charset" duration=18.55µs 2024-09-21T11:41:09.792 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.591188394Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-21T11:41:09.792 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.593034518Z level=info msg="Migration successfully executed" id="Add column team_id in preferences" duration=1.845856ms 2024-09-21T11:41:09.792 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.593382437Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-21T11:41:09.792 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.593487323Z level=info msg="Migration successfully executed" id="Update team_id column values in preferences" duration=104.866µs 2024-09-21T11:41:09.792 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.593870288Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-21T11:41:09.793 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.595733529Z level=info msg="Migration successfully executed" id="Add column week_start in preferences" duration=1.861684ms 2024-09-21T11:41:09.793 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.596090006Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.597882437Z level=info msg="Migration successfully executed" id="Add column preferences.json_data" duration=1.792567ms 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.598244578Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.598292703Z level=info msg="Migration successfully executed" id="alter preferences.json_data to mediumtext v1" duration=48.576µs 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.598771127Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.599433465Z level=info msg="Migration successfully executed" id="Add preferences index org_id" duration=662.374µs 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.599813615Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.600366749Z level=info msg="Migration successfully executed" id="Add preferences index user_id" duration=553.057µs 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.602626796Z level=info msg="Executing migration" id="create alert table v1" 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.603401415Z level=info msg="Migration successfully executed" id="create alert table v1" duration=774.498µs 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.603797296Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.604428534Z level=info msg="Migration successfully executed" id="add index alert org_id & id " duration=630.993µs 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.60652468Z level=info msg="Executing migration" id="add index alert state" 2024-09-21T11:41:09.794 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.607068781Z level=info msg="Migration successfully executed" id="add index alert state" duration=543.848µs 2024-09-21T11:41:09.795 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.60931518Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-21T11:41:09.795 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.609884051Z level=info msg="Migration successfully executed" id="add index alert dashboard_id" duration=568.657µs 2024-09-21T11:41:09.795 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.61025265Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-21T11:41:09.795 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.61072809Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v1" duration=475.339µs 2024-09-21T11:41:09.795 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.611118758Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-21T11:41:09.795 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.611704304Z level=info msg="Migration successfully executed" id="Add unique index alert_rule_tag.alert_id_tag_id" duration=587.263µs 2024-09-21T11:41:09.795 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.612076908Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-21T11:41:09.795 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.612628418Z level=info msg="Migration successfully executed" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" duration=550.145µs 2024-09-21T11:41:09.795 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.613002441Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-21T11:41:09.796 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.61869854Z level=info msg="Migration successfully executed" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" duration=5.695714ms 2024-09-21T11:41:09.796 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.619091773Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.619633774Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v2" duration=541.707µs 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.620000698Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.620553224Z level=info msg="Migration successfully executed" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" duration=552.426µs 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.620925657Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.621144522Z level=info msg="Migration successfully executed" id="copy alert_rule_tag v1 to v2" duration=218.588µs 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.62147818Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.621850172Z level=info msg="Migration successfully executed" id="drop table alert_rule_tag_v1" duration=371.74µs 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.62221614Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.62271873Z level=info msg="Migration successfully executed" id="create alert_notification table v1" duration=502.284µs 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.623067677Z level=info msg="Executing migration" id="Add column is_default" 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.625136836Z level=info msg="Migration successfully executed" id="Add column is_default" duration=2.068872ms 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.625491135Z level=info msg="Executing migration" id="Add column frequency" 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.627714271Z level=info msg="Migration successfully executed" id="Add column frequency" duration=2.221058ms 2024-09-21T11:41:09.797 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.6280921Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.630427335Z level=info msg="Migration successfully executed" id="Add column send_reminder" duration=2.333681ms 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.630801409Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.632901541Z level=info msg="Migration successfully executed" id="Add column disable_resolve_message" duration=2.099989ms 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.633303841Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.633854026Z level=info msg="Migration successfully executed" id="add index alert_notification org_id & name" duration=551.615µs 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.634307108Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.634322369Z level=info msg="Migration successfully executed" id="Update alert table charset" duration=15.791µs 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.634758512Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.63477327Z level=info msg="Migration successfully executed" id="Update alert_notification table charset" duration=15.246µs 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.635174218Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.635651706Z level=info msg="Migration successfully executed" id="create notification_journal table v1" duration=477.76µs 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.636006781Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-21T11:41:09.798 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.636539923Z level=info msg="Migration successfully executed" id="add index notification_journal org_id & alert_id & notifier_id" duration=533.405µs 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.63689669Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.6374601Z level=info msg="Migration successfully executed" id="drop alert_notification_journal" duration=563.439µs 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.63966525Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.64019936Z level=info msg="Migration successfully executed" id="create alert_notification_state table v1" duration=533.938µs 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.642296717Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.642835661Z level=info msg="Migration successfully executed" id="add index alert_notification_state org_id & alert_id & notifier_id" duration=538.774µs 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.645074572Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.647204223Z level=info msg="Migration successfully executed" id="Add for to alert table" duration=2.129238ms 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.647781639Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.650018602Z level=info msg="Migration successfully executed" id="Add column uid in alert_notification" duration=2.236492ms 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.650639495Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-21T11:41:09.799 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.650780267Z level=info msg="Migration successfully executed" id="Update uid column values in alert_notification" duration=139.556µs 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.65327942Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.653830925Z level=info msg="Migration successfully executed" id="Add unique index alert_notification_org_id_uid" duration=551.468µs 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.656082532Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.656631666Z level=info msg="Migration successfully executed" id="Remove unique index org_id_name" duration=549.033µs 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.658843573Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.661100329Z level=info msg="Migration successfully executed" id="Add column secure_settings in alert_notification" duration=2.256459ms 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.661487805Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.661535726Z level=info msg="Migration successfully executed" id="alter alert.settings to mediumtext" duration=48.131µs 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.664133447Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.664650982Z level=info msg="Migration successfully executed" id="Add non-unique index alert_notification_state_alert_id" duration=517.347µs 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.666790356Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.667399086Z level=info msg="Migration successfully executed" id="Add non-unique index alert_rule_tag_alert_id" duration=609.81µs 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.66943738Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-21T11:41:09.800 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.669526496Z level=info msg="Migration successfully executed" id="Drop old annotation table v4" duration=89.188µs 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.672195849Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.672778881Z level=info msg="Migration successfully executed" id="create annotation table v5" duration=582.767µs 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.674868198Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.67542009Z level=info msg="Migration successfully executed" id="add index annotation 0 v3" duration=550.269µs 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.677549774Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.678106271Z level=info msg="Migration successfully executed" id="add index annotation 1 v3" duration=556.204µs 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.680308134Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.68086538Z level=info msg="Migration successfully executed" id="add index annotation 2 v3" duration=556.978µs 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.683025247Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.683627568Z level=info msg="Migration successfully executed" id="add index annotation 3 v3" duration=601.907µs 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.685770619Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.686392391Z level=info msg="Migration successfully executed" id="add index annotation 4 v3" duration=621.341µs 2024-09-21T11:41:09.801 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.68842576Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.688444548Z level=info msg="Migration successfully executed" id="Update annotation table charset" duration=19.358µs 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.69112999Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.69366116Z level=info msg="Migration successfully executed" id="Add column region_id to annotation table" duration=2.530759ms 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.694048632Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.694597167Z level=info msg="Migration successfully executed" id="Drop category_id index" duration=548.757µs 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.694957204Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.697222485Z level=info msg="Migration successfully executed" id="Add column tags to annotation table" duration=2.264983ms 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.697596944Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.698055616Z level=info msg="Migration successfully executed" id="Create annotation_tag table v2" duration=458.514µs 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.70040242Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.701021059Z level=info msg="Migration successfully executed" id="Add unique index annotation_tag.annotation_id_tag_id" duration=618.396µs 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.703107538Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.703638694Z level=info msg="Migration successfully executed" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" duration=530.93µs 2024-09-21T11:41:09.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.704002185Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.710319346Z level=info msg="Migration successfully executed" id="Rename table annotation_tag to annotation_tag_v2 - v2" duration=6.316389ms 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.710709902Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.711194979Z level=info msg="Migration successfully executed" id="Create annotation_tag table v3" duration=480.08µs 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.711557018Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.712113478Z level=info msg="Migration successfully executed" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" duration=556.182µs 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.712473556Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.712672286Z level=info msg="Migration successfully executed" id="copy annotation_tag v2 to v3" duration=198.477µs 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.71323743Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.713606589Z level=info msg="Migration successfully executed" id="drop table annotation_tag_v2" duration=368.563µs 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.714086731Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.714206688Z level=info msg="Migration successfully executed" id="Update alert annotations and set TEXT to empty" duration=119.961µs 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.714902756Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.717206573Z level=info msg="Migration successfully executed" id="Add created time to annotation table" duration=2.303393ms 2024-09-21T11:41:09.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.717571946Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.719862866Z level=info msg="Migration successfully executed" id="Add updated time to annotation table" duration=2.290606ms 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.72023754Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.720769201Z level=info msg="Migration successfully executed" id="Add index for created in annotation table" duration=531.286µs 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.721128455Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.721670479Z level=info msg="Migration successfully executed" id="Add index for updated in annotation table" duration=542.838µs 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.722047228Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.722195916Z level=info msg="Migration successfully executed" id="Convert existing annotations from seconds to milliseconds" duration=149.573µs 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.722873998Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.7252493Z level=info msg="Migration successfully executed" id="Add epoch_end column" duration=2.374805ms 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.725601451Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.726151241Z level=info msg="Migration successfully executed" id="Add index for epoch_end" duration=549.389µs 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.726537816Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.72665194Z level=info msg="Migration successfully executed" id="Make epoch_end the same as epoch" duration=113.857µs 2024-09-21T11:41:09.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.727319306Z level=info msg="Executing migration" id="Move region to single row" 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.7275549Z level=info msg="Migration successfully executed" id="Move region to single row" duration=235.507µs 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.728142852Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.728674856Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch from annotation table" duration=531.818µs 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.72905991Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.729654001Z level=info msg="Migration successfully executed" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" duration=594.814µs 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.730024101Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.730647057Z level=info msg="Migration successfully executed" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" duration=622.224µs 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.730995686Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.731541571Z level=info msg="Migration successfully executed" id="Add index for org_id_epoch_end_epoch on annotation table" duration=545.289µs 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.731875192Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.732428668Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch_epoch_end from annotation table" duration=553.755µs 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.732769489Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-21T11:41:09.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.733312127Z level=info msg="Migration successfully executed" id="Add index for alert_id on annotation table" duration=541.315µs 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.733647692Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.733691121Z level=info msg="Migration successfully executed" id="Increase tags column to length 4096" duration=44.307µs 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.73412873Z level=info msg="Executing migration" id="create test_data table" 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.734663825Z level=info msg="Migration successfully executed" id="create test_data table" duration=535.263µs 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.735022861Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.735545488Z level=info msg="Migration successfully executed" id="create dashboard_version table v1" duration=522.854µs 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.735889215Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.736430482Z level=info msg="Migration successfully executed" id="add index dashboard_version.dashboard_id" duration=541.65µs 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.73677714Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.737346575Z level=info msg="Migration successfully executed" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" duration=569.606µs 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.737686538Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-21T11:41:09.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.737808618Z level=info msg="Migration successfully executed" id="Set dashboard version to 1 where 0" duration=122.469µs 2024-09-21T11:41:09.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.738546997Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-21T11:41:09.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.738785555Z level=info msg="Migration successfully executed" id="save existing dashboard data in dashboard_version table v1" duration=238.82µs 2024-09-21T11:41:09.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.739400541Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-21T11:41:09.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.739440793Z level=info msg="Migration successfully executed" id="alter dashboard_version.data to mediumtext v1" duration=41.019µs 2024-09-21T11:41:09.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.740240894Z level=info msg="Executing migration" id="create team table" 2024-09-21T11:41:09.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.740721687Z level=info msg="Migration successfully executed" id="create team table" duration=481.075µs 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.741097936Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.741735147Z level=info msg="Migration successfully executed" id="add index team.org_id" duration=637.434µs 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.742092006Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.742638875Z level=info msg="Migration successfully executed" id="add unique index team_org_id_name" duration=547.101µs 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.742993494Z level=info msg="Executing migration" id="Add column uid in team" 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.745561115Z level=info msg="Migration successfully executed" id="Add column uid in team" duration=2.567539ms 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.745917099Z level=info msg="Executing migration" id="Update uid column values in team" 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.746043133Z level=info msg="Migration successfully executed" id="Update uid column values in team" duration=125.784µs 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.746765095Z level=info msg="Executing migration" id="Add unique index team_org_id_uid" 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.7473589Z level=info msg="Migration successfully executed" id="Add unique index team_org_id_uid" duration=593.541µs 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.747747991Z level=info msg="Executing migration" id="create team member table" 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.74828007Z level=info msg="Migration successfully executed" id="create team member table" duration=531.868µs 2024-09-21T11:41:09.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.750465142Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.751045075Z level=info msg="Migration successfully executed" id="add index team_member.org_id" duration=581.018µs 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.753372861Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.753981746Z level=info msg="Migration successfully executed" id="add unique index team_member_org_id_team_id_user_id" duration=608.736µs 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.756106435Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.756671303Z level=info msg="Migration successfully executed" id="add index team_member.team_id" duration=563.296µs 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.758818Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.761718112Z level=info msg="Migration successfully executed" id="Add column email to team table" duration=2.895308ms 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.762120359Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.764977453Z level=info msg="Migration successfully executed" id="Add column external to team_member table" duration=2.856805ms 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.765344094Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.768016554Z level=info msg="Migration successfully executed" id="Add column permission to team_member table" duration=2.672136ms 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.768388724Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-21T11:41:09.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.769067038Z level=info msg="Migration successfully executed" id="create dashboard acl table" duration=677.949µs 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.771096529Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.771689584Z level=info msg="Migration successfully executed" id="add index dashboard_acl_dashboard_id" duration=592.833µs 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.772090572Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.772740143Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_user_id" duration=649.455µs 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.774702652Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.775309663Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_team_id" duration=606.817µs 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.777419266Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.778000394Z level=info msg="Migration successfully executed" id="add index dashboard_acl_user_id" duration=581.799µs 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.78023917Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.780811404Z level=info msg="Migration successfully executed" id="add index dashboard_acl_team_id" duration=572.234µs 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.782962493Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.783612394Z level=info msg="Migration successfully executed" id="add index dashboard_acl_org_id_role" duration=649.632µs 2024-09-21T11:41:09.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.785571818Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-21T11:41:09.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.786199376Z level=info msg="Migration successfully executed" id="add index dashboard_permission" duration=627.617µs 2024-09-21T11:41:09.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.78657418Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-21T11:41:09.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.786938542Z level=info msg="Migration successfully executed" id="save default acl rules in dashboard_acl table" duration=364.525µs 2024-09-21T11:41:09.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.789270985Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-21T11:41:09.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.789436956Z level=info msg="Migration successfully executed" id="delete acl rules for deleted dashboards and folders" duration=164.757µs 2024-09-21T11:41:09.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.792024407Z level=info msg="Executing migration" id="create tag table" 2024-09-21T11:41:09.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.792565872Z level=info msg="Migration successfully executed" id="create tag table" duration=542.049µs 2024-09-21T11:41:09.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:09.880 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:09.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:09.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:09.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:10.003 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:10.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:10.004 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:10.035 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.794692129Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-21T11:41:10.035 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.795284852Z level=info msg="Migration successfully executed" id="add index tag.key_value" duration=590.487µs 2024-09-21T11:41:10.035 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.795655634Z level=info msg="Executing migration" id="create login attempt table" 2024-09-21T11:41:10.035 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.796149384Z level=info msg="Migration successfully executed" id="create login attempt table" duration=494.271µs 2024-09-21T11:41:10.035 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.798428818Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-21T11:41:10.035 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.799009141Z level=info msg="Migration successfully executed" id="add index login_attempt.username" duration=580.028µs 2024-09-21T11:41:10.035 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.80107653Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-21T11:41:10.035 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.801645806Z level=info msg="Migration successfully executed" id="drop index IDX_login_attempt_username - v1" duration=569.188µs 2024-09-21T11:41:10.035 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.803703165Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-21T11:41:10.035 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.811785616Z level=info msg="Migration successfully executed" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" duration=8.08205ms 2024-09-21T11:41:10.036 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.812194806Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-21T11:41:10.036 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.812688177Z level=info msg="Migration successfully executed" id="create login_attempt v2" duration=494.509µs 2024-09-21T11:41:10.036 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.814786158Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-21T11:41:10.036 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.815391889Z level=info msg="Migration successfully executed" id="create index IDX_login_attempt_username - v2" duration=605.611µs 2024-09-21T11:41:10.036 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.817528531Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-21T11:41:10.036 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.817764754Z level=info msg="Migration successfully executed" id="copy login_attempt v1 to v2" duration=236.922µs 2024-09-21T11:41:10.036 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.820199069Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-21T11:41:10.036 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.820631695Z level=info msg="Migration successfully executed" id="drop login_attempt_tmp_qwerty" duration=432.287µs 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.821014554Z level=info msg="Executing migration" id="create user auth table" 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.82151409Z level=info msg="Migration successfully executed" id="create user auth table" duration=499.594µs 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.821890136Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.822475057Z level=info msg="Migration successfully executed" id="create index IDX_user_auth_auth_module_auth_id - v1" duration=584.935µs 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.82285105Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.822889103Z level=info msg="Migration successfully executed" id="alter user_auth.auth_id to length 190" duration=38.36µs 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.823718951Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.826697526Z level=info msg="Migration successfully executed" id="Add OAuth access token to user_auth" duration=2.978602ms 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.82705156Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.829916384Z level=info msg="Migration successfully executed" id="Add OAuth refresh token to user_auth" duration=2.864579ms 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.830281929Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.833137958Z level=info msg="Migration successfully executed" id="Add OAuth token type to user_auth" duration=2.855696ms 2024-09-21T11:41:10.037 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.833487525Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.836456983Z level=info msg="Migration successfully executed" id="Add OAuth expiry to user_auth" duration=2.969218ms 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.836816524Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.837396098Z level=info msg="Migration successfully executed" id="Add index to user_id column in user_auth" duration=579.534µs 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.837844882Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.840799173Z level=info msg="Migration successfully executed" id="Add OAuth ID token to user_auth" duration=2.954964ms 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.841168506Z level=info msg="Executing migration" id="create server_lock table" 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.841682603Z level=info msg="Migration successfully executed" id="create server_lock table" duration=513.664µs 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.842058807Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.842617958Z level=info msg="Migration successfully executed" id="add index server_lock.operation_uid" duration=558.957µs 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.842972972Z level=info msg="Executing migration" id="create user auth token table" 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.843520163Z level=info msg="Migration successfully executed" id="create user auth token table" duration=546.913µs 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.843861468Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.844443731Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.auth_token" duration=582.242µs 2024-09-21T11:41:10.038 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.844799538Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-21T11:41:10.039 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.845389039Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.prev_auth_token" duration=589.461µs 2024-09-21T11:41:10.039 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.845743683Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-21T11:41:10.039 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.846398363Z level=info msg="Migration successfully executed" id="add index user_auth_token.user_id" duration=654.502µs 2024-09-21T11:41:10.039 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.846764965Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-21T11:41:10.039 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.849881504Z level=info msg="Migration successfully executed" id="Add revoked_at to the user auth token" duration=3.115288ms 2024-09-21T11:41:10.039 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.850250279Z level=info msg="Executing migration" id="add index user_auth_token.revoked_at" 2024-09-21T11:41:10.039 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.851034331Z level=info msg="Migration successfully executed" id="add index user_auth_token.revoked_at" duration=783.593µs 2024-09-21T11:41:10.039 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.851390306Z level=info msg="Executing migration" id="create cache_data table" 2024-09-21T11:41:10.040 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.851972162Z level=info msg="Migration successfully executed" id="create cache_data table" duration=581.671µs 2024-09-21T11:41:10.040 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.852319206Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-21T11:41:10.040 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.852888273Z level=info msg="Migration successfully executed" id="add unique index cache_data.cache_key" duration=568.838µs 2024-09-21T11:41:10.040 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.85324591Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-21T11:41:10.040 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.853810212Z level=info msg="Migration successfully executed" id="create short_url table v1" duration=562.303µs 2024-09-21T11:41:10.040 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.854218506Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.854870695Z level=info msg="Migration successfully executed" id="add index short_url.org_id-uid" duration=652.721µs 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.85683236Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.856887798Z level=info msg="Migration successfully executed" id="alter table short_url alter column created_by type to bigint" duration=55.792µs 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.859488579Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.859557677Z level=info msg="Migration successfully executed" id="delete alert_definition table" duration=68.62µs 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.862212109Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.862796759Z level=info msg="Migration successfully executed" id="recreate alert_definition table" duration=584.279µs 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.863172133Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.863783194Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and title columns" duration=610.824µs 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.865873784Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.866496157Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and uid columns" duration=622.157µs 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.868560849Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-21T11:41:10.041 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.868621696Z level=info msg="Migration successfully executed" id="alter alert_definition table data column to mediumtext in mysql" duration=61.225µs 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.871550122Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.872171468Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and title columns" duration=619.418µs 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.874303253Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.874943558Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and uid columns" duration=640.264µs 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.877096719Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.877749447Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and title columns" duration=651.216µs 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.879904634Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.880558971Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and uid columns" duration=652.707µs 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.882570422Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.886069055Z level=info msg="Migration successfully executed" id="Add column paused in alert_definition" duration=3.498145ms 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.886422168Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.887115413Z level=info msg="Migration successfully executed" id="drop alert_definition table" duration=693.252µs 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.889196521Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-21T11:41:10.042 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.889260941Z level=info msg="Migration successfully executed" id="delete alert_definition_version table" duration=61.199µs 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.891891261Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.892507332Z level=info msg="Migration successfully executed" id="recreate alert_definition_version table" duration=615.722µs 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.894381411Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.895014032Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_id and version columns" duration=632.475µs 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.895381398Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.896044468Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_uid and version columns" duration=662.541µs 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.897809617Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.897864782Z level=info msg="Migration successfully executed" id="alter alert_definition_version table data column to mediumtext in mysql" duration=55.522µs 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.900329658Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.901019438Z level=info msg="Migration successfully executed" id="drop alert_definition_version table" duration=688.036µs 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.903038569Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.903689296Z level=info msg="Migration successfully executed" id="create alert_instance table" duration=650.368µs 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.904053584Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-21T11:41:10.043 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.904688873Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" duration=633.964µs 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.906745508Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.907395154Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, current_state columns" duration=649.359µs 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.909125463Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.912620891Z level=info msg="Migration successfully executed" id="add column current_state_end to alert_instance" duration=3.495037ms 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.91303375Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.91364733Z level=info msg="Migration successfully executed" id="remove index def_org_id, def_uid, current_state on alert_instance" duration=613.629µs 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.914024001Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.914596037Z level=info msg="Migration successfully executed" id="remove index def_org_id, current_state on alert_instance" duration=571.649µs 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.916479059Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.934506458Z level=info msg="Migration successfully executed" id="rename def_org_id to rule_org_id in alert_instance" duration=18.027188ms 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.934969434Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.952192537Z level=info msg="Migration successfully executed" id="rename def_uid to rule_uid in alert_instance" duration=17.220775ms 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.952651753Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-21T11:41:10.044 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.953295526Z level=info msg="Migration successfully executed" id="add index rule_org_id, rule_uid, current_state on alert_instance" duration=643.069µs 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.953650826Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.954278557Z level=info msg="Migration successfully executed" id="add index rule_org_id, current_state on alert_instance" duration=627.554µs 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.956209032Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.95949822Z level=info msg="Migration successfully executed" id="add current_reason column related to current_state" duration=3.289182ms 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.959861617Z level=info msg="Executing migration" id="add result_fingerprint column to alert_instance" 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.963121648Z level=info msg="Migration successfully executed" id="add result_fingerprint column to alert_instance" duration=3.261295ms 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.963483015Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.964113614Z level=info msg="Migration successfully executed" id="create alert_rule table" duration=630.212µs 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.964472813Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.965098689Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and title columns" duration=625.587µs 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.967008211Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.967620064Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and uid columns" duration=611.395µs 2024-09-21T11:41:10.045 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.969659342Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.970357431Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" duration=696.341µs 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.970717837Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.97076516Z level=info msg="Migration successfully executed" id="alter alert_rule table data column to mediumtext in mysql" duration=47.462µs 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.971190716Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.974556464Z level=info msg="Migration successfully executed" id="add column for to alert_rule" duration=3.366273ms 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.974920236Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.978288443Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule" duration=3.366928ms 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.978653731Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.982028217Z level=info msg="Migration successfully executed" id="add column labels to alert_rule" duration=3.374042ms 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.982384695Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.982983716Z level=info msg="Migration successfully executed" id="remove unique index from alert_rule on org_id, title columns" duration=598.898µs 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.983330728Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.983958341Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespase_uid and title columns" duration=627.26µs 2024-09-21T11:41:10.046 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.984298737Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.98763379Z level=info msg="Migration successfully executed" id="add dashboard_uid column to alert_rule" duration=3.334738ms 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.988020786Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.991298973Z level=info msg="Migration successfully executed" id="add panel_id column to alert_rule" duration=3.277998ms 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.991664497Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.9923001Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" duration=635.349µs 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.992693108Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.995984299Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule" duration=3.290954ms 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.996348378Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.999570966Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule table" duration=3.222268ms 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.999942731Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:09.99999266Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule table" duration=50.981µs 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.000741998Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-21T11:41:10.047 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.001470241Z level=info msg="Migration successfully executed" id="create alert_rule_version table" duration=727.876µs 2024-09-21T11:41:10.048 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.001838056Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-21T11:41:10.048 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.002471878Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" duration=632.994µs 2024-09-21T11:41:10.048 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.0028239Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-21T11:41:10.048 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.003488782Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" duration=664.533µs 2024-09-21T11:41:10.048 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.003872926Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-21T11:41:10.048 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.003919025Z level=info msg="Migration successfully executed" id="alter alert_rule_version table data column to mediumtext in mysql" duration=46.446µs 2024-09-21T11:41:10.048 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.004666013Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-21T11:41:10.048 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.008064101Z level=info msg="Migration successfully executed" id="add column for to alert_rule_version" duration=3.397702ms 2024-09-21T11:41:10.048 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.008422217Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.011873586Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule_version" duration=3.450301ms 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.012242867Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.015681191Z level=info msg="Migration successfully executed" id="add column labels to alert_rule_version" duration=3.438075ms 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.016043467Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.019468157Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule_version" duration=3.424351ms 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.019830642Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.023274445Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule_versions table" duration=3.44446ms 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.023625874Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.023672432Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule_version table" duration=46.838µs 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.02440286Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.024889347Z level=info msg="Migration successfully executed" id=create_alert_configuration_table duration=486.172µs 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.025250101Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-21T11:41:10.049 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.028880106Z level=info msg="Migration successfully executed" id="Add column default in alert_configuration" duration=3.629859ms 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.029249879Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.029297349Z level=info msg="Migration successfully executed" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" duration=47.721µs 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.02969211Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.033343262Z level=info msg="Migration successfully executed" id="add column org_id in alert_configuration" duration=3.650805ms 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.033758814Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.03442058Z level=info msg="Migration successfully executed" id="add index in alert_configuration table on org_id column" duration=661.532µs 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.03478758Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.038710578Z level=info msg="Migration successfully executed" id="add configuration_hash column to alert_configuration" duration=3.923144ms 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.039077701Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.039605913Z level=info msg="Migration successfully executed" id=create_ngalert_configuration_table duration=528.041µs 2024-09-21T11:41:10.050 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.040002501Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-21T11:41:10.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.040654824Z level=info msg="Migration successfully executed" id="add index in ngalert_configuration on org_id column" duration=653.252µs 2024-09-21T11:41:10.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.041026038Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-21T11:41:10.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-21T11:41:10.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:41:10.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:10.143 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:10.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:10.144 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:10.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:10.273 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:10.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:10.274 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:10.293 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.044776378Z level=info msg="Migration successfully executed" id="add column send_alerts_to in ngalert_configuration" duration=3.750495ms 2024-09-21T11:41:10.293 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.045151746Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.045662555Z level=info msg="Migration successfully executed" id="create provenance_type table" duration=511.695µs 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.046047347Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.046686001Z level=info msg="Migration successfully executed" id="add index to uniquify (record_key, record_type, org_id) columns" duration=638.312µs 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.047047663Z level=info msg="Executing migration" id="create alert_image table" 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.047557238Z level=info msg="Migration successfully executed" id="create alert_image table" duration=509.184µs 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.047917085Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.048528202Z level=info msg="Migration successfully executed" id="add unique index on token to alert_image table" duration=610.387µs 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.050713424Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.050755308Z level=info msg="Migration successfully executed" id="support longer URLs in alert_image table" duration=42.675µs 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.053562796Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.054280792Z level=info msg="Migration successfully executed" id=create_alert_configuration_history_table duration=718.128µs 2024-09-21T11:41:10.294 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.054640693Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-21T11:41:10.295 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.055240286Z level=info msg="Migration successfully executed" id="drop non-unique orgID index on alert_configuration" duration=599.215µs 2024-09-21T11:41:10.295 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.05745498Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-21T11:41:10.295 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.057699479Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-21T11:41:10.295 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.060128782Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-21T11:41:10.295 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.060399279Z level=info msg="Migration successfully executed" id="extract alertmanager configuration history to separate table" duration=269.427µs 2024-09-21T11:41:10.295 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.062832078Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.063488142Z level=info msg="Migration successfully executed" id="add unique index on orgID to alert_configuration" duration=655.75µs 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.063847324Z level=info msg="Executing migration" id="add last_applied column to alert_configuration_history" 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.06763503Z level=info msg="Migration successfully executed" id="add last_applied column to alert_configuration_history" duration=3.787163ms 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.068033462Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.068721638Z level=info msg="Migration successfully executed" id="create library_element table v1" duration=687.998µs 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.070822223Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.071480828Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_id-name-kind" duration=658.788µs 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.073422916Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.074006024Z level=info msg="Migration successfully executed" id="create library_element_connection table v1" duration=582.943µs 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.076082793Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.07673954Z level=info msg="Migration successfully executed" id="add index library_element_connection element_id-kind-connection_id" duration=656.695µs 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.078818281Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.079464107Z level=info msg="Migration successfully executed" id="add unique index library_element org_id_uid" duration=645.536µs 2024-09-21T11:41:10.296 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.081654732Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.081690117Z level=info msg="Migration successfully executed" id="increase max description length to 2048" duration=35.277µs 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.084460081Z level=info msg="Executing migration" id="alter library_element model to mediumtext" 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.08452485Z level=info msg="Migration successfully executed" id="alter library_element model to mediumtext" duration=64.757µs 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.084919128Z level=info msg="Executing migration" id="add library_element folder uid" 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.089053064Z level=info msg="Migration successfully executed" id="add library_element folder uid" duration=4.133405ms 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.089399347Z level=info msg="Executing migration" id="populate library_element folder_uid" 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.089645243Z level=info msg="Migration successfully executed" id="populate library_element folder_uid" duration=244.63µs 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.092183101Z level=info msg="Executing migration" id="add index library_element org_id-folder_uid-name-kind" 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.092857266Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_uid-name-kind" duration=673.983µs 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.095025188Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.095205342Z level=info msg="Migration successfully executed" id="clone move dashboard alerts to unified alerting" duration=180.873µs 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.095595826Z level=info msg="Executing migration" id="create data_keys table" 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.096251156Z level=info msg="Migration successfully executed" id="create data_keys table" duration=654.826µs 2024-09-21T11:41:10.297 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.098236933Z level=info msg="Executing migration" id="create secrets table" 2024-09-21T11:41:10.298 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.098810283Z level=info msg="Migration successfully executed" id="create secrets table" duration=589.249µs 2024-09-21T11:41:10.298 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.100655859Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-21T11:41:10.298 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.122984742Z level=info msg="Migration successfully executed" id="rename data_keys name column to id" duration=22.328444ms 2024-09-21T11:41:10.298 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.123352075Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-21T11:41:10.298 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.127497734Z level=info msg="Migration successfully executed" id="add name column into data_keys" duration=4.145344ms 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.127849634Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.127961323Z level=info msg="Migration successfully executed" id="copy data_keys id column values into name" duration=111.724µs 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.130187513Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.151731632Z level=info msg="Migration successfully executed" id="rename data_keys name column to label" duration=21.542101ms 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.152168354Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.173819469Z level=info msg="Migration successfully executed" id="rename data_keys id column back to name" duration=21.651172ms 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.174221944Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.174785567Z level=info msg="Migration successfully executed" id="create kv_store table v1" duration=563.661µs 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.176824865Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.177540837Z level=info msg="Migration successfully executed" id="add index kv_store.org_id-namespace-key" duration=715.729µs 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.179269908Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-21T11:41:10.299 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.179427227Z level=info msg="Migration successfully executed" id="update dashboard_uid and panel_id from existing annotations" duration=157.298µs 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.181677601Z level=info msg="Executing migration" id="create permission table" 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.182268818Z level=info msg="Migration successfully executed" id="create permission table" duration=591.994µs 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.184351249Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.184992066Z level=info msg="Migration successfully executed" id="add unique index permission.role_id" duration=640.725µs 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.186912836Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.187587734Z level=info msg="Migration successfully executed" id="add unique index role_id_action_scope" duration=674.648µs 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.189512832Z level=info msg="Executing migration" id="create role table" 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.190102044Z level=info msg="Migration successfully executed" id="create role table" duration=588.716µs 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.192022122Z level=info msg="Executing migration" id="add column display_name" 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.196572609Z level=info msg="Migration successfully executed" id="add column display_name" duration=4.549067ms 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.196959981Z level=info msg="Executing migration" id="add column group_name" 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.201503749Z level=info msg="Migration successfully executed" id="add column group_name" duration=4.543644ms 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.201869485Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-21T11:41:10.300 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.202515035Z level=info msg="Migration successfully executed" id="add index role.org_id" duration=644.463µs 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.202864304Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.203532473Z level=info msg="Migration successfully executed" id="add unique index role_org_id_name" duration=667.975µs 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.203880557Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.204558166Z level=info msg="Migration successfully executed" id="add index role_org_id_uid" duration=677.205µs 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.204907312Z level=info msg="Executing migration" id="create team role table" 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.205545586Z level=info msg="Migration successfully executed" id="create team role table" duration=638.019µs 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.205915539Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.206577724Z level=info msg="Migration successfully executed" id="add index team_role.org_id" duration=661.816µs 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.206913193Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.207578304Z level=info msg="Migration successfully executed" id="add unique index team_role_org_id_team_id_role_id" duration=664.735µs 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.207915904Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.208565502Z level=info msg="Migration successfully executed" id="add index team_role.team_id" duration=649.31µs 2024-09-21T11:41:10.301 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.20890828Z level=info msg="Executing migration" id="create user role table" 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.209485217Z level=info msg="Migration successfully executed" id="create user role table" duration=576.641µs 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.209832846Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.210501493Z level=info msg="Migration successfully executed" id="add index user_role.org_id" duration=667.672µs 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.210857848Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.211538794Z level=info msg="Migration successfully executed" id="add unique index user_role_org_id_user_id_role_id" duration=679.596µs 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.21196764Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.212630458Z level=info msg="Migration successfully executed" id="add index user_role.user_id" duration=662.551µs 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.212991969Z level=info msg="Executing migration" id="create builtin role table" 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.213562762Z level=info msg="Migration successfully executed" id="create builtin role table" duration=570.529µs 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.213936356Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.214618149Z level=info msg="Migration successfully executed" id="add index builtin_role.role_id" duration=689.69µs 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.21497946Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-21T11:41:10.302 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.215643663Z level=info msg="Migration successfully executed" id="add index builtin_role.name" duration=663.77µs 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.215999877Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.220595207Z level=info msg="Migration successfully executed" id="Add column org_id to builtin_role table" duration=4.595086ms 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.220968956Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.221629678Z level=info msg="Migration successfully executed" id="add index builtin_role.org_id" duration=660.548µs 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.222058299Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.222713871Z level=info msg="Migration successfully executed" id="add unique index builtin_role_org_id_role_id_role" duration=655.382µs 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.223070903Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.223712016Z level=info msg="Migration successfully executed" id="Remove unique index role_org_id_uid" duration=641.108µs 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.224050737Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.224669222Z level=info msg="Migration successfully executed" id="add unique index role.uid" duration=617.38µs 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.224994318Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.225468815Z level=info msg="Migration successfully executed" id="create seed assignment table" duration=474.202µs 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.225796591Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-21T11:41:10.303 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.226469013Z level=info msg="Migration successfully executed" id="add unique index builtin_role_role_name" duration=672.028µs 2024-09-21T11:41:10.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.226797288Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-21T11:41:10.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.231472003Z level=info msg="Migration successfully executed" id="add column hidden to role table" duration=4.674476ms 2024-09-21T11:41:10.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.231815837Z level=info msg="Executing migration" id="permission kind migration" 2024-09-21T11:41:10.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.236542513Z level=info msg="Migration successfully executed" id="permission kind migration" duration=4.726483ms 2024-09-21T11:41:10.304 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.236891481Z level=info msg="Executing migration" id="permission attribute migration" 2024-09-21T11:41:10.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.241507884Z level=info msg="Migration successfully executed" id="permission attribute migration" duration=4.614624ms 2024-09-21T11:41:10.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.241856258Z level=info msg="Executing migration" id="permission identifier migration" 2024-09-21T11:41:10.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.246407059Z level=info msg="Migration successfully executed" id="permission identifier migration" duration=4.550497ms 2024-09-21T11:41:10.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.246740744Z level=info msg="Executing migration" id="add permission identifier index" 2024-09-21T11:41:10.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.247410843Z level=info msg="Migration successfully executed" id="add permission identifier index" duration=669.926µs 2024-09-21T11:41:10.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.247759385Z level=info msg="Executing migration" id="add permission action scope role_id index" 2024-09-21T11:41:10.306 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.248415909Z level=info msg="Migration successfully executed" id="add permission action scope role_id index" duration=656.328µs 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.248755935Z level=info msg="Executing migration" id="remove permission role_id action scope index" 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.249393332Z level=info msg="Migration successfully executed" id="remove permission role_id action scope index" duration=637.195µs 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.249733849Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.250293389Z level=info msg="Migration successfully executed" id="create query_history table v1" duration=559.151µs 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.250654762Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.25131022Z level=info msg="Migration successfully executed" id="add index query_history.org_id-created_by-datasource_uid" duration=655.111µs 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.25163228Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.251670878Z level=info msg="Migration successfully executed" id="alter table query_history alter column created_by type to bigint" duration=38.919µs 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.252052507Z level=info msg="Executing migration" id="rbac disabled migrator" 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.252071402Z level=info msg="Migration successfully executed" id="rbac disabled migrator" duration=19.415µs 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.252449596Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.252693524Z level=info msg="Migration successfully executed" id="teams permissions migration" duration=243.855µs 2024-09-21T11:41:10.307 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.253034178Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.253355566Z level=info msg="Migration successfully executed" id="dashboard permissions" duration=321.912µs 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.253722734Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.254118493Z level=info msg="Migration successfully executed" id="dashboard permissions uid scopes" duration=395.696µs 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.2544883Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.25461551Z level=info msg="Migration successfully executed" id="drop managed folder create actions" duration=127.121µs 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.254978242Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.255251829Z level=info msg="Migration successfully executed" id="alerting notification permissions" duration=273.632µs 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.25564046Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.256137277Z level=info msg="Migration successfully executed" id="create query_history_star table v1" duration=496.554µs 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.256460459Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.25716715Z level=info msg="Migration successfully executed" id="add index query_history.user_id-query_uid" duration=706.385µs 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.257505375Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.262157068Z level=info msg="Migration successfully executed" id="add column org_id in query_history_star" duration=4.651343ms 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.262488091Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-21T11:41:10.308 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.262524854Z level=info msg="Migration successfully executed" id="alter table query_history_star_mig column user_id type to bigint" duration=37.035µs 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.262911285Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.263523065Z level=info msg="Migration successfully executed" id="create correlation table v1" duration=611.354µs 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.26386694Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.264619899Z level=info msg="Migration successfully executed" id="add index correlations.uid" duration=753.692µs 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.265015355Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.265703561Z level=info msg="Migration successfully executed" id="add index correlations.source_uid" duration=686.799µs 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.266037048Z level=info msg="Executing migration" id="add correlation config column" 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.270779018Z level=info msg="Migration successfully executed" id="add correlation config column" duration=4.741691ms 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.271141305Z level=info msg="Executing migration" id="drop index IDX_correlation_uid - v1" 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.271768893Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_uid - v1" duration=627.383µs 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.272103608Z level=info msg="Executing migration" id="drop index IDX_correlation_source_uid - v1" 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.272727787Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_source_uid - v1" duration=623.958µs 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.273059383Z level=info msg="Executing migration" id="Rename table correlation to correlation_tmp_qwerty - v1" 2024-09-21T11:41:10.309 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.291995981Z level=info msg="Migration successfully executed" id="Rename table correlation to correlation_tmp_qwerty - v1" duration=18.931228ms 2024-09-21T11:41:10.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.292880702Z level=info msg="Executing migration" id="create correlation v2" 2024-09-21T11:41:10.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.294661307Z level=info msg="Migration successfully executed" id="create correlation v2" duration=1.780175ms 2024-09-21T11:41:10.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.295656293Z level=info msg="Executing migration" id="create index IDX_correlation_uid - v2" 2024-09-21T11:41:10.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.298238872Z level=info msg="Migration successfully executed" id="create index IDX_correlation_uid - v2" duration=2.579339ms 2024-09-21T11:41:10.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.298885369Z level=info msg="Executing migration" id="create index IDX_correlation_source_uid - v2" 2024-09-21T11:41:10.310 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.300030772Z level=info msg="Migration successfully executed" id="create index IDX_correlation_source_uid - v2" duration=1.145345ms 2024-09-21T11:41:10.371 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-21T11:41:10.371 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[36454]: osdmap e82: 8 total, 8 up, 8 in 2024-09-21T11:41:10.371 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.371 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.371 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[36454]: Deploying daemon node-exporter.a on smithi043 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[29710]: osdmap e82: 8 total, 8 up, 8 in 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[29710]: Deploying daemon node-exporter.a on smithi043 2024-09-21T11:41:10.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:10 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.376 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:10.397 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:10.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:10.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:10.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:10.521 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:10.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:10.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:10.544 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-21T11:41:10.544 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-mon[32613]: osdmap e82: 8 total, 8 up, 8 in 2024-09-21T11:41:10.544 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.545 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.545 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.545 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.545 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-mon[32613]: Deploying daemon node-exporter.a on smithi043 2024-09-21T11:41:10.545 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:10.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.300620285Z level=info msg="Executing migration" id="create index IDX_correlation_org_id - v2" 2024-09-21T11:41:10.545 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.301747089Z level=info msg="Migration successfully executed" id="create index IDX_correlation_org_id - v2" duration=1.126129ms 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.302448659Z level=info msg="Executing migration" id="copy correlation v1 to v2" 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.302709456Z level=info msg="Migration successfully executed" id="copy correlation v1 to v2" duration=261.019µs 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.303478507Z level=info msg="Executing migration" id="drop correlation_tmp_qwerty" 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.304984074Z level=info msg="Migration successfully executed" id="drop correlation_tmp_qwerty" duration=1.503878ms 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.305720393Z level=info msg="Executing migration" id="add provisioning column" 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.311581494Z level=info msg="Migration successfully executed" id="add provisioning column" duration=5.859382ms 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.312031077Z level=info msg="Executing migration" id="create entity_events table" 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.312579227Z level=info msg="Migration successfully executed" id="create entity_events table" duration=548.171µs 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.312949142Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.313602115Z level=info msg="Migration successfully executed" id="create dashboard public config v1" duration=652.624µs 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.313958659Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-21T11:41:10.546 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.314220495Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.314788094Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.315049693Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.315427706Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.315988227Z level=info msg="Migration successfully executed" id="Drop old dashboard public config table" duration=558.355µs 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.316328808Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.316981432Z level=info msg="Migration successfully executed" id="recreate dashboard public config v1" duration=652.3µs 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.317320093Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.31797481Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v1" duration=654.366µs 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.320233308Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.321037676Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" duration=803.527µs 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.322873599Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.323556367Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_public_config_uid - v2" duration=682.619µs 2024-09-21T11:41:10.547 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.32557158Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-21T11:41:10.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.326348716Z level=info msg="Migration successfully executed" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=776.81µs 2024-09-21T11:41:10.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.328266757Z level=info msg="Executing migration" id="Drop public config table" 2024-09-21T11:41:10.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.328838129Z level=info msg="Migration successfully executed" id="Drop public config table" duration=571.062µs 2024-09-21T11:41:10.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.331056624Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-21T11:41:10.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.331826098Z level=info msg="Migration successfully executed" id="Recreate dashboard public config v2" duration=769.984µs 2024-09-21T11:41:10.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.333960146Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-21T11:41:10.548 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.334644712Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v2" duration=684.461µs 2024-09-21T11:41:10.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.336723772Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-21T11:41:10.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.337421332Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=697.269µs 2024-09-21T11:41:10.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.339611154Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-21T11:41:10.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.340386098Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_access_token - v2" duration=781.228µs 2024-09-21T11:41:10.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.342339779Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-21T11:41:10.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.35825907Z level=info msg="Migration successfully executed" id="Rename table dashboard_public_config to dashboard_public - v2" duration=15.918281ms 2024-09-21T11:41:10.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.358726295Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-21T11:41:10.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.363987834Z level=info msg="Migration successfully executed" id="add annotations_enabled column" duration=5.261329ms 2024-09-21T11:41:10.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.364319622Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-21T11:41:10.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.36929068Z level=info msg="Migration successfully executed" id="add time_selection_enabled column" duration=4.970727ms 2024-09-21T11:41:10.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.369646707Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-21T11:41:10.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.369788035Z level=info msg="Migration successfully executed" id="delete orphaned public dashboards" duration=141.21µs 2024-09-21T11:41:10.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.372339294Z level=info msg="Executing migration" id="add share column" 2024-09-21T11:41:10.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.377377693Z level=info msg="Migration successfully executed" id="add share column" duration=5.038132ms 2024-09-21T11:41:10.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.377727221Z level=info msg="Executing migration" id="backfill empty share column fields with default of public" 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.377858053Z level=info msg="Migration successfully executed" id="backfill empty share column fields with default of public" duration=128.308µs 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.380446435Z level=info msg="Executing migration" id="create file table" 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.381131605Z level=info msg="Migration successfully executed" id="create file table" duration=684.818µs 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.383161079Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.383955373Z level=info msg="Migration successfully executed" id="file table idx: path natural pk" duration=794.968µs 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.3844233Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.385175849Z level=info msg="Migration successfully executed" id="file table idx: parent_folder_path_hash fast folder retrieval" duration=752.775µs 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.38712656Z level=info msg="Executing migration" id="create file_meta table" 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.387747245Z level=info msg="Migration successfully executed" id="create file_meta table" duration=620.511µs 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.389858104Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-21T11:41:10.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.390583947Z level=info msg="Migration successfully executed" id="file table idx: path key" duration=725.551µs 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.392639138Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.392679045Z level=info msg="Migration successfully executed" id="set path collation in file table" duration=40.433µs 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.395285036Z level=info msg="Executing migration" id="migrate contents column to mediumblob for MySQL" 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.395339599Z level=info msg="Migration successfully executed" id="migrate contents column to mediumblob for MySQL" duration=54.91µs 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.397995651Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.398360491Z level=info msg="Migration successfully executed" id="managed permissions migration" duration=365.68µs 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.400745857Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.400917041Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions migration" duration=175.771µs 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.403156601Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.403972857Z level=info msg="Migration successfully executed" id="RBAC action name migrator" duration=815.887µs 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.405917127Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.411254033Z level=info msg="Migration successfully executed" id="Add UID column to playlist" duration=5.336493ms 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.411633657Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-21T11:41:10.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.411741667Z level=info msg="Migration successfully executed" id="Update uid column values in playlist" duration=114.692µs 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.414395982Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.415126727Z level=info msg="Migration successfully executed" id="Add index for uid in playlist" duration=730.454µs 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.417065403Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.41728737Z level=info msg="Migration successfully executed" id="update group index for alert rules" duration=222.321µs 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.419826962Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.420063451Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated migration" duration=238.507µs 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.422488952Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.422854031Z level=info msg="Migration successfully executed" id="admin only folder/dashboard permission" duration=366.977µs 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.425037904Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.429975521Z level=info msg="Migration successfully executed" id="add action column to seed_assignment" duration=4.937297ms 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.430312179Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.435763622Z level=info msg="Migration successfully executed" id="add scope column to seed_assignment" duration=5.450999ms 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.436136142Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.436817902Z level=info msg="Migration successfully executed" id="remove unique index builtin_role_role_name before nullable update" duration=681.814µs 2024-09-21T11:41:10.553 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.438732545Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.494826992Z level=info msg="Migration successfully executed" id="update seed_assignment role_name column to nullable" duration=56.094315ms 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.495278497Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.496045907Z level=info msg="Migration successfully executed" id="add unique index builtin_role_name back" duration=768.521µs 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.497909901Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.498664995Z level=info msg="Migration successfully executed" id="add unique index builtin_role_action_scope" duration=757.373µs 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.499061786Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.516280494Z level=info msg="Migration successfully executed" id="add primary key to seed_assigment" duration=17.218423ms 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.516659526Z level=info msg="Executing migration" id="add origin column to seed_assignment" 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.521666254Z level=info msg="Migration successfully executed" id="add origin column to seed_assignment" duration=5.006563ms 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.522044742Z level=info msg="Executing migration" id="add origin to plugin seed_assignment" 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.522235671Z level=info msg="Migration successfully executed" id="add origin to plugin seed_assignment" duration=191.092µs 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.52284404Z level=info msg="Executing migration" id="prevent seeding OnCall access" 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.522974173Z level=info msg="Migration successfully executed" id="prevent seeding OnCall access" duration=130.482µs 2024-09-21T11:41:10.554 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.5236614Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.523784319Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated fixed migration" duration=122.757µs 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.524503749Z level=info msg="Executing migration" id="managed folder permissions library panel actions migration" 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.524626324Z level=info msg="Migration successfully executed" id="managed folder permissions library panel actions migration" duration=123.557µs 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.525457173Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.525701458Z level=info msg="Migration successfully executed" id="migrate external alertmanagers to datsourcse" duration=247.714µs 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.526101661Z level=info msg="Executing migration" id="create folder table" 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.526737981Z level=info msg="Migration successfully executed" id="create folder table" duration=636.088µs 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.527122356Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.527813501Z level=info msg="Migration successfully executed" id="Add index for parent_uid" duration=690.986µs 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.52816615Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.528866514Z level=info msg="Migration successfully executed" id="Add unique index for folder.uid and folder.org_id" duration=698.625µs 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.529217757Z level=info msg="Executing migration" id="Update folder title length" 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.52923194Z level=info msg="Migration successfully executed" id="Update folder title length" duration=15.793µs 2024-09-21T11:41:10.555 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.530199236Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.531064199Z level=info msg="Migration successfully executed" id="Add unique index for folder.title and folder.parent_uid" duration=866.563µs 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.531424358Z level=info msg="Executing migration" id="Remove unique index for folder.title and folder.parent_uid" 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.532085364Z level=info msg="Migration successfully executed" id="Remove unique index for folder.title and folder.parent_uid" duration=660.975µs 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.532446559Z level=info msg="Executing migration" id="Add unique index for title, parent_uid, and org_id" 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.533123586Z level=info msg="Migration successfully executed" id="Add unique index for title, parent_uid, and org_id" duration=676.782µs 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.533480677Z level=info msg="Executing migration" id="Sync dashboard and folder table" 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.533765648Z level=info msg="Migration successfully executed" id="Sync dashboard and folder table" duration=284.805µs 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.534327974Z level=info msg="Executing migration" id="Remove ghost folders from the folder table" 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.534514356Z level=info msg="Migration successfully executed" id="Remove ghost folders from the folder table" duration=186.276µs 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.535116302Z level=info msg="Executing migration" id="Remove unique index UQE_folder_uid_org_id" 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.53579186Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_uid_org_id" duration=675.366µs 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.537567497Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_uid" 2024-09-21T11:41:10.556 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.538339144Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_uid" duration=773.139µs 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.539935006Z level=info msg="Executing migration" id="Remove unique index UQE_folder_title_parent_uid_org_id" 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.540610498Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_title_parent_uid_org_id" duration=675.178µs 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.542549441Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_parent_uid_title" 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.543281334Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_parent_uid_title" duration=733.609µs 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.543623386Z level=info msg="Executing migration" id="Remove index IDX_folder_parent_uid_org_id" 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.54441036Z level=info msg="Migration successfully executed" id="Remove index IDX_folder_parent_uid_org_id" duration=786.462µs 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.544744669Z level=info msg="Executing migration" id="create anon_device table" 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.545295592Z level=info msg="Migration successfully executed" id="create anon_device table" duration=550.648µs 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.545628062Z level=info msg="Executing migration" id="add unique index anon_device.device_id" 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.546371117Z level=info msg="Migration successfully executed" id="add unique index anon_device.device_id" duration=739.743µs 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.546746285Z level=info msg="Executing migration" id="add index anon_device.updated_at" 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.54745644Z level=info msg="Migration successfully executed" id="add index anon_device.updated_at" duration=709.959µs 2024-09-21T11:41:10.557 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.547805624Z level=info msg="Executing migration" id="create signing_key table" 2024-09-21T11:41:10.623 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:10.644 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:10.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:10.646 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:10.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:10.761 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:10.772 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:10.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:10.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:10.788 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:10.860 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:41:10.860 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.548372367Z level=info msg="Migration successfully executed" id="create signing_key table" duration=566.568µs 2024-09-21T11:41:10.860 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.54874042Z level=info msg="Executing migration" id="add unique index signing_key.key_id" 2024-09-21T11:41:10.860 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.549549157Z level=info msg="Migration successfully executed" id="add unique index signing_key.key_id" duration=807.03µs 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.549943029Z level=info msg="Executing migration" id="set legacy alert migration status in kvstore" 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.55091352Z level=info msg="Migration successfully executed" id="set legacy alert migration status in kvstore" duration=970.94µs 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.551271705Z level=info msg="Executing migration" id="migrate record of created folders during legacy migration to kvstore" 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.551440339Z level=info msg="Migration successfully executed" id="migrate record of created folders during legacy migration to kvstore" duration=169.201µs 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.552019893Z level=info msg="Executing migration" id="Add folder_uid for dashboard" 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.557480947Z level=info msg="Migration successfully executed" id="Add folder_uid for dashboard" duration=5.461266ms 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.557840212Z level=info msg="Executing migration" id="Populate dashboard folder_uid column" 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.558326039Z level=info msg="Migration successfully executed" id="Populate dashboard folder_uid column" duration=486.819µs 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.558727403Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title" 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.558741199Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title" duration=14.521µs 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.559467608Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_id_title" 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.560155982Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_id_title" duration=688.811µs 2024-09-21T11:41:10.861 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.560490897Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_uid_title" 2024-09-21T11:41:10.862 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.560499931Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_uid_title" duration=9.529µs 2024-09-21T11:41:10.862 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.56135024Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" 2024-09-21T11:41:10.862 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.562082679Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" duration=730.855µs 2024-09-21T11:41:10.862 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.562424712Z level=info msg="Executing migration" id="Restore index for dashboard_org_id_folder_id_title" 2024-09-21T11:41:10.862 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.563144404Z level=info msg="Migration successfully executed" id="Restore index for dashboard_org_id_folder_id_title" duration=719.257µs 2024-09-21T11:41:10.862 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.563514276Z level=info msg="Executing migration" id="create sso_setting table" 2024-09-21T11:41:10.862 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.564180292Z level=info msg="Migration successfully executed" id="create sso_setting table" duration=665.827µs 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.564544667Z level=info msg="Executing migration" id="copy kvstore migration status to each org" 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.565058666Z level=info msg="Migration successfully executed" id="copy kvstore migration status to each org" duration=513.909µs 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.565423613Z level=info msg="Executing migration" id="add back entry for orgid=0 migrated status" 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.565589919Z level=info msg="Migration successfully executed" id="add back entry for orgid=0 migrated status" duration=167.504µs 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.566248196Z level=info msg="Executing migration" id="alter kv_store.value to longtext" 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.566305683Z level=info msg="Migration successfully executed" id="alter kv_store.value to longtext" duration=57.688µs 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.566965303Z level=info msg="Executing migration" id="add notification_settings column to alert_rule table" 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.57213487Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule table" duration=5.169793ms 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.572505589Z level=info msg="Executing migration" id="add notification_settings column to alert_rule_version table" 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.577798021Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule_version table" duration=5.291727ms 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.57815801Z level=info msg="Executing migration" id="removing scope from alert.instances:read action migration" 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.578372711Z level=info msg="Migration successfully executed" id="removing scope from alert.instances:read action migration" duration=215.125µs 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=migrator t=2024-09-21T11:41:10.578914783Z level=info msg="migrations completed" performed=551 skipped=0 duration=1.26560535s 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore t=2024-09-21T11:41:10.579791688Z level=info msg="Created default organization" 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=secrets t=2024-09-21T11:41:10.580328333Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-21T11:41:10.863 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=plugin.store t=2024-09-21T11:41:10.594506676Z level=info msg="Loading plugins..." 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=local.finder t=2024-09-21T11:41:10.636814656Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=plugin.store t=2024-09-21T11:41:10.636833978Z level=info msg="Plugins loaded" count=55 duration=42.32781ms 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=query_data t=2024-09-21T11:41:10.638905227Z level=info msg="Query Service initialization" 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=live.push_http t=2024-09-21T11:41:10.641671866Z level=info msg="Live Push Gateway initialization" 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.migration t=2024-09-21T11:41:10.643030442Z level=info msg=Starting 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.migration t=2024-09-21T11:41:10.643322223Z level=info msg="Applying transition" currentType=Legacy desiredType=UnifiedAlerting cleanOnDowngrade=false cleanOnUpgrade=false 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.migration orgID=1 t=2024-09-21T11:41:10.643586919Z level=info msg="Migrating alerts for organisation" 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.migration orgID=1 t=2024-09-21T11:41:10.644061726Z level=info msg="Alerts found to migrate" alerts=0 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.migration t=2024-09-21T11:41:10.645377459Z level=info msg="Completed alerting migration" 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.state.manager t=2024-09-21T11:41:10.656445856Z level=info msg="Running in alternative execution of Error/NoData mode" 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=infra.usagestats.collector t=2024-09-21T11:41:10.657718887Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=provisioning.datasources t=2024-09-21T11:41:10.659311831Z level=info msg="inserting datasource from configuration" name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=provisioning.datasources t=2024-09-21T11:41:10.671716607Z level=info msg="inserting datasource from configuration" name=Loki uid=P8E80F9AEF21F6940 2024-09-21T11:41:10.864 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=provisioning.alerting t=2024-09-21T11:41:10.684214784Z level=info msg="starting to provision alerting" 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=provisioning.alerting t=2024-09-21T11:41:10.684225105Z level=info msg="finished to provision alerting" 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=grafanaStorageLogger t=2024-09-21T11:41:10.684344897Z level=info msg="Storage starting" 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=http.server t=2024-09-21T11:41:10.68642328Z level=info msg="HTTP Server TLS settings" MinTLSVersion=TLS1.2 configuredciphers=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.state.manager t=2024-09-21T11:41:10.686791236Z level=info msg="Warming state cache for startup" 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=http.server t=2024-09-21T11:41:10.68683419Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.state.manager t=2024-09-21T11:41:10.686967169Z level=info msg="State cache has been initialized" states=0 duration=175.312µs 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.multiorg.alertmanager t=2024-09-21T11:41:10.686991604Z level=info msg="Starting MultiOrg Alertmanager" 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.scheduler t=2024-09-21T11:41:10.687011415Z level=info msg="Starting scheduler" tickInterval=10s maxAttempts=1 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ticker t=2024-09-21T11:41:10.687044954Z level=info msg=starting first_tick=2024-09-21T11:41:20Z 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore.transactions t=2024-09-21T11:41:10.695487903Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=provisioning.dashboard t=2024-09-21T11:41:10.697843874Z level=info msg="starting to provision dashboards" 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore.transactions t=2024-09-21T11:41:10.712943301Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore.transactions t=2024-09-21T11:41:10.724889645Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=plugins.update.checker t=2024-09-21T11:41:10.767717111Z level=info msg="Update check succeeded" duration=83.053055ms 2024-09-21T11:41:10.865 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=grafana.update.checker t=2024-09-21T11:41:10.774847018Z level=info msg="Update check succeeded" duration=88.27231ms 2024-09-21T11:41:10.866 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=provisioning.dashboard t=2024-09-21T11:41:10.859170351Z level=info msg="finished to provision dashboards" 2024-09-21T11:41:10.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:10.897 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:10.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:10.898 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:10.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:11.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:11.022 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:11.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:11.123 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:11.146 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:11.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:11.147 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:11.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:11.260 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=grafana-apiserver t=2024-09-21T11:41:11.056497831Z level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager" 2024-09-21T11:41:11.260 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:41:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=grafana-apiserver t=2024-09-21T11:41:11.056867766Z level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager" 2024-09-21T11:41:11.269 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:11.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:11.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:11.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:11.395 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:11.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:11.396 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:11.449 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:11 smithi043 ceph-mon[36454]: pgmap v32: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 673 B/s rd, 0 op/s 2024-09-21T11:41:11.449 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:11 smithi043 ceph-mon[36454]: osdmap e83: 8 total, 8 up, 8 in 2024-09-21T11:41:11.449 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:11 smithi043 ceph-mon[36454]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:41:11.449 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:11 smithi043 ceph-mon[29710]: pgmap v32: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 673 B/s rd, 0 op/s 2024-09-21T11:41:11.450 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:11 smithi043 ceph-mon[29710]: osdmap e83: 8 total, 8 up, 8 in 2024-09-21T11:41:11.450 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:11 smithi043 ceph-mon[29710]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:41:11.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:11.521 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:11.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:11.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:11.574 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:41:11.574 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:41:11.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:11.647 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:11.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:11.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:11.739 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:11.749 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:11.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:11 smithi175 ceph-mon[32613]: pgmap v32: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 673 B/s rd, 0 op/s 2024-09-21T11:41:11.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:11 smithi175 ceph-mon[32613]: osdmap e83: 8 total, 8 up, 8 in 2024-09-21T11:41:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:11 smithi175 ceph-mon[32613]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:41:11.771 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:11.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:11.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:11.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:11.895 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:11.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:11.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:11.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:12.019 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:12.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:12.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:12.121 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:12.146 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:12.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:12.147 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:12.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:41:12.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:12.273 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:12.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:12.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:12.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:12.398 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:12.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:12.399 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:12.421 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:12 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:12.421 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:12 smithi043 ceph-mon[36454]: osdmap e84: 8 total, 8 up, 8 in 2024-09-21T11:41:12.421 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:12 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:12.421 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:12 smithi043 ceph-mon[29710]: osdmap e84: 8 total, 8 up, 8 in 2024-09-21T11:41:12.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:12.525 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:12.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:12.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:12.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:12.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:12.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:12.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:41:12.751 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:12.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:12 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:12.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:12 smithi175 ceph-mon[32613]: osdmap e84: 8 total, 8 up, 8 in 2024-09-21T11:41:12.777 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:12.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:12.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:12.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:12.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:12.903 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:13.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:13.029 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:13.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:13.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:13.061 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:13.095 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:13.132 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:13.157 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:13.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:13.158 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:13.259 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:13.284 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:13.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:13.285 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:13.313 INFO:teuthology.orchestra.run.smithi043.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-21T11:41:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:41:13.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:13.408 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:13.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:13.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:13.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:13 smithi043 ceph-mon[29710]: pgmap v35: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:41:13.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2774884108' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:41:13.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3107055029' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:41:13.420 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:13 smithi043 ceph-mon[36454]: pgmap v35: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:41:13.420 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2774884108' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:41:13.420 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3107055029' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:41:13.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:13.533 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:13.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:13.534 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:13.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:13.658 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:13.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:13.659 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:13.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:13.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:13 smithi175 ceph-mon[32613]: pgmap v35: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:41:13.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2774884108' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:41:13.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3107055029' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:41:13.779 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:41:13.779 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:41:13.779 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-21T11:41:13.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.0 'trim stale osdmaps' 2024-09-21T11:41:13.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:13.785 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:13.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:13.886 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:13.909 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:13.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:13.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:14.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:14.037 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:14.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:14.038 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:14.138 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:14.164 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:14.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:14.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:14.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:14.290 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:14.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:14.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:14.370 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:41:14.370 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:41:14.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:41:14.370 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:41:14.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.370 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14430.0:2 3.12 3:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-21T11:41:14.370 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:00.305385+0000", 2024-09-21T11:41:14.370 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.043101041, 2024-09-21T11:41:14.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059996429999999998, 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14430", 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/1706821443", 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.305385+0000", 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.305385+0000", 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.305389+0000", 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7979999999999999e-06 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.305394+0000", 2024-09-21T11:41:14.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4689999999999996e-06 2024-09-21T11:41:14.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.305404+0000", 2024-09-21T11:41:14.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7729999999999994e-06 2024-09-21T11:41:14.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.305414+0000", 2024-09-21T11:41:14.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0638e-05 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.305531+0000", 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011696099999999999 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.305540+0000", 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1189999999999997e-06 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.311303+0000", 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057633659999999998 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.311384+0000", 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1519000000000003e-05 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.24343.0:1 2.1 e59/57 2:c4c92e5a:::rbd_trash:head v 59'1, mlcod=0'0)", 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:00.314328+0000", 2024-09-21T11:41:14.375 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.034157618999998, 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028142140000000002, 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.314328+0000", 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.314328+0000", 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.314329+0000", 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0500000000000003e-07 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.314334+0000", 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.485e-06 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.314343+0000", 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0019999999999995e-06 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.314351+0000", 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7780000000000007e-06 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.314522+0000", 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000170772 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.314541+0000", 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8817999999999998e-05 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.317102+0000", 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025610149999999998 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:00.317142+0000", 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0639000000000002e-05 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:57 5.3 5:c609908c:::notify.5:head [watch watch cookie 93908746616704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:04.348142+0000", 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.000344059, 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017661409999999999, 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.380 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 57 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348142+0000", 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348142+0000", 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348143+0000", 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.249e-06 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348149+0000", 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0730000000000001e-06 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348165+0000", 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5571999999999999e-05 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348173+0000", 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4980000000000006e-06 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348244+0000", 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1008999999999997e-05 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348339+0000", 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4721000000000003e-05 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348543+0000", 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000203902 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348595+0000", 2024-09-21T11:41:14.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2259999999999998e-05 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348745+0000", 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015009299999999999 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.349087+0000", 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034165699999999999 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.349430+0000", 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034363299999999998 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.349737+0000", 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030622499999999998 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.349794+0000", 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7031000000000003e-05 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.349908+0000", 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114218 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14481.0:57 5.3 5:c609908c:::notify.5:head [watch watch cookie 94366406333600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:04.348697+0000", 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.999788741000003, 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002278043, 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14481", 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/1749088654", 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 57 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348697+0000", 2024-09-21T11:41:14.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348697+0000", 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348698+0000", 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1880000000000001e-06 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348703+0000", 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1370000000000001e-06 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348716+0000", 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2619000000000001e-05 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348725+0000", 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6600000000000001e-06 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.348974+0000", 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024918299999999999 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for rw locks", 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.349038+0000", 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4553000000000003e-05 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.349970+0000", 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000931952 2024-09-21T11:41:14.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.350024+0000", 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4132000000000003e-05 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.350136+0000", 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111605 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.350570+0000", 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00043380699999999998 2024-09-21T11:41:14.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.350819+0000", 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000249335 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.350909+0000", 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9828999999999997e-05 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.350938+0000", 2024-09-21T11:41:14.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9201e-05 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:04.350975+0000", 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6841999999999997e-05 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:207 4.b 4:d242335b:gc::gc.2:head [create,call version.check_conds in=18b,call rgw_gc.rgw_gc_queue_init in=23b,call version.set in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.358922+0000", 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.989563637000003, 2024-09-21T11:41:14.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001512303, 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 207 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.358922+0000", 2024-09-21T11:41:14.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.358922+0000", 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.358925+0000", 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7209999999999999e-06 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.358933+0000", 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.038e-06 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.358948+0000", 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5264000000000002e-05 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.358957+0000", 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1370000000000008e-06 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.359032+0000", 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4190999999999995e-05 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.359154+0000", 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012280099999999999 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.359435+0000", 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000281014 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.359905+0000", 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00046960099999999999 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.360137+0000", 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023245000000000001 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.360372+0000", 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000234961 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.360406+0000", 2024-09-21T11:41:14.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3924999999999998e-05 2024-09-21T11:41:14.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.360434+0000", 2024-09-21T11:41:14.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8200000000000001e-05 2024-09-21T11:41:14.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.401 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:221 4.3 4:c623b173:gc::gc.16:head [create,call version.check_conds in=18b,call rgw_gc.rgw_gc_queue_init in=23b,call version.set in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.378588+0000", 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.969897778999993, 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014235249999999999, 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 221 2024-09-21T11:41:14.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.378588+0000", 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.378588+0000", 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.378590+0000", 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4980000000000001e-06 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.378596+0000", 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6420000000000001e-06 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.378612+0000", 2024-09-21T11:41:14.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6206999999999999e-05 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.378623+0000", 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0869999999999999e-05 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.378690+0000", 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6323999999999997e-05 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.378823+0000", 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013388699999999999 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.379066+0000", 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024223699999999999 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.379489+0000", 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042299599999999997 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.379905+0000", 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00041665000000000002 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.379946+0000", 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0185999999999998e-05 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.379982+0000", 2024-09-21T11:41:14.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6575999999999997e-05 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.380012+0000", 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9451999999999999e-05 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:273 4.3 4:c31f41d7:gc::gc.10:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.406112+0000", 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.942373997999994, 2024-09-21T11:41:14.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012547929999999999, 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 273 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.406112+0000", 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.406112+0000", 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.406114+0000", 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7749999999999999e-06 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.406123+0000", 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7950000000000002e-06 2024-09-21T11:41:14.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.406142+0000", 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8125e-05 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.406149+0000", 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3289999999999996e-06 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.406211+0000", 2024-09-21T11:41:14.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2612000000000003e-05 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.406304+0000", 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2516000000000007e-05 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.406493+0000", 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018876900000000001 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.406898+0000", 2024-09-21T11:41:14.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00040563200000000001 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.407071+0000", 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000172814 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.407195+0000", 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000124177 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.407320+0000", 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012496 2024-09-21T11:41:14.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.407367+0000", 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6288999999999997e-05 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:294 4.3 4:c31f41d7:gc::gc.10:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.409128+0000", 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.939357494000006, 2024-09-21T11:41:14.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015123070000000001, 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 294 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.409128+0000", 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.409128+0000", 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.409129+0000", 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.409136+0000", 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5679999999999996e-06 2024-09-21T11:41:14.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.409149+0000", 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3042e-05 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.409157+0000", 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7549999999999994e-06 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.409216+0000", 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8980000000000001e-05 2024-09-21T11:41:14.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.409284+0000", 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8063000000000002e-05 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.409389+0000", 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010576100000000001 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.409692+0000", 2024-09-21T11:41:14.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030244700000000001 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.410301+0000", 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00060873999999999998 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.410560+0000", 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00025921000000000001 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.410604+0000", 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4629999999999998e-05 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.410641+0000", 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6139000000000001e-05 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:361 4.3 4:c3a682d4:reshard::reshard.0000000004:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.420 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.417279+0000", 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.931206380000006, 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001243547, 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 361 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.417279+0000", 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.417279+0000", 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.417281+0000", 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.418e-06 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.417286+0000", 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.113e-06 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.417299+0000", 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3244000000000001e-05 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.417307+0000", 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0099999999999995e-06 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.417379+0000", 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2290000000000001e-05 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.417488+0000", 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010823399999999999 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.417693+0000", 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000205446 2024-09-21T11:41:14.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.418064+0000", 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037072599999999999 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.418413+0000", 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034925499999999998 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.418466+0000", 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2709e-05 2024-09-21T11:41:14.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.418496+0000", 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9903000000000002e-05 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.418523+0000", 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7199000000000002e-05 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.426 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:384 4.6 4:66924383:reshard::reshard.0000000005:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.420551+0000", 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.927934605000004, 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016424110000000001, 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 384 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.420551+0000", 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.420551+0000", 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.420552+0000", 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5799999999999998e-07 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.420557+0000", 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1900000000000003e-06 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.420572+0000", 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4643e-05 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.420580+0000", 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4370000000000007e-06 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.420653+0000", 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2356999999999999e-05 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.420744+0000", 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0821000000000003e-05 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.420925+0000", 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000181387 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.421322+0000", 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00039693900000000002 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.421873+0000", 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00055110000000000001 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422091+0000", 2024-09-21T11:41:14.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021771 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422150+0000", 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9846000000000003e-05 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422194+0000", 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3123e-05 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.432 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:393 4.6 4:66924383:reshard::reshard.0000000005:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.422661+0000", 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.925825196999995, 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001526192, 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 393 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422661+0000", 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422661+0000", 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422661+0000", 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2799999999999995e-07 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422665+0000", 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5609999999999999e-06 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422674+0000", 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0280000000000006e-06 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422681+0000", 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8000000000000001e-06 2024-09-21T11:41:14.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422729+0000", 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8436999999999997e-05 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422807+0000", 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7470000000000002e-05 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.422969+0000", 2024-09-21T11:41:14.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016249299999999999 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.423415+0000", 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00044531200000000001 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.423790+0000", 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037546699999999999 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424108+0000", 2024-09-21T11:41:14.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000317585 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424154+0000", 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6019000000000001e-05 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424187+0000", 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3191999999999997e-05 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:403 4.16 4:6dc3b799:reshard::reshard.0000000006:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.424302+0000", 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.924183692, 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001646706, 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 403 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424302+0000", 2024-09-21T11:41:14.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:14.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424302+0000", 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424303+0000", 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9399999999999993e-07 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424308+0000", 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5979999999999999e-06 2024-09-21T11:41:14.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424320+0000", 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1881e-05 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424325+0000", 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.344e-06 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424372+0000", 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7147000000000001e-05 2024-09-21T11:41:14.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424461+0000", 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8442000000000005e-05 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424574+0000", 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113097 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.424933+0000", 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00035964200000000002 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.425175+0000", 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024209200000000001 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.425861+0000", 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00068539800000000004 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.425909+0000", 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8389e-05 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.425949+0000", 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9681999999999997e-05 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:433 4.6 4:64760185:::obj_delete_at_hint.0000000010:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.433521+0000", 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.914964338000004, 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012646529999999999, 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 433 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.433521+0000", 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.433521+0000", 2024-09-21T11:41:14.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.433523+0000", 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.353e-06 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.433528+0000", 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.344e-06 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.433540+0000", 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1936000000000001e-05 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.433546+0000", 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6640000000000003e-06 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.433605+0000", 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8915000000000001e-05 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.433685+0000", 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0227000000000003e-05 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.433824+0000", 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013930799999999999 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.434448+0000", 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00062368999999999999 2024-09-21T11:41:14.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.434572+0000", 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012359799999999999 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.434692+0000", 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120124 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.434745+0000", 2024-09-21T11:41:14.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3368000000000001e-05 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.434786+0000", 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1125999999999999e-05 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:440 4.6 4:64760185:::obj_delete_at_hint.0000000010:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.435320+0000", 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.913165750999994, 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014456849999999999, 2024-09-21T11:41:14.452 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 440 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.435320+0000", 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.435320+0000", 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.435321+0000", 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.435325+0000", 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8909999999999997e-06 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.435335+0000", 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6849999999999996e-06 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.435342+0000", 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0110000000000003e-06 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.435398+0000", 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6567999999999998e-05 2024-09-21T11:41:14.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.435491+0000", 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3374000000000004e-05 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.435680+0000", 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018862800000000001 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.436245+0000", 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056488299999999997 2024-09-21T11:41:14.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.436525+0000", 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028012399999999998 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.436662+0000", 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013719000000000001 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.436719+0000", 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7067000000000001e-05 2024-09-21T11:41:14.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.436766+0000", 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6390000000000001e-05 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:450 4.3 4:c623b173:gc::gc.16:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.438549+0000", 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.909936830999996, 2024-09-21T11:41:14.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014372219999999999, 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 450 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.438549+0000", 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.438549+0000", 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.438550+0000", 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.004e-06 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.438554+0000", 2024-09-21T11:41:14.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1239999999999998e-06 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.438571+0000", 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6639000000000001e-05 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.438579+0000", 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2630000000000003e-06 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.438637+0000", 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7967999999999998e-05 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.438723+0000", 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6187999999999995e-05 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.438893+0000", 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016988699999999999 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.439290+0000", 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00039667700000000001 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.439455+0000", 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016539899999999999 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.439883+0000", 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000428162 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.439951+0000", 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7498999999999999e-05 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.439986+0000", 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5411999999999999e-05 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:454 4.16 4:6849fd06:gc::gc.17:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.464 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.440223+0000", 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.908263141000006, 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001329897, 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 454 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.440223+0000", 2024-09-21T11:41:14.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.440223+0000", 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.440224+0000", 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0389999999999999e-06 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.440228+0000", 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8190000000000002e-06 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.440238+0000", 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0832999999999999e-05 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.440246+0000", 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7719999999999994e-06 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.440318+0000", 2024-09-21T11:41:14.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1644999999999996e-05 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.440410+0000", 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2307999999999998e-05 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.440597+0000", 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018653699999999999 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.440932+0000", 2024-09-21T11:41:14.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00033564899999999998 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.441352+0000", 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042000400000000003 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.441458+0000", 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010597799999999999 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.441515+0000", 2024-09-21T11:41:14.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6285000000000001e-05 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.441553+0000", 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8028000000000003e-05 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:466 4.3 4:c57bc89b:::obj_delete_at_hint.0000000013:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.443303+0000", 2024-09-21T11:41:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.905182378999996, 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001271681, 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 466 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443303+0000", 2024-09-21T11:41:14.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443303+0000", 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443304+0000", 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2399999999999996e-07 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443309+0000", 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6199999999999998e-06 2024-09-21T11:41:14.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443321+0000", 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2140000000000001e-05 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443328+0000", 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2099999999999996e-06 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443390+0000", 2024-09-21T11:41:14.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1470999999999995e-05 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443500+0000", 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011031600000000001 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443666+0000", 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016540599999999999 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.444038+0000", 2024-09-21T11:41:14.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000372796 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.444340+0000", 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030188300000000002 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.444474+0000", 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000133454 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.444535+0000", 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1352999999999993e-05 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.444575+0000", 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0108000000000003e-05 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:467 4.16 4:6849fd06:gc::gc.17:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.443371+0000", 2024-09-21T11:41:14.476 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.905115116999994, 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001744911, 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 467 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443371+0000", 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443371+0000", 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443372+0000", 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6800000000000009e-07 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443376+0000", 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2719999999999999e-06 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443384+0000", 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0989999999999999e-06 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443390+0000", 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1680000000000001e-06 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443438+0000", 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7772000000000003e-05 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443501+0000", 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2830000000000007e-05 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.443660+0000", 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000159149 2024-09-21T11:41:14.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.444160+0000", 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00049983199999999997 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.444288+0000", 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012821800000000001 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445014+0000", 2024-09-21T11:41:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072601299999999996 2024-09-21T11:41:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445075+0000", 2024-09-21T11:41:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1472999999999999e-05 2024-09-21T11:41:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445116+0000", 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0117000000000002e-05 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:471 4.3 4:c57bc89b:::obj_delete_at_hint.0000000013:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.445083+0000", 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.903402791000005, 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013426530000000001, 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 471 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445083+0000", 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445083+0000", 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445084+0000", 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.046e-06 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445089+0000", 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7840000000000003e-06 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445100+0000", 2024-09-21T11:41:14.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0923e-05 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445109+0000", 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5829999999999996e-06 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445180+0000", 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0487999999999993e-05 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445277+0000", 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7548999999999997e-05 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445453+0000", 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000175346 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.445991+0000", 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00053831400000000002 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.446111+0000", 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119536 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.446330+0000", 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021978699999999999 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.446386+0000", 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5877000000000001e-05 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.446426+0000", 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9419999999999999e-05 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14445.0:556 4.b 4:d201668e:::obj_delete_at_hint.0000000023:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:40:07.473048+0000", 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 66.875437792, 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001240501, 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:14.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14445", 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 556 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473048+0000", 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:14.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473048+0000", 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473049+0000", 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3600000000000003e-07 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473052+0000", 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.49e-06 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:14.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473060+0000", 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.276e-06 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473065+0000", 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7690000000000004e-06 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473113+0000", 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7329e-05 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473197+0000", 2024-09-21T11:41:14.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4115000000000004e-05 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473378+0000", 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018112799999999999 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473779+0000", 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00040118400000000001 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.473976+0000", 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019726699999999999 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.474158+0000", 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018191299999999999 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.474213+0000", 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4580999999999998e-05 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:40:07.474289+0000", 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5813000000000001e-05 2024-09-21T11:41:14.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:14.495 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:14.495 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:41:14.503 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:41:14.503 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:41:14.503 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:41:14.503 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:41:14.504 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:41:14.504 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:41:14.504 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:41:14.504 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:41:14.504 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:41:14.504 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:41:14.504 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:41:14.504 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:41:14.505 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:41:14.505 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:41:14.505 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:41:14.505 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:41:14.517 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:14.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:14.518 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:14.619 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:14.641 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:14.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:14.642 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:14.701 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:41:14.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:41:14.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:14.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2774884108' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:41:14.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3107055029' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:41:14.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[36454]: osdmap e85: 8 total, 8 up, 8 in 2024-09-21T11:41:14.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/699081690' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:14.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:41:14.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:41:14.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:14.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2774884108' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:41:14.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3107055029' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:41:14.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[29710]: osdmap e85: 8 total, 8 up, 8 in 2024-09-21T11:41:14.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/699081690' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:14.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:14 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:41:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:14 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:41:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:14 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2774884108' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:41:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3107055029' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:41:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:14 smithi175 ceph-mon[32613]: osdmap e85: 8 total, 8 up, 8 in 2024-09-21T11:41:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/699081690' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:14.766 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:14.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:14.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:14.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:14.889 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:14.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:14.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:14.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:15.014 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:15.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:15.015 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:15.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:15.140 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:15.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:15.141 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:15.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:15.264 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:15.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:15.265 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:15.309 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:41:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:41:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:41:15.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:15.388 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:15.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:15.389 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:15.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:15.514 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:15.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:15.515 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:15.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:15.642 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:15.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:15.643 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:15.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:15 smithi043 ceph-mon[36454]: pgmap v37: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:41:15.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:15 smithi043 ceph-mon[36454]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:41:15.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:15 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:15.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:15 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:15.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:15 smithi043 ceph-mon[29710]: pgmap v37: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:41:15.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:15 smithi043 ceph-mon[29710]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:41:15.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:15 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:15.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:15 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:15.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:15.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:15 smithi175 ceph-mon[32613]: pgmap v37: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:41:15.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:15 smithi175 ceph-mon[32613]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:41:15.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:15 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:15.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:15 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:15.770 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:15.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:15.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:15.871 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:15.897 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:15.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:15.897 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:15.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:15 smithi043 bash[69916]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-21T11:41:15.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:16.023 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:16.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:16.024 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:16.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:16.151 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:16.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:16.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:16.252 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:16.278 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:16.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:16.279 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:16.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:16.407 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:16.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:16.407 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:16.491 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:16 smithi043 bash[69916]: Writing manifest to image destination 2024-09-21T11:41:16.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:16.534 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:16.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:16.535 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:16.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:16.663 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:16.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:16.664 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:16.765 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:16.791 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:16.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:16.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:16.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:16.920 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:16.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:16.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:16.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:16 smithi043 podman[69916]: 2024-09-21 11:41:16.613364736 +0000 UTC m=+3.591638091 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-21T11:41:17.022 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:17.048 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:17.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:17.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:17.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:17.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:17.175 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:17.175 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:17.276 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:17.302 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:17.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:17.303 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:17.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:17.428 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:17.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:17.429 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:17.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:17.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:17.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:17.636 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:17 smithi043 ceph-mon[36454]: pgmap v38: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:41:17.636 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:17 smithi043 ceph-mon[36454]: osdmap e86: 8 total, 8 up, 8 in 2024-09-21T11:41:17.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:17 smithi043 ceph-mon[29710]: pgmap v38: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:41:17.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:17 smithi043 ceph-mon[29710]: osdmap e86: 8 total, 8 up, 8 in 2024-09-21T11:41:17.655 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:17.680 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:17.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:17.681 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:17.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:17 smithi175 ceph-mon[32613]: pgmap v38: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:41:17.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:17 smithi175 ceph-mon[32613]: osdmap e86: 8 total, 8 up, 8 in 2024-09-21T11:41:17.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:17.807 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:17.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:17.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:17.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:17.934 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:17.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:17.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:18.035 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:18.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:41:18.057 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:18.060 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:18.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:18.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:18.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:18.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:18.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:18.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:18.302 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 podman[69916]: 2024-09-21 11:41:18.1765615 +0000 UTC m=+5.154834853 container create 984bd8ee5471ac8ada2dd1e5b57c1a07cfc27478d14878a8692b56b6eabbe77d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T11:41:18.311 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:18.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:18.312 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:18.411 DEBUG:teuthology.orchestra.run.smithi043:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-21T11:41:18.413 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:18.414 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-21T11:41:18.414 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-21T11:41:18.439 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:18.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:18.440 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:18.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:18.567 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:18.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:18.568 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:18.643 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:hints = 1 2024-09-21T11:41:18.643 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-21T11:41:18.643 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Object prefix: benchmark_data_smithi043_70646 2024-09-21T11:41:18.644 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:41:18.644 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 0 0 0 0 0 0 - 0 2024-09-21T11:41:18.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:18.709 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:18.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:18.710 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:18.743 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 podman[69916]: 2024-09-21 11:41:18.623524273 +0000 UTC m=+5.601797639 container init 984bd8ee5471ac8ada2dd1e5b57c1a07cfc27478d14878a8692b56b6eabbe77d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T11:41:18.743 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 podman[69916]: 2024-09-21 11:41:18.626967478 +0000 UTC m=+5.605240832 container start 984bd8ee5471ac8ada2dd1e5b57c1a07cfc27478d14878a8692b56b6eabbe77d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T11:41:18.743 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.630Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-21T11:41:18.743 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.630Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.630Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.630Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=arp 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-21T11:41:18.744 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=edac 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-21T11:41:18.745 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=os 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=stat 2024-09-21T11:41:18.746 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-21T11:41:18.747 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-21T11:41:18.747 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-21T11:41:18.747 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=time 2024-09-21T11:41:18.747 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-21T11:41:18.747 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=uname 2024-09-21T11:41:18.747 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.631Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-21T11:41:18.747 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.632Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-21T11:41:18.747 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.632Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-21T11:41:18.747 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.632Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-21T11:41:18.747 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a[70621]: ts=2024-09-21T11:41:18.632Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-21T11:41:18.811 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:18.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:18.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:18.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:18.981 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:18.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:18.984 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:19.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:41:19.071 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 bash[69916]: 984bd8ee5471ac8ada2dd1e5b57c1a07cfc27478d14878a8692b56b6eabbe77d 2024-09-21T11:41:19.072 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 11:41:18 smithi043 systemd[1]: Started Ceph node-exporter.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:41:19.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:19.113 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:19.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:19.114 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:19.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:19.255 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:19.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:19.256 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:19.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:19.410 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:19.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:19.411 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:19.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:19 smithi175 ceph-mon[32613]: pgmap v40: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 799 B/s rd, 0 op/s 2024-09-21T11:41:19.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:19 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:19.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:19 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:19.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:19 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:19.511 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:19.514 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:19.515 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:41:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:41:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:41:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:41:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:41:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:41:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:41:19.516 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:41:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:41:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:41:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:41:19.517 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:41:19.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:41:19.542 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:19.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:19.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:19.589 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:19 smithi043 ceph-mon[29710]: pgmap v40: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 799 B/s rd, 0 op/s 2024-09-21T11:41:19.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:19 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:19.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:19 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:19.590 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:19 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:19.590 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:19 smithi043 ceph-mon[36454]: pgmap v40: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 799 B/s rd, 0 op/s 2024-09-21T11:41:19.590 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:19 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:19.590 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:19 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:19.590 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:19 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:19.644 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 1 16 2872 2856 178.49 178.5 0.00535423 0.00557942 2024-09-21T11:41:19.644 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:19.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:19.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:19.777 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:19.809 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:19.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:19.810 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:19.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:19.953 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:19.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:19.954 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:20.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:20.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:20.095 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:20.095 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:20.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:20.240 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:20.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:20.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:20.243 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:41:20.243 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:41:20.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.1 'trim stale osdmaps' 2024-09-21T11:41:20.342 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:20.343 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:20.383 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:20 smithi175 ceph-mon[32613]: Deploying daemon node-exporter.b on smithi175 2024-09-21T11:41:20.383 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:20 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:20.396 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:20.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:20.398 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:20.459 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:20 smithi043 ceph-mon[29710]: Deploying daemon node-exporter.b on smithi175 2024-09-21T11:41:20.459 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:20 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:20.460 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:20 smithi043 ceph-mon[36454]: Deploying daemon node-exporter.b on smithi175 2024-09-21T11:41:20.460 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:20 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:20.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:20.561 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:20.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:20.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:20.644 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 2 16 5694 5678 177.412 176.375 0.00604561 0.00562162 2024-09-21T11:41:20.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:20.708 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:20.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:20.709 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:20.734 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:20 smithi175 bash[62305]: Trying to pull quay.io/prometheus/node-exporter:v1.7.0... 2024-09-21T11:41:20.734 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:20 smithi175 bash[62305]: Getting image source signatures 2024-09-21T11:41:20.734 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:20 smithi175 bash[62305]: Copying blob sha256:324153f2810a9927fcce320af9e4e291e0b6e805cbdd1f338386c756b9defa24 2024-09-21T11:41:20.734 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:20 smithi175 bash[62305]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-21T11:41:20.734 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:20 smithi175 bash[62305]: Copying blob sha256:2abcce694348cd2c949c0e98a7400ebdfd8341021bcf6b541bc72033ce982510 2024-09-21T11:41:20.810 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:20.844 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:20.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:20.845 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:20.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:20.984 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:20.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:20.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:21.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:41:20 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:41:21.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:21.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:21.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:21.231 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:21.235 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:41:21.235 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:6802 7.b 7:d18d1ee3:::benchmark_data_smithi043_70646_object6801:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:21.040853+0000", 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0016554619999999999, 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001689367, 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 6802 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:21.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.040853+0000", 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.040853+0000", 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.040854+0000", 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.367e-06 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.040878+0000", 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3637000000000001e-05 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.040889+0000", 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1746000000000001e-05 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.040908+0000", 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9049e-05 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.040965+0000", 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6236000000000002e-05 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.041075+0000", 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010988 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.041206+0000", 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013112399999999999 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:21.239 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:6803 7.4 e87/82 7:21b884b8:::benchmark_data_smithi043_70646_object6802:head v 87'401, mlcod=87'399)", 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:21.042049+0000", 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00045899999999999999, 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056416500000000004, 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:21.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042049+0000", 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042049+0000", 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042050+0000", 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.029e-06 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042087+0000", 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6452000000000003e-05 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042101+0000", 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4333e-05 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042108+0000", 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.216e-06 2024-09-21T11:41:21.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042176+0000", 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7639999999999996e-05 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042202+0000", 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5936e-05 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.243 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:6802 7.b e87/82 ondisk, result = 0)", 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:21.042550+0000", 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9999585, 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107068, 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042550+0000", 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:21.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042550+0000", 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042550+0000", 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5700000000000002e-07 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042555+0000", 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3769999999999998e-06 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:21.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042566+0000", 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1600000000000001e-05 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042573+0000", 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3210000000000001e-06 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042638+0000", 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5609000000000002e-05 2024-09-21T11:41:21.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042645+0000", 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.905e-06 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.042657+0000", 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1499e-05 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 3 2024-09-21T11:41:21.247 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:41:21.263 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:21.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:21.264 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:21.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:21.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:21.408 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:21.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:21.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:21.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:21.644 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 3 16 8559 8543 177.957 179.062 0.00370604 0.00561095 2024-09-21T11:41:21.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:21.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:21.692 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:21.692 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:21 smithi043 ceph-mon[36454]: pgmap v41: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:41:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:21 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:41:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:21 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:41:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:21 smithi043 ceph-mon[36454]: osdmap e87: 8 total, 8 up, 8 in 2024-09-21T11:41:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:21 smithi043 ceph-mon[29710]: pgmap v41: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:41:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:21 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:41:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:21 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:41:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:21 smithi043 ceph-mon[29710]: osdmap e87: 8 total, 8 up, 8 in 2024-09-21T11:41:21.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:21 smithi175 ceph-mon[32613]: pgmap v41: 148 pgs: 148 active+clean; 583 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:41:21.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:21 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:41:21.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:21 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:41:21.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:21 smithi175 ceph-mon[32613]: osdmap e87: 8 total, 8 up, 8 in 2024-09-21T11:41:21.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:21.838 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:21.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:21.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:21.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:21.971 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:21.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:21.972 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:22.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:22.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:22.114 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:22.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:22.255 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:22.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:22.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:22.357 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:22.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:41:22.385 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:22.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:22.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:22.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:22.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:22.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-mon[32613]: osdmap e88: 8 total, 8 up, 8 in 2024-09-21T11:41:22.511 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 bash[62305]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-21T11:41:22.511 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 bash[62305]: Writing manifest to image destination 2024-09-21T11:41:22.515 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:22.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:22.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:22.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:22.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:22 smithi043 ceph-mon[29710]: osdmap e88: 8 total, 8 up, 8 in 2024-09-21T11:41:22.612 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:22.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:22 smithi043 ceph-mon[36454]: osdmap e88: 8 total, 8 up, 8 in 2024-09-21T11:41:22.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:22.644 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 4 16 11363 11347 177.275 175.25 0.00647574 0.00563142 2024-09-21T11:41:22.667 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:22.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:22.668 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:22.752 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:22.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:22.830 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 podman[62305]: 2024-09-21 11:41:22.564857086 +0000 UTC m=+2.328945892 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-21T11:41:22.830 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 podman[62305]: 2024-09-21 11:41:22.687889186 +0000 UTC m=+2.451977982 container create 9d48face5bd78948b1ec07fac1e0d5b737bd8c49eccf36b19a563b06f7a42a86 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T11:41:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:22.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:22.838 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:22.938 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:22.971 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:22.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:22.972 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:23.072 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:23.100 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:23.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:23.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:23.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:23.230 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:23.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:23.231 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:23.261 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 podman[62305]: 2024-09-21 11:41:22.940962436 +0000 UTC m=+2.705051229 container init 9d48face5bd78948b1ec07fac1e0d5b737bd8c49eccf36b19a563b06f7a42a86 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 podman[62305]: 2024-09-21 11:41:22.94600835 +0000 UTC m=+2.710097138 container start 9d48face5bd78948b1ec07fac1e0d5b737bd8c49eccf36b19a563b06f7a42a86 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.950Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.950Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.951Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.951Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=arp 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-21T11:41:23.262 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-21T11:41:23.263 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-21T11:41:23.263 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-21T11:41:23.263 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-21T11:41:23.263 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-21T11:41:23.263 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=edac 2024-09-21T11:41:23.263 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-21T11:41:23.263 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-21T11:41:23.264 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=os 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=stat 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=time 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.953Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.954Z caller=node_exporter.go:117 level=info collector=uname 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.954Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-21T11:41:23.265 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.954Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-21T11:41:23.266 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.954Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-21T11:41:23.266 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.954Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-21T11:41:23.266 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b[62460]: ts=2024-09-21T11:41:22.954Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-21T11:41:23.266 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:23 smithi175 bash[62305]: 9d48face5bd78948b1ec07fac1e0d5b737bd8c49eccf36b19a563b06f7a42a86 2024-09-21T11:41:23.266 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 11:41:23 smithi175 systemd[1]: Started Ceph node-exporter.b for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:41:23.332 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:23.361 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:23.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:23.362 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:23.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:23.495 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:23.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:23.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:23.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:23.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: pgmap v43: 148 pgs: 148 active+clean; 366 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 46 MiB/s wr, 731 op/s 2024-09-21T11:41:23.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:41:23.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:41:23.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: osdmap e89: 8 total, 8 up, 8 in 2024-09-21T11:41:23.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2683301778' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: pgmap v43: 148 pgs: 148 active+clean; 366 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 46 MiB/s wr, 731 op/s 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: osdmap e89: 8 total, 8 up, 8 in 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2683301778' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:41:23.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:23 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:41:23.645 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 5 16 13519 13503 168.738 134.75 0.00520058 0.00591719 2024-09-21T11:41:23.650 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:23.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:23.651 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:23.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: pgmap v43: 148 pgs: 148 active+clean; 366 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 46 MiB/s wr, 731 op/s 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: osdmap e89: 8 total, 8 up, 8 in 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2683301778' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:41:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:23 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:41:23.796 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:23.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:23.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:23.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:23.928 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:23.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:23.929 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:24.029 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:24.060 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:24.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:24.061 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:24.142 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:41:24.142 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:41:24.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:41:24.185 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:41:24.191 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:41:24.206 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:24.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:24.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:24.229 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:41:24.229 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:41:24.269 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:41:24.269 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:41:24.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:24.309 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:41:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:41:24.331 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:24.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:24.332 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:24.333 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:41:24.333 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:41:24.373 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:41:24.373 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:41:24.409 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:41:24.410 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:41:24.433 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:24.462 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:24.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:24.463 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:24.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:24 smithi043 ceph-mon[36454]: pgmap v46: 148 pgs: 148 active+clean; 366 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 974 op/s 2024-09-21T11:41:24.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:24 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3961166510' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:24.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:24 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:24.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:24 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2683301778' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:41:24.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:24 smithi043 ceph-mon[36454]: osdmap e90: 8 total, 8 up, 8 in 2024-09-21T11:41:24.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:24 smithi043 ceph-mon[29710]: pgmap v46: 148 pgs: 148 active+clean; 366 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 974 op/s 2024-09-21T11:41:24.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:24 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3961166510' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:24.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:24 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:24.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:24 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2683301778' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:41:24.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:24 smithi043 ceph-mon[29710]: osdmap e90: 8 total, 8 up, 8 in 2024-09-21T11:41:24.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:24.596 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:24.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:24.597 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:24.645 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 6 16 15989 15973 166.342 154.375 0.00532501 0.00600472 2024-09-21T11:41:24.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:24.739 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:24.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:24.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:24.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:24 smithi175 ceph-mon[32613]: pgmap v46: 148 pgs: 148 active+clean; 366 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 974 op/s 2024-09-21T11:41:24.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:24 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3961166510' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:24.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:24 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:24.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:24 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2683301778' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:41:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:24 smithi175 ceph-mon[32613]: osdmap e90: 8 total, 8 up, 8 in 2024-09-21T11:41:24.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:24.874 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:25.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:25.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:25.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:25.366 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:25.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:25.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:25.467 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:25.490 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:41:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:41:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:41:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:25 smithi043 ceph-mon[29710]: 5.b scrub starts 2024-09-21T11:41:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:25 smithi043 ceph-mon[29710]: 5.b scrub ok 2024-09-21T11:41:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:25 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:25 smithi043 ceph-mon[36454]: 5.b scrub starts 2024-09-21T11:41:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:25 smithi043 ceph-mon[36454]: 5.b scrub ok 2024-09-21T11:41:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:25 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:25.503 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:25.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:25.504 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:25.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:25.636 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:25.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:25.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:25.645 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 7 16 18424 18408 164.318 152.188 0.00788347 0.00607904 2024-09-21T11:41:25.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:25.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:25 smithi175 ceph-mon[32613]: 5.b scrub starts 2024-09-21T11:41:25.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:25 smithi175 ceph-mon[32613]: 5.b scrub ok 2024-09-21T11:41:25.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:25 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:25.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:25.777 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:25.777 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:25.877 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:25.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:25.905 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:25.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:25.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:25.968 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:41:25.968 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:41:25.968 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:41:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:41:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:41:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:41:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:41:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:41:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:41:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:41:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:41:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:41:26.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:26.052 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:26.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:26.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:26.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:26.191 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:26.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:26.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:26.293 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:26.329 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:26.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:26.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:26.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:26.461 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:26.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:26.461 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:26.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:26.598 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:26.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:26.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:26.645 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 8 15 20950 20935 163.519 157.938 0.00726482 0.00610885 2024-09-21T11:41:26.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:26 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:41:26.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:26 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:41:26.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:26 smithi175 ceph-mon[32613]: 4.17 scrub starts 2024-09-21T11:41:26.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:26 smithi175 ceph-mon[32613]: 4.17 scrub ok 2024-09-21T11:41:26.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:26 smithi175 ceph-mon[32613]: pgmap v48: 148 pgs: 148 active+clean; 561 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 954 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-21T11:41:26.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:26.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:26 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:41:26.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:26 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:41:26.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:26 smithi043 ceph-mon[36454]: 4.17 scrub starts 2024-09-21T11:41:26.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:26 smithi043 ceph-mon[36454]: 4.17 scrub ok 2024-09-21T11:41:26.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:26 smithi043 ceph-mon[36454]: pgmap v48: 148 pgs: 148 active+clean; 561 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 954 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-21T11:41:26.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:26 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:41:26.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:26 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:41:26.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:26 smithi043 ceph-mon[29710]: 4.17 scrub starts 2024-09-21T11:41:26.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:26 smithi043 ceph-mon[29710]: 4.17 scrub ok 2024-09-21T11:41:26.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:26 smithi043 ceph-mon[29710]: pgmap v48: 148 pgs: 148 active+clean; 561 MiB data, 1.9 GiB used, 713 GiB / 715 GiB avail; 954 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-21T11:41:26.727 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:26.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:26.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:26.774 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:41:26.774 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:41:26.774 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.2 'trim stale osdmaps' 2024-09-21T11:41:26.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:26.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:26.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:26.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:27.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:27.038 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:27.139 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:27.168 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:27.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:27.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:27.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:27.299 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:27.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:27.300 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:27.401 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:27.424 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:27.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:27.425 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:27.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:41:27.526 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:27.553 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:27.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:27.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:27.645 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 9 16 22928 22912 159.079 123.562 0.0028402 0.0062772 2024-09-21T11:41:27.655 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:27.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:27.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:27 smithi043 ceph-mon[29710]: 4.18 scrub starts 2024-09-21T11:41:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:27 smithi043 ceph-mon[29710]: 4.18 scrub ok 2024-09-21T11:41:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:27 smithi043 ceph-mon[36454]: 4.18 scrub starts 2024-09-21T11:41:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:27 smithi043 ceph-mon[36454]: 4.18 scrub ok 2024-09-21T11:41:27.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:27 smithi175 ceph-mon[32613]: 4.18 scrub starts 2024-09-21T11:41:27.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:27 smithi175 ceph-mon[32613]: 4.18 scrub ok 2024-09-21T11:41:27.778 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:27.812 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:27.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:27.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:27.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:41:27.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:27.939 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:27.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:27.940 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:28.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:28.101 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:28.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:28.102 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:28.203 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:28.227 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:28.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:28.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:28.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:28.351 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:28.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:28.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:28.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:28.492 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:28.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:28.493 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:28.533 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:28 smithi043 ceph-mon[29710]: 6.b scrub starts 2024-09-21T11:41:28.533 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:28 smithi043 ceph-mon[29710]: 6.b scrub ok 2024-09-21T11:41:28.533 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:28 smithi043 ceph-mon[29710]: pgmap v49: 148 pgs: 148 active+clean; 1.1 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 132 MiB/s wr, 2.12k op/s 2024-09-21T11:41:28.533 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:28 smithi043 ceph-mon[29710]: 6.1e scrub starts 2024-09-21T11:41:28.533 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:28 smithi043 ceph-mon[29710]: 6.1e scrub ok 2024-09-21T11:41:28.534 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:28 smithi043 ceph-mon[36454]: 6.b scrub starts 2024-09-21T11:41:28.534 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:28 smithi043 ceph-mon[36454]: 6.b scrub ok 2024-09-21T11:41:28.534 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:28 smithi043 ceph-mon[36454]: pgmap v49: 148 pgs: 148 active+clean; 1.1 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 132 MiB/s wr, 2.12k op/s 2024-09-21T11:41:28.534 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:28 smithi043 ceph-mon[36454]: 6.1e scrub starts 2024-09-21T11:41:28.534 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:28 smithi043 ceph-mon[36454]: 6.1e scrub ok 2024-09-21T11:41:28.534 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:41:28.534 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:41:28.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:28.638 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:28.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:28.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:28.645 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 10 16 24832 24816 155.071 119 0.00771974 0.00644288 2024-09-21T11:41:28.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:28.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:28 smithi175 ceph-mon[32613]: 6.b scrub starts 2024-09-21T11:41:28.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:28 smithi175 ceph-mon[32613]: 6.b scrub ok 2024-09-21T11:41:28.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:28 smithi175 ceph-mon[32613]: pgmap v49: 148 pgs: 148 active+clean; 1.1 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 132 MiB/s wr, 2.12k op/s 2024-09-21T11:41:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:28 smithi175 ceph-mon[32613]: 6.1e scrub starts 2024-09-21T11:41:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:28 smithi175 ceph-mon[32613]: 6.1e scrub ok 2024-09-21T11:41:28.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:28.782 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:28.782 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:28.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:28.912 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:28.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:28.913 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:29.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:29.052 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:29.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:29.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:29.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:29.176 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:29.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:29.177 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:29.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:29.307 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:29.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:29.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:29.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:29.448 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:29.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:29.450 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:29.550 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:29.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:29.589 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:29.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:29.646 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 11 16 26681 26665 151.478 115.562 0.00978919 0.00659516 2024-09-21T11:41:29.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:29.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:29.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:29.833 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:29.861 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:29.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:29.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:29.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[36454]: 5.11 scrub starts 2024-09-21T11:41:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[36454]: 5.11 scrub ok 2024-09-21T11:41:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T11:41:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T11:41:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[29710]: 5.11 scrub starts 2024-09-21T11:41:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[29710]: 5.11 scrub ok 2024-09-21T11:41:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T11:41:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:29 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T11:41:29.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:29.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:29.993 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:30.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:29 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:29 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:29 smithi175 ceph-mon[32613]: 5.11 scrub starts 2024-09-21T11:41:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:29 smithi175 ceph-mon[32613]: 5.11 scrub ok 2024-09-21T11:41:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:29 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T11:41:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:29 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T11:41:30.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:30.120 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:30.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:30.121 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:30.222 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:30.267 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:30.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:30.268 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:30.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:30.408 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:30.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:30.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:30.510 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:30.538 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:30.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:30.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:30.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:30.646 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 12 16 28713 28697 149.438 127 0.00800519 0.00668608 2024-09-21T11:41:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:30.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:30.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:30.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:30.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:30.804 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:30.805 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:30.905 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:30.926 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:30.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:30.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: 3.4 scrub starts 2024-09-21T11:41:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: 6.10 scrub starts 2024-09-21T11:41:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: 3.4 scrub ok 2024-09-21T11:41:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: 6.10 scrub ok 2024-09-21T11:41:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: 5.6 scrub starts 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: 5.6 scrub ok 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: 3.7 scrub starts 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: 3.7 scrub ok 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: pgmap v50: 148 pgs: 148 active+clean; 1.1 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 674 B/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: 3.a scrub starts 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: 3.a scrub ok 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-21T11:41:30.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:41:30.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: 3.4 scrub starts 2024-09-21T11:41:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: 6.10 scrub starts 2024-09-21T11:41:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: 3.4 scrub ok 2024-09-21T11:41:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: 6.10 scrub ok 2024-09-21T11:41:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: 5.6 scrub starts 2024-09-21T11:41:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: 5.6 scrub ok 2024-09-21T11:41:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: 3.7 scrub starts 2024-09-21T11:41:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: 3.7 scrub ok 2024-09-21T11:41:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: pgmap v50: 148 pgs: 148 active+clean; 1.1 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 674 B/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-21T11:41:30.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: 3.a scrub starts 2024-09-21T11:41:30.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:41:30.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: 3.a scrub ok 2024-09-21T11:41:30.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-21T11:41:30.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:41:30.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:41:30.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-21T11:41:30.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:41:30.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:31.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:41:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: 3.4 scrub starts 2024-09-21T11:41:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: 6.10 scrub starts 2024-09-21T11:41:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: 3.4 scrub ok 2024-09-21T11:41:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: 6.10 scrub ok 2024-09-21T11:41:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: 5.6 scrub starts 2024-09-21T11:41:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: 5.6 scrub ok 2024-09-21T11:41:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: 3.7 scrub starts 2024-09-21T11:41:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: 3.7 scrub ok 2024-09-21T11:41:31.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: pgmap v50: 148 pgs: 148 active+clean; 1.1 GiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 674 B/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-21T11:41:31.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: 3.a scrub starts 2024-09-21T11:41:31.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:41:31.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: 3.a scrub ok 2024-09-21T11:41:31.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-21T11:41:31.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]} : dispatch 2024-09-21T11:41:31.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:41:31.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]} : dispatch 2024-09-21T11:41:31.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:41:31.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:31.029 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:31.057 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:31.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:31.058 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:31.159 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:31.180 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:31.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:31.181 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:31.282 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:31.309 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:31.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:31.310 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:31.411 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:31.440 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:31.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:31.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:31.542 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:31.563 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:31.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:31.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:31.646 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 13 15 30183 30168 145.015 91.9375 0.00681127 0.00688903 2024-09-21T11:41:31.664 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:31.692 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:31.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:31.693 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:31.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:31.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:31.846 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:31.947 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[29710]: 6.16 scrub starts 2024-09-21T11:41:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[29710]: 6.16 scrub ok 2024-09-21T11:41:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[29710]: 2.1 scrub starts 2024-09-21T11:41:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[29710]: 2.1 scrub ok 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[29710]: osdmap e91: 8 total, 8 up, 8 in 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[36454]: 6.16 scrub starts 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[36454]: 6.16 scrub ok 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[36454]: 2.1 scrub starts 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[36454]: 2.1 scrub ok 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-21T11:41:31.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-21T11:41:31.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-21T11:41:31.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[36454]: osdmap e91: 8 total, 8 up, 8 in 2024-09-21T11:41:31.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:31 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:31.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:31.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:31 smithi175 ceph-mon[32613]: 6.16 scrub starts 2024-09-21T11:41:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:31 smithi175 ceph-mon[32613]: 6.16 scrub ok 2024-09-21T11:41:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:31 smithi175 ceph-mon[32613]: 2.1 scrub starts 2024-09-21T11:41:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:31 smithi175 ceph-mon[32613]: 2.1 scrub ok 2024-09-21T11:41:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-21T11:41:32.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-21T11:41:32.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-21T11:41:32.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:31 smithi175 ceph-mon[32613]: osdmap e91: 8 total, 8 up, 8 in 2024-09-21T11:41:32.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:31 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:32.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:32.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:32.124 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:32.124 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:32.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:32.272 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:32.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:32.273 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:32.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:32.415 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:32.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:32.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:32.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:32.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:32.547 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:32.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:32.646 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 14 16 32197 32181 143.642 125.812 0.00363029 0.00695508 2024-09-21T11:41:32.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:32.688 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:32.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:32.689 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:32.735 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:32.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:41:32.790 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:32.818 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:32.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:32.887 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:32.887 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:32.935 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[36454]: 5.1e scrub starts 2024-09-21T11:41:32.936 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[36454]: 6.4 scrub starts 2024-09-21T11:41:32.936 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[36454]: 5.1e scrub ok 2024-09-21T11:41:32.936 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[36454]: 6.4 scrub ok 2024-09-21T11:41:32.936 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[36454]: 5.d scrub starts 2024-09-21T11:41:32.936 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[36454]: 5.d scrub ok 2024-09-21T11:41:32.936 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[36454]: osdmap e92: 8 total, 8 up, 8 in 2024-09-21T11:41:32.936 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[36454]: pgmap v53: 148 pgs: 3 peering, 145 active+clean; 1.8 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.3 KiB/s rd, 191 MiB/s wr, 3.06k op/s 2024-09-21T11:41:32.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[29710]: 5.1e scrub starts 2024-09-21T11:41:32.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[29710]: 6.4 scrub starts 2024-09-21T11:41:32.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[29710]: 5.1e scrub ok 2024-09-21T11:41:32.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[29710]: 6.4 scrub ok 2024-09-21T11:41:32.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[29710]: 5.d scrub starts 2024-09-21T11:41:32.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[29710]: 5.d scrub ok 2024-09-21T11:41:32.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[29710]: osdmap e92: 8 total, 8 up, 8 in 2024-09-21T11:41:32.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:32 smithi043 ceph-mon[29710]: pgmap v53: 148 pgs: 3 peering, 145 active+clean; 1.8 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.3 KiB/s rd, 191 MiB/s wr, 3.06k op/s 2024-09-21T11:41:32.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:33.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:32 smithi175 ceph-mon[32613]: 5.1e scrub starts 2024-09-21T11:41:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:32 smithi175 ceph-mon[32613]: 6.4 scrub starts 2024-09-21T11:41:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:32 smithi175 ceph-mon[32613]: 5.1e scrub ok 2024-09-21T11:41:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:32 smithi175 ceph-mon[32613]: 6.4 scrub ok 2024-09-21T11:41:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:32 smithi175 ceph-mon[32613]: 5.d scrub starts 2024-09-21T11:41:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:32 smithi175 ceph-mon[32613]: 5.d scrub ok 2024-09-21T11:41:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:32 smithi175 ceph-mon[32613]: osdmap e92: 8 total, 8 up, 8 in 2024-09-21T11:41:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:32 smithi175 ceph-mon[32613]: pgmap v53: 148 pgs: 3 peering, 145 active+clean; 1.8 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.3 KiB/s rd, 191 MiB/s wr, 3.06k op/s 2024-09-21T11:41:33.015 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:33.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:33.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:33.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:33.178 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:33.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:33.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:33.280 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:33.317 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:33.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:33.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:33.364 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:33.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:33.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:33.477 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:33.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:33.578 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:33.607 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:33.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:33.608 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:33.646 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 15 16 33872 33856 141.045 104.688 0.00768473 0.00708433 2024-09-21T11:41:33.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:41:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:41:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:41:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:41:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:41:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:41:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:41:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:41:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:41:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:41:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:41:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:41:33.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:33.753 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:33.754 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:33.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[36454]: 4.12 scrub starts 2024-09-21T11:41:33.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[36454]: 4.12 scrub ok 2024-09-21T11:41:33.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[36454]: 4.1c scrub starts 2024-09-21T11:41:33.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[36454]: 4.1c scrub ok 2024-09-21T11:41:33.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[36454]: 4.10 scrub starts 2024-09-21T11:41:33.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[36454]: 4.10 scrub ok 2024-09-21T11:41:33.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[36454]: 5.1b scrub starts 2024-09-21T11:41:33.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[36454]: 5.1b scrub ok 2024-09-21T11:41:33.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[29710]: 4.12 scrub starts 2024-09-21T11:41:33.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[29710]: 4.12 scrub ok 2024-09-21T11:41:33.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[29710]: 4.1c scrub starts 2024-09-21T11:41:33.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[29710]: 4.1c scrub ok 2024-09-21T11:41:33.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[29710]: 4.10 scrub starts 2024-09-21T11:41:33.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[29710]: 4.10 scrub ok 2024-09-21T11:41:33.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[29710]: 5.1b scrub starts 2024-09-21T11:41:33.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:33 smithi043 ceph-mon[29710]: 5.1b scrub ok 2024-09-21T11:41:33.835 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:41:33.835 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:41:33.852 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:41:33.852 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:41:33.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:33.890 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:41:33.890 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:41:33.896 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:33.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:33.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:33.932 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:41:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:41:33.965 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:41:33.966 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:41:33.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:34.001 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:41:34.002 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:41:34.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:33 smithi175 ceph-mon[32613]: 4.12 scrub starts 2024-09-21T11:41:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:33 smithi175 ceph-mon[32613]: 4.12 scrub ok 2024-09-21T11:41:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:33 smithi175 ceph-mon[32613]: 4.1c scrub starts 2024-09-21T11:41:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:33 smithi175 ceph-mon[32613]: 4.1c scrub ok 2024-09-21T11:41:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:33 smithi175 ceph-mon[32613]: 4.10 scrub starts 2024-09-21T11:41:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:33 smithi175 ceph-mon[32613]: 4.10 scrub ok 2024-09-21T11:41:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:33 smithi175 ceph-mon[32613]: 5.1b scrub starts 2024-09-21T11:41:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:33 smithi175 ceph-mon[32613]: 5.1b scrub ok 2024-09-21T11:41:34.036 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:34.036 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:41:34.037 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:41:34.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:34.038 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:34.075 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:41:34.076 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:41:34.139 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:34.163 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:34.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:34.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:34.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:34.292 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:34.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:34.293 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:34.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:34.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:34.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:34.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:34.574 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:34.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:34.575 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:34.646 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 16 16 35651 35635 139.178 111.188 0.00810424 0.0071785 2024-09-21T11:41:34.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:34.720 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:34.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:34.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:34.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:34 smithi043 ceph-mon[36454]: pgmap v54: 148 pgs: 3 peering, 145 active+clean; 1.8 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-21T11:41:34.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1529219957' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:34.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:34 smithi043 ceph-mon[36454]: 4.15 scrub starts 2024-09-21T11:41:34.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:34 smithi043 ceph-mon[36454]: 4.15 scrub ok 2024-09-21T11:41:34.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:34 smithi043 ceph-mon[29710]: pgmap v54: 148 pgs: 3 peering, 145 active+clean; 1.8 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-21T11:41:34.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1529219957' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:34.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:34 smithi043 ceph-mon[29710]: 4.15 scrub starts 2024-09-21T11:41:34.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:34 smithi043 ceph-mon[29710]: 4.15 scrub ok 2024-09-21T11:41:34.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:34.866 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:34.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:34.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:34.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:35.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:35.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:35.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:34 smithi175 ceph-mon[32613]: pgmap v54: 148 pgs: 3 peering, 145 active+clean; 1.8 GiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-21T11:41:35.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1529219957' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:35.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:34 smithi175 ceph-mon[32613]: 4.15 scrub starts 2024-09-21T11:41:35.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:34 smithi175 ceph-mon[32613]: 4.15 scrub ok 2024-09-21T11:41:35.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:35.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:35.151 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:35.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:35.252 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:35.278 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:35.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:35.280 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:35.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:35.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:35.407 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:35.407 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:41:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:41:35.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:35.536 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:35.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:35.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:35.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:35.648 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 17 16 37488 37472 137.731 114.812 0.00531413 0.00725552 2024-09-21T11:41:35.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:35.682 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:35.683 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:35.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:35.813 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:35.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:35.814 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:35.915 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:35.943 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:35.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:35.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:41:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:41:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: 4.1b scrub starts 2024-09-21T11:41:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: 4.1b scrub ok 2024-09-21T11:41:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: 3.6 scrub starts 2024-09-21T11:41:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: 3.6 scrub ok 2024-09-21T11:41:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: 6.1 scrub starts 2024-09-21T11:41:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: 6.1 scrub ok 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: 3.e scrub starts 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: 3.e scrub ok 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[36454]: osdmap e93: 8 total, 8 up, 8 in 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: 4.1b scrub starts 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: 4.1b scrub ok 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: 3.6 scrub starts 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: 3.6 scrub ok 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: 6.1 scrub starts 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: 6.1 scrub ok 2024-09-21T11:41:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: 3.e scrub starts 2024-09-21T11:41:35.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: 3.e scrub ok 2024-09-21T11:41:35.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:35 smithi043 ceph-mon[29710]: osdmap e93: 8 total, 8 up, 8 in 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: 4.1b scrub starts 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: 4.1b scrub ok 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: 3.6 scrub starts 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: 3.6 scrub ok 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: 6.1 scrub starts 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: 6.1 scrub ok 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: 3.e scrub starts 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: 3.e scrub ok 2024-09-21T11:41:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:35 smithi175 ceph-mon[32613]: osdmap e93: 8 total, 8 up, 8 in 2024-09-21T11:41:36.045 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:36.074 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:36.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:36.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:36.176 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:36.203 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:36.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:36.204 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:36.305 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:36.309 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:41:36.310 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:41:36.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.3 'trim stale osdmaps' 2024-09-21T11:41:36.331 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:36.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:36.332 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:36.433 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:36.458 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:36.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:36.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:36.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:36.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:36.616 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:36.648 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 18 16 39214 39198 136.072 107.875 0.00757316 0.00734238 2024-09-21T11:41:36.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:36.744 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:36.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:36.746 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:36.846 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:36.875 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:36.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:36.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:36.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:36 smithi043 ceph-mon[29710]: 4.1d scrub starts 2024-09-21T11:41:36.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:36 smithi043 ceph-mon[29710]: 4.1d scrub ok 2024-09-21T11:41:36.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:36 smithi043 ceph-mon[29710]: pgmap v56: 148 pgs: 1 peering, 147 active+clean; 2.0 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 152 MiB/s wr, 2.43k op/s; 36 B/s, 1 objects/s recovering 2024-09-21T11:41:36.947 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:36 smithi043 ceph-mon[36454]: 4.1d scrub starts 2024-09-21T11:41:36.947 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:36 smithi043 ceph-mon[36454]: 4.1d scrub ok 2024-09-21T11:41:36.947 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:36 smithi043 ceph-mon[36454]: pgmap v56: 148 pgs: 1 peering, 147 active+clean; 2.0 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 152 MiB/s wr, 2.43k op/s; 36 B/s, 1 objects/s recovering 2024-09-21T11:41:36.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:37.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:36 smithi175 ceph-mon[32613]: 4.1d scrub starts 2024-09-21T11:41:37.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:36 smithi175 ceph-mon[32613]: 4.1d scrub ok 2024-09-21T11:41:37.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:36 smithi175 ceph-mon[32613]: pgmap v56: 148 pgs: 1 peering, 147 active+clean; 2.0 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 152 MiB/s wr, 2.43k op/s; 36 B/s, 1 objects/s recovering 2024-09-21T11:41:37.045 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:37.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:37.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:37.122 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:41:37.147 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:37.172 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:37.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:37.173 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:37.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:37.302 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:37.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:37.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:37.404 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:37.428 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:37.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:37.429 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:37.529 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:37.558 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:37.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:37.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:37.648 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 19 15 40902 40887 134.466 105.562 0.0151519 0.0074313 2024-09-21T11:41:37.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:37.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:37.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:37.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:37.840 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:37.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:37.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:37.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:37.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: 3.8 scrub starts 2024-09-21T11:41:37.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: 3.8 scrub ok 2024-09-21T11:41:37.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: 2.0 scrub starts 2024-09-21T11:41:37.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: 2.0 scrub ok 2024-09-21T11:41:37.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: 3.16 scrub starts 2024-09-21T11:41:37.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: 3.16 scrub ok 2024-09-21T11:41:37.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1201799252' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:41:37.961 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[29710]: osdmap e94: 8 total, 8 up, 8 in 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: 3.8 scrub starts 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: 3.8 scrub ok 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: 2.0 scrub starts 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: 2.0 scrub ok 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: 3.16 scrub starts 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: 3.16 scrub ok 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1201799252' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:41:37.962 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:41:37.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:41:37.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:41:37.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:41:37.963 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:37 smithi043 ceph-mon[36454]: osdmap e94: 8 total, 8 up, 8 in 2024-09-21T11:41:37.967 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:37.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:37.968 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: 3.8 scrub starts 2024-09-21T11:41:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: 3.8 scrub ok 2024-09-21T11:41:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: 2.0 scrub starts 2024-09-21T11:41:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: 2.0 scrub ok 2024-09-21T11:41:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: 3.16 scrub starts 2024-09-21T11:41:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: 3.16 scrub ok 2024-09-21T11:41:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1201799252' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:41:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:41:38.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:41:38.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:41:38.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:41:38.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:37 smithi175 ceph-mon[32613]: osdmap e94: 8 total, 8 up, 8 in 2024-09-21T11:41:38.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:38.108 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:38.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:38.109 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:38.210 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:38.235 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:38.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:38.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:38.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:38.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:38.378 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:38.378 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:38.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:38.522 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:38.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:38.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:38.623 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:38.648 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:41:38.647601+0000 min lat: 0.0011233 max lat: 0.139917 avg lat: 0.00749562 2024-09-21T11:41:38.648 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:41:38.648 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 20 16 42684 42668 133.308 111.312 0.0207663 0.00749562 2024-09-21T11:41:38.651 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:38.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:38.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:38.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:38.781 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:38.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:38.782 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:38.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:38.911 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:38.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:38.912 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:38.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:38 smithi043 ceph-mon[29710]: 4.4 scrub starts 2024-09-21T11:41:38.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:38 smithi043 ceph-mon[29710]: 4.4 scrub ok 2024-09-21T11:41:38.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:38 smithi043 ceph-mon[29710]: pgmap v58: 148 pgs: 148 active+clean; 2.3 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 841 B/s rd, 90 MiB/s wr, 1.44k op/s; 36 B/s, 1 objects/s recovering 2024-09-21T11:41:38.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:38 smithi043 ceph-mon[36454]: 4.4 scrub starts 2024-09-21T11:41:38.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:38 smithi043 ceph-mon[36454]: 4.4 scrub ok 2024-09-21T11:41:38.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:38 smithi043 ceph-mon[36454]: pgmap v58: 148 pgs: 148 active+clean; 2.3 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 841 B/s rd, 90 MiB/s wr, 1.44k op/s; 36 B/s, 1 objects/s recovering 2024-09-21T11:41:38.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:41:39.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:38 smithi175 ceph-mon[32613]: 4.4 scrub starts 2024-09-21T11:41:39.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:38 smithi175 ceph-mon[32613]: 4.4 scrub ok 2024-09-21T11:41:39.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:38 smithi175 ceph-mon[32613]: pgmap v58: 148 pgs: 148 active+clean; 2.3 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 841 B/s rd, 90 MiB/s wr, 1.44k op/s; 36 B/s, 1 objects/s recovering 2024-09-21T11:41:39.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:39.040 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:39.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:39.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:39.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:39.202 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:39.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:39.207 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:39.207 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:39.225 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:41:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:41:39.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:3009 7.b 7:d1f679bc:::benchmark_data_smithi043_70646_object3008:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:19.687381+0000", 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.532235330999999, 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.027295179999999999, 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3009 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.687381+0000", 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.687381+0000", 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.687383+0000", 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8309999999999999e-06 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.687407+0000", 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4000000000000001e-05 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.687424+0000", 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7649999999999999e-05 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.687433+0000", 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7159999999999992e-06 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.687501+0000", 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7886999999999996e-05 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.687629+0000", 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000128019 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.687766+0000", 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013671500000000001 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.688509+0000", 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074300199999999999 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.689480+0000", 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000971684 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.714429+0000", 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024948788 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.714624+0000", 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194413 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.714676+0000", 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2475000000000002e-05 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:3012 7.b 7:d7c35d54:::benchmark_data_smithi043_70646_object3011:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:19.688759+0000", 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.530856794999998, 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026109677000000001, 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3012 2024-09-21T11:41:39.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.688759+0000", 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.688759+0000", 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.688761+0000", 2024-09-21T11:41:39.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.145e-06 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.688786+0000", 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5224000000000001e-05 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.688800+0000", 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4674999999999999e-05 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.688808+0000", 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2420000000000003e-06 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.688862+0000", 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4617000000000002e-05 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.688981+0000", 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011822100000000001 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.689104+0000", 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123237 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.689992+0000", 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00088847000000000004 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.690832+0000", 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00084017100000000002 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.714727+0000", 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023894579999999999 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.714830+0000", 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102598 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.714869+0000", 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9496999999999998e-05 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:3123 7.b 7:d141faf1:::benchmark_data_smithi043_70646_object3122:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:19.750757+0000", 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 19.468858984000001, 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015095945, 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 3123 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.238 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.750757+0000", 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.750757+0000", 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.750759+0000", 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6819999999999999e-06 2024-09-21T11:41:39.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.750786+0000", 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6947000000000001e-05 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.750801+0000", 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5534999999999999e-05 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.750809+0000", 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4800000000000004e-06 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.751487+0000", 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00067777799999999997 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.751606+0000", 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119047 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.751737+0000", 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013165 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.752678+0000", 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00094089200000000003 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.757455+0000", 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047765259999999997 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.765721+0000", 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0082657789999999991 2024-09-21T11:41:39.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.765795+0000", 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4085000000000006e-05 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.765853+0000", 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8544000000000001e-05 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.243 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:5270 7.b 7:df52303f:::benchmark_data_smithi043_70646_object5269:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:20.486908+0000", 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.732707943000001, 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018366830000000001, 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5270 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.486908+0000", 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.486908+0000", 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.486910+0000", 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1639999999999999e-06 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.486935+0000", 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5034000000000001e-05 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.486956+0000", 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0250000000000001e-05 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.486964+0000", 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0129999999999993e-06 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.487062+0000", 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7849000000000005e-05 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.487173+0000", 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111407 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.487300+0000", 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012687899999999999 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.488787+0000", 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001486788 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.505130+0000", 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016343052 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.505167+0000", 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6927000000000001e-05 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.505223+0000", 2024-09-21T11:41:39.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6551000000000001e-05 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.505275+0000", 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1916000000000002e-05 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:5272 7.b 7:d640db44:::benchmark_data_smithi043_70646_object5271:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:20.486984+0000", 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.732631835999999, 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018436449000000001, 2024-09-21T11:41:39.249 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5272 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.486984+0000", 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.486984+0000", 2024-09-21T11:41:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.486985+0000", 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7100000000000002e-07 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.487004+0000", 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8762000000000001e-05 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.487010+0000", 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3260000000000001e-06 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.487016+0000", 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1970000000000001e-06 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.487762+0000", 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074616799999999996 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.487855+0000", 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3128999999999996e-05 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.487962+0000", 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106496 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.491060+0000", 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00309823 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.254 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.505309+0000", 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014248558 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.505338+0000", 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8813e-05 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.505383+0000", 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5068999999999998e-05 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.505421+0000", 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8229999999999998e-05 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:5298 7.b 7:d6d35bac:::benchmark_data_smithi043_70646_object5297:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:20.498014+0000", 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.721602412999999, 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026814195999999998, 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.256 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5298 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.498014+0000", 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.498014+0000", 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.498016+0000", 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.838e-06 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.498040+0000", 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4752e-05 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.498057+0000", 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6319000000000001e-05 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.498064+0000", 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4970000000000004e-06 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.498119+0000", 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4589999999999997e-05 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.498229+0000", 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011052000000000001 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.498366+0000", 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013689799999999999 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.498949+0000", 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00058251799999999997 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.260 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.505457+0000", 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0065081080000000003 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.524677+0000", 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019220511999999999 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.524779+0000", 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101549 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.524828+0000", 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9094999999999997e-05 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:5368 7.b 7:d81138ab:::benchmark_data_smithi043_70646_object5367:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:20.529898+0000", 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 18.6897181, 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01466606, 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.262 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 5368 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.529898+0000", 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.529898+0000", 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.529900+0000", 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1780000000000002e-06 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.529925+0000", 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5069999999999999e-05 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.529943+0000", 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7493e-05 2024-09-21T11:41:39.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.529951+0000", 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9160000000000001e-06 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.530008+0000", 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7006999999999998e-05 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.530112+0000", 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103795 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.530246+0000", 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013467500000000001 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.530944+0000", 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00069734400000000002 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.531664+0000", 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00072071500000000003 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.544425+0000", 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012760495 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.544519+0000", 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4388000000000005e-05 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:20.544564+0000", 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4984000000000002e-05 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.267 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:35104 7.b 7:d4141b8e:::benchmark_data_smithi043_70646_object35103:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:34.341086+0000", 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.878530123, 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013508279999999999, 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 35104 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.268 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.341086+0000", 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.341086+0000", 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.341089+0000", 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4650000000000001e-06 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.341114+0000", 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5412999999999999e-05 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.341136+0000", 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2147999999999999e-05 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.341146+0000", 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9879999999999999e-06 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.341239+0000", 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2380000000000007e-05 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.341373+0000", 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013487199999999999 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.341515+0000", 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014133400000000001 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.343365+0000", 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018503199999999999 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.343551+0000", 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018607199999999999 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.353507+0000", 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0099559520000000006 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.354223+0000", 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000715721 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.354594+0000", 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000371615 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:35654 7.0 e92/82 7:0a46c00c:::benchmark_data_smithi043_70646_object35653:head v 92'2154, mlcod=92'2152)", 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:34.650841+0000", 2024-09-21T11:41:39.273 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.5687756530000003, 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013410335000000001, 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.650841+0000", 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.650841+0000", 2024-09-21T11:41:39.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.650841+0000", 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9999999999999996e-07 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.650883+0000", 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1221999999999999e-05 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.650893+0000", 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0192e-05 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.650900+0000", 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9829999999999999e-06 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.656500+0000", 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056000210000000002 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.656519+0000", 2024-09-21T11:41:39.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8941e-05 2024-09-21T11:41:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.664207+0000", 2024-09-21T11:41:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076880569999999999 2024-09-21T11:41:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:34.664251+0000", 2024-09-21T11:41:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4019e-05 2024-09-21T11:41:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:37277 7.b 7:d13dff0d:::benchmark_data_smithi043_70646_object37276:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:41:39.278 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:35.480704+0000", 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7389118529999998, 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036890579, 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37277 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480704+0000", 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480704+0000", 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480706+0000", 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0339999999999999e-06 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480728+0000", 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1526e-05 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480738+0000", 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0083e-05 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480745+0000", 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481292+0000", 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00054686700000000003 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481399+0000", 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107378 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481563+0000", 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000163833 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484047+0000", 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024838680000000002 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484084+0000", 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7339000000000003e-05 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517265+0000", 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.033180236000000002 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517535+0000", 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00027041599999999999 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517595+0000", 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0001999999999999e-05 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.284 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:37278 7.b 7:d6b83a2c:::benchmark_data_smithi043_70646_object37277:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:35.481164+0000", 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.7384526579999999, 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036602902, 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37278 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481164+0000", 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481164+0000", 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481164+0000", 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8899999999999998e-07 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481196+0000", 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1829000000000003e-05 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481206+0000", 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8940000000000005e-06 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481213+0000", 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2080000000000003e-06 2024-09-21T11:41:39.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484174+0000", 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029610600000000002 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484285+0000", 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110468 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484411+0000", 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125847 2024-09-21T11:41:39.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.486414+0000", 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020031049999999998 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.486427+0000", 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3563e-05 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517660+0000", 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031232167000000002 2024-09-21T11:41:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517723+0000", 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3660999999999999e-05 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517766+0000", 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3210999999999997e-05 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.290 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:38791 7.b 7:d058df55:::benchmark_data_smithi043_70646_object38790:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:36.361348+0000", 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8582687099999999, 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013183242, 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 38791 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.291 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.361348+0000", 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.361348+0000", 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.361348+0000", 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9100000000000004e-07 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.292 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.361369+0000", 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1263999999999998e-05 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.361375+0000", 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0700000000000003e-06 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.361384+0000", 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5760000000000006e-06 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.293 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.365153+0000", 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037689339999999998 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.365348+0000", 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019548499999999999 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.365491+0000", 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014226500000000001 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.294 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.369422+0000", 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039309719999999996 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.372332+0000", 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002910045 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.374429+0000", 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020972370000000001 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.374477+0000", 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7914000000000002e-05 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.374531+0000", 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3989000000000001e-05 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:39236 7.b 7:d2094da9:::benchmark_data_smithi043_70646_object39235:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:36.661785+0000", 2024-09-21T11:41:39.296 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.5578311989999998, 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016282799000000001, 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 39236 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.661785+0000", 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.661785+0000", 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.661786+0000", 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.341e-06 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.661810+0000", 2024-09-21T11:41:39.298 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3581e-05 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.661821+0000", 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0968000000000001e-05 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.661828+0000", 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5390000000000002e-06 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.662446+0000", 2024-09-21T11:41:39.299 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00061755200000000005 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.666668+0000", 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042224369999999999 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.666827+0000", 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015844599999999999 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.669817+0000", 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029905449999999998 2024-09-21T11:41:39.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.677876+0000", 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080583830000000006 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.677912+0000", 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5732999999999998e-05 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.678012+0000", 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010049500000000001 2024-09-21T11:41:39.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.678068+0000", 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5779000000000002e-05 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:39265 7.b 7:da3e25b7:::benchmark_data_smithi043_70646_object39264:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:36.678465+0000", 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.5411510150000001, 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017179976999999999, 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 39265 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.678465+0000", 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.678465+0000", 2024-09-21T11:41:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.678466+0000", 2024-09-21T11:41:39.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0299999999999999e-07 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.678486+0000", 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0547000000000002e-05 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.678495+0000", 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2349999999999999e-06 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.678502+0000", 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9569999999999996e-06 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.685910+0000", 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074087780000000004 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.686026+0000", 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115223 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.686150+0000", 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012450800000000001 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.688955+0000", 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002805246 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.688993+0000", 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7571999999999999e-05 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.695340+0000", 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063471939999999996 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.695593+0000", 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000252544 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.695645+0000", 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.257e-05 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.308 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:39413 7.0 e93/82 7:0047e634:::benchmark_data_smithi043_70646_object39412:head v 93'2395, mlcod=93'2390)", 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:36.768789+0000", 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.4508271069999998, 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012808783000000001, 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.768789+0000", 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.768789+0000", 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.768789+0000", 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3099999999999999e-07 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.310 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.768816+0000", 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6922000000000001e-05 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.768822+0000", 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.768827+0000", 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6639999999999997e-06 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.311 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.775223+0000", 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063957249999999997 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.775249+0000", 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5740000000000001e-05 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.781557+0000", 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063089060000000004 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.781598+0000", 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0469000000000002e-05 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:39414 7.0 e93/82 7:03b8de04:::benchmark_data_smithi043_70646_object39413:head v 93'2396, mlcod=93'2390)", 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:36.769562+0000", 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.4500543929999998, 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01353916, 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.313 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.769562+0000", 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.769562+0000", 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.769563+0000", 2024-09-21T11:41:39.314 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7800000000000001e-07 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.769641+0000", 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8269999999999994e-05 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.769649+0000", 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.687e-06 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.769656+0000", 2024-09-21T11:41:39.315 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5320000000000003e-06 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.776944+0000", 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072884020000000002 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.776962+0000", 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7716000000000001e-05 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.783058+0000", 2024-09-21T11:41:39.316 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060961629999999999 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.783101+0000", 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2712e-05 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:39827 7.b 7:d0d12f85:::benchmark_data_smithi043_70646_object39826:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:37.030452+0000", 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.1891642139999998, 2024-09-21T11:41:39.317 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019448764, 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 39827 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.030452+0000", 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.030452+0000", 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.030454+0000", 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0169999999999999e-06 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.030488+0000", 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3812000000000001e-05 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.030498+0000", 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7799999999999995e-06 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.030505+0000", 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3590000000000002e-06 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.041954+0000", 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011448823 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.320 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.045177+0000", 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032229289999999998 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.046043+0000", 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00086640899999999995 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.049548+0000", 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035050599999999999 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.321 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.049789+0000", 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024071100000000001 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.049804+0000", 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5528000000000001e-05 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.049857+0000", 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2179000000000003e-05 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:37.049901+0000", 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4156999999999998e-05 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:42580 7.4 e94/82 7:2f775980:::benchmark_data_smithi043_70646_object42579:head v 94'2586, mlcod=94'2581)", 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:38.588030+0000", 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.63158619299999996, 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012736364999999999, 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.323 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.588030+0000", 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.588030+0000", 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.588031+0000", 2024-09-21T11:41:39.324 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8899999999999998e-07 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.588055+0000", 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3847999999999999e-05 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.588072+0000", 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7007000000000002e-05 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.588081+0000", 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.054e-06 2024-09-21T11:41:39.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.588175+0000", 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4585000000000003e-05 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.588243+0000", 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7712000000000005e-05 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.600324+0000", 2024-09-21T11:41:39.326 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012081155999999999 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.600766+0000", 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00044221400000000002 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:42831 7.b 7:dd5bd818:::benchmark_data_smithi043_70646_object42830:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:38.727155+0000", 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.49246084200000001, 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0127063, 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.327 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 42831 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.727155+0000", 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.328 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.727155+0000", 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.730472+0000", 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033165159999999998 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.730515+0000", 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3377000000000002e-05 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.730524+0000", 2024-09-21T11:41:39.329 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9430000000000006e-06 2024-09-21T11:41:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.330 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.330 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.730530+0000", 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.48e-06 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.736842+0000", 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063119420000000001 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.736962+0000", 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120238 2024-09-21T11:41:39.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.737104+0000", 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014237399999999999 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.739079+0000", 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001974768 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.739379+0000", 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030022600000000001 2024-09-21T11:41:39.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.739412+0000", 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2700000000000002e-05 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.739497+0000", 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5535e-05 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.739862+0000", 2024-09-21T11:41:39.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00036420099999999998 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:43117 7.d e94/82 7:b6b189e5:::benchmark_data_smithi043_70646_object43116:head v 94'2739, mlcod=94'2737)", 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:38.884720+0000", 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.33489592600000001, 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012777019000000001, 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:41:39.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.884720+0000", 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.884720+0000", 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.884721+0000", 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3600000000000002e-07 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.884760+0000", 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.871e-05 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.884774+0000", 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4233000000000001e-05 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.884781+0000", 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9369999999999998e-06 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.885881+0000", 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010996129999999999 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.885899+0000", 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8635999999999999e-05 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.897447+0000", 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011547945 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:41:39.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:41:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.897497+0000", 2024-09-21T11:41:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0009000000000002e-05 2024-09-21T11:41:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:41:39.338 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:41:39.338 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:41:39.383 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:39.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:39.386 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:39.486 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:39.515 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:39.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:39.517 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:39.617 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:39.644 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:39.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:39.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:39.648 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 21 16 44289 44273 131.736 100.312 0.00990661 0.00758455 2024-09-21T11:41:39.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:39.785 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:39.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:39.786 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:39.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:39.914 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:39.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:39.915 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:39 smithi043 ceph-mon[36454]: 6.c scrub starts 2024-09-21T11:41:39.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:39 smithi043 ceph-mon[36454]: 6.c scrub ok 2024-09-21T11:41:39.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:39 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:41:39.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:39 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:41:39.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:39 smithi043 ceph-mon[29710]: 6.c scrub starts 2024-09-21T11:41:39.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:39 smithi043 ceph-mon[29710]: 6.c scrub ok 2024-09-21T11:41:39.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:39 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:41:39.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:39 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:41:40.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:39 smithi175 ceph-mon[32613]: 6.c scrub starts 2024-09-21T11:41:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:39 smithi175 ceph-mon[32613]: 6.c scrub ok 2024-09-21T11:41:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:39 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:41:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:39 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:41:40.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:40.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:41:40.075 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:40.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:40.076 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:40.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:40.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:40.228 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:40.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:40.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:40.379 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:40.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:40.381 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:40.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:40.513 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:40.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:40.514 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:40.614 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:40.642 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:40.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:40.643 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:40.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:41:40.648 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 22 16 45901 45885 130.328 100.75 0.00380842 0.00766746 2024-09-21T11:41:40.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:40.769 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:40.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:40.770 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:40.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:40.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[36454]: 6.1d scrub starts 2024-09-21T11:41:40.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[36454]: 6.1d scrub ok 2024-09-21T11:41:40.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[36454]: 6.3 scrub starts 2024-09-21T11:41:40.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[36454]: 6.3 scrub ok 2024-09-21T11:41:40.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[36454]: pgmap v59: 148 pgs: 148 active+clean; 2.3 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 68 MiB/s wr, 1.09k op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:41:40.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[36454]: osdmap e95: 8 total, 8 up, 8 in 2024-09-21T11:41:40.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[36454]: 6.a scrub starts 2024-09-21T11:41:40.921 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[36454]: 6.a scrub ok 2024-09-21T11:41:40.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[29710]: 6.1d scrub starts 2024-09-21T11:41:40.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[29710]: 6.1d scrub ok 2024-09-21T11:41:40.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[29710]: 6.3 scrub starts 2024-09-21T11:41:40.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[29710]: 6.3 scrub ok 2024-09-21T11:41:40.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[29710]: pgmap v59: 148 pgs: 148 active+clean; 2.3 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 68 MiB/s wr, 1.09k op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:41:40.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[29710]: osdmap e95: 8 total, 8 up, 8 in 2024-09-21T11:41:40.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[29710]: 6.a scrub starts 2024-09-21T11:41:40.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:40 smithi043 ceph-mon[29710]: 6.a scrub ok 2024-09-21T11:41:40.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:40.923 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:40.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:40 smithi175 ceph-mon[32613]: 6.1d scrub starts 2024-09-21T11:41:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:40 smithi175 ceph-mon[32613]: 6.1d scrub ok 2024-09-21T11:41:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:40 smithi175 ceph-mon[32613]: 6.3 scrub starts 2024-09-21T11:41:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:40 smithi175 ceph-mon[32613]: 6.3 scrub ok 2024-09-21T11:41:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:40 smithi175 ceph-mon[32613]: pgmap v59: 148 pgs: 148 active+clean; 2.3 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 68 MiB/s wr, 1.09k op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:41:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:40 smithi175 ceph-mon[32613]: osdmap e95: 8 total, 8 up, 8 in 2024-09-21T11:41:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:40 smithi175 ceph-mon[32613]: 6.a scrub starts 2024-09-21T11:41:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:40 smithi175 ceph-mon[32613]: 6.a scrub ok 2024-09-21T11:41:41.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:41:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:41:41.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:41:41.037 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:41:41.048 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:41.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:41.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:41.150 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:41.177 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:41.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:41.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:41.279 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:41.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:41.334 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:41.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:41.476 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:41.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:41.477 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:41.578 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:41.604 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:41.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:41.605 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:41.649 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 23 16 47311 47295 128.49 88.125 0.00748285 0.0077771 2024-09-21T11:41:41.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:41.753 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:41.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:41.754 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:41.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:41.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[29710]: 5.1 scrub starts 2024-09-21T11:41:41.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[29710]: 5.1 scrub ok 2024-09-21T11:41:41.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[29710]: 6.0 scrub starts 2024-09-21T11:41:41.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[29710]: 6.0 scrub ok 2024-09-21T11:41:41.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[29710]: 6.1f scrub starts 2024-09-21T11:41:41.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[29710]: 6.1f scrub ok 2024-09-21T11:41:41.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:41.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T11:41:41.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T11:41:41.875 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[29710]: 4.1 scrub starts 2024-09-21T11:41:41.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[36454]: 5.1 scrub starts 2024-09-21T11:41:41.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[36454]: 5.1 scrub ok 2024-09-21T11:41:41.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[36454]: 6.0 scrub starts 2024-09-21T11:41:41.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[36454]: 6.0 scrub ok 2024-09-21T11:41:41.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[36454]: 6.1f scrub starts 2024-09-21T11:41:41.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[36454]: 6.1f scrub ok 2024-09-21T11:41:41.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:41.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T11:41:41.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T11:41:41.876 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:41 smithi043 ceph-mon[36454]: 4.1 scrub starts 2024-09-21T11:41:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:41.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:41.897 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:41.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:42.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:41 smithi175 ceph-mon[32613]: 5.1 scrub starts 2024-09-21T11:41:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:41 smithi175 ceph-mon[32613]: 5.1 scrub ok 2024-09-21T11:41:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:41 smithi175 ceph-mon[32613]: 6.0 scrub starts 2024-09-21T11:41:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:41 smithi175 ceph-mon[32613]: 6.0 scrub ok 2024-09-21T11:41:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:41 smithi175 ceph-mon[32613]: 6.1f scrub starts 2024-09-21T11:41:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:41 smithi175 ceph-mon[32613]: 6.1f scrub ok 2024-09-21T11:41:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:41 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:41 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T11:41:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:41 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T11:41:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:41 smithi175 ceph-mon[32613]: 4.1 scrub starts 2024-09-21T11:41:42.027 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:41:42.027 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:41:42.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.4 'trim stale osdmaps' 2024-09-21T11:41:42.042 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:42.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:42.043 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:42.074 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:42.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:42.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:42.194 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:42.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:42.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:42.349 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:42.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:42.351 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:42.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:42.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:42.537 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:42.638 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:42.649 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 24 16 48589 48573 126.462 79.875 0.00411498 0.00790154 2024-09-21T11:41:42.662 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:42.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:42.663 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:42.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:42.787 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:42.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:42.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:42.795 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:42.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[29710]: 4.1 scrub ok 2024-09-21T11:41:42.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[29710]: pgmap v61: 148 pgs: 148 active+clean; 2.8 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 152 MiB/s wr, 2.44k op/s; 31 B/s, 1 objects/s recovering 2024-09-21T11:41:42.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[29710]: osdmap e96: 8 total, 8 up, 8 in 2024-09-21T11:41:42.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:42.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:42.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:41:42.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:41:42.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:42.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[36454]: 4.1 scrub ok 2024-09-21T11:41:42.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[36454]: pgmap v61: 148 pgs: 148 active+clean; 2.8 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 152 MiB/s wr, 2.44k op/s; 31 B/s, 1 objects/s recovering 2024-09-21T11:41:42.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[36454]: osdmap e96: 8 total, 8 up, 8 in 2024-09-21T11:41:42.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:42.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:42.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:41:42.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:41:42.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:42 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:42.889 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:42.917 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:42.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:42.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:43.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:42 smithi175 ceph-mon[32613]: 4.1 scrub ok 2024-09-21T11:41:43.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:42 smithi175 ceph-mon[32613]: pgmap v61: 148 pgs: 148 active+clean; 2.8 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 152 MiB/s wr, 2.44k op/s; 31 B/s, 1 objects/s recovering 2024-09-21T11:41:43.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:42 smithi175 ceph-mon[32613]: osdmap e96: 8 total, 8 up, 8 in 2024-09-21T11:41:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:42 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:42 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:42 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:41:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:42 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:41:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:42 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:43.018 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:43.044 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:43.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:43.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:43.146 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:43.166 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:41:43.167 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:41:43.171 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:43.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:43.172 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:43.206 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:41:43.206 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:41:43.240 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:41:43.241 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:41:43.273 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:43.291 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:41:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:41:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:43.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:43.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:43.326 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:41:43.326 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:41:43.364 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:41:43.364 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:41:43.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:43.402 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:41:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:41:43.443 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:41:43.443 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:41:43.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:43.449 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:43.449 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:43.550 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:43.578 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:43.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:43.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:43.650 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 25 16 49857 49841 124.573 79.25 0.0136785 0.00802133 2024-09-21T11:41:43.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:43.720 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:43.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:43.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:43.725 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:41:43.725 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:41:43.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:43.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:43.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:43.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:43 smithi043 ceph-mon[36454]: 3.1f scrub starts 2024-09-21T11:41:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:43 smithi043 ceph-mon[36454]: 3.1f scrub ok 2024-09-21T11:41:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:43 smithi043 ceph-mon[36454]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-21T11:41:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:43 smithi043 ceph-mon[36454]: Reconfiguring daemon alertmanager.a on smithi043 2024-09-21T11:41:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:43 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1798239038' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:43 smithi043 ceph-mon[29710]: 3.1f scrub starts 2024-09-21T11:41:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:43 smithi043 ceph-mon[29710]: 3.1f scrub ok 2024-09-21T11:41:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:43 smithi043 ceph-mon[29710]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-21T11:41:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:43 smithi043 ceph-mon[29710]: Reconfiguring daemon alertmanager.a on smithi043 2024-09-21T11:41:43.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:43 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1798239038' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:44.001 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:44.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:44.002 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:44.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:43 smithi175 ceph-mon[32613]: 3.1f scrub starts 2024-09-21T11:41:44.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:43 smithi175 ceph-mon[32613]: 3.1f scrub ok 2024-09-21T11:41:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:43 smithi175 ceph-mon[32613]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-21T11:41:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:43 smithi175 ceph-mon[32613]: Reconfiguring daemon alertmanager.a on smithi043 2024-09-21T11:41:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:43 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1798239038' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:41:44.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:44.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:44.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:44.249 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:44.275 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:44.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:44.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:44.377 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:44.406 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:44.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:44.407 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:44.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:44.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:44.551 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:44.650 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 26 16 51495 51479 123.719 102.375 0.0102063 0.00807648 2024-09-21T11:41:44.652 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:44.665 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:44.677 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:44.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:44.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:44.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:44.805 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:44.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:44.806 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:44.907 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:44.935 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:44.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:44.937 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[36454]: 4.1a scrub starts 2024-09-21T11:41:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[36454]: 4.1a scrub ok 2024-09-21T11:41:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[36454]: 5.9 scrub starts 2024-09-21T11:41:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[36454]: 5.9 scrub ok 2024-09-21T11:41:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[36454]: pgmap v63: 148 pgs: 148 active+clean; 2.8 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 776 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-21T11:41:44.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:41:44.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:41:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[29710]: 4.1a scrub starts 2024-09-21T11:41:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[29710]: 4.1a scrub ok 2024-09-21T11:41:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[29710]: 5.9 scrub starts 2024-09-21T11:41:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[29710]: 5.9 scrub ok 2024-09-21T11:41:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[29710]: pgmap v63: 148 pgs: 148 active+clean; 2.8 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 776 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-21T11:41:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:41:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:44 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:41:45.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:44 smithi175 ceph-mon[32613]: 4.1a scrub starts 2024-09-21T11:41:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:44 smithi175 ceph-mon[32613]: 4.1a scrub ok 2024-09-21T11:41:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:44 smithi175 ceph-mon[32613]: 5.9 scrub starts 2024-09-21T11:41:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:44 smithi175 ceph-mon[32613]: 5.9 scrub ok 2024-09-21T11:41:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:44 smithi175 ceph-mon[32613]: pgmap v63: 148 pgs: 148 active+clean; 2.8 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 776 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-21T11:41:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:44 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:41:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:44 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:41:45.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:45.064 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:45.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:45.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:45.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:45.205 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:45.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:45.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:45.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:41:45.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:45.334 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:45.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:45.335 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:45.436 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:45.463 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:45.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:45.464 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:45.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:41:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:41:45.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:45.612 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:45.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:45.613 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:45.650 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 27 15 53013 52998 122.653 94.9375 0.00865839 0.00814726 2024-09-21T11:41:45.714 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:45.740 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:45.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:45.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:45.842 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:45.871 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:45.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:45.872 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:45.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[29710]: 6.17 scrub starts 2024-09-21T11:41:45.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[29710]: 6.17 scrub ok 2024-09-21T11:41:45.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[29710]: 6.15 scrub starts 2024-09-21T11:41:45.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[29710]: 6.15 scrub ok 2024-09-21T11:41:45.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[29710]: osdmap e97: 8 total, 8 up, 8 in 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[29710]: 6.7 scrub starts 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[29710]: 6.7 scrub ok 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[36454]: 6.17 scrub starts 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[36454]: 6.17 scrub ok 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[36454]: 6.15 scrub starts 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[36454]: 6.15 scrub ok 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[36454]: osdmap e97: 8 total, 8 up, 8 in 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[36454]: 6.7 scrub starts 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[36454]: 6.7 scrub ok 2024-09-21T11:41:45.939 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:45.940 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:45 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:45.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:45.997 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:45.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:45.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:46.099 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:46.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:46.128 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:46.128 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:46.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:46.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:45 smithi175 ceph-mon[32613]: 6.17 scrub starts 2024-09-21T11:41:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:45 smithi175 ceph-mon[32613]: 6.17 scrub ok 2024-09-21T11:41:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:45 smithi175 ceph-mon[32613]: 6.15 scrub starts 2024-09-21T11:41:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:45 smithi175 ceph-mon[32613]: 6.15 scrub ok 2024-09-21T11:41:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:45 smithi175 ceph-mon[32613]: osdmap e97: 8 total, 8 up, 8 in 2024-09-21T11:41:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:45 smithi175 ceph-mon[32613]: 6.7 scrub starts 2024-09-21T11:41:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:45 smithi175 ceph-mon[32613]: 6.7 scrub ok 2024-09-21T11:41:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:45 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:41:46.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:45 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:41:46.268 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:46.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:46.270 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:46.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:46.413 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:46.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:46.415 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:46.516 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:46.543 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:46.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:46.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:46.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:46.650 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 28 16 54479 54463 121.543 91.5625 0.00655518 0.00822087 2024-09-21T11:41:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:46.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:46.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:46.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:46.819 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:46.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:46.820 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:46.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:46.949 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:46.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:46.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:46.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:46 smithi043 ceph-mon[36454]: 4.9 scrub starts 2024-09-21T11:41:46.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:46 smithi043 ceph-mon[36454]: 4.9 scrub ok 2024-09-21T11:41:46.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:46 smithi043 ceph-mon[36454]: pgmap v65: 148 pgs: 148 active+clean; 2.9 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 106 MiB/s wr, 1.70k op/s 2024-09-21T11:41:46.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:46 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3342366963' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:41:46.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:46 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:41:46.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:46 smithi043 ceph-mon[29710]: 4.9 scrub starts 2024-09-21T11:41:46.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:46 smithi043 ceph-mon[29710]: 4.9 scrub ok 2024-09-21T11:41:46.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:46 smithi043 ceph-mon[29710]: pgmap v65: 148 pgs: 148 active+clean; 2.9 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 106 MiB/s wr, 1.70k op/s 2024-09-21T11:41:46.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:46 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3342366963' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:41:46.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:46 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:41:47.051 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:47.090 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:47.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:47.091 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:47.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:47.233 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:47.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:47.234 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:47.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:46 smithi175 ceph-mon[32613]: 4.9 scrub starts 2024-09-21T11:41:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:46 smithi175 ceph-mon[32613]: 4.9 scrub ok 2024-09-21T11:41:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:46 smithi175 ceph-mon[32613]: pgmap v65: 148 pgs: 148 active+clean; 2.9 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 106 MiB/s wr, 1.70k op/s 2024-09-21T11:41:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:46 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3342366963' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:41:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:46 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:41:47.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:47.378 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:47.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:47.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:47.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:47.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:47.529 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:47.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:47.651 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 29 16 55984 55968 120.594 94.0625 0.0121334 0.00828673 2024-09-21T11:41:47.674 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:47.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:47.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:47.696 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:41:47.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:47.814 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:47.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:47.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:47.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:47.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:47.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:48.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[36454]: 4.13 scrub starts 2024-09-21T11:41:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[36454]: 4.13 scrub ok 2024-09-21T11:41:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[36454]: 3.d scrub starts 2024-09-21T11:41:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[36454]: 3.d scrub ok 2024-09-21T11:41:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:41:48.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[36454]: osdmap e98: 8 total, 8 up, 8 in 2024-09-21T11:41:48.020 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[29710]: 4.13 scrub starts 2024-09-21T11:41:48.020 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[29710]: 4.13 scrub ok 2024-09-21T11:41:48.020 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[29710]: 3.d scrub starts 2024-09-21T11:41:48.020 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[29710]: 3.d scrub ok 2024-09-21T11:41:48.021 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:48.021 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:41:48.021 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:47 smithi043 ceph-mon[29710]: osdmap e98: 8 total, 8 up, 8 in 2024-09-21T11:41:48.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:48.086 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:48.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:48.087 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:48.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:48.225 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:48.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:48.226 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:48.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:47 smithi175 ceph-mon[32613]: 4.13 scrub starts 2024-09-21T11:41:48.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:47 smithi175 ceph-mon[32613]: 4.13 scrub ok 2024-09-21T11:41:48.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:47 smithi175 ceph-mon[32613]: 3.d scrub starts 2024-09-21T11:41:48.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:47 smithi175 ceph-mon[32613]: 3.d scrub ok 2024-09-21T11:41:48.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:47 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:41:48.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:47 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:41:48.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:47 smithi175 ceph-mon[32613]: osdmap e98: 8 total, 8 up, 8 in 2024-09-21T11:41:48.326 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:48.356 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:48.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:48.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:48.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:48.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:48.514 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:48.514 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:48.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:48.642 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:48.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:48.643 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:48.650 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 30 16 57485 57469 119.701 93.8125 0.0081675 0.00834891 2024-09-21T11:41:48.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:48.773 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:48.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:48.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:48.874 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:48.903 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:48.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:48.904 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:49.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:49.049 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:49.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:49.052 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:49.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:49.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:49.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:49.292 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:49.321 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:49.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:49.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:49.411 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:49 smithi043 ceph-mon[29710]: pgmap v67: 148 pgs: 148 active+clean; 3.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-21T11:41:49.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:49 smithi043 ceph-mon[36454]: pgmap v67: 148 pgs: 148 active+clean; 3.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-21T11:41:49.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:49.461 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:49.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:49.462 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:49.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:49 smithi175 ceph-mon[32613]: pgmap v67: 148 pgs: 148 active+clean; 3.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-21T11:41:49.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:49.592 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:49.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:49.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:49.606 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:49.650 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 31 16 58994 58978 118.882 94.3125 0.0163387 0.00840657 2024-09-21T11:41:49.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:49.726 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:49.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:49.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:49.741 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:41:49 smithi043 systemd[1]: Stopping Ceph alertmanager.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:41:49.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:49.859 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:49.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:49.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:49.961 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:49.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:49.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:49.994 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:50.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:50.125 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:50.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:50.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:50.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:50.270 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:50.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:50.272 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:50.372 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:50.393 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:50.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:50.394 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:50.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:41:50.495 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:50.582 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:50.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:50.583 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:50.651 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 32 16 60408 60392 117.927 88.375 0.00970931 0.00847377 2024-09-21T11:41:50.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:50.715 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:50.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:50.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:50.817 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:50.865 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:50.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:50.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:50.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:50.995 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:50.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:50.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:51.097 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:51.128 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:41:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:41:51.133 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:51.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:51.134 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:51.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:51.264 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:51.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:51.265 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:51.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:51.403 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:51.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:51.404 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:51.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:51 smithi043 ceph-mon[36454]: pgmap v68: 148 pgs: 148 active+clean; 3.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 674 B/s rd, 59 MiB/s wr, 937 op/s 2024-09-21T11:41:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:51 smithi043 ceph-mon[29710]: pgmap v68: 148 pgs: 148 active+clean; 3.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 674 B/s rd, 59 MiB/s wr, 937 op/s 2024-09-21T11:41:51.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:51.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:51 smithi175 ceph-mon[32613]: pgmap v68: 148 pgs: 148 active+clean; 3.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 674 B/s rd, 59 MiB/s wr, 937 op/s 2024-09-21T11:41:51.547 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:51.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:51.548 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:51.649 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:51.651 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 33 16 61794 61778 116.979 86.625 0.00913659 0.00854306 2024-09-21T11:41:51.678 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:51.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:51.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:51.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:51.804 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:51.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:51.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:51.905 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:51.932 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:51.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:51.933 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:52.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:52.070 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:52.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:52.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:52.172 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:52.197 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:52.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:52.198 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:52.299 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:52.323 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:52.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:52.324 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:52.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:52.465 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:52.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:52.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:52.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:52.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:52.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:41:52.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:52.589 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:52.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:52.590 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:52.651 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 34 16 62901 62885 115.573 69.1875 0.0161826 0.0086473 2024-09-21T11:41:52.690 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:52.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:52.721 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:52.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:52.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:52.860 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:52.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:52.861 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:52.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:53.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:53.004 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:53.005 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:53.105 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:53.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:53.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:53.252 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:53.272 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:53.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:53.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:53.374 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:53.404 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:53.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:53.405 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:53.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:53 smithi043 ceph-mon[36454]: pgmap v69: 148 pgs: 148 active+clean; 3.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-21T11:41:53.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:53 smithi043 ceph-mon[29710]: pgmap v69: 148 pgs: 148 active+clean; 3.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-21T11:41:53.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:53.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:53 smithi175 ceph-mon[32613]: pgmap v69: 148 pgs: 148 active+clean; 3.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-21T11:41:53.548 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:53.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:53.550 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:53.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:53.651 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 35 16 64123 64107 114.453 76.375 0.00669375 0.00873158 2024-09-21T11:41:53.687 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:53.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:53.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:53.789 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:53.815 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:53.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:53.816 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:53.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:53.944 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:53.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:53.945 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:54.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:54.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:54.091 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:54.192 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:54.216 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:54.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:54.217 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:54.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:54.347 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:54.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:54.353 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:54.453 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:54.480 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:54.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:54.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:54.582 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:54.609 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:54.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:54.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:54.651 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 36 16 65369 65353 113.437 77.875 0.0411629 0.0088097 2024-09-21T11:41:54.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:54.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:54.751 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:54.796 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:54.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:54.905 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:54.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:54.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:55.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:55.036 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:55.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:55.037 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:55.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:55.182 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:55.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:55.183 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:55.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:55.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:55.325 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:55.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:55.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:55.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:41:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:41:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:41:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:55 smithi043 ceph-mon[36454]: pgmap v70: 148 pgs: 148 active+clean; 3.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-21T11:41:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:55 smithi043 ceph-mon[29710]: pgmap v70: 148 pgs: 148 active+clean; 3.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-21T11:41:55.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:55 smithi175 ceph-mon[32613]: pgmap v70: 148 pgs: 148 active+clean; 3.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-21T11:41:55.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:55.601 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:55.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:55.602 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:55.651 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 37 16 66624 66608 112.491 78.4375 0.00589551 0.00888432 2024-09-21T11:41:55.704 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:55.729 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:55.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:55.730 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:55.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:55.858 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:55.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:55.860 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:55.961 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:55.987 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:55.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:55.989 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:56.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:56.115 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:56.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:56.116 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:56.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:56.266 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:56.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:56.267 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:56.347 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:41:56.348 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:41:56.369 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:56.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:56.394 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:56.394 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:56.495 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:56.520 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:56.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:56.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:56.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:56.652 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 38 16 67763 67747 111.4 71.1875 0.0104802 0.0089712 2024-09-21T11:41:56.664 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:56.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:56.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:56.682 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:41:56.682 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:41:56.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:56.793 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:56.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:56.794 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:56.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:41:56.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:56.957 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:56.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:56.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:57.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:57.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:57.120 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:57.221 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:57.247 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:57.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:57.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:57.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:57.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:57.399 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:57.482 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:57 smithi043 ceph-mon[29710]: pgmap v71: 148 pgs: 148 active+clean; 3.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-21T11:41:57.482 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:57 smithi043 ceph-mon[36454]: pgmap v71: 148 pgs: 148 active+clean; 3.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-21T11:41:57.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:57.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:57 smithi175 ceph-mon[32613]: pgmap v71: 148 pgs: 148 active+clean; 3.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-21T11:41:57.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:57.528 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:57.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:57.532 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:41:57.533 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:41:57.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.5 'trim stale osdmaps' 2024-09-21T11:41:57.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:57.652 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 39 16 68838 68822 110.266 67.1875 0.0187688 0.00906248 2024-09-21T11:41:57.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:57.685 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:57.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:57.786 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:57.810 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:57.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:57.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:57.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:41:57.934 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:57.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:57.935 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:41:58.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:41:58.079 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:58.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:58.080 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:41:58.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:58.208 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:58.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:58.209 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:41:58.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:58.339 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:58.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:58.340 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:58.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:58.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:58.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:58.470 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:58.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:58 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:41:58.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:58 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:41:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:58 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:41:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:58 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:41:58.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:58 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:41:58.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:58 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:41:58.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:58.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:41:58.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:58.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:58.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:58.621 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:58.655 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:41:58.652270+0000 min lat: 0.0011233 max lat: 0.151666 avg lat: 0.00914481 2024-09-21T11:41:58.655 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:41:58.655 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 40 16 69955 69939 109.255 69.8125 0.0170189 0.00914481 2024-09-21T11:41:58.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:41:58.730 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:58.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:58.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:41:58.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:41:58.859 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:58.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:58.861 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:41:58.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:58.994 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:58.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:58.995 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:59.096 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:59.122 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:59.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:59.123 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:59.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:59.271 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:59.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:59.273 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:59.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:41:59.417 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:41:59.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:59.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:41:59.461 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:59 smithi043 ceph-mon[29710]: pgmap v72: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 966 B/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-21T11:41:59.462 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:41:59 smithi043 ceph-mon[29710]: osdmap e99: 8 total, 8 up, 8 in 2024-09-21T11:41:59.462 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:59 smithi043 ceph-mon[36454]: pgmap v72: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 966 B/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-21T11:41:59.462 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:41:59 smithi043 ceph-mon[36454]: osdmap e99: 8 total, 8 up, 8 in 2024-09-21T11:41:59.462 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:41:59 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[67905]: ts=2024-09-21T11:41:59.082Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-21T11:41:59.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:59 smithi175 ceph-mon[32613]: pgmap v72: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 966 B/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-21T11:41:59.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:41:59 smithi175 ceph-mon[32613]: osdmap e99: 8 total, 8 up, 8 in 2024-09-21T11:41:59.519 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:41:59.547 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:59.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:59.548 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:41:59.649 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:59.653 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 41 16 71140 71124 108.396 74.0625 0.0112385 0.00922028 2024-09-21T11:41:59.676 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:59.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:59.677 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:59.688 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:41:59.741 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:41:59 smithi043 podman[75141]: 2024-09-21 11:41:59.667351123 +0000 UTC m=+1.074979095 container cleanup f988a64964725524b3a25c9126c8195df2911e33fb686b30209662ed3f6757f3 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:41:59.741 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:41:59 smithi043 bash[75141]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a 2024-09-21T11:41:59.778 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:41:59.808 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:59.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:59.809 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:41:59.910 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:41:59.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:41:59.938 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:41:59.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:00.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:00.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:00.086 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:00.187 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:00.219 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:00.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:00.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:00.321 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:00.347 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:00.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:00.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:00.448 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:00.475 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:00.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:00.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:00.577 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:00.603 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:00.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:00.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:00.653 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 42 16 72331 72315 107.588 74.4375 0.01296 0.00928874 2024-09-21T11:42:00.706 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:00.732 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:00.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:00.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:00.741 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:42:00.741 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:42:00.778 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:42:00.778 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:42:00.820 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:42:00.820 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:42:00.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:00.868 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:42:00.869 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:42:00.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:00.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:00.912 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:42:00.912 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:42:00.953 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:42:00.953 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:42:00.979 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:00.994 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:42:00.994 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:42:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:42:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:42:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:42:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:42:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:42:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:42:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:42:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:42:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:42:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:42:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:42:00.998 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:42:01.005 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:01.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:01.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:01.031 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:42:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:42:01.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:01.132 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:01.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:01.133 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:01.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:01.256 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:42:00 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:42:01.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:01.272 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:01.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:01.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:01.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:01.416 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:01.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:01.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:01 smithi175 ceph-mon[32613]: pgmap v74: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-21T11:42:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:01 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/501804348' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:01.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:01.563 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:01.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:01.564 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:01.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:01 smithi043 ceph-mon[29710]: pgmap v74: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-21T11:42:01.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:01 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:01.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:01.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/501804348' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:01.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:01 smithi043 ceph-mon[36454]: pgmap v74: 148 pgs: 148 active+clean; 4.1 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-21T11:42:01.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:01 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:01.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:01.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/501804348' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:01.653 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 43 16 73434 73418 106.689 68.9375 0.00838207 0.00936804 2024-09-21T11:42:01.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:01.708 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:01.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:01.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:01.810 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:01.840 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:01.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:01.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:01.942 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:01.968 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:01.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:01.969 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:01.991 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:01 smithi043 podman[75239]: 2024-09-21 11:42:01.626354124 +0000 UTC m=+2.490263105 container remove f988a64964725524b3a25c9126c8195df2911e33fb686b30209662ed3f6757f3 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:42:01.991 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:01 smithi043 podman[75239]: 2024-09-21 11:42:01.753039734 +0000 UTC m=+2.616948722 volume remove b000671e77d16f4ea924ab2d7e39ec8530f66e690278175041fd009df8a4c966 2024-09-21T11:42:02.070 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:02.099 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:02.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:02.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:02.130 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:42:02.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:02.243 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:02.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:02.244 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:02.345 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:02.372 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:02.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:02.373 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:02.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3748762210' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[36454]: osdmap e100: 8 total, 8 up, 8 in 2024-09-21T11:42:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:02.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3748762210' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:02.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:02.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:02.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:02.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:02 smithi043 ceph-mon[29710]: osdmap e100: 8 total, 8 up, 8 in 2024-09-21T11:42:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:02.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:02.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:02.617 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:02.644 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:02.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:02.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:02.653 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 44 16 74673 74657 106.024 77.4375 0.0132704 0.00942681 2024-09-21T11:42:02.746 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:02.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:02.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3748762210' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:02 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:02 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:02 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:02 smithi175 ceph-mon[32613]: osdmap e100: 8 total, 8 up, 8 in 2024-09-21T11:42:02.769 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:02.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:02.770 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:02.773 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:42:02.773 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:42:02.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.6 'trim stale osdmaps' 2024-09-21T11:42:02.870 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:02.894 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:02.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:02.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:02.991 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:02 smithi043 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@alertmanager.a.service: Deactivated successfully. 2024-09-21T11:42:02.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:02 smithi043 systemd[1]: Stopped Ceph alertmanager.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:42:02.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:02 smithi043 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@alertmanager.a.service: Consumed 1.261s CPU time. 2024-09-21T11:42:02.992 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:02 smithi043 systemd[1]: Starting Ceph alertmanager.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:42:02.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:03.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:03.044 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:03.044 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:03.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:03.172 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:03.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:03.173 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:03.273 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:03.299 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:03.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:03.300 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:03.401 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:03.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:03.432 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:03.432 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:03.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:03 smithi043 ceph-mon[29710]: pgmap v75: 148 pgs: 148 active+clean; 4.4 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:42:03.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:03 smithi043 ceph-mon[36454]: pgmap v75: 148 pgs: 148 active+clean; 4.4 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:42:03.533 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:03.560 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:03.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:03.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:03.653 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 45 16 75854 75838 105.308 73.8125 0.0119169 0.00949084 2024-09-21T11:42:03.662 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:03.691 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:03.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:03.692 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:03.758 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:03.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:03 smithi175 ceph-mon[32613]: pgmap v75: 148 pgs: 148 active+clean; 4.4 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:42:03.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:03.817 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:03.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:03.818 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:03.837 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:03.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:03.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:03.965 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:03.997 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:03 smithi043 podman[75726]: 2024-09-21 11:42:03.854090125 +0000 UTC m=+0.032029552 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-21T11:42:04.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:04.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:04.112 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:04.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:04.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:04.248 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:04.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:04.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:04.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:04.396 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:04.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:04.397 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:04.472 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:42:04.472 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:42:04.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:2994 7.8 7:16f65b36:::benchmark_data_smithi043_70646_object2993:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:19.681958+0000", 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.785952684999998, 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031920342999999997, 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2994 2024-09-21T11:42:04.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.681958+0000", 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.681958+0000", 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.681959+0000", 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1399999999999996e-07 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.681981+0000", 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1625999999999999e-05 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.681988+0000", 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6750000000000002e-06 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.681994+0000", 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9549999999999997e-06 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.682914+0000", 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00092033700000000002 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.683041+0000", 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012619199999999999 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,7", 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.683162+0000", 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120934 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.684049+0000", 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00088769399999999996 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.684761+0000", 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00071152200000000002 2024-09-21T11:42:04.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.713794+0000", 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029033519000000001 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.713837+0000", 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2928e-05 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:19.713879+0000", 2024-09-21T11:42:04.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1247e-05 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:8321 7.e 7:708013b0:::benchmark_data_smithi043_70646_object8320:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:21.543004+0000", 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 42.924907302000001, 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029992009, 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 8321 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.543004+0000", 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.543004+0000", 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.543005+0000", 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3820000000000001e-06 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.543030+0000", 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4678e-05 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.543041+0000", 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1619e-05 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.543054+0000", 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2332e-05 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.543113+0000", 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9888e-05 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.543254+0000", 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014051800000000001 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.543414+0000", 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160457 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.568328+0000", 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024913339 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.568535+0000", 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020713900000000001 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.572749+0000", 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042141330000000001 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.572898+0000", 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014891000000000001 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:21.572996+0000", 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7614000000000004e-05 2024-09-21T11:42:04.484 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:12353 7.e 7:7ee27185:::benchmark_data_smithi043_70646_object12352:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:23.088834+0000", 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.379076716999997, 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.033889932999999997, 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.486 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.486 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12353 2024-09-21T11:42:04.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.486 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.088834+0000", 2024-09-21T11:42:04.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.088834+0000", 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.088836+0000", 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.392e-06 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.088855+0000", 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9911e-05 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.088866+0000", 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0423e-05 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.088873+0000", 2024-09-21T11:42:04.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2559999999999996e-06 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.088933+0000", 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0013000000000003e-05 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.089046+0000", 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112613 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.089179+0000", 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013359400000000001 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.090070+0000", 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00089015399999999999 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.090418+0000", 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034861899999999999 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.122633+0000", 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.032214813000000002 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.122686+0000", 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3519e-05 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.122724+0000", 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7626000000000002e-05 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.491 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:12992 7.e 7:7e6c1046:::benchmark_data_smithi043_70646_object12991:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:23.375903+0000", 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.092008319999998, 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044075507999999999, 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12992 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.492 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.375903+0000", 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.375903+0000", 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.375904+0000", 2024-09-21T11:42:04.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2309999999999999e-06 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.375926+0000", 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2058000000000001e-05 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.375934+0000", 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3999999999999992e-06 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.375940+0000", 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.241e-06 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.376011+0000", 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0030999999999994e-05 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.376112+0000", 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010176299999999999 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.376238+0000", 2024-09-21T11:42:04.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012545000000000001 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.377373+0000", 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001135526 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378305+0000", 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00093215599999999996 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.419872+0000", 2024-09-21T11:42:04.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041566893000000001 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.419933+0000", 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0708999999999998e-05 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.419978+0000", 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5049999999999997e-05 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:12996 7.e 7:78b72738:::benchmark_data_smithi043_70646_object12995:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:23.377998+0000", 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.089912425000001, 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044143705999999998, 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12996 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.377998+0000", 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.377998+0000", 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378000+0000", 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5430000000000001e-06 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378022+0000", 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1576000000000001e-05 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378032+0000", 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0448000000000001e-05 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378039+0000", 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6959999999999999e-06 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378335+0000", 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00029643300000000003 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378454+0000", 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011907 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378583+0000", 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012849199999999999 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.380435+0000", 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018527039999999999 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.380693+0000", 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00025739799999999999 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.422043+0000", 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041349782000000002 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.422098+0000", 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5071000000000002e-05 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.422142+0000", 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4493000000000002e-05 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:12998 7.e 7:742544ae:::benchmark_data_smithi043_70646_object12997:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T11:42:04.503 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:23.378474+0000", 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.089436841000001, 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044037881000000001, 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 12998 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378474+0000", 2024-09-21T11:42:04.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378474+0000", 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378475+0000", 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.08e-06 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378494+0000", 2024-09-21T11:42:04.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9134e-05 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378505+0000", 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0349e-05 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.378511+0000", 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8750000000000002e-06 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379318+0000", 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00080648299999999998 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379435+0000", 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117343 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379645+0000", 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021009499999999999 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.380443+0000", 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000797429 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.380782+0000", 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00033966699999999999 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.422181+0000", 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041398311 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.422475+0000", 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00029429899999999998 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.422512+0000", 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6816000000000001e-05 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:13000 7.8 7:1c32cb06:::benchmark_data_smithi043_70646_object12999:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T11:42:04.509 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:23.379196+0000", 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.088714596000003, 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.037570800000000001, 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 13000 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379196+0000", 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379196+0000", 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379197+0000", 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3900000000000004e-07 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379224+0000", 2024-09-21T11:42:04.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7297999999999999e-05 2024-09-21T11:42:04.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379268+0000", 2024-09-21T11:42:04.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3571e-05 2024-09-21T11:42:04.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.512 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:04.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379278+0000", 2024-09-21T11:42:04.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6670000000000008e-06 2024-09-21T11:42:04.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379379+0000", 2024-09-21T11:42:04.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101649 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379740+0000", 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00036037100000000001 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,7", 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.379898+0000", 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015853800000000001 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.380644+0000", 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074540800000000003 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416586+0000", 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035942288000000003 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416653+0000", 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7527999999999994e-05 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416717+0000", 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3686e-05 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416767+0000", 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9957000000000003e-05 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:13000 7.8 e89/82 ondisk, result = 0)", 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:23.381022+0000", 2024-09-21T11:42:04.516 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.086888698999999, 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035572013999999999, 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.381022+0000", 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.381022+0000", 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.381023+0000", 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.61e-07 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.381026+0000", 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9040000000000001e-06 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.381036+0000", 2024-09-21T11:42:04.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0363000000000001e-05 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.381040+0000", 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9609999999999999e-06 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416568+0000", 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035527764000000003 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416578+0000", 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0349999999999999e-05 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416594+0000", 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6011e-05 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:13000 7.8 e89/82 ondisk, result = 0)", 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:23.381057+0000", 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 41.086854129000002, 2024-09-21T11:42:04.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035716942000000002, 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.381057+0000", 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.381057+0000", 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.381057+0000", 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6100000000000002e-07 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.381059+0000", 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.708e-06 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416435+0000", 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035375877999999999 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416449+0000", 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4810000000000001e-05 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416643+0000", 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019303899999999999 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416649+0000", 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.19e-06 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:23.416774+0000", 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012495599999999999 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:37383 7.e 7:7ebcbb39:::benchmark_data_smithi043_70646_object37382:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:35.541628+0000", 2024-09-21T11:42:04.524 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 28.926282999000001, 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063848501000000002, 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37383 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541628+0000", 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541628+0000", 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541629+0000", 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.626e-06 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541653+0000", 2024-09-21T11:42:04.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3728000000000001e-05 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541665+0000", 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1732999999999999e-05 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541673+0000", 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3720000000000005e-06 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541744+0000", 2024-09-21T11:42:04.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0279999999999998e-05 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541856+0000", 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112046 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541982+0000", 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012619899999999999 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.543479+0000", 2024-09-21T11:42:04.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014970300000000001 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.544149+0000", 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00066999599999999996 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.605387+0000", 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.061237993999999997 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.605438+0000", 2024-09-21T11:42:04.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1025e-05 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.605476+0000", 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8472e-05 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:39144 7.e 7:7215a9f9:::benchmark_data_smithi043_70646_object39143:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:36.599144+0000", 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.868766764, 2024-09-21T11:42:04.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.033218046000000001, 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 39144 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.599144+0000", 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.599144+0000", 2024-09-21T11:42:04.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.599145+0000", 2024-09-21T11:42:04.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8800000000000002e-07 2024-09-21T11:42:04.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.599169+0000", 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4640000000000001e-05 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.599178+0000", 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3750000000000003e-06 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.599184+0000", 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0780000000000001e-06 2024-09-21T11:42:04.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.609615+0000", 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010431039 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.609826+0000", 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021100399999999999 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.609961+0000", 2024-09-21T11:42:04.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013515599999999999 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.614001+0000", 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040401220000000002 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.614864+0000", 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00086315800000000005 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.632268+0000", 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017404069000000001 2024-09-21T11:42:04.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.632321+0000", 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2989999999999999e-05 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.632362+0000", 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0827000000000001e-05 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:39151 7.e 7:759a9712:::benchmark_data_smithi043_70646_object39150:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:42:04.537 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:36.601954+0000", 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 27.865956861000001, 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030523046000000002, 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 39151 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.601954+0000", 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.601954+0000", 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.601956+0000", 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.787e-06 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.601981+0000", 2024-09-21T11:42:04.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5193000000000001e-05 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.601992+0000", 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1234e-05 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.601999+0000", 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6159999999999998e-06 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.614892+0000", 2024-09-21T11:42:04.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012893254 2024-09-21T11:42:04.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.615009+0000", 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117001 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.615141+0000", 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013234600000000001 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.617244+0000", 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021021299999999998 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.621507+0000", 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042629540000000002 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.632403+0000", 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010896273999999999 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.632446+0000", 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3689999999999997e-05 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:36.632477+0000", 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0567000000000001e-05 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:42623 7.8 7:12c558a9:::benchmark_data_smithi043_70646_object42622:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:38.612359+0000", 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 25.855551467000002, 2024-09-21T11:42:04.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034680402999999999, 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 42623 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.612359+0000", 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.612359+0000", 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.612360+0000", 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3399999999999998e-07 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.612381+0000", 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.056e-05 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.612392+0000", 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.119e-05 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.612406+0000", 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3546e-05 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.615722+0000", 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033161079999999999 2024-09-21T11:42:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.615844+0000", 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000122559 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,7", 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.615969+0000", 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000124807 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.617858+0000", 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018892500000000001 2024-09-21T11:42:04.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.618026+0000", 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000167925 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.646933+0000", 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028906812 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.646993+0000", 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0189999999999998e-05 2024-09-21T11:42:04.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:38.647040+0000", 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6622000000000002e-05 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:48382 7.e 7:71a8e214:::benchmark_data_smithi043_70646_object48381:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:42.476781+0000", 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.991129595, 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.028897583000000001, 2024-09-21T11:42:04.550 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 48382 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.476781+0000", 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.476781+0000", 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.476783+0000", 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.686e-06 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.476810+0000", 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7110999999999998e-05 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.476819+0000", 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1509999999999993e-06 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.476826+0000", 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8589999999999999e-06 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.476863+0000", 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6876999999999999e-05 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.477017+0000", 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015401299999999999 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.477162+0000", 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014458599999999999 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.478938+0000", 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017762590000000001 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.480161+0000", 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001222901 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.505576+0000", 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.025415222000000001 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.505637+0000", 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0715999999999999e-05 2024-09-21T11:42:04.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.505679+0000", 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2202e-05 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:48426 7.e 7:71c81b20:::benchmark_data_smithi043_70646_object48425:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:42.520302+0000", 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.947608820999999, 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029967805, 2024-09-21T11:42:04.556 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 48426 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.520302+0000", 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.520302+0000", 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.520303+0000", 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.341e-06 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.520332+0000", 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9045999999999999e-05 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.520344+0000", 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1136e-05 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.520351+0000", 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5939999999999996e-06 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.533578+0000", 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013226921000000001 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.533697+0000", 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118845 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.533830+0000", 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013320300000000001 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.536011+0000", 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021805520000000001 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.544444+0000", 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0084332790000000001 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.550027+0000", 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055831880000000002 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.550209+0000", 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018171100000000001 2024-09-21T11:42:04.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:42.550270+0000", 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0989000000000002e-05 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:49081 7.e 7:72f5509c:::benchmark_data_smithi043_70646_object49080:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:43.084984+0000", 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 21.382926803, 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029592692, 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 49081 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.084984+0000", 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.084984+0000", 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.084986+0000", 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.477e-06 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.085011+0000", 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5950000000000001e-05 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.085022+0000", 2024-09-21T11:42:04.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0664e-05 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.085029+0000", 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.25e-06 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.085215+0000", 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000185887 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.085359+0000", 2024-09-21T11:42:04.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014376999999999999 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.085506+0000", 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147047 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.091697+0000", 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061911029999999999 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.091749+0000", 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1665000000000001e-05 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.114452+0000", 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022703015999999999 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.114534+0000", 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1660000000000001e-05 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:43.114577+0000", 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3203e-05 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:72752 7.e 7:76e7011d:::benchmark_data_smithi043_70646_object72751:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:01.034284+0000", 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.4336273309999998, 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036770767000000003, 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.568 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 72752 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.034284+0000", 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.034284+0000", 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.034284+0000", 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.85e-07 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.034313+0000", 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9068999999999999e-05 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.034323+0000", 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6350000000000001e-06 2024-09-21T11:42:04.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.034331+0000", 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9000000000000006e-06 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.041215+0000", 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0068844129999999998 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.046212+0000", 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049963079999999997 2024-09-21T11:42:04.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.046366+0000", 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000154666 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.049371+0000", 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003004793 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.049386+0000", 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4331999999999999e-05 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.070949+0000", 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021563432 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.071004+0000", 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4979999999999999e-05 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:01.071054+0000", 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0353999999999999e-05 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:74681 7.e 7:7557e768:::benchmark_data_smithi043_70646_object74680:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:02.659067+0000", 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.808844111, 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.030142927, 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 74681 2024-09-21T11:42:04.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.659067+0000", 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.659067+0000", 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.659067+0000", 2024-09-21T11:42:04.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9800000000000003e-07 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.659091+0000", 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3518000000000001e-05 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.659100+0000", 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6300000000000004e-06 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.659106+0000", 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6560000000000003e-06 2024-09-21T11:42:04.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.664432+0000", 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0053253880000000003 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.664548+0000", 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115955 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.664670+0000", 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012241899999999999 2024-09-21T11:42:04.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.668940+0000", 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042694930000000001 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.672186+0000", 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032469339999999999 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.688925+0000", 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016738466 2024-09-21T11:42:04.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.689091+0000", 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016577900000000001 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.689210+0000", 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118991 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.579 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:74748 7.e 7:70f79183:::benchmark_data_smithi043_70646_object74747:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:04.580 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:02.713483+0000", 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.7544274180000001, 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.02943111, 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 74748 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.713483+0000", 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.713483+0000", 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.713486+0000", 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1220000000000002e-06 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.713516+0000", 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0077e-05 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.713603+0000", 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7106000000000003e-05 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.713613+0000", 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0125e-05 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.713813+0000", 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020047399999999999 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.713958+0000", 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000145067 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.714105+0000", 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014686199999999999 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.720056+0000", 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059503359999999996 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.722772+0000", 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002716048 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.742478+0000", 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019706218000000001 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.742853+0000", 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037529700000000002 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.742915+0000", 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1377999999999994e-05 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:75020 7.e 7:7eb5901e:::benchmark_data_smithi043_70646_object75019:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:02.970512+0000", 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.4973987129999999, 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031201653999999999, 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 75020 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.970512+0000", 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.970512+0000", 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.970514+0000", 2024-09-21T11:42:04.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.423e-06 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.970536+0000", 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2699000000000001e-05 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.970547+0000", 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1082e-05 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.970554+0000", 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9190000000000002e-06 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.974415+0000", 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038607369999999999 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.974531+0000", 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011633699999999999 2024-09-21T11:42:04.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.974658+0000", 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000127107 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.978030+0000", 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033717220000000002 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:02.978055+0000", 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4481000000000002e-05 2024-09-21T11:42:04.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:03.001586+0000", 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023531718 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:03.001661+0000", 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5069999999999998e-05 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:03.001714+0000", 2024-09-21T11:42:04.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2358999999999998e-05 2024-09-21T11:42:04.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.593 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:04.593 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:04.593 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:42:04.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:04.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:04.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:04.653 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 46 15 77017 77002 104.6 72.75 0.00821062 0.00955493 2024-09-21T11:42:04.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:04.722 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:04.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:04.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:04.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:04.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:04.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:04.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:04.995 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:04.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:04.996 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:05.097 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:05.121 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:05.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:05.122 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:05.218 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:04 smithi043 podman[75726]: 2024-09-21 11:42:04.943129418 +0000 UTC m=+1.121068830 volume create f20d59193b5121e7a04a1131f719db91635bc3c75194a1d1b55e4e6e822f16f0 2024-09-21T11:42:05.218 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 podman[75726]: 2024-09-21 11:42:05.111205239 +0000 UTC m=+1.289144656 container create 366f1f2e4d315753050610509a0eb0f98c426a7779b74a6b48589a8c8092cd43 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:42:05.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:05.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:05.265 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:05.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:42:05.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:05.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:05.426 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:05.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:05.469 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:42:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:42:05.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-mon[29710]: pgmap v77: 148 pgs: 148 active+clean; 4.4 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:42:05.470 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-mon[36454]: pgmap v77: 148 pgs: 148 active+clean; 4.4 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:42:05.527 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:05.555 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:05.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:05.556 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:05.654 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 47 16 78194 78178 103.937 73.5 0.0246851 0.00961592 2024-09-21T11:42:05.657 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:05.683 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:05.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:05.684 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:05.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:05 smithi175 ceph-mon[32613]: pgmap v77: 148 pgs: 148 active+clean; 4.4 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:42:05.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:05.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:05.812 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:05.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:05.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:42:05.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:05.924 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 podman[75726]: 2024-09-21 11:42:05.62428359 +0000 UTC m=+1.802223008 container init 366f1f2e4d315753050610509a0eb0f98c426a7779b74a6b48589a8c8092cd43 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:42:05.924 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 podman[75726]: 2024-09-21 11:42:05.632590482 +0000 UTC m=+1.810529899 container start 366f1f2e4d315753050610509a0eb0f98c426a7779b74a6b48589a8c8092cd43 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T11:42:05.924 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T11:42:05.655Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-21T11:42:05.924 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T11:42:05.655Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-21T11:42:05.924 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T11:42:05.656Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.43 port=9094 2024-09-21T11:42:05.924 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T11:42:05.660Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-21T11:42:05.924 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T11:42:05.708Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-21T11:42:05.924 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T11:42:05.708Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-21T11:42:05.924 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T11:42:05.711Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-21T11:42:05.925 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T11:42:05.712Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-21T11:42:05.925 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 bash[75726]: 366f1f2e4d315753050610509a0eb0f98c426a7779b74a6b48589a8c8092cd43 2024-09-21T11:42:05.925 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:05 smithi043 systemd[1]: Started Ceph alertmanager.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:42:05.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:05.969 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:05.969 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:06.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:06.112 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:06.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:06.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:06.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:06.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:06.251 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:06.251 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:06.352 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:06.382 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:06.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:06.383 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:06.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:06.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:06.508 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:06.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:06.610 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:06.640 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:06.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:06.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:06.654 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 48 16 79217 79201 103.104 63.9375 0.00274115 0.00969184 2024-09-21T11:42:06.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:06.783 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:06.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:06.784 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:06.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:06.913 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:06.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:06.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:07.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:07.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:07.055 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:07.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:07.104 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:06 smithi175 ceph-mon[32613]: pgmap v78: 148 pgs: 148 active+clean; 4.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:42:07.104 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:06 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:07.104 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:06 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:07.104 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:06 smithi175 ceph-mon[32613]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-21T11:42:07.104 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:06 smithi175 ceph-mon[32613]: Reconfiguring daemon prometheus.a on smithi175 2024-09-21T11:42:07.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:07.197 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:07.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:07.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:07.215 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:06 smithi043 ceph-mon[29710]: pgmap v78: 148 pgs: 148 active+clean; 4.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:42:07.215 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:06 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:07.216 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:06 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:07.216 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:06 smithi043 ceph-mon[29710]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-21T11:42:07.216 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:06 smithi043 ceph-mon[29710]: Reconfiguring daemon prometheus.a on smithi175 2024-09-21T11:42:07.216 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:06 smithi043 ceph-mon[36454]: pgmap v78: 148 pgs: 148 active+clean; 4.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:42:07.216 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:06 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:07.216 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:06 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:07.216 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:06 smithi043 ceph-mon[36454]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-21T11:42:07.216 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:06 smithi043 ceph-mon[36454]: Reconfiguring daemon prometheus.a on smithi175 2024-09-21T11:42:07.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:07.344 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:07.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:07.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:07.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:07.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:07.488 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:07.488 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:07.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:07.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:42:07.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:07.616 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:07.616 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:07.655 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 49 16 80365 80349 102.463 71.75 0.0102595 0.0097543 2024-09-21T11:42:07.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:07.782 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:07.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:07.783 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:07.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:07.890 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:07 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T11:42:07.661Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00072118s 2024-09-21T11:42:07.892 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:42:07.892 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:42:07.892 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:42:07.892 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:42:07.892 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:42:07.893 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:42:07.893 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:42:07.893 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:42:07.893 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:42:07.893 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:42:07.893 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:42:07.893 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:42:07.895 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:07.906 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:07.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:07.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:08.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:08.057 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:08.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:08.058 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:08.159 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:08.205 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:08.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:08.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:08.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:08.336 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:08.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:08.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:08.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:08.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:08.488 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:08.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:08.617 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:08.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:08.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:08.655 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 50 16 81364 81348 101.662 62.4375 0.0064431 0.00983139 2024-09-21T11:42:08.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:08.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:08.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:08.765 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:08.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:08.895 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:08.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:08.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:08.939 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:08.954 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:42:08.955 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:42:08.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.7 'trim stale osdmaps' 2024-09-21T11:42:08.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:09.044 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:09.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:09.045 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:09.146 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:09.163 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:08 smithi175 ceph-mon[32613]: pgmap v79: 148 pgs: 148 active+clean; 4.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-21T11:42:09.172 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:09.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:09.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:09.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:08 smithi043 ceph-mon[29710]: pgmap v79: 148 pgs: 148 active+clean; 4.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-21T11:42:09.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:08 smithi043 ceph-mon[36454]: pgmap v79: 148 pgs: 148 active+clean; 4.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-21T11:42:09.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:09.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:09.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:09.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:09.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:09.465 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:09.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:09.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:09.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:09.621 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:09.638 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:09.655 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 51 16 82404 82388 100.943 65 0.0322632 0.00990113 2024-09-21T11:42:09.722 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:09.749 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:09.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:09.750 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:09.760 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=infra.usagestats t=2024-09-21T11:42:09.69377952Z level=info msg="Usage stats are ready to report" 2024-09-21T11:42:09.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 systemd[1]: Stopping Ceph prometheus.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:42:09.851 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:09.877 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:09.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:09.879 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:09.979 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:10.004 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:10.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:10.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:10.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:10.146 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:10.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:10.147 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:10.239 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.931Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.931Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.931Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.931Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.931Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.931Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.931Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.931Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.931Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.932Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.932Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[59619]: ts=2024-09-21T11:42:09.932Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-21T11:42:10.240 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:09 smithi175 podman[66290]: 2024-09-21 11:42:09.941410048 +0000 UTC m=+0.128913107 container died 0082b47ebe2bedd2f3f5989a604a6cf98346871d8600a15bf87f35515e349a9f (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:42:10.248 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:10.289 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:10.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:10.290 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:10.390 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:10.401 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:42:10.402 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:42:10.423 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:10.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:10.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:10.440 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:42:10.441 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:42:10.480 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:42:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:42:10.510 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:10 smithi175 podman[66290]: 2024-09-21 11:42:10.345862531 +0000 UTC m=+0.533365587 container cleanup 0082b47ebe2bedd2f3f5989a604a6cf98346871d8600a15bf87f35515e349a9f (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:42:10.510 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:10 smithi175 bash[66290]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a 2024-09-21T11:42:10.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:10.526 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:42:10.526 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:42:10.549 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:10.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:10.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:10.569 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:42:10.569 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:42:10.618 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:42:10.619 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:42:10.651 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:10.656 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 52 16 83266 83250 100.038 53.875 0.0187014 0.00999068 2024-09-21T11:42:10.673 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:42:10.673 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:42:10.677 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:10.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:10.678 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:10.686 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:42:10.686 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:42:10.722 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:42:10.722 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:42:10.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:10.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:10.807 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:10.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:10.869 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:42:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:42:10.870 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:10 smithi175 podman[66321]: 2024-09-21 11:42:10.614202758 +0000 UTC m=+0.673475555 container remove 0082b47ebe2bedd2f3f5989a604a6cf98346871d8600a15bf87f35515e349a9f (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:42:10.870 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:10 smithi175 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@prometheus.a.service: Deactivated successfully. 2024-09-21T11:42:10.870 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:10 smithi175 systemd[1]: Stopped Ceph prometheus.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:42:10.870 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:10 smithi175 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@prometheus.a.service: Consumed 1.161s CPU time. 2024-09-21T11:42:10.870 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:10 smithi175 systemd[1]: Starting Ceph prometheus.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:42:10.890 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:42:10.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:10.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:10.940 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:10.940 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:11.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:11.080 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:11.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:11.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:11.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:11.211 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:11.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:11.212 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:11.217 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:10 smithi175 ceph-mon[32613]: pgmap v80: 148 pgs: 148 active+clean; 4.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.16k op/s 2024-09-21T11:42:11.217 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/589612928' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:11.217 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:10 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:11.217 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1979067716' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:11.217 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 podman[66510]: 2024-09-21 11:42:10.961017289 +0000 UTC m=+0.026271078 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-21T11:42:11.218 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 podman[66510]: 2024-09-21 11:42:11.090648428 +0000 UTC m=+0.155902208 container create 36bc2df62037dfcfdc7817617cd5537af1fceb5f47cdaf8cf7b2bf394eb69549 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:42:11.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:10 smithi043 ceph-mon[29710]: pgmap v80: 148 pgs: 148 active+clean; 4.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.16k op/s 2024-09-21T11:42:11.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/589612928' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:10 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:11.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1979067716' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:10 smithi043 ceph-mon[36454]: pgmap v80: 148 pgs: 148 active+clean; 4.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 72 MiB/s wr, 1.16k op/s 2024-09-21T11:42:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/589612928' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:10 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1979067716' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:11.312 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:11.341 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:11.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:11.342 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:11.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:11.470 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 podman[66510]: 2024-09-21 11:42:11.359889603 +0000 UTC m=+0.425143401 container init 36bc2df62037dfcfdc7817617cd5537af1fceb5f47cdaf8cf7b2bf394eb69549 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 podman[66510]: 2024-09-21 11:42:11.363770528 +0000 UTC m=+0.429024314 container start 36bc2df62037dfcfdc7817617cd5537af1fceb5f47cdaf8cf7b2bf394eb69549 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.389Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.389Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.389Z caller=main.go:623 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi175 (none))" 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.389Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.389Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.391Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.392Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.394Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.394Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.398Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.398Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.225µs 2024-09-21T11:42:11.471 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.398Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-21T11:42:11.472 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.399Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-21T11:42:11.472 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.399Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-21T11:42:11.472 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.399Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=26.913µs wal_replay_duration=754.2µs wbl_replay_duration=93ns total_replay_duration=800.8µs 2024-09-21T11:42:11.472 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.401Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-21T11:42:11.472 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.401Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-21T11:42:11.472 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.402Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-21T11:42:11.472 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.424Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=22.404149ms db_storage=1.175µs remote_storage=1.58µs web_handler=718ns query_engine=1.07µs scrape=598.988µs scrape_sd=652.549µs notify=20.708µs notify_sd=18.398µs rules=20.537563ms tracing=8.243µs 2024-09-21T11:42:11.472 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.424Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-21T11:42:11.472 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T11:42:11.424Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-21T11:42:11.472 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 bash[66510]: 36bc2df62037dfcfdc7817617cd5537af1fceb5f47cdaf8cf7b2bf394eb69549 2024-09-21T11:42:11.482 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:11.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:11.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:11.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:11.621 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:11.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:11.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:11.655 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 53 16 84273 84257 99.3382 62.9375 0.0112844 0.0100604 2024-09-21T11:42:11.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:11.731 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:11] ENGINE Bus STOPPING 2024-09-21T11:42:11.748 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:11.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:11.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:11.760 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 11:42:11 smithi175 systemd[1]: Started Ceph prometheus.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T11:42:11.850 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:11.877 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:11.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:11.878 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:11.979 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:11.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T11:42:11.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:11] ENGINE Bus STOPPED 2024-09-21T11:42:11.991 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:11] ENGINE Bus STARTING 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: osdmap e101: 8 total, 8 up, 8 in 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: 3.13 scrub starts 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: 3.13 scrub ok 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: 4.16 scrub starts 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: 4.16 scrub ok 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-21T11:42:11.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"} : dispatch 2024-09-21T11:42:11.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:11.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-21T11:42:11.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"} : dispatch 2024-09-21T11:42:11.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:11.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-21T11:42:11.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"} : dispatch 2024-09-21T11:42:11.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:11.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:11.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: osdmap e101: 8 total, 8 up, 8 in 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: 3.13 scrub starts 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: 3.13 scrub ok 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: 4.16 scrub starts 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: 4.16 scrub ok 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:11.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-21T11:42:11.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"} : dispatch 2024-09-21T11:42:11.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:11.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-21T11:42:11.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"} : dispatch 2024-09-21T11:42:11.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:11.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-21T11:42:11.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"} : dispatch 2024-09-21T11:42:11.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:11 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:12.002 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:12.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:12.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:12.012 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:12.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:12.132 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:12.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:12.133 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:12.234 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:42:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: osdmap e101: 8 total, 8 up, 8 in 2024-09-21T11:42:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T11:42:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T11:42:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: 3.13 scrub starts 2024-09-21T11:42:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: 3.13 scrub ok 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: 4.16 scrub starts 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: 4.16 scrub ok 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"} : dispatch 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"} : dispatch 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-21T11:42:12.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"} : dispatch 2024-09-21T11:42:12.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:11 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:12.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:12.265 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:12.265 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:12.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:12.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:12.403 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:12.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:12.427 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:12] ENGINE Serving on http://:::9283 2024-09-21T11:42:12.427 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:12] ENGINE Bus STARTED 2024-09-21T11:42:12.427 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:12] ENGINE Bus STOPPING 2024-09-21T11:42:12.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:42:12.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:12.603 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:12.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:12.604 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:12.655 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 54 16 85288 85272 98.6733 63.4375 0.0183106 0.0101285 2024-09-21T11:42:12.684 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:12] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T11:42:12.684 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:12] ENGINE Bus STOPPED 2024-09-21T11:42:12.684 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:12] ENGINE Bus STARTING 2024-09-21T11:42:12.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:12.742 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:12.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:12.743 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:12.844 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:12.886 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:12.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:12.888 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:12.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T11:42:12.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T11:42:12.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T11:42:12.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T11:42:12.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T11:42:12.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T11:42:12.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: pgmap v82: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-21T11:42:12.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:42:12.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:42:12.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:42:12.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:42:12.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: 6.19 scrub starts 2024-09-21T11:42:12.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: 6.19 scrub ok 2024-09-21T11:42:12.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: osdmap e102: 8 total, 8 up, 8 in 2024-09-21T11:42:12.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:42:12.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:42:12.942 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:12] ENGINE Serving on http://:::9283 2024-09-21T11:42:12.943 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:12] ENGINE Bus STARTED 2024-09-21T11:42:12.943 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:12] ENGINE Bus STOPPING 2024-09-21T11:42:12.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T11:42:12.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T11:42:12.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T11:42:12.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T11:42:12.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T11:42:12.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T11:42:12.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: pgmap v82: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-21T11:42:12.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:42:12.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:42:12.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:42:12.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:42:12.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: 6.19 scrub starts 2024-09-21T11:42:12.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: 6.19 scrub ok 2024-09-21T11:42:12.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: osdmap e102: 8 total, 8 up, 8 in 2024-09-21T11:42:12.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:42:12.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:12 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:42:12.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-21T11:42:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-09-21T11:42:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-21T11:42:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"}]: dispatch 2024-09-21T11:42:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-21T11:42:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"}]: dispatch 2024-09-21T11:42:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: pgmap v82: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.0 KiB/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-21T11:42:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:42:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:42:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:42:13.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:42:13.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: 6.19 scrub starts 2024-09-21T11:42:13.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: 6.19 scrub ok 2024-09-21T11:42:13.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: osdmap e102: 8 total, 8 up, 8 in 2024-09-21T11:42:13.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:42:13.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:12 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:42:13.020 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:13.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:13.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:13.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:13.153 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:13.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:13.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:13.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:13.332 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:13.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:13.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:13.434 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:13.465 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T11:42:13.465 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:13] ENGINE Bus STOPPED 2024-09-21T11:42:13.465 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:13] ENGINE Bus STARTING 2024-09-21T11:42:13.465 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:13] ENGINE Serving on http://:::9283 2024-09-21T11:42:13.465 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:11:42:13] ENGINE Bus STARTED 2024-09-21T11:42:13.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:13.484 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:13.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:13.576 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:13.585 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:13.588 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:42:13.591 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:42:13.591 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:42:13.591 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:42:13.591 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:42:13.591 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:42:13.592 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:42:13.592 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:42:13.592 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:42:13.592 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:42:13.592 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:42:13.592 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:42:13.611 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:13.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:13.612 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:13.655 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 55 16 86308 86292 98.0379 63.75 0.0154113 0.0101956 2024-09-21T11:42:13.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:13.750 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:13.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:13.752 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:13.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:42:13.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:13.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:13.916 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:14.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:14.044 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:14.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:14.044 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:14.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:14.167 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:14.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:14.168 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:14.232 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:13 smithi175 ceph-mon[32613]: 6.13 scrub starts 2024-09-21T11:42:14.232 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:13 smithi175 ceph-mon[32613]: 6.13 scrub ok 2024-09-21T11:42:14.232 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:13 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:42:14.232 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:13 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:42:14.232 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:13 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:42:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-mon[36454]: 6.13 scrub starts 2024-09-21T11:42:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-mon[36454]: 6.13 scrub ok 2024-09-21T11:42:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:42:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:42:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:42:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-mon[29710]: 6.13 scrub starts 2024-09-21T11:42:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-mon[29710]: 6.13 scrub ok 2024-09-21T11:42:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:42:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T11:42:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:13 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T11:42:14.269 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:14.308 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:14.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:14.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:14.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:14.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:14.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:14.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:14.575 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:14.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:14.576 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:14.656 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 56 16 87304 87288 97.3988 62.25 0.0108208 0.0102625 2024-09-21T11:42:14.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:14.708 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:14.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:14.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:14.810 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:14.847 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:14.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:14.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:14.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:42:14.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:15.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:15.052 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:15.052 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:15.153 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:15.180 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:15.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:15.180 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:15.196 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:14 smithi175 ceph-mon[32613]: pgmap v84: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-21T11:42:15.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:14 smithi043 ceph-mon[29710]: pgmap v84: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-21T11:42:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:14 smithi043 ceph-mon[36454]: pgmap v84: 148 pgs: 148 active+clean; 5.1 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-21T11:42:15.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:15.343 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:15.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:15.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:15.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:15.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:15.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:15.612 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:15.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:15.644 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:15.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:15.656 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 57 16 88352 88336 96.8391 65.5 0.00965528 0.0103211 2024-09-21T11:42:15.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:15.779 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T11:42:15.664Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004127639s 2024-09-21T11:42:15.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:15.797 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:15.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:15.819 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:15.834 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:42:15.834 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:42:15.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:15.950 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:15.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:15.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:16.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:42:16.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:42:16.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[29710]: 5.1f scrub starts 2024-09-21T11:42:16.040 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[29710]: 5.1f scrub ok 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[29710]: osdmap e103: 8 total, 8 up, 8 in 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[29710]: 4.b scrub starts 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[29710]: 4.b scrub ok 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[36454]: 5.1f scrub starts 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[36454]: 5.1f scrub ok 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[36454]: osdmap e103: 8 total, 8 up, 8 in 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T11:42:16.041 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T11:42:16.042 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:16.042 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[36454]: 4.b scrub starts 2024-09-21T11:42:16.042 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:15 smithi043 ceph-mon[36454]: 4.b scrub ok 2024-09-21T11:42:16.051 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:16.080 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:16.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:16.081 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:16.182 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:16.210 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:16.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:16.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:15 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:42:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:15 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:42:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:15 smithi175 ceph-mon[32613]: 5.1f scrub starts 2024-09-21T11:42:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:15 smithi175 ceph-mon[32613]: 5.1f scrub ok 2024-09-21T11:42:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:15 smithi175 ceph-mon[32613]: osdmap e103: 8 total, 8 up, 8 in 2024-09-21T11:42:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:15 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T11:42:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:15 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T11:42:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:15 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:15 smithi175 ceph-mon[32613]: 4.b scrub starts 2024-09-21T11:42:16.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:15 smithi175 ceph-mon[32613]: 4.b scrub ok 2024-09-21T11:42:16.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:16.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:16.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:16.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:16.497 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:16.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:16.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:16.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:16.630 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:16.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:16.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:16.656 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 58 16 89393 89377 96.2904 65.0625 0.0218285 0.0103794 2024-09-21T11:42:16.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:16.765 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:16.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:16.766 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:16.867 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:16.902 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:16.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:16.903 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:16.961 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:16 smithi175 ceph-mon[32613]: 4.2 scrub starts 2024-09-21T11:42:16.961 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:16 smithi175 ceph-mon[32613]: 4.2 scrub ok 2024-09-21T11:42:16.961 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:16 smithi175 ceph-mon[32613]: 4.11 scrub starts 2024-09-21T11:42:16.961 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:16 smithi175 ceph-mon[32613]: 4.11 scrub ok 2024-09-21T11:42:16.961 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:16 smithi175 ceph-mon[32613]: pgmap v86: 148 pgs: 148 active+clean; 5.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-21T11:42:16.961 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:16 smithi175 ceph-mon[32613]: 7.d scrub starts 2024-09-21T11:42:17.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:17.029 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:17.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:17.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:17.130 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:17.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:17.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:17.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[36454]: 4.2 scrub starts 2024-09-21T11:42:17.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[36454]: 4.2 scrub ok 2024-09-21T11:42:17.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[36454]: 4.11 scrub starts 2024-09-21T11:42:17.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[36454]: 4.11 scrub ok 2024-09-21T11:42:17.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[36454]: pgmap v86: 148 pgs: 148 active+clean; 5.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-21T11:42:17.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[36454]: 7.d scrub starts 2024-09-21T11:42:17.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[29710]: 4.2 scrub starts 2024-09-21T11:42:17.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[29710]: 4.2 scrub ok 2024-09-21T11:42:17.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[29710]: 4.11 scrub starts 2024-09-21T11:42:17.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[29710]: 4.11 scrub ok 2024-09-21T11:42:17.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[29710]: pgmap v86: 148 pgs: 148 active+clean; 5.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-21T11:42:17.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:16 smithi043 ceph-mon[29710]: 7.d scrub starts 2024-09-21T11:42:17.258 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:42:17.258 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:42:17.268 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:17.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:17.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:17.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:17.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:17.511 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:17.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:17.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:17.657 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 59 16 90476 90460 95.8043 67.6875 0.0137366 0.010432 2024-09-21T11:42:17.682 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:17.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:17.683 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:17.784 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:17.809 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:17.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:17.810 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:17.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:17.935 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:17.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:17.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:18.014 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:18.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:18.062 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:18.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:18.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:18.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:18.209 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:18.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:18.210 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:18.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:18.353 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:18.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:18.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:18.439 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[29710]: osdmap e104: 8 total, 8 up, 8 in 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[29710]: 2.2 scrub starts 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[29710]: 2.2 scrub ok 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[29710]: 7.d scrub ok 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[29710]: 5.c scrub starts 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[29710]: 5.c scrub ok 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[36454]: osdmap e104: 8 total, 8 up, 8 in 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[36454]: 2.2 scrub starts 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[36454]: 2.2 scrub ok 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:18.440 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:18.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[36454]: 7.d scrub ok 2024-09-21T11:42:18.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[36454]: 5.c scrub starts 2024-09-21T11:42:18.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:18 smithi043 ceph-mon[36454]: 5.c scrub ok 2024-09-21T11:42:18.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:18.494 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:18.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:18.495 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:18.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:18 smithi175 ceph-mon[32613]: osdmap e104: 8 total, 8 up, 8 in 2024-09-21T11:42:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:18 smithi175 ceph-mon[32613]: 2.2 scrub starts 2024-09-21T11:42:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:18 smithi175 ceph-mon[32613]: 2.2 scrub ok 2024-09-21T11:42:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:18 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:18 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:18 smithi175 ceph-mon[32613]: 7.d scrub ok 2024-09-21T11:42:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:18 smithi175 ceph-mon[32613]: 5.c scrub starts 2024-09-21T11:42:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:18 smithi175 ceph-mon[32613]: 5.c scrub ok 2024-09-21T11:42:18.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:18.603 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:42:18.636 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:18.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:18.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:18.657 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:42:18.656764+0000 min lat: 0.0011233 max lat: 0.151666 avg lat: 0.0104873 2024-09-21T11:42:18.657 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:42:18.657 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 60 16 91520 91504 95.295 65.25 0.0128693 0.0104873 2024-09-21T11:42:18.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:18.765 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:18.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:18.766 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:18.867 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:18.891 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:18.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:18.892 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:18.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:19.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:19.044 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:19.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:19.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:19.153 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:42:19.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:19.203 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:19.209 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:42:19.209 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:42:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[36454]: pgmap v88: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 853 op/s 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[36454]: 3.1c scrub starts 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[36454]: 3.1c scrub ok 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2430827766' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[36454]: 5.a scrub starts 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[36454]: 5.a scrub ok 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[29710]: pgmap v88: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 853 op/s 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[29710]: 3.1c scrub starts 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[29710]: 3.1c scrub ok 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2430827766' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:19.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[29710]: 5.a scrub starts 2024-09-21T11:42:19.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:19 smithi043 ceph-mon[29710]: 5.a scrub ok 2024-09-21T11:42:19.256 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:42:19.256 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:42:19.293 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:42:19.293 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:42:19.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:19.328 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:42:19.329 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:42:19.354 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:19.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:19.355 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:19.366 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:42:19.366 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:42:19.398 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:42:19.399 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:42:19.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:19 smithi175 ceph-mon[32613]: pgmap v88: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 853 op/s 2024-09-21T11:42:19.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:19 smithi175 ceph-mon[32613]: 3.1c scrub starts 2024-09-21T11:42:19.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:19 smithi175 ceph-mon[32613]: 3.1c scrub ok 2024-09-21T11:42:19.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:19 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2430827766' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:19.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:19 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:19.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:19 smithi175 ceph-mon[32613]: 5.a scrub starts 2024-09-21T11:42:19.411 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:19 smithi175 ceph-mon[32613]: 5.a scrub ok 2024-09-21T11:42:19.438 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:42:19.438 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:42:19.456 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:19.476 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:42:19.476 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:42:19.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:19.486 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:19.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:19.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:19.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:19.636 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:19.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:19.657 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 61 16 92483 92467 94.7195 60.1875 0.00814514 0.0105523 2024-09-21T11:42:19.737 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:19.782 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:19.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:19.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:19.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:19.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:19.931 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:19.931 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:20.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:20.078 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:20.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:20.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:20.180 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:20.209 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:20.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:20.210 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:20.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:20.339 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:20.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:20.340 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:20.440 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:20.467 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:20.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:20.468 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:20.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:42:20.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:42:20.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4079866334' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:20.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:20.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[36454]: osdmap e105: 8 total, 8 up, 8 in 2024-09-21T11:42:20.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[36454]: 5.8 scrub starts 2024-09-21T11:42:20.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[36454]: 5.8 scrub ok 2024-09-21T11:42:20.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[36454]: 5.3 scrub starts 2024-09-21T11:42:20.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[36454]: 5.3 scrub ok 2024-09-21T11:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4079866334' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[29710]: osdmap e105: 8 total, 8 up, 8 in 2024-09-21T11:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[29710]: 5.8 scrub starts 2024-09-21T11:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[29710]: 5.8 scrub ok 2024-09-21T11:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[29710]: 5.3 scrub starts 2024-09-21T11:42:20.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:20 smithi043 ceph-mon[29710]: 5.3 scrub ok 2024-09-21T11:42:20.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:20 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:42:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:20 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:42:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4079866334' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:20 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:20 smithi175 ceph-mon[32613]: osdmap e105: 8 total, 8 up, 8 in 2024-09-21T11:42:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:20 smithi175 ceph-mon[32613]: 5.8 scrub starts 2024-09-21T11:42:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:20 smithi175 ceph-mon[32613]: 5.8 scrub ok 2024-09-21T11:42:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:20 smithi175 ceph-mon[32613]: 5.3 scrub starts 2024-09-21T11:42:20.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:20 smithi175 ceph-mon[32613]: 5.3 scrub ok 2024-09-21T11:42:20.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:20.607 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:20.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:20.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:20.657 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 62 15 93588 93573 94.3066 69.125 0.00420757 0.0105963 2024-09-21T11:42:20.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:20.756 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:20.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:20.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:20.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:20.896 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:20.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:20.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:20.998 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:21.028 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:21.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:21.029 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:21.130 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:21.153 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:42:20 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:42:21.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:21.158 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:21.158 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:21.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:21.303 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:21.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:21.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:21.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:21.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:21.449 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[29710]: 3.f scrub starts 2024-09-21T11:42:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[29710]: 3.f scrub ok 2024-09-21T11:42:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[29710]: 4.5 scrub starts 2024-09-21T11:42:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[29710]: 4.5 scrub ok 2024-09-21T11:42:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[29710]: pgmap v90: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 853 op/s 2024-09-21T11:42:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[29710]: 5.1d scrub starts 2024-09-21T11:42:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[29710]: 5.1d scrub ok 2024-09-21T11:42:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[36454]: 3.f scrub starts 2024-09-21T11:42:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[36454]: 3.f scrub ok 2024-09-21T11:42:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[36454]: 4.5 scrub starts 2024-09-21T11:42:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[36454]: 4.5 scrub ok 2024-09-21T11:42:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[36454]: pgmap v90: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 853 op/s 2024-09-21T11:42:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[36454]: 5.1d scrub starts 2024-09-21T11:42:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:21 smithi043 ceph-mon[36454]: 5.1d scrub ok 2024-09-21T11:42:21.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:21 smithi175 ceph-mon[32613]: 3.f scrub starts 2024-09-21T11:42:21.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:21 smithi175 ceph-mon[32613]: 3.f scrub ok 2024-09-21T11:42:21.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:21 smithi175 ceph-mon[32613]: 4.5 scrub starts 2024-09-21T11:42:21.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:21 smithi175 ceph-mon[32613]: 4.5 scrub ok 2024-09-21T11:42:21.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:21 smithi175 ceph-mon[32613]: pgmap v90: 148 pgs: 148 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 853 op/s 2024-09-21T11:42:21.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:21 smithi175 ceph-mon[32613]: 5.1d scrub starts 2024-09-21T11:42:21.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:21 smithi175 ceph-mon[32613]: 5.1d scrub ok 2024-09-21T11:42:21.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:21.580 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:21.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:21.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:21.658 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 63 16 94655 94639 93.8672 66.625 0.00447133 0.0106481 2024-09-21T11:42:21.682 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:21.709 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:21.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:21.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:21.811 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:21.840 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:21.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:21.841 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:21.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:21.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:21.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:22.098 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:22.125 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:22.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:22.126 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:22.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:42:22.227 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:22.254 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:22.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:22.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:22.259 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:42:22.259 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:42:22.260 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-21T11:42:22.260 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T11:42:22.260 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T11:42:22.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-21T11:42:22.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:22.380 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:22.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:22.381 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:22.443 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:22 smithi043 ceph-mon[29710]: 7.7 scrub starts 2024-09-21T11:42:22.443 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:22.443 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:22 smithi043 ceph-mon[29710]: osdmap e106: 8 total, 8 up, 8 in 2024-09-21T11:42:22.444 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:22 smithi043 ceph-mon[36454]: 7.7 scrub starts 2024-09-21T11:42:22.444 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:22.444 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:22 smithi043 ceph-mon[36454]: osdmap e106: 8 total, 8 up, 8 in 2024-09-21T11:42:22.444 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:42:22.444 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:42:22.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:42:22.444 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:42:22.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:37377 7.a 7:50bba3dd:::benchmark_data_smithi043_70646_object37376:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:35.539587+0000", 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.895838372, 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054190330000000002, 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37377 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.445 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.539587+0000", 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.539587+0000", 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.539588+0000", 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6139999999999999e-06 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.539668+0000", 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9834999999999997e-05 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.539680+0000", 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1334e-05 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.539687+0000", 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6920000000000002e-06 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.540686+0000", 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00099832999999999996 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.540794+0000", 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010829599999999999 2024-09-21T11:42:22.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.540915+0000", 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121094 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593600+0000", 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052685106000000002 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593638+0000", 2024-09-21T11:42:22.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7598000000000002e-05 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593665+0000", 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7047999999999999e-05 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593725+0000", 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0251999999999998e-05 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593777+0000", 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2130999999999999e-05 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:37380 7.a 7:55314f00:::benchmark_data_smithi043_70646_object37379:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:35.540641+0000", 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.894784506000001, 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.053313910999999999, 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37380 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.540641+0000", 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.540641+0000", 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.540642+0000", 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.176e-06 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.540663+0000", 2024-09-21T11:42:22.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0992000000000002e-05 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.540674+0000", 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1143e-05 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.540681+0000", 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2660000000000004e-06 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.590828+0000", 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050147205 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.590971+0000", 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014243200000000001 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.591099+0000", 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012797700000000001 2024-09-21T11:42:22.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593788+0000", 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026892959999999999 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593829+0000", 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1167000000000001e-05 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593886+0000", 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6811999999999997e-05 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593933+0000", 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7293999999999998e-05 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593955+0000", 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1151000000000001e-05 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:37385 7.a 7:5667064f:::benchmark_data_smithi043_70646_object37384:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:35.541881+0000", 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.893543680999997, 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052282151999999998, 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37385 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541881+0000", 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541881+0000", 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541882+0000", 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5899999999999995e-07 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541899+0000", 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6277999999999999e-05 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541907+0000", 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0169999999999996e-06 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.541911+0000", 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6970000000000001e-06 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.592365+0000", 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.050453359000000003 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.592474+0000", 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108843 2024-09-21T11:42:22.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.592566+0000", 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2443999999999998e-05 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.593963+0000", 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013967680000000001 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.594010+0000", 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7698000000000003e-05 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.594076+0000", 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5106999999999999e-05 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.594125+0000", 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9234000000000003e-05 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.594164+0000", 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8847999999999998e-05 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:79198 7.a 7:591bc924:::benchmark_data_smithi043_70646_object79197:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:06.629979+0000", 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.805445939, 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.055021826000000003, 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 79198 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.629979+0000", 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.629979+0000", 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.629981+0000", 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.578e-06 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.630002+0000", 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1483000000000001e-05 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.630014+0000", 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1477999999999999e-05 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.630021+0000", 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5190000000000003e-06 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.630102+0000", 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0859000000000006e-05 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.630229+0000", 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000126642 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.630370+0000", 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140989 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.634522+0000", 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041519520000000004 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.634565+0000", 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3043000000000001e-05 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.639870+0000", 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0053051679999999999 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.684945+0000", 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.04507535 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.685001+0000", 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5764999999999999e-05 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:79200 7.1 e100/82 7:832d4e39:::benchmark_data_smithi043_70646_object79199:head v 100'4962, mlcod=100'4960)", 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:06.632767+0000", 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.802658028, 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054911258999999997, 2024-09-21T11:42:22.470 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.632767+0000", 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.632767+0000", 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.632768+0000", 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9500000000000001e-07 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.632795+0000", 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6744e-05 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.632804+0000", 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2960000000000004e-06 2024-09-21T11:42:22.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.632810+0000", 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4999999999999996e-06 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.633462+0000", 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000651923 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.633483+0000", 2024-09-21T11:42:22.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0585e-05 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.684950+0000", 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051467127000000001 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.687678+0000", 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027282890000000001 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:79203 7.2 7:4820fa04:::benchmark_data_smithi043_70646_object79202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:06.634952+0000", 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.800472754999999, 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.053169043999999999, 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 79203 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.634952+0000", 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.634952+0000", 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.634954+0000", 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4330000000000001e-06 2024-09-21T11:42:22.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.634979+0000", 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.508e-05 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.634989+0000", 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0429e-05 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.634997+0000", 2024-09-21T11:42:22.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5920000000000003e-06 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635439+0000", 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000442003 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635545+0000", 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105789 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:42:22.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.636438+0000", 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00089300700000000002 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.687810+0000", 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051372534999999997 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.687889+0000", 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8652000000000005e-05 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.687947+0000", 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7645999999999997e-05 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.688078+0000", 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013117900000000001 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.688121+0000", 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3699000000000003e-05 2024-09-21T11:42:22.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:79205 7.a 7:5652c04d:::benchmark_data_smithi043_70646_object79204:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:06.635011+0000", 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.800413817000001, 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052944936999999997, 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 79205 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635011+0000", 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635011+0000", 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635012+0000", 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5199999999999996e-07 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635035+0000", 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3031e-05 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635043+0000", 2024-09-21T11:42:22.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7360000000000001e-06 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635049+0000", 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6229999999999999e-06 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.636548+0000", 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001498573 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.636691+0000", 2024-09-21T11:42:22.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000142886 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.636865+0000", 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017441099999999999 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.685046+0000", 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048180643000000002 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.685059+0000", 2024-09-21T11:42:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3602e-05 2024-09-21T11:42:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.687854+0000", 2024-09-21T11:42:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002794132 2024-09-21T11:42:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.687911+0000", 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7207000000000003e-05 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.687956+0000", 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5441000000000001e-05 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:79208 7.3 7:ce642e61:::benchmark_data_smithi043_70646_object79207:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:22.488 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:06.635951+0000", 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.799474266000001, 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054708367000000001, 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 79208 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635951+0000", 2024-09-21T11:42:22.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635951+0000", 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635952+0000", 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2999999999999999e-07 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635970+0000", 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8607999999999999e-05 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635979+0000", 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9330000000000007e-06 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.635987+0000", 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2429999999999999e-06 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.638098+0000", 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021114670000000001 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.685101+0000", 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.047002720999999997 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.685261+0000", 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160204 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.687575+0000", 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023137050000000001 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.687993+0000", 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00041867400000000001 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.690343+0000", 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023498899999999999 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.690505+0000", 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000162084 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.690659+0000", 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015390799999999999 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:79211 7.2 7:42cb9b7d:::benchmark_data_smithi043_70646_object79210:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:22.494 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:06.636346+0000", 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 15.799078772, 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.054951070999999997, 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 79211 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.636346+0000", 2024-09-21T11:42:22.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.636346+0000", 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.636347+0000", 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9899999999999999e-07 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.636369+0000", 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1647000000000001e-05 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.636378+0000", 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2469999999999997e-06 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.636384+0000", 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2500000000000003e-06 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.684982+0000", 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.048597643000000003 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.685138+0000", 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015594599999999999 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.685308+0000", 2024-09-21T11:42:22.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000169493 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.691122+0000", 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058149509999999996 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.691137+0000", 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4722000000000001e-05 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.691214+0000", 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7317999999999994e-05 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.691263+0000", 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8071000000000001e-05 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:06.691297+0000", 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4884e-05 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:80899 7.7 e100/82 7:e95db70c:::benchmark_data_smithi043_70646_object80898:head v 100'5181, mlcod=100'5178)", 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:08.200013+0000", 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.235412114000001, 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.056394249, 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.200013+0000", 2024-09-21T11:42:22.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.200013+0000", 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.200013+0000", 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.51e-07 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.200046+0000", 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2215999999999998e-05 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.200052+0000", 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7610000000000002e-06 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.200057+0000", 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7110000000000003e-06 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.203719+0000", 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036615089999999999 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.203740+0000", 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0914e-05 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.256358+0000", 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052618406999999999 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.256407+0000", 2024-09-21T11:42:22.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9280000000000003e-05 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:80905 7.7 e100/82 7:e8b89d02:::benchmark_data_smithi043_70646_object80904:head v 100'5182, mlcod=100'5178)", 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:08.207239+0000", 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 14.228185783000001, 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.051833606999999997, 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.505 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.207239+0000", 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.207239+0000", 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.207240+0000", 2024-09-21T11:42:22.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2900000000000003e-07 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.207287+0000", 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7372e-05 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.207297+0000", 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7559999999999994e-06 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.207303+0000", 2024-09-21T11:42:22.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.105e-06 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.209622+0000", 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023181930000000001 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.209637+0000", 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5923e-05 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.258992+0000", 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049354175 2024-09-21T11:42:22.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.259073+0000", 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1353999999999994e-05 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:82889 7.a 7:54634fd4:::benchmark_data_smithi043_70646_object82888:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:10.227135+0000", 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.208289941, 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.059807836000000003, 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 82889 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227135+0000", 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227135+0000", 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227137+0000", 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5850000000000001e-06 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227160+0000", 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3376000000000001e-05 2024-09-21T11:42:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227173+0000", 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2534999999999999e-05 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227180+0000", 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4919999999999996e-06 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.234781+0000", 2024-09-21T11:42:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076003010000000003 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.234904+0000", 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012354799999999999 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.235041+0000", 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136692 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.240713+0000", 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056720939999999999 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.240723+0000", 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0536e-05 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.286839+0000", 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046115399000000001 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.286897+0000", 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8635999999999999e-05 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.286943+0000", 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5642000000000001e-05 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:82890 7.2 7:4f821b1e:::benchmark_data_smithi043_70646_object82889:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:10.227193+0000", 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.208231794, 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052047090999999997, 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 82890 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227193+0000", 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227193+0000", 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227194+0000", 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2200000000000004e-07 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227217+0000", 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2838000000000001e-05 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227224+0000", 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.627e-06 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.227231+0000", 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0750000000000003e-06 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.236081+0000", 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088507550000000001 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.236206+0000", 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012473900000000001 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.236359+0000", 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000152678 2024-09-21T11:42:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.240770+0000", 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044111810000000001 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.241245+0000", 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00047497000000000001 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.279137+0000", 2024-09-21T11:42:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0378917 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.279194+0000", 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7346000000000003e-05 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.279240+0000", 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6560000000000001e-05 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:82896 7.a 7:51134ea0:::benchmark_data_smithi043_70646_object82895:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:10.236910+0000", 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.198515349000001, 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052194042000000003, 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 82896 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.236910+0000", 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.236910+0000", 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.236912+0000", 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7409999999999999e-06 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.236933+0000", 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1396e-05 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.236942+0000", 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7590000000000003e-06 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.236948+0000", 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1149999999999999e-06 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.240802+0000", 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038540530000000001 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.241471+0000", 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00066902499999999996 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.241610+0000", 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013924200000000001 2024-09-21T11:42:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.248683+0000", 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070732080000000001 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.249049+0000", 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00036601100000000002 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.288937+0000", 2024-09-21T11:42:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.039887480000000003 2024-09-21T11:42:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.527 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:22 smithi175 ceph-mon[32613]: 7.7 scrub starts 2024-09-21T11:42:22.527 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:22.528 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:22 smithi175 ceph-mon[32613]: osdmap e106: 8 total, 8 up, 8 in 2024-09-21T11:42:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.289054+0000", 2024-09-21T11:42:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117354 2024-09-21T11:42:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:10.289104+0000", 2024-09-21T11:42:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9657999999999998e-05 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:93572 7.2 7:428d15be:::benchmark_data_smithi043_70646_object93571:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:20.623167+0000", 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.8122577230000001, 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.070536225999999994, 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 93572 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623167+0000", 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623167+0000", 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623169+0000", 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8050000000000001e-06 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623201+0000", 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1458000000000003e-05 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623213+0000", 2024-09-21T11:42:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2719e-05 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623221+0000", 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9799999999999998e-06 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623481+0000", 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00025943400000000001 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623594+0000", 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113324 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623730+0000", 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013556700000000001 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.628687+0000", 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049576359999999996 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.628722+0000", 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4223000000000001e-05 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.691338+0000", 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.062616284999999994 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.693650+0000", 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023116460000000001 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.693704+0000", 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4148999999999999e-05 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:93573 7.2 7:46121aa5:::benchmark_data_smithi043_70646_object93572:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:20.623248+0000", 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.8121770179999999, 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078731641000000005, 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 93573 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623248+0000", 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623248+0000", 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623249+0000", 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7100000000000002e-07 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623351+0000", 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001027 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623361+0000", 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3209999999999994e-06 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623367+0000", 2024-09-21T11:42:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.144e-06 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.626247+0000", 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002880489 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.626362+0000", 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011466 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:42:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.626461+0000", 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9253000000000007e-05 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.628935+0000", 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00247335 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.643053+0000", 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014118575 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.701746+0000", 2024-09-21T11:42:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.058693161000000001 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.701939+0000", 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019299200000000001 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.701980+0000", 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0425e-05 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:93575 7.a 7:5e5b3032:::benchmark_data_smithi043_70646_object93574:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:20.623380+0000", 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.8120453589999999, 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078208274999999994, 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 93575 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623380+0000", 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623380+0000", 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623380+0000", 2024-09-21T11:42:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7700000000000004e-07 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623422+0000", 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1940000000000002e-05 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623429+0000", 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1940000000000003e-06 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623434+0000", 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4999999999999999e-06 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.626193+0000", 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027593769999999999 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.626310+0000", 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116562 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.626443+0000", 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000133325 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.628789+0000", 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002346093 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.628874+0000", 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5073000000000004e-05 2024-09-21T11:42:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.701337+0000", 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.072462917000000002 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.701543+0000", 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000205343 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.701588+0000", 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5374000000000003e-05 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:93579 7.3 7:c18b9a21:::benchmark_data_smithi043_70646_object93578:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:20.646689+0000", 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.7887363540000001, 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052735796000000001, 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 93579 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.646689+0000", 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.646689+0000", 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.646691+0000", 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-06 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.646714+0000", 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3263000000000001e-05 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.646729+0000", 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4450999999999999e-05 2024-09-21T11:42:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.646737+0000", 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4630000000000001e-06 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.646800+0000", 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2953000000000005e-05 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.646956+0000", 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015568099999999999 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.647122+0000", 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000166324 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.653097+0000", 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059752349999999997 2024-09-21T11:42:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.653166+0000", 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8257999999999996e-05 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.699112+0000", 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045946847999999998 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.699373+0000", 2024-09-21T11:42:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026048 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.699425+0000", 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1669999999999998e-05 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:93580 7.3 7:c39492de:::benchmark_data_smithi043_70646_object93579:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:20.647273+0000", 2024-09-21T11:42:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.788152381, 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052601255, 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 93580 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.647273+0000", 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.647273+0000", 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.647274+0000", 2024-09-21T11:42:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0020000000000001e-06 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.647297+0000", 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3366e-05 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.647308+0000", 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0983e-05 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.647315+0000", 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7279999999999998e-06 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.649288+0000", 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019729679999999999 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.649418+0000", 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012980000000000001 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.649558+0000", 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140111 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.653154+0000", 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035963570000000001 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.653170+0000", 2024-09-21T11:42:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5911000000000001e-05 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.699462+0000", 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046292453999999997 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.699832+0000", 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00036937899999999997 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.699874+0000", 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2196000000000001e-05 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:95292 7.2 7:4facb8ac:::benchmark_data_smithi043_70646_object95291:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:22.359513+0000", 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.075911821000000004, 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052261229999999999, 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 95292 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.359513+0000", 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.359513+0000", 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.359514+0000", 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1399999999999997e-07 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.359556+0000", 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2148999999999999e-05 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.359562+0000", 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7420000000000003e-06 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.359566+0000", 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7040000000000002e-06 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.374193+0000", 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014626801 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.374316+0000", 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012252399999999999 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.374451+0000", 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013493700000000001 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.385103+0000", 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010652735 2024-09-21T11:42:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.394316+0000", 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092131309999999994 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.410525+0000", 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016208231 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.411695+0000", 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011698419999999999 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:42:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:22.411775+0000", 2024-09-21T11:42:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0019999999999996e-05 2024-09-21T11:42:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:22.566 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:22.566 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:42:22.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:22.572 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:22.572 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:22.658 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 64 16 95532 95516 93.2564 54.8125 0.00423236 0.0107179 2024-09-21T11:42:22.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:22.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:22.702 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:22.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:22.803 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:22.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:22.849 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:22.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:23.002 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:23.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:23.003 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:23.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:23.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:23.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:23.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:23.281 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:23.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:23.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:23.383 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:23.409 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:23.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:23.410 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[36454]: pgmap v91: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.5 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-21T11:42:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[36454]: 4.e scrub starts 2024-09-21T11:42:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[36454]: 4.e scrub ok 2024-09-21T11:42:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[36454]: 7.7 scrub ok 2024-09-21T11:42:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[36454]: 3.12 scrub starts 2024-09-21T11:42:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[36454]: 3.12 scrub ok 2024-09-21T11:42:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[29710]: pgmap v91: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.5 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-21T11:42:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[29710]: 4.e scrub starts 2024-09-21T11:42:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[29710]: 4.e scrub ok 2024-09-21T11:42:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[29710]: 7.7 scrub ok 2024-09-21T11:42:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[29710]: 3.12 scrub starts 2024-09-21T11:42:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:23 smithi043 ceph-mon[29710]: 3.12 scrub ok 2024-09-21T11:42:23.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:23 smithi175 ceph-mon[32613]: pgmap v91: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.5 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-21T11:42:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:23 smithi175 ceph-mon[32613]: 4.e scrub starts 2024-09-21T11:42:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:23 smithi175 ceph-mon[32613]: 4.e scrub ok 2024-09-21T11:42:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:23 smithi175 ceph-mon[32613]: 7.7 scrub ok 2024-09-21T11:42:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:23 smithi175 ceph-mon[32613]: 3.12 scrub starts 2024-09-21T11:42:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:23 smithi175 ceph-mon[32613]: 3.12 scrub ok 2024-09-21T11:42:23.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:23.553 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:23.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:23.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:23.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:23.658 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 65 16 96648 96632 92.8947 69.75 0.0103983 0.0107597 2024-09-21T11:42:23.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:23.697 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:23.697 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:23.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:42:23.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:23.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:23.841 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:23.842 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:23.942 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:23.967 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:23.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:23.968 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:24.068 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:24.093 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:24.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:24.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:24.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:24.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:24.235 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:24.236 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:24.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:24.379 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:24.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:24.380 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:24.457 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[36454]: 3.11 scrub starts 2024-09-21T11:42:24.457 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[36454]: 3.11 scrub ok 2024-09-21T11:42:24.457 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[36454]: 6.f scrub starts 2024-09-21T11:42:24.458 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[36454]: 6.f scrub ok 2024-09-21T11:42:24.458 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[36454]: 3.19 scrub starts 2024-09-21T11:42:24.458 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[36454]: 3.19 scrub ok 2024-09-21T11:42:24.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[29710]: 3.11 scrub starts 2024-09-21T11:42:24.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[29710]: 3.11 scrub ok 2024-09-21T11:42:24.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[29710]: 6.f scrub starts 2024-09-21T11:42:24.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[29710]: 6.f scrub ok 2024-09-21T11:42:24.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[29710]: 3.19 scrub starts 2024-09-21T11:42:24.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:24 smithi043 ceph-mon[29710]: 3.19 scrub ok 2024-09-21T11:42:24.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:24.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:24 smithi175 ceph-mon[32613]: 3.11 scrub starts 2024-09-21T11:42:24.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:24 smithi175 ceph-mon[32613]: 3.11 scrub ok 2024-09-21T11:42:24.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:24 smithi175 ceph-mon[32613]: 6.f scrub starts 2024-09-21T11:42:24.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:24 smithi175 ceph-mon[32613]: 6.f scrub ok 2024-09-21T11:42:24.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:24 smithi175 ceph-mon[32613]: 3.19 scrub starts 2024-09-21T11:42:24.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:24 smithi175 ceph-mon[32613]: 3.19 scrub ok 2024-09-21T11:42:24.525 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:24.539 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:42:24.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:24.542 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:24.542 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:24.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:24.659 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 66 16 97636 97620 92.4211 61.75 0.0141214 0.010815 2024-09-21T11:42:24.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:24.677 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:24.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:24.778 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:24.820 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:24.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:24.821 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:24.899 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:24.922 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:24.951 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:24.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:24.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:25.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:25.093 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:25.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:25.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:25.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:25.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:25.242 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:25.243 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:25.272 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:42:25.272 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T11:42:25.272 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:42:25.308 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:25 smithi175 ceph-mon[32613]: 5.13 scrub starts 2024-09-21T11:42:25.309 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:25 smithi175 ceph-mon[32613]: 5.13 scrub ok 2024-09-21T11:42:25.309 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:25 smithi175 ceph-mon[32613]: pgmap v93: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 776 B/s rd, 50 MiB/s wr, 793 op/s 2024-09-21T11:42:25.309 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:25 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:42:25.309 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:25 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:42:25.309 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:25 smithi175 ceph-mon[32613]: 6.14 scrub starts 2024-09-21T11:42:25.309 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:25 smithi175 ceph-mon[32613]: 6.14 scrub ok 2024-09-21T11:42:25.309 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:25 smithi175 ceph-mon[32613]: 5.5 scrub starts 2024-09-21T11:42:25.309 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:25 smithi175 ceph-mon[32613]: 5.5 scrub ok 2024-09-21T11:42:25.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:25.396 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:25.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:25.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:25.446 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:42:25.446 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:42:25.447 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:42:25.447 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:42:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:42:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[36454]: 5.13 scrub starts 2024-09-21T11:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[36454]: 5.13 scrub ok 2024-09-21T11:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[36454]: pgmap v93: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 776 B/s rd, 50 MiB/s wr, 793 op/s 2024-09-21T11:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[36454]: 6.14 scrub starts 2024-09-21T11:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[36454]: 6.14 scrub ok 2024-09-21T11:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[36454]: 5.5 scrub starts 2024-09-21T11:42:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[36454]: 5.5 scrub ok 2024-09-21T11:42:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[29710]: 5.13 scrub starts 2024-09-21T11:42:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[29710]: 5.13 scrub ok 2024-09-21T11:42:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[29710]: pgmap v93: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 776 B/s rd, 50 MiB/s wr, 793 op/s 2024-09-21T11:42:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:42:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:42:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[29710]: 6.14 scrub starts 2024-09-21T11:42:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[29710]: 6.14 scrub ok 2024-09-21T11:42:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[29710]: 5.5 scrub starts 2024-09-21T11:42:25.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:25 smithi043 ceph-mon[29710]: 5.5 scrub ok 2024-09-21T11:42:25.498 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:25.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:25.522 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:25.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:25.623 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:25.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:25.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:25.661 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 67 16 98542 98526 91.8862 56.625 0.0287875 0.0108775 2024-09-21T11:42:25.761 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:25.786 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:25.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:25.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:25.888 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:25.910 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:25.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:25.911 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:26.012 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:26.036 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:26.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:26.037 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:26.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:26.179 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:26.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:26.180 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:26.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:26.305 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:26.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:26.321 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:26.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:26.443 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:26.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:26.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:26.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 4.14 scrub starts 2024-09-21T11:42:26.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 4.14 scrub ok 2024-09-21T11:42:26.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 3.1d scrub starts 2024-09-21T11:42:26.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 3.1d scrub ok 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: osdmap e107: 8 total, 8 up, 8 in 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 6.5 scrub starts 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 6.5 scrub ok 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 5.12 scrub starts 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 5.12 scrub ok 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 7.e scrub starts 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 5.e scrub starts 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[36454]: 5.e scrub ok 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 4.14 scrub starts 2024-09-21T11:42:26.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 4.14 scrub ok 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 3.1d scrub starts 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 3.1d scrub ok 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: osdmap e107: 8 total, 8 up, 8 in 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 6.5 scrub starts 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 6.5 scrub ok 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 5.12 scrub starts 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 5.12 scrub ok 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 7.e scrub starts 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 5.e scrub starts 2024-09-21T11:42:26.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:26 smithi043 ceph-mon[29710]: 5.e scrub ok 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 4.14 scrub starts 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 4.14 scrub ok 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 3.1d scrub starts 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 3.1d scrub ok 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: osdmap e107: 8 total, 8 up, 8 in 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 6.5 scrub starts 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 6.5 scrub ok 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 5.12 scrub starts 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 5.12 scrub ok 2024-09-21T11:42:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 7.e scrub starts 2024-09-21T11:42:26.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 5.e scrub starts 2024-09-21T11:42:26.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:26 smithi175 ceph-mon[32613]: 5.e scrub ok 2024-09-21T11:42:26.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:26.569 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:26.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:26.570 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:26.660 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 68 15 99230 99215 91.1682 43.0625 0.0131088 0.010963 2024-09-21T11:42:26.670 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:26.699 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:26.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:26.700 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:26.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:26.826 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:26.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:26.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:26.927 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:26.953 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:26.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:26.954 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:27.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:27.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:27.095 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:27.095 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:27.198 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:27.225 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:27.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:27.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:27.327 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:27.352 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:27.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:27.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:27.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:27 smithi043 ceph-mon[36454]: pgmap v95: 148 pgs: 148 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.5 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:42:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:27 smithi043 ceph-mon[36454]: 7.f scrub starts 2024-09-21T11:42:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:27 smithi043 ceph-mon[36454]: osdmap e108: 8 total, 8 up, 8 in 2024-09-21T11:42:27.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:27 smithi043 ceph-mon[29710]: pgmap v95: 148 pgs: 148 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.5 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:42:27.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:27 smithi043 ceph-mon[29710]: 7.f scrub starts 2024-09-21T11:42:27.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:27 smithi043 ceph-mon[29710]: osdmap e108: 8 total, 8 up, 8 in 2024-09-21T11:42:27.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:27.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:27.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:27 smithi175 ceph-mon[32613]: pgmap v95: 148 pgs: 148 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.5 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:42:27.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:27 smithi175 ceph-mon[32613]: 7.f scrub starts 2024-09-21T11:42:27.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:27 smithi175 ceph-mon[32613]: osdmap e108: 8 total, 8 up, 8 in 2024-09-21T11:42:27.592 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:42:27.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:27.621 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:27.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:27.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:27.660 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 69 16 100090 100074 90.625 53.6875 0.0713878 0.0110265 2024-09-21T11:42:27.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:27.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:27.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:27.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:27.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:27.902 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:27.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:27.903 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:27.904 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:28.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:28.051 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:28.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:28.052 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:28.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:28.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:28.186 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:28.186 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:28.281 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:28 smithi175 ceph-mon[32613]: 3.0 scrub starts 2024-09-21T11:42:28.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:28.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:28.332 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:28.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:28.478 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:28.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:28.480 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:28.580 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:28.602 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:28.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:28.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:28.619 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:28 smithi175 ceph-mon[32613]: 3.0 scrub ok 2024-09-21T11:42:28.619 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:28 smithi175 ceph-mon[32613]: 3.17 scrub starts 2024-09-21T11:42:28.619 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:28 smithi175 ceph-mon[32613]: 3.17 scrub ok 2024-09-21T11:42:28.625 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:42:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:42:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:42:28.625 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:42:28.625 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:42:28.660 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 70 16 100944 100928 90.0929 53.375 0.0072298 0.0110917 2024-09-21T11:42:28.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:28.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:28 smithi043 ceph-mon[29710]: 3.0 scrub starts 2024-09-21T11:42:28.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:28 smithi043 ceph-mon[29710]: 3.0 scrub ok 2024-09-21T11:42:28.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:28 smithi043 ceph-mon[29710]: 3.17 scrub starts 2024-09-21T11:42:28.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:28 smithi043 ceph-mon[29710]: 3.17 scrub ok 2024-09-21T11:42:28.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:28 smithi043 ceph-mon[36454]: 3.0 scrub starts 2024-09-21T11:42:28.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:28 smithi043 ceph-mon[36454]: 3.0 scrub ok 2024-09-21T11:42:28.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:28 smithi043 ceph-mon[36454]: 3.17 scrub starts 2024-09-21T11:42:28.745 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:28 smithi043 ceph-mon[36454]: 3.17 scrub ok 2024-09-21T11:42:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:28.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:28.753 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:28.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:28.884 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:28.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:28.885 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:28.985 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:29.014 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:29.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:29.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:29.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:29.140 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:29.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:29.141 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:29.241 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:29.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:29.283 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:29.283 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:29.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:29.427 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:29.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:29.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:29 smithi043 ceph-mon[29710]: pgmap v97: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 51 MiB/s wr, 823 op/s 2024-09-21T11:42:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:29 smithi043 ceph-mon[29710]: 7.e scrub ok 2024-09-21T11:42:29.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:29 smithi043 ceph-mon[36454]: pgmap v97: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 51 MiB/s wr, 823 op/s 2024-09-21T11:42:29.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:29 smithi043 ceph-mon[36454]: 7.e scrub ok 2024-09-21T11:42:29.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:29 smithi175 ceph-mon[32613]: pgmap v97: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 51 MiB/s wr, 823 op/s 2024-09-21T11:42:29.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:29 smithi175 ceph-mon[32613]: 7.e scrub ok 2024-09-21T11:42:29.529 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:29.559 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:29.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:29.560 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:29.661 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:29.663 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 71 16 101841 101825 89.6131 56.0625 0.0228027 0.0111537 2024-09-21T11:42:29.686 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:29.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:29.687 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:29.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:42:29.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:29.798 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:29.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:29.851 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:29.851 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:29.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:29.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:29.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:29.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:30.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:30.079 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:30.126 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:30.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:30.127 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:30.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:30.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:30.329 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:30.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:30.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:30.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:30.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:30 smithi175 ceph-mon[32613]: 3.b scrub starts 2024-09-21T11:42:30.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:30 smithi175 ceph-mon[32613]: 3.b scrub ok 2024-09-21T11:42:30.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:30 smithi175 ceph-mon[32613]: 7.b scrub starts 2024-09-21T11:42:30.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:30 smithi175 ceph-mon[32613]: 7.6 scrub starts 2024-09-21T11:42:30.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:30 smithi175 ceph-mon[32613]: 7.f scrub ok 2024-09-21T11:42:30.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:30.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:30.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:30.573 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[36454]: 3.b scrub starts 2024-09-21T11:42:30.573 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[36454]: 3.b scrub ok 2024-09-21T11:42:30.573 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[36454]: 7.b scrub starts 2024-09-21T11:42:30.573 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[36454]: 7.6 scrub starts 2024-09-21T11:42:30.573 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[36454]: 7.f scrub ok 2024-09-21T11:42:30.573 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:30.573 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:30.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[29710]: 3.b scrub starts 2024-09-21T11:42:30.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[29710]: 3.b scrub ok 2024-09-21T11:42:30.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[29710]: 7.b scrub starts 2024-09-21T11:42:30.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[29710]: 7.6 scrub starts 2024-09-21T11:42:30.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[29710]: 7.f scrub ok 2024-09-21T11:42:30.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:30.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:30.599 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:30.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:30.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:30.614 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:42:30.614 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:42:30.661 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 72 16 102578 102562 89.0082 46.0625 0.0127286 0.0112295 2024-09-21T11:42:30.675 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:42:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:42:30.681 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:42:30.681 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:42:30.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:42:30.681 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:42:30.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.681 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:2998 7.1 7:8ac39051:::benchmark_data_smithi043_70646_object2997:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T11:42:30.681 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:41:19.683955+0000", 2024-09-21T11:42:30.681 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.992639905000004, 2024-09-21T11:42:30.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031732397000000002, 2024-09-21T11:42:30.681 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 2998 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.683955+0000", 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.683955+0000", 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.683957+0000", 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2089999999999999e-06 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.684004+0000", 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7315999999999999e-05 2024-09-21T11:42:30.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.684016+0000", 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2174000000000001e-05 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.684024+0000", 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5170000000000002e-06 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.684085+0000", 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1099000000000006e-05 2024-09-21T11:42:30.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.711925+0000", 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.027840169000000001 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.712209+0000", 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028430300000000002 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.713718+0000", 2024-09-21T11:42:30.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015085579999999999 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.714495+0000", 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00077714799999999997 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.715618+0000", 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001123144 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.715650+0000", 2024-09-21T11:42:30.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2132000000000003e-05 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.715687+0000", 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6628000000000002e-05 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:3002 7.1 7:89ff5df4:::benchmark_data_smithi043_70646_object3001:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:41:19.684044+0000", 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.992550291000001, 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034224275999999998, 2024-09-21T11:42:30.687 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3002 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.684044+0000", 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.684044+0000", 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.684045+0000", 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9400000000000005e-07 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.684077+0000", 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2144999999999998e-05 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.684084+0000", 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6719999999999998e-06 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.684089+0000", 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.519e-06 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.713109+0000", 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.029019534 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.713242+0000", 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013308300000000001 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.713397+0000", 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000155594 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.715711+0000", 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023140050000000001 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.715728+0000", 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7110999999999999e-05 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.718169+0000", 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024406279999999998 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.718222+0000", 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3341999999999999e-05 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:19.718268+0000", 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6048999999999999e-05 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:12999 7.1 7:8c997897:::benchmark_data_smithi043_70646_object12998:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:41:23.379009+0000", 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 67.297585225999995, 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037840428000000002, 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.693 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.694 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.694 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 12999 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.379009+0000", 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.379009+0000", 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.379011+0000", 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.815e-06 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.379059+0000", 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8183000000000003e-05 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.379070+0000", 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0579e-05 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.379078+0000", 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7030000000000006e-06 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.379139+0000", 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1404999999999999e-05 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.379233+0000", 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4103000000000003e-05 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.379378+0000", 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000144917 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.380208+0000", 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00082996299999999997 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.381252+0000", 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010438139999999999 2024-09-21T11:42:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.416765+0000", 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.035513337999999998 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.416806+0000", 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0692999999999998e-05 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:41:23.416850+0000", 2024-09-21T11:42:30.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3915000000000002e-05 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:76472 7.1 7:8f4d0c01:::benchmark_data_smithi043_70646_object76471:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:04.179798+0000", 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.496796925000002, 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.039281345000000002, 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.700 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 76472 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.179798+0000", 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.179798+0000", 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.179799+0000", 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3540000000000001e-06 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.179845+0000", 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5865000000000002e-05 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.179858+0000", 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3607e-05 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.179865+0000", 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.263e-06 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.179938+0000", 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3424999999999996e-05 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.180061+0000", 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012275900000000001 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.180193+0000", 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013246699999999999 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.183314+0000", 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0031203189999999999 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.184583+0000", 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012689859999999999 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.218985+0000", 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.034402147000000001 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.219036+0000", 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1678999999999998e-05 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.219079+0000", 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2474e-05 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:76489 7.1 7:8de22e1a:::benchmark_data_smithi043_70646_object76488:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:04.197425+0000", 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.479169250999998, 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.038308130000000003, 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.706 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 76489 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.197425+0000", 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.197425+0000", 2024-09-21T11:42:30.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.197427+0000", 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6300000000000001e-06 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.197478+0000", 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1035999999999997e-05 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.197491+0000", 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3509000000000001e-05 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.197499+0000", 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8490000000000006e-06 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.197552+0000", 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2373999999999997e-05 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.197644+0000", 2024-09-21T11:42:30.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2035999999999998e-05 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.197725+0000", 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1792000000000006e-05 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.202386+0000", 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0046606299999999998 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.202430+0000", 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4159999999999997e-05 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.235657+0000", 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033226319999999997 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.235689+0000", 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2771000000000002e-05 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.235733+0000", 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4023000000000002e-05 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:76492 7.1 7:890fe7d8:::benchmark_data_smithi043_70646_object76491:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:04.198285+0000", 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 26.478309247999999, 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.037592049000000002, 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.712 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 76492 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.198285+0000", 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.198285+0000", 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.198286+0000", 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1990000000000001e-06 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.198364+0000", 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7758999999999999e-05 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.198379+0000", 2024-09-21T11:42:30.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4431e-05 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.198387+0000", 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0779999999999996e-06 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.200064+0000", 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001677483 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.200227+0000", 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000162628 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.200367+0000", 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013971200000000001 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.202556+0000", 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021896099999999998 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.203075+0000", 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051842499999999998 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.235790+0000", 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.032715401999999998 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.235835+0000", 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5531999999999997e-05 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:04.235877+0000", 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1789999999999998e-05 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:79200 7.1 7:832d4e39:::benchmark_data_smithi043_70646_object79199:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:06.631560+0000", 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 24.045034669, 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056443642000000002, 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.718 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 79200 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.631560+0000", 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.631560+0000", 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.631561+0000", 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5540000000000001e-06 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.631640+0000", 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9059000000000003e-05 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.631656+0000", 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5378000000000001e-05 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.631663+0000", 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2989999999999999e-06 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.631728+0000", 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4880000000000002e-05 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.721 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.631817+0000", 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9042999999999994e-05 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.632221+0000", 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040417900000000001 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.635180+0000", 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002958989 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.635633+0000", 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045249500000000002 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.687889+0000", 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052256660000000003 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.687955+0000", 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5810999999999997e-05 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.688003+0000", 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8294999999999998e-05 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:79210 7.1 7:8314a296:::benchmark_data_smithi043_70646_object79209:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:06.636321+0000", 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 24.040273066000001, 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.051780843, 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.724 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 79210 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636321+0000", 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636321+0000", 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636323+0000", 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5510000000000001e-06 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636367+0000", 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4329000000000002e-05 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636380+0000", 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.226e-05 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636386+0000", 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6749999999999996e-06 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636439+0000", 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3016000000000002e-05 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636575+0000", 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013594599999999999 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636734+0000", 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015869899999999999 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.639184+0000", 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024499140000000001 2024-09-21T11:42:30.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.639581+0000", 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039699899999999998 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.688042+0000", 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.048460743000000001 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.688077+0000", 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5298e-05 2024-09-21T11:42:30.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.688102+0000", 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5412999999999999e-05 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:79212 7.1 7:8cc77a04:::benchmark_data_smithi043_70646_object79211:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:06.639996+0000", 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 24.036598152, 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.050526639999999998, 2024-09-21T11:42:30.730 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 79212 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.639996+0000", 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.639996+0000", 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.640003+0000", 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3130000000000004e-06 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.640041+0000", 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8534000000000001e-05 2024-09-21T11:42:30.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.640055+0000", 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3579e-05 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.640062+0000", 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1729999999999998e-06 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.640110+0000", 2024-09-21T11:42:30.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8177000000000003e-05 2024-09-21T11:42:30.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:30.735 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:42:30.735 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:42:30.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.736 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.640224+0000", 2024-09-21T11:42:30.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114232 2024-09-21T11:42:30.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.736 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.640363+0000", 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000138617 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.642847+0000", 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024839139999999998 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.643084+0000", 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023713200000000001 2024-09-21T11:42:30.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.690427+0000", 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047342535999999998 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.690482+0000", 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5448999999999997e-05 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.690523+0000", 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0983999999999999e-05 2024-09-21T11:42:30.738 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:80908 7.1 7:85251c17:::benchmark_data_smithi043_70646_object80907:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:08.205653+0000", 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.470941367999998, 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047573866999999999, 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.739 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 80908 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.205653+0000", 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.205653+0000", 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.205655+0000", 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.491e-06 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.205705+0000", 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0850999999999998e-05 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.205717+0000", 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1109e-05 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.205722+0000", 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2229999999999998e-06 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.205774+0000", 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2301e-05 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.205898+0000", 2024-09-21T11:42:30.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012425700000000001 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206041+0000", 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014257899999999999 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.208641+0000", 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025996399999999999 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.743 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.208697+0000", 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6156999999999998e-05 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.253153+0000", 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044456017 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.253192+0000", 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9214000000000001e-05 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.253227+0000", 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5027999999999997e-05 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:80913 7.1 7:8f8add98:::benchmark_data_smithi043_70646_object80912:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:08.211475+0000", 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 22.465119318999999, 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.046313861999999997, 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.745 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 80913 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211475+0000", 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211475+0000", 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211477+0000", 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.61e-06 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211534+0000", 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7534000000000002e-05 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211550+0000", 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5639000000000001e-05 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211558+0000", 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8250000000000005e-06 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211627+0000", 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8986999999999995e-05 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211769+0000", 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014271499999999999 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211921+0000", 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015117900000000001 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.214143+0000", 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022226289999999998 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.214482+0000", 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033857599999999999 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.257703+0000", 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.043220789000000003 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.257745+0000", 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2718e-05 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.257789+0000", 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3661000000000001e-05 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:82054 7.1 7:8b6da476:::benchmark_data_smithi043_70646_object82053:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:09.289903+0000", 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 21.386691256999999, 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.049607945000000001, 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.751 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 82054 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.289903+0000", 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.752 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.289903+0000", 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.289904+0000", 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5099999999999998e-07 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.309499+0000", 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.019595311000000001 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.309511+0000", 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1143e-05 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.309518+0000", 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4569999999999999e-06 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.309580+0000", 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2348999999999995e-05 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.309731+0000", 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015050999999999999 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.309889+0000", 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000158085 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.312744+0000", 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028547059999999998 2024-09-21T11:42:30.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.312883+0000", 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000139271 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.339372+0000", 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.026488807999999999 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.339467+0000", 2024-09-21T11:42:30.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4871999999999996e-05 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.339511+0000", 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4481999999999999e-05 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:82531 7.1 7:8a8dcfe1:::benchmark_data_smithi043_70646_object82530:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:09.821956+0000", 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.854638766000001, 2024-09-21T11:42:30.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033489421999999998, 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 82531 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.821956+0000", 2024-09-21T11:42:30.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.821956+0000", 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.821957+0000", 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.66e-06 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.821996+0000", 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8299e-05 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.822006+0000", 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0034000000000001e-05 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.822011+0000", 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1460000000000004e-06 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.822066+0000", 2024-09-21T11:42:30.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5368000000000003e-05 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.822208+0000", 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014164900000000001 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.822361+0000", 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015350300000000001 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.824686+0000", 2024-09-21T11:42:30.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023246130000000001 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.826974+0000", 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002287982 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.855260+0000", 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.028286507999999998 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.855359+0000", 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8319000000000005e-05 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.855445+0000", 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6341000000000006e-05 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:82532 7.1 7:8f26de71:::benchmark_data_smithi043_70646_object82531:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:09.824534+0000", 2024-09-21T11:42:30.763 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.852060657999999, 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.031027836, 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 82532 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.824534+0000", 2024-09-21T11:42:30.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.824534+0000", 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.824535+0000", 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.443e-06 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.824649+0000", 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113334 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.824661+0000", 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2482e-05 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.824669+0000", 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8869999999999992e-06 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.824716+0000", 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7172000000000002e-05 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.824851+0000", 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013474099999999999 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.825016+0000", 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016512499999999999 2024-09-21T11:42:30.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.827295+0000", 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022793750000000002 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.827574+0000", 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027877199999999999 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.855493+0000", 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.027919112999999999 2024-09-21T11:42:30.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.855532+0000", 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8377000000000002e-05 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:09.855562+0000", 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0015e-05 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:94027 7.1 7:88c7e39e:::benchmark_data_smithi043_70646_object94026:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:21.071061+0000", 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.6055334810000002, 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.029943205000000001, 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 94027 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071061+0000", 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071061+0000", 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071062+0000", 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1340000000000001e-06 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071108+0000", 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5649000000000003e-05 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071118+0000", 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0291e-05 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071135+0000", 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6654e-05 2024-09-21T11:42:30.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071199+0000", 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4140000000000006e-05 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071320+0000", 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000121507 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071451+0000", 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013077799999999999 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.076102+0000", 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00465082 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.076145+0000", 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2851e-05 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.100909+0000", 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024764050999999999 2024-09-21T11:42:30.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.100960+0000", 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1581e-05 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.101004+0000", 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3748999999999997e-05 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:94029 7.1 7:8e64762a:::benchmark_data_smithi043_70646_object94028:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:21.071165+0000", 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.6054297379999998, 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.029944755, 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 94029 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071165+0000", 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071165+0000", 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071165+0000", 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8599999999999998e-07 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.777 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071193+0000", 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7813e-05 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071201+0000", 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4560000000000003e-06 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.071207+0000", 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3040000000000001e-06 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.073896+0000", 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026888540000000001 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.074025+0000", 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012907000000000001 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.074174+0000", 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000148794 2024-09-21T11:42:30.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.076227+0000", 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020536030000000002 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.076292+0000", 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4424999999999994e-05 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.101049+0000", 2024-09-21T11:42:30.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.024757601000000001 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.101086+0000", 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6282999999999998e-05 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:21.101109+0000", 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3966000000000001e-05 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.781 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:100081 7.1 7:84f19035:::benchmark_data_smithi043_70646_object100080:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:27.648861+0000", 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.0277336369999999, 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.033003049, 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100081 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.648861+0000", 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.648861+0000", 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.648862+0000", 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3489999999999999e-06 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.783 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.648913+0000", 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1035999999999997e-05 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.648924+0000", 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0791999999999999e-05 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.648939+0000", 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5024e-05 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.648990+0000", 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1348999999999999e-05 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.649069+0000", 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8429000000000004e-05 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.649153+0000", 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3778999999999999e-05 2024-09-21T11:42:30.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.651281+0000", 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002128291 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.651833+0000", 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00055241299999999997 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.681763+0000", 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.029929824000000001 2024-09-21T11:42:30.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.681819+0000", 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5909000000000003e-05 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:27.681864+0000", 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4854000000000001e-05 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.789 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:100727 7.1 7:8b911d35:::benchmark_data_smithi043_70646_object100726:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:42:30.789 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:28.388440+0000", 2024-09-21T11:42:30.789 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.288154069, 2024-09-21T11:42:30.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030210059000000001, 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100727 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388440+0000", 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388440+0000", 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388442+0000", 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.638e-06 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388487+0000", 2024-09-21T11:42:30.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5284000000000002e-05 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388498+0000", 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1042e-05 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388504+0000", 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7400000000000001e-06 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388549+0000", 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4492e-05 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388634+0000", 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5754000000000006e-05 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388724+0000", 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0138999999999999e-05 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.396422+0000", 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0076975660000000003 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.396433+0000", 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0951999999999999e-05 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.418555+0000", 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022122297999999999 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.418607+0000", 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1246999999999999e-05 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.418650+0000", 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3906999999999998e-05 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.795 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:100729 7.1 7:8fd6eafe:::benchmark_data_smithi043_70646_object100728:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:28.388585+0000", 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.2880092049999998, 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.030185475999999999, 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100729 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388585+0000", 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388585+0000", 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388586+0000", 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5199999999999996e-07 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388649+0000", 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2874999999999997e-05 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388654+0000", 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0050000000000004e-06 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.388657+0000", 2024-09-21T11:42:30.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3569999999999998e-06 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.391129+0000", 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024716260000000002 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.391247+0000", 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118436 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.391391+0000", 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143514 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.396461+0000", 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0050701640000000003 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.396470+0000", 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.884e-06 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.418699+0000", 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.022229576000000001 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.418748+0000", 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8945e-05 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.418771+0000", 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2442e-05 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:101051 7.1 7:86930eb5:::benchmark_data_smithi043_70646_object101050:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:28.760976+0000", 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.9156187089999999, 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.058579606999999999, 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 101051 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:42:30.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.760976+0000", 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.760976+0000", 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.760977+0000", 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.328e-06 2024-09-21T11:42:30.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761027+0000", 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9629000000000002e-05 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761039+0000", 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2199e-05 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761044+0000", 2024-09-21T11:42:30.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761093+0000", 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.871e-05 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761199+0000", 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105896 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:42:30.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761295+0000", 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5965999999999996e-05 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.764871+0000", 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035761209999999998 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.764946+0000", 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4849000000000001e-05 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:30.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.819492+0000", 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054546866999999999 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.819528+0000", 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5142000000000003e-05 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.819555+0000", 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7747999999999999e-05 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.807 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:42:30.808 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:42:30.808 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:42:30.809 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:42:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:42:30.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:30.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:30.819 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:42:30.819 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:42:30.874 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:42:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:42:30.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:30.923 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:42:30.923 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:42:30.938 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:30.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:30.939 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:30.963 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:42:30.963 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:42:31.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:31.085 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:31.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:31.086 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:31.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:31.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:31.242 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:31.254 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:42:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:42:31.343 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:31.370 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:31.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:31.371 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:31.472 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:31.498 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:31.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:31.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: 6.6 scrub starts 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: 6.6 scrub ok 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: pgmap v98: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 674 B/s rd, 41 MiB/s wr, 651 op/s 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: 6.12 scrub starts 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: 6.12 scrub ok 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4000383369' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3226713973' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:42:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:31 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:42:31.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:31.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:31.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:31.661 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 73 15 103430 103415 88.5193 53.3125 0.0282092 0.0112923 2024-09-21T11:42:31.669 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:42:31.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: 6.6 scrub starts 2024-09-21T11:42:31.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: 6.6 scrub ok 2024-09-21T11:42:31.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: pgmap v98: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 674 B/s rd, 41 MiB/s wr, 651 op/s 2024-09-21T11:42:31.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: 6.12 scrub starts 2024-09-21T11:42:31.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: 6.12 scrub ok 2024-09-21T11:42:31.673 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:31.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:31.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:42:31.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:42:31.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:31.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4000383369' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:31.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3226713973' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:42:31.674 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:42:31.674 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: 6.6 scrub starts 2024-09-21T11:42:31.674 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: 6.6 scrub ok 2024-09-21T11:42:31.674 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: pgmap v98: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 674 B/s rd, 41 MiB/s wr, 651 op/s 2024-09-21T11:42:31.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: 6.12 scrub starts 2024-09-21T11:42:31.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: 6.12 scrub ok 2024-09-21T11:42:31.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:31.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:31.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:42:31.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:42:31.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:31.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4000383369' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:31.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3226713973' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:42:31.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:31 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:42:31.719 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:42:31.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:31.773 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:31.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:31.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:31.875 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:31.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:31.910 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:31.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:32.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:32.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:32.045 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:32.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:32.107 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:32.146 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:32.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:32.192 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:32.192 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:32.293 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:32.323 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:32.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:32.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:32.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:32.467 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:32.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:32.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:32.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:32.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:32 smithi175 ceph-mon[32613]: 6.1c scrub starts 2024-09-21T11:42:32.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:32 smithi175 ceph-mon[32613]: 6.1c scrub ok 2024-09-21T11:42:32.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:32 smithi175 ceph-mon[32613]: 7.b scrub ok 2024-09-21T11:42:32.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:32 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:32.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:32 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:42:32.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:32 smithi175 ceph-mon[32613]: osdmap e109: 8 total, 8 up, 8 in 2024-09-21T11:42:32.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:32 smithi175 ceph-mon[32613]: osdmap e110: 8 total, 8 up, 8 in 2024-09-21T11:42:32.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:32.604 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:32.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:32.605 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:32.661 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 74 16 104441 104425 88.1761 63.125 0.0114615 0.0113359 2024-09-21T11:42:32.689 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:42:32.689 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:42:32.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:32.728 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:32.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:32.729 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:32.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:32.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[36454]: 6.1c scrub starts 2024-09-21T11:42:32.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[36454]: 6.1c scrub ok 2024-09-21T11:42:32.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[36454]: 7.b scrub ok 2024-09-21T11:42:32.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:32.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:42:32.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[36454]: osdmap e109: 8 total, 8 up, 8 in 2024-09-21T11:42:32.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[36454]: osdmap e110: 8 total, 8 up, 8 in 2024-09-21T11:42:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[29710]: 6.1c scrub starts 2024-09-21T11:42:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[29710]: 6.1c scrub ok 2024-09-21T11:42:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[29710]: 7.b scrub ok 2024-09-21T11:42:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:42:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[29710]: osdmap e109: 8 total, 8 up, 8 in 2024-09-21T11:42:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:32 smithi043 ceph-mon[29710]: osdmap e110: 8 total, 8 up, 8 in 2024-09-21T11:42:32.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:32.869 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:32.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:32.869 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:32.970 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:33.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:33.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:33.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:33.147 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:33.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:33.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:33.249 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:33.275 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:33.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:33.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:33.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:33.414 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:42:33.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:33.418 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:33.518 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:33.550 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:33.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:33.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:33.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:33.661 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 75 16 105484 105468 87.8695 65.1875 0.0116257 0.0113754 2024-09-21T11:42:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:33.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:33.698 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:33 smithi043 ceph-mon[36454]: 7.a scrub starts 2024-09-21T11:42:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:33 smithi043 ceph-mon[36454]: pgmap v100: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.5 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-21T11:42:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:33 smithi043 ceph-mon[29710]: 7.a scrub starts 2024-09-21T11:42:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:33 smithi043 ceph-mon[29710]: pgmap v100: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.5 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-21T11:42:33.759 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:33.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:33 smithi175 ceph-mon[32613]: 7.a scrub starts 2024-09-21T11:42:33.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:33 smithi175 ceph-mon[32613]: pgmap v100: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.5 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-21T11:42:33.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:33.846 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:33.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:33.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:33.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:33.990 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:33.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:33.992 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:34.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:34.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:34.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:34.237 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:42:34.238 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:42:34.238 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:42:34.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd out 1 2024-09-21T11:42:34.238 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:34.269 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:34.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:34.271 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:34.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:34.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:34.416 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:34.417 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:34.503 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:42:34.504 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:42:34.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:34.551 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:34.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:34.552 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:34.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:34.663 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 76 16 106527 106511 87.5691 65.1875 0.0197303 0.011414 2024-09-21T11:42:34.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:34.696 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:34.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:34.724 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:34.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:34.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:34.841 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:34.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:34.942 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:34.970 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:34.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:34.971 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:35.072 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:35.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:35.100 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:35.101 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:35.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:35.254 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:35.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:35.255 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:35.331 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:42:35.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:35.402 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:35.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:35.403 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:42:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:42:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:35 smithi043 ceph-mon[36454]: pgmap v102: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 776 B/s rd, 38 MiB/s wr, 612 op/s 2024-09-21T11:42:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:35 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:42:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:35 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:42:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:35 smithi043 ceph-mon[29710]: pgmap v102: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 776 B/s rd, 38 MiB/s wr, 612 op/s 2024-09-21T11:42:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:35 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:42:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:35 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:42:35.504 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:35.534 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:35.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:35.535 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:35.636 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:35.663 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 77 16 107579 107563 87.2857 65.75 0.0213699 0.0114511 2024-09-21T11:42:35.669 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:35.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:35.670 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:35.692 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:35 smithi175 ceph-mon[32613]: pgmap v102: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 776 B/s rd, 38 MiB/s wr, 612 op/s 2024-09-21T11:42:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:35 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:42:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:35 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:42:35.771 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:35.797 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:35.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:35.798 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:35.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:42:35.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:35.922 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:35.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:35.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:36.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:36.052 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:36.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:36.053 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:36.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:36.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:36.210 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:36.210 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:36.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a[29706]: 2024-09-21T11:42:35.780+0000 7f5b548dc640 -1 mon.a@0(leader).osd e111 definitely_dead 0 2024-09-21T11:42:36.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:42:36.303 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:36.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:36.314 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:42:36.314 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:42:36.340 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:36.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:36.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:36.352 INFO:teuthology.orchestra.run.smithi043.stderr:marked out osd.1. 2024-09-21T11:42:36.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:36.462 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:36.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:36.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:36.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:36 smithi175 ceph-mon[32613]: osdmap e111: 8 total, 8 up, 8 in 2024-09-21T11:42:36.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:36 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2198645946' entity='client.admin' cmd={"prefix": "osd out", "ids": ["1"]} : dispatch 2024-09-21T11:42:36.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:36 smithi175 ceph-mon[32613]: Client client.admin marked osd.1 out, while it was still marked up 2024-09-21T11:42:36.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:36.585 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:36.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:36.586 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:36 smithi043 ceph-mon[36454]: osdmap e111: 8 total, 8 up, 8 in 2024-09-21T11:42:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:36 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2198645946' entity='client.admin' cmd={"prefix": "osd out", "ids": ["1"]} : dispatch 2024-09-21T11:42:36.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:36 smithi043 ceph-mon[36454]: Client client.admin marked osd.1 out, while it was still marked up 2024-09-21T11:42:36.644 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:36 smithi043 ceph-mon[29710]: osdmap e111: 8 total, 8 up, 8 in 2024-09-21T11:42:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:36 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2198645946' entity='client.admin' cmd={"prefix": "osd out", "ids": ["1"]} : dispatch 2024-09-21T11:42:36.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:36 smithi043 ceph-mon[29710]: Client client.admin marked osd.1 out, while it was still marked up 2024-09-21T11:42:36.663 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 78 16 108290 108274 86.7364 44.4375 0.00341743 0.011484 2024-09-21T11:42:36.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:36.707 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:36.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:36.708 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:36.809 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:36.830 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:36.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:36.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:36.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:36.953 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:36.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:36.954 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:36.995 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:42:36.995 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:42:37.001 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:42:37.015 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:42:37.016 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:42:37.034 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:42:37.034 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:42:37.053 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:42:37.053 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:42:37.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:37.072 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:42:37.072 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:42:37.080 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:37.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:37.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:37.089 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:42:37.089 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:42:37.107 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:42:37.107 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:42:37.124 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:42:37.124 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:42:37.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:37.213 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:37.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:37.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:37.275 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:37.315 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:37.336 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:37.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:37.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:37.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:37.461 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:37.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:37.461 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:37.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:37.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:37.586 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:37.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:37.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:37 smithi043 ceph-mon[36454]: pgmap v104: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-09-21T11:42:37.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:37 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2198645946' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-09-21T11:42:37.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:37 smithi043 ceph-mon[36454]: osdmap e112: 8 total, 8 up, 7 in 2024-09-21T11:42:37.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:37 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/492653170' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:37.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:37 smithi043 ceph-mon[36454]: osdmap e113: 8 total, 8 up, 7 in 2024-09-21T11:42:37.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:37 smithi043 ceph-mon[29710]: pgmap v104: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-09-21T11:42:37.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:37 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2198645946' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-09-21T11:42:37.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:37 smithi043 ceph-mon[29710]: osdmap e112: 8 total, 8 up, 7 in 2024-09-21T11:42:37.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:37 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/492653170' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:37.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:37 smithi043 ceph-mon[29710]: osdmap e113: 8 total, 8 up, 7 in 2024-09-21T11:42:37.663 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 79 16 108290 108274 85.6386 0 - 0.011484 2024-09-21T11:42:37.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:37.708 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:37.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:37.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:37.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:37 smithi175 ceph-mon[32613]: pgmap v104: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.7 KiB/s rd, 61 MiB/s wr, 983 op/s 2024-09-21T11:42:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:37 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2198645946' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-09-21T11:42:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:37 smithi175 ceph-mon[32613]: osdmap e112: 8 total, 8 up, 7 in 2024-09-21T11:42:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:37 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/492653170' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:37 smithi175 ceph-mon[32613]: osdmap e113: 8 total, 8 up, 7 in 2024-09-21T11:42:37.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:37.834 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:37.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:37.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:37.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:37.959 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:37.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:37.960 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:38.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:38.082 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:38.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:38.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:38.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:38.209 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:38.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:38.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:38.311 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:38.326 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:42:38.326 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:42:38.326 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:42:38.326 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:42:38.342 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:38.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:38.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:38.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:38.478 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:38.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:38.479 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:38.580 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:38.607 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:38.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:38.609 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:38.628 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:38 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:42:38.628 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:38 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:42:38.628 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:38 smithi043 ceph-mon[36454]: 7.2 starting backfill to osd.7 from (0'0,0'0] MAX to 111'6623 2024-09-21T11:42:38.628 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:38 smithi043 ceph-mon[36454]: osdmap e114: 8 total, 8 up, 7 in 2024-09-21T11:42:38.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:38 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:42:38.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:38 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:42:38.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:38 smithi043 ceph-mon[29710]: 7.2 starting backfill to osd.7 from (0'0,0'0] MAX to 111'6623 2024-09-21T11:42:38.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:38 smithi043 ceph-mon[29710]: osdmap e114: 8 total, 8 up, 7 in 2024-09-21T11:42:38.663 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:42:38.662791+0000 min lat: 0.0011233 max lat: 1.99904 avg lat: 0.0117788 2024-09-21T11:42:38.663 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:42:38.663 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 80 16 108667 108651 84.8628 11.7812 0.0146696 0.0117788 2024-09-21T11:42:38.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:38.742 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:38.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:38.743 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:38.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:38 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:42:38.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:38 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:42:38.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:38 smithi175 ceph-mon[32613]: 7.2 starting backfill to osd.7 from (0'0,0'0] MAX to 111'6623 2024-09-21T11:42:38.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:38 smithi175 ceph-mon[32613]: osdmap e114: 8 total, 8 up, 7 in 2024-09-21T11:42:38.844 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:38.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:38.874 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:38.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:38.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:39.018 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:39.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:39.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:39.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:42:39.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:39.168 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:39.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:39.170 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:39.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:39.305 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:39.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:39.306 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:39.378 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:42:39.399 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:39.407 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:39.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:39.448 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:39.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:39.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:39.590 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:39.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:39.594 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:39.663 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 81 16 109597 109581 84.5326 58.125 0.0135569 0.011825 2024-09-21T11:42:39.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:39.718 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:39.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:39.719 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[36454]: pgmap v107: 148 pgs: 148 active+clean; 6.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 915 B/s rd, 57 MiB/s wr, 909 op/s 2024-09-21T11:42:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[36454]: 7.b starting backfill to osd.2 from (0'0,0'0] MAX to 111'6701 2024-09-21T11:42:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[36454]: 7.0 starting backfill to osd.2 from (0'0,0'0] MAX to 111'6791 2024-09-21T11:42:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[36454]: 7.d starting backfill to osd.3 from (0'0,0'0] MAX to 111'6782 2024-09-21T11:42:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[36454]: 7.4 starting backfill to osd.6 from (0'0,0'0] MAX to 111'6665 2024-09-21T11:42:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[36454]: 7.9 starting backfill to osd.7 from (0'0,0'0] MAX to 111'6695 2024-09-21T11:42:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2775728668' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[29710]: pgmap v107: 148 pgs: 148 active+clean; 6.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 915 B/s rd, 57 MiB/s wr, 909 op/s 2024-09-21T11:42:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[29710]: 7.b starting backfill to osd.2 from (0'0,0'0] MAX to 111'6701 2024-09-21T11:42:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[29710]: 7.0 starting backfill to osd.2 from (0'0,0'0] MAX to 111'6791 2024-09-21T11:42:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[29710]: 7.d starting backfill to osd.3 from (0'0,0'0] MAX to 111'6782 2024-09-21T11:42:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[29710]: 7.4 starting backfill to osd.6 from (0'0,0'0] MAX to 111'6665 2024-09-21T11:42:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[29710]: 7.9 starting backfill to osd.7 from (0'0,0'0] MAX to 111'6695 2024-09-21T11:42:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2775728668' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:39 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:39.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:39 smithi175 ceph-mon[32613]: pgmap v107: 148 pgs: 148 active+clean; 6.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 915 B/s rd, 57 MiB/s wr, 909 op/s 2024-09-21T11:42:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:39 smithi175 ceph-mon[32613]: 7.b starting backfill to osd.2 from (0'0,0'0] MAX to 111'6701 2024-09-21T11:42:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:39 smithi175 ceph-mon[32613]: 7.0 starting backfill to osd.2 from (0'0,0'0] MAX to 111'6791 2024-09-21T11:42:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:39 smithi175 ceph-mon[32613]: 7.d starting backfill to osd.3 from (0'0,0'0] MAX to 111'6782 2024-09-21T11:42:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:39 smithi175 ceph-mon[32613]: 7.4 starting backfill to osd.6 from (0'0,0'0] MAX to 111'6665 2024-09-21T11:42:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:39 smithi175 ceph-mon[32613]: 7.9 starting backfill to osd.7 from (0'0,0'0] MAX to 111'6695 2024-09-21T11:42:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2775728668' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:39 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:39.821 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:39.849 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:39.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:39.850 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:39.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:39.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:39.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:40.099 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:40.131 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:40.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:40.132 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:40.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:40.261 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:40.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:40.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:40.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:40.406 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:40.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:40.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:40.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:40.538 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:40.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:40.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:40.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:42:40.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:40.655 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:40 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:40.655 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:40 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:40.655 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:40 smithi175 ceph-mon[32613]: osdmap e115: 8 total, 8 up, 7 in 2024-09-21T11:42:40.665 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 82 16 110531 110515 84.2118 58.375 0.0183262 0.0118698 2024-09-21T11:42:40.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:40 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:40.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:40 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:40.677 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:40 smithi043 ceph-mon[29710]: osdmap e115: 8 total, 8 up, 7 in 2024-09-21T11:42:40.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:40 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:40.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:40 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:40.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:40 smithi043 ceph-mon[36454]: osdmap e115: 8 total, 8 up, 7 in 2024-09-21T11:42:40.702 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:42:40.703 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:42:40.703 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:42:40.703 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:42:40.703 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:42:40.723 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:40.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:40.724 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:40.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:40.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:40.874 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:40.874 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:40.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:41.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:42:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:42:41.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:41.043 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:41.043 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:41.120 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:41.144 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:41.173 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:41.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:41.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:41.275 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:41.304 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:41.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:41.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:41.406 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:41.434 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:41.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:41.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:41.518 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:41 smithi043 ceph-mon[36454]: pgmap v110: 148 pgs: 148 active+clean; 6.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 50 MiB/s wr, 804 op/s 2024-09-21T11:42:41.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:41 smithi043 ceph-mon[29710]: pgmap v110: 148 pgs: 148 active+clean; 6.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 50 MiB/s wr, 804 op/s 2024-09-21T11:42:41.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:41.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:42:41.586 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:41.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:41.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:41.665 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 83 16 111363 111347 83.8238 52 0.0150911 0.0119238 2024-09-21T11:42:41.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:41.719 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:41.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:41.720 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:41.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:41 smithi175 ceph-mon[32613]: pgmap v110: 148 pgs: 148 active+clean; 6.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 50 MiB/s wr, 804 op/s 2024-09-21T11:42:41.812 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:42:41.813 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:42:41.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:41.859 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:42:41.859 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:42:41.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:41.861 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:41.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:41.890 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:41.896 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:42:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:42:41.939 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:42:41.939 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:42:41.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:41.973 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:42:41.973 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:42:41.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:41.992 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:41.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:42.009 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:42:42.009 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:42:42.043 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:42:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:42:42.078 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:42:42.078 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:42:42.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:42.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:42.148 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:42.249 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:42.278 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:42.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:42.279 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:42.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:42.408 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:42.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:42.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:42.478 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:42:42.479 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:42:42.510 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:42.549 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:42.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:42.550 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:42.651 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:42.665 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 84 16 112178 112162 83.4323 50.9375 0.00886199 0.01198 2024-09-21T11:42:42.683 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:42.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:42.684 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/381077757' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:42 smithi043 ceph-mon[36454]: osdmap e116: 8 total, 8 up, 7 in 2024-09-21T11:42:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:42.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/381077757' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:42.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:42 smithi043 ceph-mon[29710]: osdmap e116: 8 total, 8 up, 7 in 2024-09-21T11:42:42.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:42.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/381077757' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:42 smithi175 ceph-mon[32613]: osdmap e116: 8 total, 8 up, 7 in 2024-09-21T11:42:42.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:42.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:42.825 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:42.825 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:42.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:42.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:42:42.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:42.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:43.051 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:42:43.051 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 0 1 2024-09-21T11:42:43.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:43.095 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:43.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:43.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:43.196 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:43.200 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:42:43.227 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:43.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:43.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:43.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:43.359 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:43.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:43.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:43.461 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:43.501 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:43.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:43.504 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:43.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:43.616 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:43.648 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:43.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:43.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:43.665 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 85 16 112981 112965 83.0412 50.1875 0.025632 0.0120372 2024-09-21T11:42:43.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:43 smithi043 ceph-mon[36454]: pgmap v111: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 34 MiB/s wr, 538 op/s; 39313/332514 objects misplaced (11.823%); 21 MiB/s, 361 objects/s recovering 2024-09-21T11:42:43.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:43 smithi043 ceph-mon[29710]: pgmap v111: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 34 MiB/s wr, 538 op/s; 39313/332514 objects misplaced (11.823%); 21 MiB/s, 361 objects/s recovering 2024-09-21T11:42:43.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:43.758 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:43 smithi175 ceph-mon[32613]: pgmap v111: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 34 MiB/s wr, 538 op/s; 39313/332514 objects misplaced (11.823%); 21 MiB/s, 361 objects/s recovering 2024-09-21T11:42:43.800 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:43.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:43.801 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:43.901 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:43.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:43.928 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:43.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:44.030 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:44.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:44.081 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:44.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:44.149 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:44.182 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:44.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:44.219 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:44.219 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:44.239 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:42:44.239 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:42:44.320 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:44.347 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:44.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:44.348 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:44.448 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:44.475 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:44.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:44.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:44.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:44.622 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:44.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:44.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:44.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:44 smithi043 ceph-mon[29710]: pgmap v113: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 30 MiB/s wr, 482 op/s; 39313/332514 objects misplaced (11.823%); 19 MiB/s, 323 objects/s recovering 2024-09-21T11:42:44.635 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:44 smithi043 ceph-mon[36454]: pgmap v113: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 30 MiB/s wr, 482 op/s; 39313/332514 objects misplaced (11.823%); 19 MiB/s, 323 objects/s recovering 2024-09-21T11:42:44.665 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 86 16 113830 113814 82.6926 53.0625 0.0166846 0.0120875 2024-09-21T11:42:44.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:44.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:44 smithi175 ceph-mon[32613]: pgmap v113: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 30 MiB/s wr, 482 op/s; 39313/332514 objects misplaced (11.823%); 19 MiB/s, 323 objects/s recovering 2024-09-21T11:42:44.766 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:44.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:44.767 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:44.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:44.918 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:44.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:44.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:45.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:42:45.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:45.048 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:42:45.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:45.058 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:45.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:45.159 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:45.193 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:45.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:45.194 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:45.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:45.335 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:45.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:45.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:45.366 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:45.417 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:42:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:42:45.420 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T11:42:45.426 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:42:45.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:45.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:45.497 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:45.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:45.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:45.626 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:45.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:45.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:45.666 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 87 16 114658 114642 82.3369 51.75 0.0156715 0.012135 2024-09-21T11:42:45.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:45.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/151503091' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/555695619' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/151503091' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/555695619' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:45.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:45 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:45.758 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:45.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:45.759 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/151503091' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:45 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:42:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/555695619' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:42:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:45 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:42:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:45 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:42:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:45 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:42:45.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:45.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:45.903 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:46.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:46.033 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:46.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:46.034 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:46.134 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:46.161 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:46.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:46.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:46.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:46.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:46.311 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:46.311 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:46.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:46.460 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:42:46.460 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:42:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:46.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:46.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:46.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:46.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:46.611 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:46.666 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 88 15 115561 115546 82.0432 56.5 0.0228612 0.0121827 2024-09-21T11:42:46.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:46.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:46 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:46.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:46 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:42:46.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:46 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:42:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:46 smithi043 ceph-mon[36454]: osdmap e117: 8 total, 8 up, 7 in 2024-09-21T11:42:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:46 smithi043 ceph-mon[36454]: pgmap v115: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 41 MiB/s wr, 657 op/s; 38960/336372 objects misplaced (11.582%); 24 MiB/s, 397 objects/s recovering 2024-09-21T11:42:46.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:46 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:46.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:46 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:42:46.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:46 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:42:46.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:46 smithi043 ceph-mon[29710]: osdmap e117: 8 total, 8 up, 7 in 2024-09-21T11:42:46.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:46 smithi043 ceph-mon[29710]: pgmap v115: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 41 MiB/s wr, 657 op/s; 38960/336372 objects misplaced (11.582%); 24 MiB/s, 397 objects/s recovering 2024-09-21T11:42:46.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:46 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:42:46.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:46 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:42:46.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:46 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:42:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:46 smithi175 ceph-mon[32613]: osdmap e117: 8 total, 8 up, 7 in 2024-09-21T11:42:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:46 smithi175 ceph-mon[32613]: pgmap v115: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 41 MiB/s wr, 657 op/s; 38960/336372 objects misplaced (11.582%); 24 MiB/s, 397 objects/s recovering 2024-09-21T11:42:46.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:46.766 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:46.767 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:46.869 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:46.903 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:46.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:46.904 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:47.005 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:47.033 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:47.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:47.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:47.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:47.150 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:42:47.186 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:47.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:47.187 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:47.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:47.329 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:47.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:47.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:47.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:47.477 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:47.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:47.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:47.515 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:42:47.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:47.622 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:47.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:47.623 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:47.666 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 89 16 116468 116452 81.7576 56.625 0.00893272 0.012226 2024-09-21T11:42:47.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:47.760 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:47.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:47.762 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:47.862 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:47.897 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:47.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:47.899 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:47.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:48.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:48.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:48.083 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:42:48.083 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:42:48.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:48.164 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:48.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:48.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:48.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:48 smithi175 ceph-mon[32613]: osdmap e118: 8 total, 8 up, 7 in 2024-09-21T11:42:48.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:48.293 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:48.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:48.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:48.395 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:48.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:48.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:48.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:48 smithi043 ceph-mon[36454]: osdmap e118: 8 total, 8 up, 7 in 2024-09-21T11:42:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:48 smithi043 ceph-mon[29710]: osdmap e118: 8 total, 8 up, 7 in 2024-09-21T11:42:48.528 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:48.556 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:48.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:48.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:48.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:48.666 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 90 6 117405 117399 81.5068 59.1875 0.0115711 0.012262 2024-09-21T11:42:48.687 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:48.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:48.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:48.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:48.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:42:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:48.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:48.821 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:48.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:48.948 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:48.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:48.949 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:49.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:49.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:49.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:49.151 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:49.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 2 1 2024-09-21T11:42:49.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:49.207 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:49.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:49.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:49.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:49.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:42:49.336 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:49.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:49.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:49.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:49.451 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:49 smithi043 ceph-mon[36454]: pgmap v117: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 43 MiB/s wr, 691 op/s; 37313/344958 objects misplaced (10.817%); 36 MiB/s, 574 objects/s recovering 2024-09-21T11:42:49.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:49 smithi043 ceph-mon[29710]: pgmap v117: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 43 MiB/s wr, 691 op/s; 37313/344958 objects misplaced (10.817%); 36 MiB/s, 574 objects/s recovering 2024-09-21T11:42:49.480 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:49.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:49.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:49.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:49 smithi175 ceph-mon[32613]: pgmap v117: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 853 B/s rd, 43 MiB/s wr, 691 op/s; 37313/344958 objects misplaced (10.817%); 36 MiB/s, 574 objects/s recovering 2024-09-21T11:42:49.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:49.618 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:49.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:49.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:49.666 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Total time run: 90.0485 2024-09-21T11:42:49.666 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Total writes made: 117405 2024-09-21T11:42:49.667 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Write size: 65536 2024-09-21T11:42:49.667 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Object size: 65536 2024-09-21T11:42:49.667 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Bandwidth (MB/sec): 81.4873 2024-09-21T11:42:49.667 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev Bandwidth: 34.4954 2024-09-21T11:42:49.667 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max bandwidth (MB/sec): 179.062 2024-09-21T11:42:49.667 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min bandwidth (MB/sec): 0 2024-09-21T11:42:49.668 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Average IOPS: 1303 2024-09-21T11:42:49.668 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev IOPS: 551.938 2024-09-21T11:42:49.668 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max IOPS: 2865 2024-09-21T11:42:49.668 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min IOPS: 0 2024-09-21T11:42:49.668 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Average Latency(s): 0.0122642 2024-09-21T11:42:49.668 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev Latency(s): 0.0237743 2024-09-21T11:42:49.668 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max latency(s): 1.99904 2024-09-21T11:42:49.669 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min latency(s): 0.0011233 2024-09-21T11:42:49.673 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Cleaning up (deleting benchmark objects) 2024-09-21T11:42:49.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:49.739 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:49.746 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:49.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:49.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:49.844 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:42:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:42:49.848 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:49.876 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:49.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:49.877 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:49.882 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:42:49.882 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:42:49.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:42:49.920 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:42:49.958 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:42:49.958 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:42:49.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:49.992 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:42:49.992 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:42:50.027 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:50.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:50.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:50.032 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:42:50.033 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:42:50.074 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:42:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:42:50.117 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:42:50.117 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:42:50.130 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:50.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:50.161 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:50.161 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:50.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:50.312 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:50.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:50.315 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:50.336 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1604036925' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:50.337 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1604036925' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:118003 7.7 e119/82 7:e25a38e7:::benchmark_data_smithi043_70646_object595:head v 119'7516, mlcod=119'7512)", 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:50.376422+0000", 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.017909943000000001, 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017936346999999998, 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:50.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.376422+0000", 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.376422+0000", 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.376423+0000", 2024-09-21T11:42:50.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1900000000000003e-07 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.376426+0000", 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5760000000000002e-06 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.376432+0000", 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7289999999999997e-06 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.376436+0000", 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3769999999999998e-06 2024-09-21T11:42:50.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.385100+0000", 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0086635940000000002 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.385120+0000", 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9779000000000002e-05 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.400 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:118011 7.7 e119/82 7:e751eca2:::benchmark_data_smithi043_70646_object603:head v 119'7517, mlcod=119'7513)", 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:50.385561+0000", 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0087711990000000004, 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088514690000000007, 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.385561+0000", 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.385561+0000", 2024-09-21T11:42:50.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.385561+0000", 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7899999999999998e-07 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.385573+0000", 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.188e-05 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:50.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.385583+0000", 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0071999999999999e-05 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.385590+0000", 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3339999999999998e-06 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.389040+0000", 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034505959999999998 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:50.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.389058+0000", 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7340999999999999e-05 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:118014 7.6 7:61b55b55:::benchmark_data_smithi043_70646_object606:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:50.387050+0000", 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0072821250000000004, 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007413439, 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:42:50.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 118014 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.387050+0000", 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.387050+0000", 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:50.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.387055+0000", 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2979999999999999e-06 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.387060+0000", 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0209999999999999e-06 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.387070+0000", 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0237999999999999e-05 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.387080+0000", 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2420000000000006e-06 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.387134+0000", 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4682000000000003e-05 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.387243+0000", 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108732 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.387383+0000", 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013978100000000001 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.393507+0000", 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061242110000000001 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.393561+0000", 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4153999999999997e-05 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:50.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:118015 7.7 e119/82 7:e1dec1fc:::benchmark_data_smithi043_70646_object607:head v 119'7518, mlcod=119'7513)", 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:50.389278+0000", 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0050543799999999998, 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052528779999999999, 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.389278+0000", 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.389278+0000", 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.389278+0000", 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3799999999999997e-07 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.389283+0000", 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2179999999999998e-06 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.389291+0000", 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3459999999999996e-06 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.389297+0000", 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6460000000000004e-06 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.391710+0000", 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024121440000000002 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:42:50.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:50.391728+0000", 2024-09-21T11:42:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8420999999999999e-05 2024-09-21T11:42:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:42:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:42:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:42:50.412 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 4 2024-09-21T11:42:50.412 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:42:50.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:50.446 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:50.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:50.447 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:50.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1604036925' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:42:50.548 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:50.577 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:50.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:50.578 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:50.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:50.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:50.707 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:50.707 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:50.808 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:50.840 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:50.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:50.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:50.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:50.986 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:50.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:50.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:51.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:51.137 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:51.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:51.138 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:51.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:42:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:42:51.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:51.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:51.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:51.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:51.417 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:51.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:51.418 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:51.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[29710]: pgmap v118: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 674 B/s rd, 34 MiB/s wr, 547 op/s; 37313/344958 objects misplaced (10.817%); 28 MiB/s, 454 objects/s recovering 2024-09-21T11:42:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:42:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:42:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[29710]: 6.e scrub starts 2024-09-21T11:42:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[29710]: 6.e scrub ok 2024-09-21T11:42:51.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[29710]: osdmap e119: 8 total, 8 up, 7 in 2024-09-21T11:42:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[36454]: pgmap v118: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 674 B/s rd, 34 MiB/s wr, 547 op/s; 37313/344958 objects misplaced (10.817%); 28 MiB/s, 454 objects/s recovering 2024-09-21T11:42:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:42:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:42:51.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[36454]: 6.e scrub starts 2024-09-21T11:42:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[36454]: 6.e scrub ok 2024-09-21T11:42:51.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:51 smithi043 ceph-mon[36454]: osdmap e119: 8 total, 8 up, 7 in 2024-09-21T11:42:51.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:51 smithi175 ceph-mon[32613]: pgmap v118: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.0 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 674 B/s rd, 34 MiB/s wr, 547 op/s; 37313/344958 objects misplaced (10.817%); 28 MiB/s, 454 objects/s recovering 2024-09-21T11:42:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:51 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:42:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:51 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:42:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:51 smithi175 ceph-mon[32613]: 6.e scrub starts 2024-09-21T11:42:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:51 smithi175 ceph-mon[32613]: 6.e scrub ok 2024-09-21T11:42:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:51 smithi175 ceph-mon[32613]: osdmap e119: 8 total, 8 up, 7 in 2024-09-21T11:42:51.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:51.561 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:51.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:51.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:51.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:51.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:51.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:51.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:51.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:51.846 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:51.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:51.947 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:51.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:42:51.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:51.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:51.976 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:52.077 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:52.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:52.106 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:52.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:52.207 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:52.234 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:52.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:52.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:52.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:52.396 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:52.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:52.397 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:52.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:52.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:52 smithi043 ceph-mon[29710]: osdmap e120: 8 total, 8 up, 7 in 2024-09-21T11:42:52.466 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:52.466 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:52 smithi043 ceph-mon[36454]: osdmap e120: 8 total, 8 up, 7 in 2024-09-21T11:42:52.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:52.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:42:52.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:52 smithi175 ceph-mon[32613]: osdmap e120: 8 total, 8 up, 7 in 2024-09-21T11:42:52.543 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:52.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:52.544 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:52.645 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:52.674 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:52.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:52.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:52.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:52.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:52.827 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:52.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:52.990 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:52.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:52.991 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:53.024 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:42:53.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:53.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:53.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:53.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:42:53.228 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T11:42:53.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:53.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:53.322 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:53.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:53.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:53 smithi043 ceph-mon[29710]: pgmap v120: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.7 KiB/s rd, 67 MiB/s wr, 1.24k op/s; 32064/349584 objects misplaced (9.172%); 91 MiB/s, 1.46k objects/s recovering 2024-09-21T11:42:53.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:53 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2678699437' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:42:53.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:53 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:42:53.421 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:53 smithi043 ceph-mon[36454]: pgmap v120: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.7 KiB/s rd, 67 MiB/s wr, 1.24k op/s; 32064/349584 objects misplaced (9.172%); 91 MiB/s, 1.46k objects/s recovering 2024-09-21T11:42:53.421 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:53 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2678699437' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:42:53.421 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:53 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:42:53.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:53.503 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:53.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:53.504 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:53 smithi175 ceph-mon[32613]: pgmap v120: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.7 KiB/s rd, 67 MiB/s wr, 1.24k op/s; 32064/349584 objects misplaced (9.172%); 91 MiB/s, 1.46k objects/s recovering 2024-09-21T11:42:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:53 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2678699437' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:42:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:53 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:42:53.604 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:53.678 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:53.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:53.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:53.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:53.811 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:53.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:53.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:53.913 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:53.941 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:53.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:53.942 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:54.043 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:54.073 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:54.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:54.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:54.175 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:54.206 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:54.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:54.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:54.233 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:42:54.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:54.333 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:54.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:54.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:54.434 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:54.478 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:54.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:54.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:54 smithi043 ceph-mon[36454]: 3.3 scrub starts 2024-09-21T11:42:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:54 smithi043 ceph-mon[36454]: 3.3 scrub ok 2024-09-21T11:42:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:54 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:42:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:54 smithi043 ceph-mon[36454]: osdmap e121: 8 total, 8 up, 7 in 2024-09-21T11:42:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:54 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1691340531' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:54.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:54 smithi043 ceph-mon[29710]: 3.3 scrub starts 2024-09-21T11:42:54.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:54 smithi043 ceph-mon[29710]: 3.3 scrub ok 2024-09-21T11:42:54.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:54 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:42:54.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:54 smithi043 ceph-mon[29710]: osdmap e121: 8 total, 8 up, 7 in 2024-09-21T11:42:54.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:54 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1691340531' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:54.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:54 smithi175 ceph-mon[32613]: 3.3 scrub starts 2024-09-21T11:42:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:54 smithi175 ceph-mon[32613]: 3.3 scrub ok 2024-09-21T11:42:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:54 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:42:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:54 smithi175 ceph-mon[32613]: osdmap e121: 8 total, 8 up, 7 in 2024-09-21T11:42:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:54 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1691340531' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:42:54.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:54.611 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:54.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:54.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:54.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:54.757 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:54.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:54.758 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:54.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:54.899 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:54.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:54.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:55.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:55.039 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:55.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:55.040 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:55.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:55.177 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:55.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:55.177 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:55.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:55.308 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:55.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:55.309 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:55.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:55.432 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:55.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:55.432 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:42:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:42:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-mon[36454]: pgmap v123: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 27 MiB/s wr, 616 op/s; 32064/349584 objects misplaced (9.172%); 60 MiB/s, 960 objects/s recovering 2024-09-21T11:42:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:42:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:42:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1691340531' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-mon[36454]: osdmap e122: 8 total, 8 up, 7 in 2024-09-21T11:42:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-mon[29710]: pgmap v123: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 27 MiB/s wr, 616 op/s; 32064/349584 objects misplaced (9.172%); 60 MiB/s, 960 objects/s recovering 2024-09-21T11:42:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:42:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:42:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1691340531' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:55 smithi043 ceph-mon[29710]: osdmap e122: 8 total, 8 up, 7 in 2024-09-21T11:42:55.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:55 smithi175 ceph-mon[32613]: pgmap v123: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.1 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1023 B/s rd, 27 MiB/s wr, 616 op/s; 32064/349584 objects misplaced (9.172%); 60 MiB/s, 960 objects/s recovering 2024-09-21T11:42:55.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:55 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:42:55.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:55 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:42:55.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:55 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1691340531' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:42:55.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:55 smithi175 ceph-mon[32613]: osdmap e122: 8 total, 8 up, 7 in 2024-09-21T11:42:55.533 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:55.563 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:55.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:55.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:55.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:55.703 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:55.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:55.704 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:55.804 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:55.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:55.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:55.948 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:55.969 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:55.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:55.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:56.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:56.099 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:56.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:56.100 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:56.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:56.243 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:56.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:56.244 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:56.344 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:56.374 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:56.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:56.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:56.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:42:56.505 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:56.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:56.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:42:56.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:56.652 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:56.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:56.652 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:56.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:56.778 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:56.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:56.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:56.880 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:56.911 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:56.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:56.912 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:57.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:57.091 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:57.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:57.092 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:57.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:57.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:57.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:57.316 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:57.346 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:57.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:57.347 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:57.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:57.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:57.479 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:57.480 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:57.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:57 smithi175 ceph-mon[32613]: pgmap v125: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 2.0 KiB/s rd, 29 MiB/s wr, 935 op/s; 31252/345111 objects misplaced (9.056%); 72 MiB/s, 1.15k objects/s recovering 2024-09-21T11:42:57.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:57.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:57.615 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:57.615 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:57.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:57.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:57 smithi043 ceph-mon[36454]: pgmap v125: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 2.0 KiB/s rd, 29 MiB/s wr, 935 op/s; 31252/345111 objects misplaced (9.056%); 72 MiB/s, 1.15k objects/s recovering 2024-09-21T11:42:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:57 smithi043 ceph-mon[29710]: pgmap v125: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 7.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 2.0 KiB/s rd, 29 MiB/s wr, 935 op/s; 31252/345111 objects misplaced (9.056%); 72 MiB/s, 1.15k objects/s recovering 2024-09-21T11:42:57.742 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:57.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:57.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:57.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:57.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:57.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:57.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:42:58.015 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:58.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:58.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:42:58.116 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:58.144 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:58.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:58.145 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:58.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:58.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:58.288 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:58.389 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:42:58.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:58.424 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:58.424 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:42:58.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:58.552 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:58.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:58.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:58.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:58.691 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:58.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:58.693 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:58.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:42:58.822 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:42:58.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:58.823 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:42:58.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:58.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:58.972 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:59.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:42:59.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:59.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:42:59.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:59.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:59.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:59.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:59.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:59.386 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:59.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:42:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:59.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:59.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:42:59.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:59 smithi175 ceph-mon[32613]: pgmap v126: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 749 op/s; 27242/336099 objects misplaced (8.105%); 39 MiB/s, 622 objects/s recovering 2024-09-21T11:42:59.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:59 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T11:42:59.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:42:59 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T11:42:59.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:59.654 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:59.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:59.655 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:42:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:59 smithi043 ceph-mon[36454]: pgmap v126: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 749 op/s; 27242/336099 objects misplaced (8.105%); 39 MiB/s, 622 objects/s recovering 2024-09-21T11:42:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:59 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T11:42:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:42:59 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T11:42:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:59 smithi043 ceph-mon[29710]: pgmap v126: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 749 op/s; 27242/336099 objects misplaced (8.105%); 39 MiB/s, 622 objects/s recovering 2024-09-21T11:42:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:59 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T11:42:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:42:59 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T11:42:59.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:42:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:59.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:59.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:42:59.903 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:42:59.946 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:42:59.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:42:59.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:00.048 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:00.092 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:00.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:00.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:00.195 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:00.232 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:00.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:00.233 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:00.244 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:43:00.244 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:43:00.244 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:43:00.244 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:43:00.244 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:43:00.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:00.380 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:00.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:00.381 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:00.482 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:00.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:00 smithi043 ceph-mon[29710]: 6.1a scrub starts 2024-09-21T11:43:00.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:00 smithi043 ceph-mon[29710]: 6.1a scrub ok 2024-09-21T11:43:00.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:00 smithi043 ceph-mon[36454]: 6.1a scrub starts 2024-09-21T11:43:00.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:00 smithi043 ceph-mon[36454]: 6.1a scrub ok 2024-09-21T11:43:00.507 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:00.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:00.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:00.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:00 smithi175 ceph-mon[32613]: 6.1a scrub starts 2024-09-21T11:43:00.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:00 smithi175 ceph-mon[32613]: 6.1a scrub ok 2024-09-21T11:43:00.609 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:00.636 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:00.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:00.637 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:00.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:00.778 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:00.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:00.782 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:00.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:00.914 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:00.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:00.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:00.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:43:01.015 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:01.042 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:01.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:01.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:01.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:01.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:01.193 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:01.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:01.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:43:00 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:43:01.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:01.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:01.344 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:01.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 3 1 2024-09-21T11:43:01.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:01.472 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:01.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:01.505 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:01.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:01.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:01 smithi043 ceph-mon[29710]: pgmap v127: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 673 B/s rd, 0 B/s wr, 592 op/s; 27242/336099 objects misplaced (8.105%); 31 MiB/s, 492 objects/s recovering 2024-09-21T11:43:01.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:01 smithi043 ceph-mon[29710]: 3.14 scrub starts 2024-09-21T11:43:01.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:01 smithi043 ceph-mon[29710]: 3.14 scrub ok 2024-09-21T11:43:01.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:01 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:01.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:01 smithi043 ceph-mon[36454]: pgmap v127: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 673 B/s rd, 0 B/s wr, 592 op/s; 27242/336099 objects misplaced (8.105%); 31 MiB/s, 492 objects/s recovering 2024-09-21T11:43:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:01 smithi043 ceph-mon[36454]: 3.14 scrub starts 2024-09-21T11:43:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:01 smithi043 ceph-mon[36454]: 3.14 scrub ok 2024-09-21T11:43:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:01 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:01.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:01.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:01.674 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:01.674 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:01.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:01 smithi175 ceph-mon[32613]: pgmap v127: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.8 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 673 B/s rd, 0 B/s wr, 592 op/s; 27242/336099 objects misplaced (8.105%); 31 MiB/s, 492 objects/s recovering 2024-09-21T11:43:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:01 smithi175 ceph-mon[32613]: 3.14 scrub starts 2024-09-21T11:43:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:01 smithi175 ceph-mon[32613]: 3.14 scrub ok 2024-09-21T11:43:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:01 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:01.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:01.802 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:01.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:01.803 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:01.904 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:01.932 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:01.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:01.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:02.034 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:02.061 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:02.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:02.062 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:02.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:43:02.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:02.258 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:02.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:02.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:02.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:02.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:02.411 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:02.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:02.455 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:02.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:02.494 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:43:02.494 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:43:02.512 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:02.535 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:02.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:02.538 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:02.539 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:43:02.540 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:43:02.556 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:43:02.556 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:43:02.579 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:43:02.580 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:43:02.626 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:43:02.626 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:43:02.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:02.670 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:43:02.670 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:43:02.690 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:02.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:02.691 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:02.705 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:43:02.705 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:43:02.746 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:43:02.746 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:43:02.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:02.782 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:43:02.782 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:43:02.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:02.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:02.851 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:02.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:02.973 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:02.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:02.974 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:03.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:03.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:03.120 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:03.221 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:03.241 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:03.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:03.242 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:03.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:03.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:03.385 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:03.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:03.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:03.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:03.524 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:03.524 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:03.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:03.666 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:03.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:03.667 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:03 smithi043 ceph-mon[29710]: pgmap v128: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s; 22348/321711 objects misplaced (6.947%); 55 MiB/s, 887 objects/s recovering 2024-09-21T11:43:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:03 smithi043 ceph-mon[29710]: 5.1c scrub starts 2024-09-21T11:43:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:03 smithi043 ceph-mon[29710]: 5.1c scrub ok 2024-09-21T11:43:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:03 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4228534670' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:03 smithi043 ceph-mon[36454]: pgmap v128: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s; 22348/321711 objects misplaced (6.947%); 55 MiB/s, 887 objects/s recovering 2024-09-21T11:43:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:03 smithi043 ceph-mon[36454]: 5.1c scrub starts 2024-09-21T11:43:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:03 smithi043 ceph-mon[36454]: 5.1c scrub ok 2024-09-21T11:43:03.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:03 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4228534670' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:03 smithi175 ceph-mon[32613]: pgmap v128: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s; 22348/321711 objects misplaced (6.947%); 55 MiB/s, 887 objects/s recovering 2024-09-21T11:43:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:03 smithi175 ceph-mon[32613]: 5.1c scrub starts 2024-09-21T11:43:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:03 smithi175 ceph-mon[32613]: 5.1c scrub ok 2024-09-21T11:43:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:03 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4228534670' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:03.768 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:03.796 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:03.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:03.797 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:03.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:03.934 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:03.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:03.935 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:04.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:04.067 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:04.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:04.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:04.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:04.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:04.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:04.315 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:04.347 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:04.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:04.349 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:04.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:04.488 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:04.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:04.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:04.590 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:04.622 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:04.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:04.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:04.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:04 smithi043 ceph-mon[29710]: 7.1 scrub starts 2024-09-21T11:43:04.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:04 smithi043 ceph-mon[36454]: 7.1 scrub starts 2024-09-21T11:43:04.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:04 smithi175 ceph-mon[32613]: 7.1 scrub starts 2024-09-21T11:43:04.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:04.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:04.765 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:04.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:04.897 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:04.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:04.898 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:04.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:05.021 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:05.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:05.022 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:05.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:05.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:05.159 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:05.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:05.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:05.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:05.306 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:05.307 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:05.408 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:05.439 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:05.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:05.440 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:05.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:05 smithi043 ceph-mon[29710]: pgmap v129: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 929 op/s; 22348/321711 objects misplaced (6.947%); 47 MiB/s, 756 objects/s recovering 2024-09-21T11:43:05.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:05 smithi043 ceph-mon[29710]: 5.18 scrub starts 2024-09-21T11:43:05.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:05 smithi043 ceph-mon[29710]: 5.18 scrub ok 2024-09-21T11:43:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:43:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:43:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:43:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:05 smithi043 ceph-mon[36454]: pgmap v129: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 929 op/s; 22348/321711 objects misplaced (6.947%); 47 MiB/s, 756 objects/s recovering 2024-09-21T11:43:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:05 smithi043 ceph-mon[36454]: 5.18 scrub starts 2024-09-21T11:43:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:05 smithi043 ceph-mon[36454]: 5.18 scrub ok 2024-09-21T11:43:05.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:05.586 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:05.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:05.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:05.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:05.734 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:05.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:05.735 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:05.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:05 smithi175 ceph-mon[32613]: pgmap v129: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1022 B/s rd, 0 B/s wr, 929 op/s; 22348/321711 objects misplaced (6.947%); 47 MiB/s, 756 objects/s recovering 2024-09-21T11:43:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:05 smithi175 ceph-mon[32613]: 5.18 scrub starts 2024-09-21T11:43:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:05 smithi175 ceph-mon[32613]: 5.18 scrub ok 2024-09-21T11:43:05.835 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:05.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:05.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:43:05.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:05.883 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:05.884 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:05.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:06.037 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:06.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:06.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:06.139 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:06.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:06.168 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:06.168 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:06.269 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:06.296 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:06.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:06.297 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:06.371 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:06.398 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:06.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:06.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:06.430 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:06.504 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:06 smithi043 ceph-mon[29710]: 7.1 scrub ok 2024-09-21T11:43:06.504 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:06 smithi043 ceph-mon[29710]: pgmap v130: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 924 op/s; 21653/317700 objects misplaced (6.816%); 44 MiB/s, 705 objects/s recovering 2024-09-21T11:43:06.505 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:06 smithi043 ceph-mon[36454]: 7.1 scrub ok 2024-09-21T11:43:06.505 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:06 smithi043 ceph-mon[36454]: pgmap v130: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 924 op/s; 21653/317700 objects misplaced (6.816%); 44 MiB/s, 705 objects/s recovering 2024-09-21T11:43:06.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:06.579 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:06.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:06.580 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:06.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:06.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:06.733 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:06.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:06.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:06 smithi175 ceph-mon[32613]: 7.1 scrub ok 2024-09-21T11:43:06.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:06 smithi175 ceph-mon[32613]: pgmap v130: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.5 GiB data, 19 GiB used, 607 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 924 op/s; 21653/317700 objects misplaced (6.816%); 44 MiB/s, 705 objects/s recovering 2024-09-21T11:43:06.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:06.883 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:06.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:06.884 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:06.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:07.030 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:07.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:07.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:07.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:07.179 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:07.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:07.180 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:07.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:07.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:07.341 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:07.342 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:07.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:07.468 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:07.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:07.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:07.503 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:07.570 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:07.595 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:07.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:07.596 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:07.682 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T11:43:07.696 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:07.721 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:07.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:07.722 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:07.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:07.880 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:07.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:07.881 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:07.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:07.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/538410922' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:43:07.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:07 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:43:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/538410922' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:43:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:07 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:43:08.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/538410922' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:43:08.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:07 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:43:08.027 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:08.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:08.028 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:08.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:08.158 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:08.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:08.159 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:08.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:08.287 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:08.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:08.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:08.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:43:08.359 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:43:08.359 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:43:08.389 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:08.415 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:08.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:08.416 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:08.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:08.573 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:08.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:08.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:08.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:08.703 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:43:08.727 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:08.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:08.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:08.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:08.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 4 1 2024-09-21T11:43:08.869 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:08.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:08.870 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:08.970 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[29710]: 4.c scrub starts 2024-09-21T11:43:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[29710]: 4.c scrub ok 2024-09-21T11:43:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:43:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[29710]: osdmap e123: 8 total, 8 up, 7 in 2024-09-21T11:43:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[29710]: pgmap v132: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.3 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 897 op/s; 18197/309189 objects misplaced (5.885%); 45 MiB/s, 716 objects/s recovering 2024-09-21T11:43:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/412261183' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[36454]: 4.c scrub starts 2024-09-21T11:43:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[36454]: 4.c scrub ok 2024-09-21T11:43:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:43:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[36454]: osdmap e123: 8 total, 8 up, 7 in 2024-09-21T11:43:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[36454]: pgmap v132: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.3 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 897 op/s; 18197/309189 objects misplaced (5.885%); 45 MiB/s, 716 objects/s recovering 2024-09-21T11:43:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/412261183' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:08 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:09.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:08 smithi175 ceph-mon[32613]: 4.c scrub starts 2024-09-21T11:43:09.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:08 smithi175 ceph-mon[32613]: 4.c scrub ok 2024-09-21T11:43:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:08 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:43:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:08 smithi175 ceph-mon[32613]: osdmap e123: 8 total, 8 up, 7 in 2024-09-21T11:43:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:08 smithi175 ceph-mon[32613]: pgmap v132: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.3 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 897 op/s; 18197/309189 objects misplaced (5.885%); 45 MiB/s, 716 objects/s recovering 2024-09-21T11:43:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:08 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/412261183' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:08 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:09.022 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:09.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:09.023 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:09.124 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:09.154 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:09.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:09.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:09.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:09.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:09.315 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:09.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:09.462 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:09.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:09.464 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:09.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:09.609 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:09.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:09.610 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:09.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:09.719 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:43:09.740 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:09.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:09.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:09.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:09.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:09.897 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:09.897 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:09 smithi043 ceph-mon[36454]: 4.1b scrub starts 2024-09-21T11:43:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:09 smithi043 ceph-mon[36454]: 4.1b scrub ok 2024-09-21T11:43:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:09 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:09 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:43:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:09 smithi043 ceph-mon[36454]: osdmap e124: 8 total, 8 up, 7 in 2024-09-21T11:43:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:09 smithi043 ceph-mon[29710]: 4.1b scrub starts 2024-09-21T11:43:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:09 smithi043 ceph-mon[29710]: 4.1b scrub ok 2024-09-21T11:43:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:09 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:09 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:43:09.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:09 smithi043 ceph-mon[29710]: osdmap e124: 8 total, 8 up, 7 in 2024-09-21T11:43:09.998 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:09 smithi175 ceph-mon[32613]: 4.1b scrub starts 2024-09-21T11:43:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:09 smithi175 ceph-mon[32613]: 4.1b scrub ok 2024-09-21T11:43:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:09 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:09 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:43:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:09 smithi175 ceph-mon[32613]: osdmap e124: 8 total, 8 up, 7 in 2024-09-21T11:43:10.026 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:10.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:10.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:10.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:10.217 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:10.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:10.219 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:10.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:10.344 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:10.351 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:10.402 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:10.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:10.404 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:10.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:10.534 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:10.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:10.535 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:10.636 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:10.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:10.667 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:10.668 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:10.758 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:10 smithi043 ceph-mon[36454]: pgmap v134: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.3 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 521 op/s; 18197/309189 objects misplaced (5.885%); 26 MiB/s, 418 objects/s recovering 2024-09-21T11:43:10.758 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:10 smithi043 ceph-mon[29710]: pgmap v134: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.3 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 521 op/s; 18197/309189 objects misplaced (5.885%); 26 MiB/s, 418 objects/s recovering 2024-09-21T11:43:10.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:10.835 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:10.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:10.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:10.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:10.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:10.982 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:11.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:10 smithi175 ceph-mon[32613]: pgmap v134: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.3 GiB data, 18 GiB used, 607 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 521 op/s; 18197/309189 objects misplaced (5.885%); 26 MiB/s, 418 objects/s recovering 2024-09-21T11:43:11.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:43:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:43:11.083 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:11.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:11.125 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:11.125 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:11.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:11.265 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:11.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:11.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:11.366 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:11.377 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:43:11.377 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:43:11.394 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:11.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:11.395 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:11.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:11.538 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:11.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:11.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:11.616 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:11.640 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:11.667 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:11.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:11.668 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:11.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:11.815 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:11.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:11.816 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:11.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:11.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:11.969 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:11 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:11.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:11 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:12.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:11 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:12.070 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:12.095 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:12.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:12.096 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:12.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:43:12.197 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:12.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:12.225 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:12.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:12.325 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:12.351 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:12.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:12.352 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:12.453 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:12.483 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:12.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:12.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:12.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:12.610 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:12.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:12.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:12.618 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:43:12.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:12.739 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:12.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:12.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:12.841 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:12.890 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:12.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:43:12.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:43:12.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[36454]: osdmap e125: 8 total, 8 up, 7 in 2024-09-21T11:43:12.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[36454]: pgmap v136: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.24k op/s; 13881/295380 objects misplaced (4.699%); 66 MiB/s, 1.06k objects/s recovering 2024-09-21T11:43:12.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/691685686' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:43:12.901 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:43:12.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:43:12.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:43:12.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[29710]: osdmap e125: 8 total, 8 up, 7 in 2024-09-21T11:43:12.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[29710]: pgmap v136: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.24k op/s; 13881/295380 objects misplaced (4.699%); 66 MiB/s, 1.06k objects/s recovering 2024-09-21T11:43:12.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/691685686' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:43:12.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:12 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:43:12.902 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:43:12.903 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:43:12.918 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:12.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:12.919 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:12.950 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:43:12.951 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:43:12.986 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:43:12.986 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:43:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:12 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:43:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:12 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:43:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:12 smithi175 ceph-mon[32613]: osdmap e125: 8 total, 8 up, 7 in 2024-09-21T11:43:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:12 smithi175 ceph-mon[32613]: pgmap v136: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 1.24k op/s; 13881/295380 objects misplaced (4.699%); 66 MiB/s, 1.06k objects/s recovering 2024-09-21T11:43:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:12 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/691685686' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:43:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:12 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:43:13.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:13.022 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:43:13.022 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:43:13.057 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:43:13.057 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:43:13.075 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:13.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:13.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:13.088 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:43:13.088 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:43:13.127 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:43:13.127 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:43:13.161 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:43:13.161 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:43:13.167 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T11:43:13.177 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:13.206 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:13.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:13.207 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:13.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:13.336 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:13.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:13.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:13.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:13.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:13.485 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:13.485 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:13.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:13.635 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:13.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:13.636 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:13.737 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:13.762 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:13.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:13.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:13.864 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:13.895 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:13.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:13.896 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/124734180' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:13 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:43:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:13 smithi043 ceph-mon[29710]: osdmap e126: 8 total, 8 up, 7 in 2024-09-21T11:43:13.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/124734180' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:13.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:13 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:43:13.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:13 smithi043 ceph-mon[36454]: osdmap e126: 8 total, 8 up, 7 in 2024-09-21T11:43:13.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:14.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/124734180' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:14.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:13 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:43:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:13 smithi175 ceph-mon[32613]: osdmap e126: 8 total, 8 up, 7 in 2024-09-21T11:43:14.037 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:14.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:14.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:14.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:14.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:14.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:14.283 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:14.314 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:14.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:14.315 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:14.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:14.461 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:14.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:14.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:14.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:14.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:14.592 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:14.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:14.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:14.738 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:14.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:14.739 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:14.766 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:43:14.827 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:14 smithi043 ceph-mon[29710]: pgmap v138: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 767 op/s; 13881/295380 objects misplaced (4.699%); 37 MiB/s, 596 objects/s recovering 2024-09-21T11:43:14.828 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/122473704' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:43:14.828 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:14 smithi043 ceph-mon[36454]: pgmap v138: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 767 op/s; 13881/295380 objects misplaced (4.699%); 37 MiB/s, 596 objects/s recovering 2024-09-21T11:43:14.828 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/122473704' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:43:14.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:14.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 5 1 2024-09-21T11:43:14.870 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:14.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:14.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:14.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:14.999 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:15.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:15.000 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:15.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:15.183 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:15.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:15.184 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:15.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:14 smithi175 ceph-mon[32613]: pgmap v138: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 6.0 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 767 op/s; 13881/295380 objects misplaced (4.699%); 37 MiB/s, 596 objects/s recovering 2024-09-21T11:43:15.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/122473704' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:43:15.285 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:15.339 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:15.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:15.340 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:15.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:15.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:15.472 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:15.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:15.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:43:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:43:15.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:15.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:15.619 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:15.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:15.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:15.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:15.751 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:15.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:15.852 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:15.877 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:15.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:15.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:15.947 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:15.979 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:16.008 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:16.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:16.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:16.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:16.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/122473704' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:43:16.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-mon[29710]: osdmap e127: 8 total, 8 up, 7 in 2024-09-21T11:43:16.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:16.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:16.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:16.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/122473704' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:43:16.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-mon[36454]: osdmap e127: 8 total, 8 up, 7 in 2024-09-21T11:43:16.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:16.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:15 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:16.110 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:16.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:43:16.142 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:16.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:16.143 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:16.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:16.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:15 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/122473704' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:43:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:15 smithi175 ceph-mon[32613]: osdmap e127: 8 total, 8 up, 7 in 2024-09-21T11:43:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:15 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:15 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:16.301 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:16.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:16.302 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:16.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:16.446 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:16.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:16.447 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:16.548 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:16.575 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:16.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:16.576 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:16.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:16.704 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:16.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:16.705 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:16.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:16.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:16.853 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:16.927 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:16 smithi043 ceph-mon[36454]: pgmap v140: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.01k op/s; 13152/290985 objects misplaced (4.520%); 44 MiB/s, 703 objects/s recovering 2024-09-21T11:43:16.927 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:16 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:43:16.927 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:16 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:43:16.927 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:16 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:43:16.928 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:16 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:43:16.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:16 smithi043 ceph-mon[29710]: pgmap v140: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.01k op/s; 13152/290985 objects misplaced (4.520%); 44 MiB/s, 703 objects/s recovering 2024-09-21T11:43:16.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:16 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:43:16.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:16 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:43:16.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:16 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:43:16.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:16 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:43:16.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:16.997 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:16.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:16.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:17.099 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:17.136 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:17.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:17.137 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:17.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:17.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:16 smithi175 ceph-mon[32613]: pgmap v140: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.9 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.01k op/s; 13152/290985 objects misplaced (4.520%); 44 MiB/s, 703 objects/s recovering 2024-09-21T11:43:17.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:16 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:43:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:16 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:43:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:16 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:43:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:16 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:43:17.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:17.278 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:17.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:17.379 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:17.410 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:17.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:17.411 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:17.512 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:17.544 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:17.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:17.544 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:17.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:17.688 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:17.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:17.689 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:17.789 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:17.821 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:17.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:43:17.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:17.822 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:17.880 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:43:17.880 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:43:17.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:43:17.880 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:43:17.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.880 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:37277 7.b 7:d13dff0d:::benchmark_data_smithi043_70646_object37276:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:35.480704+0000", 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 101.443478277, 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036890579, 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37277 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480704+0000", 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480704+0000", 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480706+0000", 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0339999999999999e-06 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480728+0000", 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1526e-05 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480738+0000", 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0083e-05 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480745+0000", 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481292+0000", 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00054686700000000003 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481399+0000", 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107378 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481563+0000", 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000163833 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484047+0000", 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024838680000000002 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484084+0000", 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7339000000000003e-05 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517265+0000", 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.033180236000000002 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517535+0000", 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00027041599999999999 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517595+0000", 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0001999999999999e-05 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:37278 7.b 7:d6b83a2c:::benchmark_data_smithi043_70646_object37277:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:43:17.886 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:35.481164+0000", 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 101.44301908200001, 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036602902, 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37278 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481164+0000", 2024-09-21T11:43:17.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481164+0000", 2024-09-21T11:43:17.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481164+0000", 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8899999999999998e-07 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481196+0000", 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1829000000000003e-05 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481206+0000", 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8940000000000005e-06 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481213+0000", 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2080000000000003e-06 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484174+0000", 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029610600000000002 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484285+0000", 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110468 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484411+0000", 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125847 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.486414+0000", 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020031049999999998 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.486427+0000", 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3563e-05 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517660+0000", 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031232167000000002 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517723+0000", 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3660999999999999e-05 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517766+0000", 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3210999999999997e-05 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:92195 7.b 7:dccd3640:::benchmark_data_smithi043_70646_object92194:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:19.376248+0000", 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 57.547934726999998, 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043184081999999999, 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 92195 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.893 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376248+0000", 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376248+0000", 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376250+0000", 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.809e-06 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376279+0000", 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9685000000000001e-05 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376294+0000", 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4382000000000001e-05 2024-09-21T11:43:17.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376302+0000", 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7489999999999998e-06 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.381875+0000", 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055737729999999997 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.382223+0000", 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034789100000000002 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.382499+0000", 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00027602900000000001 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394336+0000", 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011837089 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394380+0000", 2024-09-21T11:43:17.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3791999999999997e-05 2024-09-21T11:43:17.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419127+0000", 2024-09-21T11:43:17.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024747149 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419380+0000", 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00025246999999999997 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419432+0000", 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2264e-05 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.900 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:92197 7.b 7:d6014428:::benchmark_data_smithi043_70646_object92196:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:43:17.900 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:19.377823+0000", 2024-09-21T11:43:17.900 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 57.546359443999997, 2024-09-21T11:43:17.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041873795999999998, 2024-09-21T11:43:17.900 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.900 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 92197 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377823+0000", 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377823+0000", 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377825+0000", 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.316e-06 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377847+0000", 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2942000000000002e-05 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377863+0000", 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5571999999999999e-05 2024-09-21T11:43:17.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377871+0000", 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6830000000000008e-06 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.386479+0000", 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0086077989999999993 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.386599+0000", 2024-09-21T11:43:17.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012036699999999999 2024-09-21T11:43:17.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.389012+0000", 2024-09-21T11:43:17.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024130559999999998 2024-09-21T11:43:17.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394424+0000", 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054119509999999999 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394464+0000", 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0262000000000002e-05 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419493+0000", 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.025028741 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419655+0000", 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000162349 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419697+0000", 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1758000000000002e-05 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:92198 7.b 7:de413604:::benchmark_data_smithi043_70646_object92197:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:19.377883+0000", 2024-09-21T11:43:17.907 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 57.546299409, 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042019793999999999, 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 92198 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377883+0000", 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377883+0000", 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377884+0000", 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1399999999999997e-07 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377904+0000", 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0386000000000001e-05 2024-09-21T11:43:17.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377910+0000", 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3219999999999998e-06 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377916+0000", 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394496+0000", 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016580785000000001 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394620+0000", 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123736 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394768+0000", 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147365 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.410416+0000", 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015648695000000001 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.410469+0000", 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2683999999999999e-05 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419710+0000", 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092409359999999999 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419862+0000", 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015258599999999999 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419903+0000", 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0590000000000003e-05 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:94692 7.9 e105/82 7:92d687ee:::benchmark_data_smithi043_70646_object94691:head v 105'5840, mlcod=105'5838)", 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:21.710418+0000", 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.213764337999997, 2024-09-21T11:43:17.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043836149999999997, 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710418+0000", 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710418+0000", 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710419+0000", 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710450+0000", 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1259e-05 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710457+0000", 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540000000000003e-06 2024-09-21T11:43:17.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710463+0000", 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.712306+0000", 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001843021 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.712326+0000", 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0322999999999998e-05 2024-09-21T11:43:17.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.754204+0000", 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041877978000000003 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.754254+0000", 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0479999999999998e-05 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.917 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:94698 7.9 e105/82 7:9acdce21:::benchmark_data_smithi043_70646_object94697:head v 105'5841, mlcod=105'5838)", 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:21.717113+0000", 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 55.207069208999997, 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.038293739, 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717113+0000", 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.918 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717113+0000", 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717115+0000", 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.181e-06 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717141+0000", 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6837000000000001e-05 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717153+0000", 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1262e-05 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717160+0000", 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3669999999999999e-06 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.719727+0000", 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025664730000000001 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.719850+0000", 2024-09-21T11:43:17.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012304899999999999 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.754262+0000", 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034412700999999997 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.755407+0000", 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001144869 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:96882 7.0 e106/82 7:0b4ad799:::benchmark_data_smithi043_70646_object96881:head v 106'6094, mlcod=106'6090)", 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:23.906882+0000", 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 53.017300954, 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046078277000000001, 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906882+0000", 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906882+0000", 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906883+0000", 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2999999999999999e-07 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906918+0000", 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5533e-05 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906935+0000", 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6473999999999998e-05 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906943+0000", 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0430000000000008e-06 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.911336+0000", 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004393798 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.911401+0000", 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4611999999999997e-05 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.952912+0000", 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041511406000000001 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.952960+0000", 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7481000000000002e-05 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:96890 7.0 e106/82 7:0e02357d:::benchmark_data_smithi043_70646_object96889:head v 106'6095, mlcod=106'6090)", 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:23.916357+0000", 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 53.007826025, 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036631786999999999, 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916357+0000", 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916357+0000", 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916358+0000", 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-07 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916454+0000", 2024-09-21T11:43:17.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6772000000000001e-05 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916465+0000", 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0536999999999999e-05 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916473+0000", 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7389999999999999e-06 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.919650+0000", 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031771780000000001 2024-09-21T11:43:17.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.919713+0000", 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3551000000000002e-05 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.952967+0000", 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.033253222999999998 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.952988+0000", 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1821999999999999e-05 2024-09-21T11:43:17.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:98325 7.b 7:de6ec310:::benchmark_data_smithi043_70646_object98324:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:25.372481+0000", 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.551701944999998, 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034115872999999998, 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.930 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 98325 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372481+0000", 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372481+0000", 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372482+0000", 2024-09-21T11:43:17.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6500000000000001e-06 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372504+0000", 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1647000000000001e-05 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372521+0000", 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7022999999999999e-05 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372530+0000", 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5439999999999999e-06 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372611+0000", 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1329999999999996e-05 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.373014+0000", 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00040339400000000001 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.373144+0000", 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000129935 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.403034+0000", 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.029890232999999999 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.403110+0000", 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5434999999999999e-05 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.406336+0000", 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032260549999999998 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.406549+0000", 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000212611 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.406597+0000", 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8016000000000002e-05 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:98329 7.b 7:dc8c13a9:::benchmark_data_smithi043_70646_object98328:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:43:17.935 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:25.372738+0000", 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 51.551444852000003, 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034055026000000002, 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 98329 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372738+0000", 2024-09-21T11:43:17.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372738+0000", 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372739+0000", 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3399999999999998e-07 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372758+0000", 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9649000000000001e-05 2024-09-21T11:43:17.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372767+0000", 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0710000000000001e-06 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.372774+0000", 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7159999999999997e-06 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.387516+0000", 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014742132 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.387633+0000", 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011697999999999999 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.387769+0000", 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013582399999999999 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.403146+0000", 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015377332000000001 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.403185+0000", 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9048999999999998e-05 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.406608+0000", 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034221809999999998 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.406755+0000", 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014697500000000001 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.406793+0000", 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8282999999999999e-05 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:100678 7.b 7:d782bdfb:::benchmark_data_smithi043_70646_object100677:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:28.332750+0000", 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 48.591432415, 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044281528000000001, 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 100678 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332750+0000", 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332750+0000", 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332751+0000", 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7199999999999999e-07 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332772+0000", 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1446000000000001e-05 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332779+0000", 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.17e-06 2024-09-21T11:43:17.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332785+0000", 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0240000000000003e-06 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.335754+0000", 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002968498 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.335876+0000", 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012237399999999999 2024-09-21T11:43:17.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.336024+0000", 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014732999999999999 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.341309+0000", 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052852929999999999 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.342298+0000", 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098862199999999998 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.376908+0000", 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034610502000000001 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.376980+0000", 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1679000000000003e-05 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.377032+0000", 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2018000000000002e-05 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:101832 7.b 7:dcbd601b:::benchmark_data_smithi043_70646_object101831:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:29.648649+0000", 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 47.275534145999998, 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052835252999999999, 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 101832 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648649+0000", 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648649+0000", 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648650+0000", 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.778e-06 2024-09-21T11:43:17.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648675+0000", 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4752e-05 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648690+0000", 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4932e-05 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648697+0000", 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5070000000000003e-06 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.649977+0000", 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012800159999999999 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.650104+0000", 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000126236 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.650252+0000", 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014790499999999999 2024-09-21T11:43:17.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.655289+0000", 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050370090000000003 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.655495+0000", 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020639200000000001 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.701323+0000", 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045828277000000001 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.701429+0000", 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.701484+0000", 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5038999999999999e-05 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:102078 7.b 7:df4f0289:::benchmark_data_smithi043_70646_object102077:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:43:17.952 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:29.972683+0000", 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.951499212999998, 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.040813940999999999, 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 102078 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972683+0000", 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972683+0000", 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972685+0000", 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.756e-06 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972708+0000", 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2775000000000001e-05 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972720+0000", 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2408999999999999e-05 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972728+0000", 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8790000000000003e-06 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.978889+0000", 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061604269999999996 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.979014+0000", 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012558 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.979160+0000", 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014582899999999999 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.991453+0000", 2024-09-21T11:43:17.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012293282000000001 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.994408+0000", 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029544100000000002 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.013365+0000", 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01895728 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.013432+0000", 2024-09-21T11:43:17.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6841999999999995e-05 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.013497+0000", 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5471999999999999e-05 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:102215 7.b 7:d049ffb3:::benchmark_data_smithi043_70646_object102214:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:30.137277+0000", 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.786905191999999, 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.039707500999999999, 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 102215 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137277+0000", 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137277+0000", 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137279+0000", 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9970000000000001e-06 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137306+0000", 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6928e-05 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137324+0000", 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7448e-05 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137333+0000", 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.414e-06 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137998+0000", 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00066516999999999998 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.138109+0000", 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110862 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.138265+0000", 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000155384 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.169637+0000", 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031372672999999997 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.169671+0000", 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4156000000000003e-05 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.176870+0000", 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071988649999999996 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.176924+0000", 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3944999999999999e-05 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.176985+0000", 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0659000000000003e-05 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:102229 7.b 7:d60ee21f:::benchmark_data_smithi043_70646_object102228:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:30.141884+0000", 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.782298203000003, 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035204337000000002, 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.964 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 102229 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.141884+0000", 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.141884+0000", 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.141887+0000", 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1550000000000001e-06 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.141910+0000", 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3862e-05 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.141927+0000", 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6705999999999999e-05 2024-09-21T11:43:17.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.141936+0000", 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.772e-06 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.165985+0000", 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024049197000000001 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.166142+0000", 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000156541 2024-09-21T11:43:17.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.166608+0000", 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00046652199999999998 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.169707+0000", 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003098347 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.176819+0000", 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071128129999999999 2024-09-21T11:43:17.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.176994+0000", 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017498599999999999 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.177040+0000", 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5303999999999999e-05 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.177089+0000", 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9132000000000003e-05 2024-09-21T11:43:17.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:102266 7.b 7:d1444c05:::benchmark_data_smithi043_70646_object102265:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:30.193280+0000", 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.730903040000001, 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.037717675999999999, 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 102266 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193280+0000", 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193280+0000", 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193282+0000", 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.153e-06 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193309+0000", 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6948e-05 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193327+0000", 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8445999999999999e-05 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193337+0000", 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6840000000000008e-06 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.204924+0000", 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011586905999999999 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.205037+0000", 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011285099999999999 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:43:17.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.205177+0000", 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140803 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.211726+0000", 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0065485650000000001 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.230657+0000", 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018930939000000001 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.230719+0000", 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1834000000000005e-05 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.230907+0000", 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000188121 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.230997+0000", 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0426000000000005e-05 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:102389 7.d e108/82 7:bd869319:::benchmark_data_smithi043_70646_object102388:head v 108'6421, mlcod=108'6419)", 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:30.370305+0000", 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 46.553877980000003, 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034160173000000002, 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.370305+0000", 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.370305+0000", 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.370306+0000", 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0330000000000001e-06 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.371777+0000", 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001471095 2024-09-21T11:43:17.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.371788+0000", 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0711e-05 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.371796+0000", 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0860000000000002e-06 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.377315+0000", 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055198900000000004 2024-09-21T11:43:17.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.377337+0000", 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.122e-05 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.404232+0000", 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026895345000000001 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.404465+0000", 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023279299999999999 2024-09-21T11:43:17.979 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:120685 7.b e120/113 7:d17da732:::benchmark_data_smithi043_70646_object3277:head v 120'7456, mlcod=120'7454)", 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:53.199775+0000", 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 23.724407677999999, 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036220908000000003, 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:53.199775+0000", 2024-09-21T11:43:17.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:53.199775+0000", 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:53.199777+0000", 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6810000000000001e-06 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:53.199865+0000", 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8442999999999993e-05 2024-09-21T11:43:17.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:53.199886+0000", 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1262000000000001e-05 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:53.209410+0000", 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0095235530000000006 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:53.209692+0000", 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000282346 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:53.209716+0000", 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3269e-05 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:53.233216+0000", 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.023500145 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:53.235996+0000", 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027802090000000001 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:140623 7.0 e127/113 7:04380214:::benchmark_data_smithi043_70646_object23215:head v 127'8735, mlcod=127'8732)", 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:43:15.346210+0000", 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.5779729850000002, 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.035318034999999998, 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:15.346210+0000", 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:15.346210+0000", 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:15.346210+0000", 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4799999999999998e-07 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:15.346219+0000", 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6810000000000004e-06 2024-09-21T11:43:17.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:15.346226+0000", 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2779999999999996e-06 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:15.346232+0000", 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3229999999999997e-06 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:15.348815+0000", 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025833290000000001 2024-09-21T11:43:17.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:15.348837+0000", 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2493e-05 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:15.381477+0000", 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.032639730999999998 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:15.381528+0000", 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0652000000000002e-05 2024-09-21T11:43:17.987 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:17.988 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:17.988 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:43:17.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:18.013 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:18.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:18.014 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:18.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:18.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:18.172 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:18.172 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:18.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:17 smithi043 ceph-mon[29710]: osdmap e128: 8 total, 8 up, 7 in 2024-09-21T11:43:18.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:17 smithi043 ceph-mon[36454]: osdmap e128: 8 total, 8 up, 7 in 2024-09-21T11:43:18.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:17 smithi175 ceph-mon[32613]: osdmap e128: 8 total, 8 up, 7 in 2024-09-21T11:43:18.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:18.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:18.317 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:18.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:18.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:18.458 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:18.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:18.459 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:18.560 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:18.589 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:18.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:18.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:18.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:18.741 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:18.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:18.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:18.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:18.889 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:18.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:18.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:18.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:19.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:19.020 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:19.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:19.121 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:19.147 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:19.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:19.148 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:19.161 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:18 smithi175 ceph-mon[32613]: pgmap v142: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 772 op/s; 9683/281481 objects misplaced (3.440%); 38 MiB/s, 610 objects/s recovering 2024-09-21T11:43:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:18 smithi043 ceph-mon[36454]: pgmap v142: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 772 op/s; 9683/281481 objects misplaced (3.440%); 38 MiB/s, 610 objects/s recovering 2024-09-21T11:43:19.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:18 smithi043 ceph-mon[29710]: pgmap v142: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 772 op/s; 9683/281481 objects misplaced (3.440%); 38 MiB/s, 610 objects/s recovering 2024-09-21T11:43:19.249 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:19.277 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:19.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:19.278 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:19.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:43:19.378 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:19.405 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:19.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:19.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:19.506 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:19.530 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:19.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:19.532 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:19.607 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:19.633 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:19.659 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:19.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:19.660 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:19.761 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:19.788 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:19.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:19.789 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T11:43:19.890 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:19.919 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:19.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:19.920 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:19.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:19 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1797185816' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:43:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:19 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1797185816' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:43:20.021 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:20.049 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:20.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:20.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:20.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:20.199 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:20.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:20.200 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:20.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:19 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1797185816' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:43:20.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:20.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:20.343 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:20.343 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:20.346 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:43:20.346 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:43:20.391 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:43:20.391 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:43:20.429 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:43:20.429 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:43:20.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:20.472 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:43:20.473 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:43:20.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:20.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:20.511 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:43:20.511 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:43:20.547 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:43:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:43:20.578 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:43:20.578 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:43:20.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:20.629 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:43:20.629 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:43:20.631 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:20.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:20.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:20.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:20.779 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:20.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:20.780 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:20.881 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:20.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:20.903 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:20.904 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:20 smithi043 ceph-mon[29710]: pgmap v143: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 776 B/s rd, 0 B/s wr, 703 op/s; 9683/281481 objects misplaced (3.440%); 35 MiB/s, 555 objects/s recovering 2024-09-21T11:43:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1797185816' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:43:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:20 smithi043 ceph-mon[29710]: osdmap e129: 8 total, 8 up, 7 in 2024-09-21T11:43:20.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1040108498' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:20.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:20 smithi043 ceph-mon[36454]: pgmap v143: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 776 B/s rd, 0 B/s wr, 703 op/s; 9683/281481 objects misplaced (3.440%); 35 MiB/s, 555 objects/s recovering 2024-09-21T11:43:20.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1797185816' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:43:20.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:20 smithi043 ceph-mon[36454]: osdmap e129: 8 total, 8 up, 7 in 2024-09-21T11:43:20.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1040108498' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:21.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:21.032 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:21.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:21.033 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:21.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:21.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:21.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:21.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:20 smithi175 ceph-mon[32613]: pgmap v143: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.7 GiB data, 17 GiB used, 608 GiB / 626 GiB avail; 776 B/s rd, 0 B/s wr, 703 op/s; 9683/281481 objects misplaced (3.440%); 35 MiB/s, 555 objects/s recovering 2024-09-21T11:43:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1797185816' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:43:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:20 smithi175 ceph-mon[32613]: osdmap e129: 8 total, 8 up, 7 in 2024-09-21T11:43:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1040108498' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:21.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:43:20 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:43:21.282 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:21.305 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:21.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:21.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:21.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:21.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:21.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:21.552 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:21.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:21.579 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:21.580 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:21.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:21.711 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:21.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:21.712 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:21.813 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:21.841 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:21.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:21.842 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:21.943 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:21.967 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:21.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:21.968 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:22.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:22.102 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:22.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:22.103 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:22.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:22.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:22.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:22.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:21 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:22.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3740924219' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:43:22.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:21 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:22.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:21 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3740924219' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:43:22.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:21 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:21 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3740924219' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:43:22.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:22.363 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:22.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:22.364 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:22.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:22.487 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:22.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:22.488 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:22.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:22.616 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:22.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:22.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:22.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:22.747 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:22.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:22.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:22.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:22.877 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:22.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:22.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:22.979 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:23.017 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:23.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:23.018 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:23.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:23.155 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:23.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:23.156 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:23.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:22 smithi043 ceph-mon[36454]: pgmap v145: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.29k op/s; 5631/268389 objects misplaced (2.098%); 66 MiB/s, 1.05k objects/s recovering 2024-09-21T11:43:23.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:22 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3740924219' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:43:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:22 smithi043 ceph-mon[36454]: osdmap e130: 8 total, 8 up, 7 in 2024-09-21T11:43:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:22 smithi043 ceph-mon[29710]: pgmap v145: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.29k op/s; 5631/268389 objects misplaced (2.098%); 66 MiB/s, 1.05k objects/s recovering 2024-09-21T11:43:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:22 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3740924219' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:43:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:22 smithi043 ceph-mon[29710]: osdmap e130: 8 total, 8 up, 7 in 2024-09-21T11:43:23.257 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:23.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:22 smithi175 ceph-mon[32613]: pgmap v145: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.29k op/s; 5631/268389 objects misplaced (2.098%); 66 MiB/s, 1.05k objects/s recovering 2024-09-21T11:43:23.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:22 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:23.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3740924219' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:43:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:22 smithi175 ceph-mon[32613]: osdmap e130: 8 total, 8 up, 7 in 2024-09-21T11:43:23.287 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:23.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:23.288 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:23.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:23.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:23.430 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:23.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:23.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:23.573 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:23.574 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:23.674 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:23.703 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:23.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:23.704 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:23.804 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:23.826 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:23.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:23.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:23.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:23.957 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:23.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:23.958 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:24.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:24.085 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:24.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:24.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:24.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:24.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:24.218 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:24.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:23 smithi043 ceph-mon[29710]: 6.1e scrub starts 2024-09-21T11:43:24.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:23 smithi043 ceph-mon[29710]: 6.1e scrub ok 2024-09-21T11:43:24.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:23 smithi043 ceph-mon[29710]: osdmap e131: 8 total, 8 up, 7 in 2024-09-21T11:43:24.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:23 smithi043 ceph-mon[36454]: 6.1e scrub starts 2024-09-21T11:43:24.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:23 smithi043 ceph-mon[36454]: 6.1e scrub ok 2024-09-21T11:43:24.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:23 smithi043 ceph-mon[36454]: osdmap e131: 8 total, 8 up, 7 in 2024-09-21T11:43:24.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:23 smithi175 ceph-mon[32613]: 6.1e scrub starts 2024-09-21T11:43:24.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:23 smithi175 ceph-mon[32613]: 6.1e scrub ok 2024-09-21T11:43:24.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:23 smithi175 ceph-mon[32613]: osdmap e131: 8 total, 8 up, 7 in 2024-09-21T11:43:24.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:24.346 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:24.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:24.348 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:24.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:24.477 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:24.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:24.478 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:24.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:24.601 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:24.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:24.602 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:24.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:24.729 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:24.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:24.730 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:24.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:24.855 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:24.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:24.857 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:24.958 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:24.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:24.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:25.088 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:25.116 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:25.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:25.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:25.217 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:24 smithi043 ceph-mon[36454]: pgmap v148: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 727 op/s; 5631/268389 objects misplaced (2.098%); 38 MiB/s, 613 objects/s recovering 2024-09-21T11:43:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:24 smithi043 ceph-mon[36454]: osdmap e132: 8 total, 8 up, 7 in 2024-09-21T11:43:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:24 smithi043 ceph-mon[29710]: pgmap v148: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 727 op/s; 5631/268389 objects misplaced (2.098%); 38 MiB/s, 613 objects/s recovering 2024-09-21T11:43:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:24 smithi043 ceph-mon[29710]: osdmap e132: 8 total, 8 up, 7 in 2024-09-21T11:43:25.241 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:25.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:25.242 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:25.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:24 smithi175 ceph-mon[32613]: pgmap v148: 148 pgs: 6 active+remapped+backfilling, 142 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 727 op/s; 5631/268389 objects misplaced (2.098%); 38 MiB/s, 613 objects/s recovering 2024-09-21T11:43:25.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:24 smithi175 ceph-mon[32613]: osdmap e132: 8 total, 8 up, 7 in 2024-09-21T11:43:25.343 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:25.366 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:25.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:25.367 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:25.468 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:25.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:25.491 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:25.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:25.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:43:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:43:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:43:25.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:25.614 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:25.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:25.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:25.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:25.740 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:25.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:25.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:25.842 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:25.864 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:25.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:25.865 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:25.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:26.003 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:26.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:26.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:26.105 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:26.131 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:26.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:26.132 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:26.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:26.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:25 smithi043 ceph-mon[36454]: 6.17 scrub starts 2024-09-21T11:43:26.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:25 smithi043 ceph-mon[36454]: 6.17 scrub ok 2024-09-21T11:43:26.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:25 smithi043 ceph-mon[36454]: osdmap e133: 8 total, 8 up, 7 in 2024-09-21T11:43:26.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:25 smithi043 ceph-mon[29710]: 6.17 scrub starts 2024-09-21T11:43:26.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:25 smithi043 ceph-mon[29710]: 6.17 scrub ok 2024-09-21T11:43:26.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:25 smithi043 ceph-mon[29710]: osdmap e133: 8 total, 8 up, 7 in 2024-09-21T11:43:26.254 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:26.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:26.255 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:26.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:25 smithi175 ceph-mon[32613]: 6.17 scrub starts 2024-09-21T11:43:26.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:25 smithi175 ceph-mon[32613]: 6.17 scrub ok 2024-09-21T11:43:26.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:25 smithi175 ceph-mon[32613]: osdmap e133: 8 total, 8 up, 7 in 2024-09-21T11:43:26.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:26.377 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:26.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:26.378 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:26.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:26.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:26.502 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:26.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:26.604 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:26.625 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:26.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:26.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:26.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:26.750 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:26.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:26.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:26.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:26.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:26.875 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:26.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:27.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:27.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:27.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:27.130 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:27.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:27.131 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:27.232 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:27.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[36454]: 5.10 scrub starts 2024-09-21T11:43:27.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[36454]: 5.10 scrub ok 2024-09-21T11:43:27.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[36454]: pgmap v151: 148 pgs: 1 active+clean+remapped, 5 active+remapped+backfilling, 142 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 202 op/s; 4669/265977 objects misplaced (1.755%); 15 MiB/s, 234 objects/s recovering 2024-09-21T11:43:27.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[36454]: osdmap e134: 8 total, 8 up, 7 in 2024-09-21T11:43:27.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[36454]: 5.b scrub starts 2024-09-21T11:43:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[36454]: 5.b scrub ok 2024-09-21T11:43:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[29710]: 5.10 scrub starts 2024-09-21T11:43:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[29710]: 5.10 scrub ok 2024-09-21T11:43:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[29710]: pgmap v151: 148 pgs: 1 active+clean+remapped, 5 active+remapped+backfilling, 142 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 202 op/s; 4669/265977 objects misplaced (1.755%); 15 MiB/s, 234 objects/s recovering 2024-09-21T11:43:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[29710]: osdmap e134: 8 total, 8 up, 7 in 2024-09-21T11:43:27.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[29710]: 5.b scrub starts 2024-09-21T11:43:27.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:26 smithi043 ceph-mon[29710]: 5.b scrub ok 2024-09-21T11:43:27.258 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:27.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:27.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:27.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:26 smithi175 ceph-mon[32613]: 5.10 scrub starts 2024-09-21T11:43:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:26 smithi175 ceph-mon[32613]: 5.10 scrub ok 2024-09-21T11:43:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:26 smithi175 ceph-mon[32613]: pgmap v151: 148 pgs: 1 active+clean+remapped, 5 active+remapped+backfilling, 142 active+clean; 5.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 202 op/s; 4669/265977 objects misplaced (1.755%); 15 MiB/s, 234 objects/s recovering 2024-09-21T11:43:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:26 smithi175 ceph-mon[32613]: osdmap e134: 8 total, 8 up, 7 in 2024-09-21T11:43:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:26 smithi175 ceph-mon[32613]: 5.b scrub starts 2024-09-21T11:43:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:26 smithi175 ceph-mon[32613]: 5.b scrub ok 2024-09-21T11:43:27.359 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:27.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:27.382 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:27.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:27.482 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:27.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:27.504 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:27.504 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:27.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:27.630 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:27.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:27.631 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:27.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:27.799 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:27.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:27.800 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:27.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:27.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:27.929 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:28.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:28.058 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:28.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:28.059 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:28.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:28.193 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:28.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:28.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[36454]: 4.1a scrub starts 2024-09-21T11:43:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[36454]: 4.1a scrub ok 2024-09-21T11:43:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[36454]: 3.15 scrub starts 2024-09-21T11:43:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[36454]: 3.15 scrub ok 2024-09-21T11:43:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[36454]: 5.2 scrub starts 2024-09-21T11:43:28.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[36454]: 5.2 scrub ok 2024-09-21T11:43:28.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[36454]: osdmap e135: 8 total, 8 up, 7 in 2024-09-21T11:43:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[29710]: 4.1a scrub starts 2024-09-21T11:43:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[29710]: 4.1a scrub ok 2024-09-21T11:43:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[29710]: 3.15 scrub starts 2024-09-21T11:43:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[29710]: 3.15 scrub ok 2024-09-21T11:43:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[29710]: 5.2 scrub starts 2024-09-21T11:43:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[29710]: 5.2 scrub ok 2024-09-21T11:43:28.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:27 smithi043 ceph-mon[29710]: osdmap e135: 8 total, 8 up, 7 in 2024-09-21T11:43:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:27 smithi175 ceph-mon[32613]: 4.1a scrub starts 2024-09-21T11:43:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:27 smithi175 ceph-mon[32613]: 4.1a scrub ok 2024-09-21T11:43:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:27 smithi175 ceph-mon[32613]: 3.15 scrub starts 2024-09-21T11:43:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:27 smithi175 ceph-mon[32613]: 3.15 scrub ok 2024-09-21T11:43:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:27 smithi175 ceph-mon[32613]: 5.2 scrub starts 2024-09-21T11:43:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:27 smithi175 ceph-mon[32613]: 5.2 scrub ok 2024-09-21T11:43:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:27 smithi175 ceph-mon[32613]: osdmap e135: 8 total, 8 up, 7 in 2024-09-21T11:43:28.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:28.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:28.339 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:28.440 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:28.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:28.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:28.470 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:28.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:28.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:28.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:28.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:28.740 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:28.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:28.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:28.841 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:28.875 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:28.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:28.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:28.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:29.015 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:29.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:29.016 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:29.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:29.151 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:29.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:29.152 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[36454]: 4.a scrub starts 2024-09-21T11:43:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[36454]: 4.a scrub ok 2024-09-21T11:43:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[36454]: pgmap v154: 148 pgs: 2 active+clean+remapped, 1 active+remapped+backfilling, 145 active+clean; 5.3 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 607 op/s; 327/261111 objects misplaced (0.125%); 81 MiB/s, 1.30k objects/s recovering 2024-09-21T11:43:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[36454]: osdmap e136: 8 total, 8 up, 7 in 2024-09-21T11:43:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[36454]: 5.15 scrub starts 2024-09-21T11:43:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[36454]: 5.15 scrub ok 2024-09-21T11:43:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[36454]: 5.6 scrub starts 2024-09-21T11:43:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[36454]: 5.6 scrub ok 2024-09-21T11:43:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[29710]: 4.a scrub starts 2024-09-21T11:43:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[29710]: 4.a scrub ok 2024-09-21T11:43:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[29710]: pgmap v154: 148 pgs: 2 active+clean+remapped, 1 active+remapped+backfilling, 145 active+clean; 5.3 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 607 op/s; 327/261111 objects misplaced (0.125%); 81 MiB/s, 1.30k objects/s recovering 2024-09-21T11:43:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[29710]: osdmap e136: 8 total, 8 up, 7 in 2024-09-21T11:43:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[29710]: 5.15 scrub starts 2024-09-21T11:43:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[29710]: 5.15 scrub ok 2024-09-21T11:43:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[29710]: 5.6 scrub starts 2024-09-21T11:43:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:28 smithi043 ceph-mon[29710]: 5.6 scrub ok 2024-09-21T11:43:29.253 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:29.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:28 smithi175 ceph-mon[32613]: 4.a scrub starts 2024-09-21T11:43:29.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:28 smithi175 ceph-mon[32613]: 4.a scrub ok 2024-09-21T11:43:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:28 smithi175 ceph-mon[32613]: pgmap v154: 148 pgs: 2 active+clean+remapped, 1 active+remapped+backfilling, 145 active+clean; 5.3 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 607 op/s; 327/261111 objects misplaced (0.125%); 81 MiB/s, 1.30k objects/s recovering 2024-09-21T11:43:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:28 smithi175 ceph-mon[32613]: osdmap e136: 8 total, 8 up, 7 in 2024-09-21T11:43:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:28 smithi175 ceph-mon[32613]: 5.15 scrub starts 2024-09-21T11:43:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:28 smithi175 ceph-mon[32613]: 5.15 scrub ok 2024-09-21T11:43:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:28 smithi175 ceph-mon[32613]: 5.6 scrub starts 2024-09-21T11:43:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:28 smithi175 ceph-mon[32613]: 5.6 scrub ok 2024-09-21T11:43:29.280 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:29.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:29.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:29.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:29.412 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:29.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:29.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:29.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:29.535 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:29.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:29.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:29.636 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:29.669 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:29.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:29.670 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:29.771 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:29.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:29.792 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:29.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:29.893 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:29.927 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:29.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:29.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:30.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:30.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:30.070 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:30.070 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:30.170 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:30.202 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:30.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:30.203 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:30.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:43:30.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:43:30.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[29710]: 6.1b scrub starts 2024-09-21T11:43:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[29710]: 6.1b scrub ok 2024-09-21T11:43:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[29710]: osdmap e137: 8 total, 8 up, 7 in 2024-09-21T11:43:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[29710]: 5.0 scrub starts 2024-09-21T11:43:30.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[29710]: 5.0 scrub ok 2024-09-21T11:43:30.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:43:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:43:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[36454]: 6.1b scrub starts 2024-09-21T11:43:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[36454]: 6.1b scrub ok 2024-09-21T11:43:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[36454]: osdmap e137: 8 total, 8 up, 7 in 2024-09-21T11:43:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[36454]: 5.0 scrub starts 2024-09-21T11:43:30.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:29 smithi043 ceph-mon[36454]: 5.0 scrub ok 2024-09-21T11:43:30.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:29 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:43:30.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:29 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:43:30.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:29 smithi175 ceph-mon[32613]: 6.1b scrub starts 2024-09-21T11:43:30.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:29 smithi175 ceph-mon[32613]: 6.1b scrub ok 2024-09-21T11:43:30.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:29 smithi175 ceph-mon[32613]: osdmap e137: 8 total, 8 up, 7 in 2024-09-21T11:43:30.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:29 smithi175 ceph-mon[32613]: 5.0 scrub starts 2024-09-21T11:43:30.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:29 smithi175 ceph-mon[32613]: 5.0 scrub ok 2024-09-21T11:43:30.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:30.343 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:30.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:30.344 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:30.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 6 1 2024-09-21T11:43:30.445 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:30.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:30.471 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:30.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:30.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:30.600 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:30.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:30.601 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:30.671 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:43:30.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:30.734 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:30.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:30.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:30.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:30.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:30.890 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:30.890 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:30.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:31.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:31.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:31.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:31.166 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:31.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:31.167 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:31.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: 4.1e scrub starts 2024-09-21T11:43:31.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: 4.1e scrub ok 2024-09-21T11:43:31.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: 6.8 scrub starts 2024-09-21T11:43:31.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: 6.8 scrub ok 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: pgmap v157: 148 pgs: 2 active+clean+remapped, 1 active+remapped+backfilling, 145 active+clean; 5.3 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 0 B/s wr, 405 op/s; 327/261111 objects misplaced (0.125%); 66 MiB/s, 1.06k objects/s recovering 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: 3.2 scrub starts 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: 3.2 scrub ok 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: 3.4 scrub starts 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: 3.4 scrub ok 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]} : dispatch 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 2]} : dispatch 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]} : dispatch 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 2]} : dispatch 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: 4.1e scrub starts 2024-09-21T11:43:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: 4.1e scrub ok 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: 6.8 scrub starts 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: 6.8 scrub ok 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: pgmap v157: 148 pgs: 2 active+clean+remapped, 1 active+remapped+backfilling, 145 active+clean; 5.3 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 0 B/s wr, 405 op/s; 327/261111 objects misplaced (0.125%); 66 MiB/s, 1.06k objects/s recovering 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: 3.2 scrub starts 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: 3.2 scrub ok 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: 3.4 scrub starts 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: 3.4 scrub ok 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]} : dispatch 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 2]} : dispatch 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]} : dispatch 2024-09-21T11:43:31.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 2]} : dispatch 2024-09-21T11:43:31.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:43:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: 4.1e scrub starts 2024-09-21T11:43:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: 4.1e scrub ok 2024-09-21T11:43:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: 6.8 scrub starts 2024-09-21T11:43:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: 6.8 scrub ok 2024-09-21T11:43:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: pgmap v157: 148 pgs: 2 active+clean+remapped, 1 active+remapped+backfilling, 145 active+clean; 5.3 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 0 B/s wr, 405 op/s; 327/261111 objects misplaced (0.125%); 66 MiB/s, 1.06k objects/s recovering 2024-09-21T11:43:31.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: 3.2 scrub starts 2024-09-21T11:43:31.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: 3.2 scrub ok 2024-09-21T11:43:31.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:31.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: 3.4 scrub starts 2024-09-21T11:43:31.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: 3.4 scrub ok 2024-09-21T11:43:31.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]} : dispatch 2024-09-21T11:43:31.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 2]} : dispatch 2024-09-21T11:43:31.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]} : dispatch 2024-09-21T11:43:31.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 2]} : dispatch 2024-09-21T11:43:31.268 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:31.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:31.310 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:31.393 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:31.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:31.464 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:31.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:31.465 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:31.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:31.602 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:31.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:31.603 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:31.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:31.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:31.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:31.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:31.897 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:31.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:31.898 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:31.998 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:32.025 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:32.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:32.026 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:32.083 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:43:32.083 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:43:32.102 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:32.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:32.159 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:32.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:32.160 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:32.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[29710]: 3.10 scrub starts 2024-09-21T11:43:32.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[29710]: 3.10 scrub ok 2024-09-21T11:43:32.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:32.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]}]': finished 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 2]}]': finished 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[29710]: osdmap e138: 8 total, 8 up, 7 in 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[29710]: 4.1c scrub starts 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[29710]: 4.1c scrub ok 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[36454]: 3.10 scrub starts 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[36454]: 3.10 scrub ok 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]}]': finished 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 2]}]': finished 2024-09-21T11:43:32.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[36454]: osdmap e138: 8 total, 8 up, 7 in 2024-09-21T11:43:32.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:43:32.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[36454]: 4.1c scrub starts 2024-09-21T11:43:32.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:31 smithi043 ceph-mon[36454]: 4.1c scrub ok 2024-09-21T11:43:32.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:31 smithi175 ceph-mon[32613]: 3.10 scrub starts 2024-09-21T11:43:32.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:31 smithi175 ceph-mon[32613]: 3.10 scrub ok 2024-09-21T11:43:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:31 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [3, 5]}]': finished 2024-09-21T11:43:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [3, 2]}]': finished 2024-09-21T11:43:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:31 smithi175 ceph-mon[32613]: osdmap e138: 8 total, 8 up, 7 in 2024-09-21T11:43:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:43:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:31 smithi175 ceph-mon[32613]: 4.1c scrub starts 2024-09-21T11:43:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:31 smithi175 ceph-mon[32613]: 4.1c scrub ok 2024-09-21T11:43:32.261 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:32.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:32.306 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:32.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:32.407 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:32.435 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:32.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:32.436 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:32.536 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:32.567 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:32.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:32.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:32.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:43:32.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:32.740 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:32.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:32.741 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:32.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:43:32.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:32.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:32.902 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:33.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:33.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:33.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:33.154 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:33.183 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:33.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:33.184 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:33.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:32 smithi043 ceph-mon[36454]: 5.4 scrub starts 2024-09-21T11:43:33.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:32 smithi043 ceph-mon[36454]: 5.4 scrub ok 2024-09-21T11:43:33.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:32 smithi043 ceph-mon[36454]: pgmap v159: 148 pgs: 1 peering, 147 active+clean; 5.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 865 op/s; 4.2 MiB/s, 67 objects/s recovering 2024-09-21T11:43:33.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:32 smithi043 ceph-mon[36454]: osdmap e139: 8 total, 8 up, 7 in 2024-09-21T11:43:33.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:32 smithi043 ceph-mon[29710]: 5.4 scrub starts 2024-09-21T11:43:33.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:32 smithi043 ceph-mon[29710]: 5.4 scrub ok 2024-09-21T11:43:33.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:32 smithi043 ceph-mon[29710]: pgmap v159: 148 pgs: 1 peering, 147 active+clean; 5.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 865 op/s; 4.2 MiB/s, 67 objects/s recovering 2024-09-21T11:43:33.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:32 smithi043 ceph-mon[29710]: osdmap e139: 8 total, 8 up, 7 in 2024-09-21T11:43:33.258 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:32 smithi175 ceph-mon[32613]: 5.4 scrub starts 2024-09-21T11:43:33.258 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:32 smithi175 ceph-mon[32613]: 5.4 scrub ok 2024-09-21T11:43:33.258 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:32 smithi175 ceph-mon[32613]: pgmap v159: 148 pgs: 1 peering, 147 active+clean; 5.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 865 op/s; 4.2 MiB/s, 67 objects/s recovering 2024-09-21T11:43:33.258 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:32 smithi175 ceph-mon[32613]: osdmap e139: 8 total, 8 up, 7 in 2024-09-21T11:43:33.285 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:33.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:33.329 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:33.329 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:33.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:43:33.430 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:33.455 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:33.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:33.456 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:33.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:33.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:33.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:33.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:33.734 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:33.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:33.735 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:33.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:33.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:33.872 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:33.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:33.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:34.002 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T11:43:34.018 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:34.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:34.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:34.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:34.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:34.166 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:34.167 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:34.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:34 smithi175 ceph-mon[32613]: 4.9 scrub starts 2024-09-21T11:43:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:34 smithi175 ceph-mon[32613]: 4.9 scrub ok 2024-09-21T11:43:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:34 smithi175 ceph-mon[32613]: 3.1b scrub starts 2024-09-21T11:43:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:34 smithi175 ceph-mon[32613]: 3.1b scrub ok 2024-09-21T11:43:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:34 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:43:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:34 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:43:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3406599125' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:43:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:34 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:43:34.268 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:34.298 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:34.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:34.299 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:34.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:34.439 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:34.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:34.440 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:34.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[36454]: 4.9 scrub starts 2024-09-21T11:43:34.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[36454]: 4.9 scrub ok 2024-09-21T11:43:34.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[36454]: 3.1b scrub starts 2024-09-21T11:43:34.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[36454]: 3.1b scrub ok 2024-09-21T11:43:34.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:43:34.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:43:34.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3406599125' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:43:34.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[29710]: 4.9 scrub starts 2024-09-21T11:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[29710]: 4.9 scrub ok 2024-09-21T11:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[29710]: 3.1b scrub starts 2024-09-21T11:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[29710]: 3.1b scrub ok 2024-09-21T11:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3406599125' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:43:34.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:34 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:43:34.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:34.572 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:34.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:34.573 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:34.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:34.712 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:34.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:34.713 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:34.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:34.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:34.857 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:34.857 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:34.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 7 1 2024-09-21T11:43:34.948 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:34.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:34.984 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:34.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:34.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:35.086 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:35.113 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:35.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:35.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:35.132 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:35 smithi043 ceph-mon[29710]: pgmap v161: 148 pgs: 1 peering, 147 active+clean; 5.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 877 B/s rd, 0 B/s wr, 718 op/s; 3.5 MiB/s, 55 objects/s recovering 2024-09-21T11:43:35.132 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:35 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:43:35.132 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:35 smithi043 ceph-mon[29710]: osdmap e140: 8 total, 8 up, 7 in 2024-09-21T11:43:35.132 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:35 smithi043 ceph-mon[36454]: pgmap v161: 148 pgs: 1 peering, 147 active+clean; 5.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 877 B/s rd, 0 B/s wr, 718 op/s; 3.5 MiB/s, 55 objects/s recovering 2024-09-21T11:43:35.132 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:35 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:43:35.132 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:35 smithi043 ceph-mon[36454]: osdmap e140: 8 total, 8 up, 7 in 2024-09-21T11:43:35.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:35.263 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:35.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:35.264 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:35.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:35.401 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:35.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:35.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:35.453 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:43:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:43:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:43:35.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:35.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:35 smithi175 ceph-mon[32613]: pgmap v161: 148 pgs: 1 peering, 147 active+clean; 5.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 877 B/s rd, 0 B/s wr, 718 op/s; 3.5 MiB/s, 55 objects/s recovering 2024-09-21T11:43:35.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:35 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:43:35.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:35 smithi175 ceph-mon[32613]: osdmap e140: 8 total, 8 up, 7 in 2024-09-21T11:43:35.534 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:35.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:35.535 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:35.636 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:35.678 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:35.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:35.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:35.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:35.810 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:35.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:35.811 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:35.845 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:43:35.845 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:43:35.880 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:43:35.880 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:43:35.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:35.928 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:43:35.928 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:43:35.955 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:35.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:35.956 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:35.958 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:43:35.958 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:43:36.035 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:43:36.035 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:43:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:43:36.036 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:43:36.036 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:43:36.054 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:43:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:43:36.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:36.082 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:36.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:36.083 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:36.098 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:43:36.098 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:43:36.132 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:43:36.133 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:43:36.167 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:43:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:43:36.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:36.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:36.217 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:36.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:36.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:36.367 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:36.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:36.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:36.370 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:36 smithi043 ceph-mon[29710]: 7.c scrub starts 2024-09-21T11:43:36.370 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:36 smithi043 ceph-mon[29710]: 7.8 scrub starts 2024-09-21T11:43:36.370 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:36 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/338453666' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:36.370 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:36 smithi043 ceph-mon[36454]: 7.c scrub starts 2024-09-21T11:43:36.370 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:36 smithi043 ceph-mon[36454]: 7.8 scrub starts 2024-09-21T11:43:36.370 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:36 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/338453666' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:36.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:36.508 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:36.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:36.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:36.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:36 smithi175 ceph-mon[32613]: 7.c scrub starts 2024-09-21T11:43:36.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:36 smithi175 ceph-mon[32613]: 7.8 scrub starts 2024-09-21T11:43:36.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:36 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/338453666' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:36.607 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:36.607 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:36.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:36.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:36.654 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:36.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:36.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:36.798 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:36.798 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:36.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:36.938 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:36.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:36.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:37.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:37.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:37.087 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:37.087 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:37.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:37.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:37.228 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:37.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:37.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[29710]: pgmap v163: 148 pgs: 1 active+clean+scrubbing, 1 peering, 146 active+clean; 4.9 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.12k op/s; 3.4 MiB/s, 54 objects/s recovering 2024-09-21T11:43:37.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[29710]: 7.c scrub ok 2024-09-21T11:43:37.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[29710]: 3.18 scrub starts 2024-09-21T11:43:37.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[29710]: 3.18 scrub ok 2024-09-21T11:43:37.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[29710]: 2.1 scrub starts 2024-09-21T11:43:37.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[29710]: 2.1 scrub ok 2024-09-21T11:43:37.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[36454]: pgmap v163: 148 pgs: 1 active+clean+scrubbing, 1 peering, 146 active+clean; 4.9 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.12k op/s; 3.4 MiB/s, 54 objects/s recovering 2024-09-21T11:43:37.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[36454]: 7.c scrub ok 2024-09-21T11:43:37.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[36454]: 3.18 scrub starts 2024-09-21T11:43:37.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[36454]: 3.18 scrub ok 2024-09-21T11:43:37.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[36454]: 2.1 scrub starts 2024-09-21T11:43:37.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:37 smithi043 ceph-mon[36454]: 2.1 scrub ok 2024-09-21T11:43:37.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:37.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:37.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:37.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:37 smithi175 ceph-mon[32613]: pgmap v163: 148 pgs: 1 active+clean+scrubbing, 1 peering, 146 active+clean; 4.9 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.12k op/s; 3.4 MiB/s, 54 objects/s recovering 2024-09-21T11:43:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:37 smithi175 ceph-mon[32613]: 7.c scrub ok 2024-09-21T11:43:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:37 smithi175 ceph-mon[32613]: 3.18 scrub starts 2024-09-21T11:43:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:37 smithi175 ceph-mon[32613]: 3.18 scrub ok 2024-09-21T11:43:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:37 smithi175 ceph-mon[32613]: 2.1 scrub starts 2024-09-21T11:43:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:37 smithi175 ceph-mon[32613]: 2.1 scrub ok 2024-09-21T11:43:37.523 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:37.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:37.526 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:37.627 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:37.665 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:37.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:37.666 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:37.768 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:37.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:43:37.796 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:37.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:37.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:37.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:37.925 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:37.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:37.926 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:38.027 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:38.052 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:38.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:38.053 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:38.153 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:38.181 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:38.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:38.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:38.282 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:38.310 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:38.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:38.311 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:38.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:38.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:38.466 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:38.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:38.551 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:38.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:38.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:38.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:38.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[36454]: 3.a scrub starts 2024-09-21T11:43:38.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[36454]: 3.a scrub ok 2024-09-21T11:43:38.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[36454]: 5.19 scrub starts 2024-09-21T11:43:38.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[36454]: 5.19 scrub ok 2024-09-21T11:43:38.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[36454]: 7.8 scrub ok 2024-09-21T11:43:38.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:38.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:38.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[36454]: 4.4 scrub starts 2024-09-21T11:43:38.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[36454]: 4.4 scrub ok 2024-09-21T11:43:38.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[29710]: 3.a scrub starts 2024-09-21T11:43:38.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[29710]: 3.a scrub ok 2024-09-21T11:43:38.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[29710]: 5.19 scrub starts 2024-09-21T11:43:38.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[29710]: 5.19 scrub ok 2024-09-21T11:43:38.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[29710]: 7.8 scrub ok 2024-09-21T11:43:38.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:38.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:38.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[29710]: 4.4 scrub starts 2024-09-21T11:43:38.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:38 smithi043 ceph-mon[29710]: 4.4 scrub ok 2024-09-21T11:43:38.733 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:38.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:38.762 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:38.762 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:38.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:38 smithi175 ceph-mon[32613]: 3.a scrub starts 2024-09-21T11:43:38.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:38 smithi175 ceph-mon[32613]: 3.a scrub ok 2024-09-21T11:43:38.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:38 smithi175 ceph-mon[32613]: 5.19 scrub starts 2024-09-21T11:43:38.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:38 smithi175 ceph-mon[32613]: 5.19 scrub ok 2024-09-21T11:43:38.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:38 smithi175 ceph-mon[32613]: 7.8 scrub ok 2024-09-21T11:43:38.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:38 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:38.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:38 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:38.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:38 smithi175 ceph-mon[32613]: 4.4 scrub starts 2024-09-21T11:43:38.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:38 smithi175 ceph-mon[32613]: 4.4 scrub ok 2024-09-21T11:43:38.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:38.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:38.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:38.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:39.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:39.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:43:39.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:39.042 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:39.042 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:39.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:39.191 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:39.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:39.192 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:39.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:39.332 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: pgmap v164: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 752 B/s rd, 0 B/s wr, 1.12k op/s; 68 B/s, 2 objects/s recovering 2024-09-21T11:43:39.332 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: 6.1 scrub starts 2024-09-21T11:43:39.332 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: 6.1 scrub ok 2024-09-21T11:43:39.332 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:43:39.332 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:43:39.332 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: osdmap e141: 8 total, 8 up, 7 in 2024-09-21T11:43:39.332 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: 5.d scrub starts 2024-09-21T11:43:39.332 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: 5.d scrub ok 2024-09-21T11:43:39.332 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: 5.17 scrub starts 2024-09-21T11:43:39.333 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: 5.17 scrub ok 2024-09-21T11:43:39.333 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/231579283' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:43:39.333 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3151524603' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:43:39.346 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:39.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:39.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:39.351 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:43:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T11:43:39.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:39.496 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:39.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:39.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:39.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: pgmap v164: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 752 B/s rd, 0 B/s wr, 1.12k op/s; 68 B/s, 2 objects/s recovering 2024-09-21T11:43:39.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: 6.1 scrub starts 2024-09-21T11:43:39.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: 6.1 scrub ok 2024-09-21T11:43:39.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:43:39.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:43:39.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: osdmap e141: 8 total, 8 up, 7 in 2024-09-21T11:43:39.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: 5.d scrub starts 2024-09-21T11:43:39.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: 5.d scrub ok 2024-09-21T11:43:39.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: 5.17 scrub starts 2024-09-21T11:43:39.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: 5.17 scrub ok 2024-09-21T11:43:39.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/231579283' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:43:39.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3151524603' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:43:39.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:39.636 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:39.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:39.637 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:39.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: pgmap v164: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 752 B/s rd, 0 B/s wr, 1.12k op/s; 68 B/s, 2 objects/s recovering 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: 6.1 scrub starts 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: 6.1 scrub ok 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: osdmap e141: 8 total, 8 up, 7 in 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: 5.d scrub starts 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: 5.d scrub ok 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: 5.17 scrub starts 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: 5.17 scrub ok 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/231579283' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:43:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3151524603' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:43:39.780 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:39.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:39.781 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:39.786 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:80915 7.e 7:75bc4e1c:::benchmark_data_smithi043_70646_object80914:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:08.211886+0000", 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.376899863000006, 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084894413000000002, 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 80915 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.211886+0000", 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.211886+0000", 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.211888+0000", 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8789999999999999e-06 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.211908+0000", 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0400000000000001e-05 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.211918+0000", 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5859999999999994e-06 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.211924+0000", 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4659999999999996e-06 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.211966+0000", 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2181999999999997e-05 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.212051+0000", 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4627000000000001e-05 2024-09-21T11:43:39.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.212148+0000", 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.734e-05 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.216081+0000", 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039329359999999997 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.216106+0000", 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5164000000000002e-05 2024-09-21T11:43:39.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.296245+0000", 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.080138941000000005 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.296620+0000", 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037499799999999998 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.296780+0000", 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000159894 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.792 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:80916 7.e 7:717134b6:::benchmark_data_smithi043_70646_object80915:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:08.212687+0000", 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.376098150000004, 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084871045000000006, 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 80916 2024-09-21T11:43:39.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.212687+0000", 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.212687+0000", 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.212688+0000", 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1209999999999999e-06 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.212712+0000", 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3152000000000001e-05 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.212723+0000", 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.134e-05 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.212730+0000", 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5699999999999998e-06 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.214418+0000", 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001688567 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.214511+0000", 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3187000000000001e-05 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.214597+0000", 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6003999999999999e-05 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.216221+0000", 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00162345 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.216285+0000", 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4064000000000002e-05 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.296971+0000", 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.080686042999999999 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.297261+0000", 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028991800000000001 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:08.297558+0000", 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00029762899999999999 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:93557 7.8 7:18589ab9:::benchmark_data_smithi043_70646_object93556:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:20.612115+0000", 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.976670604000006, 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078399602999999998, 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.800 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.800 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.800 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 93557 2024-09-21T11:43:39.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.800 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612115+0000", 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612115+0000", 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612115+0000", 2024-09-21T11:43:39.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.51e-07 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612135+0000", 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9241000000000001e-05 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612141+0000", 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8440000000000003e-06 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612145+0000", 2024-09-21T11:43:39.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7310000000000001e-06 2024-09-21T11:43:39.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.618295+0000", 2024-09-21T11:43:39.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061494619999999996 2024-09-21T11:43:39.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.618424+0000", 2024-09-21T11:43:39.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012948500000000001 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,7", 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.618589+0000", 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016432299999999999 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.622660+0000", 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040709479999999996 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623393+0000", 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00073394200000000001 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.626858+0000", 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034649149999999998 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.690465+0000", 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.063606857000000003 2024-09-21T11:43:39.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.690515+0000", 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9304000000000001e-05 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:93558 7.e 7:7d43af41:::benchmark_data_smithi043_70646_object93557:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:20.612152+0000", 2024-09-21T11:43:39.807 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.976633415999999, 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078917778999999993, 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 93558 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612152+0000", 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612152+0000", 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612153+0000", 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-07 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612172+0000", 2024-09-21T11:43:39.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.946e-05 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612178+0000", 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8510000000000004e-06 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.612183+0000", 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8590000000000004e-06 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.618524+0000", 2024-09-21T11:43:39.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063413890000000002 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.618624+0000", 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9708000000000001e-05 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.618750+0000", 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012588500000000001 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.622619+0000", 2024-09-21T11:43:39.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038690280000000001 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.623932+0000", 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013134119999999999 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.690789+0000", 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.066856958999999994 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.690972+0000", 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018273399999999999 2024-09-21T11:43:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:20.691070+0000", 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8053999999999993e-05 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:98458 7.e 7:70351833:::benchmark_data_smithi043_70646_object98457:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:25.506976+0000", 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 74.081809683000003, 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.086509519000000007, 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 98458 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.506976+0000", 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.506976+0000", 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.506977+0000", 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0559999999999999e-06 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.507008+0000", 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1207999999999997e-05 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.507019+0000", 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.099e-05 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.507026+0000", 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3470000000000001e-06 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.507082+0000", 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6019e-05 2024-09-21T11:43:39.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.507189+0000", 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106572 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.507337+0000", 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014766000000000001 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.511500+0000", 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041634740000000003 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.511515+0000", 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4367e-05 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.590394+0000", 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.078879371000000004 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.590459+0000", 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4914999999999998e-05 2024-09-21T11:43:39.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.593485+0000", 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030265399999999999 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:98652 7.e 7:726aac7d:::benchmark_data_smithi043_70646_object98651:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:25.800920+0000", 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.787865099000001, 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11258921700000001, 2024-09-21T11:43:39.819 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 98652 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.800920+0000", 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.800920+0000", 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.800922+0000", 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9530000000000002e-06 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.800947+0000", 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4706e-05 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.800958+0000", 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0923e-05 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.800966+0000", 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0919999999999998e-06 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.874288+0000", 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.073322271999999994 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.874402+0000", 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113786 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.874526+0000", 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012398699999999999 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.886844+0000", 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012317524 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.887088+0000", 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000244539 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.913312+0000", 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026224081999999999 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.913431+0000", 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118903 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:25.913510+0000", 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8449999999999996e-05 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:99015 7.e 7:74e80bab:::benchmark_data_smithi043_70646_object99014:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.320644+0000", 2024-09-21T11:43:39.826 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.268141864, 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12664123799999999, 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 99015 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320644+0000", 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320644+0000", 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320646+0000", 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8899999999999999e-06 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320673+0000", 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7270000000000001e-05 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320685+0000", 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2326e-05 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320693+0000", 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6559999999999992e-06 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320812+0000", 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118983 2024-09-21T11:43:39.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320928+0000", 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115841 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.321051+0000", 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123437 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.324857+0000", 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038061000000000002 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334002+0000", 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0091444979999999992 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447169+0000", 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.113167226 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447221+0000", 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2080000000000003e-05 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447285+0000", 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3930999999999995e-05 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.832 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:99021 7.e 7:7a4ac083:::benchmark_data_smithi043_70646_object99020:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.325207+0000", 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.263578253999995, 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13657670399999999, 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 99021 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325207+0000", 2024-09-21T11:43:39.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325207+0000", 2024-09-21T11:43:39.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325208+0000", 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8800000000000002e-07 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325229+0000", 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1180999999999999e-05 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325239+0000", 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8400000000000007e-06 2024-09-21T11:43:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325247+0000", 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3790000000000006e-06 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330846+0000", 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055982740000000003 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330952+0000", 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106563 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.331082+0000", 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013003799999999999 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334345+0000", 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032629339999999999 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447059+0000", 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11271336 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461671+0000", 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014612823 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461721+0000", 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9688000000000003e-05 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461784+0000", 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2836000000000006e-05 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:99029 7.e 7:77218e16:::benchmark_data_smithi043_70646_object99028:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.334806+0000", 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.253979037999997, 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12710760600000001, 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 99029 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334806+0000", 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334806+0000", 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334808+0000", 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.64e-06 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334835+0000", 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240999999999999e-05 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334848+0000", 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2415000000000001e-05 2024-09-21T11:43:39.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334856+0000", 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0209999999999999e-06 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335270+0000", 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000413905 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335377+0000", 2024-09-21T11:43:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107583 2024-09-21T11:43:39.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335502+0000", 2024-09-21T11:43:39.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000124595 2024-09-21T11:43:39.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447122+0000", 2024-09-21T11:43:39.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.111620521 2024-09-21T11:43:39.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447133+0000", 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0701e-05 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461829+0000", 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014696074999999999 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461873+0000", 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3374999999999997e-05 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461914+0000", 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1533999999999999e-05 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:99021 7.e e107/82 ondisk, result = 0)", 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.338128+0000", 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.250657716999996, 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.108948584, 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:43:39.846 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338128+0000", 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338128+0000", 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338129+0000", 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0500000000000002e-07 2024-09-21T11:43:39.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338133+0000", 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3069999999999996e-06 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338142+0000", 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4779999999999997e-06 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338149+0000", 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9149999999999999e-06 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447043+0000", 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.108893407 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447051+0000", 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6119999999999999e-06 2024-09-21T11:43:39.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447076+0000", 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4960000000000002e-05 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:99029 7.e e107/82 ondisk, result = 0)", 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.338301+0000", 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.250484235000002, 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.10883254000000001, 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.850 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338301+0000", 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338301+0000", 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338302+0000", 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9699999999999995e-07 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338306+0000", 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2989999999999998e-06 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338317+0000", 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1044e-05 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338325+0000", 2024-09-21T11:43:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8150000000000006e-06 2024-09-21T11:43:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447107+0000", 2024-09-21T11:43:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.108781515 2024-09-21T11:43:39.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447117+0000", 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0353999999999999e-05 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447134+0000", 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6716000000000001e-05 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.854 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:99033 7.5 e107/82 7:a309294f:::benchmark_data_smithi043_70646_object99032:head v 107'6223, mlcod=107'6221)", 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.338400+0000", 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 73.250385906999995, 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.095287362, 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338400+0000", 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338400+0000", 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338400+0000", 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3e-07 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338489+0000", 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8869000000000006e-05 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338499+0000", 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8109999999999997e-06 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338505+0000", 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.241e-06 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338577+0000", 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2095999999999995e-05 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338596+0000", 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9001e-05 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433593+0000", 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.094996454999999994 2024-09-21T11:43:39.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433687+0000", 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4159000000000004e-05 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:100307 7.e 7:7acab972:::benchmark_data_smithi043_70646_object100306:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:27.888309+0000", 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.700476253999994, 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.085580681000000006, 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.859 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 100307 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.888309+0000", 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.888309+0000", 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.888311+0000", 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.592e-06 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.888334+0000", 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.338e-05 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.862 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.888345+0000", 2024-09-21T11:43:39.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0769e-05 2024-09-21T11:43:39.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.888353+0000", 2024-09-21T11:43:39.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5329999999999997e-06 2024-09-21T11:43:39.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.888647+0000", 2024-09-21T11:43:39.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00029410500000000002 2024-09-21T11:43:39.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.888809+0000", 2024-09-21T11:43:39.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016244400000000001 2024-09-21T11:43:39.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.888969+0000", 2024-09-21T11:43:39.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016025300000000001 2024-09-21T11:43:39.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.953909+0000", 2024-09-21T11:43:39.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.064939909000000004 2024-09-21T11:43:39.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.953922+0000", 2024-09-21T11:43:39.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2261e-05 2024-09-21T11:43:39.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.973751+0000", 2024-09-21T11:43:39.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019829497000000001 2024-09-21T11:43:39.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.973811+0000", 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9627999999999999e-05 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:27.973890+0000", 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9309999999999998e-05 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:100923 7.e 7:767f7391:::benchmark_data_smithi043_70646_object100922:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:43:39.868 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:28.574418+0000", 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.014367453999995, 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.089125826000000005, 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 100923 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.574418+0000", 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.574418+0000", 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.574419+0000", 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2219999999999999e-06 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.574437+0000", 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8167000000000001e-05 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.574446+0000", 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4479999999999994e-06 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.574450+0000", 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5580000000000002e-06 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.574492+0000", 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1510000000000001e-05 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.574575+0000", 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3090000000000006e-05 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.574658+0000", 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3175000000000002e-05 2024-09-21T11:43:39.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577036+0000", 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023778089999999998 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577511+0000", 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00047537700000000003 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.663372+0000", 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.085860383999999998 2024-09-21T11:43:39.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.663484+0000", 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011265900000000001 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.663544+0000", 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9426999999999999e-05 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.874 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:100924 7.e 7:79043a28:::benchmark_data_smithi043_70646_object100923:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:28.577498+0000", 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 71.011287160999998, 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.086310008999999993, 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 100924 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.875 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577498+0000", 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577498+0000", 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577499+0000", 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2600000000000001e-07 2024-09-21T11:43:39.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577516+0000", 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6773999999999999e-05 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577522+0000", 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.19e-06 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577526+0000", 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0219999999999998e-06 2024-09-21T11:43:39.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577559+0000", 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2663000000000002e-05 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577631+0000", 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1874999999999999e-05 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.577710+0000", 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8751999999999994e-05 2024-09-21T11:43:39.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.579546+0000", 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018366109999999999 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.579689+0000", 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014250800000000001 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.663696+0000", 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084007033999999994 2024-09-21T11:43:39.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.663774+0000", 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8567999999999997e-05 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.663808+0000", 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4085999999999999e-05 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.880 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:101064 7.e 7:77d6e359:::benchmark_data_smithi043_70646_object101063:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:28.769023+0000", 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 70.819762401000006, 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.093500502999999999, 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 101064 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.769023+0000", 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.769023+0000", 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.769025+0000", 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.871e-06 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.769051+0000", 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6503e-05 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.769065+0000", 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3791000000000001e-05 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.769074+0000", 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5590000000000006e-06 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.769127+0000", 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3427999999999997e-05 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.769238+0000", 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011097100000000001 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.769339+0000", 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100523 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.771779+0000", 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002440298 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.771794+0000", 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4637000000000001e-05 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.862419+0000", 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.090625150000000002 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.862479+0000", 2024-09-21T11:43:39.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9951999999999998e-05 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.862524+0000", 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4820000000000001e-05 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:122606 7.e 7:765e29c7:::benchmark_data_smithi043_70646_object5198:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:43:39.886 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:55.382480+0000", 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 44.206305909000001, 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.077920689000000001, 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 122606 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.382480+0000", 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.382480+0000", 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.382481+0000", 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4899999999999999e-06 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.382486+0000", 2024-09-21T11:43:39.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5140000000000003e-06 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.382497+0000", 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1481000000000001e-05 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.382505+0000", 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6750000000000002e-06 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.385854+0000", 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003349318 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.385962+0000", 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107874 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.386068+0000", 2024-09-21T11:43:39.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105924 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.388482+0000", 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024144890000000001 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.448152+0000", 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.059669801000000001 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.460301+0000", 2024-09-21T11:43:39.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012149126999999999 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.460355+0000", 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3358e-05 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.460400+0000", 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5637999999999999e-05 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:123059 7.e 7:726734e6:::benchmark_data_smithi043_70646_object5651:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:55.885335+0000", 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.703450721999999, 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.104585872, 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 123059 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.885335+0000", 2024-09-21T11:43:39.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.885335+0000", 2024-09-21T11:43:39.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.885336+0000", 2024-09-21T11:43:39.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.207e-06 2024-09-21T11:43:39.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.885341+0000", 2024-09-21T11:43:39.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1460000000000004e-06 2024-09-21T11:43:39.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.885353+0000", 2024-09-21T11:43:39.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1768e-05 2024-09-21T11:43:39.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.885362+0000", 2024-09-21T11:43:39.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0000000000000002e-06 2024-09-21T11:43:39.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.885422+0000", 2024-09-21T11:43:39.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0285000000000003e-05 2024-09-21T11:43:39.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.885534+0000", 2024-09-21T11:43:39.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112105 2024-09-21T11:43:39.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.885636+0000", 2024-09-21T11:43:39.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101672 2024-09-21T11:43:39.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.891498+0000", 2024-09-21T11:43:39.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058624910000000001 2024-09-21T11:43:39.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.891514+0000", 2024-09-21T11:43:39.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5267000000000001e-05 2024-09-21T11:43:39.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.989831+0000", 2024-09-21T11:43:39.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.098317317000000001 2024-09-21T11:43:39.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.989877+0000", 2024-09-21T11:43:39.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5967999999999998e-05 2024-09-21T11:43:39.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.989921+0000", 2024-09-21T11:43:39.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3646000000000002e-05 2024-09-21T11:43:39.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.902 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.902 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:123061 7.e 7:7a37ff1e:::benchmark_data_smithi043_70646_object5653:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:43:39.902 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:55.888061+0000", 2024-09-21T11:43:39.903 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.700724802000003, 2024-09-21T11:43:39.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.102026987, 2024-09-21T11:43:39.903 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.903 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:43:39.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:43:39.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:43:39.903 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:43:39.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 123061 2024-09-21T11:43:39.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.904 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.888061+0000", 2024-09-21T11:43:39.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.888061+0000", 2024-09-21T11:43:39.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.905 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.905 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.888062+0000", 2024-09-21T11:43:39.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.331e-06 2024-09-21T11:43:39.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.888067+0000", 2024-09-21T11:43:39.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0760000000000002e-06 2024-09-21T11:43:39.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.888078+0000", 2024-09-21T11:43:39.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1037e-05 2024-09-21T11:43:39.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.888086+0000", 2024-09-21T11:43:39.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7430000000000002e-06 2024-09-21T11:43:39.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.888782+0000", 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00069585299999999997 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.888887+0000", 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104802 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.888999+0000", 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112661 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.891555+0000", 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025554879999999999 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.891579+0000", 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4102999999999999e-05 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.990005+0000", 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.098425795999999996 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.990050+0000", 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5565999999999997e-05 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.990088+0000", 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7530999999999997e-05 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "MOSDPGPush(7.b 122/113 [PushOp(7:d536a18c:::benchmark_data_smithi043_70646_object36599:head, version: 93'2238, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7:d536a18c:::benchmark_data_smithi043_70646_object36599:head@93'2238, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-21T11:43:39.910 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:55.893694+0000", 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 43.695091582000003, 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.076779557999999998, 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.893694+0000", 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.893694+0000", 2024-09-21T11:43:39.911 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.893694+0000", 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0699999999999997e-07 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.893747+0000", 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2818999999999997e-05 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.968211+0000", 2024-09-21T11:43:39.912 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.074463800999999996 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.968316+0000", 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010487199999999999 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.970274+0000", 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001958188 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.970286+0000", 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1779e-05 2024-09-21T11:43:39.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:43:39.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:43:39.914 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:43:39.914 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:55.970473+0000", 2024-09-21T11:43:39.914 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018759199999999999 2024-09-21T11:43:39.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.914 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:43:39.914 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:43:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:43:39.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:39.953 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:39.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:39.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:40.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:40.083 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:40.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:40.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:40.184 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:40.214 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:40.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:40.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:40.316 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:40.346 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:40.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:40.347 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:40.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:40.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:40.489 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:40.490 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:40.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:40.626 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:40.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:40.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:40.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[29710]: 5.1a scrub starts 2024-09-21T11:43:40.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[29710]: 5.1a scrub ok 2024-09-21T11:43:40.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[29710]: 4.d scrub starts 2024-09-21T11:43:40.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[29710]: 4.d scrub ok 2024-09-21T11:43:40.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:43:40.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:43:40.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/231579283' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:43:40.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3151524603' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:43:40.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[29710]: osdmap e142: 8 total, 8 up, 7 in 2024-09-21T11:43:40.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[36454]: 5.1a scrub starts 2024-09-21T11:43:40.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[36454]: 5.1a scrub ok 2024-09-21T11:43:40.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[36454]: 4.d scrub starts 2024-09-21T11:43:40.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[36454]: 4.d scrub ok 2024-09-21T11:43:40.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:43:40.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:43:40.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/231579283' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:43:40.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3151524603' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:43:40.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:40 smithi043 ceph-mon[36454]: osdmap e142: 8 total, 8 up, 7 in 2024-09-21T11:43:40.727 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:40.756 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:40.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:40.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:40 smithi175 ceph-mon[32613]: 5.1a scrub starts 2024-09-21T11:43:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:40 smithi175 ceph-mon[32613]: 5.1a scrub ok 2024-09-21T11:43:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:40 smithi175 ceph-mon[32613]: 4.d scrub starts 2024-09-21T11:43:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:40 smithi175 ceph-mon[32613]: 4.d scrub ok 2024-09-21T11:43:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:40 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:43:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:40 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:43:40.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:40 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/231579283' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:43:40.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:40 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3151524603' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:43:40.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:40 smithi175 ceph-mon[32613]: osdmap e142: 8 total, 8 up, 7 in 2024-09-21T11:43:40.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:40.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:40.893 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:40.893 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:40.994 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:41.027 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:41.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:41.028 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:41.128 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:41.156 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:41.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:41.157 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:41.167 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T11:43:41.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:43:41.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:41.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:43:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:43:41.294 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:41.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:41.295 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:41.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:41.425 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:41.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:41.426 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: pgmap v167: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 943 op/s; 77 B/s, 2 objects/s recovering 2024-09-21T11:43:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: 5.11 scrub starts 2024-09-21T11:43:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: 5.11 scrub ok 2024-09-21T11:43:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: 4.13 scrub starts 2024-09-21T11:43:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: 4.13 scrub ok 2024-09-21T11:43:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: 4.0 scrub starts 2024-09-21T11:43:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: 4.0 scrub ok 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: 5.8 scrub starts 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: 5.8 scrub ok 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: 4.6 scrub starts 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[36454]: 4.6 scrub ok 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: pgmap v167: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 943 op/s; 77 B/s, 2 objects/s recovering 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: 5.11 scrub starts 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: 5.11 scrub ok 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: 4.13 scrub starts 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: 4.13 scrub ok 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: 4.0 scrub starts 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: 4.0 scrub ok 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: 5.8 scrub starts 2024-09-21T11:43:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: 5.8 scrub ok 2024-09-21T11:43:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: 4.6 scrub starts 2024-09-21T11:43:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:41 smithi043 ceph-mon[29710]: 4.6 scrub ok 2024-09-21T11:43:41.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:41.572 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:41.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:41.573 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:41.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:41.713 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:41.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:41.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: pgmap v167: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 4.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 852 B/s rd, 0 B/s wr, 943 op/s; 77 B/s, 2 objects/s recovering 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: 5.11 scrub starts 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: 5.11 scrub ok 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: 4.13 scrub starts 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: 4.13 scrub ok 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: 4.0 scrub starts 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: 4.0 scrub ok 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: 5.8 scrub starts 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: 5.8 scrub ok 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: 4.6 scrub starts 2024-09-21T11:43:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:41 smithi175 ceph-mon[32613]: 4.6 scrub ok 2024-09-21T11:43:41.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:41.854 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:41.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:41.855 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:41.956 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:41.989 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:41.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:41.990 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:42.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:42.131 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:42.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:42.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:42.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:42.262 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:42.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:42.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:42.363 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:42.396 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:42.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:42.397 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:42.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:42.535 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:42.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:42.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:42.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:43:42.626 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:42.628 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:42.636 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:42.662 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:42.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:42.662 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:42.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:42.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:43:42.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:43:42.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[29710]: 5.c scrub starts 2024-09-21T11:43:42.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[29710]: 5.c scrub ok 2024-09-21T11:43:42.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[29710]: osdmap e143: 8 total, 8 up, 7 in 2024-09-21T11:43:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:43:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:43:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[36454]: 5.c scrub starts 2024-09-21T11:43:42.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[36454]: 5.c scrub ok 2024-09-21T11:43:42.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:42 smithi043 ceph-mon[36454]: osdmap e143: 8 total, 8 up, 7 in 2024-09-21T11:43:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:42 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:43:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:42 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:43:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:42 smithi175 ceph-mon[32613]: 5.c scrub starts 2024-09-21T11:43:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:42 smithi175 ceph-mon[32613]: 5.c scrub ok 2024-09-21T11:43:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:42 smithi175 ceph-mon[32613]: osdmap e143: 8 total, 8 up, 7 in 2024-09-21T11:43:42.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:42.823 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:42.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:42.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:42.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:42.975 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:42.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:42.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:43.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:43.117 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:43.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:43.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:43.219 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:43.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:43.251 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:43.252 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:43.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:43.393 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:43.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:43.394 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:43.495 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:43.544 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:43.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:43.545 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:43.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[29710]: pgmap v169: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.53k op/s; 77 B/s, 2 objects/s recovering 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[29710]: 4.12 scrub starts 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[29710]: 4.12 scrub ok 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[29710]: 6.14 scrub starts 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[29710]: 6.14 scrub ok 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[29710]: 3.1 scrub starts 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[29710]: 3.1 scrub ok 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[36454]: pgmap v169: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.53k op/s; 77 B/s, 2 objects/s recovering 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[36454]: 4.12 scrub starts 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[36454]: 4.12 scrub ok 2024-09-21T11:43:43.605 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[36454]: 6.14 scrub starts 2024-09-21T11:43:43.606 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[36454]: 6.14 scrub ok 2024-09-21T11:43:43.606 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[36454]: 3.1 scrub starts 2024-09-21T11:43:43.606 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:43 smithi043 ceph-mon[36454]: 3.1 scrub ok 2024-09-21T11:43:43.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:43.674 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:43.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:43.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:43 smithi175 ceph-mon[32613]: pgmap v169: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.53k op/s; 77 B/s, 2 objects/s recovering 2024-09-21T11:43:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:43 smithi175 ceph-mon[32613]: 4.12 scrub starts 2024-09-21T11:43:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:43 smithi175 ceph-mon[32613]: 4.12 scrub ok 2024-09-21T11:43:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:43 smithi175 ceph-mon[32613]: 6.14 scrub starts 2024-09-21T11:43:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:43 smithi175 ceph-mon[32613]: 6.14 scrub ok 2024-09-21T11:43:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:43 smithi175 ceph-mon[32613]: 3.1 scrub starts 2024-09-21T11:43:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:43 smithi175 ceph-mon[32613]: 3.1 scrub ok 2024-09-21T11:43:43.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:43.829 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:43.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:43.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:43.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:43.958 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:43.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:43.959 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:44.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:44.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:44.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:44.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:44.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:44.289 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:44.312 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:43:44.313 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:43:44.361 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:43:44.361 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:43:44.390 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:44.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:44.418 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:44.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:44.422 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:43:44.422 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:43:44.475 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:43:44.475 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:43:44.492 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:44.520 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:44.527 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:43:44.528 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:43:44.546 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:44.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:44.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:44.595 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:43:44.595 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:43:44.631 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:43:44.631 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:43:44.648 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:44.668 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:43:44.669 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:43:44.674 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:44.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:44.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: 5.1d scrub starts 2024-09-21T11:43:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: 5.1d scrub ok 2024-09-21T11:43:44.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: 7.5 scrub starts 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: pgmap v170: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/941285361' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: 4.f scrub starts 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[29710]: 4.f scrub ok 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: 5.1d scrub starts 2024-09-21T11:43:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: 5.1d scrub ok 2024-09-21T11:43:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: 7.5 scrub starts 2024-09-21T11:43:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: pgmap v170: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:43:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/941285361' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:43:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:43:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: 4.f scrub starts 2024-09-21T11:43:44.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:44 smithi043 ceph-mon[36454]: 4.f scrub ok 2024-09-21T11:43:44.776 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:43:44.797 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:43:44.804 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:44.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:44.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:44.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:44.948 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:44.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:44.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: 5.1d scrub starts 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: 5.1d scrub ok 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: 7.5 scrub starts 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: pgmap v170: 148 pgs: 148 active+clean; 4.3 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/941285361' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: 4.f scrub starts 2024-09-21T11:43:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:44 smithi175 ceph-mon[32613]: 4.f scrub ok 2024-09-21T11:43:45.051 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:45.079 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:45.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:45.080 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:45.180 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:45.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:45.212 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:45.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:45.297 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:43:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:43:45.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:45.342 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:45.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:45.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:45.443 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:45.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:45.472 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:45.472 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:45.570 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:43:45.571 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:43:45.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:45.603 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:45.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:45.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:45.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:45.734 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:45.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:45.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:45.836 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:45.868 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:45.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:45.869 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:45.970 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[36454]: 5.f scrub starts 2024-09-21T11:43:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[36454]: 5.f scrub ok 2024-09-21T11:43:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[36454]: 3.e scrub starts 2024-09-21T11:43:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[36454]: 3.e scrub ok 2024-09-21T11:43:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[36454]: 7.5 scrub ok 2024-09-21T11:43:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[36454]: 6.18 scrub starts 2024-09-21T11:43:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[36454]: 6.18 scrub ok 2024-09-21T11:43:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[36454]: from='client.24817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:43:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[29710]: 5.f scrub starts 2024-09-21T11:43:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[29710]: 5.f scrub ok 2024-09-21T11:43:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[29710]: 3.e scrub starts 2024-09-21T11:43:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[29710]: 3.e scrub ok 2024-09-21T11:43:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[29710]: 7.5 scrub ok 2024-09-21T11:43:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[29710]: 6.18 scrub starts 2024-09-21T11:43:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[29710]: 6.18 scrub ok 2024-09-21T11:43:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[29710]: from='client.24817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:43:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:45 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:46.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:46.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:46.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:45 smithi175 ceph-mon[32613]: 5.f scrub starts 2024-09-21T11:43:46.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:45 smithi175 ceph-mon[32613]: 5.f scrub ok 2024-09-21T11:43:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:45 smithi175 ceph-mon[32613]: 3.e scrub starts 2024-09-21T11:43:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:45 smithi175 ceph-mon[32613]: 3.e scrub ok 2024-09-21T11:43:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:45 smithi175 ceph-mon[32613]: 7.5 scrub ok 2024-09-21T11:43:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:45 smithi175 ceph-mon[32613]: 6.18 scrub starts 2024-09-21T11:43:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:45 smithi175 ceph-mon[32613]: 6.18 scrub ok 2024-09-21T11:43:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:45 smithi175 ceph-mon[32613]: from='client.24817 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:43:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:45 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:43:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:45 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:43:46.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:46.149 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:46.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:46.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:46.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:46.260 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-21T11:43:43.777808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4649327155,"num_objects":71134,"num_object_clones":0,"num_object_copies":213402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71134,"num_whiteouts":0,"num_read":1051,"num_read_kb":908,"num_write":164426,"num_write_kb":7515594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38553,"num_bytes_recovered":2517897553,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77114,"ondisk_log_size":77114,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":14715904,"kb_used_data":13781096,"kb_used_omap":10,"kb_used_meta":934517,"kb_avail":641500160,"statfs":{"total":671965249536,"available":656896163840,"internally_reserved":0,"allocated":14111842304,"data_stored":14109057702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11156,"internal_metadata":956945516},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,10,4,13],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-396492800,"num_objects":-6050,"num_object_clones":0,"num_object_copies":-18150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6050,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6050,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001578"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368432+0000","last_change":"2024-09-21T11:41:42.291880+0000","last_active":"2024-09-21T11:43:39.368432+0000","last_peered":"2024-09-21T11:43:39.368432+0000","last_clean":"2024-09-21T11:43:39.368432+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:43:39.368432+0000","last_undegraded":"2024-09-21T11:43:39.368432+0000","last_fullsized":"2024-09-21T11:43:39.368432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:42.290188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:42.290188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":216,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368032+0000","last_change":"2024-09-21T11:42:37.191052+0000","last_active":"2024-09-21T11:43:39.368032+0000","last_peered":"2024-09-21T11:43:39.368032+0000","last_clean":"2024-09-21T11:43:39.368032+0000","last_became_active":"2024-09-21T11:42:37.164858+0000","last_became_peered":"2024-09-21T11:42:37.164858+0000","last_unstale":"2024-09-21T11:43:39.368032+0000","last_undegraded":"2024-09-21T11:43:39.368032+0000","last_fullsized":"2024-09-21T11:43:39.368032+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:26.994148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:26.994148+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:16.329886+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":80,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.349499+0000","last_change":"2024-09-21T11:43:37.050216+0000","last_active":"2024-09-21T11:43:39.349499+0000","last_peered":"2024-09-21T11:43:39.349499+0000","last_clean":"2024-09-21T11:43:39.349499+0000","last_became_active":"2024-09-21T11:42:37.161713+0000","last_became_peered":"2024-09-21T11:42:37.161713+0000","last_unstale":"2024-09-21T11:43:39.349499+0000","last_undegraded":"2024-09-21T11:43:39.349499+0000","last_fullsized":"2024-09-21T11:43:39.349499+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:52.112502+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":182,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368100+0000","last_change":"2024-09-21T11:42:59.201398+0000","last_active":"2024-09-21T11:43:39.368100+0000","last_peered":"2024-09-21T11:43:39.368100+0000","last_clean":"2024-09-21T11:43:39.368100+0000","last_became_active":"2024-09-21T11:42:37.158907+0000","last_became_peered":"2024-09-21T11:42:37.158907+0000","last_unstale":"2024-09-21T11:43:39.368100+0000","last_undegraded":"2024-09-21T11:43:39.368100+0000","last_fullsized":"2024-09-21T11:43:39.368100+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:59.201330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:59.201330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:02.317360+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":175,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.358089+0000","last_change":"2024-09-21T11:43:28.461458+0000","last_active":"2024-09-21T11:43:39.358089+0000","last_peered":"2024-09-21T11:43:39.358089+0000","last_clean":"2024-09-21T11:43:39.358089+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:43:39.358089+0000","last_undegraded":"2024-09-21T11:43:39.358089+0000","last_fullsized":"2024-09-21T11:43:39.358089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.461378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.461378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.850394+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":179,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.359827+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:43:39.359827+0000","last_peered":"2024-09-21T11:43:39.359827+0000","last_clean":"2024-09-21T11:43:39.359827+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:43:39.359827+0000","last_undegraded":"2024-09-21T11:43:39.359827+0000","last_fullsized":"2024-09-21T11:43:39.359827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":200,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.360732+0000","last_change":"2024-09-21T11:40:02.315669+0000","last_active":"2024-09-21T11:43:39.360732+0000","last_peered":"2024-09-21T11:43:39.360732+0000","last_clean":"2024-09-21T11:43:39.360732+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:43:39.360732+0000","last_undegraded":"2024-09-21T11:43:39.360732+0000","last_fullsized":"2024-09-21T11:43:39.360732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":194,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.367962+0000","last_change":"2024-09-21T11:43:04.373004+0000","last_active":"2024-09-21T11:43:39.367962+0000","last_peered":"2024-09-21T11:43:39.367962+0000","last_clean":"2024-09-21T11:43:39.367962+0000","last_became_active":"2024-09-21T11:42:37.154511+0000","last_became_peered":"2024-09-21T11:42:37.154511+0000","last_unstale":"2024-09-21T11:43:39.367962+0000","last_undegraded":"2024-09-21T11:43:39.367962+0000","last_fullsized":"2024-09-21T11:43:39.367962+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:04.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:04.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:06.903626+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350440+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:43:39.350440+0000","last_peered":"2024-09-21T11:43:39.350440+0000","last_clean":"2024-09-21T11:43:39.350440+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:43:39.350440+0000","last_undegraded":"2024-09-21T11:43:39.350440+0000","last_fullsized":"2024-09-21T11:43:39.350440+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":207,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368222+0000","last_change":"2024-09-21T11:43:26.258023+0000","last_active":"2024-09-21T11:43:39.368222+0000","last_peered":"2024-09-21T11:43:39.368222+0000","last_clean":"2024-09-21T11:43:39.368222+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:43:39.368222+0000","last_undegraded":"2024-09-21T11:43:39.368222+0000","last_fullsized":"2024-09-21T11:43:39.368222+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:26.257967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.257967+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:51.865812+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":178,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350128+0000","last_change":"2024-09-21T11:41:32.885455+0000","last_active":"2024-09-21T11:43:39.350128+0000","last_peered":"2024-09-21T11:43:39.350128+0000","last_clean":"2024-09-21T11:43:39.350128+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:43:39.350128+0000","last_undegraded":"2024-09-21T11:43:39.350128+0000","last_fullsized":"2024-09-21T11:43:39.350128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:32.885407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:32.885407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":173,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.365029+0000","last_change":"2024-09-21T11:42:37.156247+0000","last_active":"2024-09-21T11:43:39.365029+0000","last_peered":"2024-09-21T11:43:39.365029+0000","last_clean":"2024-09-21T11:43:39.365029+0000","last_became_active":"2024-09-21T11:42:37.155879+0000","last_became_peered":"2024-09-21T11:42:37.155879+0000","last_unstale":"2024-09-21T11:43:39.365029+0000","last_undegraded":"2024-09-21T11:43:39.365029+0000","last_fullsized":"2024-09-21T11:43:39.365029+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":201,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350494+0000","last_change":"2024-09-21T11:42:37.156936+0000","last_active":"2024-09-21T11:43:39.350494+0000","last_peered":"2024-09-21T11:43:39.350494+0000","last_clean":"2024-09-21T11:43:39.350494+0000","last_became_active":"2024-09-21T11:42:37.156575+0000","last_became_peered":"2024-09-21T11:42:37.156575+0000","last_unstale":"2024-09-21T11:43:39.350494+0000","last_undegraded":"2024-09-21T11:43:39.350494+0000","last_fullsized":"2024-09-21T11:43:39.350494+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.101955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.101955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":214,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368425+0000","last_change":"2024-09-21T11:43:08.326923+0000","last_active":"2024-09-21T11:43:39.368425+0000","last_peered":"2024-09-21T11:43:39.368425+0000","last_clean":"2024-09-21T11:43:39.368425+0000","last_became_active":"2024-09-21T11:41:31.634863+0000","last_became_peered":"2024-09-21T11:41:31.634863+0000","last_unstale":"2024-09-21T11:43:39.368425+0000","last_undegraded":"2024-09-21T11:43:39.368425+0000","last_fullsized":"2024-09-21T11:43:39.368425+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:08.326846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:08.326846+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:14.001930+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":185,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362021+0000","last_change":"2024-09-21T11:43:38.567194+0000","last_active":"2024-09-21T11:43:39.362021+0000","last_peered":"2024-09-21T11:43:39.362021+0000","last_clean":"2024-09-21T11:43:39.362021+0000","last_became_active":"2024-09-21T11:42:37.164285+0000","last_became_peered":"2024-09-21T11:42:37.164285+0000","last_unstale":"2024-09-21T11:43:39.362021+0000","last_undegraded":"2024-09-21T11:43:39.362021+0000","last_fullsized":"2024-09-21T11:43:39.362021+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.567131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.567131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.325966+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":181,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.349518+0000","last_change":"2024-09-21T11:42:37.163213+0000","last_active":"2024-09-21T11:43:39.349518+0000","last_peered":"2024-09-21T11:43:39.349518+0000","last_clean":"2024-09-21T11:43:39.349518+0000","last_became_active":"2024-09-21T11:42:37.162912+0000","last_became_peered":"2024-09-21T11:42:37.162912+0000","last_unstale":"2024-09-21T11:43:39.349518+0000","last_undegraded":"2024-09-21T11:43:39.349518+0000","last_fullsized":"2024-09-21T11:43:39.349518+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:12.028860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:12.028860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:40.276311+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":183,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368656+0000","last_change":"2024-09-21T11:43:22.300966+0000","last_active":"2024-09-21T11:43:39.368656+0000","last_peered":"2024-09-21T11:43:39.368656+0000","last_clean":"2024-09-21T11:43:39.368656+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:43:39.368656+0000","last_undegraded":"2024-09-21T11:43:39.368656+0000","last_fullsized":"2024-09-21T11:43:39.368656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:22.300885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:22.300885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.343801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.366750+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:43:39.366750+0000","last_peered":"2024-09-21T11:43:39.366750+0000","last_clean":"2024-09-21T11:43:39.366750+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:43:39.366750+0000","last_undegraded":"2024-09-21T11:43:39.366750+0000","last_fullsized":"2024-09-21T11:43:39.366750+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:50.720157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":262,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.378914+0000","last_change":"2024-09-21T11:43:31.498356+0000","last_active":"2024-09-21T11:43:39.378914+0000","last_peered":"2024-09-21T11:43:39.378914+0000","last_clean":"2024-09-21T11:43:39.378914+0000","last_became_active":"2024-09-21T11:42:37.166901+0000","last_became_peered":"2024-09-21T11:42:37.166901+0000","last_unstale":"2024-09-21T11:43:39.378914+0000","last_undegraded":"2024-09-21T11:43:39.378914+0000","last_fullsized":"2024-09-21T11:43:39.378914+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:31.498142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.498142+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:33.717651+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":72,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368556+0000","last_change":"2024-09-21T11:42:37.154086+0000","last_active":"2024-09-21T11:43:39.368556+0000","last_peered":"2024-09-21T11:43:39.368556+0000","last_clean":"2024-09-21T11:43:39.368556+0000","last_became_active":"2024-09-21T11:42:37.153799+0000","last_became_peered":"2024-09-21T11:42:37.153799+0000","last_unstale":"2024-09-21T11:43:39.368556+0000","last_undegraded":"2024-09-21T11:43:39.368556+0000","last_fullsized":"2024-09-21T11:43:39.368556+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:20.335521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:20.335521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":175,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.363298+0000","last_change":"2024-09-21T11:41:40.448106+0000","last_active":"2024-09-21T11:43:39.363298+0000","last_peered":"2024-09-21T11:43:39.363298+0000","last_clean":"2024-09-21T11:43:39.363298+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:43:39.363298+0000","last_undegraded":"2024-09-21T11:43:39.363298+0000","last_fullsized":"2024-09-21T11:43:39.363298+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.447908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.447908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368312+0000","last_change":"2024-09-21T11:42:37.154002+0000","last_active":"2024-09-21T11:43:39.368312+0000","last_peered":"2024-09-21T11:43:39.368312+0000","last_clean":"2024-09-21T11:43:39.368312+0000","last_became_active":"2024-09-21T11:42:37.153694+0000","last_became_peered":"2024-09-21T11:42:37.153694+0000","last_unstale":"2024-09-21T11:43:39.368312+0000","last_undegraded":"2024-09-21T11:43:39.368312+0000","last_fullsized":"2024-09-21T11:43:39.368312+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:43.155873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":225,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.363218+0000","last_change":"2024-09-21T11:42:37.189830+0000","last_active":"2024-09-21T11:43:39.363218+0000","last_peered":"2024-09-21T11:43:39.363218+0000","last_clean":"2024-09-21T11:43:39.363218+0000","last_became_active":"2024-09-21T11:42:37.165964+0000","last_became_peered":"2024-09-21T11:42:37.165964+0000","last_unstale":"2024-09-21T11:43:39.363218+0000","last_undegraded":"2024-09-21T11:43:39.363218+0000","last_fullsized":"2024-09-21T11:43:39.363218+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:41:35.427210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:35.427210+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":179,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.369821+0000","last_change":"2024-09-21T11:43:02.320628+0000","last_active":"2024-09-21T11:43:39.369821+0000","last_peered":"2024-09-21T11:43:39.369821+0000","last_clean":"2024-09-21T11:43:39.369821+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:43:39.369821+0000","last_undegraded":"2024-09-21T11:43:39.369821+0000","last_fullsized":"2024-09-21T11:43:39.369821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:02.320559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:02.320559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:24.560814+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":183,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362212+0000","last_change":"2024-09-21T11:42:31.278897+0000","last_active":"2024-09-21T11:43:39.362212+0000","last_peered":"2024-09-21T11:43:39.362212+0000","last_clean":"2024-09-21T11:43:39.362212+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:43:39.362212+0000","last_undegraded":"2024-09-21T11:43:39.362212+0000","last_fullsized":"2024-09-21T11:43:39.362212+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:42:31.278824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:31.278824+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":84,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.360728+0000","last_change":"2024-09-21T11:42:37.185608+0000","last_active":"2024-09-21T11:43:39.360728+0000","last_peered":"2024-09-21T11:43:39.360728+0000","last_clean":"2024-09-21T11:43:39.360728+0000","last_became_active":"2024-09-21T11:42:37.166144+0000","last_became_peered":"2024-09-21T11:42:37.166144+0000","last_unstale":"2024-09-21T11:43:39.360728+0000","last_undegraded":"2024-09-21T11:43:39.360728+0000","last_fullsized":"2024-09-21T11:43:39.360728+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:23.349301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.349301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:45.201792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":197,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.367603+0000","last_change":"2024-09-21T11:43:29.595122+0000","last_active":"2024-09-21T11:43:39.367603+0000","last_peered":"2024-09-21T11:43:39.367603+0000","last_clean":"2024-09-21T11:43:39.367603+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:43:39.367603+0000","last_undegraded":"2024-09-21T11:43:39.367603+0000","last_fullsized":"2024-09-21T11:43:39.367603+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:43:29.595040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.595040+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:41.200221+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":242,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.356295+0000","last_change":"2024-09-21T11:42:14.814180+0000","last_active":"2024-09-21T11:43:39.356295+0000","last_peered":"2024-09-21T11:43:39.356295+0000","last_clean":"2024-09-21T11:43:39.356295+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:43:39.356295+0000","last_undegraded":"2024-09-21T11:43:39.356295+0000","last_fullsized":"2024-09-21T11:43:39.356295+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:14.814041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:14.814041+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"142'10328","reported_seq":10517,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.732640+0000","last_change":"2024-09-21T11:43:35.987574+0000","last_active":"2024-09-21T11:43:40.732640+0000","last_peered":"2024-09-21T11:43:40.732640+0000","last_clean":"2024-09-21T11:43:40.732640+0000","last_became_active":"2024-09-21T11:41:10.304754+0000","last_became_peered":"2024-09-21T11:41:10.304754+0000","last_unstale":"2024-09-21T11:43:40.732640+0000","last_undegraded":"2024-09-21T11:43:40.732640+0000","last_fullsized":"2024-09-21T11:43:40.732640+0000","mapping_epoch":82,"log_start":"102'5346","ondisk_log_start":"102'5346","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'9957","last_scrub_stamp":"2024-09-21T11:43:35.987394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:35.987394+0000","objects_scrubbed":4941,"log_size":4982,"log_dups_size":0,"ondisk_log_size":4982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:47.433252+0000","scrub_duration":1245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":296878080,"num_objects":4530,"num_object_clones":0,"num_object_copies":13590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10328,"num_write_kb":475456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":132,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.363483+0000","last_change":"2024-09-21T11:42:37.202244+0000","last_active":"2024-09-21T11:43:39.363483+0000","last_peered":"2024-09-21T11:43:39.363483+0000","last_clean":"2024-09-21T11:43:39.363483+0000","last_became_active":"2024-09-21T11:42:37.169939+0000","last_became_peered":"2024-09-21T11:42:37.169939+0000","last_unstale":"2024-09-21T11:43:39.363483+0000","last_undegraded":"2024-09-21T11:43:39.363483+0000","last_fullsized":"2024-09-21T11:43:39.363483+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":194,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361913+0000","last_change":"2024-09-21T11:42:37.172945+0000","last_active":"2024-09-21T11:43:39.361913+0000","last_peered":"2024-09-21T11:43:39.361913+0000","last_clean":"2024-09-21T11:43:39.361913+0000","last_became_active":"2024-09-21T11:42:37.171365+0000","last_became_peered":"2024-09-21T11:42:37.171365+0000","last_unstale":"2024-09-21T11:43:39.361913+0000","last_undegraded":"2024-09-21T11:43:39.361913+0000","last_fullsized":"2024-09-21T11:43:39.361913+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.454207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.454207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":232,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368749+0000","last_change":"2024-09-21T11:42:26.008760+0000","last_active":"2024-09-21T11:43:39.368749+0000","last_peered":"2024-09-21T11:43:39.368749+0000","last_clean":"2024-09-21T11:43:39.368749+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:43:39.368749+0000","last_undegraded":"2024-09-21T11:43:39.368749+0000","last_fullsized":"2024-09-21T11:43:39.368749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:26.008646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:26.008646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:48.227954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":181,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350204+0000","last_change":"2024-09-21T11:42:37.157278+0000","last_active":"2024-09-21T11:43:39.350204+0000","last_peered":"2024-09-21T11:43:39.350204+0000","last_clean":"2024-09-21T11:43:39.350204+0000","last_became_active":"2024-09-21T11:42:37.156952+0000","last_became_peered":"2024-09-21T11:42:37.156952+0000","last_unstale":"2024-09-21T11:43:39.350204+0000","last_undegraded":"2024-09-21T11:43:39.350204+0000","last_fullsized":"2024-09-21T11:43:39.350204+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.066324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.066324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"142'10180","reported_seq":10312,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.082791+0000","last_change":"2024-09-21T11:41:10.297546+0000","last_active":"2024-09-21T11:43:41.082791+0000","last_peered":"2024-09-21T11:43:41.082791+0000","last_clean":"2024-09-21T11:43:41.082791+0000","last_became_active":"2024-09-21T11:41:10.297301+0000","last_became_peered":"2024-09-21T11:41:10.297301+0000","last_unstale":"2024-09-21T11:43:41.082791+0000","last_undegraded":"2024-09-21T11:43:41.082791+0000","last_fullsized":"2024-09-21T11:43:41.082791+0000","mapping_epoch":82,"log_start":"106'5951","ondisk_log_start":"106'5951","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4229,"log_dups_size":0,"ondisk_log_size":4229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":284688384,"num_objects":4344,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10180,"num_write_kb":464768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":215,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.423834+0000","last_change":"2024-09-21T11:43:40.423834+0000","last_active":"2024-09-21T11:43:40.423834+0000","last_peered":"2024-09-21T11:43:40.423834+0000","last_clean":"2024-09-21T11:43:40.423834+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:43:40.423834+0000","last_undegraded":"2024-09-21T11:43:40.423834+0000","last_fullsized":"2024-09-21T11:43:40.423834+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:43:40.423750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.423750+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:01.136315+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":187,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.363737+0000","last_change":"2024-09-21T11:41:29.388381+0000","last_active":"2024-09-21T11:43:39.363737+0000","last_peered":"2024-09-21T11:43:39.363737+0000","last_clean":"2024-09-21T11:43:39.363737+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:43:39.363737+0000","last_undegraded":"2024-09-21T11:43:39.363737+0000","last_fullsized":"2024-09-21T11:43:39.363737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:29.388322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:29.388322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":179,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368098+0000","last_change":"2024-09-21T11:41:40.030553+0000","last_active":"2024-09-21T11:43:39.368098+0000","last_peered":"2024-09-21T11:43:39.368098+0000","last_clean":"2024-09-21T11:43:39.368098+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:43:39.368098+0000","last_undegraded":"2024-09-21T11:43:39.368098+0000","last_fullsized":"2024-09-21T11:43:39.368098+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.030493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.030493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:11.327373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":167,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368193+0000","last_change":"2024-09-21T11:40:06.343742+0000","last_active":"2024-09-21T11:43:39.368193+0000","last_peered":"2024-09-21T11:43:39.368193+0000","last_clean":"2024-09-21T11:43:39.368193+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:43:39.368193+0000","last_undegraded":"2024-09-21T11:43:39.368193+0000","last_fullsized":"2024-09-21T11:43:39.368193+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:10.356658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"142'10090","reported_seq":22831,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.068079+0000","last_change":"2024-09-21T11:43:25.943980+0000","last_active":"2024-09-21T11:43:41.068079+0000","last_peered":"2024-09-21T11:43:41.068079+0000","last_clean":"2024-09-21T11:43:41.068079+0000","last_became_active":"2024-09-21T11:43:25.930319+0000","last_became_peered":"2024-09-21T11:43:25.930319+0000","last_unstale":"2024-09-21T11:43:41.068079+0000","last_undegraded":"2024-09-21T11:43:41.068079+0000","last_fullsized":"2024-09-21T11:43:41.068079+0000","mapping_epoch":133,"log_start":"106'5814","ondisk_log_start":"106'5814","created":82,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4276,"log_dups_size":0,"ondisk_log_size":4276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":284557312,"num_objects":4342,"num_object_clones":0,"num_object_copies":13026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10090,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6296,"num_bytes_recovered":412614656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":204,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368221+0000","last_change":"2024-09-21T11:41:41.007761+0000","last_active":"2024-09-21T11:43:39.368221+0000","last_peered":"2024-09-21T11:43:39.368221+0000","last_clean":"2024-09-21T11:43:39.368221+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:43:39.368221+0000","last_undegraded":"2024-09-21T11:43:39.368221+0000","last_fullsized":"2024-09-21T11:43:39.368221+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:41:41.007695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:41.007695+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:15.335018+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":198,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.367847+0000","last_change":"2024-09-21T11:42:37.182056+0000","last_active":"2024-09-21T11:43:39.367847+0000","last_peered":"2024-09-21T11:43:39.367847+0000","last_clean":"2024-09-21T11:43:39.367847+0000","last_became_active":"2024-09-21T11:42:37.159390+0000","last_became_peered":"2024-09-21T11:42:37.159390+0000","last_unstale":"2024-09-21T11:43:39.367847+0000","last_undegraded":"2024-09-21T11:43:39.367847+0000","last_fullsized":"2024-09-21T11:43:39.367847+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:41:34.236667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:34.236667+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"114'9","reported_seq":244,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361478+0000","last_change":"2024-09-21T11:43:29.417021+0000","last_active":"2024-09-21T11:43:39.361478+0000","last_peered":"2024-09-21T11:43:39.361478+0000","last_clean":"2024-09-21T11:43:39.361478+0000","last_became_active":"2024-09-21T11:42:37.169838+0000","last_became_peered":"2024-09-21T11:42:37.169838+0000","last_unstale":"2024-09-21T11:43:39.361478+0000","last_undegraded":"2024-09-21T11:43:39.361478+0000","last_fullsized":"2024-09-21T11:43:39.361478+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:29.416946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.416946+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:30.090032+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":177,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361939+0000","last_change":"2024-09-21T11:41:39.527210+0000","last_active":"2024-09-21T11:43:39.361939+0000","last_peered":"2024-09-21T11:43:39.361939+0000","last_clean":"2024-09-21T11:43:39.361939+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:43:39.361939+0000","last_undegraded":"2024-09-21T11:43:39.361939+0000","last_fullsized":"2024-09-21T11:43:39.361939+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.527151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.527151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"142'10192","reported_seq":10398,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.687595+0000","last_change":"2024-09-21T11:43:05.145633+0000","last_active":"2024-09-21T11:43:40.687595+0000","last_peered":"2024-09-21T11:43:40.687595+0000","last_clean":"2024-09-21T11:43:40.687595+0000","last_became_active":"2024-09-21T11:41:10.300541+0000","last_became_peered":"2024-09-21T11:41:10.300541+0000","last_unstale":"2024-09-21T11:43:40.687595+0000","last_undegraded":"2024-09-21T11:43:40.687595+0000","last_fullsized":"2024-09-21T11:43:40.687595+0000","mapping_epoch":82,"log_start":"103'5633","ondisk_log_start":"103'5633","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'8200","last_scrub_stamp":"2024-09-21T11:43:05.145547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:05.145547+0000","objects_scrubbed":6512,"log_size":4559,"log_dups_size":0,"ondisk_log_size":4559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.858838+0000","scrub_duration":1824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":292945920,"num_objects":4470,"num_object_clones":0,"num_object_copies":13410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10192,"num_write_kb":469184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":118,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.349973+0000","last_change":"2024-09-21T11:42:37.207015+0000","last_active":"2024-09-21T11:43:39.349973+0000","last_peered":"2024-09-21T11:43:39.349973+0000","last_clean":"2024-09-21T11:43:39.349973+0000","last_became_active":"2024-09-21T11:42:37.165120+0000","last_became_peered":"2024-09-21T11:42:37.165120+0000","last_unstale":"2024-09-21T11:43:39.349973+0000","last_undegraded":"2024-09-21T11:43:39.349973+0000","last_fullsized":"2024-09-21T11:43:39.349973+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:15.360632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.360632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:42.919793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350536+0000","last_change":"2024-09-21T11:41:40.866256+0000","last_active":"2024-09-21T11:43:39.350536+0000","last_peered":"2024-09-21T11:43:39.350536+0000","last_clean":"2024-09-21T11:43:39.350536+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:43:39.350536+0000","last_undegraded":"2024-09-21T11:43:39.350536+0000","last_fullsized":"2024-09-21T11:43:39.350536+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.866156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.866156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":234,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.366419+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:43:39.366419+0000","last_peered":"2024-09-21T11:43:39.366419+0000","last_clean":"2024-09-21T11:43:39.366419+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:43:39.366419+0000","last_undegraded":"2024-09-21T11:43:39.366419+0000","last_fullsized":"2024-09-21T11:43:39.366419+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:44.871068+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":174,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.365155+0000","last_change":"2024-09-21T11:41:40.278404+0000","last_active":"2024-09-21T11:43:39.365155+0000","last_peered":"2024-09-21T11:43:39.365155+0000","last_clean":"2024-09-21T11:43:39.365155+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:43:39.365155+0000","last_undegraded":"2024-09-21T11:43:39.365155+0000","last_fullsized":"2024-09-21T11:43:39.365155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.278315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.278315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"142'10189","reported_seq":23028,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.729457+0000","last_change":"2024-09-21T11:43:27.957009+0000","last_active":"2024-09-21T11:43:40.729457+0000","last_peered":"2024-09-21T11:43:40.729457+0000","last_clean":"2024-09-21T11:43:40.729457+0000","last_became_active":"2024-09-21T11:43:27.938453+0000","last_became_peered":"2024-09-21T11:43:27.938453+0000","last_unstale":"2024-09-21T11:43:40.729457+0000","last_undegraded":"2024-09-21T11:43:40.729457+0000","last_fullsized":"2024-09-21T11:43:40.729457+0000","mapping_epoch":135,"log_start":"100'4964","ondisk_log_start":"100'4964","created":82,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":5225,"log_dups_size":0,"ondisk_log_size":5225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":296550400,"num_objects":4525,"num_object_clones":0,"num_object_copies":13575,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10189,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6345,"num_bytes_recovered":415825920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":218,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.366511+0000","last_change":"2024-09-21T11:41:07.065583+0000","last_active":"2024-09-21T11:43:39.366511+0000","last_peered":"2024-09-21T11:43:39.366511+0000","last_clean":"2024-09-21T11:43:39.366511+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:43:39.366511+0000","last_undegraded":"2024-09-21T11:43:39.366511+0000","last_fullsized":"2024-09-21T11:43:39.366511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:41:07.065507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.065507+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":79,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.378832+0000","last_change":"2024-09-21T11:43:30.509571+0000","last_active":"2024-09-21T11:43:39.378832+0000","last_peered":"2024-09-21T11:43:39.378832+0000","last_clean":"2024-09-21T11:43:39.378832+0000","last_became_active":"2024-09-21T11:42:37.157562+0000","last_became_peered":"2024-09-21T11:42:37.157562+0000","last_unstale":"2024-09-21T11:43:39.378832+0000","last_undegraded":"2024-09-21T11:43:39.378832+0000","last_fullsized":"2024-09-21T11:43:39.378832+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.509505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.509505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:42.953900+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":178,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.355640+0000","last_change":"2024-09-21T11:43:26.704117+0000","last_active":"2024-09-21T11:43:39.355640+0000","last_peered":"2024-09-21T11:43:39.355640+0000","last_clean":"2024-09-21T11:43:39.355640+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:43:39.355640+0000","last_undegraded":"2024-09-21T11:43:39.355640+0000","last_fullsized":"2024-09-21T11:43:39.355640+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.704073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.704073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:55.097023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":79,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.355635+0000","last_change":"2024-09-21T11:43:37.819912+0000","last_active":"2024-09-21T11:43:39.355635+0000","last_peered":"2024-09-21T11:43:39.355635+0000","last_clean":"2024-09-21T11:43:39.355635+0000","last_became_active":"2024-09-21T11:42:37.155437+0000","last_became_peered":"2024-09-21T11:42:37.155437+0000","last_unstale":"2024-09-21T11:43:39.355635+0000","last_undegraded":"2024-09-21T11:43:39.355635+0000","last_fullsized":"2024-09-21T11:43:39.355635+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.544593+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"142'10489","reported_seq":10689,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.713420+0000","last_change":"2024-09-21T11:42:22.352124+0000","last_active":"2024-09-21T11:43:40.713420+0000","last_peered":"2024-09-21T11:43:40.713420+0000","last_clean":"2024-09-21T11:43:40.713420+0000","last_became_active":"2024-09-21T11:41:10.304723+0000","last_became_peered":"2024-09-21T11:41:10.304723+0000","last_unstale":"2024-09-21T11:43:40.713420+0000","last_undegraded":"2024-09-21T11:43:40.713420+0000","last_fullsized":"2024-09-21T11:43:40.713420+0000","mapping_epoch":82,"log_start":"102'5563","ondisk_log_start":"102'5563","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'6088","last_scrub_stamp":"2024-09-21T11:42:22.351940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.351940+0000","objects_scrubbed":6032,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:42.753165+0000","scrub_duration":1670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293011456,"num_objects":4471,"num_object_clones":0,"num_object_copies":13413,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10489,"num_write_kb":478720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"140'30","reported_seq":214,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.378222+0000","last_change":"2024-09-21T11:43:37.416987+0000","last_active":"2024-09-21T11:43:39.378222+0000","last_peered":"2024-09-21T11:43:39.378222+0000","last_clean":"2024-09-21T11:43:39.378222+0000","last_became_active":"2024-09-21T11:43:32.003036+0000","last_became_peered":"2024-09-21T11:43:32.003036+0000","last_unstale":"2024-09-21T11:43:39.378222+0000","last_undegraded":"2024-09-21T11:43:39.378222+0000","last_fullsized":"2024-09-21T11:43:39.378222+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.678954+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":193,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368607+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:43:39.368607+0000","last_peered":"2024-09-21T11:43:39.368607+0000","last_clean":"2024-09-21T11:43:39.368607+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:43:39.368607+0000","last_undegraded":"2024-09-21T11:43:39.368607+0000","last_fullsized":"2024-09-21T11:43:39.368607+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:14.022118+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":207,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350125+0000","last_change":"2024-09-21T11:42:37.193243+0000","last_active":"2024-09-21T11:43:39.350125+0000","last_peered":"2024-09-21T11:43:39.350125+0000","last_clean":"2024-09-21T11:43:39.350125+0000","last_became_active":"2024-09-21T11:42:37.164723+0000","last_became_peered":"2024-09-21T11:42:37.164723+0000","last_unstale":"2024-09-21T11:43:39.350125+0000","last_undegraded":"2024-09-21T11:43:39.350125+0000","last_fullsized":"2024-09-21T11:43:39.350125+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:42:17.142454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.142454+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":185,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.369710+0000","last_change":"2024-09-21T11:42:37.164162+0000","last_active":"2024-09-21T11:43:39.369710+0000","last_peered":"2024-09-21T11:43:39.369710+0000","last_clean":"2024-09-21T11:43:39.369710+0000","last_became_active":"2024-09-21T11:42:37.163682+0000","last_became_peered":"2024-09-21T11:42:37.163682+0000","last_unstale":"2024-09-21T11:43:39.369710+0000","last_undegraded":"2024-09-21T11:43:39.369710+0000","last_fullsized":"2024-09-21T11:43:39.369710+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:24.468909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.468909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.266739+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":175,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.359614+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:43:39.359614+0000","last_peered":"2024-09-21T11:43:39.359614+0000","last_clean":"2024-09-21T11:43:39.359614+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:43:39.359614+0000","last_undegraded":"2024-09-21T11:43:39.359614+0000","last_fullsized":"2024-09-21T11:43:39.359614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:50.884536+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"142'10101","reported_seq":23204,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.209901+0000","last_change":"2024-09-21T11:43:25.018476+0000","last_active":"2024-09-21T11:43:41.209901+0000","last_peered":"2024-09-21T11:43:41.209901+0000","last_clean":"2024-09-21T11:43:41.209901+0000","last_became_active":"2024-09-21T11:43:24.947584+0000","last_became_peered":"2024-09-21T11:43:24.947584+0000","last_unstale":"2024-09-21T11:43:41.209901+0000","last_undegraded":"2024-09-21T11:43:41.209901+0000","last_fullsized":"2024-09-21T11:43:41.209901+0000","mapping_epoch":132,"log_start":"104'5520","ondisk_log_start":"104'5520","created":82,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4581,"log_dups_size":0,"ondisk_log_size":4581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":292093952,"num_objects":4457,"num_object_clones":0,"num_object_copies":13371,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":465856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6476,"num_bytes_recovered":424280064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":130,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350669+0000","last_change":"2024-09-21T11:42:37.188687+0000","last_active":"2024-09-21T11:43:39.350669+0000","last_peered":"2024-09-21T11:43:39.350669+0000","last_clean":"2024-09-21T11:43:39.350669+0000","last_became_active":"2024-09-21T11:42:37.161797+0000","last_became_peered":"2024-09-21T11:42:37.161797+0000","last_unstale":"2024-09-21T11:43:39.350669+0000","last_undegraded":"2024-09-21T11:43:39.350669+0000","last_fullsized":"2024-09-21T11:43:39.350669+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":84,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.377376+0000","last_change":"2024-09-21T11:42:37.182044+0000","last_active":"2024-09-21T11:43:39.377376+0000","last_peered":"2024-09-21T11:43:39.377376+0000","last_clean":"2024-09-21T11:43:39.377376+0000","last_became_active":"2024-09-21T11:42:37.156191+0000","last_became_peered":"2024-09-21T11:42:37.156191+0000","last_unstale":"2024-09-21T11:43:39.377376+0000","last_undegraded":"2024-09-21T11:43:39.377376+0000","last_fullsized":"2024-09-21T11:43:39.377376+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:52.863101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":204,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.377019+0000","last_change":"2024-09-21T11:43:36.446391+0000","last_active":"2024-09-21T11:43:39.377019+0000","last_peered":"2024-09-21T11:43:39.377019+0000","last_clean":"2024-09-21T11:43:39.377019+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:43:39.377019+0000","last_undegraded":"2024-09-21T11:43:39.377019+0000","last_fullsized":"2024-09-21T11:43:39.377019+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:43:36.446341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.446341+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.312398+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":187,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.379298+0000","last_change":"2024-09-21T11:43:28.505134+0000","last_active":"2024-09-21T11:43:39.379298+0000","last_peered":"2024-09-21T11:43:39.379298+0000","last_clean":"2024-09-21T11:43:39.379298+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:43:39.379298+0000","last_undegraded":"2024-09-21T11:43:39.379298+0000","last_fullsized":"2024-09-21T11:43:39.379298+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.504894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.504894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.779358+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":174,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362143+0000","last_change":"2024-09-21T11:42:25.245624+0000","last_active":"2024-09-21T11:43:39.362143+0000","last_peered":"2024-09-21T11:43:39.362143+0000","last_clean":"2024-09-21T11:43:39.362143+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:43:39.362143+0000","last_undegraded":"2024-09-21T11:43:39.362143+0000","last_fullsized":"2024-09-21T11:43:39.362143+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.245565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.245565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"142'10342","reported_seq":10474,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.680752+0000","last_change":"2024-09-21T11:41:10.303217+0000","last_active":"2024-09-21T11:43:40.680752+0000","last_peered":"2024-09-21T11:43:40.680752+0000","last_clean":"2024-09-21T11:43:40.680752+0000","last_became_active":"2024-09-21T11:41:10.302903+0000","last_became_peered":"2024-09-21T11:41:10.302903+0000","last_unstale":"2024-09-21T11:43:40.680752+0000","last_undegraded":"2024-09-21T11:43:40.680752+0000","last_fullsized":"2024-09-21T11:43:40.680752+0000","mapping_epoch":82,"log_start":"105'5828","ondisk_log_start":"105'5828","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":293339136,"num_objects":4476,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10342,"num_write_kb":474176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":221,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.607996+0000","last_change":"2024-09-21T11:43:40.607996+0000","last_active":"2024-09-21T11:43:40.607996+0000","last_peered":"2024-09-21T11:43:40.607996+0000","last_clean":"2024-09-21T11:43:40.607996+0000","last_became_active":"2024-09-21T11:42:37.164059+0000","last_became_peered":"2024-09-21T11:42:37.164059+0000","last_unstale":"2024-09-21T11:43:40.607996+0000","last_undegraded":"2024-09-21T11:43:40.607996+0000","last_fullsized":"2024-09-21T11:43:40.607996+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:43:40.607801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.607801+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:59.037561+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":178,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.365552+0000","last_change":"2024-09-21T11:40:00.312222+0000","last_active":"2024-09-21T11:43:39.365552+0000","last_peered":"2024-09-21T11:43:39.365552+0000","last_clean":"2024-09-21T11:43:39.365552+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:43:39.365552+0000","last_undegraded":"2024-09-21T11:43:39.365552+0000","last_fullsized":"2024-09-21T11:43:39.365552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":423,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361885+0000","last_change":"2024-09-21T11:42:37.176176+0000","last_active":"2024-09-21T11:43:39.361885+0000","last_peered":"2024-09-21T11:43:39.361885+0000","last_clean":"2024-09-21T11:43:39.361885+0000","last_became_active":"2024-09-21T11:42:37.166141+0000","last_became_peered":"2024-09-21T11:42:37.166141+0000","last_unstale":"2024-09-21T11:43:39.361885+0000","last_undegraded":"2024-09-21T11:43:39.361885+0000","last_fullsized":"2024-09-21T11:43:39.361885+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:41:36.501543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.501543+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":214,"num_read_kb":209,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":177,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350867+0000","last_change":"2024-09-21T11:42:37.173054+0000","last_active":"2024-09-21T11:43:39.350867+0000","last_peered":"2024-09-21T11:43:39.350867+0000","last_clean":"2024-09-21T11:43:39.350867+0000","last_became_active":"2024-09-21T11:42:37.172270+0000","last_became_peered":"2024-09-21T11:42:37.172270+0000","last_unstale":"2024-09-21T11:43:39.350867+0000","last_undegraded":"2024-09-21T11:43:39.350867+0000","last_fullsized":"2024-09-21T11:43:39.350867+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":71,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350472+0000","last_change":"2024-09-21T11:42:37.173082+0000","last_active":"2024-09-21T11:43:39.350472+0000","last_peered":"2024-09-21T11:43:39.350472+0000","last_clean":"2024-09-21T11:43:39.350472+0000","last_became_active":"2024-09-21T11:42:37.172645+0000","last_became_peered":"2024-09-21T11:42:37.172645+0000","last_unstale":"2024-09-21T11:43:39.350472+0000","last_undegraded":"2024-09-21T11:43:39.350472+0000","last_fullsized":"2024-09-21T11:43:39.350472+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:31.250421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:31.250421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"142'10425","reported_seq":10605,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.203334+0000","last_change":"2024-09-21T11:42:31.712433+0000","last_active":"2024-09-21T11:43:41.203334+0000","last_peered":"2024-09-21T11:43:41.203334+0000","last_clean":"2024-09-21T11:43:41.203334+0000","last_became_active":"2024-09-21T11:41:10.302497+0000","last_became_peered":"2024-09-21T11:41:10.302497+0000","last_unstale":"2024-09-21T11:43:41.203334+0000","last_undegraded":"2024-09-21T11:43:41.203334+0000","last_fullsized":"2024-09-21T11:43:41.203334+0000","mapping_epoch":82,"log_start":"105'5842","ondisk_log_start":"105'5842","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":4349,"log_size":4583,"log_dups_size":0,"ondisk_log_size":4583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":292356096,"num_objects":4461,"num_object_clones":0,"num_object_copies":13383,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10425,"num_write_kb":476352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":249,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.356743+0000","last_change":"2024-09-21T11:42:19.703071+0000","last_active":"2024-09-21T11:43:39.356743+0000","last_peered":"2024-09-21T11:43:39.356743+0000","last_clean":"2024-09-21T11:43:39.356743+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:43:39.356743+0000","last_undegraded":"2024-09-21T11:43:39.356743+0000","last_fullsized":"2024-09-21T11:43:39.356743+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:42:19.703007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.703007+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:46.340331+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":187,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.360758+0000","last_change":"2024-09-21T11:43:30.437283+0000","last_active":"2024-09-21T11:43:39.360758+0000","last_peered":"2024-09-21T11:43:39.360758+0000","last_clean":"2024-09-21T11:43:39.360758+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:43:39.360758+0000","last_undegraded":"2024-09-21T11:43:39.360758+0000","last_fullsized":"2024-09-21T11:43:39.360758+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.437080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.437080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.933810+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":231,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362084+0000","last_change":"2024-09-21T11:42:24.254810+0000","last_active":"2024-09-21T11:43:39.362084+0000","last_peered":"2024-09-21T11:43:39.362084+0000","last_clean":"2024-09-21T11:43:39.362084+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:43:39.362084+0000","last_undegraded":"2024-09-21T11:43:39.362084+0000","last_fullsized":"2024-09-21T11:43:39.362084+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:42:24.254749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.254749+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":178,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362309+0000","last_change":"2024-09-21T11:43:31.630737+0000","last_active":"2024-09-21T11:43:39.362309+0000","last_peered":"2024-09-21T11:43:39.362309+0000","last_clean":"2024-09-21T11:43:39.362309+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:43:39.362309+0000","last_undegraded":"2024-09-21T11:43:39.362309+0000","last_fullsized":"2024-09-21T11:43:39.362309+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:31.630670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.630670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.782053+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":174,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350793+0000","last_change":"2024-09-21T11:41:44.929146+0000","last_active":"2024-09-21T11:43:39.350793+0000","last_peered":"2024-09-21T11:43:39.350793+0000","last_clean":"2024-09-21T11:43:39.350793+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:43:39.350793+0000","last_undegraded":"2024-09-21T11:43:39.350793+0000","last_fullsized":"2024-09-21T11:43:39.350793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:44.929079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:44.929079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"142'10271","reported_seq":23362,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.706318+0000","last_change":"2024-09-21T11:43:26.996320+0000","last_active":"2024-09-21T11:43:40.706318+0000","last_peered":"2024-09-21T11:43:40.706318+0000","last_clean":"2024-09-21T11:43:40.706318+0000","last_became_active":"2024-09-21T11:43:26.984538+0000","last_became_peered":"2024-09-21T11:43:26.984538+0000","last_unstale":"2024-09-21T11:43:40.706318+0000","last_undegraded":"2024-09-21T11:43:40.706318+0000","last_fullsized":"2024-09-21T11:43:40.706318+0000","mapping_epoch":134,"log_start":"101'5264","ondisk_log_start":"101'5264","created":82,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'5645","last_scrub_stamp":"2024-09-21T11:42:17.339535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.339535+0000","objects_scrubbed":5593,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":290979864,"num_objects":4441,"num_object_clones":0,"num_object_copies":13323,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4441,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10271,"num_write_kb":470721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6438,"num_bytes_recovered":421855256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":221,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368735+0000","last_change":"2024-09-21T11:42:37.179956+0000","last_active":"2024-09-21T11:43:39.368735+0000","last_peered":"2024-09-21T11:43:39.368735+0000","last_clean":"2024-09-21T11:43:39.368735+0000","last_became_active":"2024-09-21T11:42:37.158158+0000","last_became_peered":"2024-09-21T11:42:37.158158+0000","last_unstale":"2024-09-21T11:43:39.368735+0000","last_undegraded":"2024-09-21T11:43:39.368735+0000","last_fullsized":"2024-09-21T11:43:39.368735+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:22.041174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.041174+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:48.441691+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":187,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368278+0000","last_change":"2024-09-21T11:42:58.177899+0000","last_active":"2024-09-21T11:43:39.368278+0000","last_peered":"2024-09-21T11:43:39.368278+0000","last_clean":"2024-09-21T11:43:39.368278+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:43:39.368278+0000","last_undegraded":"2024-09-21T11:43:39.368278+0000","last_fullsized":"2024-09-21T11:43:39.368278+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:58.177844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:58.177844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:06.969504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":170,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350586+0000","last_change":"2024-09-21T11:40:04.329480+0000","last_active":"2024-09-21T11:43:39.350586+0000","last_peered":"2024-09-21T11:43:39.350586+0000","last_clean":"2024-09-21T11:43:39.350586+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:43:39.350586+0000","last_undegraded":"2024-09-21T11:43:39.350586+0000","last_fullsized":"2024-09-21T11:43:39.350586+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":175,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.360549+0000","last_change":"2024-09-21T11:41:38.453843+0000","last_active":"2024-09-21T11:43:39.360549+0000","last_peered":"2024-09-21T11:43:39.360549+0000","last_clean":"2024-09-21T11:43:39.360549+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:43:39.360549+0000","last_undegraded":"2024-09-21T11:43:39.360549+0000","last_fullsized":"2024-09-21T11:43:39.360549+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:38.453579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:38.453579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"142'10267","reported_seq":10468,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.834297+0000","last_change":"2024-09-21T11:42:28.141767+0000","last_active":"2024-09-21T11:43:40.834297+0000","last_peered":"2024-09-21T11:43:40.834297+0000","last_clean":"2024-09-21T11:43:40.834297+0000","last_became_active":"2024-09-21T11:41:10.300107+0000","last_became_peered":"2024-09-21T11:41:10.300107+0000","last_unstale":"2024-09-21T11:43:40.834297+0000","last_undegraded":"2024-09-21T11:43:40.834297+0000","last_fullsized":"2024-09-21T11:43:40.834297+0000","mapping_epoch":82,"log_start":"100'4764","ondisk_log_start":"100'4764","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6297","last_scrub_stamp":"2024-09-21T11:42:28.141670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.141670+0000","objects_scrubbed":6222,"log_size":5503,"log_dups_size":0,"ondisk_log_size":5503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2853,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":290127872,"num_objects":4427,"num_object_clones":0,"num_object_copies":13281,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10267,"num_write_kb":470208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":236,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368893+0000","last_change":"2024-09-21T11:43:39.233466+0000","last_active":"2024-09-21T11:43:39.368893+0000","last_peered":"2024-09-21T11:43:39.368893+0000","last_clean":"2024-09-21T11:43:39.368893+0000","last_became_active":"2024-09-21T11:42:37.160867+0000","last_became_peered":"2024-09-21T11:42:37.160867+0000","last_unstale":"2024-09-21T11:43:39.368893+0000","last_undegraded":"2024-09-21T11:43:39.368893+0000","last_fullsized":"2024-09-21T11:43:39.368893+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:43:39.233386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.233386+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.064116+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":201,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.355877+0000","last_change":"2024-09-21T11:43:36.784531+0000","last_active":"2024-09-21T11:43:39.355877+0000","last_peered":"2024-09-21T11:43:39.355877+0000","last_clean":"2024-09-21T11:43:39.355877+0000","last_became_active":"2024-09-21T11:42:37.167043+0000","last_became_peered":"2024-09-21T11:42:37.167043+0000","last_unstale":"2024-09-21T11:43:39.355877+0000","last_undegraded":"2024-09-21T11:43:39.355877+0000","last_fullsized":"2024-09-21T11:43:39.355877+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.784467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.784467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:39.372586+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":72,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.369753+0000","last_change":"2024-09-21T11:42:37.160975+0000","last_active":"2024-09-21T11:43:39.369753+0000","last_peered":"2024-09-21T11:43:39.369753+0000","last_clean":"2024-09-21T11:43:39.369753+0000","last_became_active":"2024-09-21T11:42:37.160659+0000","last_became_peered":"2024-09-21T11:42:37.160659+0000","last_unstale":"2024-09-21T11:43:39.369753+0000","last_undegraded":"2024-09-21T11:43:39.369753+0000","last_fullsized":"2024-09-21T11:43:39.369753+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:17.392806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.392806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":175,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.378347+0000","last_change":"2024-09-21T11:42:23.339239+0000","last_active":"2024-09-21T11:43:39.378347+0000","last_peered":"2024-09-21T11:43:39.378347+0000","last_clean":"2024-09-21T11:43:39.378347+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:43:39.378347+0000","last_undegraded":"2024-09-21T11:43:39.378347+0000","last_fullsized":"2024-09-21T11:43:39.378347+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:23.339154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.339154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"142'10185","reported_seq":10388,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.166680+0000","last_change":"2024-09-21T11:42:29.618177+0000","last_active":"2024-09-21T11:43:41.166680+0000","last_peered":"2024-09-21T11:43:41.166680+0000","last_clean":"2024-09-21T11:43:41.166680+0000","last_became_active":"2024-09-21T11:41:10.303333+0000","last_became_peered":"2024-09-21T11:41:10.303333+0000","last_unstale":"2024-09-21T11:43:41.166680+0000","last_undegraded":"2024-09-21T11:43:41.166680+0000","last_fullsized":"2024-09-21T11:43:41.166680+0000","mapping_epoch":82,"log_start":"103'5544","ondisk_log_start":"103'5544","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6263","last_scrub_stamp":"2024-09-21T11:42:29.617794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.617794+0000","objects_scrubbed":6197,"log_size":4641,"log_dups_size":0,"ondisk_log_size":4641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283836416,"num_objects":4331,"num_object_clones":0,"num_object_copies":12993,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10185,"num_write_kb":464512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":206,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.369846+0000","last_change":"2024-09-21T11:43:07.347397+0000","last_active":"2024-09-21T11:43:39.369846+0000","last_peered":"2024-09-21T11:43:39.369846+0000","last_clean":"2024-09-21T11:43:39.369846+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:43:39.369846+0000","last_undegraded":"2024-09-21T11:43:39.369846+0000","last_fullsized":"2024-09-21T11:43:39.369846+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:43:07.347346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:07.347346+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.982030+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":187,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.358467+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:43:39.358467+0000","last_peered":"2024-09-21T11:43:39.358467+0000","last_clean":"2024-09-21T11:43:39.358467+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:43:39.358467+0000","last_undegraded":"2024-09-21T11:43:39.358467+0000","last_fullsized":"2024-09-21T11:43:39.358467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:47.593290+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":240,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.375639+0000","last_change":"2024-09-21T11:43:38.421538+0000","last_active":"2024-09-21T11:43:39.375639+0000","last_peered":"2024-09-21T11:43:39.375639+0000","last_clean":"2024-09-21T11:43:39.375639+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:43:39.375639+0000","last_undegraded":"2024-09-21T11:43:39.375639+0000","last_fullsized":"2024-09-21T11:43:39.375639+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:43:38.421251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.421251+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:48.875567+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":182,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368569+0000","last_change":"2024-09-21T11:42:50.131895+0000","last_active":"2024-09-21T11:43:39.368569+0000","last_peered":"2024-09-21T11:43:39.368569+0000","last_clean":"2024-09-21T11:43:39.368569+0000","last_became_active":"2024-09-21T11:42:37.157732+0000","last_became_peered":"2024-09-21T11:42:37.157732+0000","last_unstale":"2024-09-21T11:43:39.368569+0000","last_undegraded":"2024-09-21T11:43:39.368569+0000","last_fullsized":"2024-09-21T11:43:39.368569+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:50.131830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:50.131830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:52.943303+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"142'10300","reported_seq":10490,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.809545+0000","last_change":"2024-09-21T11:43:37.155559+0000","last_active":"2024-09-21T11:43:40.809545+0000","last_peered":"2024-09-21T11:43:40.809545+0000","last_clean":"2024-09-21T11:43:40.809545+0000","last_became_active":"2024-09-21T11:41:10.304149+0000","last_became_peered":"2024-09-21T11:41:10.304149+0000","last_unstale":"2024-09-21T11:43:40.809545+0000","last_undegraded":"2024-09-21T11:43:40.809545+0000","last_fullsized":"2024-09-21T11:43:40.809545+0000","mapping_epoch":82,"log_start":"100'4802","ondisk_log_start":"100'4802","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'10015","last_scrub_stamp":"2024-09-21T11:43:37.155379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.155379+0000","objects_scrubbed":4788,"log_size":5498,"log_dups_size":0,"ondisk_log_size":5498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:48.799347+0000","scrub_duration":1732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291110912,"num_objects":4442,"num_object_clones":0,"num_object_copies":13326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10300,"num_write_kb":471744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":215,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.366779+0000","last_change":"2024-09-21T11:42:37.192718+0000","last_active":"2024-09-21T11:43:39.366779+0000","last_peered":"2024-09-21T11:43:39.366779+0000","last_clean":"2024-09-21T11:43:39.366779+0000","last_became_active":"2024-09-21T11:42:37.168751+0000","last_became_peered":"2024-09-21T11:42:37.168751+0000","last_unstale":"2024-09-21T11:43:39.366779+0000","last_undegraded":"2024-09-21T11:43:39.366779+0000","last_fullsized":"2024-09-21T11:43:39.366779+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:15.417817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.417817+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350594+0000","last_change":"2024-09-21T11:41:28.936695+0000","last_active":"2024-09-21T11:43:39.350594+0000","last_peered":"2024-09-21T11:43:39.350594+0000","last_clean":"2024-09-21T11:43:39.350594+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:43:39.350594+0000","last_undegraded":"2024-09-21T11:43:39.350594+0000","last_fullsized":"2024-09-21T11:43:39.350594+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:28.936645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:28.936645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":179,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.376455+0000","last_change":"2024-09-21T11:42:18.265457+0000","last_active":"2024-09-21T11:43:39.376455+0000","last_peered":"2024-09-21T11:43:39.376455+0000","last_clean":"2024-09-21T11:43:39.376455+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:43:39.376455+0000","last_undegraded":"2024-09-21T11:43:39.376455+0000","last_fullsized":"2024-09-21T11:43:39.376455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.265392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.265392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:45.700703+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":166,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.365494+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:43:39.365494+0000","last_peered":"2024-09-21T11:43:39.365494+0000","last_clean":"2024-09-21T11:43:39.365494+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:43:39.365494+0000","last_undegraded":"2024-09-21T11:43:39.365494+0000","last_fullsized":"2024-09-21T11:43:39.365494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"142'10213","reported_seq":23463,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.201265+0000","last_change":"2024-09-21T11:43:22.892096+0000","last_active":"2024-09-21T11:43:41.201265+0000","last_peered":"2024-09-21T11:43:41.201265+0000","last_clean":"2024-09-21T11:43:41.201265+0000","last_became_active":"2024-09-21T11:43:22.875072+0000","last_became_peered":"2024-09-21T11:43:22.875072+0000","last_unstale":"2024-09-21T11:43:41.201265+0000","last_undegraded":"2024-09-21T11:43:41.201265+0000","last_fullsized":"2024-09-21T11:43:41.201265+0000","mapping_epoch":130,"log_start":"104'5650","ondisk_log_start":"104'5650","created":82,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4563,"log_dups_size":0,"ondisk_log_size":4563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283705344,"num_objects":4329,"num_object_clones":0,"num_object_copies":12987,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10213,"num_write_kb":465344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6549,"num_bytes_recovered":429195264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":240,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.355742+0000","last_change":"2024-09-21T11:43:27.726443+0000","last_active":"2024-09-21T11:43:39.355742+0000","last_peered":"2024-09-21T11:43:39.355742+0000","last_clean":"2024-09-21T11:43:39.355742+0000","last_became_active":"2024-09-21T11:42:37.166373+0000","last_became_peered":"2024-09-21T11:42:37.166373+0000","last_unstale":"2024-09-21T11:43:39.355742+0000","last_undegraded":"2024-09-21T11:43:39.355742+0000","last_fullsized":"2024-09-21T11:43:39.355742+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:43:27.726370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:27.726370+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:36.734856+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362408+0000","last_change":"2024-09-21T11:41:46.471940+0000","last_active":"2024-09-21T11:43:39.362408+0000","last_peered":"2024-09-21T11:43:39.362408+0000","last_clean":"2024-09-21T11:43:39.362408+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:43:39.362408+0000","last_undegraded":"2024-09-21T11:43:39.362408+0000","last_fullsized":"2024-09-21T11:43:39.362408+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:46.471872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:46.471872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":187,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.376816+0000","last_change":"2024-09-21T11:43:26.484397+0000","last_active":"2024-09-21T11:43:39.376816+0000","last_peered":"2024-09-21T11:43:39.376816+0000","last_clean":"2024-09-21T11:43:39.376816+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:43:39.376816+0000","last_undegraded":"2024-09-21T11:43:39.376816+0000","last_fullsized":"2024-09-21T11:43:39.376816+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.484353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.484353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:38.261052+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":174,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361982+0000","last_change":"2024-09-21T11:43:29.650485+0000","last_active":"2024-09-21T11:43:39.361982+0000","last_peered":"2024-09-21T11:43:39.361982+0000","last_clean":"2024-09-21T11:43:39.361982+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:43:39.361982+0000","last_undegraded":"2024-09-21T11:43:39.361982+0000","last_fullsized":"2024-09-21T11:43:39.361982+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:29.650427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.650427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:32.208091+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"142'10247","reported_seq":10386,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.090413+0000","last_change":"2024-09-21T11:42:31.693667+0000","last_active":"2024-09-21T11:43:41.090413+0000","last_peered":"2024-09-21T11:43:41.090413+0000","last_clean":"2024-09-21T11:43:41.090413+0000","last_became_active":"2024-09-21T11:41:10.300287+0000","last_became_peered":"2024-09-21T11:41:10.300287+0000","last_unstale":"2024-09-21T11:43:41.090413+0000","last_undegraded":"2024-09-21T11:43:41.090413+0000","last_fullsized":"2024-09-21T11:43:41.090413+0000","mapping_epoch":82,"log_start":"106'5954","ondisk_log_start":"106'5954","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":449,"log_size":4293,"log_dups_size":0,"ondisk_log_size":4293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291307520,"num_objects":4445,"num_object_clones":0,"num_object_copies":13335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10247,"num_write_kb":470144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":233,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368162+0000","last_change":"2024-09-21T11:43:32.231463+0000","last_active":"2024-09-21T11:43:39.368162+0000","last_peered":"2024-09-21T11:43:39.368162+0000","last_clean":"2024-09-21T11:43:39.368162+0000","last_became_active":"2024-09-21T11:42:37.165364+0000","last_became_peered":"2024-09-21T11:42:37.165364+0000","last_unstale":"2024-09-21T11:43:39.368162+0000","last_undegraded":"2024-09-21T11:43:39.368162+0000","last_fullsized":"2024-09-21T11:43:39.368162+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:43:32.231381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.231381+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:36.630886+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":193,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362584+0000","last_change":"2024-09-21T11:42:37.165805+0000","last_active":"2024-09-21T11:43:39.362584+0000","last_peered":"2024-09-21T11:43:39.362584+0000","last_clean":"2024-09-21T11:43:39.362584+0000","last_became_active":"2024-09-21T11:42:37.165470+0000","last_became_peered":"2024-09-21T11:42:37.165470+0000","last_unstale":"2024-09-21T11:43:39.362584+0000","last_undegraded":"2024-09-21T11:43:39.362584+0000","last_fullsized":"2024-09-21T11:43:39.362584+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:34.481939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:34.481939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":194,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.502911+0000","last_change":"2024-09-21T11:43:40.502911+0000","last_active":"2024-09-21T11:43:40.502911+0000","last_peered":"2024-09-21T11:43:40.502911+0000","last_clean":"2024-09-21T11:43:40.502911+0000","last_became_active":"2024-09-21T11:42:37.155675+0000","last_became_peered":"2024-09-21T11:42:37.155675+0000","last_unstale":"2024-09-21T11:43:40.502911+0000","last_undegraded":"2024-09-21T11:43:40.502911+0000","last_fullsized":"2024-09-21T11:43:40.502911+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:40.502747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.502747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:57.625738+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":182,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.358178+0000","last_change":"2024-09-21T11:42:37.170697+0000","last_active":"2024-09-21T11:43:39.358178+0000","last_peered":"2024-09-21T11:43:39.358178+0000","last_clean":"2024-09-21T11:43:39.358178+0000","last_became_active":"2024-09-21T11:42:37.168686+0000","last_became_peered":"2024-09-21T11:42:37.168686+0000","last_unstale":"2024-09-21T11:43:39.358178+0000","last_undegraded":"2024-09-21T11:43:39.358178+0000","last_fullsized":"2024-09-21T11:43:39.358178+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:27.327920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:27.327920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"142'10058","reported_seq":22898,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.605705+0000","last_change":"2024-09-21T11:43:27.000045+0000","last_active":"2024-09-21T11:43:40.605705+0000","last_peered":"2024-09-21T11:43:40.605705+0000","last_clean":"2024-09-21T11:43:40.605705+0000","last_became_active":"2024-09-21T11:43:26.983757+0000","last_became_peered":"2024-09-21T11:43:26.983757+0000","last_unstale":"2024-09-21T11:43:40.605705+0000","last_undegraded":"2024-09-21T11:43:40.605705+0000","last_fullsized":"2024-09-21T11:43:40.605705+0000","mapping_epoch":134,"log_start":"100'4880","ondisk_log_start":"100'4880","created":82,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6370","last_scrub_stamp":"2024-09-21T11:42:31.280157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:31.280157+0000","objects_scrubbed":6318,"log_size":5178,"log_dups_size":0,"ondisk_log_size":5178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:58.540352+0000","scrub_duration":2077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":291241984,"num_objects":4444,"num_object_clones":0,"num_object_copies":13332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10058,"num_write_kb":464064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6310,"num_bytes_recovered":413532160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":199,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.349518+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:43:39.349518+0000","last_peered":"2024-09-21T11:43:39.349518+0000","last_clean":"2024-09-21T11:43:39.349518+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:43:39.349518+0000","last_undegraded":"2024-09-21T11:43:39.349518+0000","last_fullsized":"2024-09-21T11:43:39.349518+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":198,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362540+0000","last_change":"2024-09-21T11:42:19.352990+0000","last_active":"2024-09-21T11:43:39.362540+0000","last_peered":"2024-09-21T11:43:39.362540+0000","last_clean":"2024-09-21T11:43:39.362540+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:43:39.362540+0000","last_undegraded":"2024-09-21T11:43:39.362540+0000","last_fullsized":"2024-09-21T11:43:39.362540+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:42:19.352895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.352895+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":231,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361932+0000","last_change":"2024-09-21T11:41:43.495040+0000","last_active":"2024-09-21T11:43:39.361932+0000","last_peered":"2024-09-21T11:43:39.361932+0000","last_clean":"2024-09-21T11:43:39.361932+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:43:39.361932+0000","last_undegraded":"2024-09-21T11:43:39.361932+0000","last_fullsized":"2024-09-21T11:43:39.361932+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:41:43.494960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:43.494960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":174,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.349873+0000","last_change":"2024-09-21T11:41:39.867162+0000","last_active":"2024-09-21T11:43:39.349873+0000","last_peered":"2024-09-21T11:43:39.349873+0000","last_clean":"2024-09-21T11:43:39.349873+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:43:39.349873+0000","last_undegraded":"2024-09-21T11:43:39.349873+0000","last_fullsized":"2024-09-21T11:43:39.349873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.867108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.867108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.355518+0000","last_change":"2024-09-21T11:43:30.734704+0000","last_active":"2024-09-21T11:43:39.355518+0000","last_peered":"2024-09-21T11:43:39.355518+0000","last_clean":"2024-09-21T11:43:39.355518+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:43:39.355518+0000","last_undegraded":"2024-09-21T11:43:39.355518+0000","last_fullsized":"2024-09-21T11:43:39.355518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.734361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.734361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:35.190586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":207,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.363579+0000","last_change":"2024-09-21T11:42:37.201056+0000","last_active":"2024-09-21T11:43:39.363579+0000","last_peered":"2024-09-21T11:43:39.363579+0000","last_clean":"2024-09-21T11:43:39.363579+0000","last_became_active":"2024-09-21T11:42:37.171563+0000","last_became_peered":"2024-09-21T11:42:37.171563+0000","last_unstale":"2024-09-21T11:43:39.363579+0000","last_undegraded":"2024-09-21T11:43:39.363579+0000","last_fullsized":"2024-09-21T11:43:39.363579+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:41:25.310957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:25.310957+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":185,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.352186+0000","last_change":"2024-09-21T11:42:37.170710+0000","last_active":"2024-09-21T11:43:39.352186+0000","last_peered":"2024-09-21T11:43:39.352186+0000","last_clean":"2024-09-21T11:43:39.352186+0000","last_became_active":"2024-09-21T11:42:37.170365+0000","last_became_peered":"2024-09-21T11:42:37.170365+0000","last_unstale":"2024-09-21T11:43:39.352186+0000","last_undegraded":"2024-09-21T11:43:39.352186+0000","last_fullsized":"2024-09-21T11:43:39.352186+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:15.090108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.090108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":174,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362105+0000","last_change":"2024-09-21T11:41:44.479681+0000","last_active":"2024-09-21T11:43:39.362105+0000","last_peered":"2024-09-21T11:43:39.362105+0000","last_clean":"2024-09-21T11:43:39.362105+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:43:39.362105+0000","last_undegraded":"2024-09-21T11:43:39.362105+0000","last_fullsized":"2024-09-21T11:43:39.362105+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:44.479617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:44.479617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":215,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.367765+0000","last_change":"2024-09-21T11:43:32.077598+0000","last_active":"2024-09-21T11:43:39.367765+0000","last_peered":"2024-09-21T11:43:39.367765+0000","last_clean":"2024-09-21T11:43:39.367765+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:43:39.367765+0000","last_undegraded":"2024-09-21T11:43:39.367765+0000","last_fullsized":"2024-09-21T11:43:39.367765+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:11.469922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.469922+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362146+0000","last_change":"2024-09-21T11:42:23.241888+0000","last_active":"2024-09-21T11:43:39.362146+0000","last_peered":"2024-09-21T11:43:39.362146+0000","last_clean":"2024-09-21T11:43:39.362146+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:43:39.362146+0000","last_undegraded":"2024-09-21T11:43:39.362146+0000","last_fullsized":"2024-09-21T11:43:39.362146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:23.241796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.241796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:44.385176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.363391+0000","last_change":"2024-09-21T11:43:38.461076+0000","last_active":"2024-09-21T11:43:39.363391+0000","last_peered":"2024-09-21T11:43:39.363391+0000","last_clean":"2024-09-21T11:43:39.363391+0000","last_became_active":"2024-09-21T11:42:37.172423+0000","last_became_peered":"2024-09-21T11:42:37.172423+0000","last_unstale":"2024-09-21T11:43:39.363391+0000","last_undegraded":"2024-09-21T11:43:39.363391+0000","last_fullsized":"2024-09-21T11:43:39.363391+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.461014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.461014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:44.750422+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":175,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.377864+0000","last_change":"2024-09-21T11:42:24.275745+0000","last_active":"2024-09-21T11:43:39.377864+0000","last_peered":"2024-09-21T11:43:39.377864+0000","last_clean":"2024-09-21T11:43:39.377864+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:43:39.377864+0000","last_undegraded":"2024-09-21T11:43:39.377864+0000","last_fullsized":"2024-09-21T11:43:39.377864+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:24.275686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.275686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":198,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350434+0000","last_change":"2024-09-21T11:41:33.917474+0000","last_active":"2024-09-21T11:43:39.350434+0000","last_peered":"2024-09-21T11:43:39.350434+0000","last_clean":"2024-09-21T11:43:39.350434+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:43:39.350434+0000","last_undegraded":"2024-09-21T11:43:39.350434+0000","last_fullsized":"2024-09-21T11:43:39.350434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:33.917389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:33.917389+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.366092+0000","last_change":"2024-09-21T11:42:22.478692+0000","last_active":"2024-09-21T11:43:39.366092+0000","last_peered":"2024-09-21T11:43:39.366092+0000","last_clean":"2024-09-21T11:43:39.366092+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:43:39.366092+0000","last_undegraded":"2024-09-21T11:43:39.366092+0000","last_fullsized":"2024-09-21T11:43:39.366092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:22.478619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.478619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.263376+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":224,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.360960+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:43:39.360960+0000","last_peered":"2024-09-21T11:43:39.360960+0000","last_clean":"2024-09-21T11:43:39.360960+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:43:39.360960+0000","last_undegraded":"2024-09-21T11:43:39.360960+0000","last_fullsized":"2024-09-21T11:43:39.360960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":183,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368799+0000","last_change":"2024-09-21T11:43:24.300706+0000","last_active":"2024-09-21T11:43:39.368799+0000","last_peered":"2024-09-21T11:43:39.368799+0000","last_clean":"2024-09-21T11:43:39.368799+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:43:39.368799+0000","last_undegraded":"2024-09-21T11:43:39.368799+0000","last_fullsized":"2024-09-21T11:43:39.368799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:24.300642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:24.300642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.984128+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":217,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361153+0000","last_change":"2024-09-21T11:42:37.207434+0000","last_active":"2024-09-21T11:43:39.361153+0000","last_peered":"2024-09-21T11:43:39.361153+0000","last_clean":"2024-09-21T11:43:39.361153+0000","last_became_active":"2024-09-21T11:42:37.171345+0000","last_became_peered":"2024-09-21T11:42:37.171345+0000","last_unstale":"2024-09-21T11:43:39.361153+0000","last_undegraded":"2024-09-21T11:43:39.361153+0000","last_fullsized":"2024-09-21T11:43:39.361153+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.379125+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362230+0000","last_change":"2024-09-21T11:42:11.317760+0000","last_active":"2024-09-21T11:43:39.362230+0000","last_peered":"2024-09-21T11:43:39.362230+0000","last_clean":"2024-09-21T11:43:39.362230+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:43:39.362230+0000","last_undegraded":"2024-09-21T11:43:39.362230+0000","last_fullsized":"2024-09-21T11:43:39.362230+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.317696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.317696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"114'9","reported_seq":243,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350346+0000","last_change":"2024-09-21T11:43:28.003108+0000","last_active":"2024-09-21T11:43:39.350346+0000","last_peered":"2024-09-21T11:43:39.350346+0000","last_clean":"2024-09-21T11:43:39.350346+0000","last_became_active":"2024-09-21T11:42:37.163771+0000","last_became_peered":"2024-09-21T11:42:37.163771+0000","last_unstale":"2024-09-21T11:43:39.350346+0000","last_undegraded":"2024-09-21T11:43:39.350346+0000","last_fullsized":"2024-09-21T11:43:39.350346+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:28.003036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.003036+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:32.544327+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":174,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.366999+0000","last_change":"2024-09-21T11:41:30.274387+0000","last_active":"2024-09-21T11:43:39.366999+0000","last_peered":"2024-09-21T11:43:39.366999+0000","last_clean":"2024-09-21T11:43:39.366999+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:43:39.366999+0000","last_undegraded":"2024-09-21T11:43:39.366999+0000","last_fullsized":"2024-09-21T11:43:39.366999+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:30.274201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:30.274201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":229,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:40.233160+0000","last_change":"2024-09-21T11:43:40.233160+0000","last_active":"2024-09-21T11:43:40.233160+0000","last_peered":"2024-09-21T11:43:40.233160+0000","last_clean":"2024-09-21T11:43:40.233160+0000","last_became_active":"2024-09-21T11:42:37.154835+0000","last_became_peered":"2024-09-21T11:42:37.154835+0000","last_unstale":"2024-09-21T11:43:40.233160+0000","last_undegraded":"2024-09-21T11:43:40.233160+0000","last_fullsized":"2024-09-21T11:43:40.233160+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:43:40.233067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.233067+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:41.481160+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":187,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.369975+0000","last_change":"2024-09-21T11:43:00.245058+0000","last_active":"2024-09-21T11:43:39.369975+0000","last_peered":"2024-09-21T11:43:39.369975+0000","last_clean":"2024-09-21T11:43:39.369975+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:43:39.369975+0000","last_undegraded":"2024-09-21T11:43:39.369975+0000","last_fullsized":"2024-09-21T11:43:39.369975+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:00.244985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:00.244985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:22.792144+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":72,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.349909+0000","last_change":"2024-09-21T11:42:37.163592+0000","last_active":"2024-09-21T11:43:39.349909+0000","last_peered":"2024-09-21T11:43:39.349909+0000","last_clean":"2024-09-21T11:43:39.349909+0000","last_became_active":"2024-09-21T11:42:37.163261+0000","last_became_peered":"2024-09-21T11:42:37.163261+0000","last_unstale":"2024-09-21T11:43:39.349909+0000","last_undegraded":"2024-09-21T11:43:39.349909+0000","last_fullsized":"2024-09-21T11:43:39.349909+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.301406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.301406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.735442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":167,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.360321+0000","last_change":"2024-09-21T11:40:06.353577+0000","last_active":"2024-09-21T11:43:39.360321+0000","last_peered":"2024-09-21T11:43:39.360321+0000","last_clean":"2024-09-21T11:43:39.360321+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:43:39.360321+0000","last_undegraded":"2024-09-21T11:43:39.360321+0000","last_fullsized":"2024-09-21T11:43:39.360321+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":117,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.355841+0000","last_change":"2024-09-21T11:42:37.178093+0000","last_active":"2024-09-21T11:43:39.355841+0000","last_peered":"2024-09-21T11:43:39.355841+0000","last_clean":"2024-09-21T11:43:39.355841+0000","last_became_active":"2024-09-21T11:42:37.167265+0000","last_became_peered":"2024-09-21T11:42:37.167265+0000","last_unstale":"2024-09-21T11:43:39.355841+0000","last_undegraded":"2024-09-21T11:43:39.355841+0000","last_fullsized":"2024-09-21T11:43:39.355841+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:32.296640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:32.296640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.362336+0000","last_change":"2024-09-21T11:43:26.586603+0000","last_active":"2024-09-21T11:43:39.362336+0000","last_peered":"2024-09-21T11:43:39.362336+0000","last_clean":"2024-09-21T11:43:39.362336+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:43:39.362336+0000","last_undegraded":"2024-09-21T11:43:39.362336+0000","last_fullsized":"2024-09-21T11:43:39.362336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.586562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.586562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:55.496321+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.358254+0000","last_change":"2024-09-21T11:42:37.167224+0000","last_active":"2024-09-21T11:43:39.358254+0000","last_peered":"2024-09-21T11:43:39.358254+0000","last_clean":"2024-09-21T11:43:39.358254+0000","last_became_active":"2024-09-21T11:42:37.166917+0000","last_became_peered":"2024-09-21T11:42:37.166917+0000","last_unstale":"2024-09-21T11:43:39.358254+0000","last_undegraded":"2024-09-21T11:43:39.358254+0000","last_fullsized":"2024-09-21T11:43:39.358254+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:23.514572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.514572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.364890+0000","last_change":"2024-09-21T11:42:37.177478+0000","last_active":"2024-09-21T11:43:39.364890+0000","last_peered":"2024-09-21T11:43:39.364890+0000","last_clean":"2024-09-21T11:43:39.364890+0000","last_became_active":"2024-09-21T11:42:37.161589+0000","last_became_peered":"2024-09-21T11:42:37.161589+0000","last_unstale":"2024-09-21T11:43:39.364890+0000","last_undegraded":"2024-09-21T11:43:39.364890+0000","last_fullsized":"2024-09-21T11:43:39.364890+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:41:29.236463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:29.236463+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":202,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.360541+0000","last_change":"2024-09-21T11:42:15.470041+0000","last_active":"2024-09-21T11:43:39.360541+0000","last_peered":"2024-09-21T11:43:39.360541+0000","last_clean":"2024-09-21T11:43:39.360541+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:43:39.360541+0000","last_undegraded":"2024-09-21T11:43:39.360541+0000","last_fullsized":"2024-09-21T11:43:39.360541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:15.469421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.469421+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":193,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.352244+0000","last_change":"2024-09-21T11:42:37.173431+0000","last_active":"2024-09-21T11:43:39.352244+0000","last_peered":"2024-09-21T11:43:39.352244+0000","last_clean":"2024-09-21T11:43:39.352244+0000","last_became_active":"2024-09-21T11:42:37.172859+0000","last_became_peered":"2024-09-21T11:42:37.172859+0000","last_unstale":"2024-09-21T11:43:39.352244+0000","last_undegraded":"2024-09-21T11:43:39.352244+0000","last_fullsized":"2024-09-21T11:43:39.352244+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.952335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.952335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":178,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361654+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:43:39.361654+0000","last_peered":"2024-09-21T11:43:39.361654+0000","last_clean":"2024-09-21T11:43:39.361654+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:43:39.361654+0000","last_undegraded":"2024-09-21T11:43:39.361654+0000","last_fullsized":"2024-09-21T11:43:39.361654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.612728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":175,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.358391+0000","last_change":"2024-09-21T11:42:12.409717+0000","last_active":"2024-09-21T11:43:39.358391+0000","last_peered":"2024-09-21T11:43:39.358391+0000","last_clean":"2024-09-21T11:43:39.358391+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:43:39.358391+0000","last_undegraded":"2024-09-21T11:43:39.358391+0000","last_fullsized":"2024-09-21T11:43:39.358391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:12.409641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:12.409641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":215,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361614+0000","last_change":"2024-09-21T11:42:37.195076+0000","last_active":"2024-09-21T11:43:39.361614+0000","last_peered":"2024-09-21T11:43:39.361614+0000","last_clean":"2024-09-21T11:43:39.361614+0000","last_became_active":"2024-09-21T11:42:37.169482+0000","last_became_peered":"2024-09-21T11:42:37.169482+0000","last_unstale":"2024-09-21T11:43:39.361614+0000","last_undegraded":"2024-09-21T11:43:39.361614+0000","last_fullsized":"2024-09-21T11:43:39.361614+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:41:32.414244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:32.414244+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":194,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.368504+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:43:39.368504+0000","last_peered":"2024-09-21T11:43:39.368504+0000","last_clean":"2024-09-21T11:43:39.368504+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:43:39.368504+0000","last_undegraded":"2024-09-21T11:43:39.368504+0000","last_fullsized":"2024-09-21T11:43:39.368504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:41.100704+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":186,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.838067+0000","last_change":"2024-09-21T11:43:39.838067+0000","last_active":"2024-09-21T11:43:39.838067+0000","last_peered":"2024-09-21T11:43:39.838067+0000","last_clean":"2024-09-21T11:43:39.838067+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:43:39.838067+0000","last_undegraded":"2024-09-21T11:43:39.838067+0000","last_fullsized":"2024-09-21T11:43:39.838067+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:39.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.838002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.931489+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":174,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361856+0000","last_change":"2024-09-21T11:42:30.269470+0000","last_active":"2024-09-21T11:43:39.361856+0000","last_peered":"2024-09-21T11:43:39.361856+0000","last_clean":"2024-09-21T11:43:39.361856+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:43:39.361856+0000","last_undegraded":"2024-09-21T11:43:39.361856+0000","last_fullsized":"2024-09-21T11:43:39.361856+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:30.269408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:30.269408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:53.929437+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":71,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.350296+0000","last_change":"2024-09-21T11:42:37.155891+0000","last_active":"2024-09-21T11:43:39.350296+0000","last_peered":"2024-09-21T11:43:39.350296+0000","last_clean":"2024-09-21T11:43:39.350296+0000","last_became_active":"2024-09-21T11:42:37.155620+0000","last_became_peered":"2024-09-21T11:42:37.155620+0000","last_unstale":"2024-09-21T11:43:39.350296+0000","last_undegraded":"2024-09-21T11:43:39.350296+0000","last_fullsized":"2024-09-21T11:43:39.350296+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.358678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.358678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":194,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.361838+0000","last_change":"2024-09-21T11:43:36.413767+0000","last_active":"2024-09-21T11:43:39.361838+0000","last_peered":"2024-09-21T11:43:39.361838+0000","last_clean":"2024-09-21T11:43:39.361838+0000","last_became_active":"2024-09-21T11:42:37.167248+0000","last_became_peered":"2024-09-21T11:42:37.167248+0000","last_unstale":"2024-09-21T11:43:39.361838+0000","last_undegraded":"2024-09-21T11:43:39.361838+0000","last_fullsized":"2024-09-21T11:43:39.361838+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.413725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.413725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.736856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":228,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.356338+0000","last_change":"2024-09-21T11:43:32.709031+0000","last_active":"2024-09-21T11:43:39.356338+0000","last_peered":"2024-09-21T11:43:39.356338+0000","last_clean":"2024-09-21T11:43:39.356338+0000","last_became_active":"2024-09-21T11:42:37.171253+0000","last_became_peered":"2024-09-21T11:42:37.171253+0000","last_unstale":"2024-09-21T11:43:39.356338+0000","last_undegraded":"2024-09-21T11:43:39.356338+0000","last_fullsized":"2024-09-21T11:43:39.356338+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:43:32.708849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.708849+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.362664+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":178,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-21T11:43:39.367482+0000","last_change":"2024-09-21T11:41:31.249298+0000","last_active":"2024-09-21T11:43:39.367482+0000","last_peered":"2024-09-21T11:43:39.367482+0000","last_clean":"2024-09-21T11:43:39.367482+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:43:39.367482+0000","last_undegraded":"2024-09-21T11:43:39.367482+0000","last_fullsized":"2024-09-21T11:43:39.367482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:31.249215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:31.249215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":4648730648,"num_objects":70935,"num_object_clones":0,"num_object_copies":212805,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70935,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163877,"num_write_kb":7513921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38414,"num_bytes_recovered":2517303320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15910584320,"data_stored":15910568032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76558,"ondisk_log_size":76558,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":5367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":214,"num_read_kb":209,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234033,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2177252,"kb_used_data":1997524,"kb_used_omap":1,"kb_used_meta":179710,"kb_avail":91567900,"statfs":{"total":95995035648,"available":93765529600,"internally_reserved":0,"allocated":2045464576,"data_stored":2045061803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":184023498},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462966,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1884988,"kb_used_data":1749668,"kb_used_omap":1,"kb_used_meta":135294,"kb_avail":91860164,"statfs":{"total":95995035648,"available":94064807936,"internally_reserved":0,"allocated":1791660032,"data_stored":1791241724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138541510},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1221984,"kb_used_data":1158856,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":92523168,"statfs":{"total":95995035648,"available":94743724032,"internally_reserved":0,"allocated":1186668544,"data_stored":1186276894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1519832,"kb_used_data":1434376,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":92225320,"statfs":{"total":95995035648,"available":94438727680,"internally_reserved":0,"allocated":1468801024,"data_stored":1468410022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117063,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3027236,"kb_used_data":2869708,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":90717916,"statfs":{"total":95995035648,"available":92895145984,"internally_reserved":0,"allocated":2938580992,"data_stored":2938185462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":161282503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2475272,"kb_used_data":2287216,"kb_used_omap":1,"kb_used_meta":188030,"kb_avail":91269880,"statfs":{"total":95995035648,"available":93460357120,"internally_reserved":0,"allocated":2342109184,"data_stored":2341741964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":192543173},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":55834574932,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2409340,"kb_used_data":2283748,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":91335812,"statfs":{"total":95995035648,"available":93527871488,"internally_reserved":0,"allocated":2338557952,"data_stored":2338139833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2308706304,"data_stored":2308702232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1958547456,"data_stored":1958543384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2325479424,"data_stored":2325479424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2897678336,"data_stored":2897674264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1453129728,"data_stored":1453129728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1171324928,"data_stored":1171324928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1759580160,"data_stored":1759576088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2036137984,"data_stored":2036137984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:43:46.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:43:46.280 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:46.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:46.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:46.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:46.419 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:46.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:46.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:46.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:46.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:46.563 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:46.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:46.664 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:46.692 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:46.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:46.693 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[29710]: 5.7 scrub starts 2024-09-21T11:43:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[29710]: 5.7 scrub ok 2024-09-21T11:43:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[29710]: 3.1a scrub starts 2024-09-21T11:43:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[29710]: 3.1a scrub ok 2024-09-21T11:43:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[29710]: 7.e scrub starts 2024-09-21T11:43:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[29710]: 6.15 scrub starts 2024-09-21T11:43:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[29710]: 6.15 scrub ok 2024-09-21T11:43:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[29710]: pgmap v171: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:43:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[36454]: 5.7 scrub starts 2024-09-21T11:43:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[36454]: 5.7 scrub ok 2024-09-21T11:43:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[36454]: 3.1a scrub starts 2024-09-21T11:43:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[36454]: 3.1a scrub ok 2024-09-21T11:43:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[36454]: 7.e scrub starts 2024-09-21T11:43:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[36454]: 6.15 scrub starts 2024-09-21T11:43:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[36454]: 6.15 scrub ok 2024-09-21T11:43:46.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:46 smithi043 ceph-mon[36454]: pgmap v171: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:43:46.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:46.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:46.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:46.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:46.971 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:46.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:46.972 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:46 smithi175 ceph-mon[32613]: 5.7 scrub starts 2024-09-21T11:43:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:46 smithi175 ceph-mon[32613]: 5.7 scrub ok 2024-09-21T11:43:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:46 smithi175 ceph-mon[32613]: 3.1a scrub starts 2024-09-21T11:43:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:46 smithi175 ceph-mon[32613]: 3.1a scrub ok 2024-09-21T11:43:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:46 smithi175 ceph-mon[32613]: 7.e scrub starts 2024-09-21T11:43:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:46 smithi175 ceph-mon[32613]: 6.15 scrub starts 2024-09-21T11:43:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:46 smithi175 ceph-mon[32613]: 6.15 scrub ok 2024-09-21T11:43:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:46 smithi175 ceph-mon[32613]: pgmap v171: 148 pgs: 148 active+clean; 4.2 GiB data, 14 GiB used, 612 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:43:47.022 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:43:47.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:47.126 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:47.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:47.127 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:47.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:47.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:47.290 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:47.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:47.391 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:47.425 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:47.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:47.426 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:47.446 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:47.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:47.590 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:47.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:47.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:47.692 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:47.721 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:47.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:47.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:47 smithi043 ceph-mon[29710]: 6.f scrub starts 2024-09-21T11:43:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:47 smithi043 ceph-mon[29710]: 6.f scrub ok 2024-09-21T11:43:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:47 smithi043 ceph-mon[36454]: 6.f scrub starts 2024-09-21T11:43:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:47 smithi043 ceph-mon[36454]: 6.f scrub ok 2024-09-21T11:43:47.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:47.854 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:47.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:47.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:47.957 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:47.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:47.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:47.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:48.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:47 smithi175 ceph-mon[32613]: 6.f scrub starts 2024-09-21T11:43:48.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:47 smithi175 ceph-mon[32613]: 6.f scrub ok 2024-09-21T11:43:48.091 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:48.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:48.105 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:48.118 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:48.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:48.120 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:48.220 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:48.249 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:48.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:48.250 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:48.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:48.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:48.393 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:48.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:48.494 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:48.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:48.515 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:48.515 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:43:48.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:48.644 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:48.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:48.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[29710]: 5.1f scrub starts 2024-09-21T11:43:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[29710]: 5.1f scrub ok 2024-09-21T11:43:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[29710]: 7.e scrub ok 2024-09-21T11:43:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[29710]: 2.0 scrub starts 2024-09-21T11:43:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[29710]: 2.0 scrub ok 2024-09-21T11:43:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[29710]: pgmap v172: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2453724721' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[36454]: 5.1f scrub starts 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[36454]: 5.1f scrub ok 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[36454]: 7.e scrub ok 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[36454]: 2.0 scrub starts 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[36454]: 2.0 scrub ok 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[36454]: pgmap v172: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:43:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:43:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2453724721' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:48.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:48 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:48.745 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:48.776 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:48.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:48.777 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:48.877 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:48.911 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:48.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:48.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:48 smithi175 ceph-mon[32613]: 5.1f scrub starts 2024-09-21T11:43:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:48 smithi175 ceph-mon[32613]: 5.1f scrub ok 2024-09-21T11:43:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:48 smithi175 ceph-mon[32613]: 7.e scrub ok 2024-09-21T11:43:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:48 smithi175 ceph-mon[32613]: 2.0 scrub starts 2024-09-21T11:43:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:48 smithi175 ceph-mon[32613]: 2.0 scrub ok 2024-09-21T11:43:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:48 smithi175 ceph-mon[32613]: pgmap v172: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.45k op/s 2024-09-21T11:43:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:48 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:43:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:48 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:43:49.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:48 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2453724721' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:49.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:48 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:43:49.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:49.048 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:49.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:49.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:49.151 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:49.184 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:49.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:49.185 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:49.285 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:49.314 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:49.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:49.315 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:49.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:49.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:49.453 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:49.454 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:49.479 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:43:49.480 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:43:49.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:43:49.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:49.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:49.600 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:49.600 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:49.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:49.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:49.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:49.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:49.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:49.889 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:49.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:49 smithi043 ceph-mon[29710]: 7.d scrub starts 2024-09-21T11:43:49.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:49 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:49.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:49 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:43:49.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:49 smithi043 ceph-mon[29710]: osdmap e144: 8 total, 8 up, 7 in 2024-09-21T11:43:49.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:49 smithi043 ceph-mon[36454]: 7.d scrub starts 2024-09-21T11:43:49.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:49 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:49.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:49 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:43:49.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:49 smithi043 ceph-mon[36454]: osdmap e144: 8 total, 8 up, 7 in 2024-09-21T11:43:49.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:49 smithi175 ceph-mon[32613]: 7.d scrub starts 2024-09-21T11:43:49.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:49 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:43:49.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:49 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:43:49.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:49 smithi175 ceph-mon[32613]: osdmap e144: 8 total, 8 up, 7 in 2024-09-21T11:43:49.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:50.048 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:50.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:50.049 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:50.150 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:50.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:50.190 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:50.190 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:50.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:50.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:50.338 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:50.338 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:50.439 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:50.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:50.472 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:50.472 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:50.476 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:43:50.479 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:43:50.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:50.604 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:50.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:50.605 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:50.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:50.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:50 smithi043 ceph-mon[36454]: pgmap v174: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 608 B/s rd, 0 B/s wr, 733 op/s 2024-09-21T11:43:50.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:50 smithi043 ceph-mon[29710]: pgmap v174: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 608 B/s rd, 0 B/s wr, 733 op/s 2024-09-21T11:43:50.749 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:50.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:50.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:50.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:50 smithi175 ceph-mon[32613]: pgmap v174: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 4.0 GiB data, 13 GiB used, 613 GiB / 626 GiB avail; 608 B/s rd, 0 B/s wr, 733 op/s 2024-09-21T11:43:50.850 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:50.889 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:50.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:50.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:50.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:50.992 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:51.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:51.034 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:51.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:51.140 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:43:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:43:51.176 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:51.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:51.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:51.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:51.325 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:51.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:51.326 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:51.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:51.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:51.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:51.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:51.590 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:51.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:51.591 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:51.692 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:51.724 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:51.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:51.725 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:51 smithi043 ceph-mon[29710]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:43:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:51 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:51 smithi043 ceph-mon[36454]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:43:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:51 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:51.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:51.848 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:51.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:51.849 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:51.950 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:51.982 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:51.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:51.983 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:52.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:51 smithi175 ceph-mon[32613]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:43:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:51 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:43:52.083 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:52.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:43:52.115 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:52.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:52.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:52.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:52.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:52.260 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:52.260 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:52.361 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:52.391 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:52.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:52.393 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:52.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:52.524 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:52.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:52.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:52.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:52.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:52.664 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:52.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:52.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:52.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:52.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:52.931 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:52.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:52.932 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:52.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:52 smithi043 ceph-mon[29710]: pgmap v175: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:43:52.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:52 smithi043 ceph-mon[36454]: pgmap v175: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:43:53.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:52 smithi175 ceph-mon[32613]: pgmap v175: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:43:53.033 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:53.065 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:53.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:53.066 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:53.166 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:53.198 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:53.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:53.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:53.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:53.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:53.340 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:53.341 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:53.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:53.473 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:53.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:53.474 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:53.575 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:53.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:53.619 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:53.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:53.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:53.748 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:53.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:53.749 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:53.768 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:43:53.769 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:43:53.806 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:43:53.807 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:43:53.844 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:43:53.844 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:43:53.845 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:53 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/251230493' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:53.845 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:53 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/251230493' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:53.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:53.880 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:43:53.881 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:43:53.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:53.899 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:53.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:53.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:43:53.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:43:53.919 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:43:53.952 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:43:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:43:53.990 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:43:53.990 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:43:54.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:54.026 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:43:54.027 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:43:54.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:54.028 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:54.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:54.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:54.156 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:54.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:54.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:54.165 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-21T11:43:49.779551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4245625395,"num_objects":64974,"num_object_clones":0,"num_object_copies":194922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64974,"num_whiteouts":0,"num_read":1056,"num_read_kb":913,"num_write":170586,"num_write_kb":7515594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38553,"num_bytes_recovered":2517897553,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77074,"ondisk_log_size":77074,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":13553516,"kb_used_data":12592380,"kb_used_omap":10,"kb_used_meta":960885,"kb_avail":642662548,"statfs":{"total":671965249536,"available":658086449152,"internally_reserved":0,"allocated":12894597120,"data_stored":12891774417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11156,"internal_metadata":983946348},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-403701760,"num_objects":-6160,"num_object_clones":0,"num_object_copies":-18480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6160,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6160,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.409493"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.379593+0000","last_change":"2024-09-21T11:41:42.291880+0000","last_active":"2024-09-21T11:43:41.379593+0000","last_peered":"2024-09-21T11:43:41.379593+0000","last_clean":"2024-09-21T11:43:41.379593+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:43:41.379593+0000","last_undegraded":"2024-09-21T11:43:41.379593+0000","last_fullsized":"2024-09-21T11:43:41.379593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:42.290188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:42.290188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":218,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.370095+0000","last_change":"2024-09-21T11:42:37.191052+0000","last_active":"2024-09-21T11:43:41.370095+0000","last_peered":"2024-09-21T11:43:41.370095+0000","last_clean":"2024-09-21T11:43:41.370095+0000","last_became_active":"2024-09-21T11:42:37.164858+0000","last_became_peered":"2024-09-21T11:42:37.164858+0000","last_unstale":"2024-09-21T11:43:41.370095+0000","last_undegraded":"2024-09-21T11:43:41.370095+0000","last_fullsized":"2024-09-21T11:43:41.370095+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:26.994148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:26.994148+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:16.329886+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":82,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.383715+0000","last_change":"2024-09-21T11:43:37.050216+0000","last_active":"2024-09-21T11:43:41.383715+0000","last_peered":"2024-09-21T11:43:41.383715+0000","last_clean":"2024-09-21T11:43:41.383715+0000","last_became_active":"2024-09-21T11:42:37.161713+0000","last_became_peered":"2024-09-21T11:42:37.161713+0000","last_unstale":"2024-09-21T11:43:41.383715+0000","last_undegraded":"2024-09-21T11:43:41.383715+0000","last_fullsized":"2024-09-21T11:43:41.383715+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:52.112502+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":184,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.371132+0000","last_change":"2024-09-21T11:42:59.201398+0000","last_active":"2024-09-21T11:43:41.371132+0000","last_peered":"2024-09-21T11:43:41.371132+0000","last_clean":"2024-09-21T11:43:41.371132+0000","last_became_active":"2024-09-21T11:42:37.158907+0000","last_became_peered":"2024-09-21T11:42:37.158907+0000","last_unstale":"2024-09-21T11:43:41.371132+0000","last_undegraded":"2024-09-21T11:43:41.371132+0000","last_fullsized":"2024-09-21T11:43:41.371132+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:59.201330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:59.201330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:02.317360+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":177,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374321+0000","last_change":"2024-09-21T11:43:28.461458+0000","last_active":"2024-09-21T11:43:41.374321+0000","last_peered":"2024-09-21T11:43:41.374321+0000","last_clean":"2024-09-21T11:43:41.374321+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:43:41.374321+0000","last_undegraded":"2024-09-21T11:43:41.374321+0000","last_fullsized":"2024-09-21T11:43:41.374321+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.461378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.461378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.850394+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":181,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374056+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:43:41.374056+0000","last_peered":"2024-09-21T11:43:41.374056+0000","last_clean":"2024-09-21T11:43:41.374056+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:43:41.374056+0000","last_undegraded":"2024-09-21T11:43:41.374056+0000","last_fullsized":"2024-09-21T11:43:41.374056+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":202,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372246+0000","last_change":"2024-09-21T11:40:02.315669+0000","last_active":"2024-09-21T11:43:41.372246+0000","last_peered":"2024-09-21T11:43:41.372246+0000","last_clean":"2024-09-21T11:43:41.372246+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:43:41.372246+0000","last_undegraded":"2024-09-21T11:43:41.372246+0000","last_fullsized":"2024-09-21T11:43:41.372246+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":196,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.369955+0000","last_change":"2024-09-21T11:43:04.373004+0000","last_active":"2024-09-21T11:43:41.369955+0000","last_peered":"2024-09-21T11:43:41.369955+0000","last_clean":"2024-09-21T11:43:41.369955+0000","last_became_active":"2024-09-21T11:42:37.154511+0000","last_became_peered":"2024-09-21T11:42:37.154511+0000","last_unstale":"2024-09-21T11:43:41.369955+0000","last_undegraded":"2024-09-21T11:43:41.369955+0000","last_fullsized":"2024-09-21T11:43:41.369955+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:04.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:04.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:06.903626+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384895+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:43:41.384895+0000","last_peered":"2024-09-21T11:43:41.384895+0000","last_clean":"2024-09-21T11:43:41.384895+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:43:41.384895+0000","last_undegraded":"2024-09-21T11:43:41.384895+0000","last_fullsized":"2024-09-21T11:43:41.384895+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":209,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.371284+0000","last_change":"2024-09-21T11:43:26.258023+0000","last_active":"2024-09-21T11:43:41.371284+0000","last_peered":"2024-09-21T11:43:41.371284+0000","last_clean":"2024-09-21T11:43:41.371284+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:43:41.371284+0000","last_undegraded":"2024-09-21T11:43:41.371284+0000","last_fullsized":"2024-09-21T11:43:41.371284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:26.257967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.257967+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:51.865812+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":180,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385201+0000","last_change":"2024-09-21T11:41:32.885455+0000","last_active":"2024-09-21T11:43:41.385201+0000","last_peered":"2024-09-21T11:43:41.385201+0000","last_clean":"2024-09-21T11:43:41.385201+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:43:41.385201+0000","last_undegraded":"2024-09-21T11:43:41.385201+0000","last_fullsized":"2024-09-21T11:43:41.385201+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:32.885407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:32.885407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":183,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:44.648053+0000","last_change":"2024-09-21T11:43:44.648053+0000","last_active":"2024-09-21T11:43:44.648053+0000","last_peered":"2024-09-21T11:43:44.648053+0000","last_clean":"2024-09-21T11:43:44.648053+0000","last_became_active":"2024-09-21T11:42:37.155879+0000","last_became_peered":"2024-09-21T11:42:37.155879+0000","last_unstale":"2024-09-21T11:43:44.648053+0000","last_undegraded":"2024-09-21T11:43:44.648053+0000","last_fullsized":"2024-09-21T11:43:44.648053+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.647985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.647985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:56.842445+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":203,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384171+0000","last_change":"2024-09-21T11:42:37.156936+0000","last_active":"2024-09-21T11:43:41.384171+0000","last_peered":"2024-09-21T11:43:41.384171+0000","last_clean":"2024-09-21T11:43:41.384171+0000","last_became_active":"2024-09-21T11:42:37.156575+0000","last_became_peered":"2024-09-21T11:42:37.156575+0000","last_unstale":"2024-09-21T11:43:41.384171+0000","last_undegraded":"2024-09-21T11:43:41.384171+0000","last_fullsized":"2024-09-21T11:43:41.384171+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.101955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.101955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":216,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.371365+0000","last_change":"2024-09-21T11:43:08.326923+0000","last_active":"2024-09-21T11:43:41.371365+0000","last_peered":"2024-09-21T11:43:41.371365+0000","last_clean":"2024-09-21T11:43:41.371365+0000","last_became_active":"2024-09-21T11:41:31.634863+0000","last_became_peered":"2024-09-21T11:41:31.634863+0000","last_unstale":"2024-09-21T11:43:41.371365+0000","last_undegraded":"2024-09-21T11:43:41.371365+0000","last_fullsized":"2024-09-21T11:43:41.371365+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:08.326846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:08.326846+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:14.001930+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":187,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374510+0000","last_change":"2024-09-21T11:43:38.567194+0000","last_active":"2024-09-21T11:43:41.374510+0000","last_peered":"2024-09-21T11:43:41.374510+0000","last_clean":"2024-09-21T11:43:41.374510+0000","last_became_active":"2024-09-21T11:42:37.164285+0000","last_became_peered":"2024-09-21T11:42:37.164285+0000","last_unstale":"2024-09-21T11:43:41.374510+0000","last_undegraded":"2024-09-21T11:43:41.374510+0000","last_fullsized":"2024-09-21T11:43:41.374510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.567131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.567131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.325966+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":183,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.383752+0000","last_change":"2024-09-21T11:42:37.163213+0000","last_active":"2024-09-21T11:43:41.383752+0000","last_peered":"2024-09-21T11:43:41.383752+0000","last_clean":"2024-09-21T11:43:41.383752+0000","last_became_active":"2024-09-21T11:42:37.162912+0000","last_became_peered":"2024-09-21T11:42:37.162912+0000","last_unstale":"2024-09-21T11:43:41.383752+0000","last_undegraded":"2024-09-21T11:43:41.383752+0000","last_fullsized":"2024-09-21T11:43:41.383752+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:12.028860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:12.028860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":185,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372050+0000","last_change":"2024-09-21T11:43:22.300966+0000","last_active":"2024-09-21T11:43:41.372050+0000","last_peered":"2024-09-21T11:43:41.372050+0000","last_clean":"2024-09-21T11:43:41.372050+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:43:41.372050+0000","last_undegraded":"2024-09-21T11:43:41.372050+0000","last_fullsized":"2024-09-21T11:43:41.372050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:22.300885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:22.300885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.343801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.378974+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:43:41.378974+0000","last_peered":"2024-09-21T11:43:41.378974+0000","last_clean":"2024-09-21T11:43:41.378974+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:43:41.378974+0000","last_undegraded":"2024-09-21T11:43:41.378974+0000","last_fullsized":"2024-09-21T11:43:41.378974+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:50.720157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":264,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385226+0000","last_change":"2024-09-21T11:43:31.498356+0000","last_active":"2024-09-21T11:43:41.385226+0000","last_peered":"2024-09-21T11:43:41.385226+0000","last_clean":"2024-09-21T11:43:41.385226+0000","last_became_active":"2024-09-21T11:42:37.166901+0000","last_became_peered":"2024-09-21T11:42:37.166901+0000","last_unstale":"2024-09-21T11:43:41.385226+0000","last_undegraded":"2024-09-21T11:43:41.385226+0000","last_fullsized":"2024-09-21T11:43:41.385226+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:31.498142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.498142+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:33.717651+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":82,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:43.147999+0000","last_change":"2024-09-21T11:43:43.147999+0000","last_active":"2024-09-21T11:43:43.147999+0000","last_peered":"2024-09-21T11:43:43.147999+0000","last_clean":"2024-09-21T11:43:43.147999+0000","last_became_active":"2024-09-21T11:42:37.153799+0000","last_became_peered":"2024-09-21T11:42:37.153799+0000","last_unstale":"2024-09-21T11:43:43.147999+0000","last_undegraded":"2024-09-21T11:43:43.147999+0000","last_fullsized":"2024-09-21T11:43:43.147999+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:43.147942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:43.147942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:46.559755+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":177,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.380638+0000","last_change":"2024-09-21T11:41:40.448106+0000","last_active":"2024-09-21T11:43:41.380638+0000","last_peered":"2024-09-21T11:43:41.380638+0000","last_clean":"2024-09-21T11:43:41.380638+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:43:41.380638+0000","last_undegraded":"2024-09-21T11:43:41.380638+0000","last_fullsized":"2024-09-21T11:43:41.380638+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.447908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.447908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":196,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.093396+0000","last_change":"2024-09-21T11:43:45.093396+0000","last_active":"2024-09-21T11:43:45.093396+0000","last_peered":"2024-09-21T11:43:45.093396+0000","last_clean":"2024-09-21T11:43:45.093396+0000","last_became_active":"2024-09-21T11:42:37.153694+0000","last_became_peered":"2024-09-21T11:42:37.153694+0000","last_unstale":"2024-09-21T11:43:45.093396+0000","last_undegraded":"2024-09-21T11:43:45.093396+0000","last_fullsized":"2024-09-21T11:43:45.093396+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.093325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.093325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:14.273876+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":227,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.376478+0000","last_change":"2024-09-21T11:42:37.189830+0000","last_active":"2024-09-21T11:43:41.376478+0000","last_peered":"2024-09-21T11:43:41.376478+0000","last_clean":"2024-09-21T11:43:41.376478+0000","last_became_active":"2024-09-21T11:42:37.165964+0000","last_became_peered":"2024-09-21T11:42:37.165964+0000","last_unstale":"2024-09-21T11:43:41.376478+0000","last_undegraded":"2024-09-21T11:43:41.376478+0000","last_fullsized":"2024-09-21T11:43:41.376478+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:41:35.427210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:35.427210+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":181,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374314+0000","last_change":"2024-09-21T11:43:02.320628+0000","last_active":"2024-09-21T11:43:41.374314+0000","last_peered":"2024-09-21T11:43:41.374314+0000","last_clean":"2024-09-21T11:43:41.374314+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:43:41.374314+0000","last_undegraded":"2024-09-21T11:43:41.374314+0000","last_fullsized":"2024-09-21T11:43:41.374314+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:02.320559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:02.320559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:24.560814+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":185,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373396+0000","last_change":"2024-09-21T11:42:31.278897+0000","last_active":"2024-09-21T11:43:41.373396+0000","last_peered":"2024-09-21T11:43:41.373396+0000","last_clean":"2024-09-21T11:43:41.373396+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:43:41.373396+0000","last_undegraded":"2024-09-21T11:43:41.373396+0000","last_fullsized":"2024-09-21T11:43:41.373396+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:42:31.278824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:31.278824+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":86,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372155+0000","last_change":"2024-09-21T11:42:37.185608+0000","last_active":"2024-09-21T11:43:41.372155+0000","last_peered":"2024-09-21T11:43:41.372155+0000","last_clean":"2024-09-21T11:43:41.372155+0000","last_became_active":"2024-09-21T11:42:37.166144+0000","last_became_peered":"2024-09-21T11:42:37.166144+0000","last_unstale":"2024-09-21T11:43:41.372155+0000","last_undegraded":"2024-09-21T11:43:41.372155+0000","last_fullsized":"2024-09-21T11:43:41.372155+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:23.349301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.349301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:45.201792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":199,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.379314+0000","last_change":"2024-09-21T11:43:29.595122+0000","last_active":"2024-09-21T11:43:41.379314+0000","last_peered":"2024-09-21T11:43:41.379314+0000","last_clean":"2024-09-21T11:43:41.379314+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:43:41.379314+0000","last_undegraded":"2024-09-21T11:43:41.379314+0000","last_fullsized":"2024-09-21T11:43:41.379314+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:43:29.595040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.595040+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:41.200221+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":245,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.378776+0000","last_change":"2024-09-21T11:42:14.814180+0000","last_active":"2024-09-21T11:43:41.378776+0000","last_peered":"2024-09-21T11:43:41.378776+0000","last_clean":"2024-09-21T11:43:41.378776+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:43:41.378776+0000","last_undegraded":"2024-09-21T11:43:41.378776+0000","last_fullsized":"2024-09-21T11:43:41.378776+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:14.814041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:14.814041+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"143'10723","reported_seq":10914,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.692181+0000","last_change":"2024-09-21T11:43:35.987574+0000","last_active":"2024-09-21T11:43:45.692181+0000","last_peered":"2024-09-21T11:43:45.692181+0000","last_clean":"2024-09-21T11:43:45.692181+0000","last_became_active":"2024-09-21T11:41:10.304754+0000","last_became_peered":"2024-09-21T11:41:10.304754+0000","last_unstale":"2024-09-21T11:43:45.692181+0000","last_undegraded":"2024-09-21T11:43:45.692181+0000","last_fullsized":"2024-09-21T11:43:45.692181+0000","mapping_epoch":82,"log_start":"104'5746","ondisk_log_start":"104'5746","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'9957","last_scrub_stamp":"2024-09-21T11:43:35.987394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:35.987394+0000","objects_scrubbed":4941,"log_size":4977,"log_dups_size":0,"ondisk_log_size":4977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:47.433252+0000","scrub_duration":1245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270991360,"num_objects":4135,"num_object_clones":0,"num_object_copies":12405,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10723,"num_write_kb":475456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:44.440472+0000","last_change":"2024-09-21T11:43:44.440472+0000","last_active":"2024-09-21T11:43:44.440472+0000","last_peered":"2024-09-21T11:43:44.440472+0000","last_clean":"2024-09-21T11:43:44.440472+0000","last_became_active":"2024-09-21T11:42:37.169939+0000","last_became_peered":"2024-09-21T11:42:37.169939+0000","last_unstale":"2024-09-21T11:43:44.440472+0000","last_undegraded":"2024-09-21T11:43:44.440472+0000","last_fullsized":"2024-09-21T11:43:44.440472+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:44.440374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.440374+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:10.450444+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":196,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372669+0000","last_change":"2024-09-21T11:42:37.172945+0000","last_active":"2024-09-21T11:43:41.372669+0000","last_peered":"2024-09-21T11:43:41.372669+0000","last_clean":"2024-09-21T11:43:41.372669+0000","last_became_active":"2024-09-21T11:42:37.171365+0000","last_became_peered":"2024-09-21T11:42:37.171365+0000","last_unstale":"2024-09-21T11:43:41.372669+0000","last_undegraded":"2024-09-21T11:43:41.372669+0000","last_fullsized":"2024-09-21T11:43:41.372669+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.454207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.454207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":235,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372176+0000","last_change":"2024-09-21T11:42:26.008760+0000","last_active":"2024-09-21T11:43:41.372176+0000","last_peered":"2024-09-21T11:43:41.372176+0000","last_clean":"2024-09-21T11:43:41.372176+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:43:41.372176+0000","last_undegraded":"2024-09-21T11:43:41.372176+0000","last_fullsized":"2024-09-21T11:43:41.372176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:26.008646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:26.008646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:48.227954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":183,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.383711+0000","last_change":"2024-09-21T11:42:37.157278+0000","last_active":"2024-09-21T11:43:41.383711+0000","last_peered":"2024-09-21T11:43:41.383711+0000","last_clean":"2024-09-21T11:43:41.383711+0000","last_became_active":"2024-09-21T11:42:37.156952+0000","last_became_peered":"2024-09-21T11:42:37.156952+0000","last_unstale":"2024-09-21T11:43:41.383711+0000","last_undegraded":"2024-09-21T11:43:41.383711+0000","last_fullsized":"2024-09-21T11:43:41.383711+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.066324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.066324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"143'10586","reported_seq":10720,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:46.099185+0000","last_change":"2024-09-21T11:41:10.297546+0000","last_active":"2024-09-21T11:43:46.099185+0000","last_peered":"2024-09-21T11:43:46.099185+0000","last_clean":"2024-09-21T11:43:46.099185+0000","last_became_active":"2024-09-21T11:41:10.297301+0000","last_became_peered":"2024-09-21T11:41:10.297301+0000","last_unstale":"2024-09-21T11:43:46.099185+0000","last_undegraded":"2024-09-21T11:43:46.099185+0000","last_fullsized":"2024-09-21T11:43:46.099185+0000","mapping_epoch":82,"log_start":"108'6351","ondisk_log_start":"108'6351","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4235,"log_dups_size":0,"ondisk_log_size":4235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258080768,"num_objects":3938,"num_object_clones":0,"num_object_copies":11814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10586,"num_write_kb":464768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":217,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372466+0000","last_change":"2024-09-21T11:43:40.423834+0000","last_active":"2024-09-21T11:43:41.372466+0000","last_peered":"2024-09-21T11:43:41.372466+0000","last_clean":"2024-09-21T11:43:41.372466+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:43:41.372466+0000","last_undegraded":"2024-09-21T11:43:41.372466+0000","last_fullsized":"2024-09-21T11:43:41.372466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:43:40.423750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.423750+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:01.136315+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":189,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.381074+0000","last_change":"2024-09-21T11:41:29.388381+0000","last_active":"2024-09-21T11:43:41.381074+0000","last_peered":"2024-09-21T11:43:41.381074+0000","last_clean":"2024-09-21T11:43:41.381074+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:43:41.381074+0000","last_undegraded":"2024-09-21T11:43:41.381074+0000","last_fullsized":"2024-09-21T11:43:41.381074+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:29.388322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:29.388322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":181,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.370379+0000","last_change":"2024-09-21T11:41:40.030553+0000","last_active":"2024-09-21T11:43:41.370379+0000","last_peered":"2024-09-21T11:43:41.370379+0000","last_clean":"2024-09-21T11:43:41.370379+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:43:41.370379+0000","last_undegraded":"2024-09-21T11:43:41.370379+0000","last_fullsized":"2024-09-21T11:43:41.370379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.030493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.030493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:11.327373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":169,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.371255+0000","last_change":"2024-09-21T11:40:06.343742+0000","last_active":"2024-09-21T11:43:41.371255+0000","last_peered":"2024-09-21T11:43:41.371255+0000","last_clean":"2024-09-21T11:43:41.371255+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:43:41.371255+0000","last_undegraded":"2024-09-21T11:43:41.371255+0000","last_fullsized":"2024-09-21T11:43:41.371255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:10.356658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"143'10485","reported_seq":23228,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:46.099668+0000","last_change":"2024-09-21T11:43:25.943980+0000","last_active":"2024-09-21T11:43:46.099668+0000","last_peered":"2024-09-21T11:43:46.099668+0000","last_clean":"2024-09-21T11:43:46.099668+0000","last_became_active":"2024-09-21T11:43:25.930319+0000","last_became_peered":"2024-09-21T11:43:25.930319+0000","last_unstale":"2024-09-21T11:43:46.099668+0000","last_undegraded":"2024-09-21T11:43:46.099668+0000","last_fullsized":"2024-09-21T11:43:46.099668+0000","mapping_epoch":133,"log_start":"108'6214","ondisk_log_start":"108'6214","created":82,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4271,"log_dups_size":0,"ondisk_log_size":4271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258670592,"num_objects":3947,"num_object_clones":0,"num_object_copies":11841,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10485,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6296,"num_bytes_recovered":412614656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":206,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.370609+0000","last_change":"2024-09-21T11:41:41.007761+0000","last_active":"2024-09-21T11:43:41.370609+0000","last_peered":"2024-09-21T11:43:41.370609+0000","last_clean":"2024-09-21T11:43:41.370609+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:43:41.370609+0000","last_undegraded":"2024-09-21T11:43:41.370609+0000","last_fullsized":"2024-09-21T11:43:41.370609+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:41:41.007695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:41.007695+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:15.335018+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":200,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.379509+0000","last_change":"2024-09-21T11:42:37.182056+0000","last_active":"2024-09-21T11:43:41.379509+0000","last_peered":"2024-09-21T11:43:41.379509+0000","last_clean":"2024-09-21T11:43:41.379509+0000","last_became_active":"2024-09-21T11:42:37.159390+0000","last_became_peered":"2024-09-21T11:42:37.159390+0000","last_unstale":"2024-09-21T11:43:41.379509+0000","last_undegraded":"2024-09-21T11:43:41.379509+0000","last_fullsized":"2024-09-21T11:43:41.379509+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:41:34.236667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:34.236667+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"114'9","reported_seq":247,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372274+0000","last_change":"2024-09-21T11:43:29.417021+0000","last_active":"2024-09-21T11:43:41.372274+0000","last_peered":"2024-09-21T11:43:41.372274+0000","last_clean":"2024-09-21T11:43:41.372274+0000","last_became_active":"2024-09-21T11:42:37.169838+0000","last_became_peered":"2024-09-21T11:42:37.169838+0000","last_unstale":"2024-09-21T11:43:41.372274+0000","last_undegraded":"2024-09-21T11:43:41.372274+0000","last_fullsized":"2024-09-21T11:43:41.372274+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:29.416946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.416946+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:30.090032+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":179,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374312+0000","last_change":"2024-09-21T11:41:39.527210+0000","last_active":"2024-09-21T11:43:41.374312+0000","last_peered":"2024-09-21T11:43:41.374312+0000","last_clean":"2024-09-21T11:43:41.374312+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:43:41.374312+0000","last_undegraded":"2024-09-21T11:43:41.374312+0000","last_fullsized":"2024-09-21T11:43:41.374312+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.527151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.527151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"143'10587","reported_seq":10795,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.711543+0000","last_change":"2024-09-21T11:43:05.145633+0000","last_active":"2024-09-21T11:43:45.711543+0000","last_peered":"2024-09-21T11:43:45.711543+0000","last_clean":"2024-09-21T11:43:45.711543+0000","last_became_active":"2024-09-21T11:41:10.300541+0000","last_became_peered":"2024-09-21T11:41:10.300541+0000","last_unstale":"2024-09-21T11:43:45.711543+0000","last_undegraded":"2024-09-21T11:43:45.711543+0000","last_fullsized":"2024-09-21T11:43:45.711543+0000","mapping_epoch":82,"log_start":"106'6033","ondisk_log_start":"106'6033","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'8200","last_scrub_stamp":"2024-09-21T11:43:05.145547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:05.145547+0000","objects_scrubbed":6512,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.858838+0000","scrub_duration":1824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267059200,"num_objects":4075,"num_object_clones":0,"num_object_copies":12225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10587,"num_write_kb":469184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":120,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385510+0000","last_change":"2024-09-21T11:42:37.207015+0000","last_active":"2024-09-21T11:43:41.385510+0000","last_peered":"2024-09-21T11:43:41.385510+0000","last_clean":"2024-09-21T11:43:41.385510+0000","last_became_active":"2024-09-21T11:42:37.165120+0000","last_became_peered":"2024-09-21T11:42:37.165120+0000","last_unstale":"2024-09-21T11:43:41.385510+0000","last_undegraded":"2024-09-21T11:43:41.385510+0000","last_fullsized":"2024-09-21T11:43:41.385510+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:15.360632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.360632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:42.919793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384983+0000","last_change":"2024-09-21T11:41:40.866256+0000","last_active":"2024-09-21T11:43:41.384983+0000","last_peered":"2024-09-21T11:43:41.384983+0000","last_clean":"2024-09-21T11:43:41.384983+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:43:41.384983+0000","last_undegraded":"2024-09-21T11:43:41.384983+0000","last_fullsized":"2024-09-21T11:43:41.384983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.866156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.866156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":237,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.378637+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:43:41.378637+0000","last_peered":"2024-09-21T11:43:41.378637+0000","last_clean":"2024-09-21T11:43:41.378637+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:43:41.378637+0000","last_undegraded":"2024-09-21T11:43:41.378637+0000","last_fullsized":"2024-09-21T11:43:41.378637+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":176,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.377633+0000","last_change":"2024-09-21T11:41:40.278404+0000","last_active":"2024-09-21T11:43:41.377633+0000","last_peered":"2024-09-21T11:43:41.377633+0000","last_clean":"2024-09-21T11:43:41.377633+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:43:41.377633+0000","last_undegraded":"2024-09-21T11:43:41.377633+0000","last_fullsized":"2024-09-21T11:43:41.377633+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.278315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.278315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"143'10607","reported_seq":23447,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.751947+0000","last_change":"2024-09-21T11:43:27.957009+0000","last_active":"2024-09-21T11:43:45.751947+0000","last_peered":"2024-09-21T11:43:45.751947+0000","last_clean":"2024-09-21T11:43:45.751947+0000","last_became_active":"2024-09-21T11:43:27.938453+0000","last_became_peered":"2024-09-21T11:43:27.938453+0000","last_unstale":"2024-09-21T11:43:45.751947+0000","last_undegraded":"2024-09-21T11:43:45.751947+0000","last_fullsized":"2024-09-21T11:43:45.751947+0000","mapping_epoch":135,"log_start":"102'5364","ondisk_log_start":"102'5364","created":82,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":5243,"log_dups_size":0,"ondisk_log_size":5243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":269156352,"num_objects":4107,"num_object_clones":0,"num_object_copies":12321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10607,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6345,"num_bytes_recovered":415825920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":220,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.378685+0000","last_change":"2024-09-21T11:41:07.065583+0000","last_active":"2024-09-21T11:43:41.378685+0000","last_peered":"2024-09-21T11:43:41.378685+0000","last_clean":"2024-09-21T11:43:41.378685+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:43:41.378685+0000","last_undegraded":"2024-09-21T11:43:41.378685+0000","last_fullsized":"2024-09-21T11:43:41.378685+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:41:07.065507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.065507+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":81,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385327+0000","last_change":"2024-09-21T11:43:30.509571+0000","last_active":"2024-09-21T11:43:41.385327+0000","last_peered":"2024-09-21T11:43:41.385327+0000","last_clean":"2024-09-21T11:43:41.385327+0000","last_became_active":"2024-09-21T11:42:37.157562+0000","last_became_peered":"2024-09-21T11:42:37.157562+0000","last_unstale":"2024-09-21T11:43:41.385327+0000","last_undegraded":"2024-09-21T11:43:41.385327+0000","last_fullsized":"2024-09-21T11:43:41.385327+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.509505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.509505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:42.953900+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":180,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.377805+0000","last_change":"2024-09-21T11:43:26.704117+0000","last_active":"2024-09-21T11:43:41.377805+0000","last_peered":"2024-09-21T11:43:41.377805+0000","last_clean":"2024-09-21T11:43:41.377805+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:43:41.377805+0000","last_undegraded":"2024-09-21T11:43:41.377805+0000","last_fullsized":"2024-09-21T11:43:41.377805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.704073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.704073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:55.097023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":81,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.377805+0000","last_change":"2024-09-21T11:43:37.819912+0000","last_active":"2024-09-21T11:43:41.377805+0000","last_peered":"2024-09-21T11:43:41.377805+0000","last_clean":"2024-09-21T11:43:41.377805+0000","last_became_active":"2024-09-21T11:42:37.155437+0000","last_became_peered":"2024-09-21T11:42:37.155437+0000","last_unstale":"2024-09-21T11:43:41.377805+0000","last_undegraded":"2024-09-21T11:43:41.377805+0000","last_fullsized":"2024-09-21T11:43:41.377805+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.544593+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"143'10829","reported_seq":11031,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.711625+0000","last_change":"2024-09-21T11:42:22.352124+0000","last_active":"2024-09-21T11:43:45.711625+0000","last_peered":"2024-09-21T11:43:45.711625+0000","last_clean":"2024-09-21T11:43:45.711625+0000","last_became_active":"2024-09-21T11:41:10.304723+0000","last_became_peered":"2024-09-21T11:41:10.304723+0000","last_unstale":"2024-09-21T11:43:45.711625+0000","last_undegraded":"2024-09-21T11:43:45.711625+0000","last_fullsized":"2024-09-21T11:43:45.711625+0000","mapping_epoch":82,"log_start":"104'5863","ondisk_log_start":"104'5863","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'6088","last_scrub_stamp":"2024-09-21T11:42:22.351940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.351940+0000","objects_scrubbed":6032,"log_size":4966,"log_dups_size":0,"ondisk_log_size":4966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270729216,"num_objects":4131,"num_object_clones":0,"num_object_copies":12393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10829,"num_write_kb":478720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"140'30","reported_seq":217,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385171+0000","last_change":"2024-09-21T11:43:37.416987+0000","last_active":"2024-09-21T11:43:41.385171+0000","last_peered":"2024-09-21T11:43:41.385171+0000","last_clean":"2024-09-21T11:43:41.385171+0000","last_became_active":"2024-09-21T11:43:32.003036+0000","last_became_peered":"2024-09-21T11:43:32.003036+0000","last_unstale":"2024-09-21T11:43:41.385171+0000","last_undegraded":"2024-09-21T11:43:41.385171+0000","last_fullsized":"2024-09-21T11:43:41.385171+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.678954+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":195,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.371588+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:43:41.371588+0000","last_peered":"2024-09-21T11:43:41.371588+0000","last_clean":"2024-09-21T11:43:41.371588+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:43:41.371588+0000","last_undegraded":"2024-09-21T11:43:41.371588+0000","last_fullsized":"2024-09-21T11:43:41.371588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:14.022118+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":209,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.386148+0000","last_change":"2024-09-21T11:42:37.193243+0000","last_active":"2024-09-21T11:43:41.386148+0000","last_peered":"2024-09-21T11:43:41.386148+0000","last_clean":"2024-09-21T11:43:41.386148+0000","last_became_active":"2024-09-21T11:42:37.164723+0000","last_became_peered":"2024-09-21T11:42:37.164723+0000","last_unstale":"2024-09-21T11:43:41.386148+0000","last_undegraded":"2024-09-21T11:43:41.386148+0000","last_fullsized":"2024-09-21T11:43:41.386148+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:42:17.142454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.142454+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":187,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.378523+0000","last_change":"2024-09-21T11:42:37.164162+0000","last_active":"2024-09-21T11:43:41.378523+0000","last_peered":"2024-09-21T11:43:41.378523+0000","last_clean":"2024-09-21T11:43:41.378523+0000","last_became_active":"2024-09-21T11:42:37.163682+0000","last_became_peered":"2024-09-21T11:42:37.163682+0000","last_unstale":"2024-09-21T11:43:41.378523+0000","last_undegraded":"2024-09-21T11:43:41.378523+0000","last_fullsized":"2024-09-21T11:43:41.378523+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:24.468909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.468909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.266739+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":177,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373870+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:43:41.373870+0000","last_peered":"2024-09-21T11:43:41.373870+0000","last_clean":"2024-09-21T11:43:41.373870+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:43:41.373870+0000","last_undegraded":"2024-09-21T11:43:41.373870+0000","last_fullsized":"2024-09-21T11:43:41.373870+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:50.884536+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"143'10475","reported_seq":23579,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:46.194253+0000","last_change":"2024-09-21T11:43:25.018476+0000","last_active":"2024-09-21T11:43:46.194253+0000","last_peered":"2024-09-21T11:43:46.194253+0000","last_clean":"2024-09-21T11:43:46.194253+0000","last_became_active":"2024-09-21T11:43:24.947584+0000","last_became_peered":"2024-09-21T11:43:24.947584+0000","last_unstale":"2024-09-21T11:43:46.194253+0000","last_undegraded":"2024-09-21T11:43:46.194253+0000","last_fullsized":"2024-09-21T11:43:46.194253+0000","mapping_epoch":132,"log_start":"106'5920","ondisk_log_start":"106'5920","created":82,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4555,"log_dups_size":0,"ondisk_log_size":4555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267583488,"num_objects":4083,"num_object_clones":0,"num_object_copies":12249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10475,"num_write_kb":465856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6476,"num_bytes_recovered":424280064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":132,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385646+0000","last_change":"2024-09-21T11:42:37.188687+0000","last_active":"2024-09-21T11:43:41.385646+0000","last_peered":"2024-09-21T11:43:41.385646+0000","last_clean":"2024-09-21T11:43:41.385646+0000","last_became_active":"2024-09-21T11:42:37.161797+0000","last_became_peered":"2024-09-21T11:42:37.161797+0000","last_unstale":"2024-09-21T11:43:41.385646+0000","last_undegraded":"2024-09-21T11:43:41.385646+0000","last_fullsized":"2024-09-21T11:43:41.385646+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":86,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385922+0000","last_change":"2024-09-21T11:42:37.182044+0000","last_active":"2024-09-21T11:43:41.385922+0000","last_peered":"2024-09-21T11:43:41.385922+0000","last_clean":"2024-09-21T11:43:41.385922+0000","last_became_active":"2024-09-21T11:42:37.156191+0000","last_became_peered":"2024-09-21T11:42:37.156191+0000","last_unstale":"2024-09-21T11:43:41.385922+0000","last_undegraded":"2024-09-21T11:43:41.385922+0000","last_fullsized":"2024-09-21T11:43:41.385922+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:52.863101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":206,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384480+0000","last_change":"2024-09-21T11:43:36.446391+0000","last_active":"2024-09-21T11:43:41.384480+0000","last_peered":"2024-09-21T11:43:41.384480+0000","last_clean":"2024-09-21T11:43:41.384480+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:43:41.384480+0000","last_undegraded":"2024-09-21T11:43:41.384480+0000","last_fullsized":"2024-09-21T11:43:41.384480+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:43:36.446341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.446341+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.312398+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":189,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384516+0000","last_change":"2024-09-21T11:43:28.505134+0000","last_active":"2024-09-21T11:43:41.384516+0000","last_peered":"2024-09-21T11:43:41.384516+0000","last_clean":"2024-09-21T11:43:41.384516+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:43:41.384516+0000","last_undegraded":"2024-09-21T11:43:41.384516+0000","last_fullsized":"2024-09-21T11:43:41.384516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.504894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.504894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.779358+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":176,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373737+0000","last_change":"2024-09-21T11:42:25.245624+0000","last_active":"2024-09-21T11:43:41.373737+0000","last_peered":"2024-09-21T11:43:41.373737+0000","last_clean":"2024-09-21T11:43:41.373737+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:43:41.373737+0000","last_undegraded":"2024-09-21T11:43:41.373737+0000","last_fullsized":"2024-09-21T11:43:41.373737+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.245565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.245565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"143'10708","reported_seq":10892,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.715519+0000","last_change":"2024-09-21T11:43:44.550062+0000","last_active":"2024-09-21T11:43:45.715519+0000","last_peered":"2024-09-21T11:43:45.715519+0000","last_clean":"2024-09-21T11:43:45.715519+0000","last_became_active":"2024-09-21T11:41:10.302903+0000","last_became_peered":"2024-09-21T11:41:10.302903+0000","last_unstale":"2024-09-21T11:43:45.715519+0000","last_undegraded":"2024-09-21T11:43:45.715519+0000","last_fullsized":"2024-09-21T11:43:45.715519+0000","mapping_epoch":82,"log_start":"107'6228","ondisk_log_start":"107'6228","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'10628","last_scrub_stamp":"2024-09-21T11:43:44.550004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.550004+0000","objects_scrubbed":4219,"log_size":4480,"log_dups_size":0,"ondisk_log_size":4480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:05.776791+0000","scrub_duration":1037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":269352960,"num_objects":4110,"num_object_clones":0,"num_object_copies":12330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10708,"num_write_kb":474176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":223,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.379408+0000","last_change":"2024-09-21T11:43:40.607996+0000","last_active":"2024-09-21T11:43:41.379408+0000","last_peered":"2024-09-21T11:43:41.379408+0000","last_clean":"2024-09-21T11:43:41.379408+0000","last_became_active":"2024-09-21T11:42:37.164059+0000","last_became_peered":"2024-09-21T11:42:37.164059+0000","last_unstale":"2024-09-21T11:43:41.379408+0000","last_undegraded":"2024-09-21T11:43:41.379408+0000","last_fullsized":"2024-09-21T11:43:41.379408+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:43:40.607801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.607801+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:59.037561+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:42.641937+0000","last_change":"2024-09-21T11:43:42.641937+0000","last_active":"2024-09-21T11:43:42.641937+0000","last_peered":"2024-09-21T11:43:42.641937+0000","last_clean":"2024-09-21T11:43:42.641937+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:43:42.641937+0000","last_undegraded":"2024-09-21T11:43:42.641937+0000","last_fullsized":"2024-09-21T11:43:42.641937+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:42.641865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:42.641865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:44.867283+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":430,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:44.268306+0000","last_change":"2024-09-21T11:42:37.176176+0000","last_active":"2024-09-21T11:43:44.268306+0000","last_peered":"2024-09-21T11:43:44.268306+0000","last_clean":"2024-09-21T11:43:44.268306+0000","last_became_active":"2024-09-21T11:42:37.166141+0000","last_became_peered":"2024-09-21T11:42:37.166141+0000","last_unstale":"2024-09-21T11:43:44.268306+0000","last_undegraded":"2024-09-21T11:43:44.268306+0000","last_fullsized":"2024-09-21T11:43:44.268306+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:41:36.501543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.501543+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":219,"num_read_kb":214,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":187,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.046195+0000","last_change":"2024-09-21T11:43:45.046195+0000","last_active":"2024-09-21T11:43:45.046195+0000","last_peered":"2024-09-21T11:43:45.046195+0000","last_clean":"2024-09-21T11:43:45.046195+0000","last_became_active":"2024-09-21T11:42:37.172270+0000","last_became_peered":"2024-09-21T11:42:37.172270+0000","last_unstale":"2024-09-21T11:43:45.046195+0000","last_undegraded":"2024-09-21T11:43:45.046195+0000","last_fullsized":"2024-09-21T11:43:45.046195+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.046128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.046128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:59.675637+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":73,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384252+0000","last_change":"2024-09-21T11:42:37.173082+0000","last_active":"2024-09-21T11:43:41.384252+0000","last_peered":"2024-09-21T11:43:41.384252+0000","last_clean":"2024-09-21T11:43:41.384252+0000","last_became_active":"2024-09-21T11:42:37.172645+0000","last_became_peered":"2024-09-21T11:42:37.172645+0000","last_unstale":"2024-09-21T11:43:41.384252+0000","last_undegraded":"2024-09-21T11:43:41.384252+0000","last_fullsized":"2024-09-21T11:43:41.384252+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:31.250421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:31.250421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"143'10810","reported_seq":10992,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:46.188298+0000","last_change":"2024-09-21T11:42:31.712433+0000","last_active":"2024-09-21T11:43:46.188298+0000","last_peered":"2024-09-21T11:43:46.188298+0000","last_clean":"2024-09-21T11:43:46.188298+0000","last_became_active":"2024-09-21T11:41:10.302497+0000","last_became_peered":"2024-09-21T11:41:10.302497+0000","last_unstale":"2024-09-21T11:43:46.188298+0000","last_undegraded":"2024-09-21T11:43:46.188298+0000","last_fullsized":"2024-09-21T11:43:46.188298+0000","mapping_epoch":82,"log_start":"107'6242","ondisk_log_start":"107'6242","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":4349,"log_size":4568,"log_dups_size":0,"ondisk_log_size":4568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267124736,"num_objects":4076,"num_object_clones":0,"num_object_copies":12228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10810,"num_write_kb":476352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":252,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.378638+0000","last_change":"2024-09-21T11:42:19.703071+0000","last_active":"2024-09-21T11:43:41.378638+0000","last_peered":"2024-09-21T11:43:41.378638+0000","last_clean":"2024-09-21T11:43:41.378638+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:43:41.378638+0000","last_undegraded":"2024-09-21T11:43:41.378638+0000","last_fullsized":"2024-09-21T11:43:41.378638+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:42:19.703007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.703007+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:46.340331+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":189,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.377272+0000","last_change":"2024-09-21T11:43:30.437283+0000","last_active":"2024-09-21T11:43:41.377272+0000","last_peered":"2024-09-21T11:43:41.377272+0000","last_clean":"2024-09-21T11:43:41.377272+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:43:41.377272+0000","last_undegraded":"2024-09-21T11:43:41.377272+0000","last_fullsized":"2024-09-21T11:43:41.377272+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.437080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.437080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.933810+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":233,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373258+0000","last_change":"2024-09-21T11:42:24.254810+0000","last_active":"2024-09-21T11:43:41.373258+0000","last_peered":"2024-09-21T11:43:41.373258+0000","last_clean":"2024-09-21T11:43:41.373258+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:43:41.373258+0000","last_undegraded":"2024-09-21T11:43:41.373258+0000","last_fullsized":"2024-09-21T11:43:41.373258+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:42:24.254749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.254749+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":180,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373990+0000","last_change":"2024-09-21T11:43:31.630737+0000","last_active":"2024-09-21T11:43:41.373990+0000","last_peered":"2024-09-21T11:43:41.373990+0000","last_clean":"2024-09-21T11:43:41.373990+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:43:41.373990+0000","last_undegraded":"2024-09-21T11:43:41.373990+0000","last_fullsized":"2024-09-21T11:43:41.373990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:31.630670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.630670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.782053+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":176,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385742+0000","last_change":"2024-09-21T11:41:44.929146+0000","last_active":"2024-09-21T11:43:41.385742+0000","last_peered":"2024-09-21T11:43:41.385742+0000","last_clean":"2024-09-21T11:43:41.385742+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:43:41.385742+0000","last_undegraded":"2024-09-21T11:43:41.385742+0000","last_fullsized":"2024-09-21T11:43:41.385742+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:44.929079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:44.929079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"143'10630","reported_seq":23723,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.725397+0000","last_change":"2024-09-21T11:43:26.996320+0000","last_active":"2024-09-21T11:43:45.725397+0000","last_peered":"2024-09-21T11:43:45.725397+0000","last_clean":"2024-09-21T11:43:45.725397+0000","last_became_active":"2024-09-21T11:43:26.984538+0000","last_became_peered":"2024-09-21T11:43:26.984538+0000","last_unstale":"2024-09-21T11:43:45.725397+0000","last_undegraded":"2024-09-21T11:43:45.725397+0000","last_fullsized":"2024-09-21T11:43:45.725397+0000","mapping_epoch":134,"log_start":"104'5664","ondisk_log_start":"104'5664","created":82,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'5645","last_scrub_stamp":"2024-09-21T11:42:17.339535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.339535+0000","objects_scrubbed":5593,"log_size":4966,"log_dups_size":0,"ondisk_log_size":4966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267452440,"num_objects":4082,"num_object_clones":0,"num_object_copies":12246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4082,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10630,"num_write_kb":470721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6438,"num_bytes_recovered":421855256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":223,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372227+0000","last_change":"2024-09-21T11:42:37.179956+0000","last_active":"2024-09-21T11:43:41.372227+0000","last_peered":"2024-09-21T11:43:41.372227+0000","last_clean":"2024-09-21T11:43:41.372227+0000","last_became_active":"2024-09-21T11:42:37.158158+0000","last_became_peered":"2024-09-21T11:42:37.158158+0000","last_unstale":"2024-09-21T11:43:41.372227+0000","last_undegraded":"2024-09-21T11:43:41.372227+0000","last_fullsized":"2024-09-21T11:43:41.372227+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:22.041174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.041174+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:48.441691+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":189,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.370720+0000","last_change":"2024-09-21T11:42:58.177899+0000","last_active":"2024-09-21T11:43:41.370720+0000","last_peered":"2024-09-21T11:43:41.370720+0000","last_clean":"2024-09-21T11:43:41.370720+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:43:41.370720+0000","last_undegraded":"2024-09-21T11:43:41.370720+0000","last_fullsized":"2024-09-21T11:43:41.370720+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:58.177844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:58.177844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:06.969504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":180,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:44.049478+0000","last_change":"2024-09-21T11:43:44.049478+0000","last_active":"2024-09-21T11:43:44.049478+0000","last_peered":"2024-09-21T11:43:44.049478+0000","last_clean":"2024-09-21T11:43:44.049478+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:43:44.049478+0000","last_undegraded":"2024-09-21T11:43:44.049478+0000","last_fullsized":"2024-09-21T11:43:44.049478+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.049424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.049424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:05.726834+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":177,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373928+0000","last_change":"2024-09-21T11:41:38.453843+0000","last_active":"2024-09-21T11:43:41.373928+0000","last_peered":"2024-09-21T11:43:41.373928+0000","last_clean":"2024-09-21T11:43:41.373928+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:43:41.373928+0000","last_undegraded":"2024-09-21T11:43:41.373928+0000","last_fullsized":"2024-09-21T11:43:41.373928+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:38.453579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:38.453579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"143'10662","reported_seq":10877,"reported_epoch":143,"state":"active+clean+scrubbing","last_fresh":"2024-09-21T11:43:45.877051+0000","last_change":"2024-09-21T11:43:45.482578+0000","last_active":"2024-09-21T11:43:45.877051+0000","last_peered":"2024-09-21T11:43:45.877051+0000","last_clean":"2024-09-21T11:43:45.877051+0000","last_became_active":"2024-09-21T11:41:10.300107+0000","last_became_peered":"2024-09-21T11:41:10.300107+0000","last_unstale":"2024-09-21T11:43:45.877051+0000","last_undegraded":"2024-09-21T11:43:45.877051+0000","last_fullsized":"2024-09-21T11:43:45.877051+0000","mapping_epoch":82,"log_start":"100'5164","ondisk_log_start":"100'5164","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6297","last_scrub_stamp":"2024-09-21T11:42:28.141670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.141670+0000","objects_scrubbed":999,"log_size":5498,"log_dups_size":0,"ondisk_log_size":5498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"scrubbing for 0s","scrub_duration":2853,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264241152,"num_objects":4032,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10662,"num_write_kb":470208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":238,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372008+0000","last_change":"2024-09-21T11:43:39.233466+0000","last_active":"2024-09-21T11:43:41.372008+0000","last_peered":"2024-09-21T11:43:41.372008+0000","last_clean":"2024-09-21T11:43:41.372008+0000","last_became_active":"2024-09-21T11:42:37.160867+0000","last_became_peered":"2024-09-21T11:42:37.160867+0000","last_unstale":"2024-09-21T11:43:41.372008+0000","last_undegraded":"2024-09-21T11:43:41.372008+0000","last_fullsized":"2024-09-21T11:43:41.372008+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:43:39.233386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.233386+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.064116+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":203,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.378156+0000","last_change":"2024-09-21T11:43:36.784531+0000","last_active":"2024-09-21T11:43:41.378156+0000","last_peered":"2024-09-21T11:43:41.378156+0000","last_clean":"2024-09-21T11:43:41.378156+0000","last_became_active":"2024-09-21T11:42:37.167043+0000","last_became_peered":"2024-09-21T11:42:37.167043+0000","last_unstale":"2024-09-21T11:43:41.378156+0000","last_undegraded":"2024-09-21T11:43:41.378156+0000","last_fullsized":"2024-09-21T11:43:41.378156+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.784467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.784467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:39.372586+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":82,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374251+0000","last_change":"2024-09-21T11:43:41.216278+0000","last_active":"2024-09-21T11:43:41.374251+0000","last_peered":"2024-09-21T11:43:41.374251+0000","last_clean":"2024-09-21T11:43:41.374251+0000","last_became_active":"2024-09-21T11:42:37.160659+0000","last_became_peered":"2024-09-21T11:42:37.160659+0000","last_unstale":"2024-09-21T11:43:41.374251+0000","last_undegraded":"2024-09-21T11:43:41.374251+0000","last_fullsized":"2024-09-21T11:43:41.374251+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:41.216215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:41.216215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:45.696826+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":177,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384635+0000","last_change":"2024-09-21T11:42:23.339239+0000","last_active":"2024-09-21T11:43:41.384635+0000","last_peered":"2024-09-21T11:43:41.384635+0000","last_clean":"2024-09-21T11:43:41.384635+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:43:41.384635+0000","last_undegraded":"2024-09-21T11:43:41.384635+0000","last_fullsized":"2024-09-21T11:43:41.384635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:23.339154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.339154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"143'10555","reported_seq":10760,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:46.175033+0000","last_change":"2024-09-21T11:42:29.618177+0000","last_active":"2024-09-21T11:43:46.175033+0000","last_peered":"2024-09-21T11:43:46.175033+0000","last_clean":"2024-09-21T11:43:46.175033+0000","last_became_active":"2024-09-21T11:41:10.303333+0000","last_became_peered":"2024-09-21T11:41:10.303333+0000","last_unstale":"2024-09-21T11:43:46.175033+0000","last_undegraded":"2024-09-21T11:43:46.175033+0000","last_fullsized":"2024-09-21T11:43:46.175033+0000","mapping_epoch":82,"log_start":"106'5944","ondisk_log_start":"106'5944","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6263","last_scrub_stamp":"2024-09-21T11:42:29.617794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.617794+0000","objects_scrubbed":6197,"log_size":4611,"log_dups_size":0,"ondisk_log_size":4611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259588096,"num_objects":3961,"num_object_clones":0,"num_object_copies":11883,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10555,"num_write_kb":464512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":208,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374334+0000","last_change":"2024-09-21T11:43:07.347397+0000","last_active":"2024-09-21T11:43:41.374334+0000","last_peered":"2024-09-21T11:43:41.374334+0000","last_clean":"2024-09-21T11:43:41.374334+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:43:41.374334+0000","last_undegraded":"2024-09-21T11:43:41.374334+0000","last_fullsized":"2024-09-21T11:43:41.374334+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:43:07.347346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:07.347346+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.982030+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":189,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374750+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:43:41.374750+0000","last_peered":"2024-09-21T11:43:41.374750+0000","last_clean":"2024-09-21T11:43:41.374750+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:43:41.374750+0000","last_undegraded":"2024-09-21T11:43:41.374750+0000","last_fullsized":"2024-09-21T11:43:41.374750+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:47.593290+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":243,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.386516+0000","last_change":"2024-09-21T11:43:38.421538+0000","last_active":"2024-09-21T11:43:41.386516+0000","last_peered":"2024-09-21T11:43:41.386516+0000","last_clean":"2024-09-21T11:43:41.386516+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:43:41.386516+0000","last_undegraded":"2024-09-21T11:43:41.386516+0000","last_fullsized":"2024-09-21T11:43:41.386516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:43:38.421251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.421251+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:48.875567+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":184,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.371956+0000","last_change":"2024-09-21T11:42:50.131895+0000","last_active":"2024-09-21T11:43:41.371956+0000","last_peered":"2024-09-21T11:43:41.371956+0000","last_clean":"2024-09-21T11:43:41.371956+0000","last_became_active":"2024-09-21T11:42:37.157732+0000","last_became_peered":"2024-09-21T11:42:37.157732+0000","last_unstale":"2024-09-21T11:43:41.371956+0000","last_undegraded":"2024-09-21T11:43:41.371956+0000","last_fullsized":"2024-09-21T11:43:41.371956+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:50.131830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:50.131830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:52.943303+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"143'10696","reported_seq":10886,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.847659+0000","last_change":"2024-09-21T11:43:37.155559+0000","last_active":"2024-09-21T11:43:45.847659+0000","last_peered":"2024-09-21T11:43:45.847659+0000","last_clean":"2024-09-21T11:43:45.847659+0000","last_became_active":"2024-09-21T11:41:10.304149+0000","last_became_peered":"2024-09-21T11:41:10.304149+0000","last_unstale":"2024-09-21T11:43:45.847659+0000","last_undegraded":"2024-09-21T11:43:45.847659+0000","last_fullsized":"2024-09-21T11:43:45.847659+0000","mapping_epoch":82,"log_start":"100'5202","ondisk_log_start":"100'5202","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'10015","last_scrub_stamp":"2024-09-21T11:43:37.155379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.155379+0000","objects_scrubbed":4788,"log_size":5494,"log_dups_size":0,"ondisk_log_size":5494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:48.799347+0000","scrub_duration":1732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265158656,"num_objects":4046,"num_object_clones":0,"num_object_copies":12138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10696,"num_write_kb":471744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":217,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.378666+0000","last_change":"2024-09-21T11:42:37.192718+0000","last_active":"2024-09-21T11:43:41.378666+0000","last_peered":"2024-09-21T11:43:41.378666+0000","last_clean":"2024-09-21T11:43:41.378666+0000","last_became_active":"2024-09-21T11:42:37.168751+0000","last_became_peered":"2024-09-21T11:42:37.168751+0000","last_unstale":"2024-09-21T11:43:41.378666+0000","last_undegraded":"2024-09-21T11:43:41.378666+0000","last_fullsized":"2024-09-21T11:43:41.378666+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:15.417817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.417817+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384330+0000","last_change":"2024-09-21T11:41:28.936695+0000","last_active":"2024-09-21T11:43:41.384330+0000","last_peered":"2024-09-21T11:43:41.384330+0000","last_clean":"2024-09-21T11:43:41.384330+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:43:41.384330+0000","last_undegraded":"2024-09-21T11:43:41.384330+0000","last_fullsized":"2024-09-21T11:43:41.384330+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:28.936645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:28.936645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":181,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.386341+0000","last_change":"2024-09-21T11:42:18.265457+0000","last_active":"2024-09-21T11:43:41.386341+0000","last_peered":"2024-09-21T11:43:41.386341+0000","last_clean":"2024-09-21T11:43:41.386341+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:43:41.386341+0000","last_undegraded":"2024-09-21T11:43:41.386341+0000","last_fullsized":"2024-09-21T11:43:41.386341+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.265392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.265392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:45.700703+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":168,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.380663+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:43:41.380663+0000","last_peered":"2024-09-21T11:43:41.380663+0000","last_clean":"2024-09-21T11:43:41.380663+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:43:41.380663+0000","last_undegraded":"2024-09-21T11:43:41.380663+0000","last_fullsized":"2024-09-21T11:43:41.380663+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"143'10590","reported_seq":23842,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:46.184829+0000","last_change":"2024-09-21T11:43:22.892096+0000","last_active":"2024-09-21T11:43:46.184829+0000","last_peered":"2024-09-21T11:43:46.184829+0000","last_clean":"2024-09-21T11:43:46.184829+0000","last_became_active":"2024-09-21T11:43:22.875072+0000","last_became_peered":"2024-09-21T11:43:22.875072+0000","last_unstale":"2024-09-21T11:43:46.184829+0000","last_undegraded":"2024-09-21T11:43:46.184829+0000","last_fullsized":"2024-09-21T11:43:46.184829+0000","mapping_epoch":130,"log_start":"106'5950","ondisk_log_start":"106'5950","created":82,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258998272,"num_objects":3952,"num_object_clones":0,"num_object_copies":11856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10590,"num_write_kb":465344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6549,"num_bytes_recovered":429195264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":242,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.377916+0000","last_change":"2024-09-21T11:43:27.726443+0000","last_active":"2024-09-21T11:43:41.377916+0000","last_peered":"2024-09-21T11:43:41.377916+0000","last_clean":"2024-09-21T11:43:41.377916+0000","last_became_active":"2024-09-21T11:42:37.166373+0000","last_became_peered":"2024-09-21T11:42:37.166373+0000","last_unstale":"2024-09-21T11:43:41.377916+0000","last_undegraded":"2024-09-21T11:43:41.377916+0000","last_fullsized":"2024-09-21T11:43:41.377916+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:43:27.726370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:27.726370+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:36.734856+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374701+0000","last_change":"2024-09-21T11:41:46.471940+0000","last_active":"2024-09-21T11:43:41.374701+0000","last_peered":"2024-09-21T11:43:41.374701+0000","last_clean":"2024-09-21T11:43:41.374701+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:43:41.374701+0000","last_undegraded":"2024-09-21T11:43:41.374701+0000","last_fullsized":"2024-09-21T11:43:41.374701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:46.471872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:46.471872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":189,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384480+0000","last_change":"2024-09-21T11:43:26.484397+0000","last_active":"2024-09-21T11:43:41.384480+0000","last_peered":"2024-09-21T11:43:41.384480+0000","last_clean":"2024-09-21T11:43:41.384480+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:43:41.384480+0000","last_undegraded":"2024-09-21T11:43:41.384480+0000","last_fullsized":"2024-09-21T11:43:41.384480+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.484353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.484353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:38.261052+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":176,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373201+0000","last_change":"2024-09-21T11:43:29.650485+0000","last_active":"2024-09-21T11:43:41.373201+0000","last_peered":"2024-09-21T11:43:41.373201+0000","last_clean":"2024-09-21T11:43:41.373201+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:43:41.373201+0000","last_undegraded":"2024-09-21T11:43:41.373201+0000","last_fullsized":"2024-09-21T11:43:41.373201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:29.650427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.650427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:32.208091+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"143'10633","reported_seq":10775,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:46.056747+0000","last_change":"2024-09-21T11:42:31.693667+0000","last_active":"2024-09-21T11:43:46.056747+0000","last_peered":"2024-09-21T11:43:46.056747+0000","last_clean":"2024-09-21T11:43:46.056747+0000","last_became_active":"2024-09-21T11:41:10.300287+0000","last_became_peered":"2024-09-21T11:41:10.300287+0000","last_unstale":"2024-09-21T11:43:46.056747+0000","last_undegraded":"2024-09-21T11:43:46.056747+0000","last_fullsized":"2024-09-21T11:43:46.056747+0000","mapping_epoch":82,"log_start":"108'6354","ondisk_log_start":"108'6354","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":449,"log_size":4279,"log_dups_size":0,"ondisk_log_size":4279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266010624,"num_objects":4059,"num_object_clones":0,"num_object_copies":12177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10633,"num_write_kb":470144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":235,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.370488+0000","last_change":"2024-09-21T11:43:32.231463+0000","last_active":"2024-09-21T11:43:41.370488+0000","last_peered":"2024-09-21T11:43:41.370488+0000","last_clean":"2024-09-21T11:43:41.370488+0000","last_became_active":"2024-09-21T11:42:37.165364+0000","last_became_peered":"2024-09-21T11:42:37.165364+0000","last_unstale":"2024-09-21T11:43:41.370488+0000","last_undegraded":"2024-09-21T11:43:41.370488+0000","last_fullsized":"2024-09-21T11:43:41.370488+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:43:32.231381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.231381+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:36.630886+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":203,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:44.519748+0000","last_change":"2024-09-21T11:43:44.519748+0000","last_active":"2024-09-21T11:43:44.519748+0000","last_peered":"2024-09-21T11:43:44.519748+0000","last_clean":"2024-09-21T11:43:44.519748+0000","last_became_active":"2024-09-21T11:42:37.165470+0000","last_became_peered":"2024-09-21T11:42:37.165470+0000","last_unstale":"2024-09-21T11:43:44.519748+0000","last_undegraded":"2024-09-21T11:43:44.519748+0000","last_fullsized":"2024-09-21T11:43:44.519748+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.519704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.519704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:47.181516+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":196,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385797+0000","last_change":"2024-09-21T11:43:40.502911+0000","last_active":"2024-09-21T11:43:41.385797+0000","last_peered":"2024-09-21T11:43:41.385797+0000","last_clean":"2024-09-21T11:43:41.385797+0000","last_became_active":"2024-09-21T11:42:37.155675+0000","last_became_peered":"2024-09-21T11:42:37.155675+0000","last_unstale":"2024-09-21T11:43:41.385797+0000","last_undegraded":"2024-09-21T11:43:41.385797+0000","last_fullsized":"2024-09-21T11:43:41.385797+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:40.502747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.502747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:57.625738+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":184,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374408+0000","last_change":"2024-09-21T11:42:37.170697+0000","last_active":"2024-09-21T11:43:41.374408+0000","last_peered":"2024-09-21T11:43:41.374408+0000","last_clean":"2024-09-21T11:43:41.374408+0000","last_became_active":"2024-09-21T11:42:37.168686+0000","last_became_peered":"2024-09-21T11:42:37.168686+0000","last_unstale":"2024-09-21T11:43:41.374408+0000","last_undegraded":"2024-09-21T11:43:41.374408+0000","last_fullsized":"2024-09-21T11:43:41.374408+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:27.327920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:27.327920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"143'10461","reported_seq":23302,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.762800+0000","last_change":"2024-09-21T11:43:27.000045+0000","last_active":"2024-09-21T11:43:45.762800+0000","last_peered":"2024-09-21T11:43:45.762800+0000","last_clean":"2024-09-21T11:43:45.762800+0000","last_became_active":"2024-09-21T11:43:26.983757+0000","last_became_peered":"2024-09-21T11:43:26.983757+0000","last_unstale":"2024-09-21T11:43:45.762800+0000","last_undegraded":"2024-09-21T11:43:45.762800+0000","last_fullsized":"2024-09-21T11:43:45.762800+0000","mapping_epoch":134,"log_start":"102'5280","ondisk_log_start":"102'5280","created":82,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6370","last_scrub_stamp":"2024-09-21T11:42:31.280157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:31.280157+0000","objects_scrubbed":6318,"log_size":5181,"log_dups_size":0,"ondisk_log_size":5181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:58.540352+0000","scrub_duration":2077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264830976,"num_objects":4041,"num_object_clones":0,"num_object_copies":12123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10461,"num_write_kb":464064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6310,"num_bytes_recovered":413532160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":201,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385661+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:43:41.385661+0000","last_peered":"2024-09-21T11:43:41.385661+0000","last_clean":"2024-09-21T11:43:41.385661+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:43:41.385661+0000","last_undegraded":"2024-09-21T11:43:41.385661+0000","last_fullsized":"2024-09-21T11:43:41.385661+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":200,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373958+0000","last_change":"2024-09-21T11:42:19.352990+0000","last_active":"2024-09-21T11:43:41.373958+0000","last_peered":"2024-09-21T11:43:41.373958+0000","last_clean":"2024-09-21T11:43:41.373958+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:43:41.373958+0000","last_undegraded":"2024-09-21T11:43:41.373958+0000","last_fullsized":"2024-09-21T11:43:41.373958+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:42:19.352895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.352895+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":234,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373575+0000","last_change":"2024-09-21T11:41:43.495040+0000","last_active":"2024-09-21T11:43:41.373575+0000","last_peered":"2024-09-21T11:43:41.373575+0000","last_clean":"2024-09-21T11:43:41.373575+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:43:41.373575+0000","last_undegraded":"2024-09-21T11:43:41.373575+0000","last_fullsized":"2024-09-21T11:43:41.373575+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:41:43.494960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:43.494960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":176,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385395+0000","last_change":"2024-09-21T11:41:39.867162+0000","last_active":"2024-09-21T11:43:41.385395+0000","last_peered":"2024-09-21T11:43:41.385395+0000","last_clean":"2024-09-21T11:43:41.385395+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:43:41.385395+0000","last_undegraded":"2024-09-21T11:43:41.385395+0000","last_fullsized":"2024-09-21T11:43:41.385395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.867108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.867108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.377844+0000","last_change":"2024-09-21T11:43:30.734704+0000","last_active":"2024-09-21T11:43:41.377844+0000","last_peered":"2024-09-21T11:43:41.377844+0000","last_clean":"2024-09-21T11:43:41.377844+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:43:41.377844+0000","last_undegraded":"2024-09-21T11:43:41.377844+0000","last_fullsized":"2024-09-21T11:43:41.377844+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.734361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.734361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:35.190586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":209,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.380891+0000","last_change":"2024-09-21T11:42:37.201056+0000","last_active":"2024-09-21T11:43:41.380891+0000","last_peered":"2024-09-21T11:43:41.380891+0000","last_clean":"2024-09-21T11:43:41.380891+0000","last_became_active":"2024-09-21T11:42:37.171563+0000","last_became_peered":"2024-09-21T11:42:37.171563+0000","last_unstale":"2024-09-21T11:43:41.380891+0000","last_undegraded":"2024-09-21T11:43:41.380891+0000","last_fullsized":"2024-09-21T11:43:41.380891+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:41:25.310957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:25.310957+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":187,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385413+0000","last_change":"2024-09-21T11:42:37.170710+0000","last_active":"2024-09-21T11:43:41.385413+0000","last_peered":"2024-09-21T11:43:41.385413+0000","last_clean":"2024-09-21T11:43:41.385413+0000","last_became_active":"2024-09-21T11:42:37.170365+0000","last_became_peered":"2024-09-21T11:42:37.170365+0000","last_unstale":"2024-09-21T11:43:41.385413+0000","last_undegraded":"2024-09-21T11:43:41.385413+0000","last_fullsized":"2024-09-21T11:43:41.385413+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:15.090108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.090108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":184,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:45.551832+0000","last_change":"2024-09-21T11:43:45.551832+0000","last_active":"2024-09-21T11:43:45.551832+0000","last_peered":"2024-09-21T11:43:45.551832+0000","last_clean":"2024-09-21T11:43:45.551832+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:43:45.551832+0000","last_undegraded":"2024-09-21T11:43:45.551832+0000","last_fullsized":"2024-09-21T11:43:45.551832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.551792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.551792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.431636+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":217,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.379469+0000","last_change":"2024-09-21T11:43:32.077598+0000","last_active":"2024-09-21T11:43:41.379469+0000","last_peered":"2024-09-21T11:43:41.379469+0000","last_clean":"2024-09-21T11:43:41.379469+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:43:41.379469+0000","last_undegraded":"2024-09-21T11:43:41.379469+0000","last_fullsized":"2024-09-21T11:43:41.379469+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:11.469922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.469922+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373721+0000","last_change":"2024-09-21T11:42:23.241888+0000","last_active":"2024-09-21T11:43:41.373721+0000","last_peered":"2024-09-21T11:43:41.373721+0000","last_clean":"2024-09-21T11:43:41.373721+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:43:41.373721+0000","last_undegraded":"2024-09-21T11:43:41.373721+0000","last_fullsized":"2024-09-21T11:43:41.373721+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:23.241796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.241796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:44.385176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.380730+0000","last_change":"2024-09-21T11:43:38.461076+0000","last_active":"2024-09-21T11:43:41.380730+0000","last_peered":"2024-09-21T11:43:41.380730+0000","last_clean":"2024-09-21T11:43:41.380730+0000","last_became_active":"2024-09-21T11:42:37.172423+0000","last_became_peered":"2024-09-21T11:42:37.172423+0000","last_unstale":"2024-09-21T11:43:41.380730+0000","last_undegraded":"2024-09-21T11:43:41.380730+0000","last_fullsized":"2024-09-21T11:43:41.380730+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.461014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.461014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:44.750422+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":185,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:42.448731+0000","last_change":"2024-09-21T11:43:42.448731+0000","last_active":"2024-09-21T11:43:42.448731+0000","last_peered":"2024-09-21T11:43:42.448731+0000","last_clean":"2024-09-21T11:43:42.448731+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:43:42.448731+0000","last_undegraded":"2024-09-21T11:43:42.448731+0000","last_fullsized":"2024-09-21T11:43:42.448731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:42.448669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:42.448669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.700328+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":200,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384807+0000","last_change":"2024-09-21T11:41:33.917474+0000","last_active":"2024-09-21T11:43:41.384807+0000","last_peered":"2024-09-21T11:43:41.384807+0000","last_clean":"2024-09-21T11:43:41.384807+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:43:41.384807+0000","last_undegraded":"2024-09-21T11:43:41.384807+0000","last_fullsized":"2024-09-21T11:43:41.384807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:33.917389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:33.917389+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.378351+0000","last_change":"2024-09-21T11:42:22.478692+0000","last_active":"2024-09-21T11:43:41.378351+0000","last_peered":"2024-09-21T11:43:41.378351+0000","last_clean":"2024-09-21T11:43:41.378351+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:43:41.378351+0000","last_undegraded":"2024-09-21T11:43:41.378351+0000","last_fullsized":"2024-09-21T11:43:41.378351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:22.478619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.478619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.263376+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":227,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374102+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:43:41.374102+0000","last_peered":"2024-09-21T11:43:41.374102+0000","last_clean":"2024-09-21T11:43:41.374102+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:43:41.374102+0000","last_undegraded":"2024-09-21T11:43:41.374102+0000","last_fullsized":"2024-09-21T11:43:41.374102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":185,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372257+0000","last_change":"2024-09-21T11:43:24.300706+0000","last_active":"2024-09-21T11:43:41.372257+0000","last_peered":"2024-09-21T11:43:41.372257+0000","last_clean":"2024-09-21T11:43:41.372257+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:43:41.372257+0000","last_undegraded":"2024-09-21T11:43:41.372257+0000","last_fullsized":"2024-09-21T11:43:41.372257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:24.300642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:24.300642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.984128+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":219,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374342+0000","last_change":"2024-09-21T11:42:37.207434+0000","last_active":"2024-09-21T11:43:41.374342+0000","last_peered":"2024-09-21T11:43:41.374342+0000","last_clean":"2024-09-21T11:43:41.374342+0000","last_became_active":"2024-09-21T11:42:37.171345+0000","last_became_peered":"2024-09-21T11:42:37.171345+0000","last_unstale":"2024-09-21T11:43:41.374342+0000","last_undegraded":"2024-09-21T11:43:41.374342+0000","last_fullsized":"2024-09-21T11:43:41.374342+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.379125+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374565+0000","last_change":"2024-09-21T11:42:11.317760+0000","last_active":"2024-09-21T11:43:41.374565+0000","last_peered":"2024-09-21T11:43:41.374565+0000","last_clean":"2024-09-21T11:43:41.374565+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:43:41.374565+0000","last_undegraded":"2024-09-21T11:43:41.374565+0000","last_fullsized":"2024-09-21T11:43:41.374565+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.317696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.317696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"114'9","reported_seq":246,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384632+0000","last_change":"2024-09-21T11:43:28.003108+0000","last_active":"2024-09-21T11:43:41.384632+0000","last_peered":"2024-09-21T11:43:41.384632+0000","last_clean":"2024-09-21T11:43:41.384632+0000","last_became_active":"2024-09-21T11:42:37.163771+0000","last_became_peered":"2024-09-21T11:42:37.163771+0000","last_unstale":"2024-09-21T11:43:41.384632+0000","last_undegraded":"2024-09-21T11:43:41.384632+0000","last_fullsized":"2024-09-21T11:43:41.384632+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:28.003036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.003036+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:32.544327+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":176,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.379089+0000","last_change":"2024-09-21T11:41:30.274387+0000","last_active":"2024-09-21T11:43:41.379089+0000","last_peered":"2024-09-21T11:43:41.379089+0000","last_clean":"2024-09-21T11:43:41.379089+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:43:41.379089+0000","last_undegraded":"2024-09-21T11:43:41.379089+0000","last_fullsized":"2024-09-21T11:43:41.379089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:30.274201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:30.274201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":231,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.371486+0000","last_change":"2024-09-21T11:43:40.233160+0000","last_active":"2024-09-21T11:43:41.371486+0000","last_peered":"2024-09-21T11:43:41.371486+0000","last_clean":"2024-09-21T11:43:41.371486+0000","last_became_active":"2024-09-21T11:42:37.154835+0000","last_became_peered":"2024-09-21T11:42:37.154835+0000","last_unstale":"2024-09-21T11:43:41.371486+0000","last_undegraded":"2024-09-21T11:43:41.371486+0000","last_fullsized":"2024-09-21T11:43:41.371486+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:43:40.233067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.233067+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:41.481160+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":189,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374510+0000","last_change":"2024-09-21T11:43:00.245058+0000","last_active":"2024-09-21T11:43:41.374510+0000","last_peered":"2024-09-21T11:43:41.374510+0000","last_clean":"2024-09-21T11:43:41.374510+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:43:41.374510+0000","last_undegraded":"2024-09-21T11:43:41.374510+0000","last_fullsized":"2024-09-21T11:43:41.374510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:00.244985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:00.244985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:22.792144+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":74,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385713+0000","last_change":"2024-09-21T11:42:37.163592+0000","last_active":"2024-09-21T11:43:41.385713+0000","last_peered":"2024-09-21T11:43:41.385713+0000","last_clean":"2024-09-21T11:43:41.385713+0000","last_became_active":"2024-09-21T11:42:37.163261+0000","last_became_peered":"2024-09-21T11:42:37.163261+0000","last_unstale":"2024-09-21T11:43:41.385713+0000","last_undegraded":"2024-09-21T11:43:41.385713+0000","last_fullsized":"2024-09-21T11:43:41.385713+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.301406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.301406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.735442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":169,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.371882+0000","last_change":"2024-09-21T11:40:06.353577+0000","last_active":"2024-09-21T11:43:41.371882+0000","last_peered":"2024-09-21T11:43:41.371882+0000","last_clean":"2024-09-21T11:43:41.371882+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:43:41.371882+0000","last_undegraded":"2024-09-21T11:43:41.371882+0000","last_fullsized":"2024-09-21T11:43:41.371882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":127,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.808987+0000","last_change":"2024-09-21T11:43:41.808987+0000","last_active":"2024-09-21T11:43:41.808987+0000","last_peered":"2024-09-21T11:43:41.808987+0000","last_clean":"2024-09-21T11:43:41.808987+0000","last_became_active":"2024-09-21T11:42:37.167265+0000","last_became_peered":"2024-09-21T11:42:37.167265+0000","last_unstale":"2024-09-21T11:43:41.808987+0000","last_undegraded":"2024-09-21T11:43:41.808987+0000","last_fullsized":"2024-09-21T11:43:41.808987+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:41.808900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:41.808900+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.057274+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373920+0000","last_change":"2024-09-21T11:43:26.586603+0000","last_active":"2024-09-21T11:43:41.373920+0000","last_peered":"2024-09-21T11:43:41.373920+0000","last_clean":"2024-09-21T11:43:41.373920+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:43:41.373920+0000","last_undegraded":"2024-09-21T11:43:41.373920+0000","last_fullsized":"2024-09-21T11:43:41.373920+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.586562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.586562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:55.496321+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374499+0000","last_change":"2024-09-21T11:42:37.167224+0000","last_active":"2024-09-21T11:43:41.374499+0000","last_peered":"2024-09-21T11:43:41.374499+0000","last_clean":"2024-09-21T11:43:41.374499+0000","last_became_active":"2024-09-21T11:42:37.166917+0000","last_became_peered":"2024-09-21T11:42:37.166917+0000","last_unstale":"2024-09-21T11:43:41.374499+0000","last_undegraded":"2024-09-21T11:43:41.374499+0000","last_fullsized":"2024-09-21T11:43:41.374499+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:23.514572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.514572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.377348+0000","last_change":"2024-09-21T11:42:37.177478+0000","last_active":"2024-09-21T11:43:41.377348+0000","last_peered":"2024-09-21T11:43:41.377348+0000","last_clean":"2024-09-21T11:43:41.377348+0000","last_became_active":"2024-09-21T11:42:37.161589+0000","last_became_peered":"2024-09-21T11:42:37.161589+0000","last_unstale":"2024-09-21T11:43:41.377348+0000","last_undegraded":"2024-09-21T11:43:41.377348+0000","last_fullsized":"2024-09-21T11:43:41.377348+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:41:29.236463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:29.236463+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":204,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.371983+0000","last_change":"2024-09-21T11:42:15.470041+0000","last_active":"2024-09-21T11:43:41.371983+0000","last_peered":"2024-09-21T11:43:41.371983+0000","last_clean":"2024-09-21T11:43:41.371983+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:43:41.371983+0000","last_undegraded":"2024-09-21T11:43:41.371983+0000","last_fullsized":"2024-09-21T11:43:41.371983+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:15.469421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.469421+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":195,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.385554+0000","last_change":"2024-09-21T11:42:37.173431+0000","last_active":"2024-09-21T11:43:41.385554+0000","last_peered":"2024-09-21T11:43:41.385554+0000","last_clean":"2024-09-21T11:43:41.385554+0000","last_became_active":"2024-09-21T11:42:37.172859+0000","last_became_peered":"2024-09-21T11:42:37.172859+0000","last_unstale":"2024-09-21T11:43:41.385554+0000","last_undegraded":"2024-09-21T11:43:41.385554+0000","last_fullsized":"2024-09-21T11:43:41.385554+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.952335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.952335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":180,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372978+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:43:41.372978+0000","last_peered":"2024-09-21T11:43:41.372978+0000","last_clean":"2024-09-21T11:43:41.372978+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:43:41.372978+0000","last_undegraded":"2024-09-21T11:43:41.372978+0000","last_fullsized":"2024-09-21T11:43:41.372978+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.612728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":177,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.374630+0000","last_change":"2024-09-21T11:42:12.409717+0000","last_active":"2024-09-21T11:43:41.374630+0000","last_peered":"2024-09-21T11:43:41.374630+0000","last_clean":"2024-09-21T11:43:41.374630+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:43:41.374630+0000","last_undegraded":"2024-09-21T11:43:41.374630+0000","last_fullsized":"2024-09-21T11:43:41.374630+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:12.409641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:12.409641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":217,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372381+0000","last_change":"2024-09-21T11:42:37.195076+0000","last_active":"2024-09-21T11:43:41.372381+0000","last_peered":"2024-09-21T11:43:41.372381+0000","last_clean":"2024-09-21T11:43:41.372381+0000","last_became_active":"2024-09-21T11:42:37.169482+0000","last_became_peered":"2024-09-21T11:42:37.169482+0000","last_unstale":"2024-09-21T11:43:41.372381+0000","last_undegraded":"2024-09-21T11:43:41.372381+0000","last_fullsized":"2024-09-21T11:43:41.372381+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:41:32.414244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:32.414244+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":196,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.379668+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:43:41.379668+0000","last_peered":"2024-09-21T11:43:41.379668+0000","last_clean":"2024-09-21T11:43:41.379668+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:43:41.379668+0000","last_undegraded":"2024-09-21T11:43:41.379668+0000","last_fullsized":"2024-09-21T11:43:41.379668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":188,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.377912+0000","last_change":"2024-09-21T11:43:39.838067+0000","last_active":"2024-09-21T11:43:41.377912+0000","last_peered":"2024-09-21T11:43:41.377912+0000","last_clean":"2024-09-21T11:43:41.377912+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:43:41.377912+0000","last_undegraded":"2024-09-21T11:43:41.377912+0000","last_fullsized":"2024-09-21T11:43:41.377912+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:39.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.838002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.931489+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":176,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.373284+0000","last_change":"2024-09-21T11:42:30.269470+0000","last_active":"2024-09-21T11:43:41.373284+0000","last_peered":"2024-09-21T11:43:41.373284+0000","last_clean":"2024-09-21T11:43:41.373284+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:43:41.373284+0000","last_undegraded":"2024-09-21T11:43:41.373284+0000","last_fullsized":"2024-09-21T11:43:41.373284+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:30.269408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:30.269408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:53.929437+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":73,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.384678+0000","last_change":"2024-09-21T11:42:37.155891+0000","last_active":"2024-09-21T11:43:41.384678+0000","last_peered":"2024-09-21T11:43:41.384678+0000","last_clean":"2024-09-21T11:43:41.384678+0000","last_became_active":"2024-09-21T11:42:37.155620+0000","last_became_peered":"2024-09-21T11:42:37.155620+0000","last_unstale":"2024-09-21T11:43:41.384678+0000","last_undegraded":"2024-09-21T11:43:41.384678+0000","last_fullsized":"2024-09-21T11:43:41.384678+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.358678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.358678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":196,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.372596+0000","last_change":"2024-09-21T11:43:36.413767+0000","last_active":"2024-09-21T11:43:41.372596+0000","last_peered":"2024-09-21T11:43:41.372596+0000","last_clean":"2024-09-21T11:43:41.372596+0000","last_became_active":"2024-09-21T11:42:37.167248+0000","last_became_peered":"2024-09-21T11:42:37.167248+0000","last_unstale":"2024-09-21T11:43:41.372596+0000","last_undegraded":"2024-09-21T11:43:41.372596+0000","last_fullsized":"2024-09-21T11:43:41.372596+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.413725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.413725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.736856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":230,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.378785+0000","last_change":"2024-09-21T11:43:32.709031+0000","last_active":"2024-09-21T11:43:41.378785+0000","last_peered":"2024-09-21T11:43:41.378785+0000","last_clean":"2024-09-21T11:43:41.378785+0000","last_became_active":"2024-09-21T11:42:37.171253+0000","last_became_peered":"2024-09-21T11:42:37.171253+0000","last_unstale":"2024-09-21T11:43:41.378785+0000","last_undegraded":"2024-09-21T11:43:41.378785+0000","last_fullsized":"2024-09-21T11:43:41.378785+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:43:32.708849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.708849+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.362664+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":180,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-21T11:43:41.379217+0000","last_change":"2024-09-21T11:41:31.249298+0000","last_active":"2024-09-21T11:43:41.379217+0000","last_peered":"2024-09-21T11:43:41.379217+0000","last_clean":"2024-09-21T11:43:41.379217+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:43:41.379217+0000","last_undegraded":"2024-09-21T11:43:41.379217+0000","last_fullsized":"2024-09-21T11:43:41.379217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:31.249215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:31.249215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":4245028888,"num_objects":64775,"num_object_clones":0,"num_object_copies":194325,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64775,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170037,"num_write_kb":7513921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38414,"num_bytes_recovered":2517303320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14638465024,"data_stored":14638448736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76518,"ondisk_log_size":76518,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":5367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":219,"num_read_kb":214,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234034,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1999280,"kb_used_data":1823712,"kb_used_omap":1,"kb_used_meta":175550,"kb_avail":91745872,"statfs":{"total":95995035648,"available":93947772928,"internally_reserved":0,"allocated":1867481088,"data_stored":1867072872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":179763658},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1749312,"kb_used_data":1601520,"kb_used_omap":1,"kb_used_meta":147774,"kb_avail":91995840,"statfs":{"total":95995035648,"available":94203740160,"internally_reserved":0,"allocated":1639956480,"data_stored":1639532729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151321030},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1130404,"kb_used_data":1058964,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":92614748,"statfs":{"total":95995035648,"available":94837501952,"internally_reserved":0,"allocated":1084379136,"data_stored":1083982043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1423076,"kb_used_data":1311380,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92322076,"statfs":{"total":95995035648,"available":94537805824,"internally_reserved":0,"allocated":1342853120,"data_stored":1342456675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117064,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2773544,"kb_used_data":2620184,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":90971608,"statfs":{"total":95995035648,"available":93154926592,"internally_reserved":0,"allocated":2683068416,"data_stored":2682667443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":157022663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2265292,"kb_used_data":2085436,"kb_used_omap":1,"kb_used_meta":179710,"kb_avail":91479860,"statfs":{"total":95995035648,"available":93675376640,"internally_reserved":0,"allocated":2135486464,"data_stored":2135113801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":184023493},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":55834574933,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2212608,"kb_used_data":2091184,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":91532544,"statfs":{"total":95995035648,"available":93729325056,"internally_reserved":0,"allocated":2141372416,"data_stored":2140948854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2115244032,"data_stored":2115239960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1899565056,"data_stored":1899560984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2116616192,"data_stored":2116616192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2645430272,"data_stored":2645426200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1324548096,"data_stored":1324548096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1066991616,"data_stored":1066991616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1611010048,"data_stored":1611005976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1859059712,"data_stored":1859059712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:43:54.171 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-21T11:43:54.171 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:43:54.172 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-21T11:43:54.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:43:54.257 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:54.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:53 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/251230493' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:43:54.286 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:54.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:54.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:54.388 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:54.415 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:54.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:54.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:54.516 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:54.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:54.538 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:54.539 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:54.639 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:54.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:54.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:54.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:54.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:54.813 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:54.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:54.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:54.953 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:54.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:54.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:55.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:55.096 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:55.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:55.098 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:55.199 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:55.228 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:55.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:55.229 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:54 smithi043 ceph-mon[36454]: pgmap v176: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:43:55.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:54 smithi043 ceph-mon[29710]: pgmap v176: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:43:55.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:54 smithi175 ceph-mon[32613]: pgmap v176: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:43:55.330 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:55.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:55.363 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:55.363 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:55.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:43:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:43:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:43:55.508 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:55.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:55.509 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:55.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:55.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:55.648 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:55.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:55.790 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:55.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:55.791 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:55.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:43:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:55.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:55.930 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:43:56.030 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:56.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:56.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:56.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:56.213 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:56.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:56.214 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:56.315 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:56.343 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:56.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:56.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:56.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:56.475 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:56.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:56.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:56.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:56.606 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:56.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:56.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:56.708 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:56.743 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:56.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:56.745 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:56.846 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:56.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:56.880 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:56.881 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:56.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:57.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:57.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:57.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:57.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:57.167 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:57.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:57.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:56 smithi043 ceph-mon[36454]: pgmap v177: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-21T11:43:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:56 smithi043 ceph-mon[29710]: pgmap v177: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-21T11:43:57.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:56 smithi175 ceph-mon[32613]: pgmap v177: 148 pgs: 148 active+clean; 3.4 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-21T11:43:57.268 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:43:57.296 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:57.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:57.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:43:57.398 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:57.429 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:57.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:57.429 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:57.530 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:57.551 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:57.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:57.552 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:57.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:57.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:57.692 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:57.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:57.828 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:57.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:57.829 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:57.930 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:57.956 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:57.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:57.957 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:58.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:58.088 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:58.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:58.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:58.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:58.227 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:58.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:58.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:58.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:58.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:58.370 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:58.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:58.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:58.522 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:58.522 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:58.623 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:58.647 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:58.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:58.648 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:58.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:58.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:58.785 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:58.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:58.886 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:43:58.930 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:58.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:58.931 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:43:59.032 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:43:59.063 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:59.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:59.064 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:43:59.093 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:59.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:59.210 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:59.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:59.213 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:59.290 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:43:59 smithi043 ceph-mon[36454]: pgmap v178: 148 pgs: 148 active+clean; 3.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:43:59.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:43:59 smithi043 ceph-mon[29710]: pgmap v178: 148 pgs: 148 active+clean; 3.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:43:59.305 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:43:59.314 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:43:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:59.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:59.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:43:59.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:43:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:43:59.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:59.504 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:43:59.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:43:59 smithi175 ceph-mon[32613]: pgmap v178: 148 pgs: 148 active+clean; 3.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:43:59.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:59.632 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:59.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:59.633 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:59.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:43:59.764 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:59.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:59.765 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:43:59.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:43:59.894 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:43:59.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:43:59.895 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:43:59.995 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:00.056 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:00.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:00.056 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:00.157 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:00.182 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:00.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:00.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:00.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:00.324 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:00.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:00.326 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:00.419 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:00.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:00.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:00.478 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:00.478 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:00.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:00.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:00.621 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:00.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:00.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:00.752 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:44:00.755 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:44:00.780 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:00.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:00.781 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:00.881 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:00.913 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:00.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:00.913 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:01.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:01.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:01.054 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:01.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:44:01.155 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:01.187 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:01.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:01.188 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:01.192 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:44:01.260 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:44:00 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:44:01.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:01 smithi175 ceph-mon[32613]: pgmap v179: 148 pgs: 148 active+clean; 3.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 915 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:44:01.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/204064717' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:01.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:01 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:01.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:01.289 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:01.315 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:01.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:01.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:01.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:01 smithi043 ceph-mon[29710]: pgmap v179: 148 pgs: 148 active+clean; 3.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 915 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:44:01.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/204064717' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:01.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:01 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:01.403 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:01.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:01 smithi043 ceph-mon[36454]: pgmap v179: 148 pgs: 148 active+clean; 3.2 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 915 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:44:01.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/204064717' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:01.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:01 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:01.403 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:01.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:01.461 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:01.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:01.462 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:01.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:01.598 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:44:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:44:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:44:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:44:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:99025 7.3 7:c6972747:::benchmark_data_smithi043_70646_object99024:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:44:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.329943+0000", 2024-09-21T11:44:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.069886538999995, 2024-09-21T11:44:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.10715627599999999, 2024-09-21T11:44:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 99025 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.329943+0000", 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.329943+0000", 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.329945+0000", 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5340000000000001e-06 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.329968+0000", 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3098000000000001e-05 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.329977+0000", 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0450000000000006e-06 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.329983+0000", 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4479999999999999e-06 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334571+0000", 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0045876320000000003 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334718+0000", 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000146596 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334870+0000", 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015247000000000001 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.436967+0000", 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.10209657599999999 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.436977+0000", 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0531e-05 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.437012+0000", 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.536e-05 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.437054+0000", 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1780999999999999e-05 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.437099+0000", 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5204999999999998e-05 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:99027 7.a 7:5ba6064a:::benchmark_data_smithi043_70646_object99026:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.330001+0000", 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.069828502999997, 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.10356293699999999, 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 99027 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330001+0000", 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330001+0000", 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330002+0000", 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.13e-07 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330021+0000", 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8865e-05 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330027+0000", 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2210000000000002e-06 2024-09-21T11:44:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330032+0000", 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3519999999999997e-06 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330265+0000", 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023264299999999999 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330382+0000", 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117352 2024-09-21T11:44:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330523+0000", 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140901 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.333054+0000", 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025308990000000001 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334526+0000", 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014715240000000001 2024-09-21T11:44:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433464+0000", 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.098938060999999994 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433519+0000", 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5109999999999999e-05 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433564+0000", 2024-09-21T11:44:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5395999999999997e-05 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:99028 7.3 7:c72ea3fb:::benchmark_data_smithi043_70646_object99027:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.334742+0000", 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.065087594000005, 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11164890500000001, 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 99028 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334742+0000", 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334742+0000", 2024-09-21T11:44:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334743+0000", 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.246e-06 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334770+0000", 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.65e-05 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334781+0000", 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.082e-05 2024-09-21T11:44:01.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334789+0000", 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0490000000000004e-06 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433319+0000", 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.098530329 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433490+0000", 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017107 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433660+0000", 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016998299999999999 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.436981+0000", 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033210840000000002 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.441599+0000", 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046176960000000001 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.446286+0000", 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046874270000000001 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.446348+0000", 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1736000000000007e-05 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.446391+0000", 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2964999999999999e-05 2024-09-21T11:44:01.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:99031 7.a 7:591d7a73:::benchmark_data_smithi043_70646_object99030:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.334984+0000", 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.064845258000005, 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.099384168999999994, 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:01.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 99031 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334984+0000", 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334984+0000", 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334985+0000", 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9400000000000005e-07 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335003+0000", 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7895999999999999e-05 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335011+0000", 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.537e-06 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335015+0000", 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5029999999999999e-06 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335056+0000", 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.138e-05 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335134+0000", 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7070999999999994e-05 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335232+0000", 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8692999999999999e-05 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.337583+0000", 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023505309999999999 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433408+0000", 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.095825332999999999 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.434037+0000", 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00062867299999999995 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.434325+0000", 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028800700000000001 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.434369+0000", 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3850999999999997e-05 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:99025 7.3 e107/82 ondisk, result = 0)", 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.337979+0000", 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.061850561, 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.099127779999999999, 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.337979+0000", 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.337979+0000", 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.337980+0000", 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0399999999999996e-07 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.337983+0000", 2024-09-21T11:44:01.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.427e-06 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433363+0000", 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.095380220000000002 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433371+0000", 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0669999999999992e-06 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.437003+0000", 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036318610000000001 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.437009+0000", 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2599999999999996e-06 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.437107+0000", 2024-09-21T11:44:01.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8340999999999999e-05 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:99027 7.a e107/82 ondisk, result = 0)", 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.338529+0000", 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.061300833999994, 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.095041843000000001, 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.626 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338529+0000", 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338529+0000", 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338530+0000", 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000001e-07 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.338534+0000", 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4159999999999997e-06 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.432777+0000", 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.094242836999999996 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.432788+0000", 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.097e-05 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433445+0000", 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00065723699999999997 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433458+0000", 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2799000000000001e-05 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.433571+0000", 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011284400000000001 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:126200 7.a 7:59920e1d:::benchmark_data_smithi043_70646_object8792:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:59.181462+0000", 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.218368026, 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.10560541699999999, 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.630 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126200 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.181462+0000", 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.181462+0000", 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.181462+0000", 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1299999999999999e-07 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.181467+0000", 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0990000000000001e-06 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.181476+0000", 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.183e-06 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.181483+0000", 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1250000000000004e-06 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.186834+0000", 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0053515350000000001 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.186917+0000", 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2369999999999999e-05 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.186987+0000", 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0574000000000006e-05 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.193169+0000", 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061818330000000003 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.193187+0000", 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8017999999999999e-05 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.286962+0000", 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.093774356000000003 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287022+0000", 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0900999999999998e-05 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287067+0000", 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4709999999999997e-05 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:126206 7.3 7:c572b0cf:::benchmark_data_smithi043_70646_object8798:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:59.194172+0000", 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.205657506000001, 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.098396625000000001, 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.636 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126206 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194172+0000", 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194172+0000", 2024-09-21T11:44:01.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194174+0000", 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.81e-06 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194178+0000", 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0550000000000001e-06 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194188+0000", 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6360000000000006e-06 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194195+0000", 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9609999999999999e-06 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194378+0000", 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018310099999999999 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194484+0000", 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106585 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194609+0000", 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000124831 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287363+0000", 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.092754153000000006 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:44:01.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287383+0000", 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9561000000000001e-05 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.292445+0000", 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050623550000000002 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.292518+0000", 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2872000000000003e-05 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.292569+0000", 2024-09-21T11:44:01.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0705000000000003e-05 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:126204 7.6 e122/82 7:64b29738:::benchmark_data_smithi043_70646_object8796:head v 122'8016, mlcod=122'8012)", 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:59.194233+0000", 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.205596663999998, 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.092922743000000002, 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194233+0000", 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194233+0000", 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194234+0000", 2024-09-21T11:44:01.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5400000000000001e-07 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194238+0000", 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0690000000000003e-06 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194245+0000", 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8959999999999997e-06 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194250+0000", 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.967e-06 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194906+0000", 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00065683400000000002 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.194924+0000", 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8022000000000001e-05 2024-09-21T11:44:01.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287125+0000", 2024-09-21T11:44:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.092200306999999995 2024-09-21T11:44:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287156+0000", 2024-09-21T11:44:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1093999999999997e-05 2024-09-21T11:44:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:126213 7.7 e122/82 7:efdaa55a:::benchmark_data_smithi043_70646_object8805:head v 122'8074, mlcod=122'8072)", 2024-09-21T11:44:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:59.195329+0000", 2024-09-21T11:44:01.647 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.204500508999999, 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.095698019999999995, 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.195329+0000", 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.195329+0000", 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.195330+0000", 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000001e-07 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.195335+0000", 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5349999999999998e-06 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.195342+0000", 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1180000000000003e-06 2024-09-21T11:44:01.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.195346+0000", 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.87e-06 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.195411+0000", 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4102000000000004e-05 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.195422+0000", 2024-09-21T11:44:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1651000000000001e-05 2024-09-21T11:44:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.290997+0000", 2024-09-21T11:44:01.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.095574402000000003 2024-09-21T11:44:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.291027+0000", 2024-09-21T11:44:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0602000000000003e-05 2024-09-21T11:44:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.652 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.653 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.653 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:126212 7.1 e122/82 7:86c8b838:::benchmark_data_smithi043_70646_object8804:head v 122'7890, mlcod=122'7888)", 2024-09-21T11:44:01.653 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:59.196298+0000", 2024-09-21T11:44:01.653 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.203531595999998, 2024-09-21T11:44:01.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.094481714999999994, 2024-09-21T11:44:01.654 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.654 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.654 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.654 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.196298+0000", 2024-09-21T11:44:01.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.196298+0000", 2024-09-21T11:44:01.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.196299+0000", 2024-09-21T11:44:01.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0389999999999999e-06 2024-09-21T11:44:01.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.196305+0000", 2024-09-21T11:44:01.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1330000000000004e-06 2024-09-21T11:44:01.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.283321+0000", 2024-09-21T11:44:01.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.087015599999999999 2024-09-21T11:44:01.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287128+0000", 2024-09-21T11:44:01.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038066129999999999 2024-09-21T11:44:01.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287867+0000", 2024-09-21T11:44:01.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00073916699999999999 2024-09-21T11:44:01.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287886+0000", 2024-09-21T11:44:01.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9253999999999999e-05 2024-09-21T11:44:01.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.290755+0000", 2024-09-21T11:44:01.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028686720000000001 2024-09-21T11:44:01.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.290780+0000", 2024-09-21T11:44:01.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5236999999999999e-05 2024-09-21T11:44:01.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.662 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.663 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:126200 7.a e122/82 ondisk, result = 0)", 2024-09-21T11:44:01.663 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:59.196721+0000", 2024-09-21T11:44:01.663 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.203108301, 2024-09-21T11:44:01.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.090351502, 2024-09-21T11:44:01.663 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.663 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.663 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:44:01.664 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.196721+0000", 2024-09-21T11:44:01.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.196721+0000", 2024-09-21T11:44:01.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.196722+0000", 2024-09-21T11:44:01.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5999999999999991e-07 2024-09-21T11:44:01.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.196730+0000", 2024-09-21T11:44:01.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4080000000000001e-06 2024-09-21T11:44:01.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.286736+0000", 2024-09-21T11:44:01.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.090005821 2024-09-21T11:44:01.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.286741+0000", 2024-09-21T11:44:01.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5910000000000003e-06 2024-09-21T11:44:01.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.286947+0000", 2024-09-21T11:44:01.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020590499999999999 2024-09-21T11:44:01.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.286956+0000", 2024-09-21T11:44:01.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5420000000000005e-06 2024-09-21T11:44:01.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287073+0000", 2024-09-21T11:44:01.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117275 2024-09-21T11:44:01.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.671 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.671 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:126206 7.3 e122/82 ondisk, result = 0)", 2024-09-21T11:44:01.671 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:59.197652+0000", 2024-09-21T11:44:01.672 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 62.202177984999999, 2024-09-21T11:44:01.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.089717751999999998, 2024-09-21T11:44:01.673 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.673 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.673 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:44:01.674 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.197652+0000", 2024-09-21T11:44:01.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.197652+0000", 2024-09-21T11:44:01.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.197653+0000", 2024-09-21T11:44:01.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8700000000000005e-07 2024-09-21T11:44:01.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.197656+0000", 2024-09-21T11:44:01.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4079999999999998e-06 2024-09-21T11:44:01.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.286699+0000", 2024-09-21T11:44:01.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.089043220000000006 2024-09-21T11:44:01.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.286714+0000", 2024-09-21T11:44:01.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4626e-05 2024-09-21T11:44:01.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287351+0000", 2024-09-21T11:44:01.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00063719000000000004 2024-09-21T11:44:01.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287358+0000", 2024-09-21T11:44:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0360000000000001e-06 2024-09-21T11:44:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:59.287370+0000", 2024-09-21T11:44:01.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1484999999999999e-05 2024-09-21T11:44:01.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.681 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.681 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.682 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:185188 7.3 7:cc090135:::benchmark_data_smithi043_70646_object67780:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:44:01.682 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:43:57.843835+0000", 2024-09-21T11:44:01.682 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5559948129999999, 2024-09-21T11:44:01.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.092434400999999999, 2024-09-21T11:44:01.682 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.683 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.683 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:01.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:01.683 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:01.683 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 185188 2024-09-21T11:44:01.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.684 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.843835+0000", 2024-09-21T11:44:01.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.843835+0000", 2024-09-21T11:44:01.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.843837+0000", 2024-09-21T11:44:01.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.877e-06 2024-09-21T11:44:01.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.843842+0000", 2024-09-21T11:44:01.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.04e-06 2024-09-21T11:44:01.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.843854+0000", 2024-09-21T11:44:01.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2495e-05 2024-09-21T11:44:01.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.843862+0000", 2024-09-21T11:44:01.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0080000000000002e-06 2024-09-21T11:44:01.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.845206+0000", 2024-09-21T11:44:01.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013436069999999999 2024-09-21T11:44:01.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.845316+0000", 2024-09-21T11:44:01.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110253 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.845421+0000", 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104613 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.851185+0000", 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057638869999999997 2024-09-21T11:44:01.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.936122+0000", 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.084937708000000001 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.936177+0000", 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4481000000000002e-05 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.936224+0000", 2024-09-21T11:44:01.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6767000000000001e-05 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.936269+0000", 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5664999999999997e-05 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:185185 7.f e144/82 7:f42e8c49:::benchmark_data_smithi043_70646_object67777:head v 144'11437, mlcod=144'11435)", 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:43:57.846295+0000", 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5535345380000001, 2024-09-21T11:44:01.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.094176074999999998, 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.846295+0000", 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.846295+0000", 2024-09-21T11:44:01.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.846296+0000", 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2099999999999995e-07 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.846303+0000", 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8979999999999999e-06 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.846324+0000", 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0482000000000001e-05 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.846333+0000", 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6940000000000007e-06 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.846418+0000", 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4567000000000005e-05 2024-09-21T11:44:01.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.846448+0000", 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9848999999999998e-05 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.940352+0000", 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.093904694999999996 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.940471+0000", 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011896900000000001 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:185191 7.f e144/82 7:f974449a:::benchmark_data_smithi043_70646_object67783:head v 144'11438, mlcod=144'11435)", 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:43:57.847426+0000", 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5524035380000001, 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.093172164000000002, 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.697 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.847426+0000", 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.847426+0000", 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.847427+0000", 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1800000000000005e-07 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.847433+0000", 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7690000000000002e-06 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.847442+0000", 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1360000000000003e-06 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.847448+0000", 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2940000000000002e-06 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.848848+0000", 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013995240000000001 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.848866+0000", 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8031999999999999e-05 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.940474+0000", 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.091608706999999998 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.940598+0000", 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012388399999999999 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:185197 7.3 7:c690d381:::benchmark_data_smithi043_70646_object67789:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:44:01.701 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:43:57.850030+0000", 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.549799266, 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.094613377999999998, 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 185197 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.850030+0000", 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.850030+0000", 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.850032+0000", 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.226e-06 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.850036+0000", 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4499999999999997e-06 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.850047+0000", 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0906e-05 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.850054+0000", 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0840000000000003e-06 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.851424+0000", 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013695859999999999 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.851609+0000", 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018515099999999999 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.933165+0000", 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.081555631000000003 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.940768+0000", 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0076039819999999996 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.941159+0000", 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00039013900000000001 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:01.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.944426+0000", 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003267795 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.944598+0000", 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017137599999999999 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.944644+0000", 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6051999999999999e-05 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:185198 7.c e144/82 7:3b04cd0d:::benchmark_data_smithi043_70646_object67790:head v 144'11723, mlcod=144'11721)", 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:43:57.852146+0000", 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5476835380000002, 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.088725937000000005, 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.852146+0000", 2024-09-21T11:44:01.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.852146+0000", 2024-09-21T11:44:01.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.852147+0000", 2024-09-21T11:44:01.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1999999999999998e-07 2024-09-21T11:44:01.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.852153+0000", 2024-09-21T11:44:01.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0700000000000003e-06 2024-09-21T11:44:01.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.933115+0000", 2024-09-21T11:44:01.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.080962005000000004 2024-09-21T11:44:01.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.933129+0000", 2024-09-21T11:44:01.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3705e-05 2024-09-21T11:44:01.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.933370+0000", 2024-09-21T11:44:01.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024160300000000001 2024-09-21T11:44:01.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.933391+0000", 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0704999999999999e-05 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.940827+0000", 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074362309999999997 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.940872+0000", 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4798e-05 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:185199 7.1 e144/82 7:848115c2:::benchmark_data_smithi043_70646_object67791:head v 144'11562, mlcod=144'11560)", 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:43:57.852161+0000", 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.547668732, 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.089391435000000005, 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.852161+0000", 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.852161+0000", 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.852162+0000", 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6299999999999993e-07 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.852168+0000", 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6359999999999999e-06 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.933673+0000", 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.081504956000000003 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.933683+0000", 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0168e-05 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.935956+0000", 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002273705 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.935975+0000", 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8394999999999999e-05 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.941503+0000", 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055279650000000001 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:57.941552+0000", 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9647000000000001e-05 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:186102 7.7 e144/82 7:e87d94d5:::benchmark_data_smithi043_70646_object68694:head v 144'11861, mlcod=144'11859)", 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:43:58.601557+0000", 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.7982728379999999, 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.087818758999999996, 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:01.718 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:58.601557+0000", 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:58.601557+0000", 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:58.601558+0000", 2024-09-21T11:44:01.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1900000000000001e-07 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:58.601563+0000", 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5099999999999998e-06 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:58.601572+0000", 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6479999999999992e-06 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:58.601579+0000", 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6510000000000004e-06 2024-09-21T11:44:01.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:58.601722+0000", 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014300399999999999 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:58.601740+0000", 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8734999999999999e-05 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:58.689234+0000", 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.087493286000000003 2024-09-21T11:44:01.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:01.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:01.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:44:01.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:43:58.689376+0000", 2024-09-21T11:44:01.722 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000142106 2024-09-21T11:44:01.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.722 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:01.722 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:01.722 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:44:01.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:01.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:01.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:01.853 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:01.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:01.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:01.955 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:01.984 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:01.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:01.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:02.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:02.108 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:02.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:02.109 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:02.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:02.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:02.239 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:02.239 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:02.340 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:02.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:02.375 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:02.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:02.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:02 smithi043 ceph-mon[29710]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:44:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:02 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/204064717' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:02.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:02 smithi043 ceph-mon[29710]: osdmap e145: 8 total, 8 up, 7 in 2024-09-21T11:44:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:02 smithi043 ceph-mon[36454]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:44:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:02 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/204064717' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:02 smithi043 ceph-mon[36454]: osdmap e145: 8 total, 8 up, 7 in 2024-09-21T11:44:02.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:02 smithi175 ceph-mon[32613]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:44:02.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:02 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/204064717' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:02 smithi175 ceph-mon[32613]: osdmap e145: 8 total, 8 up, 7 in 2024-09-21T11:44:02.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:02.514 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:02.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:02.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:02.651 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:02.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:02.652 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:02.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:02.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:02.782 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:02.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:02.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:02.911 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:02.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:02.912 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:03.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:03.044 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:03.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:03.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:03.146 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:03.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:03.188 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:03.188 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:03.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:03.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:03.329 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:03.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:03.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:03.435 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-21T11:43:59.783394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3413580339,"num_objects":52278,"num_object_clones":0,"num_object_copies":156834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52278,"num_whiteouts":0,"num_read":1066,"num_read_kb":923,"num_write":183282,"num_write_kb":7515594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38553,"num_bytes_recovered":2517897553,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77470,"ondisk_log_size":77470,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":11159872,"kb_used_data":10159952,"kb_used_omap":10,"kb_used_meta":999733,"kb_avail":645056192,"statfs":{"total":671965249536,"available":660537540608,"internally_reserved":0,"allocated":10403790848,"data_stored":10400930044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11156,"internal_metadata":1023726700},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,8,1,6,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-832045056,"num_objects":-12696,"num_object_clones":0,"num_object_copies":-38088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-12696,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12696,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.183107"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612642+0000","last_change":"2024-09-21T11:41:42.291880+0000","last_active":"2024-09-21T11:43:48.612642+0000","last_peered":"2024-09-21T11:43:48.612642+0000","last_clean":"2024-09-21T11:43:48.612642+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:43:48.612642+0000","last_undegraded":"2024-09-21T11:43:48.612642+0000","last_fullsized":"2024-09-21T11:43:48.612642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:42.290188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:42.290188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":220,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610167+0000","last_change":"2024-09-21T11:42:37.191052+0000","last_active":"2024-09-21T11:43:48.610167+0000","last_peered":"2024-09-21T11:43:48.610167+0000","last_clean":"2024-09-21T11:43:48.610167+0000","last_became_active":"2024-09-21T11:42:37.164858+0000","last_became_peered":"2024-09-21T11:42:37.164858+0000","last_unstale":"2024-09-21T11:43:48.610167+0000","last_undegraded":"2024-09-21T11:43:48.610167+0000","last_fullsized":"2024-09-21T11:43:48.610167+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:26.994148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:26.994148+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:16.329886+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":84,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.607088+0000","last_change":"2024-09-21T11:43:37.050216+0000","last_active":"2024-09-21T11:43:48.607088+0000","last_peered":"2024-09-21T11:43:48.607088+0000","last_clean":"2024-09-21T11:43:48.607088+0000","last_became_active":"2024-09-21T11:42:37.161713+0000","last_became_peered":"2024-09-21T11:42:37.161713+0000","last_unstale":"2024-09-21T11:43:48.607088+0000","last_undegraded":"2024-09-21T11:43:48.607088+0000","last_fullsized":"2024-09-21T11:43:48.607088+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:52.112502+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":186,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610082+0000","last_change":"2024-09-21T11:42:59.201398+0000","last_active":"2024-09-21T11:43:48.610082+0000","last_peered":"2024-09-21T11:43:48.610082+0000","last_clean":"2024-09-21T11:43:48.610082+0000","last_became_active":"2024-09-21T11:42:37.158907+0000","last_became_peered":"2024-09-21T11:42:37.158907+0000","last_unstale":"2024-09-21T11:43:48.610082+0000","last_undegraded":"2024-09-21T11:43:48.610082+0000","last_fullsized":"2024-09-21T11:43:48.610082+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:59.201330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:59.201330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:02.317360+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":179,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.605985+0000","last_change":"2024-09-21T11:43:28.461458+0000","last_active":"2024-09-21T11:43:48.605985+0000","last_peered":"2024-09-21T11:43:48.605985+0000","last_clean":"2024-09-21T11:43:48.605985+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:43:48.605985+0000","last_undegraded":"2024-09-21T11:43:48.605985+0000","last_fullsized":"2024-09-21T11:43:48.605985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.461378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.461378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.850394+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":183,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.608225+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:43:48.608225+0000","last_peered":"2024-09-21T11:43:48.608225+0000","last_clean":"2024-09-21T11:43:48.608225+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:43:48.608225+0000","last_undegraded":"2024-09-21T11:43:48.608225+0000","last_fullsized":"2024-09-21T11:43:48.608225+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":204,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606505+0000","last_change":"2024-09-21T11:40:02.315669+0000","last_active":"2024-09-21T11:43:48.606505+0000","last_peered":"2024-09-21T11:43:48.606505+0000","last_clean":"2024-09-21T11:43:48.606505+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:43:48.606505+0000","last_undegraded":"2024-09-21T11:43:48.606505+0000","last_fullsized":"2024-09-21T11:43:48.606505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":198,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.609877+0000","last_change":"2024-09-21T11:43:04.373004+0000","last_active":"2024-09-21T11:43:48.609877+0000","last_peered":"2024-09-21T11:43:48.609877+0000","last_clean":"2024-09-21T11:43:48.609877+0000","last_became_active":"2024-09-21T11:42:37.154511+0000","last_became_peered":"2024-09-21T11:42:37.154511+0000","last_unstale":"2024-09-21T11:43:48.609877+0000","last_undegraded":"2024-09-21T11:43:48.609877+0000","last_fullsized":"2024-09-21T11:43:48.609877+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:04.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:04.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:06.903626+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.607634+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:43:48.607634+0000","last_peered":"2024-09-21T11:43:48.607634+0000","last_clean":"2024-09-21T11:43:48.607634+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:43:48.607634+0000","last_undegraded":"2024-09-21T11:43:48.607634+0000","last_fullsized":"2024-09-21T11:43:48.607634+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":211,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610392+0000","last_change":"2024-09-21T11:43:26.258023+0000","last_active":"2024-09-21T11:43:48.610392+0000","last_peered":"2024-09-21T11:43:48.610392+0000","last_clean":"2024-09-21T11:43:48.610392+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:43:48.610392+0000","last_undegraded":"2024-09-21T11:43:48.610392+0000","last_fullsized":"2024-09-21T11:43:48.610392+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:26.257967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.257967+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:51.865812+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":182,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.608652+0000","last_change":"2024-09-21T11:41:32.885455+0000","last_active":"2024-09-21T11:43:48.608652+0000","last_peered":"2024-09-21T11:43:48.608652+0000","last_clean":"2024-09-21T11:43:48.608652+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:43:48.608652+0000","last_undegraded":"2024-09-21T11:43:48.608652+0000","last_fullsized":"2024-09-21T11:43:48.608652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:32.885407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:32.885407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":185,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.611960+0000","last_change":"2024-09-21T11:43:44.648053+0000","last_active":"2024-09-21T11:43:48.611960+0000","last_peered":"2024-09-21T11:43:48.611960+0000","last_clean":"2024-09-21T11:43:48.611960+0000","last_became_active":"2024-09-21T11:42:37.155879+0000","last_became_peered":"2024-09-21T11:42:37.155879+0000","last_unstale":"2024-09-21T11:43:48.611960+0000","last_undegraded":"2024-09-21T11:43:48.611960+0000","last_fullsized":"2024-09-21T11:43:48.611960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.647985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.647985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:56.842445+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":205,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606813+0000","last_change":"2024-09-21T11:42:37.156936+0000","last_active":"2024-09-21T11:43:48.606813+0000","last_peered":"2024-09-21T11:43:48.606813+0000","last_clean":"2024-09-21T11:43:48.606813+0000","last_became_active":"2024-09-21T11:42:37.156575+0000","last_became_peered":"2024-09-21T11:42:37.156575+0000","last_unstale":"2024-09-21T11:43:48.606813+0000","last_undegraded":"2024-09-21T11:43:48.606813+0000","last_fullsized":"2024-09-21T11:43:48.606813+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.101955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.101955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":218,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610073+0000","last_change":"2024-09-21T11:43:08.326923+0000","last_active":"2024-09-21T11:43:48.610073+0000","last_peered":"2024-09-21T11:43:48.610073+0000","last_clean":"2024-09-21T11:43:48.610073+0000","last_became_active":"2024-09-21T11:41:31.634863+0000","last_became_peered":"2024-09-21T11:41:31.634863+0000","last_unstale":"2024-09-21T11:43:48.610073+0000","last_undegraded":"2024-09-21T11:43:48.610073+0000","last_fullsized":"2024-09-21T11:43:48.610073+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:08.326846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:08.326846+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:14.001930+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":189,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613793+0000","last_change":"2024-09-21T11:43:38.567194+0000","last_active":"2024-09-21T11:43:48.613793+0000","last_peered":"2024-09-21T11:43:48.613793+0000","last_clean":"2024-09-21T11:43:48.613793+0000","last_became_active":"2024-09-21T11:42:37.164285+0000","last_became_peered":"2024-09-21T11:42:37.164285+0000","last_unstale":"2024-09-21T11:43:48.613793+0000","last_undegraded":"2024-09-21T11:43:48.613793+0000","last_fullsized":"2024-09-21T11:43:48.613793+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.567131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.567131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.325966+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":185,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.607197+0000","last_change":"2024-09-21T11:42:37.163213+0000","last_active":"2024-09-21T11:43:48.607197+0000","last_peered":"2024-09-21T11:43:48.607197+0000","last_clean":"2024-09-21T11:43:48.607197+0000","last_became_active":"2024-09-21T11:42:37.162912+0000","last_became_peered":"2024-09-21T11:42:37.162912+0000","last_unstale":"2024-09-21T11:43:48.607197+0000","last_undegraded":"2024-09-21T11:43:48.607197+0000","last_fullsized":"2024-09-21T11:43:48.607197+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:12.028860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:12.028860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":187,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610022+0000","last_change":"2024-09-21T11:43:22.300966+0000","last_active":"2024-09-21T11:43:48.610022+0000","last_peered":"2024-09-21T11:43:48.610022+0000","last_clean":"2024-09-21T11:43:48.610022+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:43:48.610022+0000","last_undegraded":"2024-09-21T11:43:48.610022+0000","last_fullsized":"2024-09-21T11:43:48.610022+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:22.300885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:22.300885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.343801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612648+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:43:48.612648+0000","last_peered":"2024-09-21T11:43:48.612648+0000","last_clean":"2024-09-21T11:43:48.612648+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:43:48.612648+0000","last_undegraded":"2024-09-21T11:43:48.612648+0000","last_fullsized":"2024-09-21T11:43:48.612648+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:50.720157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":266,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.614220+0000","last_change":"2024-09-21T11:43:31.498356+0000","last_active":"2024-09-21T11:43:48.614220+0000","last_peered":"2024-09-21T11:43:48.614220+0000","last_clean":"2024-09-21T11:43:48.614220+0000","last_became_active":"2024-09-21T11:42:37.166901+0000","last_became_peered":"2024-09-21T11:42:37.166901+0000","last_unstale":"2024-09-21T11:43:48.614220+0000","last_undegraded":"2024-09-21T11:43:48.614220+0000","last_fullsized":"2024-09-21T11:43:48.614220+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:31.498142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.498142+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:33.717651+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":84,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610547+0000","last_change":"2024-09-21T11:43:43.147999+0000","last_active":"2024-09-21T11:43:48.610547+0000","last_peered":"2024-09-21T11:43:48.610547+0000","last_clean":"2024-09-21T11:43:48.610547+0000","last_became_active":"2024-09-21T11:42:37.153799+0000","last_became_peered":"2024-09-21T11:42:37.153799+0000","last_unstale":"2024-09-21T11:43:48.610547+0000","last_undegraded":"2024-09-21T11:43:48.610547+0000","last_fullsized":"2024-09-21T11:43:48.610547+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:43.147942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:43.147942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:46.559755+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":179,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612543+0000","last_change":"2024-09-21T11:41:40.448106+0000","last_active":"2024-09-21T11:43:48.612543+0000","last_peered":"2024-09-21T11:43:48.612543+0000","last_clean":"2024-09-21T11:43:48.612543+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:43:48.612543+0000","last_undegraded":"2024-09-21T11:43:48.612543+0000","last_fullsized":"2024-09-21T11:43:48.612543+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.447908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.447908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":198,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610535+0000","last_change":"2024-09-21T11:43:45.093396+0000","last_active":"2024-09-21T11:43:48.610535+0000","last_peered":"2024-09-21T11:43:48.610535+0000","last_clean":"2024-09-21T11:43:48.610535+0000","last_became_active":"2024-09-21T11:42:37.153694+0000","last_became_peered":"2024-09-21T11:42:37.153694+0000","last_unstale":"2024-09-21T11:43:48.610535+0000","last_undegraded":"2024-09-21T11:43:48.610535+0000","last_fullsized":"2024-09-21T11:43:48.610535+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.093325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.093325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:14.273876+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":229,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.607866+0000","last_change":"2024-09-21T11:42:37.189830+0000","last_active":"2024-09-21T11:43:48.607866+0000","last_peered":"2024-09-21T11:43:48.607866+0000","last_clean":"2024-09-21T11:43:48.607866+0000","last_became_active":"2024-09-21T11:42:37.165964+0000","last_became_peered":"2024-09-21T11:42:37.165964+0000","last_unstale":"2024-09-21T11:43:48.607866+0000","last_undegraded":"2024-09-21T11:43:48.607866+0000","last_fullsized":"2024-09-21T11:43:48.607866+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:41:35.427210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:35.427210+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":183,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.609819+0000","last_change":"2024-09-21T11:43:02.320628+0000","last_active":"2024-09-21T11:43:48.609819+0000","last_peered":"2024-09-21T11:43:48.609819+0000","last_clean":"2024-09-21T11:43:48.609819+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:43:48.609819+0000","last_undegraded":"2024-09-21T11:43:48.609819+0000","last_fullsized":"2024-09-21T11:43:48.609819+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:02.320559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:02.320559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:24.560814+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":187,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613981+0000","last_change":"2024-09-21T11:42:31.278897+0000","last_active":"2024-09-21T11:43:48.613981+0000","last_peered":"2024-09-21T11:43:48.613981+0000","last_clean":"2024-09-21T11:43:48.613981+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:43:48.613981+0000","last_undegraded":"2024-09-21T11:43:48.613981+0000","last_fullsized":"2024-09-21T11:43:48.613981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:42:31.278824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:31.278824+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":88,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606413+0000","last_change":"2024-09-21T11:42:37.185608+0000","last_active":"2024-09-21T11:43:48.606413+0000","last_peered":"2024-09-21T11:43:48.606413+0000","last_clean":"2024-09-21T11:43:48.606413+0000","last_became_active":"2024-09-21T11:42:37.166144+0000","last_became_peered":"2024-09-21T11:42:37.166144+0000","last_unstale":"2024-09-21T11:43:48.606413+0000","last_undegraded":"2024-09-21T11:43:48.606413+0000","last_fullsized":"2024-09-21T11:43:48.606413+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:23.349301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.349301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":201,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613494+0000","last_change":"2024-09-21T11:43:29.595122+0000","last_active":"2024-09-21T11:43:48.613494+0000","last_peered":"2024-09-21T11:43:48.613494+0000","last_clean":"2024-09-21T11:43:48.613494+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:43:48.613494+0000","last_undegraded":"2024-09-21T11:43:48.613494+0000","last_fullsized":"2024-09-21T11:43:48.613494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:43:29.595040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.595040+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:41.200221+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":257,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:50.410920+0000","last_change":"2024-09-21T11:43:46.917623+0000","last_active":"2024-09-21T11:43:50.410920+0000","last_peered":"2024-09-21T11:43:50.410920+0000","last_clean":"2024-09-21T11:43:50.410920+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:43:50.410920+0000","last_undegraded":"2024-09-21T11:43:50.410920+0000","last_fullsized":"2024-09-21T11:43:50.410920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:43:46.917560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:46.917560+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:06.493510+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"144'11554","reported_seq":11747,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:55.754057+0000","last_change":"2024-09-21T11:43:35.987574+0000","last_active":"2024-09-21T11:43:55.754057+0000","last_peered":"2024-09-21T11:43:55.754057+0000","last_clean":"2024-09-21T11:43:55.754057+0000","last_became_active":"2024-09-21T11:41:10.304754+0000","last_became_peered":"2024-09-21T11:41:10.304754+0000","last_unstale":"2024-09-21T11:43:55.754057+0000","last_undegraded":"2024-09-21T11:43:55.754057+0000","last_fullsized":"2024-09-21T11:43:55.754057+0000","mapping_epoch":82,"log_start":"109'6546","ondisk_log_start":"109'6546","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'9957","last_scrub_stamp":"2024-09-21T11:43:35.987394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:35.987394+0000","objects_scrubbed":4941,"log_size":5008,"log_dups_size":0,"ondisk_log_size":5008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:47.433252+0000","scrub_duration":1245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216530944,"num_objects":3304,"num_object_clones":0,"num_object_copies":9912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11554,"num_write_kb":475456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":144,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612721+0000","last_change":"2024-09-21T11:43:44.440472+0000","last_active":"2024-09-21T11:43:48.612721+0000","last_peered":"2024-09-21T11:43:48.612721+0000","last_clean":"2024-09-21T11:43:48.612721+0000","last_became_active":"2024-09-21T11:42:37.169939+0000","last_became_peered":"2024-09-21T11:42:37.169939+0000","last_unstale":"2024-09-21T11:43:48.612721+0000","last_undegraded":"2024-09-21T11:43:48.612721+0000","last_fullsized":"2024-09-21T11:43:48.612721+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:44.440374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.440374+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:10.450444+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":198,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606670+0000","last_change":"2024-09-21T11:42:37.172945+0000","last_active":"2024-09-21T11:43:48.606670+0000","last_peered":"2024-09-21T11:43:48.606670+0000","last_clean":"2024-09-21T11:43:48.606670+0000","last_became_active":"2024-09-21T11:42:37.171365+0000","last_became_peered":"2024-09-21T11:42:37.171365+0000","last_unstale":"2024-09-21T11:43:48.606670+0000","last_undegraded":"2024-09-21T11:43:48.606670+0000","last_fullsized":"2024-09-21T11:43:48.606670+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.454207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.454207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":239,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:50.410711+0000","last_change":"2024-09-21T11:42:26.008760+0000","last_active":"2024-09-21T11:43:50.410711+0000","last_peered":"2024-09-21T11:43:50.410711+0000","last_clean":"2024-09-21T11:43:50.410711+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:43:50.410711+0000","last_undegraded":"2024-09-21T11:43:50.410711+0000","last_fullsized":"2024-09-21T11:43:50.410711+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:26.008646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:26.008646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:51.093540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":185,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606718+0000","last_change":"2024-09-21T11:42:37.157278+0000","last_active":"2024-09-21T11:43:48.606718+0000","last_peered":"2024-09-21T11:43:48.606718+0000","last_clean":"2024-09-21T11:43:48.606718+0000","last_became_active":"2024-09-21T11:42:37.156952+0000","last_became_peered":"2024-09-21T11:42:37.156952+0000","last_unstale":"2024-09-21T11:43:48.606718+0000","last_undegraded":"2024-09-21T11:43:48.606718+0000","last_fullsized":"2024-09-21T11:43:48.606718+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.066324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.066324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"144'11336","reported_seq":11472,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:56.129911+0000","last_change":"2024-09-21T11:41:10.297546+0000","last_active":"2024-09-21T11:43:56.129911+0000","last_peered":"2024-09-21T11:43:56.129911+0000","last_clean":"2024-09-21T11:43:56.129911+0000","last_became_active":"2024-09-21T11:41:10.297301+0000","last_became_peered":"2024-09-21T11:41:10.297301+0000","last_unstale":"2024-09-21T11:43:56.129911+0000","last_undegraded":"2024-09-21T11:43:56.129911+0000","last_fullsized":"2024-09-21T11:43:56.129911+0000","mapping_epoch":82,"log_start":"116'7051","ondisk_log_start":"116'7051","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4285,"log_dups_size":0,"ondisk_log_size":4285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":208928768,"num_objects":3188,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11336,"num_write_kb":464768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":219,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606435+0000","last_change":"2024-09-21T11:43:40.423834+0000","last_active":"2024-09-21T11:43:48.606435+0000","last_peered":"2024-09-21T11:43:48.606435+0000","last_clean":"2024-09-21T11:43:48.606435+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:43:48.606435+0000","last_undegraded":"2024-09-21T11:43:48.606435+0000","last_fullsized":"2024-09-21T11:43:48.606435+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:43:40.423750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.423750+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:01.136315+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":191,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612959+0000","last_change":"2024-09-21T11:41:29.388381+0000","last_active":"2024-09-21T11:43:48.612959+0000","last_peered":"2024-09-21T11:43:48.612959+0000","last_clean":"2024-09-21T11:43:48.612959+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:43:48.612959+0000","last_undegraded":"2024-09-21T11:43:48.612959+0000","last_fullsized":"2024-09-21T11:43:48.612959+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:29.388322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:29.388322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":183,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.609767+0000","last_change":"2024-09-21T11:41:40.030553+0000","last_active":"2024-09-21T11:43:48.609767+0000","last_peered":"2024-09-21T11:43:48.609767+0000","last_clean":"2024-09-21T11:43:48.609767+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:43:48.609767+0000","last_undegraded":"2024-09-21T11:43:48.609767+0000","last_fullsized":"2024-09-21T11:43:48.609767+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.030493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.030493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:11.327373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":171,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610307+0000","last_change":"2024-09-21T11:40:06.343742+0000","last_active":"2024-09-21T11:43:48.610307+0000","last_peered":"2024-09-21T11:43:48.610307+0000","last_clean":"2024-09-21T11:43:48.610307+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:43:48.610307+0000","last_undegraded":"2024-09-21T11:43:48.610307+0000","last_fullsized":"2024-09-21T11:43:48.610307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:10.356658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"144'11270","reported_seq":24015,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:56.131075+0000","last_change":"2024-09-21T11:43:25.943980+0000","last_active":"2024-09-21T11:43:56.131075+0000","last_peered":"2024-09-21T11:43:56.131075+0000","last_clean":"2024-09-21T11:43:56.131075+0000","last_became_active":"2024-09-21T11:43:25.930319+0000","last_became_peered":"2024-09-21T11:43:25.930319+0000","last_unstale":"2024-09-21T11:43:56.131075+0000","last_undegraded":"2024-09-21T11:43:56.131075+0000","last_fullsized":"2024-09-21T11:43:56.131075+0000","mapping_epoch":133,"log_start":"116'7014","ondisk_log_start":"116'7014","created":82,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4256,"log_dups_size":0,"ondisk_log_size":4256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":207224832,"num_objects":3162,"num_object_clones":0,"num_object_copies":9486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11270,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6296,"num_bytes_recovered":412614656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":208,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610207+0000","last_change":"2024-09-21T11:41:41.007761+0000","last_active":"2024-09-21T11:43:48.610207+0000","last_peered":"2024-09-21T11:43:48.610207+0000","last_clean":"2024-09-21T11:43:48.610207+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:43:48.610207+0000","last_undegraded":"2024-09-21T11:43:48.610207+0000","last_fullsized":"2024-09-21T11:43:48.610207+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:41:41.007695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:41.007695+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:15.335018+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":202,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613696+0000","last_change":"2024-09-21T11:42:37.182056+0000","last_active":"2024-09-21T11:43:48.613696+0000","last_peered":"2024-09-21T11:43:48.613696+0000","last_clean":"2024-09-21T11:43:48.613696+0000","last_became_active":"2024-09-21T11:42:37.159390+0000","last_became_peered":"2024-09-21T11:42:37.159390+0000","last_unstale":"2024-09-21T11:43:48.613696+0000","last_undegraded":"2024-09-21T11:43:48.613696+0000","last_fullsized":"2024-09-21T11:43:48.613696+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:41:34.236667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:34.236667+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"114'9","reported_seq":251,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:50.415225+0000","last_change":"2024-09-21T11:43:29.417021+0000","last_active":"2024-09-21T11:43:50.415225+0000","last_peered":"2024-09-21T11:43:50.415225+0000","last_clean":"2024-09-21T11:43:50.415225+0000","last_became_active":"2024-09-21T11:42:37.169838+0000","last_became_peered":"2024-09-21T11:42:37.169838+0000","last_unstale":"2024-09-21T11:43:50.415225+0000","last_undegraded":"2024-09-21T11:43:50.415225+0000","last_fullsized":"2024-09-21T11:43:50.415225+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:29.416946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.416946+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:30.090032+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":181,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.611787+0000","last_change":"2024-09-21T11:41:39.527210+0000","last_active":"2024-09-21T11:43:48.611787+0000","last_peered":"2024-09-21T11:43:48.611787+0000","last_clean":"2024-09-21T11:43:48.611787+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:43:48.611787+0000","last_undegraded":"2024-09-21T11:43:48.611787+0000","last_fullsized":"2024-09-21T11:43:48.611787+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.527151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.527151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"144'11375","reported_seq":11585,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:55.707260+0000","last_change":"2024-09-21T11:43:05.145633+0000","last_active":"2024-09-21T11:43:55.707260+0000","last_peered":"2024-09-21T11:43:55.707260+0000","last_clean":"2024-09-21T11:43:55.707260+0000","last_became_active":"2024-09-21T11:41:10.300541+0000","last_became_peered":"2024-09-21T11:41:10.300541+0000","last_unstale":"2024-09-21T11:43:55.707260+0000","last_undegraded":"2024-09-21T11:43:55.707260+0000","last_fullsized":"2024-09-21T11:43:55.707260+0000","mapping_epoch":82,"log_start":"111'6733","ondisk_log_start":"111'6733","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'8200","last_scrub_stamp":"2024-09-21T11:43:05.145547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:05.145547+0000","objects_scrubbed":6512,"log_size":4642,"log_dups_size":0,"ondisk_log_size":4642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.858838+0000","scrub_duration":1824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215416832,"num_objects":3287,"num_object_clones":0,"num_object_copies":9861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11375,"num_write_kb":469184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":122,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606944+0000","last_change":"2024-09-21T11:42:37.207015+0000","last_active":"2024-09-21T11:43:48.606944+0000","last_peered":"2024-09-21T11:43:48.606944+0000","last_clean":"2024-09-21T11:43:48.606944+0000","last_became_active":"2024-09-21T11:42:37.165120+0000","last_became_peered":"2024-09-21T11:42:37.165120+0000","last_unstale":"2024-09-21T11:43:48.606944+0000","last_undegraded":"2024-09-21T11:43:48.606944+0000","last_fullsized":"2024-09-21T11:43:48.606944+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:15.360632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.360632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.607708+0000","last_change":"2024-09-21T11:41:40.866256+0000","last_active":"2024-09-21T11:43:48.607708+0000","last_peered":"2024-09-21T11:43:48.607708+0000","last_clean":"2024-09-21T11:43:48.607708+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:43:48.607708+0000","last_undegraded":"2024-09-21T11:43:48.607708+0000","last_fullsized":"2024-09-21T11:43:48.607708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.866156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.866156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":241,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:50.410659+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:43:50.410659+0000","last_peered":"2024-09-21T11:43:50.410659+0000","last_clean":"2024-09-21T11:43:50.410659+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:43:50.410659+0000","last_undegraded":"2024-09-21T11:43:50.410659+0000","last_fullsized":"2024-09-21T11:43:50.410659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":178,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612108+0000","last_change":"2024-09-21T11:41:40.278404+0000","last_active":"2024-09-21T11:43:48.612108+0000","last_peered":"2024-09-21T11:43:48.612108+0000","last_clean":"2024-09-21T11:43:48.612108+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:43:48.612108+0000","last_undegraded":"2024-09-21T11:43:48.612108+0000","last_fullsized":"2024-09-21T11:43:48.612108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.278315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.278315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"144'11405","reported_seq":24247,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:55.741435+0000","last_change":"2024-09-21T11:43:27.957009+0000","last_active":"2024-09-21T11:43:55.741435+0000","last_peered":"2024-09-21T11:43:55.741435+0000","last_clean":"2024-09-21T11:43:55.741435+0000","last_became_active":"2024-09-21T11:43:27.938453+0000","last_became_peered":"2024-09-21T11:43:27.938453+0000","last_unstale":"2024-09-21T11:43:55.741435+0000","last_undegraded":"2024-09-21T11:43:55.741435+0000","last_fullsized":"2024-09-21T11:43:55.741435+0000","mapping_epoch":135,"log_start":"106'6164","ondisk_log_start":"106'6164","created":82,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":5241,"log_dups_size":0,"ondisk_log_size":5241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216858624,"num_objects":3309,"num_object_clones":0,"num_object_copies":9927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11405,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6345,"num_bytes_recovered":415825920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":222,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612253+0000","last_change":"2024-09-21T11:41:07.065583+0000","last_active":"2024-09-21T11:43:48.612253+0000","last_peered":"2024-09-21T11:43:48.612253+0000","last_clean":"2024-09-21T11:43:48.612253+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:43:48.612253+0000","last_undegraded":"2024-09-21T11:43:48.612253+0000","last_fullsized":"2024-09-21T11:43:48.612253+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:41:07.065507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.065507+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":83,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.614270+0000","last_change":"2024-09-21T11:43:30.509571+0000","last_active":"2024-09-21T11:43:48.614270+0000","last_peered":"2024-09-21T11:43:48.614270+0000","last_clean":"2024-09-21T11:43:48.614270+0000","last_became_active":"2024-09-21T11:42:37.157562+0000","last_became_peered":"2024-09-21T11:42:37.157562+0000","last_unstale":"2024-09-21T11:43:48.614270+0000","last_undegraded":"2024-09-21T11:43:48.614270+0000","last_fullsized":"2024-09-21T11:43:48.614270+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.509505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.509505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:42.953900+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":182,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.600852+0000","last_change":"2024-09-21T11:43:26.704117+0000","last_active":"2024-09-21T11:43:48.600852+0000","last_peered":"2024-09-21T11:43:48.600852+0000","last_clean":"2024-09-21T11:43:48.600852+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:43:48.600852+0000","last_undegraded":"2024-09-21T11:43:48.600852+0000","last_fullsized":"2024-09-21T11:43:48.600852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.704073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.704073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:55.097023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":83,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.603127+0000","last_change":"2024-09-21T11:43:37.819912+0000","last_active":"2024-09-21T11:43:48.603127+0000","last_peered":"2024-09-21T11:43:48.603127+0000","last_clean":"2024-09-21T11:43:48.603127+0000","last_became_active":"2024-09-21T11:42:37.155437+0000","last_became_peered":"2024-09-21T11:42:37.155437+0000","last_unstale":"2024-09-21T11:43:48.603127+0000","last_undegraded":"2024-09-21T11:43:48.603127+0000","last_fullsized":"2024-09-21T11:43:48.603127+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.544593+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"144'11641","reported_seq":11845,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:55.727827+0000","last_change":"2024-09-21T11:42:22.352124+0000","last_active":"2024-09-21T11:43:55.727827+0000","last_peered":"2024-09-21T11:43:55.727827+0000","last_clean":"2024-09-21T11:43:55.727827+0000","last_became_active":"2024-09-21T11:41:10.304723+0000","last_became_peered":"2024-09-21T11:41:10.304723+0000","last_unstale":"2024-09-21T11:43:55.727827+0000","last_undegraded":"2024-09-21T11:43:55.727827+0000","last_fullsized":"2024-09-21T11:43:55.727827+0000","mapping_epoch":82,"log_start":"110'6663","ondisk_log_start":"110'6663","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'6088","last_scrub_stamp":"2024-09-21T11:42:22.351940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.351940+0000","objects_scrubbed":6032,"log_size":4978,"log_dups_size":0,"ondisk_log_size":4978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:48.894013+0000","scrub_duration":1670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":217513984,"num_objects":3319,"num_object_clones":0,"num_object_copies":9957,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11641,"num_write_kb":478720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"140'30","reported_seq":221,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:50.412064+0000","last_change":"2024-09-21T11:43:37.416987+0000","last_active":"2024-09-21T11:43:50.412064+0000","last_peered":"2024-09-21T11:43:50.412064+0000","last_clean":"2024-09-21T11:43:50.412064+0000","last_became_active":"2024-09-21T11:43:32.003036+0000","last_became_peered":"2024-09-21T11:43:32.003036+0000","last_unstale":"2024-09-21T11:43:50.412064+0000","last_undegraded":"2024-09-21T11:43:50.412064+0000","last_fullsized":"2024-09-21T11:43:50.412064+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.678954+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":197,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610513+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:43:48.610513+0000","last_peered":"2024-09-21T11:43:48.610513+0000","last_clean":"2024-09-21T11:43:48.610513+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:43:48.610513+0000","last_undegraded":"2024-09-21T11:43:48.610513+0000","last_fullsized":"2024-09-21T11:43:48.610513+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:14.022118+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":211,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.607195+0000","last_change":"2024-09-21T11:42:37.193243+0000","last_active":"2024-09-21T11:43:48.607195+0000","last_peered":"2024-09-21T11:43:48.607195+0000","last_clean":"2024-09-21T11:43:48.607195+0000","last_became_active":"2024-09-21T11:42:37.164723+0000","last_became_peered":"2024-09-21T11:42:37.164723+0000","last_unstale":"2024-09-21T11:43:48.607195+0000","last_undegraded":"2024-09-21T11:43:48.607195+0000","last_fullsized":"2024-09-21T11:43:48.607195+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:42:17.142454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.142454+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":189,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.614249+0000","last_change":"2024-09-21T11:42:37.164162+0000","last_active":"2024-09-21T11:43:48.614249+0000","last_peered":"2024-09-21T11:43:48.614249+0000","last_clean":"2024-09-21T11:43:48.614249+0000","last_became_active":"2024-09-21T11:42:37.163682+0000","last_became_peered":"2024-09-21T11:42:37.163682+0000","last_unstale":"2024-09-21T11:43:48.614249+0000","last_undegraded":"2024-09-21T11:43:48.614249+0000","last_fullsized":"2024-09-21T11:43:48.614249+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:24.468909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.468909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.266739+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":179,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.607958+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:43:48.607958+0000","last_peered":"2024-09-21T11:43:48.607958+0000","last_clean":"2024-09-21T11:43:48.607958+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:43:48.607958+0000","last_undegraded":"2024-09-21T11:43:48.607958+0000","last_fullsized":"2024-09-21T11:43:48.607958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:50.884536+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"144'11254","reported_seq":24361,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:56.228031+0000","last_change":"2024-09-21T11:43:25.018476+0000","last_active":"2024-09-21T11:43:56.228031+0000","last_peered":"2024-09-21T11:43:56.228031+0000","last_clean":"2024-09-21T11:43:56.228031+0000","last_became_active":"2024-09-21T11:43:24.947584+0000","last_became_peered":"2024-09-21T11:43:24.947584+0000","last_unstale":"2024-09-21T11:43:56.228031+0000","last_undegraded":"2024-09-21T11:43:56.228031+0000","last_fullsized":"2024-09-21T11:43:56.228031+0000","mapping_epoch":132,"log_start":"111'6620","ondisk_log_start":"111'6620","created":82,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4634,"log_dups_size":0,"ondisk_log_size":4634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:52.616980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216530944,"num_objects":3304,"num_object_clones":0,"num_object_copies":9912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11254,"num_write_kb":465856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6476,"num_bytes_recovered":424280064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":134,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.608312+0000","last_change":"2024-09-21T11:42:37.188687+0000","last_active":"2024-09-21T11:43:48.608312+0000","last_peered":"2024-09-21T11:43:48.608312+0000","last_clean":"2024-09-21T11:43:48.608312+0000","last_became_active":"2024-09-21T11:42:37.161797+0000","last_became_peered":"2024-09-21T11:42:37.161797+0000","last_unstale":"2024-09-21T11:43:48.608312+0000","last_undegraded":"2024-09-21T11:43:48.608312+0000","last_fullsized":"2024-09-21T11:43:48.608312+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":88,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.616481+0000","last_change":"2024-09-21T11:42:37.182044+0000","last_active":"2024-09-21T11:43:48.616481+0000","last_peered":"2024-09-21T11:43:48.616481+0000","last_clean":"2024-09-21T11:43:48.616481+0000","last_became_active":"2024-09-21T11:42:37.156191+0000","last_became_peered":"2024-09-21T11:42:37.156191+0000","last_unstale":"2024-09-21T11:43:48.616481+0000","last_undegraded":"2024-09-21T11:43:48.616481+0000","last_fullsized":"2024-09-21T11:43:48.616481+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:52.863101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":208,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.614105+0000","last_change":"2024-09-21T11:43:36.446391+0000","last_active":"2024-09-21T11:43:48.614105+0000","last_peered":"2024-09-21T11:43:48.614105+0000","last_clean":"2024-09-21T11:43:48.614105+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:43:48.614105+0000","last_undegraded":"2024-09-21T11:43:48.614105+0000","last_fullsized":"2024-09-21T11:43:48.614105+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:43:36.446341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.446341+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.312398+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":191,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.614438+0000","last_change":"2024-09-21T11:43:28.505134+0000","last_active":"2024-09-21T11:43:48.614438+0000","last_peered":"2024-09-21T11:43:48.614438+0000","last_clean":"2024-09-21T11:43:48.614438+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:43:48.614438+0000","last_undegraded":"2024-09-21T11:43:48.614438+0000","last_fullsized":"2024-09-21T11:43:48.614438+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.504894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.504894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.779358+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":178,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612153+0000","last_change":"2024-09-21T11:42:25.245624+0000","last_active":"2024-09-21T11:43:48.612153+0000","last_peered":"2024-09-21T11:43:48.612153+0000","last_clean":"2024-09-21T11:43:48.612153+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:43:48.612153+0000","last_undegraded":"2024-09-21T11:43:48.612153+0000","last_fullsized":"2024-09-21T11:43:48.612153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.245565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.245565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"144'11495","reported_seq":11681,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:55.729492+0000","last_change":"2024-09-21T11:43:44.550062+0000","last_active":"2024-09-21T11:43:55.729492+0000","last_peered":"2024-09-21T11:43:55.729492+0000","last_clean":"2024-09-21T11:43:55.729492+0000","last_became_active":"2024-09-21T11:41:10.302903+0000","last_became_peered":"2024-09-21T11:41:10.302903+0000","last_unstale":"2024-09-21T11:43:55.729492+0000","last_undegraded":"2024-09-21T11:43:55.729492+0000","last_fullsized":"2024-09-21T11:43:55.729492+0000","mapping_epoch":82,"log_start":"115'6928","ondisk_log_start":"115'6928","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'10628","last_scrub_stamp":"2024-09-21T11:43:44.550004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.550004+0000","objects_scrubbed":4219,"log_size":4567,"log_dups_size":0,"ondisk_log_size":4567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:05.776791+0000","scrub_duration":1037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":217776128,"num_objects":3323,"num_object_clones":0,"num_object_copies":9969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11495,"num_write_kb":474176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":225,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613678+0000","last_change":"2024-09-21T11:43:40.607996+0000","last_active":"2024-09-21T11:43:48.613678+0000","last_peered":"2024-09-21T11:43:48.613678+0000","last_clean":"2024-09-21T11:43:48.613678+0000","last_became_active":"2024-09-21T11:42:37.164059+0000","last_became_peered":"2024-09-21T11:42:37.164059+0000","last_unstale":"2024-09-21T11:43:48.613678+0000","last_undegraded":"2024-09-21T11:43:48.613678+0000","last_fullsized":"2024-09-21T11:43:48.613678+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:43:40.607801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.607801+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:59.037561+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612932+0000","last_change":"2024-09-21T11:43:42.641937+0000","last_active":"2024-09-21T11:43:48.612932+0000","last_peered":"2024-09-21T11:43:48.612932+0000","last_clean":"2024-09-21T11:43:48.612932+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:43:48.612932+0000","last_undegraded":"2024-09-21T11:43:48.612932+0000","last_fullsized":"2024-09-21T11:43:48.612932+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:42.641865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:42.641865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:44.867283+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":450,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:54.280368+0000","last_change":"2024-09-21T11:43:47.494999+0000","last_active":"2024-09-21T11:43:54.280368+0000","last_peered":"2024-09-21T11:43:54.280368+0000","last_clean":"2024-09-21T11:43:54.280368+0000","last_became_active":"2024-09-21T11:42:37.166141+0000","last_became_peered":"2024-09-21T11:42:37.166141+0000","last_unstale":"2024-09-21T11:43:54.280368+0000","last_undegraded":"2024-09-21T11:43:54.280368+0000","last_fullsized":"2024-09-21T11:43:54.280368+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:47.494865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:47.494865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:04.635839+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":229,"num_read_kb":224,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":189,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.608130+0000","last_change":"2024-09-21T11:43:45.046195+0000","last_active":"2024-09-21T11:43:48.608130+0000","last_peered":"2024-09-21T11:43:48.608130+0000","last_clean":"2024-09-21T11:43:48.608130+0000","last_became_active":"2024-09-21T11:42:37.172270+0000","last_became_peered":"2024-09-21T11:42:37.172270+0000","last_unstale":"2024-09-21T11:43:48.608130+0000","last_undegraded":"2024-09-21T11:43:48.608130+0000","last_fullsized":"2024-09-21T11:43:48.608130+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.046128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.046128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:59.675637+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":75,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606899+0000","last_change":"2024-09-21T11:42:37.173082+0000","last_active":"2024-09-21T11:43:48.606899+0000","last_peered":"2024-09-21T11:43:48.606899+0000","last_clean":"2024-09-21T11:43:48.606899+0000","last_became_active":"2024-09-21T11:42:37.172645+0000","last_became_peered":"2024-09-21T11:42:37.172645+0000","last_unstale":"2024-09-21T11:43:48.606899+0000","last_undegraded":"2024-09-21T11:43:48.606899+0000","last_fullsized":"2024-09-21T11:43:48.606899+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:31.250421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:31.250421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"144'11619","reported_seq":11803,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:56.218534+0000","last_change":"2024-09-21T11:42:31.712433+0000","last_active":"2024-09-21T11:43:56.218534+0000","last_peered":"2024-09-21T11:43:56.218534+0000","last_clean":"2024-09-21T11:43:56.218534+0000","last_became_active":"2024-09-21T11:41:10.302497+0000","last_became_peered":"2024-09-21T11:41:10.302497+0000","last_unstale":"2024-09-21T11:43:56.218534+0000","last_undegraded":"2024-09-21T11:43:56.218534+0000","last_fullsized":"2024-09-21T11:43:56.218534+0000","mapping_epoch":82,"log_start":"115'7042","ondisk_log_start":"115'7042","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":4349,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.653561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":214106112,"num_objects":3267,"num_object_clones":0,"num_object_copies":9801,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11619,"num_write_kb":476352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":256,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:50.410684+0000","last_change":"2024-09-21T11:42:19.703071+0000","last_active":"2024-09-21T11:43:50.410684+0000","last_peered":"2024-09-21T11:43:50.410684+0000","last_clean":"2024-09-21T11:43:50.410684+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:43:50.410684+0000","last_undegraded":"2024-09-21T11:43:50.410684+0000","last_fullsized":"2024-09-21T11:43:50.410684+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:42:19.703007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.703007+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.857413+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":191,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.608659+0000","last_change":"2024-09-21T11:43:30.437283+0000","last_active":"2024-09-21T11:43:48.608659+0000","last_peered":"2024-09-21T11:43:48.608659+0000","last_clean":"2024-09-21T11:43:48.608659+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:43:48.608659+0000","last_undegraded":"2024-09-21T11:43:48.608659+0000","last_fullsized":"2024-09-21T11:43:48.608659+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.437080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.437080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.933810+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":236,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612386+0000","last_change":"2024-09-21T11:42:24.254810+0000","last_active":"2024-09-21T11:43:48.612386+0000","last_peered":"2024-09-21T11:43:48.612386+0000","last_clean":"2024-09-21T11:43:48.612386+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:43:48.612386+0000","last_undegraded":"2024-09-21T11:43:48.612386+0000","last_fullsized":"2024-09-21T11:43:48.612386+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:42:24.254749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.254749+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:55.557145+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":182,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.614116+0000","last_change":"2024-09-21T11:43:31.630737+0000","last_active":"2024-09-21T11:43:48.614116+0000","last_peered":"2024-09-21T11:43:48.614116+0000","last_clean":"2024-09-21T11:43:48.614116+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:43:48.614116+0000","last_undegraded":"2024-09-21T11:43:48.614116+0000","last_fullsized":"2024-09-21T11:43:48.614116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:31.630670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.630670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.782053+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":178,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.608005+0000","last_change":"2024-09-21T11:41:44.929146+0000","last_active":"2024-09-21T11:43:48.608005+0000","last_peered":"2024-09-21T11:43:48.608005+0000","last_clean":"2024-09-21T11:43:48.608005+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:43:48.608005+0000","last_undegraded":"2024-09-21T11:43:48.608005+0000","last_fullsized":"2024-09-21T11:43:48.608005+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:44.929079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:44.929079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"144'11456","reported_seq":24576,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:55.712193+0000","last_change":"2024-09-21T11:43:48.624147+0000","last_active":"2024-09-21T11:43:55.712193+0000","last_peered":"2024-09-21T11:43:55.712193+0000","last_clean":"2024-09-21T11:43:55.712193+0000","last_became_active":"2024-09-21T11:43:26.984538+0000","last_became_peered":"2024-09-21T11:43:26.984538+0000","last_unstale":"2024-09-21T11:43:55.712193+0000","last_undegraded":"2024-09-21T11:43:55.712193+0000","last_fullsized":"2024-09-21T11:43:55.712193+0000","mapping_epoch":134,"log_start":"108'6464","ondisk_log_start":"108'6464","created":82,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'5645","last_scrub_stamp":"2024-09-21T11:42:17.339535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.339535+0000","objects_scrubbed":2149,"log_size":4992,"log_dups_size":0,"ondisk_log_size":4992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:18.608570+0000","scrub_duration":1564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213319704,"num_objects":3256,"num_object_clones":0,"num_object_copies":9768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11456,"num_write_kb":470721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6438,"num_bytes_recovered":421855256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":225,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610550+0000","last_change":"2024-09-21T11:42:37.179956+0000","last_active":"2024-09-21T11:43:48.610550+0000","last_peered":"2024-09-21T11:43:48.610550+0000","last_clean":"2024-09-21T11:43:48.610550+0000","last_became_active":"2024-09-21T11:42:37.158158+0000","last_became_peered":"2024-09-21T11:42:37.158158+0000","last_unstale":"2024-09-21T11:43:48.610550+0000","last_undegraded":"2024-09-21T11:43:48.610550+0000","last_fullsized":"2024-09-21T11:43:48.610550+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:22.041174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.041174+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":191,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610314+0000","last_change":"2024-09-21T11:42:58.177899+0000","last_active":"2024-09-21T11:43:48.610314+0000","last_peered":"2024-09-21T11:43:48.610314+0000","last_clean":"2024-09-21T11:43:48.610314+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:43:48.610314+0000","last_undegraded":"2024-09-21T11:43:48.610314+0000","last_fullsized":"2024-09-21T11:43:48.610314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:58.177844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:58.177844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:06.969504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":182,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.607905+0000","last_change":"2024-09-21T11:43:44.049478+0000","last_active":"2024-09-21T11:43:48.607905+0000","last_peered":"2024-09-21T11:43:48.607905+0000","last_clean":"2024-09-21T11:43:48.607905+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:43:48.607905+0000","last_undegraded":"2024-09-21T11:43:48.607905+0000","last_fullsized":"2024-09-21T11:43:48.607905+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.049424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.049424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:05.726834+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":179,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.605272+0000","last_change":"2024-09-21T11:41:38.453843+0000","last_active":"2024-09-21T11:43:48.605272+0000","last_peered":"2024-09-21T11:43:48.605272+0000","last_clean":"2024-09-21T11:43:48.605272+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:43:48.605272+0000","last_undegraded":"2024-09-21T11:43:48.605272+0000","last_fullsized":"2024-09-21T11:43:48.605272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:38.453579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:38.453579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"144'11460","reported_seq":11717,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:55.847336+0000","last_change":"2024-09-21T11:43:47.086277+0000","last_active":"2024-09-21T11:43:55.847336+0000","last_peered":"2024-09-21T11:43:55.847336+0000","last_clean":"2024-09-21T11:43:55.847336+0000","last_became_active":"2024-09-21T11:41:10.300107+0000","last_became_peered":"2024-09-21T11:41:10.300107+0000","last_unstale":"2024-09-21T11:43:55.847336+0000","last_undegraded":"2024-09-21T11:43:55.847336+0000","last_fullsized":"2024-09-21T11:43:55.847336+0000","mapping_epoch":82,"log_start":"105'5964","ondisk_log_start":"105'5964","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'10760","last_scrub_stamp":"2024-09-21T11:43:47.085319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:47.085319+0000","objects_scrubbed":4008,"log_size":5496,"log_dups_size":0,"ondisk_log_size":5496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:03.663829+0000","scrub_duration":1601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":211943424,"num_objects":3234,"num_object_clones":0,"num_object_copies":9702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11460,"num_write_kb":470208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":240,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610825+0000","last_change":"2024-09-21T11:43:39.233466+0000","last_active":"2024-09-21T11:43:48.610825+0000","last_peered":"2024-09-21T11:43:48.610825+0000","last_clean":"2024-09-21T11:43:48.610825+0000","last_became_active":"2024-09-21T11:42:37.160867+0000","last_became_peered":"2024-09-21T11:42:37.160867+0000","last_unstale":"2024-09-21T11:43:48.610825+0000","last_undegraded":"2024-09-21T11:43:48.610825+0000","last_fullsized":"2024-09-21T11:43:48.610825+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:43:39.233386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.233386+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.064116+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":205,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.601260+0000","last_change":"2024-09-21T11:43:36.784531+0000","last_active":"2024-09-21T11:43:48.601260+0000","last_peered":"2024-09-21T11:43:48.601260+0000","last_clean":"2024-09-21T11:43:48.601260+0000","last_became_active":"2024-09-21T11:42:37.167043+0000","last_became_peered":"2024-09-21T11:42:37.167043+0000","last_unstale":"2024-09-21T11:43:48.601260+0000","last_undegraded":"2024-09-21T11:43:48.601260+0000","last_fullsized":"2024-09-21T11:43:48.601260+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.784467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.784467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:39.372586+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":84,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.609775+0000","last_change":"2024-09-21T11:43:41.216278+0000","last_active":"2024-09-21T11:43:48.609775+0000","last_peered":"2024-09-21T11:43:48.609775+0000","last_clean":"2024-09-21T11:43:48.609775+0000","last_became_active":"2024-09-21T11:42:37.160659+0000","last_became_peered":"2024-09-21T11:42:37.160659+0000","last_unstale":"2024-09-21T11:43:48.609775+0000","last_undegraded":"2024-09-21T11:43:48.609775+0000","last_fullsized":"2024-09-21T11:43:48.609775+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:41.216215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:41.216215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:45.696826+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":187,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.614523+0000","last_change":"2024-09-21T11:43:46.516054+0000","last_active":"2024-09-21T11:43:48.614523+0000","last_peered":"2024-09-21T11:43:48.614523+0000","last_clean":"2024-09-21T11:43:48.614523+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:43:48.614523+0000","last_undegraded":"2024-09-21T11:43:48.614523+0000","last_fullsized":"2024-09-21T11:43:48.614523+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:46.515782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:46.515782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.893835+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"144'11317","reported_seq":11524,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:56.218906+0000","last_change":"2024-09-21T11:42:29.618177+0000","last_active":"2024-09-21T11:43:56.218906+0000","last_peered":"2024-09-21T11:43:56.218906+0000","last_clean":"2024-09-21T11:43:56.218906+0000","last_became_active":"2024-09-21T11:41:10.303333+0000","last_became_peered":"2024-09-21T11:41:10.303333+0000","last_unstale":"2024-09-21T11:43:56.218906+0000","last_undegraded":"2024-09-21T11:43:56.218906+0000","last_fullsized":"2024-09-21T11:43:56.218906+0000","mapping_epoch":82,"log_start":"114'6744","ondisk_log_start":"114'6744","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6263","last_scrub_stamp":"2024-09-21T11:42:29.617794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.617794+0000","objects_scrubbed":6197,"log_size":4573,"log_dups_size":0,"ondisk_log_size":4573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209649664,"num_objects":3199,"num_object_clones":0,"num_object_copies":9597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11317,"num_write_kb":464512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":210,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.609871+0000","last_change":"2024-09-21T11:43:07.347397+0000","last_active":"2024-09-21T11:43:48.609871+0000","last_peered":"2024-09-21T11:43:48.609871+0000","last_clean":"2024-09-21T11:43:48.609871+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:43:48.609871+0000","last_undegraded":"2024-09-21T11:43:48.609871+0000","last_fullsized":"2024-09-21T11:43:48.609871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:43:07.347346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:07.347346+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.982030+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":191,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606449+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:43:48.606449+0000","last_peered":"2024-09-21T11:43:48.606449+0000","last_clean":"2024-09-21T11:43:48.606449+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:43:48.606449+0000","last_undegraded":"2024-09-21T11:43:48.606449+0000","last_fullsized":"2024-09-21T11:43:48.606449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":247,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:50.412239+0000","last_change":"2024-09-21T11:43:38.421538+0000","last_active":"2024-09-21T11:43:50.412239+0000","last_peered":"2024-09-21T11:43:50.412239+0000","last_clean":"2024-09-21T11:43:50.412239+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:43:50.412239+0000","last_undegraded":"2024-09-21T11:43:50.412239+0000","last_fullsized":"2024-09-21T11:43:50.412239+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:43:38.421251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.421251+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:48.875567+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":186,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610383+0000","last_change":"2024-09-21T11:42:50.131895+0000","last_active":"2024-09-21T11:43:48.610383+0000","last_peered":"2024-09-21T11:43:48.610383+0000","last_clean":"2024-09-21T11:43:48.610383+0000","last_became_active":"2024-09-21T11:42:37.157732+0000","last_became_peered":"2024-09-21T11:42:37.157732+0000","last_unstale":"2024-09-21T11:43:48.610383+0000","last_undegraded":"2024-09-21T11:43:48.610383+0000","last_fullsized":"2024-09-21T11:43:48.610383+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:50.131830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:50.131830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:52.943303+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"144'11496","reported_seq":11690,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:55.829879+0000","last_change":"2024-09-21T11:43:37.155559+0000","last_active":"2024-09-21T11:43:55.829879+0000","last_peered":"2024-09-21T11:43:55.829879+0000","last_clean":"2024-09-21T11:43:55.829879+0000","last_became_active":"2024-09-21T11:41:10.304149+0000","last_became_peered":"2024-09-21T11:41:10.304149+0000","last_unstale":"2024-09-21T11:43:55.829879+0000","last_undegraded":"2024-09-21T11:43:55.829879+0000","last_fullsized":"2024-09-21T11:43:55.829879+0000","mapping_epoch":82,"log_start":"105'6002","ondisk_log_start":"105'6002","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'10015","last_scrub_stamp":"2024-09-21T11:43:37.155379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.155379+0000","objects_scrubbed":4788,"log_size":5494,"log_dups_size":0,"ondisk_log_size":5494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:48.799347+0000","scrub_duration":1732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212729856,"num_objects":3246,"num_object_clones":0,"num_object_copies":9738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11496,"num_write_kb":471744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":219,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612166+0000","last_change":"2024-09-21T11:42:37.192718+0000","last_active":"2024-09-21T11:43:48.612166+0000","last_peered":"2024-09-21T11:43:48.612166+0000","last_clean":"2024-09-21T11:43:48.612166+0000","last_became_active":"2024-09-21T11:42:37.168751+0000","last_became_peered":"2024-09-21T11:42:37.168751+0000","last_unstale":"2024-09-21T11:43:48.612166+0000","last_undegraded":"2024-09-21T11:43:48.612166+0000","last_fullsized":"2024-09-21T11:43:48.612166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:15.417817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.417817+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606989+0000","last_change":"2024-09-21T11:41:28.936695+0000","last_active":"2024-09-21T11:43:48.606989+0000","last_peered":"2024-09-21T11:43:48.606989+0000","last_clean":"2024-09-21T11:43:48.606989+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:43:48.606989+0000","last_undegraded":"2024-09-21T11:43:48.606989+0000","last_fullsized":"2024-09-21T11:43:48.606989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:28.936645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:28.936645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":183,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.619571+0000","last_change":"2024-09-21T11:42:18.265457+0000","last_active":"2024-09-21T11:43:48.619571+0000","last_peered":"2024-09-21T11:43:48.619571+0000","last_clean":"2024-09-21T11:43:48.619571+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:43:48.619571+0000","last_undegraded":"2024-09-21T11:43:48.619571+0000","last_fullsized":"2024-09-21T11:43:48.619571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.265392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.265392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":170,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612847+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:43:48.612847+0000","last_peered":"2024-09-21T11:43:48.612847+0000","last_clean":"2024-09-21T11:43:48.612847+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:43:48.612847+0000","last_undegraded":"2024-09-21T11:43:48.612847+0000","last_fullsized":"2024-09-21T11:43:48.612847+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"144'11330","reported_seq":24584,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:56.219910+0000","last_change":"2024-09-21T11:43:22.892096+0000","last_active":"2024-09-21T11:43:56.219910+0000","last_peered":"2024-09-21T11:43:56.219910+0000","last_clean":"2024-09-21T11:43:56.219910+0000","last_became_active":"2024-09-21T11:43:22.875072+0000","last_became_peered":"2024-09-21T11:43:22.875072+0000","last_unstale":"2024-09-21T11:43:56.219910+0000","last_undegraded":"2024-09-21T11:43:56.219910+0000","last_fullsized":"2024-09-21T11:43:56.219910+0000","mapping_epoch":130,"log_start":"114'6750","ondisk_log_start":"114'6750","created":82,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4580,"log_dups_size":0,"ondisk_log_size":4580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":210501632,"num_objects":3212,"num_object_clones":0,"num_object_copies":9636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11330,"num_write_kb":465344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6549,"num_bytes_recovered":429195264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":244,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.601030+0000","last_change":"2024-09-21T11:43:27.726443+0000","last_active":"2024-09-21T11:43:48.601030+0000","last_peered":"2024-09-21T11:43:48.601030+0000","last_clean":"2024-09-21T11:43:48.601030+0000","last_became_active":"2024-09-21T11:42:37.166373+0000","last_became_peered":"2024-09-21T11:42:37.166373+0000","last_unstale":"2024-09-21T11:43:48.601030+0000","last_undegraded":"2024-09-21T11:43:48.601030+0000","last_fullsized":"2024-09-21T11:43:48.601030+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:43:27.726370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:27.726370+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:36.734856+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612492+0000","last_change":"2024-09-21T11:41:46.471940+0000","last_active":"2024-09-21T11:43:48.612492+0000","last_peered":"2024-09-21T11:43:48.612492+0000","last_clean":"2024-09-21T11:43:48.612492+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:43:48.612492+0000","last_undegraded":"2024-09-21T11:43:48.612492+0000","last_fullsized":"2024-09-21T11:43:48.612492+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:46.471872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:46.471872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":191,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.619425+0000","last_change":"2024-09-21T11:43:26.484397+0000","last_active":"2024-09-21T11:43:48.619425+0000","last_peered":"2024-09-21T11:43:48.619425+0000","last_clean":"2024-09-21T11:43:48.619425+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:43:48.619425+0000","last_undegraded":"2024-09-21T11:43:48.619425+0000","last_fullsized":"2024-09-21T11:43:48.619425+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.484353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.484353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:38.261052+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":178,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.611814+0000","last_change":"2024-09-21T11:43:29.650485+0000","last_active":"2024-09-21T11:43:48.611814+0000","last_peered":"2024-09-21T11:43:48.611814+0000","last_clean":"2024-09-21T11:43:48.611814+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:43:48.611814+0000","last_undegraded":"2024-09-21T11:43:48.611814+0000","last_fullsized":"2024-09-21T11:43:48.611814+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:29.650427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.650427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:32.208091+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"144'11479","reported_seq":11623,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:56.130896+0000","last_change":"2024-09-21T11:42:31.693667+0000","last_active":"2024-09-21T11:43:56.130896+0000","last_peered":"2024-09-21T11:43:56.130896+0000","last_clean":"2024-09-21T11:43:56.130896+0000","last_became_active":"2024-09-21T11:41:10.300287+0000","last_became_peered":"2024-09-21T11:41:10.300287+0000","last_unstale":"2024-09-21T11:43:56.130896+0000","last_undegraded":"2024-09-21T11:43:56.130896+0000","last_fullsized":"2024-09-21T11:43:56.130896+0000","mapping_epoch":82,"log_start":"116'7154","ondisk_log_start":"116'7154","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":449,"log_size":4325,"log_dups_size":0,"ondisk_log_size":4325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":210567168,"num_objects":3213,"num_object_clones":0,"num_object_copies":9639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11479,"num_write_kb":470144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":237,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610057+0000","last_change":"2024-09-21T11:43:32.231463+0000","last_active":"2024-09-21T11:43:48.610057+0000","last_peered":"2024-09-21T11:43:48.610057+0000","last_clean":"2024-09-21T11:43:48.610057+0000","last_became_active":"2024-09-21T11:42:37.165364+0000","last_became_peered":"2024-09-21T11:42:37.165364+0000","last_unstale":"2024-09-21T11:43:48.610057+0000","last_undegraded":"2024-09-21T11:43:48.610057+0000","last_fullsized":"2024-09-21T11:43:48.610057+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:43:32.231381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.231381+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:36.630886+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":205,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612616+0000","last_change":"2024-09-21T11:43:44.519748+0000","last_active":"2024-09-21T11:43:48.612616+0000","last_peered":"2024-09-21T11:43:48.612616+0000","last_clean":"2024-09-21T11:43:48.612616+0000","last_became_active":"2024-09-21T11:42:37.165470+0000","last_became_peered":"2024-09-21T11:42:37.165470+0000","last_unstale":"2024-09-21T11:43:48.612616+0000","last_undegraded":"2024-09-21T11:43:48.612616+0000","last_fullsized":"2024-09-21T11:43:48.612616+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.519704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.519704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:47.181516+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":198,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.616342+0000","last_change":"2024-09-21T11:43:40.502911+0000","last_active":"2024-09-21T11:43:48.616342+0000","last_peered":"2024-09-21T11:43:48.616342+0000","last_clean":"2024-09-21T11:43:48.616342+0000","last_became_active":"2024-09-21T11:42:37.155675+0000","last_became_peered":"2024-09-21T11:42:37.155675+0000","last_unstale":"2024-09-21T11:43:48.616342+0000","last_undegraded":"2024-09-21T11:43:48.616342+0000","last_fullsized":"2024-09-21T11:43:48.616342+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:40.502747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.502747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:57.625738+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":186,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606087+0000","last_change":"2024-09-21T11:42:37.170697+0000","last_active":"2024-09-21T11:43:48.606087+0000","last_peered":"2024-09-21T11:43:48.606087+0000","last_clean":"2024-09-21T11:43:48.606087+0000","last_became_active":"2024-09-21T11:42:37.168686+0000","last_became_peered":"2024-09-21T11:42:37.168686+0000","last_unstale":"2024-09-21T11:43:48.606087+0000","last_undegraded":"2024-09-21T11:43:48.606087+0000","last_fullsized":"2024-09-21T11:43:48.606087+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:27.327920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:27.327920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"144'11246","reported_seq":24090,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:55.747826+0000","last_change":"2024-09-21T11:43:27.000045+0000","last_active":"2024-09-21T11:43:55.747826+0000","last_peered":"2024-09-21T11:43:55.747826+0000","last_clean":"2024-09-21T11:43:55.747826+0000","last_became_active":"2024-09-21T11:43:26.983757+0000","last_became_peered":"2024-09-21T11:43:26.983757+0000","last_unstale":"2024-09-21T11:43:55.747826+0000","last_undegraded":"2024-09-21T11:43:55.747826+0000","last_fullsized":"2024-09-21T11:43:55.747826+0000","mapping_epoch":134,"log_start":"106'5980","ondisk_log_start":"106'5980","created":82,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6370","last_scrub_stamp":"2024-09-21T11:42:31.280157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:31.280157+0000","objects_scrubbed":6318,"log_size":5266,"log_dups_size":0,"ondisk_log_size":5266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:58.540352+0000","scrub_duration":2077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213385216,"num_objects":3256,"num_object_clones":0,"num_object_copies":9768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11246,"num_write_kb":464064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6310,"num_bytes_recovered":413532160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":203,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606605+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:43:48.606605+0000","last_peered":"2024-09-21T11:43:48.606605+0000","last_clean":"2024-09-21T11:43:48.606605+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:43:48.606605+0000","last_undegraded":"2024-09-21T11:43:48.606605+0000","last_fullsized":"2024-09-21T11:43:48.606605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":202,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613004+0000","last_change":"2024-09-21T11:42:19.352990+0000","last_active":"2024-09-21T11:43:48.613004+0000","last_peered":"2024-09-21T11:43:48.613004+0000","last_clean":"2024-09-21T11:43:48.613004+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:43:48.613004+0000","last_undegraded":"2024-09-21T11:43:48.613004+0000","last_fullsized":"2024-09-21T11:43:48.613004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:42:19.352895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.352895+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":238,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:50.410982+0000","last_change":"2024-09-21T11:41:43.495040+0000","last_active":"2024-09-21T11:43:50.410982+0000","last_peered":"2024-09-21T11:43:50.410982+0000","last_clean":"2024-09-21T11:43:50.410982+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:43:50.410982+0000","last_undegraded":"2024-09-21T11:43:50.410982+0000","last_fullsized":"2024-09-21T11:43:50.410982+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:41:43.494960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:43.494960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:51.520216+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":178,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606708+0000","last_change":"2024-09-21T11:41:39.867162+0000","last_active":"2024-09-21T11:43:48.606708+0000","last_peered":"2024-09-21T11:43:48.606708+0000","last_clean":"2024-09-21T11:43:48.606708+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:43:48.606708+0000","last_undegraded":"2024-09-21T11:43:48.606708+0000","last_fullsized":"2024-09-21T11:43:48.606708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.867108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.867108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.600591+0000","last_change":"2024-09-21T11:43:30.734704+0000","last_active":"2024-09-21T11:43:48.600591+0000","last_peered":"2024-09-21T11:43:48.600591+0000","last_clean":"2024-09-21T11:43:48.600591+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:43:48.600591+0000","last_undegraded":"2024-09-21T11:43:48.600591+0000","last_fullsized":"2024-09-21T11:43:48.600591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.734361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.734361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:35.190586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":211,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612816+0000","last_change":"2024-09-21T11:42:37.201056+0000","last_active":"2024-09-21T11:43:48.612816+0000","last_peered":"2024-09-21T11:43:48.612816+0000","last_clean":"2024-09-21T11:43:48.612816+0000","last_became_active":"2024-09-21T11:42:37.171563+0000","last_became_peered":"2024-09-21T11:42:37.171563+0000","last_unstale":"2024-09-21T11:43:48.612816+0000","last_undegraded":"2024-09-21T11:43:48.612816+0000","last_fullsized":"2024-09-21T11:43:48.612816+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:41:25.310957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:25.310957+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":189,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.608000+0000","last_change":"2024-09-21T11:42:37.170710+0000","last_active":"2024-09-21T11:43:48.608000+0000","last_peered":"2024-09-21T11:43:48.608000+0000","last_clean":"2024-09-21T11:43:48.608000+0000","last_became_active":"2024-09-21T11:42:37.170365+0000","last_became_peered":"2024-09-21T11:42:37.170365+0000","last_unstale":"2024-09-21T11:43:48.608000+0000","last_undegraded":"2024-09-21T11:43:48.608000+0000","last_fullsized":"2024-09-21T11:43:48.608000+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:15.090108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.090108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":186,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.611959+0000","last_change":"2024-09-21T11:43:45.551832+0000","last_active":"2024-09-21T11:43:48.611959+0000","last_peered":"2024-09-21T11:43:48.611959+0000","last_clean":"2024-09-21T11:43:48.611959+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:43:48.611959+0000","last_undegraded":"2024-09-21T11:43:48.611959+0000","last_fullsized":"2024-09-21T11:43:48.611959+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.551792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.551792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.431636+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":219,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613602+0000","last_change":"2024-09-21T11:43:32.077598+0000","last_active":"2024-09-21T11:43:48.613602+0000","last_peered":"2024-09-21T11:43:48.613602+0000","last_clean":"2024-09-21T11:43:48.613602+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:43:48.613602+0000","last_undegraded":"2024-09-21T11:43:48.613602+0000","last_fullsized":"2024-09-21T11:43:48.613602+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:11.469922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.469922+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612050+0000","last_change":"2024-09-21T11:42:23.241888+0000","last_active":"2024-09-21T11:43:48.612050+0000","last_peered":"2024-09-21T11:43:48.612050+0000","last_clean":"2024-09-21T11:43:48.612050+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:43:48.612050+0000","last_undegraded":"2024-09-21T11:43:48.612050+0000","last_fullsized":"2024-09-21T11:43:48.612050+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:23.241796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.241796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612635+0000","last_change":"2024-09-21T11:43:38.461076+0000","last_active":"2024-09-21T11:43:48.612635+0000","last_peered":"2024-09-21T11:43:48.612635+0000","last_clean":"2024-09-21T11:43:48.612635+0000","last_became_active":"2024-09-21T11:42:37.172423+0000","last_became_peered":"2024-09-21T11:42:37.172423+0000","last_unstale":"2024-09-21T11:43:48.612635+0000","last_undegraded":"2024-09-21T11:43:48.612635+0000","last_fullsized":"2024-09-21T11:43:48.612635+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.461014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.461014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:44.750422+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":187,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613877+0000","last_change":"2024-09-21T11:43:42.448731+0000","last_active":"2024-09-21T11:43:48.613877+0000","last_peered":"2024-09-21T11:43:48.613877+0000","last_clean":"2024-09-21T11:43:48.613877+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:43:48.613877+0000","last_undegraded":"2024-09-21T11:43:48.613877+0000","last_fullsized":"2024-09-21T11:43:48.613877+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:42.448669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:42.448669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.700328+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":202,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.607615+0000","last_change":"2024-09-21T11:41:33.917474+0000","last_active":"2024-09-21T11:43:48.607615+0000","last_peered":"2024-09-21T11:43:48.607615+0000","last_clean":"2024-09-21T11:43:48.607615+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:43:48.607615+0000","last_undegraded":"2024-09-21T11:43:48.607615+0000","last_fullsized":"2024-09-21T11:43:48.607615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:33.917389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:33.917389+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612087+0000","last_change":"2024-09-21T11:42:22.478692+0000","last_active":"2024-09-21T11:43:48.612087+0000","last_peered":"2024-09-21T11:43:48.612087+0000","last_clean":"2024-09-21T11:43:48.612087+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:43:48.612087+0000","last_undegraded":"2024-09-21T11:43:48.612087+0000","last_fullsized":"2024-09-21T11:43:48.612087+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:22.478619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.478619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.263376+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":231,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:50.412736+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:43:50.412736+0000","last_peered":"2024-09-21T11:43:50.412736+0000","last_clean":"2024-09-21T11:43:50.412736+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:43:50.412736+0000","last_undegraded":"2024-09-21T11:43:50.412736+0000","last_fullsized":"2024-09-21T11:43:50.412736+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":187,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610144+0000","last_change":"2024-09-21T11:43:24.300706+0000","last_active":"2024-09-21T11:43:48.610144+0000","last_peered":"2024-09-21T11:43:48.610144+0000","last_clean":"2024-09-21T11:43:48.610144+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:43:48.610144+0000","last_undegraded":"2024-09-21T11:43:48.610144+0000","last_fullsized":"2024-09-21T11:43:48.610144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:24.300642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:24.300642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.984128+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":221,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.605662+0000","last_change":"2024-09-21T11:42:37.207434+0000","last_active":"2024-09-21T11:43:48.605662+0000","last_peered":"2024-09-21T11:43:48.605662+0000","last_clean":"2024-09-21T11:43:48.605662+0000","last_became_active":"2024-09-21T11:42:37.171345+0000","last_became_peered":"2024-09-21T11:42:37.171345+0000","last_unstale":"2024-09-21T11:43:48.605662+0000","last_undegraded":"2024-09-21T11:43:48.605662+0000","last_fullsized":"2024-09-21T11:43:48.605662+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.379125+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612092+0000","last_change":"2024-09-21T11:42:11.317760+0000","last_active":"2024-09-21T11:43:48.612092+0000","last_peered":"2024-09-21T11:43:48.612092+0000","last_clean":"2024-09-21T11:43:48.612092+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:43:48.612092+0000","last_undegraded":"2024-09-21T11:43:48.612092+0000","last_fullsized":"2024-09-21T11:43:48.612092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.317696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.317696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"114'9","reported_seq":250,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:50.410642+0000","last_change":"2024-09-21T11:43:28.003108+0000","last_active":"2024-09-21T11:43:50.410642+0000","last_peered":"2024-09-21T11:43:50.410642+0000","last_clean":"2024-09-21T11:43:50.410642+0000","last_became_active":"2024-09-21T11:42:37.163771+0000","last_became_peered":"2024-09-21T11:42:37.163771+0000","last_unstale":"2024-09-21T11:43:50.410642+0000","last_undegraded":"2024-09-21T11:43:50.410642+0000","last_fullsized":"2024-09-21T11:43:50.410642+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:28.003036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.003036+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:32.544327+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":178,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613275+0000","last_change":"2024-09-21T11:41:30.274387+0000","last_active":"2024-09-21T11:43:48.613275+0000","last_peered":"2024-09-21T11:43:48.613275+0000","last_clean":"2024-09-21T11:43:48.613275+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:43:48.613275+0000","last_undegraded":"2024-09-21T11:43:48.613275+0000","last_fullsized":"2024-09-21T11:43:48.613275+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:30.274201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:30.274201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":233,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610281+0000","last_change":"2024-09-21T11:43:40.233160+0000","last_active":"2024-09-21T11:43:48.610281+0000","last_peered":"2024-09-21T11:43:48.610281+0000","last_clean":"2024-09-21T11:43:48.610281+0000","last_became_active":"2024-09-21T11:42:37.154835+0000","last_became_peered":"2024-09-21T11:42:37.154835+0000","last_unstale":"2024-09-21T11:43:48.610281+0000","last_undegraded":"2024-09-21T11:43:48.610281+0000","last_fullsized":"2024-09-21T11:43:48.610281+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:43:40.233067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.233067+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:41.481160+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":191,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.610192+0000","last_change":"2024-09-21T11:43:00.245058+0000","last_active":"2024-09-21T11:43:48.610192+0000","last_peered":"2024-09-21T11:43:48.610192+0000","last_clean":"2024-09-21T11:43:48.610192+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:43:48.610192+0000","last_undegraded":"2024-09-21T11:43:48.610192+0000","last_fullsized":"2024-09-21T11:43:48.610192+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:00.244985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:00.244985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:22.792144+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":76,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606835+0000","last_change":"2024-09-21T11:42:37.163592+0000","last_active":"2024-09-21T11:43:48.606835+0000","last_peered":"2024-09-21T11:43:48.606835+0000","last_clean":"2024-09-21T11:43:48.606835+0000","last_became_active":"2024-09-21T11:42:37.163261+0000","last_became_peered":"2024-09-21T11:42:37.163261+0000","last_unstale":"2024-09-21T11:43:48.606835+0000","last_undegraded":"2024-09-21T11:43:48.606835+0000","last_fullsized":"2024-09-21T11:43:48.606835+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.301406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.301406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:51.735442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":171,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.605975+0000","last_change":"2024-09-21T11:40:06.353577+0000","last_active":"2024-09-21T11:43:48.605975+0000","last_peered":"2024-09-21T11:43:48.605975+0000","last_clean":"2024-09-21T11:43:48.605975+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:43:48.605975+0000","last_undegraded":"2024-09-21T11:43:48.605975+0000","last_fullsized":"2024-09-21T11:43:48.605975+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":129,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.601169+0000","last_change":"2024-09-21T11:43:41.808987+0000","last_active":"2024-09-21T11:43:48.601169+0000","last_peered":"2024-09-21T11:43:48.601169+0000","last_clean":"2024-09-21T11:43:48.601169+0000","last_became_active":"2024-09-21T11:42:37.167265+0000","last_became_peered":"2024-09-21T11:42:37.167265+0000","last_unstale":"2024-09-21T11:43:48.601169+0000","last_undegraded":"2024-09-21T11:43:48.601169+0000","last_fullsized":"2024-09-21T11:43:48.601169+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:41.808900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:41.808900+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.057274+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612356+0000","last_change":"2024-09-21T11:43:26.586603+0000","last_active":"2024-09-21T11:43:48.612356+0000","last_peered":"2024-09-21T11:43:48.612356+0000","last_clean":"2024-09-21T11:43:48.612356+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:43:48.612356+0000","last_undegraded":"2024-09-21T11:43:48.612356+0000","last_fullsized":"2024-09-21T11:43:48.612356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.586562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.586562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:55.496321+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606188+0000","last_change":"2024-09-21T11:42:37.167224+0000","last_active":"2024-09-21T11:43:48.606188+0000","last_peered":"2024-09-21T11:43:48.606188+0000","last_clean":"2024-09-21T11:43:48.606188+0000","last_became_active":"2024-09-21T11:42:37.166917+0000","last_became_peered":"2024-09-21T11:42:37.166917+0000","last_unstale":"2024-09-21T11:43:48.606188+0000","last_undegraded":"2024-09-21T11:43:48.606188+0000","last_fullsized":"2024-09-21T11:43:48.606188+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:23.514572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.514572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.611786+0000","last_change":"2024-09-21T11:42:37.177478+0000","last_active":"2024-09-21T11:43:48.611786+0000","last_peered":"2024-09-21T11:43:48.611786+0000","last_clean":"2024-09-21T11:43:48.611786+0000","last_became_active":"2024-09-21T11:42:37.161589+0000","last_became_peered":"2024-09-21T11:42:37.161589+0000","last_unstale":"2024-09-21T11:43:48.611786+0000","last_undegraded":"2024-09-21T11:43:48.611786+0000","last_fullsized":"2024-09-21T11:43:48.611786+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:41:29.236463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:29.236463+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":206,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606130+0000","last_change":"2024-09-21T11:42:15.470041+0000","last_active":"2024-09-21T11:43:48.606130+0000","last_peered":"2024-09-21T11:43:48.606130+0000","last_clean":"2024-09-21T11:43:48.606130+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:43:48.606130+0000","last_undegraded":"2024-09-21T11:43:48.606130+0000","last_fullsized":"2024-09-21T11:43:48.606130+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:15.469421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.469421+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":197,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.608127+0000","last_change":"2024-09-21T11:42:37.173431+0000","last_active":"2024-09-21T11:43:48.608127+0000","last_peered":"2024-09-21T11:43:48.608127+0000","last_clean":"2024-09-21T11:43:48.608127+0000","last_became_active":"2024-09-21T11:42:37.172859+0000","last_became_peered":"2024-09-21T11:42:37.172859+0000","last_unstale":"2024-09-21T11:43:48.608127+0000","last_undegraded":"2024-09-21T11:43:48.608127+0000","last_fullsized":"2024-09-21T11:43:48.608127+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.952335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.952335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":182,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.611325+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:43:48.611325+0000","last_peered":"2024-09-21T11:43:48.611325+0000","last_clean":"2024-09-21T11:43:48.611325+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:43:48.611325+0000","last_undegraded":"2024-09-21T11:43:48.611325+0000","last_fullsized":"2024-09-21T11:43:48.611325+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.612728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":179,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606354+0000","last_change":"2024-09-21T11:42:12.409717+0000","last_active":"2024-09-21T11:43:48.606354+0000","last_peered":"2024-09-21T11:43:48.606354+0000","last_clean":"2024-09-21T11:43:48.606354+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:43:48.606354+0000","last_undegraded":"2024-09-21T11:43:48.606354+0000","last_fullsized":"2024-09-21T11:43:48.606354+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:12.409641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:12.409641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":219,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606247+0000","last_change":"2024-09-21T11:42:37.195076+0000","last_active":"2024-09-21T11:43:48.606247+0000","last_peered":"2024-09-21T11:43:48.606247+0000","last_clean":"2024-09-21T11:43:48.606247+0000","last_became_active":"2024-09-21T11:42:37.169482+0000","last_became_peered":"2024-09-21T11:42:37.169482+0000","last_unstale":"2024-09-21T11:43:48.606247+0000","last_undegraded":"2024-09-21T11:43:48.606247+0000","last_fullsized":"2024-09-21T11:43:48.606247+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:41:32.414244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:32.414244+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":198,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613003+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:43:48.613003+0000","last_peered":"2024-09-21T11:43:48.613003+0000","last_clean":"2024-09-21T11:43:48.613003+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:43:48.613003+0000","last_undegraded":"2024-09-21T11:43:48.613003+0000","last_fullsized":"2024-09-21T11:43:48.613003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":190,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.603209+0000","last_change":"2024-09-21T11:43:39.838067+0000","last_active":"2024-09-21T11:43:48.603209+0000","last_peered":"2024-09-21T11:43:48.603209+0000","last_clean":"2024-09-21T11:43:48.603209+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:43:48.603209+0000","last_undegraded":"2024-09-21T11:43:48.603209+0000","last_fullsized":"2024-09-21T11:43:48.603209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:39.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.838002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.931489+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":178,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.612963+0000","last_change":"2024-09-21T11:42:30.269470+0000","last_active":"2024-09-21T11:43:48.612963+0000","last_peered":"2024-09-21T11:43:48.612963+0000","last_clean":"2024-09-21T11:43:48.612963+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:43:48.612963+0000","last_undegraded":"2024-09-21T11:43:48.612963+0000","last_fullsized":"2024-09-21T11:43:48.612963+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:30.269408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:30.269408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:43:53.929437+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":75,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606820+0000","last_change":"2024-09-21T11:42:37.155891+0000","last_active":"2024-09-21T11:43:48.606820+0000","last_peered":"2024-09-21T11:43:48.606820+0000","last_clean":"2024-09-21T11:43:48.606820+0000","last_became_active":"2024-09-21T11:42:37.155620+0000","last_became_peered":"2024-09-21T11:42:37.155620+0000","last_unstale":"2024-09-21T11:43:48.606820+0000","last_undegraded":"2024-09-21T11:43:48.606820+0000","last_fullsized":"2024-09-21T11:43:48.606820+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.358678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.358678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":198,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.606593+0000","last_change":"2024-09-21T11:43:36.413767+0000","last_active":"2024-09-21T11:43:48.606593+0000","last_peered":"2024-09-21T11:43:48.606593+0000","last_clean":"2024-09-21T11:43:48.606593+0000","last_became_active":"2024-09-21T11:42:37.167248+0000","last_became_peered":"2024-09-21T11:42:37.167248+0000","last_unstale":"2024-09-21T11:43:48.606593+0000","last_undegraded":"2024-09-21T11:43:48.606593+0000","last_fullsized":"2024-09-21T11:43:48.606593+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.413725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.413725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.736856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":232,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.603005+0000","last_change":"2024-09-21T11:43:32.709031+0000","last_active":"2024-09-21T11:43:48.603005+0000","last_peered":"2024-09-21T11:43:48.603005+0000","last_clean":"2024-09-21T11:43:48.603005+0000","last_became_active":"2024-09-21T11:42:37.171253+0000","last_became_peered":"2024-09-21T11:42:37.171253+0000","last_unstale":"2024-09-21T11:43:48.603005+0000","last_undegraded":"2024-09-21T11:43:48.603005+0000","last_fullsized":"2024-09-21T11:43:48.603005+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:43:32.708849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.708849+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.362664+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":182,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-21T11:43:48.613392+0000","last_change":"2024-09-21T11:41:31.249298+0000","last_active":"2024-09-21T11:43:48.613392+0000","last_peered":"2024-09-21T11:43:48.613392+0000","last_clean":"2024-09-21T11:43:48.613392+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:43:48.613392+0000","last_undegraded":"2024-09-21T11:43:48.613392+0000","last_fullsized":"2024-09-21T11:43:48.613392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:31.249215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:31.249215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":3412983832,"num_objects":52079,"num_object_clones":0,"num_object_copies":156237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182733,"num_write_kb":7513921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38414,"num_bytes_recovered":2517303320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12024430592,"data_stored":12024414304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76914,"ondisk_log_size":76914,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":5367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":229,"num_read_kb":224,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234036,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1652220,"kb_used_data":1472492,"kb_used_omap":1,"kb_used_meta":179710,"kb_avail":92092932,"statfs":{"total":95995035648,"available":94303162368,"internally_reserved":0,"allocated":1507831808,"data_stored":1507418149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":184023498},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1444812,"kb_used_data":1297020,"kb_used_omap":1,"kb_used_meta":147774,"kb_avail":92300340,"statfs":{"total":95995035648,"available":94515548160,"internally_reserved":0,"allocated":1328148480,"data_stored":1327719286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":151321030},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724606,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":916784,"kb_used_data":849504,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":92828368,"statfs":{"total":95995035648,"available":95056248832,"internally_reserved":0,"allocated":869892096,"data_stored":869489560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1176240,"kb_used_data":1064544,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92568912,"statfs":{"total":95995035648,"available":94790565888,"internally_reserved":0,"allocated":1090093056,"data_stored":1089691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117066,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2286004,"kb_used_data":2102116,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":91459148,"statfs":{"total":95995035648,"available":93654167552,"internally_reserved":0,"allocated":2152566784,"data_stored":2152160368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":188283335},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,1,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1868824,"kb_used_data":1684872,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":91876328,"statfs":{"total":95995035648,"available":94081359872,"internally_reserved":0,"allocated":1725308928,"data_stored":1724930822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":188283333},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":55834574935,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1814988,"kb_used_data":1689404,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":91930164,"statfs":{"total":95995035648,"available":94136487936,"internally_reserved":0,"allocated":1729949696,"data_stored":1729520691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1700925440,"data_stored":1700921368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1781600256,"data_stored":1781596184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1701642240,"data_stored":1701642240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2120028160,"data_stored":2120024088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1071841280,"data_stored":1071841280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":855113728,"data_stored":855113728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1294602240,"data_stored":1294598168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1498677248,"data_stored":1498677248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:44:03.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool set .mgr pgp_num 1 2024-09-21T11:44:03.456 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:03.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:03.457 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:03.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:03 smithi043 ceph-mon[29710]: pgmap v181: 148 pgs: 148 active+clean; 2.8 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:44:03.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:03 smithi043 ceph-mon[36454]: pgmap v181: 148 pgs: 148 active+clean; 2.8 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:44:03.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:03 smithi175 ceph-mon[32613]: pgmap v181: 148 pgs: 148 active+clean; 2.8 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:44:03.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:03.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:03.587 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:03.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:03.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:03.716 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:03.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:03.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:03.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:03.848 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:03.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:03.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:03.950 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:03.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:44:03.977 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:03.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:03.978 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:04.012 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:04.069 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:04.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:04.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:04.127 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:04.127 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:04.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:04.265 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:04.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:04.266 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:04.349 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:04.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:04.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:04.401 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:04.401 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:04.502 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:04.529 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:04.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:04.530 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:04.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:04.664 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:04.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:04.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:04.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:04.816 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:04.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:04.817 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:04.918 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:04.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:44:04.955 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:44:04.955 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.14805.0:194110 7.4 e145/132 7:2be3404a:::benchmark_data_smithi043_70646_object76702:head v 145'11949, mlcod=145'11944)", 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:04.951741+0000", 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.001489665, 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015156970000000001, 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:04.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:04.951741+0000", 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:04.951741+0000", 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:04.951741+0000", 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T11:44:04.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:04.951746+0000", 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1459999999999998e-06 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:04.951755+0000", 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6690000000000001e-06 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:04.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:04.951761+0000", 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5260000000000001e-06 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:04.951819+0000", 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8173e-05 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:04.951840+0000", 2024-09-21T11:44:04.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1183999999999999e-05 2024-09-21T11:44:04.960 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:04.960 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:04.960 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:04.960 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:04.960 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:44:04.960 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T11:44:04.960 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:44:04.970 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:04.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:04.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:05.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:05.097 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:44:05.097 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:44:05.097 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:44:05.097 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:44:05.097 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:44:05.098 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:44:05.098 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:44:05.098 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:44:05.118 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:44:05.118 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:44:05.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:05.123 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:05.123 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:05.163 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:44:05.163 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:44:05.204 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:44:05.204 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:44:05.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:05.244 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:44:05.245 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:44:05.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:05 smithi043 ceph-mon[29710]: pgmap v182: 148 pgs: 148 active+clean; 2.8 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:44:05.270 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:05 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/657338655' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:05.270 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:44:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:44:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:44:05.270 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:05 smithi043 ceph-mon[36454]: pgmap v182: 148 pgs: 148 active+clean; 2.8 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:44:05.270 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:05 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/657338655' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:05.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:05.290 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:05.290 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:05.391 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:05.431 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:05.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:05.432 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:05.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:05 smithi175 ceph-mon[32613]: pgmap v182: 148 pgs: 148 active+clean; 2.8 GiB data, 9.4 GiB used, 616 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:44:05.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:05 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/657338655' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:05.533 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:05.574 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:05.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:05.575 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:05.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:05.714 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:05.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:05.716 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:05.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:05.849 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:05.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:05.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:05.951 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:05.982 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:05.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:05.983 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:06.042 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:06.083 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:06.113 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:06.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:06.115 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:06.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:44:06.212 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 1 pgp_num to 1 2024-09-21T11:44:06.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:06.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:06.252 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:06.252 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:06.353 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:06.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:06.391 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:06.391 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:06.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:06 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1765775765' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:44:06.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:06 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1765775765' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:44:06.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:06.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:06 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1765775765' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:44:06.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:06.545 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:06.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:06.599 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:06.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:06.681 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:06.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:06.682 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:06.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:06.812 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:06.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:06.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:06.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:06.947 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:06.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:06.947 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:07.048 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:07.075 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:07.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:07.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:07.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:07.216 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:44:07.216 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:44:07.217 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:07.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:07.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:07.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:07.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:07.358 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:07.358 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:07.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:07.489 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:07.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:07.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:07 smithi043 ceph-mon[36454]: pgmap v183: 148 pgs: 148 active+clean; 2.6 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:44:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1765775765' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:44:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:07 smithi043 ceph-mon[36454]: osdmap e146: 8 total, 8 up, 7 in 2024-09-21T11:44:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:07 smithi043 ceph-mon[29710]: pgmap v183: 148 pgs: 148 active+clean; 2.6 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:44:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1765775765' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:44:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:07 smithi043 ceph-mon[29710]: osdmap e146: 8 total, 8 up, 7 in 2024-09-21T11:44:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:07 smithi175 ceph-mon[32613]: pgmap v183: 148 pgs: 148 active+clean; 2.6 GiB data, 8.9 GiB used, 617 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-21T11:44:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1765775765' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:44:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:07 smithi175 ceph-mon[32613]: osdmap e146: 8 total, 8 up, 7 in 2024-09-21T11:44:07.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:07.630 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:07.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:07.631 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:07.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:07.771 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:07.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:07.772 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:07.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:07.903 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:07.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:07.904 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:07.961 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:44:07.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:44:08.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:08.077 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:08.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:08.078 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:08.178 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:08.205 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:08.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:08.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:08.216 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:44:08.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:08 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1323636720' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:44:08.304 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:08 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1323636720' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:44:08.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:08.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:08.350 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:08.350 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:08.372 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:08.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:08.490 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:08.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:08.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:08.503 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:08 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1323636720' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:44:08.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:08.629 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:08.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:08.630 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:08.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:08.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:08.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:08.765 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:08.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:08.908 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:08.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:08.909 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:09.010 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:09.013 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:44:09.013 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:44:09.041 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:09.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:09.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:09.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:09.178 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:09.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:09.186 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:09.186 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[36454]: pgmap v185: 148 pgs: 148 active+clean; 2.4 GiB data, 8.3 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-21T11:44:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1323636720' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:44:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[36454]: osdmap e147: 8 total, 8 up, 7 in 2024-09-21T11:44:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T11:44:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T11:44:09.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[29710]: pgmap v185: 148 pgs: 148 active+clean; 2.4 GiB data, 8.3 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-21T11:44:09.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:09.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1323636720' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:44:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[29710]: osdmap e147: 8 total, 8 up, 7 in 2024-09-21T11:44:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T11:44:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:09 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T11:44:09.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:09.331 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:09.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:09.332 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:09.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:09.472 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:09.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:09.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:09 smithi175 ceph-mon[32613]: pgmap v185: 148 pgs: 148 active+clean; 2.4 GiB data, 8.3 GiB used, 618 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.57k op/s 2024-09-21T11:44:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:09 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1323636720' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:44:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:09 smithi175 ceph-mon[32613]: osdmap e147: 8 total, 8 up, 7 in 2024-09-21T11:44:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:09 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T11:44:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:09 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T11:44:09.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:09.612 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:09.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:09.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:09.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:09.736 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:09.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:09.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:09.838 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:09.847 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:44:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:44:09.865 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:09.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:09.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:09.881 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:44:09.882 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:44:09.917 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:44:09.917 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:44:09.962 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:44:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:44:09.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:09.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:09.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:09.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:09.997 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:44:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:44:10.031 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:44:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:44:10.067 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:44:10.067 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:44:10.095 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:10.100 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:44:10.100 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:44:10.130 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:10.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:10.131 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:10.232 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:10.258 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:10.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:10.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:10.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:10.401 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:10.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:10.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:10.451 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:44:10.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:10 smithi175 ceph-mon[32613]: 3.13 scrub starts 2024-09-21T11:44:10.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:10 smithi175 ceph-mon[32613]: 3.13 scrub ok 2024-09-21T11:44:10.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:10 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T11:44:10.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:10 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T11:44:10.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:10 smithi175 ceph-mon[32613]: 5.1e scrub starts 2024-09-21T11:44:10.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:10 smithi175 ceph-mon[32613]: 5.1e scrub ok 2024-09-21T11:44:10.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:10 smithi175 ceph-mon[32613]: 7.2 scrub starts 2024-09-21T11:44:10.486 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1610382121' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[36454]: 3.13 scrub starts 2024-09-21T11:44:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[36454]: 3.13 scrub ok 2024-09-21T11:44:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T11:44:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T11:44:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[36454]: 5.1e scrub starts 2024-09-21T11:44:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[36454]: 5.1e scrub ok 2024-09-21T11:44:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[36454]: 7.2 scrub starts 2024-09-21T11:44:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1610382121' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[29710]: 3.13 scrub starts 2024-09-21T11:44:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[29710]: 3.13 scrub ok 2024-09-21T11:44:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T11:44:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T11:44:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[29710]: 5.1e scrub starts 2024-09-21T11:44:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[29710]: 5.1e scrub ok 2024-09-21T11:44:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[29710]: 7.2 scrub starts 2024-09-21T11:44:10.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1610382121' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:10.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:10.545 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:10.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:10.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:10.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:10.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:10.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:10.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:10.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:10.827 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:10.827 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:10.864 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:10.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:10.970 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:10.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:10.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:11.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:44:11.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:11.112 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:11.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:11.113 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:11.214 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:11.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:44:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:44:11.224 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:11 smithi175 ceph-mon[32613]: 3.11 scrub starts 2024-09-21T11:44:11.224 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:11 smithi175 ceph-mon[32613]: 3.11 scrub ok 2024-09-21T11:44:11.224 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:11 smithi175 ceph-mon[32613]: pgmap v187: 148 pgs: 148 active+clean; 2.4 GiB data, 8.3 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 790 op/s 2024-09-21T11:44:11.225 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:11 smithi175 ceph-mon[32613]: 6.4 scrub starts 2024-09-21T11:44:11.225 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:11 smithi175 ceph-mon[32613]: 6.4 scrub ok 2024-09-21T11:44:11.225 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:11 smithi175 ceph-mon[32613]: 7.2 scrub ok 2024-09-21T11:44:11.245 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:11.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:11.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:11.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:11.392 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:11.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:11.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[36454]: 3.11 scrub starts 2024-09-21T11:44:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[36454]: 3.11 scrub ok 2024-09-21T11:44:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[36454]: pgmap v187: 148 pgs: 148 active+clean; 2.4 GiB data, 8.3 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 790 op/s 2024-09-21T11:44:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[36454]: 6.4 scrub starts 2024-09-21T11:44:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[36454]: 6.4 scrub ok 2024-09-21T11:44:11.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[36454]: 7.2 scrub ok 2024-09-21T11:44:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[29710]: 3.11 scrub starts 2024-09-21T11:44:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[29710]: 3.11 scrub ok 2024-09-21T11:44:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[29710]: pgmap v187: 148 pgs: 148 active+clean; 2.4 GiB data, 8.3 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 790 op/s 2024-09-21T11:44:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[29710]: 6.4 scrub starts 2024-09-21T11:44:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[29710]: 6.4 scrub ok 2024-09-21T11:44:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:11 smithi043 ceph-mon[29710]: 7.2 scrub ok 2024-09-21T11:44:11.493 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:11.494 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:11.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:11.524 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:11.524 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:11.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:11.666 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:11.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:11.667 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:11.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:11.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:11.817 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:11.918 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:11.958 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:11.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:11.959 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:12.006 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:44:12.006 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:44:12.006 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:44:12.006 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:44:12.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:12.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:12.100 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:12.100 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:12.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:12.230 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:12.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:12.232 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:12.309 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:44:12.309 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:44:12.309 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T11:44:12.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-21T11:44:12.332 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:12.360 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:12.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:12.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:12.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:44:12.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:44:12.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:12.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[29710]: 6.1d scrub starts 2024-09-21T11:44:12.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[29710]: 6.1d scrub ok 2024-09-21T11:44:12.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[29710]: 6.2 scrub starts 2024-09-21T11:44:12.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[29710]: 6.2 scrub ok 2024-09-21T11:44:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:44:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:44:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:12.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[36454]: 6.1d scrub starts 2024-09-21T11:44:12.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[36454]: 6.1d scrub ok 2024-09-21T11:44:12.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[36454]: 6.2 scrub starts 2024-09-21T11:44:12.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:12 smithi043 ceph-mon[36454]: 6.2 scrub ok 2024-09-21T11:44:12.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:12.501 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:12.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:12.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:12 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:44:12.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:12 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:44:12.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:12 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:12.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:12 smithi175 ceph-mon[32613]: 6.1d scrub starts 2024-09-21T11:44:12.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:12 smithi175 ceph-mon[32613]: 6.1d scrub ok 2024-09-21T11:44:12.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:12 smithi175 ceph-mon[32613]: 6.2 scrub starts 2024-09-21T11:44:12.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:12 smithi175 ceph-mon[32613]: 6.2 scrub ok 2024-09-21T11:44:12.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:12.627 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:12.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:12.628 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:12.729 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:12.756 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:12.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:12.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:12.784 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:12.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:44:12.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:12.901 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:12.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:12.902 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:12.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:44:13.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:13.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:13.057 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:13.057 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:13.158 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:13.183 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:13.192 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:13.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:13.193 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:13.253 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:44:13.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:13.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:13.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:13.439 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:13.477 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:13.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:13.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[29710]: pgmap v188: 148 pgs: 148 active+clean; 2.0 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-21T11:44:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[29710]: 5.1b scrub starts 2024-09-21T11:44:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[29710]: 5.1b scrub ok 2024-09-21T11:44:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[29710]: 6.10 scrub starts 2024-09-21T11:44:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[29710]: 6.10 scrub ok 2024-09-21T11:44:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3714059409' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[29710]: 7.a scrub starts 2024-09-21T11:44:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[36454]: pgmap v188: 148 pgs: 148 active+clean; 2.0 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-21T11:44:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[36454]: 5.1b scrub starts 2024-09-21T11:44:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[36454]: 5.1b scrub ok 2024-09-21T11:44:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[36454]: 6.10 scrub starts 2024-09-21T11:44:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[36454]: 6.10 scrub ok 2024-09-21T11:44:13.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3714059409' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:13.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:13 smithi043 ceph-mon[36454]: 7.a scrub starts 2024-09-21T11:44:13.506 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:13 smithi175 ceph-mon[32613]: pgmap v188: 148 pgs: 148 active+clean; 2.0 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.59k op/s 2024-09-21T11:44:13.507 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:13 smithi175 ceph-mon[32613]: 5.1b scrub starts 2024-09-21T11:44:13.507 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:13 smithi175 ceph-mon[32613]: 5.1b scrub ok 2024-09-21T11:44:13.507 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:13 smithi175 ceph-mon[32613]: 6.10 scrub starts 2024-09-21T11:44:13.507 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:13 smithi175 ceph-mon[32613]: 6.10 scrub ok 2024-09-21T11:44:13.507 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3714059409' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:13.507 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:13 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:13.507 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:13 smithi175 ceph-mon[32613]: 7.a scrub starts 2024-09-21T11:44:13.580 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:13.606 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:13.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:13.607 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:13.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:13.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:13.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:13.836 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:44:13.837 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:44:13.837 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 148, 2024-09-21T11:44:13.837 INFO:teuthology.orchestra.run.smithi043.stdout: "fsid": "4f2c5252-780d-11ef-baf6-efdc52797490", 2024-09-21T11:44:13.837 INFO:teuthology.orchestra.run.smithi043.stdout: "created": "2024-09-21T11:34:07.249195+0000", 2024-09-21T11:44:13.837 INFO:teuthology.orchestra.run.smithi043.stdout: "modified": "2024-09-21T11:44:13.235308+0000", 2024-09-21T11:44:13.837 INFO:teuthology.orchestra.run.smithi043.stdout: "last_up_change": "2024-09-21T11:39:49.206441+0000", 2024-09-21T11:44:13.837 INFO:teuthology.orchestra.run.smithi043.stdout: "last_in_change": "2024-09-21T11:42:36.335148+0000", 2024-09-21T11:44:13.837 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T11:44:13.837 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_num": 5799936, 2024-09-21T11:44:13.838 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_set": [ 2024-09-21T11:44:13.838 INFO:teuthology.orchestra.run.smithi043.stdout: "pglog_hardlimit", 2024-09-21T11:44:13.838 INFO:teuthology.orchestra.run.smithi043.stdout: "purged_snapdirs", 2024-09-21T11:44:13.838 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_deletes", 2024-09-21T11:44:13.838 INFO:teuthology.orchestra.run.smithi043.stdout: "sortbitwise" 2024-09-21T11:44:13.838 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:44:13.838 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_version": 18, 2024-09-21T11:44:13.838 INFO:teuthology.orchestra.run.smithi043.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T11:44:13.838 INFO:teuthology.orchestra.run.smithi043.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T11:44:13.838 INFO:teuthology.orchestra.run.smithi043.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_snapshot": "", 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_max": 7, 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd": 8, 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "require_min_compat_client": "luminous", 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "min_compat_client": "luminous", 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "require_osd_release": "squid", 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_crimson": false, 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "pools": [ 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 1, 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".mgr", 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:37:26.023430+0000", 2024-09-21T11:44:13.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 1, 2024-09-21T11:44:13.840 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 1, 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 1, 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 1, 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 1, 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.841 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "146", 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:44:13.842 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:44:13.843 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:44:13.844 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_max": 32, 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_min": 1 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": {} 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 7.8899998664855957, 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 7.8899998664855957, 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.37999999523162842, 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 3, 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 3, 2024-09-21T11:44:13.845 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 2, 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "datapool", 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:58.262982+0000", 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 8193, 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:44:13.846 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 3, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 3, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 3, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 3, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 3, 2024-09-21T11:44:13.847 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "61", 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:44:13.848 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 2, 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 61, 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:44:13.849 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:44:13.850 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd": {} 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:44:13.851 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.6700000762939453, 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.6700000762939453, 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.75, 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2, 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2, 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 3, 2024-09-21T11:44:13.852 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".rgw.root", 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:59.229530+0000", 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:44:13.853 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:44:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:44:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:44:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:44:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:44:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:44:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:44:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:44:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:44:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:44:13.854 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:44:13.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:44:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "60", 2024-09-21T11:44:13.855 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:44:13.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.857 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:44:13.858 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.7400000095367432, 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.7400000095367432, 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:44:13.859 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 4, 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.log", 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:00.380059+0000", 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:44:13.860 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:44:13.861 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "62", 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:44:13.862 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:44:13.863 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:44:13.864 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.2400000095367432, 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.2400000095367432, 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.9700000286102295, 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.9700000286102295, 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:44:13.865 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 5, 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.control", 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:02.330041+0000", 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:44:13.866 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:44:13.867 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "64", 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:44:13.868 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:44:13.869 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:44:13.870 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:44:13.871 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.7400000095367432, 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.7400000095367432, 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 6, 2024-09-21T11:44:13.872 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.meta", 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:04.395413+0000", 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:44:13.873 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:44:13.874 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "66", 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:44:13.875 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:44:13.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:44:13.877 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_bias": 4 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.7400000095367432, 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.7400000095367432, 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T11:44:13.878 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 7, 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "unique_pool_0", 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:41:08.476770+0000", 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:44:13.879 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 16, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 16, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 16, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 16, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 16, 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:44:13.880 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "85", 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:44:13.881 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:44:13.882 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:44:13.883 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "rados": {} 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.9900000095367432, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.9900000095367432, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.75, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.75, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:44:13.884 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4", 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:44:13.885 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 8, 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 138, 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.886 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6802", 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6804", 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.887 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6803", 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6801/817279193", 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6802/817279193", 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6804/817279193", 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6803/817279193", 2024-09-21T11:44:13.888 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:44:13.889 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607", 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 0, 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 0, 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:44:13.890 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 13, 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 82, 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6805", 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:44:13.891 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6806", 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6808", 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.892 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6807", 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.893 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6805/263756171", 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6806/263756171", 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6808/263756171", 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6807/263756171", 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce", 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:44:13.894 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 18, 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 138, 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6809", 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.895 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6810", 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.896 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6812", 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6811", 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6809/687577863", 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6810/687577863", 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6812/687577863", 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6811/687577863", 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d", 2024-09-21T11:44:13.898 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 27, 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 133, 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.899 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6813", 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6814", 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6816", 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6815", 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6813/129493158", 2024-09-21T11:44:13.901 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6814/129493158", 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6816/129493158", 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6815/129493158", 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c", 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:44:13.902 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 33, 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 112, 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6800", 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6801", 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6803", 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6802", 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.905 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6800/1323992640", 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6801/1323992640", 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6803/1323992640", 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6802/1323992640", 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "4d647321-71a1-412b-bbf5-921441923a65", 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:44:13.906 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 39, 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 135, 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6804", 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:44:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6805", 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6807", 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:44:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6806", 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6804/1163208458", 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6805/1163208458", 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6807/1163208458", 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6806/1163208458", 2024-09-21T11:44:13.909 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354", 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:44:13.910 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 47, 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 134, 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6808", 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6809", 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6811", 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:44:13.912 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6810", 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6808/932395625", 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6809/932395625", 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6811/932395625", 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6810/932395625", 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:44:13.913 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22", 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:44:13.914 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 54, 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 112, 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6812", 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:44:13.915 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6813", 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6815", 2024-09-21T11:44:13.916 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6814", 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.917 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6812/3676713275", 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6813/3676713275", 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6815/3676713275", 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6814/3676713275", 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.918 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_xinfo": [ 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:43:38.225039+0000", 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:44:13.919 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 65536, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:43:15.836506+0000", 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:42:49.908589+0000", 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:43:11.538170+0000", 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:44:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:43:28.054157+0000", 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:44:13.922 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:43:47.789292+0000", 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:43:40.958165+0000", 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.923 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:43:15.807454+0000", 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap": [], 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_items": [ 2024-09-21T11:44:13.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.4", 2024-09-21T11:44:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:44:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:44:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:44:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.925 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.16", 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_primaries": [], 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_temp": [], 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_temp": [], 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklist": { 2024-09-21T11:44:13.926 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2419845767": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3249046048": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2518306973": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3115176321": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/2583179759": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/4225358170": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/347827280": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/1513342881": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/1113684867": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3345448895": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2454875216": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/911630798": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/983028036": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/4145025121": "2024-09-22T11:40:29.670859+0000" 2024-09-21T11:44:13.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "range_blocklist": {}, 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profiles": { 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "default": { 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-failure-domain": "osd", 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "k": "2", 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "m": "1", 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "plugin": "jerasure", 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "technique": "reed_sol_van" 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps_queue": [], 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "new_removed_snaps": [], 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "new_purged_snaps": [], 2024-09-21T11:44:13.928 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_node_flags": {}, 2024-09-21T11:44:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "device_class_flags": {}, 2024-09-21T11:44:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode": { 2024-09-21T11:44:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_enabled": false, 2024-09-21T11:44:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_bucket_count": 0, 2024-09-21T11:44:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "degraded_stretch_mode": 0, 2024-09-21T11:44:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "recovering_stretch_mode": 0, 2024-09-21T11:44:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_bucket": 0 2024-09-21T11:44:13.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:13.929 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:44:13.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:13.931 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:13.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:14.002 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:44:14.003 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:44:14.003 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:44:14.003 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:44:14.003 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:44:14.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:14.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:14.064 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:14.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:14.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:14.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: 6.12 scrub starts 2024-09-21T11:44:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: 6.12 scrub ok 2024-09-21T11:44:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: 7.a scrub ok 2024-09-21T11:44:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:44:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:44:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: osdmap e148: 8 total, 8 up, 7 in 2024-09-21T11:44:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: 6.a scrub starts 2024-09-21T11:44:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: 6.a scrub ok 2024-09-21T11:44:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: 6.16 scrub starts 2024-09-21T11:44:14.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: 6.16 scrub ok 2024-09-21T11:44:14.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: 7.3 scrub starts 2024-09-21T11:44:14.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3651666170' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:44:14.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: 5.1 scrub starts 2024-09-21T11:44:14.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:14 smithi175 ceph-mon[32613]: 5.1 scrub ok 2024-09-21T11:44:14.302 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:14.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:14.343 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:14.343 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:14.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:14.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:14.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:14.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:14.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: 6.12 scrub starts 2024-09-21T11:44:14.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: 6.12 scrub ok 2024-09-21T11:44:14.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: 7.a scrub ok 2024-09-21T11:44:14.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:44:14.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:44:14.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:14.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: osdmap e148: 8 total, 8 up, 7 in 2024-09-21T11:44:14.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: 6.a scrub starts 2024-09-21T11:44:14.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: 6.a scrub ok 2024-09-21T11:44:14.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: 6.16 scrub starts 2024-09-21T11:44:14.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: 6.16 scrub ok 2024-09-21T11:44:14.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: 7.3 scrub starts 2024-09-21T11:44:14.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3651666170' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:44:14.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: 5.1 scrub starts 2024-09-21T11:44:14.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[36454]: 5.1 scrub ok 2024-09-21T11:44:14.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: 6.12 scrub starts 2024-09-21T11:44:14.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: 6.12 scrub ok 2024-09-21T11:44:14.521 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: 7.a scrub ok 2024-09-21T11:44:14.521 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:44:14.521 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:44:14.521 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:14.521 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: osdmap e148: 8 total, 8 up, 7 in 2024-09-21T11:44:14.521 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: 6.a scrub starts 2024-09-21T11:44:14.521 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: 6.a scrub ok 2024-09-21T11:44:14.521 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: 6.16 scrub starts 2024-09-21T11:44:14.521 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: 6.16 scrub ok 2024-09-21T11:44:14.521 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: 7.3 scrub starts 2024-09-21T11:44:14.522 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3651666170' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:44:14.522 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: 5.1 scrub starts 2024-09-21T11:44:14.522 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:14 smithi043 ceph-mon[29710]: 5.1 scrub ok 2024-09-21T11:44:14.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:14.602 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:14.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:14.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:14.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:14.745 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:14.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:14.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:14.778 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:44:14.847 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:14.884 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:14.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:14.885 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:14.986 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:15.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:15.018 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:15.018 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:15.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:15.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:15.174 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:15.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:15.183 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:15.267 INFO:tasks.thrashosds.thrasher:j is {'epoch': 148, 'fsid': '4f2c5252-780d-11ef-baf6-efdc52797490', 'created': '2024-09-21T11:34:07.249195+0000', 'modified': '2024-09-21T11:44:13.235308+0000', 'last_up_change': '2024-09-21T11:39:49.206441+0000', 'last_in_change': '2024-09-21T11:42:36.335148+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T11:37:26.023430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-21T11:39:58.262982+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6700000762939453, 'score_stable': 2.6700000762939453, 'optimal_score': 0.75, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-21T11:39:59.229530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-21T11:40:00.380059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.9700000286102295, 'raw_score_stable': 1.9700000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-21T11:40:02.330041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-21T11:40:04.395413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-21T11:41:08.476770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '5229029d-4fd6-46b6-8325-a3e004df09b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6801', 'nonce': 817279193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 817279193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 817279193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6803', 'nonce': 817279193}]}, 'public_addr': '172.21.15.43:6801/817279193', 'cluster_addr': '172.21.15.43:6802/817279193', 'heartbeat_back_addr': '172.21.15.43:6804/817279193', 'heartbeat_front_addr': '172.21.15.43:6803/817279193', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a2a7b58b-ba99-4383-a6d3-3b19388b2607', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6805', 'nonce': 263756171}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 263756171}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 263756171}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6807', 'nonce': 263756171}]}, 'public_addr': '172.21.15.43:6805/263756171', 'cluster_addr': '172.21.15.43:6806/263756171', 'heartbeat_back_addr': '172.21.15.43:6808/263756171', 'heartbeat_front_addr': '172.21.15.43:6807/263756171', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c24a60b5-5713-492b-82ee-576ea7a743ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 138, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6809', 'nonce': 687577863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 687577863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 687577863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6811', 'nonce': 687577863}]}, 'public_addr': '172.21.15.43:6809/687577863', 'cluster_addr': '172.21.15.43:6810/687577863', 'heartbeat_back_addr': '172.21.15.43:6812/687577863', 'heartbeat_front_addr': '172.21.15.43:6811/687577863', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c5d43cf8-1c72-4d7d-ae41-e8136b47177d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 133, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6813', 'nonce': 129493158}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 129493158}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 129493158}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6815', 'nonce': 129493158}]}, 'public_addr': '172.21.15.43:6813/129493158', 'cluster_addr': '172.21.15.43:6814/129493158', 'heartbeat_back_addr': '172.21.15.43:6816/129493158', 'heartbeat_front_addr': '172.21.15.43:6815/129493158', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '79ee85c5-afdc-4d68-a253-89b7b0eb652c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 33, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 1323992640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6801', 'nonce': 1323992640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6803', 'nonce': 1323992640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 1323992640}]}, 'public_addr': '172.21.15.175:6800/1323992640', 'cluster_addr': '172.21.15.175:6801/1323992640', 'heartbeat_back_addr': '172.21.15.175:6803/1323992640', 'heartbeat_front_addr': '172.21.15.175:6802/1323992640', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4d647321-71a1-412b-bbf5-921441923a65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 135, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 1163208458}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6805', 'nonce': 1163208458}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6807', 'nonce': 1163208458}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 1163208458}]}, 'public_addr': '172.21.15.175:6804/1163208458', 'cluster_addr': '172.21.15.175:6805/1163208458', 'heartbeat_back_addr': '172.21.15.175:6807/1163208458', 'heartbeat_front_addr': '172.21.15.175:6806/1163208458', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3c33bea7-94b5-46c1-9905-81b4612a2354', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 47, 'up_thru': 134, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 932395625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6809', 'nonce': 932395625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6811', 'nonce': 932395625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 932395625}]}, 'public_addr': '172.21.15.175:6808/932395625', 'cluster_addr': '172.21.15.175:6809/932395625', 'heartbeat_back_addr': '172.21.15.175:6811/932395625', 'heartbeat_front_addr': '172.21.15.175:6810/932395625', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '41bc9817-d9d1-4c11-94ff-7de332af6d22', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 54, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 3676713275}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6813', 'nonce': 3676713275}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6815', 'nonce': 3676713275}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 3676713275}]}, 'public_addr': '172.21.15.175:6812/3676713275', 'cluster_addr': '172.21.15.175:6813/3676713275', 'heartbeat_back_addr': '172.21.15.175:6815/3676713275', 'heartbeat_front_addr': '172.21.15.175:6814/3676713275', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:43:38.225039+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-21T11:43:15.836506+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:42:49.908589+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:43:11.538170+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:43:28.054157+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:43:47.789292+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:43:40.958165+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:43:15.807454+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.16', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.43:0/2419845767': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3249046048': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/2518306973': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/3115176321': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:6800/2583179759': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/4225358170': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/347827280': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/1513342881': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/1113684867': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3345448895': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/2454875216': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/911630798': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/983028036': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/4145025121': '2024-09-22T11:40:29.670859+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T11:44:15.268 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:44:15.275 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:15.312 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:15.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:15.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:15.329 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:15.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:15.437 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:15.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:15.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:15.467 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:15 smithi043 ceph-mon[29710]: pgmap v190: 148 pgs: 148 active+clean; 2.0 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 675 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-21T11:44:15.467 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:15 smithi043 ceph-mon[29710]: 7.3 scrub ok 2024-09-21T11:44:15.467 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:44:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:44:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:44:15.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:15 smithi043 ceph-mon[36454]: pgmap v190: 148 pgs: 148 active+clean; 2.0 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 675 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-21T11:44:15.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:15 smithi043 ceph-mon[36454]: 7.3 scrub ok 2024-09-21T11:44:15.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:15 smithi175 ceph-mon[32613]: pgmap v190: 148 pgs: 148 active+clean; 2.0 GiB data, 7.1 GiB used, 619 GiB / 626 GiB avail; 675 B/s rd, 0 B/s wr, 1.06k op/s 2024-09-21T11:44:15.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:15 smithi175 ceph-mon[32613]: 7.3 scrub ok 2024-09-21T11:44:15.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:15.591 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:15.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:15.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:15.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:15.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:15.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:15.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:15.865 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:15.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:15.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:15.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:15.995 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:15.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:15.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:16.007 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:44:16.008 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:44:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:16.044 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:44:16.044 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:44:16.079 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:44:16.079 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:44:16.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:16.114 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:44:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:44:16.150 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:16.150 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:44:16.151 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:44:16.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:16.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:16.185 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:44:16.185 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:44:16.216 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:44:16.216 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:44:16.251 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:44:16.251 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:44:16.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:16.300 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:16.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:16.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:16.336 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:44:16.336 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:44:16.336 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[36454]: osdmap e149: 8 total, 8 up, 7 in 2024-09-21T11:44:16.336 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:16.336 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:16.336 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2046250629' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:16.354 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:44:16.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:44:16.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[29710]: osdmap e149: 8 total, 8 up, 7 in 2024-09-21T11:44:16.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:16.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:16.355 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:16 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2046250629' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:16.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:16.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:16.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:16.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:44:16.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:16 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:44:16.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:16 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:44:16.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:16 smithi175 ceph-mon[32613]: osdmap e149: 8 total, 8 up, 7 in 2024-09-21T11:44:16.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:16 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:16.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:16 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:16.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:16 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2046250629' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:16.539 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:16.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:16.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:16.644 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:44:16.644 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:44:16.678 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:16.704 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:16.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:16.705 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:16.806 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:16.834 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:16.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:16.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:16.936 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:16.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:16.967 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:16.967 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:17.068 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:17.103 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:17.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:17.104 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:17.176 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:44:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:44:17.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:17.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:17.260 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:17.261 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:17.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:17.399 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:17.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:17.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:17.474 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:44:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:17 smithi043 ceph-mon[29710]: pgmap v192: 148 pgs: 148 active+clean; 1.9 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-09-21T11:44:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:17 smithi043 ceph-mon[29710]: 4.1 scrub starts 2024-09-21T11:44:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:17 smithi043 ceph-mon[29710]: 4.1 scrub ok 2024-09-21T11:44:17.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:17 smithi043 ceph-mon[36454]: pgmap v192: 148 pgs: 148 active+clean; 1.9 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-09-21T11:44:17.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:17 smithi043 ceph-mon[36454]: 4.1 scrub starts 2024-09-21T11:44:17.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:17 smithi043 ceph-mon[36454]: 4.1 scrub ok 2024-09-21T11:44:17.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:17.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:17.559 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:17.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:17.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:17.682 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:17.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:17.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:17.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:17 smithi175 ceph-mon[32613]: pgmap v192: 148 pgs: 148 active+clean; 1.9 GiB data, 6.6 GiB used, 619 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-09-21T11:44:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:17 smithi175 ceph-mon[32613]: 4.1 scrub starts 2024-09-21T11:44:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:17 smithi175 ceph-mon[32613]: 4.1 scrub ok 2024-09-21T11:44:17.784 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:17.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:17.819 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:17.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:17.823 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:17.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:17.959 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:17.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:17.960 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:18.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:18.089 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:18.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:18.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:18.191 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:18.220 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:18.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:18.221 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:18.267 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-21T11:44:15.787955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1991383603,"num_objects":30577,"num_object_clones":0,"num_object_copies":91731,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30577,"num_whiteouts":0,"num_read":1086,"num_read_kb":943,"num_write":204983,"num_write_kb":7515594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38553,"num_bytes_recovered":2517897553,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77371,"ondisk_log_size":77371,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":6957540,"kb_used_data":5946136,"kb_used_omap":10,"kb_used_meta":1010869,"kb_avail":649258524,"statfs":{"total":671965249536,"available":664840728576,"internally_reserved":0,"allocated":6088843264,"data_stored":6086212973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11156,"internal_metadata":1035129964},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,4,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-597884928,"num_objects":-9123,"num_object_clones":0,"num_object_copies":-27369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9123,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9123,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.570660"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.229096+0000","last_change":"2024-09-21T11:41:42.291880+0000","last_active":"2024-09-21T11:44:08.229096+0000","last_peered":"2024-09-21T11:44:08.229096+0000","last_clean":"2024-09-21T11:44:08.229096+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:44:08.229096+0000","last_undegraded":"2024-09-21T11:44:08.229096+0000","last_fullsized":"2024-09-21T11:44:08.229096+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:42.290188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:42.290188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:00.579867+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":230,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270119+0000","last_change":"2024-09-21T11:42:37.191052+0000","last_active":"2024-09-21T11:44:15.270119+0000","last_peered":"2024-09-21T11:44:15.270119+0000","last_clean":"2024-09-21T11:44:15.270119+0000","last_became_active":"2024-09-21T11:42:37.164858+0000","last_became_peered":"2024-09-21T11:42:37.164858+0000","last_unstale":"2024-09-21T11:44:15.270119+0000","last_undegraded":"2024-09-21T11:44:15.270119+0000","last_fullsized":"2024-09-21T11:44:15.270119+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:26.994148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:26.994148+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:16.329886+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":94,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.279555+0000","last_change":"2024-09-21T11:43:37.050216+0000","last_active":"2024-09-21T11:44:15.279555+0000","last_peered":"2024-09-21T11:44:15.279555+0000","last_clean":"2024-09-21T11:44:15.279555+0000","last_became_active":"2024-09-21T11:42:37.161713+0000","last_became_peered":"2024-09-21T11:42:37.161713+0000","last_unstale":"2024-09-21T11:44:15.279555+0000","last_undegraded":"2024-09-21T11:44:15.279555+0000","last_fullsized":"2024-09-21T11:44:15.279555+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:52.112502+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270220+0000","last_change":"2024-09-21T11:42:59.201398+0000","last_active":"2024-09-21T11:44:15.270220+0000","last_peered":"2024-09-21T11:44:15.270220+0000","last_clean":"2024-09-21T11:44:15.270220+0000","last_became_active":"2024-09-21T11:42:37.158907+0000","last_became_peered":"2024-09-21T11:42:37.158907+0000","last_unstale":"2024-09-21T11:44:15.270220+0000","last_undegraded":"2024-09-21T11:44:15.270220+0000","last_fullsized":"2024-09-21T11:44:15.270220+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:59.201330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:59.201330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:03.936582+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":185,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.234023+0000","last_change":"2024-09-21T11:43:28.461458+0000","last_active":"2024-09-21T11:44:08.234023+0000","last_peered":"2024-09-21T11:44:08.234023+0000","last_clean":"2024-09-21T11:44:08.234023+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:44:08.234023+0000","last_undegraded":"2024-09-21T11:44:08.234023+0000","last_fullsized":"2024-09-21T11:44:08.234023+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.461378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.461378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.850394+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":190,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.233722+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:44:08.233722+0000","last_peered":"2024-09-21T11:44:08.233722+0000","last_clean":"2024-09-21T11:44:08.233722+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:44:08.233722+0000","last_undegraded":"2024-09-21T11:44:08.233722+0000","last_fullsized":"2024-09-21T11:44:08.233722+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:52.597921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":210,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.234799+0000","last_change":"2024-09-21T11:40:02.315669+0000","last_active":"2024-09-21T11:44:08.234799+0000","last_peered":"2024-09-21T11:44:08.234799+0000","last_clean":"2024-09-21T11:44:08.234799+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:44:08.234799+0000","last_undegraded":"2024-09-21T11:44:08.234799+0000","last_fullsized":"2024-09-21T11:44:08.234799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.613208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":208,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.269991+0000","last_change":"2024-09-21T11:43:04.373004+0000","last_active":"2024-09-21T11:44:15.269991+0000","last_peered":"2024-09-21T11:44:15.269991+0000","last_clean":"2024-09-21T11:44:15.269991+0000","last_became_active":"2024-09-21T11:42:37.154511+0000","last_became_peered":"2024-09-21T11:42:37.154511+0000","last_unstale":"2024-09-21T11:44:15.269991+0000","last_undegraded":"2024-09-21T11:44:15.269991+0000","last_fullsized":"2024-09-21T11:44:15.269991+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:04.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:04.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.920192+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.281766+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:44:15.281766+0000","last_peered":"2024-09-21T11:44:15.281766+0000","last_clean":"2024-09-21T11:44:15.281766+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:44:15.281766+0000","last_undegraded":"2024-09-21T11:44:15.281766+0000","last_fullsized":"2024-09-21T11:44:15.281766+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":221,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270465+0000","last_change":"2024-09-21T11:43:26.258023+0000","last_active":"2024-09-21T11:44:15.270465+0000","last_peered":"2024-09-21T11:44:15.270465+0000","last_clean":"2024-09-21T11:44:15.270465+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:44:15.270465+0000","last_undegraded":"2024-09-21T11:44:15.270465+0000","last_fullsized":"2024-09-21T11:44:15.270465+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:26.257967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.257967+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:51.865812+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.282193+0000","last_change":"2024-09-21T11:44:12.257513+0000","last_active":"2024-09-21T11:44:15.282193+0000","last_peered":"2024-09-21T11:44:15.282193+0000","last_clean":"2024-09-21T11:44:15.282193+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:44:15.282193+0000","last_undegraded":"2024-09-21T11:44:15.282193+0000","last_fullsized":"2024-09-21T11:44:15.282193+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.257454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.257454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:13.769532+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":191,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.220270+0000","last_change":"2024-09-21T11:43:44.648053+0000","last_active":"2024-09-21T11:44:08.220270+0000","last_peered":"2024-09-21T11:44:08.220270+0000","last_clean":"2024-09-21T11:44:08.220270+0000","last_became_active":"2024-09-21T11:42:37.155879+0000","last_became_peered":"2024-09-21T11:42:37.155879+0000","last_unstale":"2024-09-21T11:44:08.220270+0000","last_undegraded":"2024-09-21T11:44:08.220270+0000","last_fullsized":"2024-09-21T11:44:08.220270+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.647985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.647985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:56.842445+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":215,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280117+0000","last_change":"2024-09-21T11:42:37.156936+0000","last_active":"2024-09-21T11:44:15.280117+0000","last_peered":"2024-09-21T11:44:15.280117+0000","last_clean":"2024-09-21T11:44:15.280117+0000","last_became_active":"2024-09-21T11:42:37.156575+0000","last_became_peered":"2024-09-21T11:42:37.156575+0000","last_unstale":"2024-09-21T11:44:15.280117+0000","last_undegraded":"2024-09-21T11:44:15.280117+0000","last_fullsized":"2024-09-21T11:44:15.280117+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.101955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.101955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":228,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270334+0000","last_change":"2024-09-21T11:43:08.326923+0000","last_active":"2024-09-21T11:44:15.270334+0000","last_peered":"2024-09-21T11:44:15.270334+0000","last_clean":"2024-09-21T11:44:15.270334+0000","last_became_active":"2024-09-21T11:41:31.634863+0000","last_became_peered":"2024-09-21T11:41:31.634863+0000","last_unstale":"2024-09-21T11:44:15.270334+0000","last_undegraded":"2024-09-21T11:44:15.270334+0000","last_fullsized":"2024-09-21T11:44:15.270334+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:08.326846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:08.326846+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.278402+0000","last_change":"2024-09-21T11:43:38.567194+0000","last_active":"2024-09-21T11:44:15.278402+0000","last_peered":"2024-09-21T11:44:15.278402+0000","last_clean":"2024-09-21T11:44:15.278402+0000","last_became_active":"2024-09-21T11:42:37.164285+0000","last_became_peered":"2024-09-21T11:42:37.164285+0000","last_unstale":"2024-09-21T11:44:15.278402+0000","last_undegraded":"2024-09-21T11:44:15.278402+0000","last_fullsized":"2024-09-21T11:44:15.278402+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.567131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.567131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.325966+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.279677+0000","last_change":"2024-09-21T11:42:37.163213+0000","last_active":"2024-09-21T11:44:15.279677+0000","last_peered":"2024-09-21T11:44:15.279677+0000","last_clean":"2024-09-21T11:44:15.279677+0000","last_became_active":"2024-09-21T11:42:37.162912+0000","last_became_peered":"2024-09-21T11:42:37.162912+0000","last_unstale":"2024-09-21T11:44:15.279677+0000","last_undegraded":"2024-09-21T11:44:15.279677+0000","last_fullsized":"2024-09-21T11:44:15.279677+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:12.028860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:12.028860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":198,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.271220+0000","last_change":"2024-09-21T11:43:22.300966+0000","last_active":"2024-09-21T11:44:15.271220+0000","last_peered":"2024-09-21T11:44:15.271220+0000","last_clean":"2024-09-21T11:44:15.271220+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:44:15.271220+0000","last_undegraded":"2024-09-21T11:44:15.271220+0000","last_fullsized":"2024-09-21T11:44:15.271220+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:22.300885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:22.300885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.343801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.226763+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:44:08.226763+0000","last_peered":"2024-09-21T11:44:08.226763+0000","last_clean":"2024-09-21T11:44:08.226763+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:44:08.226763+0000","last_undegraded":"2024-09-21T11:44:08.226763+0000","last_fullsized":"2024-09-21T11:44:08.226763+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:50.720157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":272,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.237940+0000","last_change":"2024-09-21T11:43:31.498356+0000","last_active":"2024-09-21T11:44:08.237940+0000","last_peered":"2024-09-21T11:44:08.237940+0000","last_clean":"2024-09-21T11:44:08.237940+0000","last_became_active":"2024-09-21T11:42:37.166901+0000","last_became_peered":"2024-09-21T11:42:37.166901+0000","last_unstale":"2024-09-21T11:44:08.237940+0000","last_undegraded":"2024-09-21T11:44:08.237940+0000","last_fullsized":"2024-09-21T11:44:08.237940+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:31.498142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.498142+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:33.717651+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":94,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270794+0000","last_change":"2024-09-21T11:43:43.147999+0000","last_active":"2024-09-21T11:44:15.270794+0000","last_peered":"2024-09-21T11:44:15.270794+0000","last_clean":"2024-09-21T11:44:15.270794+0000","last_became_active":"2024-09-21T11:42:37.153799+0000","last_became_peered":"2024-09-21T11:42:37.153799+0000","last_unstale":"2024-09-21T11:44:15.270794+0000","last_undegraded":"2024-09-21T11:44:15.270794+0000","last_fullsized":"2024-09-21T11:44:15.270794+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:43.147942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:43.147942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:46.559755+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":185,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.240495+0000","last_change":"2024-09-21T11:41:40.448106+0000","last_active":"2024-09-21T11:44:08.240495+0000","last_peered":"2024-09-21T11:44:08.240495+0000","last_clean":"2024-09-21T11:44:08.240495+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:44:08.240495+0000","last_undegraded":"2024-09-21T11:44:08.240495+0000","last_fullsized":"2024-09-21T11:44:08.240495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.447908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.447908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":208,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270595+0000","last_change":"2024-09-21T11:43:45.093396+0000","last_active":"2024-09-21T11:44:15.270595+0000","last_peered":"2024-09-21T11:44:15.270595+0000","last_clean":"2024-09-21T11:44:15.270595+0000","last_became_active":"2024-09-21T11:42:37.153694+0000","last_became_peered":"2024-09-21T11:42:37.153694+0000","last_unstale":"2024-09-21T11:44:15.270595+0000","last_undegraded":"2024-09-21T11:44:15.270595+0000","last_fullsized":"2024-09-21T11:44:15.270595+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.093325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.093325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:14.273876+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":235,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.233817+0000","last_change":"2024-09-21T11:42:37.189830+0000","last_active":"2024-09-21T11:44:08.233817+0000","last_peered":"2024-09-21T11:44:08.233817+0000","last_clean":"2024-09-21T11:44:08.233817+0000","last_became_active":"2024-09-21T11:42:37.165964+0000","last_became_peered":"2024-09-21T11:42:37.165964+0000","last_unstale":"2024-09-21T11:44:08.233817+0000","last_undegraded":"2024-09-21T11:44:08.233817+0000","last_fullsized":"2024-09-21T11:44:08.233817+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:41:35.427210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:35.427210+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":193,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270662+0000","last_change":"2024-09-21T11:43:02.320628+0000","last_active":"2024-09-21T11:44:15.270662+0000","last_peered":"2024-09-21T11:44:15.270662+0000","last_clean":"2024-09-21T11:44:15.270662+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:44:15.270662+0000","last_undegraded":"2024-09-21T11:44:15.270662+0000","last_fullsized":"2024-09-21T11:44:15.270662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:02.320559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:02.320559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:24.560814+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":197,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.277949+0000","last_change":"2024-09-21T11:42:31.278897+0000","last_active":"2024-09-21T11:44:15.277949+0000","last_peered":"2024-09-21T11:44:15.277949+0000","last_clean":"2024-09-21T11:44:15.277949+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:44:15.277949+0000","last_undegraded":"2024-09-21T11:44:15.277949+0000","last_fullsized":"2024-09-21T11:44:15.277949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:42:31.278824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:31.278824+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:04.489162+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":94,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.234718+0000","last_change":"2024-09-21T11:42:37.185608+0000","last_active":"2024-09-21T11:44:08.234718+0000","last_peered":"2024-09-21T11:44:08.234718+0000","last_clean":"2024-09-21T11:44:08.234718+0000","last_became_active":"2024-09-21T11:42:37.166144+0000","last_became_peered":"2024-09-21T11:42:37.166144+0000","last_unstale":"2024-09-21T11:44:08.234718+0000","last_undegraded":"2024-09-21T11:44:08.234718+0000","last_fullsized":"2024-09-21T11:44:08.234718+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:23.349301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.349301+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":208,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.226872+0000","last_change":"2024-09-21T11:43:29.595122+0000","last_active":"2024-09-21T11:44:08.226872+0000","last_peered":"2024-09-21T11:44:08.226872+0000","last_clean":"2024-09-21T11:44:08.226872+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:44:08.226872+0000","last_undegraded":"2024-09-21T11:44:08.226872+0000","last_fullsized":"2024-09-21T11:44:08.226872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:43:29.595040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.595040+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:41.200221+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":272,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.282248+0000","last_change":"2024-09-21T11:43:46.917623+0000","last_active":"2024-09-21T11:44:15.282248+0000","last_peered":"2024-09-21T11:44:15.282248+0000","last_clean":"2024-09-21T11:44:15.282248+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:44:15.282248+0000","last_undegraded":"2024-09-21T11:44:15.282248+0000","last_fullsized":"2024-09-21T11:44:15.282248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:43:46.917560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:46.917560+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:06.493510+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"149'13122","reported_seq":13325,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.766953+0000","last_change":"2024-09-21T11:43:35.987574+0000","last_active":"2024-09-21T11:44:15.766953+0000","last_peered":"2024-09-21T11:44:15.766953+0000","last_clean":"2024-09-21T11:44:15.766953+0000","last_became_active":"2024-09-21T11:41:10.304754+0000","last_became_peered":"2024-09-21T11:41:10.304754+0000","last_unstale":"2024-09-21T11:44:15.766953+0000","last_undegraded":"2024-09-21T11:44:15.766953+0000","last_fullsized":"2024-09-21T11:44:15.766953+0000","mapping_epoch":82,"log_start":"122'8146","ondisk_log_start":"122'8146","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'9957","last_scrub_stamp":"2024-09-21T11:43:35.987394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:35.987394+0000","objects_scrubbed":4941,"log_size":4976,"log_dups_size":0,"ondisk_log_size":4976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:47.433252+0000","scrub_duration":1245,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113770496,"num_objects":1736,"num_object_clones":0,"num_object_copies":5208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13122,"num_write_kb":475456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":150,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.240636+0000","last_change":"2024-09-21T11:43:44.440472+0000","last_active":"2024-09-21T11:44:08.240636+0000","last_peered":"2024-09-21T11:44:08.240636+0000","last_clean":"2024-09-21T11:44:08.240636+0000","last_became_active":"2024-09-21T11:42:37.169939+0000","last_became_peered":"2024-09-21T11:42:37.169939+0000","last_unstale":"2024-09-21T11:44:08.240636+0000","last_undegraded":"2024-09-21T11:44:08.240636+0000","last_fullsized":"2024-09-21T11:44:08.240636+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:44.440374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.440374+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:10.450444+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.233446+0000","last_change":"2024-09-21T11:42:37.172945+0000","last_active":"2024-09-21T11:44:08.233446+0000","last_peered":"2024-09-21T11:44:08.233446+0000","last_clean":"2024-09-21T11:44:08.233446+0000","last_became_active":"2024-09-21T11:42:37.171365+0000","last_became_peered":"2024-09-21T11:42:37.171365+0000","last_unstale":"2024-09-21T11:44:08.233446+0000","last_undegraded":"2024-09-21T11:44:08.233446+0000","last_fullsized":"2024-09-21T11:44:08.233446+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.454207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.454207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":253,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.271346+0000","last_change":"2024-09-21T11:42:26.008760+0000","last_active":"2024-09-21T11:44:15.271346+0000","last_peered":"2024-09-21T11:44:15.271346+0000","last_clean":"2024-09-21T11:44:15.271346+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:44:15.271346+0000","last_undegraded":"2024-09-21T11:44:15.271346+0000","last_fullsized":"2024-09-21T11:44:15.271346+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:26.008646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:26.008646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:51.093540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":195,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.279945+0000","last_change":"2024-09-21T11:42:37.157278+0000","last_active":"2024-09-21T11:44:15.279945+0000","last_peered":"2024-09-21T11:44:15.279945+0000","last_clean":"2024-09-21T11:44:15.279945+0000","last_became_active":"2024-09-21T11:42:37.156952+0000","last_became_peered":"2024-09-21T11:42:37.156952+0000","last_unstale":"2024-09-21T11:44:15.279945+0000","last_undegraded":"2024-09-21T11:44:15.279945+0000","last_fullsized":"2024-09-21T11:44:15.279945+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.066324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.066324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"147'12557","reported_seq":12697,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:11.131298+0000","last_change":"2024-09-21T11:41:10.297546+0000","last_active":"2024-09-21T11:44:11.131298+0000","last_peered":"2024-09-21T11:44:11.131298+0000","last_clean":"2024-09-21T11:44:11.131298+0000","last_became_active":"2024-09-21T11:41:10.297301+0000","last_became_peered":"2024-09-21T11:41:10.297301+0000","last_unstale":"2024-09-21T11:44:11.131298+0000","last_undegraded":"2024-09-21T11:44:11.131298+0000","last_fullsized":"2024-09-21T11:44:11.131298+0000","mapping_epoch":82,"log_start":"122'8251","ondisk_log_start":"122'8251","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4306,"log_dups_size":0,"ondisk_log_size":4306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128909312,"num_objects":1967,"num_object_clones":0,"num_object_copies":5901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12557,"num_write_kb":464768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":226,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.233272+0000","last_change":"2024-09-21T11:43:40.423834+0000","last_active":"2024-09-21T11:44:08.233272+0000","last_peered":"2024-09-21T11:44:08.233272+0000","last_clean":"2024-09-21T11:44:08.233272+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:44:08.233272+0000","last_undegraded":"2024-09-21T11:44:08.233272+0000","last_fullsized":"2024-09-21T11:44:08.233272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:43:40.423750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.423750+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:01.136315+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.240774+0000","last_change":"2024-09-21T11:41:29.388381+0000","last_active":"2024-09-21T11:44:08.240774+0000","last_peered":"2024-09-21T11:44:08.240774+0000","last_clean":"2024-09-21T11:44:08.240774+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:44:08.240774+0000","last_undegraded":"2024-09-21T11:44:08.240774+0000","last_fullsized":"2024-09-21T11:44:08.240774+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:29.388322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:29.388322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":202,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.271793+0000","last_change":"2024-09-21T11:44:13.946705+0000","last_active":"2024-09-21T11:44:15.271793+0000","last_peered":"2024-09-21T11:44:15.271793+0000","last_clean":"2024-09-21T11:44:15.271793+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:44:15.271793+0000","last_undegraded":"2024-09-21T11:44:15.271793+0000","last_fullsized":"2024-09-21T11:44:15.271793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:14.029753+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":190,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270425+0000","last_change":"2024-09-21T11:44:11.930363+0000","last_active":"2024-09-21T11:44:15.270425+0000","last_peered":"2024-09-21T11:44:15.270425+0000","last_clean":"2024-09-21T11:44:15.270425+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:44:15.270425+0000","last_undegraded":"2024-09-21T11:44:15.270425+0000","last_fullsized":"2024-09-21T11:44:15.270425+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:11.930295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:11.930295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:13.258167+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"147'12404","reported_seq":25186,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:11.169040+0000","last_change":"2024-09-21T11:44:10.416880+0000","last_active":"2024-09-21T11:44:11.169040+0000","last_peered":"2024-09-21T11:44:11.169040+0000","last_clean":"2024-09-21T11:44:11.169040+0000","last_became_active":"2024-09-21T11:43:25.930319+0000","last_became_peered":"2024-09-21T11:43:25.930319+0000","last_unstale":"2024-09-21T11:44:11.169040+0000","last_undegraded":"2024-09-21T11:44:11.169040+0000","last_fullsized":"2024-09-21T11:44:11.169040+0000","mapping_epoch":133,"log_start":"122'8114","ondisk_log_start":"122'8114","created":82,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"147'12343","last_scrub_stamp":"2024-09-21T11:44:10.415692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.415692+0000","objects_scrubbed":2118,"log_size":4290,"log_dups_size":0,"ondisk_log_size":4290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:21.214528+0000","scrub_duration":837,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132907008,"num_objects":2028,"num_object_clones":0,"num_object_copies":6084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12404,"num_write_kb":461824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6296,"num_bytes_recovered":412614656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":219,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.271923+0000","last_change":"2024-09-21T11:41:41.007761+0000","last_active":"2024-09-21T11:44:15.271923+0000","last_peered":"2024-09-21T11:44:15.271923+0000","last_clean":"2024-09-21T11:44:15.271923+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:44:15.271923+0000","last_undegraded":"2024-09-21T11:44:15.271923+0000","last_fullsized":"2024-09-21T11:44:15.271923+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:41:41.007695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:41.007695+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:15.335018+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":208,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.227127+0000","last_change":"2024-09-21T11:42:37.182056+0000","last_active":"2024-09-21T11:44:08.227127+0000","last_peered":"2024-09-21T11:44:08.227127+0000","last_clean":"2024-09-21T11:44:08.227127+0000","last_became_active":"2024-09-21T11:42:37.159390+0000","last_became_peered":"2024-09-21T11:42:37.159390+0000","last_unstale":"2024-09-21T11:44:08.227127+0000","last_undegraded":"2024-09-21T11:44:08.227127+0000","last_fullsized":"2024-09-21T11:44:08.227127+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:41:34.236667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:34.236667+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:03.592456+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"114'9","reported_seq":261,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:10.415239+0000","last_change":"2024-09-21T11:43:29.417021+0000","last_active":"2024-09-21T11:44:10.415239+0000","last_peered":"2024-09-21T11:44:10.415239+0000","last_clean":"2024-09-21T11:44:10.415239+0000","last_became_active":"2024-09-21T11:42:37.169838+0000","last_became_peered":"2024-09-21T11:42:37.169838+0000","last_unstale":"2024-09-21T11:44:10.415239+0000","last_undegraded":"2024-09-21T11:44:10.415239+0000","last_fullsized":"2024-09-21T11:44:10.415239+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:29.416946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.416946+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:30.090032+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":191,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.277649+0000","last_change":"2024-09-21T11:41:39.527210+0000","last_active":"2024-09-21T11:44:15.277649+0000","last_peered":"2024-09-21T11:44:15.277649+0000","last_clean":"2024-09-21T11:44:15.277649+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:44:15.277649+0000","last_undegraded":"2024-09-21T11:44:15.277649+0000","last_fullsized":"2024-09-21T11:44:15.277649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.527151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.527151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.480178+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"149'13021","reported_seq":13241,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.697110+0000","last_change":"2024-09-21T11:43:05.145633+0000","last_active":"2024-09-21T11:44:15.697110+0000","last_peered":"2024-09-21T11:44:15.697110+0000","last_clean":"2024-09-21T11:44:15.697110+0000","last_became_active":"2024-09-21T11:41:10.300541+0000","last_became_peered":"2024-09-21T11:41:10.300541+0000","last_unstale":"2024-09-21T11:44:15.697110+0000","last_undegraded":"2024-09-21T11:44:15.697110+0000","last_fullsized":"2024-09-21T11:44:15.697110+0000","mapping_epoch":82,"log_start":"124'8433","ondisk_log_start":"124'8433","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'8200","last_scrub_stamp":"2024-09-21T11:43:05.145547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:05.145547+0000","objects_scrubbed":6512,"log_size":4588,"log_dups_size":0,"ondisk_log_size":4588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.858838+0000","scrub_duration":1824,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107544576,"num_objects":1641,"num_object_clones":0,"num_object_copies":4923,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13021,"num_write_kb":469184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":132,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280201+0000","last_change":"2024-09-21T11:42:37.207015+0000","last_active":"2024-09-21T11:44:15.280201+0000","last_peered":"2024-09-21T11:44:15.280201+0000","last_clean":"2024-09-21T11:44:15.280201+0000","last_became_active":"2024-09-21T11:42:37.165120+0000","last_became_peered":"2024-09-21T11:42:37.165120+0000","last_unstale":"2024-09-21T11:44:15.280201+0000","last_undegraded":"2024-09-21T11:44:15.280201+0000","last_fullsized":"2024-09-21T11:44:15.280201+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:15.360632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.360632+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":209,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.281853+0000","last_change":"2024-09-21T11:44:09.209354+0000","last_active":"2024-09-21T11:44:15.281853+0000","last_peered":"2024-09-21T11:44:15.281853+0000","last_clean":"2024-09-21T11:44:15.281853+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:44:15.281853+0000","last_undegraded":"2024-09-21T11:44:15.281853+0000","last_fullsized":"2024-09-21T11:44:15.281853+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.209310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.209310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:13.005605+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":250,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.226382+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:44:08.226382+0000","last_peered":"2024-09-21T11:44:08.226382+0000","last_clean":"2024-09-21T11:44:08.226382+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:44:08.226382+0000","last_undegraded":"2024-09-21T11:44:08.226382+0000","last_fullsized":"2024-09-21T11:44:08.226382+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":185,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.220437+0000","last_change":"2024-09-21T11:41:40.278404+0000","last_active":"2024-09-21T11:44:08.220437+0000","last_peered":"2024-09-21T11:44:08.220437+0000","last_clean":"2024-09-21T11:44:08.220437+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:44:08.220437+0000","last_undegraded":"2024-09-21T11:44:08.220437+0000","last_fullsized":"2024-09-21T11:44:08.220437+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.278315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.278315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"149'13010","reported_seq":25863,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.769791+0000","last_change":"2024-09-21T11:43:27.957009+0000","last_active":"2024-09-21T11:44:15.769791+0000","last_peered":"2024-09-21T11:44:15.769791+0000","last_clean":"2024-09-21T11:44:15.769791+0000","last_became_active":"2024-09-21T11:43:27.938453+0000","last_became_peered":"2024-09-21T11:43:27.938453+0000","last_unstale":"2024-09-21T11:44:15.769791+0000","last_undegraded":"2024-09-21T11:44:15.769791+0000","last_fullsized":"2024-09-21T11:44:15.769791+0000","mapping_epoch":135,"log_start":"122'7764","ondisk_log_start":"122'7764","created":82,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":5246,"log_dups_size":0,"ondisk_log_size":5246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:56.144677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111673344,"num_objects":1704,"num_object_clones":0,"num_object_copies":5112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13010,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6345,"num_bytes_recovered":415825920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":228,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.226535+0000","last_change":"2024-09-21T11:41:07.065583+0000","last_active":"2024-09-21T11:44:08.226535+0000","last_peered":"2024-09-21T11:44:08.226535+0000","last_clean":"2024-09-21T11:44:08.226535+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:44:08.226535+0000","last_undegraded":"2024-09-21T11:44:08.226535+0000","last_fullsized":"2024-09-21T11:44:08.226535+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:41:07.065507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.065507+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:56.628989+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":89,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.238043+0000","last_change":"2024-09-21T11:43:30.509571+0000","last_active":"2024-09-21T11:44:08.238043+0000","last_peered":"2024-09-21T11:44:08.238043+0000","last_clean":"2024-09-21T11:44:08.238043+0000","last_became_active":"2024-09-21T11:42:37.157562+0000","last_became_peered":"2024-09-21T11:42:37.157562+0000","last_unstale":"2024-09-21T11:44:08.238043+0000","last_undegraded":"2024-09-21T11:44:08.238043+0000","last_fullsized":"2024-09-21T11:44:08.238043+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.509505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.509505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:42.953900+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":193,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.279824+0000","last_change":"2024-09-21T11:43:26.704117+0000","last_active":"2024-09-21T11:44:15.279824+0000","last_peered":"2024-09-21T11:44:15.279824+0000","last_clean":"2024-09-21T11:44:15.279824+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:44:15.279824+0000","last_undegraded":"2024-09-21T11:44:15.279824+0000","last_fullsized":"2024-09-21T11:44:15.279824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.704073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.704073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:55.097023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":93,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.281502+0000","last_change":"2024-09-21T11:43:37.819912+0000","last_active":"2024-09-21T11:44:15.281502+0000","last_peered":"2024-09-21T11:44:15.281502+0000","last_clean":"2024-09-21T11:44:15.281502+0000","last_became_active":"2024-09-21T11:42:37.155437+0000","last_became_peered":"2024-09-21T11:42:37.155437+0000","last_unstale":"2024-09-21T11:44:15.281502+0000","last_undegraded":"2024-09-21T11:44:15.281502+0000","last_fullsized":"2024-09-21T11:44:15.281502+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.544593+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"149'13248","reported_seq":13462,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.758097+0000","last_change":"2024-09-21T11:42:22.352124+0000","last_active":"2024-09-21T11:44:15.758097+0000","last_peered":"2024-09-21T11:44:15.758097+0000","last_clean":"2024-09-21T11:44:15.758097+0000","last_became_active":"2024-09-21T11:41:10.304723+0000","last_became_peered":"2024-09-21T11:41:10.304723+0000","last_unstale":"2024-09-21T11:44:15.758097+0000","last_undegraded":"2024-09-21T11:44:15.758097+0000","last_fullsized":"2024-09-21T11:44:15.758097+0000","mapping_epoch":82,"log_start":"122'8263","ondisk_log_start":"122'8263","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"106'6088","last_scrub_stamp":"2024-09-21T11:42:22.351940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.351940+0000","objects_scrubbed":6032,"log_size":4985,"log_dups_size":0,"ondisk_log_size":4985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:48.894013+0000","scrub_duration":1670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112197632,"num_objects":1712,"num_object_clones":0,"num_object_copies":5136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13248,"num_write_kb":478720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"140'30","reported_seq":230,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.237864+0000","last_change":"2024-09-21T11:43:37.416987+0000","last_active":"2024-09-21T11:44:08.237864+0000","last_peered":"2024-09-21T11:44:08.237864+0000","last_clean":"2024-09-21T11:44:08.237864+0000","last_became_active":"2024-09-21T11:43:32.003036+0000","last_became_peered":"2024-09-21T11:43:32.003036+0000","last_unstale":"2024-09-21T11:44:08.237864+0000","last_undegraded":"2024-09-21T11:44:08.237864+0000","last_fullsized":"2024-09-21T11:44:08.237864+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.678954+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":208,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270766+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:44:15.270766+0000","last_peered":"2024-09-21T11:44:15.270766+0000","last_clean":"2024-09-21T11:44:15.270766+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:44:15.270766+0000","last_undegraded":"2024-09-21T11:44:15.270766+0000","last_fullsized":"2024-09-21T11:44:15.270766+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":222,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280355+0000","last_change":"2024-09-21T11:42:37.193243+0000","last_active":"2024-09-21T11:44:15.280355+0000","last_peered":"2024-09-21T11:44:15.280355+0000","last_clean":"2024-09-21T11:44:15.280355+0000","last_became_active":"2024-09-21T11:42:37.164723+0000","last_became_peered":"2024-09-21T11:42:37.164723+0000","last_unstale":"2024-09-21T11:44:15.280355+0000","last_undegraded":"2024-09-21T11:44:15.280355+0000","last_fullsized":"2024-09-21T11:44:15.280355+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:42:17.142454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.142454+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":195,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.225667+0000","last_change":"2024-09-21T11:42:37.164162+0000","last_active":"2024-09-21T11:44:08.225667+0000","last_peered":"2024-09-21T11:44:08.225667+0000","last_clean":"2024-09-21T11:44:08.225667+0000","last_became_active":"2024-09-21T11:42:37.163682+0000","last_became_peered":"2024-09-21T11:42:37.163682+0000","last_unstale":"2024-09-21T11:44:08.225667+0000","last_undegraded":"2024-09-21T11:44:08.225667+0000","last_fullsized":"2024-09-21T11:44:08.225667+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:24.468909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.468909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":185,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.233530+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:44:08.233530+0000","last_peered":"2024-09-21T11:44:08.233530+0000","last_clean":"2024-09-21T11:44:08.233530+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:44:08.233530+0000","last_undegraded":"2024-09-21T11:44:08.233530+0000","last_fullsized":"2024-09-21T11:44:08.233530+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"147'12430","reported_seq":25543,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:11.227372+0000","last_change":"2024-09-21T11:43:25.018476+0000","last_active":"2024-09-21T11:44:11.227372+0000","last_peered":"2024-09-21T11:44:11.227372+0000","last_clean":"2024-09-21T11:44:11.227372+0000","last_became_active":"2024-09-21T11:43:24.947584+0000","last_became_peered":"2024-09-21T11:43:24.947584+0000","last_unstale":"2024-09-21T11:44:11.227372+0000","last_undegraded":"2024-09-21T11:44:11.227372+0000","last_fullsized":"2024-09-21T11:44:11.227372+0000","mapping_epoch":132,"log_start":"122'7820","ondisk_log_start":"122'7820","created":82,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4610,"log_dups_size":0,"ondisk_log_size":4610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:52.616980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139460608,"num_objects":2128,"num_object_clones":0,"num_object_copies":6384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12430,"num_write_kb":465856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6476,"num_bytes_recovered":424280064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":144,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.281298+0000","last_change":"2024-09-21T11:42:37.188687+0000","last_active":"2024-09-21T11:44:15.281298+0000","last_peered":"2024-09-21T11:44:15.281298+0000","last_clean":"2024-09-21T11:44:15.281298+0000","last_became_active":"2024-09-21T11:42:37.161797+0000","last_became_peered":"2024-09-21T11:42:37.161797+0000","last_unstale":"2024-09-21T11:44:15.281298+0000","last_undegraded":"2024-09-21T11:44:15.281298+0000","last_fullsized":"2024-09-21T11:44:15.281298+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:57.141798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":94,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.239752+0000","last_change":"2024-09-21T11:42:37.182044+0000","last_active":"2024-09-21T11:44:08.239752+0000","last_peered":"2024-09-21T11:44:08.239752+0000","last_clean":"2024-09-21T11:44:08.239752+0000","last_became_active":"2024-09-21T11:42:37.156191+0000","last_became_peered":"2024-09-21T11:42:37.156191+0000","last_unstale":"2024-09-21T11:44:08.239752+0000","last_undegraded":"2024-09-21T11:44:08.239752+0000","last_fullsized":"2024-09-21T11:44:08.239752+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:55.333247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":215,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.237014+0000","last_change":"2024-09-21T11:43:36.446391+0000","last_active":"2024-09-21T11:44:08.237014+0000","last_peered":"2024-09-21T11:44:08.237014+0000","last_clean":"2024-09-21T11:44:08.237014+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:44:08.237014+0000","last_undegraded":"2024-09-21T11:44:08.237014+0000","last_fullsized":"2024-09-21T11:44:08.237014+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:43:36.446341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.446341+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.312398+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.238747+0000","last_change":"2024-09-21T11:43:28.505134+0000","last_active":"2024-09-21T11:44:08.238747+0000","last_peered":"2024-09-21T11:44:08.238747+0000","last_clean":"2024-09-21T11:44:08.238747+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:44:08.238747+0000","last_undegraded":"2024-09-21T11:44:08.238747+0000","last_fullsized":"2024-09-21T11:44:08.238747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.504894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.504894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.779358+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":188,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.278084+0000","last_change":"2024-09-21T11:42:25.245624+0000","last_active":"2024-09-21T11:44:15.278084+0000","last_peered":"2024-09-21T11:44:15.278084+0000","last_clean":"2024-09-21T11:44:15.278084+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:44:15.278084+0000","last_undegraded":"2024-09-21T11:44:15.278084+0000","last_fullsized":"2024-09-21T11:44:15.278084+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.245565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.245565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:59.530424+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"149'13089","reported_seq":13285,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.671697+0000","last_change":"2024-09-21T11:43:44.550062+0000","last_active":"2024-09-21T11:44:15.671697+0000","last_peered":"2024-09-21T11:44:15.671697+0000","last_clean":"2024-09-21T11:44:15.671697+0000","last_became_active":"2024-09-21T11:41:10.302903+0000","last_became_peered":"2024-09-21T11:41:10.302903+0000","last_unstale":"2024-09-21T11:44:15.671697+0000","last_undegraded":"2024-09-21T11:44:15.671697+0000","last_fullsized":"2024-09-21T11:44:15.671697+0000","mapping_epoch":82,"log_start":"123'8528","ondisk_log_start":"123'8528","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'10628","last_scrub_stamp":"2024-09-21T11:43:44.550004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.550004+0000","objects_scrubbed":4219,"log_size":4561,"log_dups_size":0,"ondisk_log_size":4561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:05.776791+0000","scrub_duration":1037,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113311744,"num_objects":1729,"num_object_clones":0,"num_object_copies":5187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13089,"num_write_kb":474176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":231,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.226954+0000","last_change":"2024-09-21T11:43:40.607996+0000","last_active":"2024-09-21T11:44:08.226954+0000","last_peered":"2024-09-21T11:44:08.226954+0000","last_clean":"2024-09-21T11:44:08.226954+0000","last_became_active":"2024-09-21T11:42:37.164059+0000","last_became_peered":"2024-09-21T11:42:37.164059+0000","last_unstale":"2024-09-21T11:44:08.226954+0000","last_undegraded":"2024-09-21T11:44:08.226954+0000","last_fullsized":"2024-09-21T11:44:08.226954+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:43:40.607801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.607801+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:59.037561+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.224741+0000","last_change":"2024-09-21T11:43:42.641937+0000","last_active":"2024-09-21T11:44:08.224741+0000","last_peered":"2024-09-21T11:44:08.224741+0000","last_clean":"2024-09-21T11:44:08.224741+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:44:08.224741+0000","last_undegraded":"2024-09-21T11:44:08.224741+0000","last_fullsized":"2024-09-21T11:44:08.224741+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:42.641865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:42.641865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:44.867283+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":481,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.278115+0000","last_change":"2024-09-21T11:43:47.494999+0000","last_active":"2024-09-21T11:44:15.278115+0000","last_peered":"2024-09-21T11:44:15.278115+0000","last_clean":"2024-09-21T11:44:15.278115+0000","last_became_active":"2024-09-21T11:42:37.166141+0000","last_became_peered":"2024-09-21T11:42:37.166141+0000","last_unstale":"2024-09-21T11:44:15.278115+0000","last_undegraded":"2024-09-21T11:44:15.278115+0000","last_fullsized":"2024-09-21T11:44:15.278115+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:47.494865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:47.494865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:04.635839+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":249,"num_read_kb":244,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.281529+0000","last_change":"2024-09-21T11:43:45.046195+0000","last_active":"2024-09-21T11:44:15.281529+0000","last_peered":"2024-09-21T11:44:15.281529+0000","last_clean":"2024-09-21T11:44:15.281529+0000","last_became_active":"2024-09-21T11:42:37.172270+0000","last_became_peered":"2024-09-21T11:42:37.172270+0000","last_unstale":"2024-09-21T11:44:15.281529+0000","last_undegraded":"2024-09-21T11:44:15.281529+0000","last_fullsized":"2024-09-21T11:44:15.281529+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.046128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.046128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:59.675637+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":93,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280205+0000","last_change":"2024-09-21T11:44:10.289926+0000","last_active":"2024-09-21T11:44:15.280205+0000","last_peered":"2024-09-21T11:44:15.280205+0000","last_clean":"2024-09-21T11:44:15.280205+0000","last_became_active":"2024-09-21T11:42:37.172645+0000","last_became_peered":"2024-09-21T11:42:37.172645+0000","last_unstale":"2024-09-21T11:44:15.280205+0000","last_undegraded":"2024-09-21T11:44:15.280205+0000","last_fullsized":"2024-09-21T11:44:15.280205+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:17.731384+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"147'12819","reported_seq":13008,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:11.246233+0000","last_change":"2024-09-21T11:42:31.712433+0000","last_active":"2024-09-21T11:44:11.246233+0000","last_peered":"2024-09-21T11:44:11.246233+0000","last_clean":"2024-09-21T11:44:11.246233+0000","last_became_active":"2024-09-21T11:41:10.302497+0000","last_became_peered":"2024-09-21T11:41:10.302497+0000","last_unstale":"2024-09-21T11:44:11.246233+0000","last_undegraded":"2024-09-21T11:44:11.246233+0000","last_fullsized":"2024-09-21T11:44:11.246233+0000","mapping_epoch":82,"log_start":"122'8242","ondisk_log_start":"122'8242","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":4349,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.653561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135462912,"num_objects":2067,"num_object_clones":0,"num_object_copies":6201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12819,"num_write_kb":476352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":271,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.453551+0000","last_change":"2024-09-21T11:42:19.703071+0000","last_active":"2024-09-21T11:44:15.453551+0000","last_peered":"2024-09-21T11:44:15.453551+0000","last_clean":"2024-09-21T11:44:15.453551+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:44:15.453551+0000","last_undegraded":"2024-09-21T11:44:15.453551+0000","last_fullsized":"2024-09-21T11:44:15.453551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:42:19.703007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.703007+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.857413+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":198,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.238610+0000","last_change":"2024-09-21T11:43:30.437283+0000","last_active":"2024-09-21T11:44:08.238610+0000","last_peered":"2024-09-21T11:44:08.238610+0000","last_clean":"2024-09-21T11:44:08.238610+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:44:08.238610+0000","last_undegraded":"2024-09-21T11:44:08.238610+0000","last_fullsized":"2024-09-21T11:44:08.238610+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.437080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.437080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.933810+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":255,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.278402+0000","last_change":"2024-09-21T11:44:10.558635+0000","last_active":"2024-09-21T11:44:15.278402+0000","last_peered":"2024-09-21T11:44:15.278402+0000","last_clean":"2024-09-21T11:44:15.278402+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:44:15.278402+0000","last_undegraded":"2024-09-21T11:44:15.278402+0000","last_fullsized":"2024-09-21T11:44:15.278402+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:44:10.558032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.558032+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:29.068228+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":193,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.277992+0000","last_change":"2024-09-21T11:43:31.630737+0000","last_active":"2024-09-21T11:44:15.277992+0000","last_peered":"2024-09-21T11:44:15.277992+0000","last_clean":"2024-09-21T11:44:15.277992+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:44:15.277992+0000","last_undegraded":"2024-09-21T11:44:15.277992+0000","last_fullsized":"2024-09-21T11:44:15.277992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:31.630670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.630670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.782053+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":188,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.281390+0000","last_change":"2024-09-21T11:41:44.929146+0000","last_active":"2024-09-21T11:44:15.281390+0000","last_peered":"2024-09-21T11:44:15.281390+0000","last_clean":"2024-09-21T11:44:15.281390+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:44:15.281390+0000","last_undegraded":"2024-09-21T11:44:15.281390+0000","last_fullsized":"2024-09-21T11:44:15.281390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:44.929079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:44.929079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:05.159956+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"149'13001","reported_seq":26131,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.757082+0000","last_change":"2024-09-21T11:43:48.624147+0000","last_active":"2024-09-21T11:44:15.757082+0000","last_peered":"2024-09-21T11:44:15.757082+0000","last_clean":"2024-09-21T11:44:15.757082+0000","last_became_active":"2024-09-21T11:43:26.984538+0000","last_became_peered":"2024-09-21T11:43:26.984538+0000","last_unstale":"2024-09-21T11:44:15.757082+0000","last_undegraded":"2024-09-21T11:44:15.757082+0000","last_fullsized":"2024-09-21T11:44:15.757082+0000","mapping_epoch":134,"log_start":"122'8064","ondisk_log_start":"122'8064","created":82,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"104'5645","last_scrub_stamp":"2024-09-21T11:42:17.339535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.339535+0000","objects_scrubbed":2149,"log_size":4937,"log_dups_size":0,"ondisk_log_size":4937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:18.608570+0000","scrub_duration":1564,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112066584,"num_objects":1711,"num_object_clones":0,"num_object_copies":5133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1711,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13001,"num_write_kb":470721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6438,"num_bytes_recovered":421855256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":235,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.271399+0000","last_change":"2024-09-21T11:42:37.179956+0000","last_active":"2024-09-21T11:44:15.271399+0000","last_peered":"2024-09-21T11:44:15.271399+0000","last_clean":"2024-09-21T11:44:15.271399+0000","last_became_active":"2024-09-21T11:42:37.158158+0000","last_became_peered":"2024-09-21T11:42:37.158158+0000","last_unstale":"2024-09-21T11:44:15.271399+0000","last_undegraded":"2024-09-21T11:44:15.271399+0000","last_fullsized":"2024-09-21T11:44:15.271399+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:22.041174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.041174+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:52.060670+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":209,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.272028+0000","last_change":"2024-09-21T11:44:08.947090+0000","last_active":"2024-09-21T11:44:15.272028+0000","last_peered":"2024-09-21T11:44:15.272028+0000","last_clean":"2024-09-21T11:44:15.272028+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:44:15.272028+0000","last_undegraded":"2024-09-21T11:44:15.272028+0000","last_fullsized":"2024-09-21T11:44:15.272028+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.946783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.946783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:11.772867+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.281207+0000","last_change":"2024-09-21T11:43:44.049478+0000","last_active":"2024-09-21T11:44:15.281207+0000","last_peered":"2024-09-21T11:44:15.281207+0000","last_clean":"2024-09-21T11:44:15.281207+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:44:15.281207+0000","last_undegraded":"2024-09-21T11:44:15.281207+0000","last_fullsized":"2024-09-21T11:44:15.281207+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.049424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.049424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:05.726834+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":185,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.235083+0000","last_change":"2024-09-21T11:41:38.453843+0000","last_active":"2024-09-21T11:44:08.235083+0000","last_peered":"2024-09-21T11:44:08.235083+0000","last_clean":"2024-09-21T11:44:08.235083+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:44:08.235083+0000","last_undegraded":"2024-09-21T11:44:08.235083+0000","last_fullsized":"2024-09-21T11:44:08.235083+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:38.453579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:38.453579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"147'12608","reported_seq":12871,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:10.879662+0000","last_change":"2024-09-21T11:43:47.086277+0000","last_active":"2024-09-21T11:44:10.879662+0000","last_peered":"2024-09-21T11:44:10.879662+0000","last_clean":"2024-09-21T11:44:10.879662+0000","last_became_active":"2024-09-21T11:41:10.300107+0000","last_became_peered":"2024-09-21T11:41:10.300107+0000","last_unstale":"2024-09-21T11:44:10.879662+0000","last_undegraded":"2024-09-21T11:44:10.879662+0000","last_fullsized":"2024-09-21T11:44:10.879662+0000","mapping_epoch":82,"log_start":"116'7064","ondisk_log_start":"116'7064","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"143'10760","last_scrub_stamp":"2024-09-21T11:43:47.085319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:47.085319+0000","objects_scrubbed":4008,"log_size":5544,"log_dups_size":0,"ondisk_log_size":5544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:03.663829+0000","scrub_duration":1601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136708096,"num_objects":2086,"num_object_clones":0,"num_object_copies":6258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12608,"num_write_kb":470208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":250,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.271101+0000","last_change":"2024-09-21T11:43:39.233466+0000","last_active":"2024-09-21T11:44:15.271101+0000","last_peered":"2024-09-21T11:44:15.271101+0000","last_clean":"2024-09-21T11:44:15.271101+0000","last_became_active":"2024-09-21T11:42:37.160867+0000","last_became_peered":"2024-09-21T11:42:37.160867+0000","last_unstale":"2024-09-21T11:44:15.271101+0000","last_undegraded":"2024-09-21T11:44:15.271101+0000","last_fullsized":"2024-09-21T11:44:15.271101+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:43:39.233386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.233386+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.064116+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":215,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280869+0000","last_change":"2024-09-21T11:43:36.784531+0000","last_active":"2024-09-21T11:44:15.280869+0000","last_peered":"2024-09-21T11:44:15.280869+0000","last_clean":"2024-09-21T11:44:15.280869+0000","last_became_active":"2024-09-21T11:42:37.167043+0000","last_became_peered":"2024-09-21T11:42:37.167043+0000","last_unstale":"2024-09-21T11:44:15.280869+0000","last_undegraded":"2024-09-21T11:44:15.280869+0000","last_fullsized":"2024-09-21T11:44:15.280869+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.784467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.784467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:39.372586+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":94,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270588+0000","last_change":"2024-09-21T11:43:41.216278+0000","last_active":"2024-09-21T11:44:15.270588+0000","last_peered":"2024-09-21T11:44:15.270588+0000","last_clean":"2024-09-21T11:44:15.270588+0000","last_became_active":"2024-09-21T11:42:37.160659+0000","last_became_peered":"2024-09-21T11:42:37.160659+0000","last_unstale":"2024-09-21T11:44:15.270588+0000","last_undegraded":"2024-09-21T11:44:15.270588+0000","last_fullsized":"2024-09-21T11:44:15.270588+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:41.216215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:41.216215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:45.696826+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":193,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.238685+0000","last_change":"2024-09-21T11:43:46.516054+0000","last_active":"2024-09-21T11:44:08.238685+0000","last_peered":"2024-09-21T11:44:08.238685+0000","last_clean":"2024-09-21T11:44:08.238685+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:44:08.238685+0000","last_undegraded":"2024-09-21T11:44:08.238685+0000","last_fullsized":"2024-09-21T11:44:08.238685+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:46.515782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:46.515782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.893835+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"147'12477","reported_seq":12690,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:11.240450+0000","last_change":"2024-09-21T11:42:29.618177+0000","last_active":"2024-09-21T11:44:11.240450+0000","last_peered":"2024-09-21T11:44:11.240450+0000","last_clean":"2024-09-21T11:44:11.240450+0000","last_became_active":"2024-09-21T11:41:10.303333+0000","last_became_peered":"2024-09-21T11:41:10.303333+0000","last_unstale":"2024-09-21T11:44:11.240450+0000","last_undegraded":"2024-09-21T11:44:11.240450+0000","last_fullsized":"2024-09-21T11:44:11.240450+0000","mapping_epoch":82,"log_start":"122'7844","ondisk_log_start":"122'7844","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6263","last_scrub_stamp":"2024-09-21T11:42:29.617794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.617794+0000","objects_scrubbed":6197,"log_size":4633,"log_dups_size":0,"ondisk_log_size":4633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133627904,"num_objects":2039,"num_object_clones":0,"num_object_copies":6117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12477,"num_write_kb":464512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":220,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270847+0000","last_change":"2024-09-21T11:43:07.347397+0000","last_active":"2024-09-21T11:44:15.270847+0000","last_peered":"2024-09-21T11:44:15.270847+0000","last_clean":"2024-09-21T11:44:15.270847+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:44:15.270847+0000","last_undegraded":"2024-09-21T11:44:15.270847+0000","last_fullsized":"2024-09-21T11:44:15.270847+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:43:07.347346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:07.347346+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.982030+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.234391+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:44:08.234391+0000","last_peered":"2024-09-21T11:44:08.234391+0000","last_clean":"2024-09-21T11:44:08.234391+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:44:08.234391+0000","last_undegraded":"2024-09-21T11:44:08.234391+0000","last_fullsized":"2024-09-21T11:44:08.234391+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":256,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.238515+0000","last_change":"2024-09-21T11:43:38.421538+0000","last_active":"2024-09-21T11:44:08.238515+0000","last_peered":"2024-09-21T11:44:08.238515+0000","last_clean":"2024-09-21T11:44:08.238515+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:44:08.238515+0000","last_undegraded":"2024-09-21T11:44:08.238515+0000","last_fullsized":"2024-09-21T11:44:08.238515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:43:38.421251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.421251+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:48.875567+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.271142+0000","last_change":"2024-09-21T11:42:50.131895+0000","last_active":"2024-09-21T11:44:15.271142+0000","last_peered":"2024-09-21T11:44:15.271142+0000","last_clean":"2024-09-21T11:44:15.271142+0000","last_became_active":"2024-09-21T11:42:37.157732+0000","last_became_peered":"2024-09-21T11:42:37.157732+0000","last_unstale":"2024-09-21T11:44:15.271142+0000","last_undegraded":"2024-09-21T11:44:15.271142+0000","last_fullsized":"2024-09-21T11:44:15.271142+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:50.131830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:50.131830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:54.040481+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"147'12669","reported_seq":12868,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:10.894266+0000","last_change":"2024-09-21T11:43:37.155559+0000","last_active":"2024-09-21T11:44:10.894266+0000","last_peered":"2024-09-21T11:44:10.894266+0000","last_clean":"2024-09-21T11:44:10.894266+0000","last_became_active":"2024-09-21T11:41:10.304149+0000","last_became_peered":"2024-09-21T11:41:10.304149+0000","last_unstale":"2024-09-21T11:44:10.894266+0000","last_undegraded":"2024-09-21T11:44:10.894266+0000","last_fullsized":"2024-09-21T11:44:10.894266+0000","mapping_epoch":82,"log_start":"116'7202","ondisk_log_start":"116'7202","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'10015","last_scrub_stamp":"2024-09-21T11:43:37.155379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.155379+0000","objects_scrubbed":4788,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:48.799347+0000","scrub_duration":1732,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135856128,"num_objects":2073,"num_object_clones":0,"num_object_copies":6219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12669,"num_write_kb":471744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":225,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.226445+0000","last_change":"2024-09-21T11:42:37.192718+0000","last_active":"2024-09-21T11:44:08.226445+0000","last_peered":"2024-09-21T11:44:08.226445+0000","last_clean":"2024-09-21T11:44:08.226445+0000","last_became_active":"2024-09-21T11:42:37.168751+0000","last_became_peered":"2024-09-21T11:42:37.168751+0000","last_unstale":"2024-09-21T11:44:08.226445+0000","last_undegraded":"2024-09-21T11:44:08.226445+0000","last_fullsized":"2024-09-21T11:44:08.226445+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:15.417817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.417817+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280301+0000","last_change":"2024-09-21T11:41:28.936695+0000","last_active":"2024-09-21T11:44:15.280301+0000","last_peered":"2024-09-21T11:44:15.280301+0000","last_clean":"2024-09-21T11:44:15.280301+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:44:15.280301+0000","last_undegraded":"2024-09-21T11:44:15.280301+0000","last_fullsized":"2024-09-21T11:44:15.280301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:28.936645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:28.936645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:04.154692+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":189,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.239007+0000","last_change":"2024-09-21T11:42:18.265457+0000","last_active":"2024-09-21T11:44:08.239007+0000","last_peered":"2024-09-21T11:44:08.239007+0000","last_clean":"2024-09-21T11:44:08.239007+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:44:08.239007+0000","last_undegraded":"2024-09-21T11:44:08.239007+0000","last_fullsized":"2024-09-21T11:44:08.239007+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.265392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.265392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:54.380641+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":176,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.224627+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:44:08.224627+0000","last_peered":"2024-09-21T11:44:08.224627+0000","last_clean":"2024-09-21T11:44:08.224627+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:44:08.224627+0000","last_undegraded":"2024-09-21T11:44:08.224627+0000","last_fullsized":"2024-09-21T11:44:08.224627+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.577967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"147'12520","reported_seq":25780,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:11.256121+0000","last_change":"2024-09-21T11:43:22.892096+0000","last_active":"2024-09-21T11:44:11.256121+0000","last_peered":"2024-09-21T11:44:11.256121+0000","last_clean":"2024-09-21T11:44:11.256121+0000","last_became_active":"2024-09-21T11:43:22.875072+0000","last_became_peered":"2024-09-21T11:43:22.875072+0000","last_unstale":"2024-09-21T11:44:11.256121+0000","last_undegraded":"2024-09-21T11:44:11.256121+0000","last_fullsized":"2024-09-21T11:44:11.256121+0000","mapping_epoch":130,"log_start":"122'7950","ondisk_log_start":"122'7950","created":82,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":0,"log_size":4570,"log_dups_size":0,"ondisk_log_size":4570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:57.625202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132513792,"num_objects":2022,"num_object_clones":0,"num_object_copies":6066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12520,"num_write_kb":465344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6549,"num_bytes_recovered":429195264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":254,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280005+0000","last_change":"2024-09-21T11:43:27.726443+0000","last_active":"2024-09-21T11:44:15.280005+0000","last_peered":"2024-09-21T11:44:15.280005+0000","last_clean":"2024-09-21T11:44:15.280005+0000","last_became_active":"2024-09-21T11:42:37.166373+0000","last_became_peered":"2024-09-21T11:42:37.166373+0000","last_unstale":"2024-09-21T11:44:15.280005+0000","last_undegraded":"2024-09-21T11:44:15.280005+0000","last_fullsized":"2024-09-21T11:44:15.280005+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:43:27.726370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:27.726370+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:36.734856+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.278693+0000","last_change":"2024-09-21T11:41:46.471940+0000","last_active":"2024-09-21T11:44:15.278693+0000","last_peered":"2024-09-21T11:44:15.278693+0000","last_clean":"2024-09-21T11:44:15.278693+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:44:15.278693+0000","last_undegraded":"2024-09-21T11:44:15.278693+0000","last_fullsized":"2024-09-21T11:44:15.278693+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:46.471872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:46.471872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:54.511285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.238702+0000","last_change":"2024-09-21T11:43:26.484397+0000","last_active":"2024-09-21T11:44:08.238702+0000","last_peered":"2024-09-21T11:44:08.238702+0000","last_clean":"2024-09-21T11:44:08.238702+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:44:08.238702+0000","last_undegraded":"2024-09-21T11:44:08.238702+0000","last_fullsized":"2024-09-21T11:44:08.238702+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.484353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.484353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:38.261052+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":188,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.278165+0000","last_change":"2024-09-21T11:43:29.650485+0000","last_active":"2024-09-21T11:44:15.278165+0000","last_peered":"2024-09-21T11:44:15.278165+0000","last_clean":"2024-09-21T11:44:15.278165+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:44:15.278165+0000","last_undegraded":"2024-09-21T11:44:15.278165+0000","last_fullsized":"2024-09-21T11:44:15.278165+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:29.650427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.650427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:32.208091+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"147'12628","reported_seq":12778,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:11.138435+0000","last_change":"2024-09-21T11:42:31.693667+0000","last_active":"2024-09-21T11:44:11.138435+0000","last_peered":"2024-09-21T11:44:11.138435+0000","last_clean":"2024-09-21T11:44:11.138435+0000","last_became_active":"2024-09-21T11:41:10.300287+0000","last_became_peered":"2024-09-21T11:41:10.300287+0000","last_unstale":"2024-09-21T11:44:11.138435+0000","last_undegraded":"2024-09-21T11:44:11.138435+0000","last_fullsized":"2024-09-21T11:44:11.138435+0000","mapping_epoch":82,"log_start":"123'8354","ondisk_log_start":"123'8354","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:41:09.278712+0000","objects_scrubbed":449,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135266304,"num_objects":2064,"num_object_clones":0,"num_object_copies":6192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12628,"num_write_kb":470144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":247,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.271837+0000","last_change":"2024-09-21T11:43:32.231463+0000","last_active":"2024-09-21T11:44:15.271837+0000","last_peered":"2024-09-21T11:44:15.271837+0000","last_clean":"2024-09-21T11:44:15.271837+0000","last_became_active":"2024-09-21T11:42:37.165364+0000","last_became_peered":"2024-09-21T11:42:37.165364+0000","last_unstale":"2024-09-21T11:44:15.271837+0000","last_undegraded":"2024-09-21T11:44:15.271837+0000","last_fullsized":"2024-09-21T11:44:15.271837+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:43:32.231381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.231381+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:36.630886+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":215,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.278249+0000","last_change":"2024-09-21T11:43:44.519748+0000","last_active":"2024-09-21T11:44:15.278249+0000","last_peered":"2024-09-21T11:44:15.278249+0000","last_clean":"2024-09-21T11:44:15.278249+0000","last_became_active":"2024-09-21T11:42:37.165470+0000","last_became_peered":"2024-09-21T11:42:37.165470+0000","last_unstale":"2024-09-21T11:44:15.278249+0000","last_undegraded":"2024-09-21T11:44:15.278249+0000","last_fullsized":"2024-09-21T11:44:15.278249+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.519704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.519704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:47.181516+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.239612+0000","last_change":"2024-09-21T11:43:40.502911+0000","last_active":"2024-09-21T11:44:08.239612+0000","last_peered":"2024-09-21T11:44:08.239612+0000","last_clean":"2024-09-21T11:44:08.239612+0000","last_became_active":"2024-09-21T11:42:37.155675+0000","last_became_peered":"2024-09-21T11:42:37.155675+0000","last_unstale":"2024-09-21T11:44:08.239612+0000","last_undegraded":"2024-09-21T11:44:08.239612+0000","last_fullsized":"2024-09-21T11:44:08.239612+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:40.502747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.502747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:57.625738+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":192,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.234100+0000","last_change":"2024-09-21T11:42:37.170697+0000","last_active":"2024-09-21T11:44:08.234100+0000","last_peered":"2024-09-21T11:44:08.234100+0000","last_clean":"2024-09-21T11:44:08.234100+0000","last_became_active":"2024-09-21T11:42:37.168686+0000","last_became_peered":"2024-09-21T11:42:37.168686+0000","last_unstale":"2024-09-21T11:44:08.234100+0000","last_undegraded":"2024-09-21T11:44:08.234100+0000","last_fullsized":"2024-09-21T11:44:08.234100+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:27.327920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:27.327920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"149'12831","reported_seq":25685,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.727116+0000","last_change":"2024-09-21T11:43:27.000045+0000","last_active":"2024-09-21T11:44:15.727116+0000","last_peered":"2024-09-21T11:44:15.727116+0000","last_clean":"2024-09-21T11:44:15.727116+0000","last_became_active":"2024-09-21T11:43:26.983757+0000","last_became_peered":"2024-09-21T11:43:26.983757+0000","last_unstale":"2024-09-21T11:44:15.727116+0000","last_undegraded":"2024-09-21T11:44:15.727116+0000","last_fullsized":"2024-09-21T11:44:15.727116+0000","mapping_epoch":134,"log_start":"122'7580","ondisk_log_start":"122'7580","created":82,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6370","last_scrub_stamp":"2024-09-21T11:42:31.280157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:41:09.278712+0000","last_clean_scrub_stamp":"2024-09-21T11:42:31.280157+0000","objects_scrubbed":6318,"log_size":5251,"log_dups_size":0,"ondisk_log_size":5251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109510656,"num_objects":1671,"num_object_clones":0,"num_object_copies":5013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12831,"num_write_kb":464064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6310,"num_bytes_recovered":413532160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":213,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.279327+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:44:15.279327+0000","last_peered":"2024-09-21T11:44:15.279327+0000","last_clean":"2024-09-21T11:44:15.279327+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:44:15.279327+0000","last_undegraded":"2024-09-21T11:44:15.279327+0000","last_fullsized":"2024-09-21T11:44:15.279327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:50.165665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":212,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.278384+0000","last_change":"2024-09-21T11:42:19.352990+0000","last_active":"2024-09-21T11:44:15.278384+0000","last_peered":"2024-09-21T11:44:15.278384+0000","last_clean":"2024-09-21T11:44:15.278384+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:44:15.278384+0000","last_undegraded":"2024-09-21T11:44:15.278384+0000","last_fullsized":"2024-09-21T11:44:15.278384+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:42:19.352895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.352895+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:02.484774+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":252,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.277578+0000","last_change":"2024-09-21T11:41:43.495040+0000","last_active":"2024-09-21T11:44:15.277578+0000","last_peered":"2024-09-21T11:44:15.277578+0000","last_clean":"2024-09-21T11:44:15.277578+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:44:15.277578+0000","last_undegraded":"2024-09-21T11:44:15.277578+0000","last_fullsized":"2024-09-21T11:44:15.277578+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:41:43.494960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:43.494960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:51.520216+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280076+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:44:15.280076+0000","last_peered":"2024-09-21T11:44:15.280076+0000","last_clean":"2024-09-21T11:44:15.280076+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:44:15.280076+0000","last_undegraded":"2024-09-21T11:44:15.280076+0000","last_fullsized":"2024-09-21T11:44:15.280076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:22.245021+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280164+0000","last_change":"2024-09-21T11:43:30.734704+0000","last_active":"2024-09-21T11:44:15.280164+0000","last_peered":"2024-09-21T11:44:15.280164+0000","last_clean":"2024-09-21T11:44:15.280164+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:44:15.280164+0000","last_undegraded":"2024-09-21T11:44:15.280164+0000","last_fullsized":"2024-09-21T11:44:15.280164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.734361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.734361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:35.190586+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":217,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.240674+0000","last_change":"2024-09-21T11:42:37.201056+0000","last_active":"2024-09-21T11:44:08.240674+0000","last_peered":"2024-09-21T11:44:08.240674+0000","last_clean":"2024-09-21T11:44:08.240674+0000","last_became_active":"2024-09-21T11:42:37.171563+0000","last_became_peered":"2024-09-21T11:42:37.171563+0000","last_unstale":"2024-09-21T11:44:08.240674+0000","last_undegraded":"2024-09-21T11:44:08.240674+0000","last_fullsized":"2024-09-21T11:44:08.240674+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:41:25.310957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:25.310957+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":199,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.281384+0000","last_change":"2024-09-21T11:42:37.170710+0000","last_active":"2024-09-21T11:44:15.281384+0000","last_peered":"2024-09-21T11:44:15.281384+0000","last_clean":"2024-09-21T11:44:15.281384+0000","last_became_active":"2024-09-21T11:42:37.170365+0000","last_became_peered":"2024-09-21T11:42:37.170365+0000","last_unstale":"2024-09-21T11:44:15.281384+0000","last_undegraded":"2024-09-21T11:44:15.281384+0000","last_fullsized":"2024-09-21T11:44:15.281384+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:15.090108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.090108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.277876+0000","last_change":"2024-09-21T11:43:45.551832+0000","last_active":"2024-09-21T11:44:15.277876+0000","last_peered":"2024-09-21T11:44:15.277876+0000","last_clean":"2024-09-21T11:44:15.277876+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:44:15.277876+0000","last_undegraded":"2024-09-21T11:44:15.277876+0000","last_fullsized":"2024-09-21T11:44:15.277876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.551792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.551792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:49.431636+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":225,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.227043+0000","last_change":"2024-09-21T11:43:32.077598+0000","last_active":"2024-09-21T11:44:08.227043+0000","last_peered":"2024-09-21T11:44:08.227043+0000","last_clean":"2024-09-21T11:44:08.227043+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:44:08.227043+0000","last_undegraded":"2024-09-21T11:44:08.227043+0000","last_fullsized":"2024-09-21T11:44:08.227043+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:11.469922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.469922+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":208,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.277732+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:44:15.277732+0000","last_peered":"2024-09-21T11:44:15.277732+0000","last_clean":"2024-09-21T11:44:15.277732+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:44:15.277732+0000","last_undegraded":"2024-09-21T11:44:15.277732+0000","last_fullsized":"2024-09-21T11:44:15.277732+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:25.473570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.240574+0000","last_change":"2024-09-21T11:43:38.461076+0000","last_active":"2024-09-21T11:44:08.240574+0000","last_peered":"2024-09-21T11:44:08.240574+0000","last_clean":"2024-09-21T11:44:08.240574+0000","last_became_active":"2024-09-21T11:42:37.172423+0000","last_became_peered":"2024-09-21T11:42:37.172423+0000","last_unstale":"2024-09-21T11:44:08.240574+0000","last_undegraded":"2024-09-21T11:44:08.240574+0000","last_fullsized":"2024-09-21T11:44:08.240574+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.461014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.461014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:44.750422+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":193,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.237317+0000","last_change":"2024-09-21T11:43:42.448731+0000","last_active":"2024-09-21T11:44:08.237317+0000","last_peered":"2024-09-21T11:44:08.237317+0000","last_clean":"2024-09-21T11:44:08.237317+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:44:08.237317+0000","last_undegraded":"2024-09-21T11:44:08.237317+0000","last_fullsized":"2024-09-21T11:44:08.237317+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:42.448669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:42.448669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.700328+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":212,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280323+0000","last_change":"2024-09-21T11:41:33.917474+0000","last_active":"2024-09-21T11:44:15.280323+0000","last_peered":"2024-09-21T11:44:15.280323+0000","last_clean":"2024-09-21T11:44:15.280323+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:44:15.280323+0000","last_undegraded":"2024-09-21T11:44:15.280323+0000","last_fullsized":"2024-09-21T11:44:15.280323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:33.917389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:33.917389+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.226024+0000","last_change":"2024-09-21T11:42:22.478692+0000","last_active":"2024-09-21T11:44:08.226024+0000","last_peered":"2024-09-21T11:44:08.226024+0000","last_clean":"2024-09-21T11:44:08.226024+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:44:08.226024+0000","last_undegraded":"2024-09-21T11:44:08.226024+0000","last_fullsized":"2024-09-21T11:44:08.226024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:22.478619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.478619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":240,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.235422+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:44:08.235422+0000","last_peered":"2024-09-21T11:44:08.235422+0000","last_clean":"2024-09-21T11:44:08.235422+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:44:08.235422+0000","last_undegraded":"2024-09-21T11:44:08.235422+0000","last_fullsized":"2024-09-21T11:44:08.235422+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:56.640568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":197,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.271567+0000","last_change":"2024-09-21T11:43:24.300706+0000","last_active":"2024-09-21T11:44:15.271567+0000","last_peered":"2024-09-21T11:44:15.271567+0000","last_clean":"2024-09-21T11:44:15.271567+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:44:15.271567+0000","last_undegraded":"2024-09-21T11:44:15.271567+0000","last_fullsized":"2024-09-21T11:44:15.271567+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:24.300642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:24.300642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:28.984128+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":227,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.235654+0000","last_change":"2024-09-21T11:42:37.207434+0000","last_active":"2024-09-21T11:44:08.235654+0000","last_peered":"2024-09-21T11:44:08.235654+0000","last_clean":"2024-09-21T11:44:08.235654+0000","last_became_active":"2024-09-21T11:42:37.171345+0000","last_became_peered":"2024-09-21T11:42:37.171345+0000","last_unstale":"2024-09-21T11:44:08.235654+0000","last_undegraded":"2024-09-21T11:44:08.235654+0000","last_fullsized":"2024-09-21T11:44:08.235654+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":208,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.277972+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:44:15.277972+0000","last_peered":"2024-09-21T11:44:15.277972+0000","last_clean":"2024-09-21T11:44:15.277972+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:44:15.277972+0000","last_undegraded":"2024-09-21T11:44:15.277972+0000","last_fullsized":"2024-09-21T11:44:15.277972+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:12.955314+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"114'9","reported_seq":264,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280139+0000","last_change":"2024-09-21T11:43:28.003108+0000","last_active":"2024-09-21T11:44:15.280139+0000","last_peered":"2024-09-21T11:44:15.280139+0000","last_clean":"2024-09-21T11:44:15.280139+0000","last_became_active":"2024-09-21T11:42:37.163771+0000","last_became_peered":"2024-09-21T11:42:37.163771+0000","last_unstale":"2024-09-21T11:44:15.280139+0000","last_undegraded":"2024-09-21T11:44:15.280139+0000","last_fullsized":"2024-09-21T11:44:15.280139+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:28.003036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.003036+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:32.544327+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":184,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.226518+0000","last_change":"2024-09-21T11:41:30.274387+0000","last_active":"2024-09-21T11:44:08.226518+0000","last_peered":"2024-09-21T11:44:08.226518+0000","last_clean":"2024-09-21T11:44:08.226518+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:44:08.226518+0000","last_undegraded":"2024-09-21T11:44:08.226518+0000","last_fullsized":"2024-09-21T11:44:08.226518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:30.274201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:30.274201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":243,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.270420+0000","last_change":"2024-09-21T11:43:40.233160+0000","last_active":"2024-09-21T11:44:15.270420+0000","last_peered":"2024-09-21T11:44:15.270420+0000","last_clean":"2024-09-21T11:44:15.270420+0000","last_became_active":"2024-09-21T11:42:37.154835+0000","last_became_peered":"2024-09-21T11:42:37.154835+0000","last_unstale":"2024-09-21T11:44:15.270420+0000","last_undegraded":"2024-09-21T11:44:15.270420+0000","last_fullsized":"2024-09-21T11:44:15.270420+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:43:40.233067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.233067+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:41.481160+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":201,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.271046+0000","last_change":"2024-09-21T11:43:00.245058+0000","last_active":"2024-09-21T11:44:15.271046+0000","last_peered":"2024-09-21T11:44:15.271046+0000","last_clean":"2024-09-21T11:44:15.271046+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:44:15.271046+0000","last_undegraded":"2024-09-21T11:44:15.271046+0000","last_fullsized":"2024-09-21T11:44:15.271046+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:00.244985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:00.244985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:22.792144+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":86,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280176+0000","last_change":"2024-09-21T11:42:37.163592+0000","last_active":"2024-09-21T11:44:15.280176+0000","last_peered":"2024-09-21T11:44:15.280176+0000","last_clean":"2024-09-21T11:44:15.280176+0000","last_became_active":"2024-09-21T11:42:37.163261+0000","last_became_peered":"2024-09-21T11:42:37.163261+0000","last_unstale":"2024-09-21T11:44:15.280176+0000","last_undegraded":"2024-09-21T11:44:15.280176+0000","last_fullsized":"2024-09-21T11:44:15.280176+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.301406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.301406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":177,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.232991+0000","last_change":"2024-09-21T11:40:06.353577+0000","last_active":"2024-09-21T11:44:08.232991+0000","last_peered":"2024-09-21T11:44:08.232991+0000","last_clean":"2024-09-21T11:44:08.232991+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:44:08.232991+0000","last_undegraded":"2024-09-21T11:44:08.232991+0000","last_fullsized":"2024-09-21T11:44:08.232991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:04.613207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":139,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280771+0000","last_change":"2024-09-21T11:43:41.808987+0000","last_active":"2024-09-21T11:44:15.280771+0000","last_peered":"2024-09-21T11:44:15.280771+0000","last_clean":"2024-09-21T11:44:15.280771+0000","last_became_active":"2024-09-21T11:42:37.167265+0000","last_became_peered":"2024-09-21T11:42:37.167265+0000","last_unstale":"2024-09-21T11:44:15.280771+0000","last_undegraded":"2024-09-21T11:44:15.280771+0000","last_fullsized":"2024-09-21T11:44:15.280771+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:41.808900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:41.808900+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.057274+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.278583+0000","last_change":"2024-09-21T11:43:26.586603+0000","last_active":"2024-09-21T11:44:15.278583+0000","last_peered":"2024-09-21T11:44:15.278583+0000","last_clean":"2024-09-21T11:44:15.278583+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:44:15.278583+0000","last_undegraded":"2024-09-21T11:44:15.278583+0000","last_fullsized":"2024-09-21T11:44:15.278583+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.586562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.586562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:55.496321+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.234173+0000","last_change":"2024-09-21T11:42:37.167224+0000","last_active":"2024-09-21T11:44:08.234173+0000","last_peered":"2024-09-21T11:44:08.234173+0000","last_clean":"2024-09-21T11:44:08.234173+0000","last_became_active":"2024-09-21T11:42:37.166917+0000","last_became_peered":"2024-09-21T11:42:37.166917+0000","last_unstale":"2024-09-21T11:44:08.234173+0000","last_undegraded":"2024-09-21T11:44:08.234173+0000","last_fullsized":"2024-09-21T11:44:08.234173+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:23.514572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:23.514572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":196,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.220113+0000","last_change":"2024-09-21T11:42:37.177478+0000","last_active":"2024-09-21T11:44:08.220113+0000","last_peered":"2024-09-21T11:44:08.220113+0000","last_clean":"2024-09-21T11:44:08.220113+0000","last_became_active":"2024-09-21T11:42:37.161589+0000","last_became_peered":"2024-09-21T11:42:37.161589+0000","last_unstale":"2024-09-21T11:44:08.220113+0000","last_undegraded":"2024-09-21T11:44:08.220113+0000","last_fullsized":"2024-09-21T11:44:08.220113+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:41:29.236463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:29.236463+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":212,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.234547+0000","last_change":"2024-09-21T11:42:15.470041+0000","last_active":"2024-09-21T11:44:08.234547+0000","last_peered":"2024-09-21T11:44:08.234547+0000","last_clean":"2024-09-21T11:44:08.234547+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:44:08.234547+0000","last_undegraded":"2024-09-21T11:44:08.234547+0000","last_fullsized":"2024-09-21T11:44:08.234547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:15.469421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.469421+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":207,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.281524+0000","last_change":"2024-09-21T11:42:37.173431+0000","last_active":"2024-09-21T11:44:15.281524+0000","last_peered":"2024-09-21T11:44:15.281524+0000","last_clean":"2024-09-21T11:44:15.281524+0000","last_became_active":"2024-09-21T11:42:37.172859+0000","last_became_peered":"2024-09-21T11:42:37.172859+0000","last_unstale":"2024-09-21T11:44:15.281524+0000","last_undegraded":"2024-09-21T11:44:15.281524+0000","last_fullsized":"2024-09-21T11:44:15.281524+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.952335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.952335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":192,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.277254+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:44:15.277254+0000","last_peered":"2024-09-21T11:44:15.277254+0000","last_clean":"2024-09-21T11:44:15.277254+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:44:15.277254+0000","last_undegraded":"2024-09-21T11:44:15.277254+0000","last_fullsized":"2024-09-21T11:44:15.277254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:31.612728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":185,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.234306+0000","last_change":"2024-09-21T11:42:12.409717+0000","last_active":"2024-09-21T11:44:08.234306+0000","last_peered":"2024-09-21T11:44:08.234306+0000","last_clean":"2024-09-21T11:44:08.234306+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:44:08.234306+0000","last_undegraded":"2024-09-21T11:44:08.234306+0000","last_fullsized":"2024-09-21T11:44:08.234306+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:12.409641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:12.409641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":225,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.233143+0000","last_change":"2024-09-21T11:42:37.195076+0000","last_active":"2024-09-21T11:44:08.233143+0000","last_peered":"2024-09-21T11:44:08.233143+0000","last_clean":"2024-09-21T11:44:08.233143+0000","last_became_active":"2024-09-21T11:42:37.169482+0000","last_became_peered":"2024-09-21T11:42:37.169482+0000","last_unstale":"2024-09-21T11:44:08.233143+0000","last_undegraded":"2024-09-21T11:44:08.233143+0000","last_fullsized":"2024-09-21T11:44:08.233143+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:41:32.414244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:32.414244+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.229188+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:44:08.229188+0000","last_peered":"2024-09-21T11:44:08.229188+0000","last_clean":"2024-09-21T11:44:08.229188+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:44:08.229188+0000","last_undegraded":"2024-09-21T11:44:08.229188+0000","last_fullsized":"2024-09-21T11:44:08.229188+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":200,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.281589+0000","last_change":"2024-09-21T11:43:39.838067+0000","last_active":"2024-09-21T11:44:15.281589+0000","last_peered":"2024-09-21T11:44:15.281589+0000","last_clean":"2024-09-21T11:44:15.281589+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:44:15.281589+0000","last_undegraded":"2024-09-21T11:44:15.281589+0000","last_fullsized":"2024-09-21T11:44:15.281589+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:39.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.838002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:53.931489+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":196,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.277922+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:44:15.277922+0000","last_peered":"2024-09-21T11:44:15.277922+0000","last_clean":"2024-09-21T11:44:15.277922+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:44:15.277922+0000","last_undegraded":"2024-09-21T11:44:15.277922+0000","last_fullsized":"2024-09-21T11:44:15.277922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:27.321443+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":93,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.280184+0000","last_change":"2024-09-21T11:44:11.214596+0000","last_active":"2024-09-21T11:44:15.280184+0000","last_peered":"2024-09-21T11:44:15.280184+0000","last_clean":"2024-09-21T11:44:15.280184+0000","last_became_active":"2024-09-21T11:42:37.155620+0000","last_became_peered":"2024-09-21T11:42:37.155620+0000","last_unstale":"2024-09-21T11:44:15.280184+0000","last_undegraded":"2024-09-21T11:44:15.280184+0000","last_fullsized":"2024-09-21T11:44:15.280184+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:11.214534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:11.214534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:16.652208+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":204,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:08.233369+0000","last_change":"2024-09-21T11:43:36.413767+0000","last_active":"2024-09-21T11:44:08.233369+0000","last_peered":"2024-09-21T11:44:08.233369+0000","last_clean":"2024-09-21T11:44:08.233369+0000","last_became_active":"2024-09-21T11:42:37.167248+0000","last_became_peered":"2024-09-21T11:42:37.167248+0000","last_unstale":"2024-09-21T11:44:08.233369+0000","last_undegraded":"2024-09-21T11:44:08.233369+0000","last_fullsized":"2024-09-21T11:44:08.233369+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.413725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.413725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.736856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":243,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-21T11:44:15.282302+0000","last_change":"2024-09-21T11:43:32.709031+0000","last_active":"2024-09-21T11:44:15.282302+0000","last_peered":"2024-09-21T11:44:15.282302+0000","last_clean":"2024-09-21T11:44:15.282302+0000","last_became_active":"2024-09-21T11:42:37.171253+0000","last_became_peered":"2024-09-21T11:42:37.171253+0000","last_unstale":"2024-09-21T11:44:15.282302+0000","last_undegraded":"2024-09-21T11:44:15.282302+0000","last_fullsized":"2024-09-21T11:44:15.282302+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:43:32.708849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.708849+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:44:43.362664+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":197,"reported_epoch":147,"state":"active+clean","last_fresh":"2024-09-21T11:44:09.512919+0000","last_change":"2024-09-21T11:44:09.512919+0000","last_active":"2024-09-21T11:44:09.512919+0000","last_peered":"2024-09-21T11:44:09.512919+0000","last_clean":"2024-09-21T11:44:09.512919+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:44:09.512919+0000","last_undegraded":"2024-09-21T11:44:09.512919+0000","last_fullsized":"2024-09-21T11:44:09.512919+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.512741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.512741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:11.430500+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":1990787096,"num_objects":30378,"num_object_clones":0,"num_object_copies":91134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30378,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204434,"num_write_kb":7513921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38414,"num_bytes_recovered":2517303320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7561035776,"data_stored":7561019488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76815,"ondisk_log_size":76815,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":5367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":249,"num_read_kb":244,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234040,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":948616,"kb_used_data":777200,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":92796536,"statfs":{"total":95995035648,"available":95023652864,"internally_reserved":0,"allocated":795852800,"data_stored":795466607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":175503818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":797012,"kb_used_data":661696,"kb_used_omap":1,"kb_used_meta":135294,"kb_avail":92948140,"statfs":{"total":95995035648,"available":95178895360,"internally_reserved":0,"allocated":677576704,"data_stored":677174976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138541510},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":536568,"kb_used_data":442916,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":93208584,"statfs":{"total":95995035648,"available":95445590016,"internally_reserved":0,"allocated":453545984,"data_stored":453170914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":95877574},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":665004,"kb_used_data":557464,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93080148,"statfs":{"total":95995035648,"available":95314071552,"internally_reserved":0,"allocated":570843136,"data_stored":570474157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110098885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117069,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1511332,"kb_used_data":1339792,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":92233820,"statfs":{"total":95995035648,"available":94447431680,"internally_reserved":0,"allocated":1371947008,"data_stored":1371578994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":175503815},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1283784,"kb_used_data":1077428,"kb_used_omap":1,"kb_used_meta":206078,"kb_avail":92461368,"statfs":{"total":95995035648,"available":94680440832,"internally_reserved":0,"allocated":1103286272,"data_stored":1102946568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":211024325},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":55834574938,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1215224,"kb_used_data":1089640,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":92529928,"statfs":{"total":95995035648,"available":94750646272,"internally_reserved":0,"allocated":1115791360,"data_stored":1115400757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1080233984,"data_stored":1080229912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1604653056,"data_stored":1604648984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1080754176,"data_stored":1080754176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1339363328,"data_stored":1339359256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":553058304,"data_stored":553058304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":445054976,"data_stored":445054976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":669782016,"data_stored":669777944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":788135936,"data_stored":788135936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:44:18.272 INFO:tasks.thrashosds.thrasher:Setting 6.c to [3, 7, 1] 2024-09-21T11:44:18.273 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '6.c', '3', '7', '1'] 2024-09-21T11:44:18.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pg-upmap 6.c 3 7 1 2024-09-21T11:44:18.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:18.375 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:44:18.375 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:44:18.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:18.377 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:18.377 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:18.404 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:18.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:18.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:18 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:44:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:18 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:44:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:18 smithi043 ceph-mon[36454]: from='client.24956 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:44:18.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:18 smithi043 ceph-mon[36454]: osdmap e150: 8 total, 8 up, 7 in 2024-09-21T11:44:18.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:18 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:44:18.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:18 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:44:18.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:18 smithi043 ceph-mon[29710]: from='client.24956 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:44:18.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:18 smithi043 ceph-mon[29710]: osdmap e150: 8 total, 8 up, 7 in 2024-09-21T11:44:18.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:18 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:44:18.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:18 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:44:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:18 smithi175 ceph-mon[32613]: from='client.24956 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:44:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:18 smithi175 ceph-mon[32613]: osdmap e150: 8 total, 8 up, 7 in 2024-09-21T11:44:18.534 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:18.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:18.536 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:18.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:18.674 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:18.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:18.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:18.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:18.804 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:18.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:18.805 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:18.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:18.935 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:18.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:18.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:19.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:19.062 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:19.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:19.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:19.063 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:19.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:19.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:19.205 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:19.206 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:19.228 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:44:19.248 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:44:19.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:19.355 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:19.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:19.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:19.457 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:19.490 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:19.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:19 smithi043 ceph-mon[36454]: pgmap v194: 148 pgs: 148 active+clean; 1.6 GiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:44:19.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:19 smithi043 ceph-mon[36454]: 3.7 scrub starts 2024-09-21T11:44:19.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:19 smithi043 ceph-mon[36454]: 3.7 scrub ok 2024-09-21T11:44:19.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:19.492 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:19.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:19 smithi043 ceph-mon[29710]: pgmap v194: 148 pgs: 148 active+clean; 1.6 GiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:44:19.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:19 smithi043 ceph-mon[29710]: 3.7 scrub starts 2024-09-21T11:44:19.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:19 smithi043 ceph-mon[29710]: 3.7 scrub ok 2024-09-21T11:44:19.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:19 smithi175 ceph-mon[32613]: pgmap v194: 148 pgs: 148 active+clean; 1.6 GiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:44:19.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:19 smithi175 ceph-mon[32613]: 3.7 scrub starts 2024-09-21T11:44:19.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:19 smithi175 ceph-mon[32613]: 3.7 scrub ok 2024-09-21T11:44:19.569 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:19.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:19.631 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:19.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:19.632 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:19.733 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:19.764 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:19.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:19.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:19.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:19.866 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:19.895 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:19.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:19.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:19.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:20.042 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:20.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:20.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:20.143 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:20.150 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:44:20.150 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:44:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:20.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:20.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:20.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:20.297 INFO:teuthology.orchestra.run.smithi043.stderr:set 6.c pg_upmap mapping to [3,7,1] 2024-09-21T11:44:20.302 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:44:20.314 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:20.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:20.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:20.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:20.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:20.452 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:20.452 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:20.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:20.592 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:20.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:20.593 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:20.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:20.730 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:20.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:20.731 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/777576848' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[36454]: 7.d scrub starts 2024-09-21T11:44:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[36454]: 4.18 scrub starts 2024-09-21T11:44:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[36454]: 4.18 scrub ok 2024-09-21T11:44:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/192844754' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.c", "id": [3, 7, 1]} : dispatch 2024-09-21T11:44:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.c", "id": [3, 7, 1]} : dispatch 2024-09-21T11:44:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/777576848' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[29710]: 7.d scrub starts 2024-09-21T11:44:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[29710]: 4.18 scrub starts 2024-09-21T11:44:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[29710]: 4.18 scrub ok 2024-09-21T11:44:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/192844754' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.c", "id": [3, 7, 1]} : dispatch 2024-09-21T11:44:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:20 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.c", "id": [3, 7, 1]} : dispatch 2024-09-21T11:44:20.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/777576848' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:20 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:20 smithi175 ceph-mon[32613]: 7.d scrub starts 2024-09-21T11:44:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:20 smithi175 ceph-mon[32613]: 4.18 scrub starts 2024-09-21T11:44:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:20 smithi175 ceph-mon[32613]: 4.18 scrub ok 2024-09-21T11:44:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/192844754' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.c", "id": [3, 7, 1]} : dispatch 2024-09-21T11:44:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:20 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.c", "id": [3, 7, 1]} : dispatch 2024-09-21T11:44:20.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:20.867 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:20.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:20.868 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:20.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:44:20.969 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:21.047 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:21.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:21.049 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:21.150 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:21.185 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:21.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:21.186 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:21.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:44:20 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:44:21.287 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:21.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:21.345 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:21.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:21.446 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:21.449 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:21.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:21.488 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:21.488 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[29710]: 4.10 scrub starts 2024-09-21T11:44:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[29710]: 4.10 scrub ok 2024-09-21T11:44:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[29710]: pgmap v195: 148 pgs: 148 active+clean; 1.6 GiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 783 B/s rd, 0 B/s wr, 928 op/s 2024-09-21T11:44:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[29710]: 7.d scrub ok 2024-09-21T11:44:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:44:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.c", "id": [3, 7, 1]}]': finished 2024-09-21T11:44:21.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[29710]: osdmap e151: 8 total, 8 up, 7 in 2024-09-21T11:44:21.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[36454]: 4.10 scrub starts 2024-09-21T11:44:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[36454]: 4.10 scrub ok 2024-09-21T11:44:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[36454]: pgmap v195: 148 pgs: 148 active+clean; 1.6 GiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 783 B/s rd, 0 B/s wr, 928 op/s 2024-09-21T11:44:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[36454]: 7.d scrub ok 2024-09-21T11:44:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:44:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.c", "id": [3, 7, 1]}]': finished 2024-09-21T11:44:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:21 smithi043 ceph-mon[36454]: osdmap e151: 8 total, 8 up, 7 in 2024-09-21T11:44:21.585 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:21 smithi175 ceph-mon[32613]: 4.10 scrub starts 2024-09-21T11:44:21.585 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:21 smithi175 ceph-mon[32613]: 4.10 scrub ok 2024-09-21T11:44:21.585 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:21 smithi175 ceph-mon[32613]: pgmap v195: 148 pgs: 148 active+clean; 1.6 GiB data, 6.0 GiB used, 620 GiB / 626 GiB avail; 783 B/s rd, 0 B/s wr, 928 op/s 2024-09-21T11:44:21.585 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:21 smithi175 ceph-mon[32613]: 7.d scrub ok 2024-09-21T11:44:21.585 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:21 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:21.586 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:21 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:44:21.586 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:21 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.c", "id": [3, 7, 1]}]': finished 2024-09-21T11:44:21.586 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:21 smithi175 ceph-mon[32613]: osdmap e151: 8 total, 8 up, 7 in 2024-09-21T11:44:21.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:21.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:21.630 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:21.630 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:21.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:21.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:21.775 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:21.876 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:21.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:21.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:22.022 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:22.054 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:44:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:44:22.054 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:44:22.055 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:44:22.056 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:22.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:22.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:22.089 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:44:22.090 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:44:22.122 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:44:22.122 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:44:22.157 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:22.160 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:44:22.160 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:44:22.186 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:22.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:22.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:22.194 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:44:22.194 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:44:22.226 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:44:22.226 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:44:22.261 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:44:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:44:22.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:22.297 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:44:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:44:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:22.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:22.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:22.434 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:22.460 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:22.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:22.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:22.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:22.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:22.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:22.594 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:22.615 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:22.615 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2827032204' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:22.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:22.729 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:22.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:22.730 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2827032204' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2827032204' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:22.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:22.860 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:44:22.870 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:22.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:22.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:22.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:23.007 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:23.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:23.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:23.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:23.145 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:23.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:23.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:23.245 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:23.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:23.287 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:23.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:23.288 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:23.389 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:23.417 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:23.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:23.418 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:23.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:23 smithi043 ceph-mon[29710]: pgmap v197: 148 pgs: 148 active+clean; 1.3 GiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.82k op/s 2024-09-21T11:44:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:23 smithi043 ceph-mon[36454]: pgmap v197: 148 pgs: 148 active+clean; 1.3 GiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.82k op/s 2024-09-21T11:44:23.519 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:23.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:23 smithi175 ceph-mon[32613]: pgmap v197: 148 pgs: 148 active+clean; 1.3 GiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.82k op/s 2024-09-21T11:44:23.559 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:23.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:23.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:23.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:23.698 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:23.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:23.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:23.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:23.835 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:23.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:23.836 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:23.844 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:44:23.844 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:44:23.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:23.844 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.14805.0:217212 7.8 e151/82 7:18cb9efb:::benchmark_data_smithi043_70646_object99804:head v 151'13669, mlcod=151'13667)", 2024-09-21T11:44:23.844 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:23.840816+0000", 2024-09-21T11:44:23.844 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.001526375, 2024-09-21T11:44:23.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015542049999999999, 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:23.840816+0000", 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:23.840816+0000", 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:23.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:23.840816+0000", 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9999999999999997e-07 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:23.840822+0000", 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8810000000000001e-06 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:23.840836+0000", 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3359e-05 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:23.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:23.840842+0000", 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0900000000000001e-06 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:23.840903+0000", 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1526999999999996e-05 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:23.840927+0000", 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3493e-05 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:23.847 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:23.848 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:44:23.848 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T11:44:23.848 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:44:23.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:23.974 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:23.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:23.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:24.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:24.107 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:24.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:24.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:24.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:24.244 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:24.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:24.245 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:24.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:24.393 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:24.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:24.394 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:24.495 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:24.519 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:24.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:24.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:24.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:24.641 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:44:24.658 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:24.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:24.659 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:24.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:24.799 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:24.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:24.800 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:24.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:24.937 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:24.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:24.938 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:25.010 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:25.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:25.085 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:25.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:25.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:25.187 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:25.222 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:25.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:25.223 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:25.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:44:25.324 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:25.352 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:25.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:25.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:25.449 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:25 smithi175 ceph-mon[32613]: pgmap v198: 148 pgs: 148 active+clean; 1.3 GiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:44:25.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:25.483 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:25.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:25.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:25 smithi043 ceph-mon[36454]: pgmap v198: 148 pgs: 148 active+clean; 1.3 GiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:44:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:44:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:44:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:44:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:25 smithi043 ceph-mon[29710]: pgmap v198: 148 pgs: 148 active+clean; 1.3 GiB data, 5.0 GiB used, 621 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:44:25.556 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:44:25.556 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:44:25.556 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:44:25.556 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:44:25.556 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:44:25.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:25.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:25.620 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:25.621 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:25.721 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:25.751 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:25.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:25.752 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:25.853 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:25.893 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:25.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:25.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:25.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:26.025 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:26.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:26.026 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:26.127 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:26.158 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:26.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:26.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:26.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:26.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:26.283 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:26.327 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:44:26.383 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:26.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:26.430 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:26.430 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:26.531 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:26.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:26.571 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:26.571 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:26.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:26.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:26.709 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:26.709 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:26.713 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:44:26.810 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:26.848 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:26.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:26.849 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:26.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:26.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:26.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:27.074 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:27.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:27.108 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:27.108 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:27.208 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:27.230 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:27.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:27.231 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:27.252 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:44:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:44:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:44:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:44:27.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:79209 7.5 7:a7c68b32:::benchmark_data_smithi043_70646_object79208:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:44:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:06.636157+0000", 2024-09-21T11:44:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.61246062699999, 2024-09-21T11:44:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.073249726000000001, 2024-09-21T11:44:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.253 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 79209 2024-09-21T11:44:27.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636157+0000", 2024-09-21T11:44:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636157+0000", 2024-09-21T11:44:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636158+0000", 2024-09-21T11:44:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4799999999999997e-07 2024-09-21T11:44:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636238+0000", 2024-09-21T11:44:27.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0576999999999998e-05 2024-09-21T11:44:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636248+0000", 2024-09-21T11:44:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1290000000000002e-06 2024-09-21T11:44:27.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636253+0000", 2024-09-21T11:44:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7579999999999998e-06 2024-09-21T11:44:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.638949+0000", 2024-09-21T11:44:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026954660000000001 2024-09-21T11:44:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.639072+0000", 2024-09-21T11:44:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123707 2024-09-21T11:44:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.639222+0000", 2024-09-21T11:44:27.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014934799999999999 2024-09-21T11:44:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.641482+0000", 2024-09-21T11:44:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022601159999999999 2024-09-21T11:44:27.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.643463+0000", 2024-09-21T11:44:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019809939999999998 2024-09-21T11:44:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.709301+0000", 2024-09-21T11:44:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.065838240000000006 2024-09-21T11:44:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.709361+0000", 2024-09-21T11:44:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9478000000000002e-05 2024-09-21T11:44:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.709407+0000", 2024-09-21T11:44:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6165000000000003e-05 2024-09-21T11:44:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:80911 7.5 7:a71a7e8a:::benchmark_data_smithi043_70646_object80910:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:44:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:08.206845+0000", 2024-09-21T11:44:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 139.04177310700001, 2024-09-21T11:44:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.089039726, 2024-09-21T11:44:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.267 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.268 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.269 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 80911 2024-09-21T11:44:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206845+0000", 2024-09-21T11:44:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206845+0000", 2024-09-21T11:44:27.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206849+0000", 2024-09-21T11:44:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7400000000000004e-06 2024-09-21T11:44:27.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206891+0000", 2024-09-21T11:44:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1918000000000001e-05 2024-09-21T11:44:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206908+0000", 2024-09-21T11:44:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6589999999999999e-05 2024-09-21T11:44:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206917+0000", 2024-09-21T11:44:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7169999999999997e-06 2024-09-21T11:44:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206987+0000", 2024-09-21T11:44:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0568999999999995e-05 2024-09-21T11:44:27.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.207072+0000", 2024-09-21T11:44:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4778999999999996e-05 2024-09-21T11:44:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.207155+0000", 2024-09-21T11:44:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3567999999999997e-05 2024-09-21T11:44:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.209349+0000", 2024-09-21T11:44:27.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021931419999999999 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211318+0000", 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019697680000000002 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.295819+0000", 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.084500668000000001 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.295854+0000", 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4601000000000003e-05 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.295884+0000", 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0666000000000001e-05 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.279 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:99033 7.5 7:a309294f:::benchmark_data_smithi043_70646_object99032:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:26.337954+0000", 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 120.91066407, 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10807878899999999, 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99033 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337954+0000", 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337954+0000", 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337955+0000", 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337993+0000", 2024-09-21T11:44:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7715000000000001e-05 2024-09-21T11:44:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338011+0000", 2024-09-21T11:44:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7676000000000002e-05 2024-09-21T11:44:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338020+0000", 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9209999999999998e-06 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338081+0000", 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0928999999999999e-05 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338172+0000", 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0969000000000003e-05 2024-09-21T11:44:27.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338260+0000", 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8417000000000004e-05 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.340226+0000", 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001966069 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.433908+0000", 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093681581999999999 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.445962+0000", 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012053895 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.446001+0000", 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9795000000000001e-05 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.446032+0000", 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0970999999999999e-05 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:100915 7.5 7:a66f7709:::benchmark_data_smithi043_70646_object100914:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:28.566552+0000", 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 118.682066152, 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.076657827999999997, 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.286 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100915 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566552+0000", 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566552+0000", 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566554+0000", 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8329999999999998e-06 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566593+0000", 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8578999999999998e-05 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566612+0000", 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9029999999999999e-05 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566622+0000", 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.009e-05 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566690+0000", 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8045000000000003e-05 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566822+0000", 2024-09-21T11:44:27.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013180700000000001 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566973+0000", 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015138099999999999 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.569472+0000", 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002498564 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.570043+0000", 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057143099999999996 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.643130+0000", 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.073087066000000006 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.643175+0000", 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4249999999999998e-05 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.643209+0000", 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4752000000000002e-05 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:100922 7.5 7:a128daba:::benchmark_data_smithi043_70646_object100921:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:28.573462+0000", 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 118.675155721, 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.074314127999999993, 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.292 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100922 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573462+0000", 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573462+0000", 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573464+0000", 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0269999999999998e-06 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573512+0000", 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7676000000000002e-05 2024-09-21T11:44:27.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573525+0000", 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3594000000000001e-05 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573534+0000", 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3110000000000006e-06 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573601+0000", 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7235000000000003e-05 2024-09-21T11:44:27.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573732+0000", 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000130872 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573835+0000", 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102905 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.576309+0000", 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024744039999999999 2024-09-21T11:44:27.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.576330+0000", 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0550000000000001e-05 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.647672+0000", 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071342384999999994 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.647729+0000", 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7358000000000002e-05 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.647776+0000", 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6811000000000003e-05 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123054 7.5 7:a4098129:::benchmark_data_smithi043_70646_object5646:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.880861+0000", 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 91.367756721999996, 2024-09-21T11:44:27.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.098889820000000003, 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123054 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880861+0000", 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880861+0000", 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880863+0000", 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6300000000000001e-06 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880867+0000", 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3239999999999996e-06 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880875+0000", 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0639999999999994e-06 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880882+0000", 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2139999999999999e-06 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880946+0000", 2024-09-21T11:44:27.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3366000000000003e-05 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.881054+0000", 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010853000000000001 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.881162+0000", 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010778 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.885528+0000", 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043657339999999996 2024-09-21T11:44:27.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889186+0000", 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036579680000000002 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979646+0000", 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090460503999999997 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979705+0000", 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9070999999999997e-05 2024-09-21T11:44:27.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979751+0000", 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5634999999999999e-05 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123057 7.5 7:a56ed89a:::benchmark_data_smithi043_70646_object5649:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.882509+0000", 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 91.366108877000002, 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.097395657999999996, 2024-09-21T11:44:27.305 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123057 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882509+0000", 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882509+0000", 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882510+0000", 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5099999999999998e-07 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882513+0000", 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7680000000000002e-06 2024-09-21T11:44:27.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882522+0000", 2024-09-21T11:44:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9549999999999998e-06 2024-09-21T11:44:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882529+0000", 2024-09-21T11:44:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2079999999999997e-06 2024-09-21T11:44:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.883324+0000", 2024-09-21T11:44:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079580500000000001 2024-09-21T11:44:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.883429+0000", 2024-09-21T11:44:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104654 2024-09-21T11:44:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.883536+0000", 2024-09-21T11:44:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107166 2024-09-21T11:44:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.885724+0000", 2024-09-21T11:44:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021879960000000002 2024-09-21T11:44:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889619+0000", 2024-09-21T11:44:27.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0038952510000000002 2024-09-21T11:44:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979813+0000", 2024-09-21T11:44:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090193465 2024-09-21T11:44:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979862+0000", 2024-09-21T11:44:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9344000000000001e-05 2024-09-21T11:44:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979904+0000", 2024-09-21T11:44:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2194999999999998e-05 2024-09-21T11:44:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123064 7.5 7:ae62c878:::benchmark_data_smithi043_70646_object5656:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:44:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.889753+0000", 2024-09-21T11:44:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 91.358864694000005, 2024-09-21T11:44:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090290819999999994, 2024-09-21T11:44:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.314 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123064 2024-09-21T11:44:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889753+0000", 2024-09-21T11:44:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889753+0000", 2024-09-21T11:44:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889755+0000", 2024-09-21T11:44:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.968e-06 2024-09-21T11:44:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889760+0000", 2024-09-21T11:44:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.113e-06 2024-09-21T11:44:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889771+0000", 2024-09-21T11:44:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1070000000000001e-05 2024-09-21T11:44:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889778+0000", 2024-09-21T11:44:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0979999999999996e-06 2024-09-21T11:44:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889826+0000", 2024-09-21T11:44:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8152999999999998e-05 2024-09-21T11:44:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889903+0000", 2024-09-21T11:44:27.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6737999999999996e-05 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889980+0000", 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7373000000000006e-05 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.891776+0000", 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001795984 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.892127+0000", 2024-09-21T11:44:27.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035003399999999999 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979962+0000", 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.087835977999999995 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.980007+0000", 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4619000000000001e-05 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.980044+0000", 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6692e-05 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126210 7.5 7:a6687ff4:::benchmark_data_smithi043_70646_object8802:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.192178+0000", 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 88.056439804999997, 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.099158476999999995, 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.322 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126210 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192178+0000", 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192178+0000", 2024-09-21T11:44:27.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192180+0000", 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.807e-06 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192185+0000", 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1250000000000001e-06 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192197+0000", 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2527000000000001e-05 2024-09-21T11:44:27.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192205+0000", 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0709999999999995e-06 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192287+0000", 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1976000000000003e-05 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192415+0000", 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127317 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192525+0000", 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110297 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194639+0000", 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021140999999999998 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200841+0000", 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062023790000000001 2024-09-21T11:44:27.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291238+0000", 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090396746 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291292+0000", 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3792000000000003e-05 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291336+0000", 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4339999999999999e-05 2024-09-21T11:44:27.327 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126211 7.5 7:a0cb9a25:::benchmark_data_smithi043_70646_object8803:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.194824+0000", 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 88.053793835999997, 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096639920000000004, 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.328 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126211 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194824+0000", 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194824+0000", 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194825+0000", 2024-09-21T11:44:27.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.609e-06 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194829+0000", 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9539999999999998e-06 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194838+0000", 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4400000000000005e-06 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194844+0000", 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2040000000000002e-06 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194897+0000", 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3405000000000001e-05 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.195017+0000", 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011910599999999999 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.195137+0000", 2024-09-21T11:44:27.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012094 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200887+0000", 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057499279999999996 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200960+0000", 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2243999999999995e-05 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291381+0000", 2024-09-21T11:44:27.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090421027000000001 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291429+0000", 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8605e-05 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291464+0000", 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4458000000000001e-05 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.333 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126214 7.5 7:a1d8d5c0:::benchmark_data_smithi043_70646_object8806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.194864+0000", 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 88.053753854999997, 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096713571999999998, 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126214 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194864+0000", 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194864+0000", 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194864+0000", 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0100000000000005e-07 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194867+0000", 2024-09-21T11:44:27.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3700000000000002e-06 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194874+0000", 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2779999999999996e-06 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194879+0000", 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4720000000000003e-06 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197151+0000", 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022713999999999998 2024-09-21T11:44:27.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197270+0000", 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000119391 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197386+0000", 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011619400000000001 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200949+0000", 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035628579999999999 2024-09-21T11:44:27.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200962+0000", 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3125000000000001e-05 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291499+0000", 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090537054000000006 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291545+0000", 2024-09-21T11:44:27.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5392000000000002e-05 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291577+0000", 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2536999999999997e-05 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126216 7.5 7:a039bdc0:::benchmark_data_smithi043_70646_object8808:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.196576+0000", 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 88.052041990999996, 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.095402531999999998, 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.339 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126216 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196576+0000", 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196576+0000", 2024-09-21T11:44:27.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196577+0000", 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.71e-07 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196581+0000", 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6360000000000002e-06 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196595+0000", 2024-09-21T11:44:27.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3653e-05 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196602+0000", 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0439999999999998e-06 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.199091+0000", 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024891990000000001 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.199198+0000", 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107163 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.199296+0000", 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8194000000000002e-05 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200996+0000", 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017000190000000001 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.201042+0000", 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5439000000000003e-05 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291891+0000", 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090849471000000001 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291941+0000", 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0096000000000003e-05 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291978+0000", 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6746999999999999e-05 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:143063 7.5 7:ae79481b:::benchmark_data_smithi043_70646_object25655:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:18.044346+0000", 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.204271820000002, 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071762571999999997, 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.345 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 143063 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044346+0000", 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044346+0000", 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044347+0000", 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1069999999999999e-06 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044351+0000", 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2370000000000003e-06 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044362+0000", 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1155000000000001e-05 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044371+0000", 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6209999999999993e-06 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044415+0000", 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4075000000000001e-05 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044522+0000", 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010684299999999999 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044632+0000", 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109656 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.048658+0000", 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0040261689999999996 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.062629+0000", 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013971167 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.116031+0000", 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053402377000000001 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.116066+0000", 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5012000000000003e-05 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.116108+0000", 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2153000000000001e-05 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.350 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:143064 7.5 7:a95993a2:::benchmark_data_smithi043_70646_object25656:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:18.044388+0000", 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.204229429999998, 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071810562999999994, 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 143064 2024-09-21T11:44:27.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044388+0000", 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044388+0000", 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044389+0000", 2024-09-21T11:44:27.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7000000000000005e-07 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044392+0000", 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.813e-06 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044398+0000", 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0630000000000002e-06 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.044404+0000", 2024-09-21T11:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0310000000000004e-06 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046778+0000", 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023742329999999999 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046857+0000", 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8768000000000002e-05 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046940+0000", 2024-09-21T11:44:27.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2908e-05 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.053703+0000", 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0067633329999999998 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.062732+0000", 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0090292770000000005 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.116142+0000", 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.053409447999999998 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.116174+0000", 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2684000000000001e-05 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.116199+0000", 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4434999999999999e-05 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.356 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:143066 7.5 7:af12805e:::benchmark_data_smithi043_70646_object25658:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:18.046672+0000", 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.201945907999999, 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.073440914999999996, 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 143066 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046672+0000", 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046672+0000", 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046673+0000", 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7000000000000003e-07 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046677+0000", 2024-09-21T11:44:27.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5519999999999998e-06 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046689+0000", 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1496e-05 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046696+0000", 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0740000000000004e-06 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.048689+0000", 2024-09-21T11:44:27.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019935370000000001 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.048794+0000", 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104295 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.049018+0000", 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000224699 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.053787+0000", 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004768881 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.062786+0000", 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089982719999999999 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.120028+0000", 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057242807 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.120077+0000", 2024-09-21T11:44:27.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8798999999999998e-05 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.120113+0000", 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5533e-05 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:185206 7.5 7:a5331398:::benchmark_data_smithi043_70646_object67798:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:57.859097+0000", 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 29.389521125999998, 2024-09-21T11:44:27.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.082023416000000002, 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 185206 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859097+0000", 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859097+0000", 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859098+0000", 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2219999999999999e-06 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859102+0000", 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5979999999999999e-06 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859113+0000", 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0317e-05 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859120+0000", 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4340000000000003e-06 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859165+0000", 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5327000000000002e-05 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859239+0000", 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3697999999999998e-05 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859307+0000", 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7611e-05 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.861065+0000", 2024-09-21T11:44:27.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001758461 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.940968+0000", 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.079902619999999994 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.941021+0000", 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3060999999999999e-05 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.941072+0000", 2024-09-21T11:44:27.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1301999999999998e-05 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.941120+0000", 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7765000000000001e-05 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:219297 7.5 7:a035a6a1:::benchmark_data_smithi043_70646_object101889:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:25.434259+0000", 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.814358503, 2024-09-21T11:44:27.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.095706954999999996, 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 219297 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434259+0000", 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434259+0000", 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434261+0000", 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.514e-06 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434264+0000", 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434272+0000", 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5429999999999996e-06 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434277+0000", 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.344e-06 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434363+0000", 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6327000000000002e-05 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434438+0000", 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4917000000000001e-05 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434508+0000", 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9884999999999999e-05 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.436738+0000", 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022296730000000002 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.437766+0000", 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010284580000000001 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.529759+0000", 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.091992554000000004 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.529890+0000", 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013153899999999999 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.529966+0000", 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5667999999999994e-05 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:219307 7.5 7:aa109419:::benchmark_data_smithi043_70646_object101899:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:25.442393+0000", 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.806224598, 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.087742547000000004, 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.374 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 219307 2024-09-21T11:44:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442393+0000", 2024-09-21T11:44:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442393+0000", 2024-09-21T11:44:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442395+0000", 2024-09-21T11:44:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.615e-06 2024-09-21T11:44:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442401+0000", 2024-09-21T11:44:27.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6760000000000001e-06 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442413+0000", 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1758e-05 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442419+0000", 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0390000000000001e-06 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442465+0000", 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5738000000000002e-05 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442543+0000", 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7843000000000007e-05 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442613+0000", 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0632999999999999e-05 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.444495+0000", 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018818229999999999 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.445347+0000", 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085143100000000004 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.530026+0000", 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.084679373000000002 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.530092+0000", 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6237999999999998e-05 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.530136+0000", 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3380000000000001e-05 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220187 7.5 7:af65a55d:::benchmark_data_smithi043_70646_object102779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.200942+0000", 2024-09-21T11:44:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.0476756899999999, 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.095579969000000001, 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220187 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.382 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200942+0000", 2024-09-21T11:44:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200942+0000", 2024-09-21T11:44:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.383 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200944+0000", 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5349999999999999e-06 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200949+0000", 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.113e-06 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200963+0000", 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4837999999999999e-05 2024-09-21T11:44:27.384 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200971+0000", 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7249999999999997e-06 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.202084+0000", 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011125530000000001 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.202191+0000", 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107434 2024-09-21T11:44:27.385 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.202297+0000", 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106078 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204798+0000", 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025004239999999998 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.206727+0000", 2024-09-21T11:44:27.386 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001928864 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.296449+0000", 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.089722510000000005 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.296488+0000", 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9178000000000003e-05 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.387 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.296522+0000", 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3717000000000003e-05 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220189 7.5 7:acec303d:::benchmark_data_smithi043_70646_object102781:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.204953+0000", 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.0436644349999999, 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.102016905, 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:44:27.388 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220189 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204953+0000", 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204953+0000", 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204954+0000", 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204959+0000", 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.296e-06 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204968+0000", 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7329999999999998e-06 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204974+0000", 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.276e-06 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205021+0000", 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7154000000000003e-05 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205112+0000", 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1334000000000003e-05 2024-09-21T11:44:27.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205186+0000", 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4011999999999995e-05 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.206522+0000", 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001336292 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.206787+0000", 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026466100000000002 2024-09-21T11:44:27.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306842+0000", 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.100054927 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306920+0000", 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8430999999999995e-05 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306970+0000", 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9784000000000003e-05 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.393 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:44:27.394 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:44:27.394 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:44:27.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:27.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:27.429 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:27.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:27 smithi175 ceph-mon[32613]: pgmap v199: 148 pgs: 148 active+clean; 1.2 GiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 886 op/s 2024-09-21T11:44:27.530 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:27.552 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:27.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:27.553 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:27.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:27.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:27.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:27.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:27 smithi043 ceph-mon[36454]: pgmap v199: 148 pgs: 148 active+clean; 1.2 GiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 886 op/s 2024-09-21T11:44:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:27 smithi043 ceph-mon[29710]: pgmap v199: 148 pgs: 148 active+clean; 1.2 GiB data, 4.6 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 886 op/s 2024-09-21T11:44:27.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:27.821 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:27.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:27.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:27.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:27.959 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:27.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:27.960 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:28.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:28.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:28.093 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:28.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:28.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:44:28.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:28.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:28.273 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:28.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:28.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:28.450 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:28.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:28.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:28.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:28.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:28.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:28.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:28.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:28.723 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:28.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:28.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:28.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:28.864 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:28.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:29.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:29.005 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:29.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:29.132 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:29.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:29.133 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:29.138 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:29.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:29.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:29.263 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:29.263 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:29.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:29.408 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:29.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:29.409 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:29.510 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:29.540 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:29.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:29.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:29.618 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:29 smithi043 ceph-mon[36454]: pgmap v200: 148 pgs: 148 active+clean; 935 MiB data, 3.8 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-21T11:44:29.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:29 smithi043 ceph-mon[29710]: pgmap v200: 148 pgs: 148 active+clean; 935 MiB data, 3.8 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-21T11:44:29.642 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:29.670 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:29.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:29.671 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:29.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:29 smithi175 ceph-mon[32613]: pgmap v200: 148 pgs: 148 active+clean; 935 MiB data, 3.8 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-21T11:44:29.772 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:29.785 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:44:29.786 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:44:29.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.786 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:224234 7.f 7:f3cc7af9:::benchmark_data_smithi043_70646_object106826:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:44:29.786 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:29.766178+0000", 2024-09-21T11:44:29.786 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.015547451, 2024-09-21T11:44:29.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015576344000000001, 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 224234 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.766178+0000", 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.766178+0000", 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.766180+0000", 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6109999999999999e-06 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.766185+0000", 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3979999999999998e-06 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.766195+0000", 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.859e-06 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.766202+0000", 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.905e-06 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.771469+0000", 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052674389999999996 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.771582+0000", 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011291299999999999 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.771703+0000", 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120872 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.779868+0000", 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0081653179999999995 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:224241 7.f 7.aa801d8f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:29.771642+0000", 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010082939000000001, 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010189468, 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:44:29.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 224241 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.771642+0000", 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.771642+0000", 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.771643+0000", 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5600000000000005e-07 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.771647+0000", 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6799999999999999e-06 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.771655+0000", 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7030000000000006e-06 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.771661+0000", 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3890000000000002e-06 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:224234 7.f e151/82 ondisk, result = 0)", 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:29.775507+0000", 2024-09-21T11:44:29.794 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0062182390000000004, 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063742549999999997, 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.775507+0000", 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.775507+0000", 2024-09-21T11:44:29.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.775508+0000", 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1500000000000005e-07 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.775511+0000", 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9009999999999998e-06 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.775520+0000", 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6840000000000008e-06 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.775526+0000", 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6339999999999998e-06 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.779857+0000", 2024-09-21T11:44:29.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043310070000000004 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.779863+0000", 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3690000000000003e-06 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:224246 7.f 7.43b50cbf (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:29.777552+0000", 2024-09-21T11:44:29.798 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0041730730000000002, 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043820300000000003, 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 224246 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.777552+0000", 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.777552+0000", 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.777554+0000", 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.717e-06 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.777560+0000", 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8710000000000002e-06 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.777652+0000", 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1800999999999997e-05 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.777661+0000", 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7580000000000004e-06 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:224247 7.7 e151/82 7:eefbf395:::benchmark_data_smithi043_70646_object106839:head v 151'14270, mlcod=151'14267)", 2024-09-21T11:44:29.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:29.778109+0000", 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0036159629999999998, 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038806209999999999, 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778109+0000", 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:29.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778109+0000", 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778110+0000", 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6000000000000002e-07 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778115+0000", 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8760000000000004e-06 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:29.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778129+0000", 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3835000000000001e-05 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778135+0000", 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0739999999999998e-06 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.779918+0000", 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017826960000000001 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.779964+0000", 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6322999999999997e-05 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:224234 7.f e151/82)", 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:29.778196+0000", 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.003529674, 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038450110000000002, 2024-09-21T11:44:29.805 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778196+0000", 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778196+0000", 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778196+0000", 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8999999999999996e-07 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778200+0000", 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.405e-06 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:29.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778207+0000", 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0570000000000004e-06 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778213+0000", 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8279999999999999e-06 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:224250 7.9 7:9dd626b0:::benchmark_data_smithi043_70646_object106842:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:29.778407+0000", 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0033186209999999999, 2024-09-21T11:44:29.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036753300000000001, 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 224250 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778407+0000", 2024-09-21T11:44:29.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778407+0000", 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778408+0000", 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026e-06 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778412+0000", 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3819999999999997e-06 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778423+0000", 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1280999999999999e-05 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778430+0000", 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0180000000000004e-06 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778530+0000", 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9601999999999999e-05 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778651+0000", 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012082399999999999 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,7", 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.778866+0000", 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021490800000000001 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.782004+0000", 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00313847 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.14805.0:224250 7.9 e151/130)", 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:29.782057+0000", 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9996686, 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1355999999999997e-05, 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:44:29.813 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.782057+0000", 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.782057+0000", 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.782058+0000", 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.142e-06 2024-09-21T11:44:29.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.782063+0000", 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8380000000000001e-06 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.782072+0000", 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8999999999999995e-06 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.782079+0000", 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1600000000000001e-06 2024-09-21T11:44:29.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:224232 7.7 e151/82 7:e7f78872:::benchmark_data_smithi043_70646_object106824:head v 151'14269, mlcod=151'14267)", 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:29.765147+0000", 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.016578432000000001, 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017054687999999998, 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.765147+0000", 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.765147+0000", 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.765148+0000", 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.283e-06 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.765153+0000", 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.853e-06 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.765167+0000", 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3567000000000001e-05 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.765174+0000", 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.25e-06 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.770021+0000", 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0048475360000000004 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:29.770145+0000", 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012386799999999999 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 9 2024-09-21T11:44:29.819 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:44:29.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:29.821 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:29.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:29.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:29.951 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:29.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:29.952 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:30.053 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:30.084 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:30.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:30.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:30.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:30.227 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:30.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:30.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:30.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:44:30.313 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:44:30.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:30.357 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:30.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:30.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:30.406 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:44:30.406 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:44:30.406 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T11:44:30.406 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-09-21T11:44:30.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 3 1 2024-09-21T11:44:30.449 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:30 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/163032666' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:30.449 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:30 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/163032666' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:30.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:30.515 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:30.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:30.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:30.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:44:30.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:30.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:30.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:30.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:30 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/163032666' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:30.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:30.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:30.844 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:30.944 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:30.972 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:30.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:30.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:31.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:31.117 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:31.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:31.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:31.219 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:31.245 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:44:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:44:31.256 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:31.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:31.257 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:31.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:31.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:31.399 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:31.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[29710]: pgmap v201: 148 pgs: 148 active+clean; 935 MiB data, 3.8 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/163032666' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[29710]: osdmap e152: 8 total, 8 up, 7 in 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[36454]: pgmap v201: 148 pgs: 148 active+clean; 935 MiB data, 3.8 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/163032666' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[36454]: osdmap e152: 8 total, 8 up, 7 in 2024-09-21T11:44:31.417 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:31.418 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:31.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:31.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:31 smithi175 ceph-mon[32613]: pgmap v201: 148 pgs: 148 active+clean; 935 MiB data, 3.8 GiB used, 622 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s 2024-09-21T11:44:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:31 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/163032666' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:31 smithi175 ceph-mon[32613]: osdmap e152: 8 total, 8 up, 7 in 2024-09-21T11:44:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:31.543 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:31.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:31.576 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:31.635 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:31.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:31.705 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:31.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:31.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:31.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:31.851 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:31.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:31.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:31.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:31.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:31.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:32.099 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:32.128 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:32.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:32.129 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:32.230 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:32.262 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:32.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:32.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:32.363 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:32.382 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:44:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:44:32.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:32.385 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:32.386 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:32.415 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:44:32.415 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:44:32.456 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:44:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:44:32.486 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:32.496 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:44:32.496 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:44:32.500 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:32.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:32 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:44:32.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:32 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:44:32.501 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:32 smithi043 ceph-mon[29710]: osdmap e153: 8 total, 8 up, 7 in 2024-09-21T11:44:32.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:32.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:32 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:44:32.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:32 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:44:32.501 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:32 smithi043 ceph-mon[36454]: osdmap e153: 8 total, 8 up, 7 in 2024-09-21T11:44:32.513 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:32.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:32.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:32.533 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:44:32.533 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:44:32.567 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:44:32.567 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:44:32.605 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:44:32.605 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:44:32.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:32.643 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:44:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:44:32.659 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:32.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:32.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:32.680 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:44:32.681 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:44:32.760 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:32 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:44:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:32 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:44:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:32 smithi175 ceph-mon[32613]: osdmap e153: 8 total, 8 up, 7 in 2024-09-21T11:44:32.789 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:32.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:32.790 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:32.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:32.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:32.930 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:33.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:33.070 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:33.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:33.071 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:33.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:33.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:33.221 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:33.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:33.370 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:33.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:33.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:33.472 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:33.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:44:33.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:33 smithi043 ceph-mon[29710]: pgmap v204: 148 pgs: 148 active+clean; 570 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s 2024-09-21T11:44:33.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:33 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2628787340' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:33.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:33 smithi043 ceph-mon[36454]: pgmap v204: 148 pgs: 148 active+clean; 570 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s 2024-09-21T11:44:33.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:33 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2628787340' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:33.500 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:33.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:33.501 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:33.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:33.625 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:33.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:33.626 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:33.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:33.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:33 smithi175 ceph-mon[32613]: pgmap v204: 148 pgs: 148 active+clean; 570 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s 2024-09-21T11:44:33.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:33 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2628787340' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:33.782 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:33.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:33.783 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:33.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:33.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:33.924 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:34.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:34.051 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:34.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:34.052 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:34.153 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:34.185 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:34.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:34.186 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:34.287 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:34.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:34.317 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:34.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:34.356 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-21T11:44:34.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:34.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/812400092' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:44:34.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:34 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:44:34.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/812400092' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:44:34.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:34 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:44:34.456 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:34.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:34.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:34.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:44:34.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:34.584 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:34.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:34.586 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:34.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:34.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:34.751 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:34.751 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:34.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/812400092' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:44:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:34 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:44:34.852 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:34.883 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:34.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:34.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:34.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:35.024 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:35.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:35.025 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:35.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:35.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:35.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:35.219 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:35.262 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:35.288 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:35.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:35.289 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:35.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:35.444 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:35.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:35.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:44:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:44:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:44:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:35 smithi043 ceph-mon[36454]: pgmap v205: 148 pgs: 148 active+clean; 570 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-21T11:44:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:35 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:44:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:35 smithi043 ceph-mon[36454]: osdmap e154: 8 total, 8 up, 7 in 2024-09-21T11:44:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:35 smithi043 ceph-mon[29710]: pgmap v205: 148 pgs: 148 active+clean; 570 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-21T11:44:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:35 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:44:35.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:35 smithi043 ceph-mon[29710]: osdmap e154: 8 total, 8 up, 7 in 2024-09-21T11:44:35.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:35.576 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:35.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:35.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:35.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:35.706 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:35.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:35.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:35.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:35 smithi175 ceph-mon[32613]: pgmap v205: 148 pgs: 148 active+clean; 570 MiB data, 2.8 GiB used, 623 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 1.32k op/s 2024-09-21T11:44:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:35 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:44:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:35 smithi175 ceph-mon[32613]: osdmap e154: 8 total, 8 up, 7 in 2024-09-21T11:44:35.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:35.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:44:35.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:35.856 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:35.856 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:35.891 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:35.957 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:35.982 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:35.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:35.983 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:36.083 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:36.122 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:36.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:36.123 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:36.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:36.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:36.286 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:36.359 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:44:36.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:36.414 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:36.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:36.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:36.515 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:36.542 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:36.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:36.543 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:36.588 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:44:36.588 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:44:36.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:36.679 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:36.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:36.680 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:36.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:36 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/176891351' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:44:36.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:36 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/176891351' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:44:36.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:36 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/176891351' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:44:36.781 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:36.823 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:36.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:36.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:36.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:36.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:36.966 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:36.966 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:37.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:37.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:37.098 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:37.098 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:37.199 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:37.227 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:37.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:37.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:37.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:37.358 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:37.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:37.359 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:37.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:37.487 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:37.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:37.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[29710]: pgmap v207: 148 pgs: 148 active+clean; 472 MiB data, 2.4 GiB used, 623 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:44:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/176891351' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:44:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[29710]: osdmap e155: 8 total, 8 up, 7 in 2024-09-21T11:44:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[29710]: 3.10 scrub starts 2024-09-21T11:44:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[29710]: 3.10 scrub ok 2024-09-21T11:44:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[36454]: pgmap v207: 148 pgs: 148 active+clean; 472 MiB data, 2.4 GiB used, 623 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:44:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/176891351' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:44:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[36454]: osdmap e155: 8 total, 8 up, 7 in 2024-09-21T11:44:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[36454]: 3.10 scrub starts 2024-09-21T11:44:37.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:37 smithi043 ceph-mon[36454]: 3.10 scrub ok 2024-09-21T11:44:37.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:37.624 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:37.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:37.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:37.725 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:37.756 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:37.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:37.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:37.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:37 smithi175 ceph-mon[32613]: pgmap v207: 148 pgs: 148 active+clean; 472 MiB data, 2.4 GiB used, 623 GiB / 626 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:44:37.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:37 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:37 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/176891351' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:44:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:37 smithi175 ceph-mon[32613]: osdmap e155: 8 total, 8 up, 7 in 2024-09-21T11:44:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:37 smithi175 ceph-mon[32613]: 3.10 scrub starts 2024-09-21T11:44:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:37 smithi175 ceph-mon[32613]: 3.10 scrub ok 2024-09-21T11:44:37.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:37.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:37.891 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:37.892 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:37.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:44:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:37.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:38.051 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:38.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:38.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:38.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:38.196 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:38.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:38.197 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:38.261 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Removed 117405 objects 2024-09-21T11:44:38.261 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Clean up completed and total clean up time :108.587 2024-09-21T11:44:38.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:38.319 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:38.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:38.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:38.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:38.431 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:38 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T11:44:38.431 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:38 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T11:44:38.431 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:38 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T11:44:38.431 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:38 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T11:44:38.442 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:38.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:38.443 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:38.538 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:44:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:44:38.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:38.552 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:44:38.552 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:44:38.568 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:38.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:38.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:38.571 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:44:38.572 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:44:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:38.589 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:44:38.589 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:44:38.612 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:44:38.613 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:44:38.629 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:44:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:44:38.647 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:44:38.647 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:44:38.664 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:44:38.664 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:44:38.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:38.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:38.695 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:38.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:38 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T11:44:38.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:38 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T11:44:38.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:38.817 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:38.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:38.818 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:38.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:38.941 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:38.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:38.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:39.043 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:39.065 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:39.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:39.066 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:39.167 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:39.189 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:39.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:39.190 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:39.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:44:39.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:39.313 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:39.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:39.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:39.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:39.437 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:39.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:39.438 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:39.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[36454]: pgmap v209: 148 pgs: 148 active+clean; 179 MiB data, 1.7 GiB used, 624 GiB / 626 GiB avail; 820 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-21T11:44:39.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[36454]: 4.a scrub starts 2024-09-21T11:44:39.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[36454]: 4.a scrub ok 2024-09-21T11:44:39.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2844584563' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:39.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[36454]: 5.b scrub starts 2024-09-21T11:44:39.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[36454]: 5.b scrub ok 2024-09-21T11:44:39.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[36454]: 6.1e scrub starts 2024-09-21T11:44:39.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[36454]: 6.1e scrub ok 2024-09-21T11:44:39.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[29710]: pgmap v209: 148 pgs: 148 active+clean; 179 MiB data, 1.7 GiB used, 624 GiB / 626 GiB avail; 820 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-21T11:44:39.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[29710]: 4.a scrub starts 2024-09-21T11:44:39.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[29710]: 4.a scrub ok 2024-09-21T11:44:39.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2844584563' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:39.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[29710]: 5.b scrub starts 2024-09-21T11:44:39.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[29710]: 5.b scrub ok 2024-09-21T11:44:39.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[29710]: 6.1e scrub starts 2024-09-21T11:44:39.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:39 smithi043 ceph-mon[29710]: 6.1e scrub ok 2024-09-21T11:44:39.539 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:39.561 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:39.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:39.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:39.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:39.684 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:39.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:39.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:39.752 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-21T11:44:39.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-21T11:44:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:39 smithi175 ceph-mon[32613]: pgmap v209: 148 pgs: 148 active+clean; 179 MiB data, 1.7 GiB used, 624 GiB / 626 GiB avail; 820 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-21T11:44:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:39 smithi175 ceph-mon[32613]: 4.a scrub starts 2024-09-21T11:44:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:39 smithi175 ceph-mon[32613]: 4.a scrub ok 2024-09-21T11:44:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2844584563' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:39 smithi175 ceph-mon[32613]: 5.b scrub starts 2024-09-21T11:44:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:39 smithi175 ceph-mon[32613]: 5.b scrub ok 2024-09-21T11:44:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:39 smithi175 ceph-mon[32613]: 6.1e scrub starts 2024-09-21T11:44:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:39 smithi175 ceph-mon[32613]: 6.1e scrub ok 2024-09-21T11:44:39.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:39.820 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:39.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:39.821 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:39.839 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:44:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:44:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-21T11:44:39.839 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:44:39.922 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:39.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:39.949 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:39.949 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:40.050 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:40.075 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:40.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:40.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:40.101 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:40.177 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:40.202 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:40.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:40.203 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:40.241 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:44:40.241 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:44:40.241 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T11:44:40.241 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T11:44:40.241 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T11:44:40.241 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 config set filestore_inject_stall 3 2024-09-21T11:44:40.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:40.327 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:40.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:40.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:40.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:40.452 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:40.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:40.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:40.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:40 smithi043 ceph-mon[29710]: 3.12 scrub starts 2024-09-21T11:44:40.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:40 smithi043 ceph-mon[29710]: 3.12 scrub ok 2024-09-21T11:44:40.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:40 smithi043 ceph-mon[36454]: 3.12 scrub starts 2024-09-21T11:44:40.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:40 smithi043 ceph-mon[36454]: 3.12 scrub ok 2024-09-21T11:44:40.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:40.577 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:40.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:40.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:40.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:44:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:40.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:40.701 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:40.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:40.702 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:40 smithi175 ceph-mon[32613]: 3.12 scrub starts 2024-09-21T11:44:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:40 smithi175 ceph-mon[32613]: 3.12 scrub ok 2024-09-21T11:44:40.803 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:40.826 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:40.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:40.827 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:40.927 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:40.953 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:40.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:40.954 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:41.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:41.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:41.078 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:41.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:41.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:41.205 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:41.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:41.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:41.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:44:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:44:41.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:41.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:41.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:41.356 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:41.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:41.457 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:41.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:41.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:41.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:44:41.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:41 smithi043 ceph-mon[36454]: pgmap v210: 148 pgs: 148 active+clean; 179 MiB data, 1.7 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 781 op/s 2024-09-21T11:44:41.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:41 smithi043 ceph-mon[36454]: 3.a scrub starts 2024-09-21T11:44:41.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:41 smithi043 ceph-mon[36454]: 3.a scrub ok 2024-09-21T11:44:41.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:41 smithi043 ceph-mon[29710]: pgmap v210: 148 pgs: 148 active+clean; 179 MiB data, 1.7 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 781 op/s 2024-09-21T11:44:41.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:41 smithi043 ceph-mon[29710]: 3.a scrub starts 2024-09-21T11:44:41.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:41 smithi043 ceph-mon[29710]: 3.a scrub ok 2024-09-21T11:44:41.556 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:44:41.557 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T11:44:41.557 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:44:41.559 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:41.584 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:41.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:41.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:41.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:41.708 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:41.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:41.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:41 smithi175 ceph-mon[32613]: pgmap v210: 148 pgs: 148 active+clean; 179 MiB data, 1.7 GiB used, 624 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 781 op/s 2024-09-21T11:44:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:41 smithi175 ceph-mon[32613]: 3.a scrub starts 2024-09-21T11:44:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:41 smithi175 ceph-mon[32613]: 3.a scrub ok 2024-09-21T11:44:41.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:41.816 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:41.835 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:41.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:41.836 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:41.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:41.959 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:41.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:41.960 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:42.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:42.086 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:42.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:42.087 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:42.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:42.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:42.211 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:42.212 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:42.312 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:42.338 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:42.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:42.339 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:44:42.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:42.461 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:42.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:42.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:42.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:42.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[36454]: 5.5 scrub starts 2024-09-21T11:44:42.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[36454]: 5.5 scrub ok 2024-09-21T11:44:42.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3649964314' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:42.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[36454]: 4.c scrub starts 2024-09-21T11:44:42.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[36454]: 4.c scrub ok 2024-09-21T11:44:42.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:42.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[29710]: 5.5 scrub starts 2024-09-21T11:44:42.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[29710]: 5.5 scrub ok 2024-09-21T11:44:42.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3649964314' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[29710]: 4.c scrub starts 2024-09-21T11:44:42.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:42 smithi043 ceph-mon[29710]: 4.c scrub ok 2024-09-21T11:44:42.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:42.585 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:42.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:42.586 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:42.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:42.708 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:42.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:42.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:42.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:42 smithi175 ceph-mon[32613]: 5.5 scrub starts 2024-09-21T11:44:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:42 smithi175 ceph-mon[32613]: 5.5 scrub ok 2024-09-21T11:44:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3649964314' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:42 smithi175 ceph-mon[32613]: 4.c scrub starts 2024-09-21T11:44:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:42 smithi175 ceph-mon[32613]: 4.c scrub ok 2024-09-21T11:44:42.809 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:42.835 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:42.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:42.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:42.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:42.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:42.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:43.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:43.088 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:43.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:43.089 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:43.189 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:43.215 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:43.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:43.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:43.317 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:43.342 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:43.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:43.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:43.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:43.470 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:43.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:43.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:43.489 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:43.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:43.597 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:43.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:43.598 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:43.699 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:43.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[36454]: pgmap v211: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:44:43.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[36454]: 4.1e scrub starts 2024-09-21T11:44:43.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[36454]: 4.1e scrub ok 2024-09-21T11:44:43.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[36454]: 6.19 scrub starts 2024-09-21T11:44:43.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[36454]: 6.19 scrub ok 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3649964314' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[36454]: osdmap e156: 8 total, 8 up, 7 in 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[29710]: pgmap v211: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[29710]: 4.1e scrub starts 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[29710]: 4.1e scrub ok 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[29710]: 6.19 scrub starts 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[29710]: 6.19 scrub ok 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3649964314' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:43.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:43 smithi043 ceph-mon[29710]: osdmap e156: 8 total, 8 up, 7 in 2024-09-21T11:44:43.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:43.724 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:43.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:43 smithi175 ceph-mon[32613]: pgmap v211: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:44:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:43 smithi175 ceph-mon[32613]: 4.1e scrub starts 2024-09-21T11:44:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:43 smithi175 ceph-mon[32613]: 4.1e scrub ok 2024-09-21T11:44:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:43 smithi175 ceph-mon[32613]: 6.19 scrub starts 2024-09-21T11:44:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:43 smithi175 ceph-mon[32613]: 6.19 scrub ok 2024-09-21T11:44:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:43 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:44:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:43 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:44:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:43 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3649964314' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:43 smithi175 ceph-mon[32613]: osdmap e156: 8 total, 8 up, 7 in 2024-09-21T11:44:43.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:43.847 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:43.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:43.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:43.943 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:44:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:44:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:44:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:44:43.944 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:44:43.949 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:43.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:43.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:43.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:44.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:44.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:44.103 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:44.103 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:44.203 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:44.229 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:44.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:44.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:44.330 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:44.354 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:44.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:44.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:44.383 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'unique_pool_0' removed 2024-09-21T11:44:44.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:44.479 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:44.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:44.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:44.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:44.602 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:44.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:44.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:44.642 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:44 smithi043 ceph-mon[29710]: 4.2 scrub starts 2024-09-21T11:44:44.642 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:44 smithi043 ceph-mon[29710]: 4.2 scrub ok 2024-09-21T11:44:44.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3549152740' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:44:44.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:44 smithi043 ceph-mon[29710]: 4.9 scrub starts 2024-09-21T11:44:44.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:44 smithi043 ceph-mon[29710]: 4.9 scrub ok 2024-09-21T11:44:44.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:44 smithi043 ceph-mon[36454]: 4.2 scrub starts 2024-09-21T11:44:44.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:44 smithi043 ceph-mon[36454]: 4.2 scrub ok 2024-09-21T11:44:44.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3549152740' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:44:44.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:44 smithi043 ceph-mon[36454]: 4.9 scrub starts 2024-09-21T11:44:44.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:44 smithi043 ceph-mon[36454]: 4.9 scrub ok 2024-09-21T11:44:44.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:44:44.704 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:44.725 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:44.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:44.726 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:44 smithi175 ceph-mon[32613]: 4.2 scrub starts 2024-09-21T11:44:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:44 smithi175 ceph-mon[32613]: 4.2 scrub ok 2024-09-21T11:44:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3549152740' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:44:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:44 smithi175 ceph-mon[32613]: 4.9 scrub starts 2024-09-21T11:44:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:44 smithi175 ceph-mon[32613]: 4.9 scrub ok 2024-09-21T11:44:44.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:44.849 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:44.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:44.850 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:44.951 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:44.972 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:44.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:44.973 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:45.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:45.097 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:45.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:45.098 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:45.102 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:44:45.102 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:44:45.124 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:44:45.124 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:44:45.142 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:44:45.142 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:44:45.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:44:45.160 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:44:45.160 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:44:45.179 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:44:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:44:45.198 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:44:45.198 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:44:45.198 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:45.216 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:44:45.217 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:44:45.219 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:45.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:45.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:45.234 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:44:45.234 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:44:45.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:45.343 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:45.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:45.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:45.348 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:44:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:44:45.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:45.466 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:45.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:45.467 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:45.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:45.590 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:45.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:45.591 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:45.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: pgmap v213: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 942 op/s 2024-09-21T11:44:45.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: 5.13 scrub starts 2024-09-21T11:44:45.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: 5.13 scrub ok 2024-09-21T11:44:45.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3549152740' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:44:45.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: osdmap e157: 8 total, 8 up, 7 in 2024-09-21T11:44:45.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:44:45.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:44:45.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: 3.4 scrub starts 2024-09-21T11:44:45.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: 3.4 scrub ok 2024-09-21T11:44:45.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: 4.13 scrub starts 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: 4.13 scrub ok 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1726020868' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: pgmap v213: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 942 op/s 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: 5.13 scrub starts 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: 5.13 scrub ok 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3549152740' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: osdmap e157: 8 total, 8 up, 7 in 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:44:45.692 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: 3.4 scrub starts 2024-09-21T11:44:45.693 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: 3.4 scrub ok 2024-09-21T11:44:45.693 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: 4.13 scrub starts 2024-09-21T11:44:45.693 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: 4.13 scrub ok 2024-09-21T11:44:45.693 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1726020868' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:45.693 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:45 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:44:45.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:45.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:45.715 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:45.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: pgmap v213: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 942 op/s 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: 5.13 scrub starts 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: 5.13 scrub ok 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3549152740' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: osdmap e157: 8 total, 8 up, 7 in 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: 3.4 scrub starts 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: 3.4 scrub ok 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: 4.13 scrub starts 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: 4.13 scrub ok 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1726020868' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:45 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:44:45.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:45.837 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:45.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:45.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:45.939 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:45.960 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:45.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:45.961 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:46.051 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-21T11:44:46.052 INFO:tasks.radosbench:Beginning radosbench... 2024-09-21T11:44:46.052 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-21T11:44:46.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool create unique_pool_1 16 2024-09-21T11:44:46.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:46.114 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:46.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:46.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:46.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:46.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:46.239 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:46.239 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:46.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:46.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:46.363 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:46.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:46.486 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:46.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:46.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:46.587 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:46.588 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:46.591 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:46.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:46.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:46.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: 3.19 scrub starts 2024-09-21T11:44:46.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: 3.19 scrub ok 2024-09-21T11:44:46.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:44:46.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:44:46.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: 3.1 scrub starts 2024-09-21T11:44:46.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: 3.1 scrub ok 2024-09-21T11:44:46.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: 5.12 scrub starts 2024-09-21T11:44:46.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: 5.12 scrub ok 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: osdmap e158: 8 total, 8 up, 7 in 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: 6.14 scrub starts 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: 6.14 scrub ok 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: 5.c scrub starts 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[29710]: 5.c scrub ok 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: 3.19 scrub starts 2024-09-21T11:44:46.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: 3.19 scrub ok 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: 3.1 scrub starts 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: 3.1 scrub ok 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: 5.12 scrub starts 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: 5.12 scrub ok 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: osdmap e158: 8 total, 8 up, 7 in 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: 6.14 scrub starts 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: 6.14 scrub ok 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: 5.c scrub starts 2024-09-21T11:44:46.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:46 smithi043 ceph-mon[36454]: 5.c scrub ok 2024-09-21T11:44:46.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:46.738 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:46.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:46.739 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: 3.19 scrub starts 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: 3.19 scrub ok 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: 3.1 scrub starts 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: 3.1 scrub ok 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: 5.12 scrub starts 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: 5.12 scrub ok 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: osdmap e158: 8 total, 8 up, 7 in 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: 6.14 scrub starts 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: 6.14 scrub ok 2024-09-21T11:44:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:46.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:44:46.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:44:46.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:44:46.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:44:46.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: 5.c scrub starts 2024-09-21T11:44:46.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:46 smithi175 ceph-mon[32613]: 5.c scrub ok 2024-09-21T11:44:46.841 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:46.863 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:46.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:46.864 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:46.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:46.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:46.987 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:46.988 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:47.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:47.111 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:47.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:47.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:47.212 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:47.236 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:47.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:47.237 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:47.319 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:44:47.319 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:44:47.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:47.361 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:47.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:47.362 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:47.463 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:47.485 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:47.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:47.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:47.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:47.603 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:44:47.603 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:44:47.612 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:47.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:47.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:47.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:47.739 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:47.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:47.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:47 smithi043 ceph-mon[36454]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T11:44:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:47 smithi043 ceph-mon[36454]: 5.15 scrub starts 2024-09-21T11:44:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:47 smithi043 ceph-mon[36454]: 5.15 scrub ok 2024-09-21T11:44:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:47 smithi043 ceph-mon[36454]: 5.1d scrub starts 2024-09-21T11:44:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:47 smithi043 ceph-mon[36454]: 5.1d scrub ok 2024-09-21T11:44:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:47 smithi043 ceph-mon[29710]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T11:44:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:47 smithi043 ceph-mon[29710]: 5.15 scrub starts 2024-09-21T11:44:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:47 smithi043 ceph-mon[29710]: 5.15 scrub ok 2024-09-21T11:44:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:47 smithi043 ceph-mon[29710]: 5.1d scrub starts 2024-09-21T11:44:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:47 smithi043 ceph-mon[29710]: 5.1d scrub ok 2024-09-21T11:44:47.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:47 smithi175 ceph-mon[32613]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T11:44:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:47 smithi175 ceph-mon[32613]: 5.15 scrub starts 2024-09-21T11:44:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:47 smithi175 ceph-mon[32613]: 5.15 scrub ok 2024-09-21T11:44:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:47 smithi175 ceph-mon[32613]: 5.1d scrub starts 2024-09-21T11:44:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:47 smithi175 ceph-mon[32613]: 5.1d scrub ok 2024-09-21T11:44:47.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:47.865 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:47.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:47.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:47.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:47.992 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:47.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:47.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:48.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:48.118 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:48.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:48.119 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:48.219 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:48.245 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:48.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:48.246 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:48.347 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:48.371 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:48.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:48.372 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:48.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:44:48.401 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:44:48.473 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:48 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1977219715' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:48 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:48 smithi043 ceph-mon[36454]: 3.e scrub starts 2024-09-21T11:44:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:48 smithi043 ceph-mon[36454]: 3.e scrub ok 2024-09-21T11:44:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:48 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1977219715' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:48 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:48 smithi043 ceph-mon[29710]: 3.e scrub starts 2024-09-21T11:44:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:48 smithi043 ceph-mon[29710]: 3.e scrub ok 2024-09-21T11:44:48.496 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:48.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:48.497 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:48.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:44:48.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:48.620 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:48.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:48.621 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:48.722 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:48.744 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:48.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:48.745 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:48.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:48 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1977219715' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:48.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:48 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:44:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:48 smithi175 ceph-mon[32613]: 3.e scrub starts 2024-09-21T11:44:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:48 smithi175 ceph-mon[32613]: 3.e scrub ok 2024-09-21T11:44:48.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:48.869 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:48.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:48.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:48.970 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:48.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:48.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:49.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:49.118 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:49.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:49.119 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:49.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:49.243 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:49.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:49.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:49.345 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:49.371 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:49.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:49.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:49.472 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:49.497 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:49.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:49.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:49.505 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:49.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:49.621 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:49.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:49.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:49.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:49 smithi043 ceph-mon[29710]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:44:49.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:49 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:49.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:49 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:44:49.636 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:49 smithi043 ceph-mon[29710]: osdmap e159: 8 total, 8 up, 7 in 2024-09-21T11:44:49.636 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:49 smithi043 ceph-mon[36454]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:44:49.636 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:49 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:49.636 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:49 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:44:49.636 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:49 smithi043 ceph-mon[36454]: osdmap e159: 8 total, 8 up, 7 in 2024-09-21T11:44:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:49 smithi175 ceph-mon[32613]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:44:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:49 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:49 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:44:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:49 smithi175 ceph-mon[32613]: osdmap e159: 8 total, 8 up, 7 in 2024-09-21T11:44:49.723 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:49.749 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:49.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:49.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:49.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:49.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:49.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:49.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:49.997 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:49.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:49.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:50.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:50.121 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:50.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:50.122 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:50.206 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:44:50.207 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:44:50.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:50.224 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:44:50.224 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:44:50.240 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:44:50.240 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:44:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:50.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:50.247 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:50.258 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:44:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:44:50.275 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:44:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:44:50.292 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:44:50.292 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:44:50.309 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:44:50.309 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:44:50.328 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:44:50.329 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:44:50.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:50.370 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:50.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:50.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:50.405 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'unique_pool_1' created 2024-09-21T11:44:50.437 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:44:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:44:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-21T11:44:50.438 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:44:50.472 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:50.494 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:50.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:50.495 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:50.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:50.618 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:50.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:50.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:50.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2183171698' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-21T11:44:50.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:50 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-21T11:44:50.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/274658096' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:50.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2183171698' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-21T11:44:50.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:50 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-21T11:44:50.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/274658096' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:50.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:50.741 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:50.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:50.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:50.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2183171698' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-21T11:44:50.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:50 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-21T11:44:50.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/274658096' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:50.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:50.865 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:50.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:50.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:50.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:50.989 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:50.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:50.990 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:51.038 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:44:51.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:51.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:51.114 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:51.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:51.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:51.239 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:51.239 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:51.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:44:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:44:51.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:51.362 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:51.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:51.363 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:51.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:51.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:51.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:51.588 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:51.611 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:51.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:51.611 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:51.657 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:51 smithi043 ceph-mon[29710]: pgmap v219: 132 pgs: 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:44:51.657 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:51 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-21T11:44:51.657 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:51 smithi043 ceph-mon[29710]: osdmap e160: 8 total, 8 up, 7 in 2024-09-21T11:44:51.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:51 smithi043 ceph-mon[36454]: pgmap v219: 132 pgs: 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:44:51.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:51 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-21T11:44:51.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:51 smithi043 ceph-mon[36454]: osdmap e160: 8 total, 8 up, 7 in 2024-09-21T11:44:51.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:44:51.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:51.734 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:51.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:51.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:51.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:51 smithi175 ceph-mon[32613]: pgmap v219: 132 pgs: 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:44:51.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:51 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-21T11:44:51.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:51 smithi175 ceph-mon[32613]: osdmap e160: 8 total, 8 up, 7 in 2024-09-21T11:44:51.836 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:51.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:51.863 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:51.863 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:51.964 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:51.988 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:51.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:51.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:52.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:52.115 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:52.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:52.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:52.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:52.242 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:52.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:52.243 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:52.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:52.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:52.368 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:52.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:52.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:52.492 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:52.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-21T11:44:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:52.588 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:52.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:52.618 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:52.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:52.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:52.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:52.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:52 smithi043 ceph-mon[29710]: osdmap e161: 8 total, 8 up, 7 in 2024-09-21T11:44:52.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:52.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:52 smithi043 ceph-mon[36454]: osdmap e161: 8 total, 8 up, 7 in 2024-09-21T11:44:52.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:52.743 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:52.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:52.745 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:52.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:44:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:52 smithi175 ceph-mon[32613]: osdmap e161: 8 total, 8 up, 7 in 2024-09-21T11:44:52.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:52.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:52.871 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:52.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:52.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:52.999 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:52.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:53.000 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:53.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:53.123 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:53.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:53.124 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:53.224 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:53.249 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:53.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:53.250 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:53.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:53.374 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:53.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:53.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:53.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:53.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:53 smithi043 ceph-mon[36454]: pgmap v222: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:44:53.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:53 smithi043 ceph-mon[36454]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:44:53.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:53 smithi043 ceph-mon[36454]: osdmap e162: 8 total, 8 up, 7 in 2024-09-21T11:44:53.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:53 smithi043 ceph-mon[29710]: pgmap v222: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:44:53.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:53 smithi043 ceph-mon[29710]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:44:53.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:53 smithi043 ceph-mon[29710]: osdmap e162: 8 total, 8 up, 7 in 2024-09-21T11:44:53.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:53.498 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:53.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:53.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:53.623 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:53.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:53.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:53.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:53.750 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:53.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:53.751 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:53.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:53 smithi175 ceph-mon[32613]: pgmap v222: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:44:53.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:53 smithi175 ceph-mon[32613]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:44:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:53 smithi175 ceph-mon[32613]: osdmap e162: 8 total, 8 up, 7 in 2024-09-21T11:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:44:53.849 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:44:53.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:53.875 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:53.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:53.877 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:53.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:54.002 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:54.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:54.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:54.083 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:44:54.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:54.131 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:54.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:54.132 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:54.232 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:54.254 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:54.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:54.255 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:54.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:54.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:54.380 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:54.421 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:44:54.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:54.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:54.504 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:54.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:54.632 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:54.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:54.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:54.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:54.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:54 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:44:54.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:54 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:44:54.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:54 smithi043 ceph-mon[29710]: osdmap e163: 8 total, 8 up, 7 in 2024-09-21T11:44:54.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:54 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/367622112' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:54.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:54 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:44:54.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:54 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:44:54.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:54 smithi043 ceph-mon[36454]: osdmap e163: 8 total, 8 up, 7 in 2024-09-21T11:44:54.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:54 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/367622112' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:54.760 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:54.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:54.761 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:54 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:44:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:54 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:44:54.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:54 smithi175 ceph-mon[32613]: osdmap e163: 8 total, 8 up, 7 in 2024-09-21T11:44:54.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:54 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/367622112' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:44:54.862 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:54.886 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:54.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:54.887 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:54.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:44:54.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:55.013 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:55.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:55.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:55.114 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:55.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:55.138 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:55.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:55.260 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:55.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:55.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:55.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:55.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:55.385 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:55.485 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:44:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:44:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:44:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:55 smithi043 ceph-mon[29710]: pgmap v225: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:44:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:55 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:55 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/367622112' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:55 smithi043 ceph-mon[29710]: osdmap e164: 8 total, 8 up, 7 in 2024-09-21T11:44:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:55 smithi043 ceph-mon[36454]: pgmap v225: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:44:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:55 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:55 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/367622112' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:55.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:55 smithi043 ceph-mon[36454]: osdmap e164: 8 total, 8 up, 7 in 2024-09-21T11:44:55.511 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:55.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:55.512 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:55.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:55.635 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:55.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:55.636 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:55.737 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:55.738 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:55.738 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:55 smithi175 ceph-mon[32613]: pgmap v225: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:44:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:55 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:44:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:55 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/367622112' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:44:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:55 smithi175 ceph-mon[32613]: osdmap e164: 8 total, 8 up, 7 in 2024-09-21T11:44:55.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:55.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:55.765 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:55.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:55.892 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:55.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:55.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:55.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:56.017 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:56.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:56.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:56.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:56.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:56.146 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:56.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:56.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:56.273 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:56.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:56.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:56.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:56.399 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:56.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:56.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:56.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:56.526 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:56.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:56.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:56.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:56.652 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:56.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:56.653 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:56.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:56.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:56.782 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:56.782 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:56.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:56.908 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:56.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:56.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:57.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:57.038 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:57.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:57.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:57.139 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:57.165 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:57.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:57.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:57.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:57.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:57.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:57.314 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:44:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:44:57.321 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:44:57.321 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:44:57.337 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:44:57.337 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:44:57.353 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:44:57.354 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:44:57.371 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:44:57.371 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:44:57.388 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:44:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:44:57.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:57.406 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:44:57.406 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:44:57.417 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:57.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:57.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:57.423 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:44:57.423 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:44:57.452 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:57 smithi043 ceph-mon[36454]: pgmap v227: 148 pgs: 11 creating+peering, 137 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:44:57.452 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:57 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/422575065' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:57.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:57 smithi043 ceph-mon[29710]: pgmap v227: 148 pgs: 11 creating+peering, 137 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:44:57.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/422575065' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:57.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:57.541 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:57.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:57.542 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:57.642 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:57.667 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:57.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:57.668 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:57.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:44:57.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:57 smithi175 ceph-mon[32613]: pgmap v227: 148 pgs: 11 creating+peering, 137 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:44:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:57 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/422575065' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:44:57.769 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:57.793 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:57.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:57.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:57.895 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:57.919 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:57.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:57.920 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:58.021 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:58.043 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:58.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:58.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:58.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:58.167 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:58.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:58.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:58.269 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:58.292 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:58.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:58.293 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:58.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:58.417 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:58.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:58.418 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:58.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:58.541 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:58.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:58.542 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:58.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:44:58.667 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:58.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:58.668 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:44:58.672 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:58.769 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:58.792 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:58.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:58.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:58.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:58.917 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:58.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:58.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:58.957 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:44:59.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:44:59.042 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:59.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:59.042 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:44:59.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:44:59.167 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:59.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:59.167 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:44:59.268 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:44:59.292 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:59.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:59.293 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:44:59.393 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:44:59.420 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:59.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:59.421 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:44:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-21T11:44:59.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:59.544 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:59.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:59.545 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:44:59.547 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:44:59.547 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:44:59.645 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:44:59.671 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:59.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:59.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:44:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[36454]: pgmap v228: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:44:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:44:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:44:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[36454]: osdmap e165: 8 total, 8 up, 7 in 2024-09-21T11:44:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2974148742' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:44:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:44:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[29710]: pgmap v228: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:44:59.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:44:59.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:44:59.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[29710]: osdmap e165: 8 total, 8 up, 7 in 2024-09-21T11:44:59.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2974148742' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:44:59.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:44:59 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:44:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:59 smithi175 ceph-mon[32613]: pgmap v228: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:44:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:59 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:44:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:59 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:44:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:59 smithi175 ceph-mon[32613]: osdmap e165: 8 total, 8 up, 7 in 2024-09-21T11:44:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:59 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2974148742' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:44:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:44:59 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:44:59.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:44:59.799 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:44:59.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:59.800 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:44:59.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:44:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:44:59.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:44:59.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:00.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:00.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:00.077 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:00.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:00.205 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:00.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:00.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:00.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:45:00.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:00.330 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:00.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:00.332 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:00.432 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:00.459 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:00.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:00.460 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:00.536 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:00 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:45:00.536 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:00 smithi043 ceph-mon[36454]: osdmap e166: 8 total, 8 up, 7 in 2024-09-21T11:45:00.537 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:00 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:45:00.537 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:00 smithi043 ceph-mon[29710]: osdmap e166: 8 total, 8 up, 7 in 2024-09-21T11:45:00.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:00.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:00.583 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:00.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:00.708 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:00.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:00.709 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:00 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:45:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:00 smithi175 ceph-mon[32613]: osdmap e166: 8 total, 8 up, 7 in 2024-09-21T11:45:00.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:00.833 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:00.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:00.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:00.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:45:00.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:00.960 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:00.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:00.961 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:01.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:01.083 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:01.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:01.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:01.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:01.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:01.207 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:01.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:01.260 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:45:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:45:01.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:01.330 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:01.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:01.331 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:01.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:01.453 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:01.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:01.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:01.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:01.578 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:01.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:01.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:01.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:01.702 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:01.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:01.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[36454]: pgmap v231: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:45:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:45:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:45:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[36454]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:45:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[36454]: osdmap e167: 8 total, 8 up, 7 in 2024-09-21T11:45:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[29710]: pgmap v231: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:45:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:45:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:45:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[29710]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:45:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[29710]: osdmap e167: 8 total, 8 up, 7 in 2024-09-21T11:45:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:01 smithi175 ceph-mon[32613]: pgmap v231: 148 pgs: 148 active+clean; 583 KiB data, 1.1 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:45:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:01 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:45:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:01 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:45:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:01 smithi175 ceph-mon[32613]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:45:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:01 smithi175 ceph-mon[32613]: osdmap e167: 8 total, 8 up, 7 in 2024-09-21T11:45:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:01 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:01.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:01.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:01.828 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:01.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:01.829 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:01.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:01.952 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:01.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:01.953 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:02.053 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:02.075 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:02.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:02.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:02.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:02.205 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:02.208 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:02.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:02.218 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:02.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:02.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:02.343 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:02.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:02.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:02.468 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:02.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:02.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:02.478 DEBUG:teuthology.orchestra.run.smithi043:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-21T11:45:02.478 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_1 2024-09-21T11:45:02.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:45:02.534 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-21T11:45:02.534 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-21T11:45:02.569 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:02.589 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:hints = 1 2024-09-21T11:45:02.589 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-21T11:45:02.589 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Object prefix: benchmark_data_smithi043_97042 2024-09-21T11:45:02.589 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:45:02.589 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 0 0 0 0 0 0 - 0 2024-09-21T11:45:02.593 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:02.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:02.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:02.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:02.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:02.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2030951424' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:45:02.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:02.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2030951424' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:45:02.726 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:02.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:02.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2030951424' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:45:02.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:02.874 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:02.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:02.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:02.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:02.994 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:45:02.994 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:45:03.010 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:03.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:03.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:03.032 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:45:03.033 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:45:03.071 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:45:03.071 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:45:03.110 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:45:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:45:03.111 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:03.142 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:03.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:03.143 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:03.148 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:45:03.148 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:45:03.186 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:45:03.186 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:45:03.197 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:45:03.223 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:45:03.223 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:45:03.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:03.253 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:45:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:45:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:2005 8.b 8:da5a3fb4:::benchmark_data_smithi043_97042_object2004:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e168)", 2024-09-21T11:45:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:03.243080+0000", 2024-09-21T11:45:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0029231790000000001, 2024-09-21T11:45:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029574409999999999, 2024-09-21T11:45:03.254 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2005 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243080+0000", 2024-09-21T11:45:03.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243080+0000", 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243081+0000", 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9899999999999999e-07 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:03.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243102+0000", 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1036e-05 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243110+0000", 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.807e-06 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243116+0000", 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2659999999999998e-06 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243164+0000", 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8060999999999999e-05 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243279+0000", 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115591 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243424+0000", 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000144635 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.244364+0000", 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00093988199999999996 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.244648+0000", 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028450900000000001 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.259 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:2006 8.7 8:ed30f281:::benchmark_data_smithi043_97042_object2005:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e168)", 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:03.243962+0000", 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0020407839999999999, 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021670069999999999, 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2006 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:03.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243962+0000", 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243962+0000", 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243963+0000", 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7400000000000002e-07 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243987+0000", 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3464000000000001e-05 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243996+0000", 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5109999999999992e-06 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.244003+0000", 2024-09-21T11:45:03.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0879999999999997e-06 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.244795+0000", 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00079226700000000004 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.244907+0000", 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111209 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:45:03.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245027+0000", 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012080500000000001 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:2011 8.7 8:e0f7c832:::benchmark_data_smithi043_97042_object2010:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e168)", 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:03.245051+0000", 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00095214699999999998, 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011529979999999999, 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:03.264 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2011 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245051+0000", 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245051+0000", 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245052+0000", 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2099999999999995e-07 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245075+0000", 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2977e-05 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245085+0000", 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0158e-05 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245092+0000", 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3950000000000003e-06 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245982+0000", 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000890065 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.246092+0000", 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010997099999999999 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:2013 8.7 8.4673c087 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e168)", 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:03.245104+0000", 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00089862000000000004, 2024-09-21T11:45:03.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011763609999999999, 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2013 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245104+0000", 2024-09-21T11:45:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245104+0000", 2024-09-21T11:45:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245105+0000", 2024-09-21T11:45:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8400000000000005e-07 2024-09-21T11:45:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245125+0000", 2024-09-21T11:45:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0119000000000002e-05 2024-09-21T11:45:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245131+0000", 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4339999999999997e-06 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245137+0000", 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4569999999999996e-06 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.272 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:2005 8.b e168/160 ondisk, result = 0)", 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:03.246126+0000", 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9998765, 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000204117, 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.246126+0000", 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:03.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.246126+0000", 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.246127+0000", 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3799999999999996e-07 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.246132+0000", 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7339999999999999e-06 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.246143+0000", 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0601999999999999e-05 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.246150+0000", 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4290000000000004e-06 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.246210+0000", 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9845e-05 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.246218+0000", 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7610000000000007e-06 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:2003 8.7 8:e2cf370f:::benchmark_data_smithi043_97042_object2002:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e168)", 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:03.242375+0000", 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0036283209999999999, 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040140660000000002, 2024-09-21T11:45:03.276 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 2003 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.242375+0000", 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.242375+0000", 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.242376+0000", 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.221e-06 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.242404+0000", 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7997999999999999e-05 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.242418+0000", 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3725e-05 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.242425+0000", 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4240000000000004e-06 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.242792+0000", 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00036669099999999999 2024-09-21T11:45:03.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.243838+0000", 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010463639999999999 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.244032+0000", 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000194081 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:03.245947+0000", 2024-09-21T11:45:03.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001914596 2024-09-21T11:45:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:45:03.281 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 6 2024-09-21T11:45:03.281 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:45:03.281 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:45:03.282 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:45:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:03.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:03.292 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:03.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:03.425 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:03.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:03.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:03.515 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:03 smithi043 ceph-mon[29710]: pgmap v233: 148 pgs: 148 active+clean; 583 KiB data, 1.0 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:45:03.515 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:03 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/550292872' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:03.515 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:03 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:03.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:03 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2030951424' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:45:03.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:03 smithi043 ceph-mon[29710]: osdmap e168: 8 total, 8 up, 7 in 2024-09-21T11:45:03.516 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:03 smithi043 ceph-mon[36454]: pgmap v233: 148 pgs: 148 active+clean; 583 KiB data, 1.0 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:45:03.516 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:03 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/550292872' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:03.516 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:03 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:03.516 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:03 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2030951424' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:45:03.516 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:03 smithi043 ceph-mon[36454]: osdmap e168: 8 total, 8 up, 7 in 2024-09-21T11:45:03.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:03.572 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:03.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:03.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:03.590 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 1 15 3229 3214 200.863 200.875 0.00288383 0.00496055 2024-09-21T11:45:03.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:03.708 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:03.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:03.709 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:03 smithi175 ceph-mon[32613]: pgmap v233: 148 pgs: 148 active+clean; 583 KiB data, 1.0 GiB used, 625 GiB / 626 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T11:45:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:03 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/550292872' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:03 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:03 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2030951424' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:45:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:03 smithi175 ceph-mon[32613]: osdmap e168: 8 total, 8 up, 7 in 2024-09-21T11:45:03.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:03.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:03.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:03.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:03.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:03.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:04.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:04.130 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:04.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:04.131 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:04.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:04.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:04.269 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:04.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:04.409 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:04.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:04.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:04.431 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:04.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:04 smithi043 ceph-mon[29710]: 6.f scrub starts 2024-09-21T11:45:04.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:04 smithi043 ceph-mon[29710]: 6.f scrub ok 2024-09-21T11:45:04.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:04.552 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:04.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:04.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:04.590 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 2 16 6837 6821 213.122 225.438 0.00375154 0.00468292 2024-09-21T11:45:04.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:04.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:04.693 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:04.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:04 smithi043 ceph-mon[36454]: 6.f scrub starts 2024-09-21T11:45:04.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:04 smithi043 ceph-mon[36454]: 6.f scrub ok 2024-09-21T11:45:04.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:04 smithi175 ceph-mon[32613]: 6.f scrub starts 2024-09-21T11:45:04.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:04 smithi175 ceph-mon[32613]: 6.f scrub ok 2024-09-21T11:45:04.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:04.829 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:04.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:04.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:04.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:04.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:45:04.963 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:04.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:04.964 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:05.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:05.109 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:05.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:05.110 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:05.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:05.236 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:05.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:05.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:05.337 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:05.371 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:05.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:05.372 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:05.453 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:45:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:45:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:45:05.473 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:05.506 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:05.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:05.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:05.590 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 3 15 10003 9988 208.055 197.938 0.00668698 0.00479817 2024-09-21T11:45:05.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:05.657 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:05.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:05.658 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:05.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:05 smithi043 ceph-mon[36454]: pgmap v235: 148 pgs: 148 active+clean; 583 KiB data, 1.0 GiB used, 625 GiB / 626 GiB avail; 952 B/s rd, 0 op/s 2024-09-21T11:45:05.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:05 smithi043 ceph-mon[29710]: pgmap v235: 148 pgs: 148 active+clean; 583 KiB data, 1.0 GiB used, 625 GiB / 626 GiB avail; 952 B/s rd, 0 op/s 2024-09-21T11:45:05.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:05 smithi175 ceph-mon[32613]: pgmap v235: 148 pgs: 148 active+clean; 583 KiB data, 1.0 GiB used, 625 GiB / 626 GiB avail; 952 B/s rd, 0 op/s 2024-09-21T11:45:05.795 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:05.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:05.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:05.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:05.936 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:05.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:05.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:06.037 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:06.080 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:06.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:06.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:06.156 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:06.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:06.211 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:06.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:06.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:06.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:45:06.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:06.344 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:06.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:06.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:06.409 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:45:06.412 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:45:06.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:06.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:06.499 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:06.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:06.591 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 4 16 12735 12719 198.647 170.688 0.00792445 0.00502439 2024-09-21T11:45:06.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:06.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:06.639 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:06.740 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[29710]: 6.11 scrub starts 2024-09-21T11:45:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[29710]: 6.11 scrub ok 2024-09-21T11:45:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[29710]: 3.6 scrub starts 2024-09-21T11:45:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[29710]: 3.6 scrub ok 2024-09-21T11:45:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[29710]: 5.8 scrub starts 2024-09-21T11:45:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[29710]: 5.8 scrub ok 2024-09-21T11:45:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[29710]: pgmap v236: 148 pgs: 148 active+clean; 207 MiB data, 1.8 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 32 MiB/s wr, 520 op/s 2024-09-21T11:45:06.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[29710]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:45:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[36454]: 6.11 scrub starts 2024-09-21T11:45:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[36454]: 6.11 scrub ok 2024-09-21T11:45:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[36454]: 3.6 scrub starts 2024-09-21T11:45:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[36454]: 3.6 scrub ok 2024-09-21T11:45:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[36454]: 5.8 scrub starts 2024-09-21T11:45:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[36454]: 5.8 scrub ok 2024-09-21T11:45:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[36454]: pgmap v236: 148 pgs: 148 active+clean; 207 MiB data, 1.8 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 32 MiB/s wr, 520 op/s 2024-09-21T11:45:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:06 smithi043 ceph-mon[36454]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:45:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:06 smithi175 ceph-mon[32613]: 6.11 scrub starts 2024-09-21T11:45:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:06 smithi175 ceph-mon[32613]: 6.11 scrub ok 2024-09-21T11:45:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:06 smithi175 ceph-mon[32613]: 3.6 scrub starts 2024-09-21T11:45:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:06 smithi175 ceph-mon[32613]: 3.6 scrub ok 2024-09-21T11:45:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:06 smithi175 ceph-mon[32613]: 5.8 scrub starts 2024-09-21T11:45:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:06 smithi175 ceph-mon[32613]: 5.8 scrub ok 2024-09-21T11:45:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:06 smithi175 ceph-mon[32613]: pgmap v236: 148 pgs: 148 active+clean; 207 MiB data, 1.8 GiB used, 624 GiB / 626 GiB avail; 1.6 KiB/s rd, 32 MiB/s wr, 520 op/s 2024-09-21T11:45:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:06 smithi175 ceph-mon[32613]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:45:06.773 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:06.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:06.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:06.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:06.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:06.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:06.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:07.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:07.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:07.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:07.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:07.168 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-21T11:45:05.806017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":217520667,"num_objects":3509,"num_object_clones":0,"num_object_copies":10527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3509,"num_whiteouts":0,"num_read":1135,"num_read_kb":992,"num_write":3859,"num_write_kb":213513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":594233,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3866,"ondisk_log_size":3866,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":1845816,"kb_used_data":736232,"kb_used_omap":10,"kb_used_meta":1109429,"kb_avail":654370248,"statfs":{"total":671965249536,"available":670075133952,"internally_reserved":0,"allocated":753901568,"data_stored":750911113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11156,"internal_metadata":1136055404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,6,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":216924160,"num_objects":3310,"num_object_clones":0,"num_object_copies":9930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3310,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3310,"num_write_kb":211840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.376308"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":237,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239487+0000","last_change":"2024-09-21T11:41:42.291880+0000","last_active":"2024-09-21T11:45:01.239487+0000","last_peered":"2024-09-21T11:45:01.239487+0000","last_clean":"2024-09-21T11:45:01.239487+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:45:01.239487+0000","last_undegraded":"2024-09-21T11:45:01.239487+0000","last_fullsized":"2024-09-21T11:45:01.239487+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:42.290188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:42.290188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":277,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193202+0000","last_change":"2024-09-21T11:44:19.929729+0000","last_active":"2024-09-21T11:45:03.193202+0000","last_peered":"2024-09-21T11:45:03.193202+0000","last_clean":"2024-09-21T11:45:03.193202+0000","last_became_active":"2024-09-21T11:42:37.164858+0000","last_became_peered":"2024-09-21T11:42:37.164858+0000","last_unstale":"2024-09-21T11:45:03.193202+0000","last_undegraded":"2024-09-21T11:45:03.193202+0000","last_fullsized":"2024-09-21T11:45:03.193202+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:44:19.929450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:19.929450+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:31.219724+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":133,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.196796+0000","last_change":"2024-09-21T11:43:37.050216+0000","last_active":"2024-09-21T11:45:03.196796+0000","last_peered":"2024-09-21T11:45:03.196796+0000","last_clean":"2024-09-21T11:45:03.196796+0000","last_became_active":"2024-09-21T11:42:37.161713+0000","last_became_peered":"2024-09-21T11:42:37.161713+0000","last_unstale":"2024-09-21T11:45:03.196796+0000","last_undegraded":"2024-09-21T11:45:03.196796+0000","last_fullsized":"2024-09-21T11:45:03.196796+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:53.271101+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":235,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193414+0000","last_change":"2024-09-21T11:42:59.201398+0000","last_active":"2024-09-21T11:45:03.193414+0000","last_peered":"2024-09-21T11:45:03.193414+0000","last_clean":"2024-09-21T11:45:03.193414+0000","last_became_active":"2024-09-21T11:42:37.158907+0000","last_became_peered":"2024-09-21T11:42:37.158907+0000","last_unstale":"2024-09-21T11:45:03.193414+0000","last_undegraded":"2024-09-21T11:45:03.193414+0000","last_fullsized":"2024-09-21T11:45:03.193414+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:59.201330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:59.201330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":225,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443488+0000","last_change":"2024-09-21T11:43:28.461458+0000","last_active":"2024-09-21T11:45:00.443488+0000","last_peered":"2024-09-21T11:45:00.443488+0000","last_clean":"2024-09-21T11:45:00.443488+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:45:00.443488+0000","last_undegraded":"2024-09-21T11:45:00.443488+0000","last_fullsized":"2024-09-21T11:45:00.443488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.461378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.461378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":230,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443962+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:45:00.443962+0000","last_peered":"2024-09-21T11:45:00.443962+0000","last_clean":"2024-09-21T11:45:00.443962+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:45:00.443962+0000","last_undegraded":"2024-09-21T11:45:00.443962+0000","last_fullsized":"2024-09-21T11:45:00.443962+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:53.756395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":251,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443197+0000","last_change":"2024-09-21T11:40:02.315669+0000","last_active":"2024-09-21T11:45:00.443197+0000","last_peered":"2024-09-21T11:45:00.443197+0000","last_clean":"2024-09-21T11:45:00.443197+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:45:00.443197+0000","last_undegraded":"2024-09-21T11:45:00.443197+0000","last_fullsized":"2024-09-21T11:45:00.443197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.613208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":247,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193197+0000","last_change":"2024-09-21T11:43:04.373004+0000","last_active":"2024-09-21T11:45:03.193197+0000","last_peered":"2024-09-21T11:45:03.193197+0000","last_clean":"2024-09-21T11:45:03.193197+0000","last_became_active":"2024-09-21T11:42:37.154511+0000","last_became_peered":"2024-09-21T11:42:37.154511+0000","last_unstale":"2024-09-21T11:45:03.193197+0000","last_undegraded":"2024-09-21T11:45:03.193197+0000","last_fullsized":"2024-09-21T11:45:03.193197+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:04.372941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:04.372941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.920192+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":239,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197519+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:45:03.197519+0000","last_peered":"2024-09-21T11:45:03.197519+0000","last_clean":"2024-09-21T11:45:03.197519+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:45:03.197519+0000","last_undegraded":"2024-09-21T11:45:03.197519+0000","last_fullsized":"2024-09-21T11:45:03.197519+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:28.425846+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":260,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193599+0000","last_change":"2024-09-21T11:43:26.258023+0000","last_active":"2024-09-21T11:45:03.193599+0000","last_peered":"2024-09-21T11:45:03.193599+0000","last_clean":"2024-09-21T11:45:03.193599+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:45:03.193599+0000","last_undegraded":"2024-09-21T11:45:03.193599+0000","last_fullsized":"2024-09-21T11:45:03.193599+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:26.257967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.257967+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:58.850809+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":239,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197074+0000","last_change":"2024-09-21T11:44:12.257513+0000","last_active":"2024-09-21T11:45:03.197074+0000","last_peered":"2024-09-21T11:45:03.197074+0000","last_clean":"2024-09-21T11:45:03.197074+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:45:03.197074+0000","last_undegraded":"2024-09-21T11:45:03.197074+0000","last_fullsized":"2024-09-21T11:45:03.197074+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.257454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.257454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:13.769532+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":232,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.237777+0000","last_change":"2024-09-21T11:43:44.648053+0000","last_active":"2024-09-21T11:45:01.237777+0000","last_peered":"2024-09-21T11:45:01.237777+0000","last_clean":"2024-09-21T11:45:01.237777+0000","last_became_active":"2024-09-21T11:42:37.155879+0000","last_became_peered":"2024-09-21T11:42:37.155879+0000","last_unstale":"2024-09-21T11:45:01.237777+0000","last_undegraded":"2024-09-21T11:45:01.237777+0000","last_fullsized":"2024-09-21T11:45:01.237777+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.647985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.647985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":254,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197456+0000","last_change":"2024-09-21T11:42:37.156936+0000","last_active":"2024-09-21T11:45:03.197456+0000","last_peered":"2024-09-21T11:45:03.197456+0000","last_clean":"2024-09-21T11:45:03.197456+0000","last_became_active":"2024-09-21T11:42:37.156575+0000","last_became_peered":"2024-09-21T11:42:37.156575+0000","last_unstale":"2024-09-21T11:45:03.197456+0000","last_undegraded":"2024-09-21T11:45:03.197456+0000","last_fullsized":"2024-09-21T11:45:03.197456+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.101955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.101955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:35.459584+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":267,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193469+0000","last_change":"2024-09-21T11:43:08.326923+0000","last_active":"2024-09-21T11:45:03.193469+0000","last_peered":"2024-09-21T11:45:03.193469+0000","last_clean":"2024-09-21T11:45:03.193469+0000","last_became_active":"2024-09-21T11:41:31.634863+0000","last_became_peered":"2024-09-21T11:41:31.634863+0000","last_unstale":"2024-09-21T11:45:03.193469+0000","last_undegraded":"2024-09-21T11:45:03.193469+0000","last_fullsized":"2024-09-21T11:45:03.193469+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:08.326846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:08.326846+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:24.972822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":238,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.201832+0000","last_change":"2024-09-21T11:43:38.567194+0000","last_active":"2024-09-21T11:45:03.201832+0000","last_peered":"2024-09-21T11:45:03.201832+0000","last_clean":"2024-09-21T11:45:03.201832+0000","last_became_active":"2024-09-21T11:42:37.164285+0000","last_became_peered":"2024-09-21T11:42:37.164285+0000","last_unstale":"2024-09-21T11:45:03.201832+0000","last_undegraded":"2024-09-21T11:45:03.201832+0000","last_fullsized":"2024-09-21T11:45:03.201832+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.567131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.567131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.325966+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":242,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.196776+0000","last_change":"2024-09-21T11:44:42.350677+0000","last_active":"2024-09-21T11:45:03.196776+0000","last_peered":"2024-09-21T11:45:03.196776+0000","last_clean":"2024-09-21T11:45:03.196776+0000","last_became_active":"2024-09-21T11:42:37.162912+0000","last_became_peered":"2024-09-21T11:42:37.162912+0000","last_unstale":"2024-09-21T11:45:03.196776+0000","last_undegraded":"2024-09-21T11:45:03.196776+0000","last_fullsized":"2024-09-21T11:45:03.196776+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:42.350621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:42.350621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:07.179165+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":244,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193785+0000","last_change":"2024-09-21T11:44:38.861427+0000","last_active":"2024-09-21T11:45:03.193785+0000","last_peered":"2024-09-21T11:45:03.193785+0000","last_clean":"2024-09-21T11:45:03.193785+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:45:03.193785+0000","last_undegraded":"2024-09-21T11:45:03.193785+0000","last_fullsized":"2024-09-21T11:45:03.193785+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:38.861369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:38.861369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:50.429021+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":237,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239359+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:45:01.239359+0000","last_peered":"2024-09-21T11:45:01.239359+0000","last_clean":"2024-09-21T11:45:01.239359+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:45:01.239359+0000","last_undegraded":"2024-09-21T11:45:01.239359+0000","last_fullsized":"2024-09-21T11:45:01.239359+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:53.172684+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":312,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.442522+0000","last_change":"2024-09-21T11:43:31.498356+0000","last_active":"2024-09-21T11:45:00.442522+0000","last_peered":"2024-09-21T11:45:00.442522+0000","last_clean":"2024-09-21T11:45:00.442522+0000","last_became_active":"2024-09-21T11:42:37.166901+0000","last_became_peered":"2024-09-21T11:42:37.166901+0000","last_unstale":"2024-09-21T11:45:00.442522+0000","last_undegraded":"2024-09-21T11:45:00.442522+0000","last_fullsized":"2024-09-21T11:45:00.442522+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:31.498142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.498142+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:34.605537+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":140,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193689+0000","last_change":"2024-09-21T11:44:46.822178+0000","last_active":"2024-09-21T11:45:03.193689+0000","last_peered":"2024-09-21T11:45:03.193689+0000","last_clean":"2024-09-21T11:45:03.193689+0000","last_became_active":"2024-09-21T11:42:37.153799+0000","last_became_peered":"2024-09-21T11:42:37.153799+0000","last_unstale":"2024-09-21T11:45:03.193689+0000","last_undegraded":"2024-09-21T11:45:03.193689+0000","last_fullsized":"2024-09-21T11:45:03.193689+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:46.822132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:46.822132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.312064+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":226,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.444379+0000","last_change":"2024-09-21T11:41:40.448106+0000","last_active":"2024-09-21T11:45:00.444379+0000","last_peered":"2024-09-21T11:45:00.444379+0000","last_clean":"2024-09-21T11:45:00.444379+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:45:00.444379+0000","last_undegraded":"2024-09-21T11:45:00.444379+0000","last_fullsized":"2024-09-21T11:45:00.444379+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.447908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.447908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:30.773729+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":247,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193745+0000","last_change":"2024-09-21T11:43:45.093396+0000","last_active":"2024-09-21T11:45:03.193745+0000","last_peered":"2024-09-21T11:45:03.193745+0000","last_clean":"2024-09-21T11:45:03.193745+0000","last_became_active":"2024-09-21T11:42:37.153694+0000","last_became_peered":"2024-09-21T11:42:37.153694+0000","last_unstale":"2024-09-21T11:45:03.193745+0000","last_undegraded":"2024-09-21T11:45:03.193745+0000","last_fullsized":"2024-09-21T11:45:03.193745+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.093325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.093325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:14.273876+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":275,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443341+0000","last_change":"2024-09-21T11:42:37.189830+0000","last_active":"2024-09-21T11:45:00.443341+0000","last_peered":"2024-09-21T11:45:00.443341+0000","last_clean":"2024-09-21T11:45:00.443341+0000","last_became_active":"2024-09-21T11:42:37.165964+0000","last_became_peered":"2024-09-21T11:42:37.165964+0000","last_unstale":"2024-09-21T11:45:00.443341+0000","last_undegraded":"2024-09-21T11:45:00.443341+0000","last_fullsized":"2024-09-21T11:45:00.443341+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:41:35.427210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:35.427210+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:28.788456+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":232,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193569+0000","last_change":"2024-09-21T11:43:02.320628+0000","last_active":"2024-09-21T11:45:03.193569+0000","last_peered":"2024-09-21T11:45:03.193569+0000","last_clean":"2024-09-21T11:45:03.193569+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:45:03.193569+0000","last_undegraded":"2024-09-21T11:45:03.193569+0000","last_fullsized":"2024-09-21T11:45:03.193569+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:02.320559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:02.320559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:31.845873+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":235,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.201925+0000","last_change":"2024-09-21T11:42:31.278897+0000","last_active":"2024-09-21T11:45:03.201925+0000","last_peered":"2024-09-21T11:45:03.201925+0000","last_clean":"2024-09-21T11:45:03.201925+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:45:03.201925+0000","last_undegraded":"2024-09-21T11:45:03.201925+0000","last_fullsized":"2024-09-21T11:45:03.201925+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:42:31.278824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:31.278824+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:04.489162+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":142,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443056+0000","last_change":"2024-09-21T11:44:44.796081+0000","last_active":"2024-09-21T11:45:00.443056+0000","last_peered":"2024-09-21T11:45:00.443056+0000","last_clean":"2024-09-21T11:45:00.443056+0000","last_became_active":"2024-09-21T11:42:37.166144+0000","last_became_peered":"2024-09-21T11:42:37.166144+0000","last_unstale":"2024-09-21T11:45:00.443056+0000","last_undegraded":"2024-09-21T11:45:00.443056+0000","last_fullsized":"2024-09-21T11:45:00.443056+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:44:44.796015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.796015+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:47.778060+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":256,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239553+0000","last_change":"2024-09-21T11:44:42.278189+0000","last_active":"2024-09-21T11:45:01.239553+0000","last_peered":"2024-09-21T11:45:01.239553+0000","last_clean":"2024-09-21T11:45:01.239553+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:45:01.239553+0000","last_undegraded":"2024-09-21T11:45:01.239553+0000","last_fullsized":"2024-09-21T11:45:01.239553+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:44:42.278119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:42.278119+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:48.020804+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":315,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.923674+0000","last_change":"2024-09-21T11:43:46.917623+0000","last_active":"2024-09-21T11:44:59.923674+0000","last_peered":"2024-09-21T11:44:59.923674+0000","last_clean":"2024-09-21T11:44:59.923674+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:44:59.923674+0000","last_undegraded":"2024-09-21T11:44:59.923674+0000","last_fullsized":"2024-09-21T11:44:59.923674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:43:46.917560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:46.917560+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:06.493510+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"168'736","reported_seq":762,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:05.788802+0000","last_change":"2024-09-21T11:44:51.412099+0000","last_active":"2024-09-21T11:45:05.788802+0000","last_peered":"2024-09-21T11:45:05.788802+0000","last_clean":"2024-09-21T11:45:05.788802+0000","last_became_active":"2024-09-21T11:44:51.411794+0000","last_became_peered":"2024-09-21T11:44:51.411794+0000","last_unstale":"2024-09-21T11:45:05.788802+0000","last_undegraded":"2024-09-21T11:45:05.788802+0000","last_fullsized":"2024-09-21T11:45:05.788802+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:11.272659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48234496,"num_objects":736,"num_object_clones":0,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":736,"num_write_kb":47104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":191,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.444470+0000","last_change":"2024-09-21T11:43:44.440472+0000","last_active":"2024-09-21T11:45:00.444470+0000","last_peered":"2024-09-21T11:45:00.444470+0000","last_clean":"2024-09-21T11:45:00.444470+0000","last_became_active":"2024-09-21T11:42:37.169939+0000","last_became_peered":"2024-09-21T11:42:37.169939+0000","last_unstale":"2024-09-21T11:45:00.444470+0000","last_undegraded":"2024-09-21T11:45:00.444470+0000","last_fullsized":"2024-09-21T11:45:00.444470+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:44.440374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.440374+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:10.450444+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":245,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.441295+0000","last_change":"2024-09-21T11:42:37.172945+0000","last_active":"2024-09-21T11:45:00.441295+0000","last_peered":"2024-09-21T11:45:00.441295+0000","last_clean":"2024-09-21T11:45:00.441295+0000","last_became_active":"2024-09-21T11:42:37.171365+0000","last_became_peered":"2024-09-21T11:42:37.171365+0000","last_unstale":"2024-09-21T11:45:00.441295+0000","last_undegraded":"2024-09-21T11:45:00.441295+0000","last_fullsized":"2024-09-21T11:45:00.441295+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.454207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.454207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:32.793544+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":302,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193895+0000","last_change":"2024-09-21T11:42:26.008760+0000","last_active":"2024-09-21T11:45:03.193895+0000","last_peered":"2024-09-21T11:45:03.193895+0000","last_clean":"2024-09-21T11:45:03.193895+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:45:03.193895+0000","last_undegraded":"2024-09-21T11:45:03.193895+0000","last_fullsized":"2024-09-21T11:45:03.193895+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:26.008646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:26.008646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:53.860408+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":234,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197204+0000","last_change":"2024-09-21T11:42:37.157278+0000","last_active":"2024-09-21T11:45:03.197204+0000","last_peered":"2024-09-21T11:45:03.197204+0000","last_clean":"2024-09-21T11:45:03.197204+0000","last_became_active":"2024-09-21T11:42:37.156952+0000","last_became_peered":"2024-09-21T11:42:37.156952+0000","last_unstale":"2024-09-21T11:45:03.197204+0000","last_undegraded":"2024-09-21T11:45:03.197204+0000","last_fullsized":"2024-09-21T11:45:03.197204+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.066324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.066324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:31.525962+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":25,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239133+0000","last_change":"2024-09-21T11:44:51.744583+0000","last_active":"2024-09-21T11:45:01.239133+0000","last_peered":"2024-09-21T11:45:01.239133+0000","last_clean":"2024-09-21T11:45:01.239133+0000","last_became_active":"2024-09-21T11:44:51.744224+0000","last_became_peered":"2024-09-21T11:44:51.744224+0000","last_unstale":"2024-09-21T11:45:01.239133+0000","last_undegraded":"2024-09-21T11:45:01.239133+0000","last_fullsized":"2024-09-21T11:45:01.239133+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.180772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":266,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.441061+0000","last_change":"2024-09-21T11:43:40.423834+0000","last_active":"2024-09-21T11:45:00.441061+0000","last_peered":"2024-09-21T11:45:00.441061+0000","last_clean":"2024-09-21T11:45:00.441061+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:45:00.441061+0000","last_undegraded":"2024-09-21T11:45:00.441061+0000","last_fullsized":"2024-09-21T11:45:00.441061+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:43:40.423750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.423750+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:01.136315+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":246,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.444611+0000","last_change":"2024-09-21T11:44:18.614485+0000","last_active":"2024-09-21T11:45:00.444611+0000","last_peered":"2024-09-21T11:45:00.444611+0000","last_clean":"2024-09-21T11:45:00.444611+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:45:00.444611+0000","last_undegraded":"2024-09-21T11:45:00.444611+0000","last_fullsized":"2024-09-21T11:45:00.444611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:18.614293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:18.614293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:45.404958+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":240,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193205+0000","last_change":"2024-09-21T11:44:13.946705+0000","last_active":"2024-09-21T11:45:03.193205+0000","last_peered":"2024-09-21T11:45:03.193205+0000","last_clean":"2024-09-21T11:45:03.193205+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:45:03.193205+0000","last_undegraded":"2024-09-21T11:45:03.193205+0000","last_fullsized":"2024-09-21T11:45:03.193205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:14.029753+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":228,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193498+0000","last_change":"2024-09-21T11:44:11.930363+0000","last_active":"2024-09-21T11:45:03.193498+0000","last_peered":"2024-09-21T11:45:03.193498+0000","last_clean":"2024-09-21T11:45:03.193498+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:45:03.193498+0000","last_undegraded":"2024-09-21T11:45:03.193498+0000","last_fullsized":"2024-09-21T11:45:03.193498+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:11.930295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:11.930295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:13.258167+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"168'661","reported_seq":688,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:05.789978+0000","last_change":"2024-09-21T11:44:51.744967+0000","last_active":"2024-09-21T11:45:05.789978+0000","last_peered":"2024-09-21T11:45:05.789978+0000","last_clean":"2024-09-21T11:45:05.789978+0000","last_became_active":"2024-09-21T11:44:51.744633+0000","last_became_peered":"2024-09-21T11:44:51.744633+0000","last_unstale":"2024-09-21T11:45:05.789978+0000","last_undegraded":"2024-09-21T11:45:05.789978+0000","last_fullsized":"2024-09-21T11:45:05.789978+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:54.651804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43319296,"num_objects":661,"num_object_clones":0,"num_object_copies":1983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":661,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":265,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193389+0000","last_change":"2024-09-21T11:44:16.970995+0000","last_active":"2024-09-21T11:45:03.193389+0000","last_peered":"2024-09-21T11:45:03.193389+0000","last_clean":"2024-09-21T11:45:03.193389+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:45:03.193389+0000","last_undegraded":"2024-09-21T11:45:03.193389+0000","last_fullsized":"2024-09-21T11:45:03.193389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:44:16.970905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:16.970905+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:27.013087+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":249,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.237989+0000","last_change":"2024-09-21T11:42:37.182056+0000","last_active":"2024-09-21T11:45:01.237989+0000","last_peered":"2024-09-21T11:45:01.237989+0000","last_clean":"2024-09-21T11:45:01.237989+0000","last_became_active":"2024-09-21T11:42:37.159390+0000","last_became_peered":"2024-09-21T11:42:37.159390+0000","last_unstale":"2024-09-21T11:45:01.237989+0000","last_undegraded":"2024-09-21T11:45:01.237989+0000","last_fullsized":"2024-09-21T11:45:01.237989+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:41:34.236667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:34.236667+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:03.592456+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"114'9","reported_seq":311,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.441405+0000","last_change":"2024-09-21T11:43:29.417021+0000","last_active":"2024-09-21T11:45:00.441405+0000","last_peered":"2024-09-21T11:45:00.441405+0000","last_clean":"2024-09-21T11:45:00.441405+0000","last_became_active":"2024-09-21T11:42:37.169838+0000","last_became_peered":"2024-09-21T11:42:37.169838+0000","last_unstale":"2024-09-21T11:45:00.441405+0000","last_undegraded":"2024-09-21T11:45:00.441405+0000","last_fullsized":"2024-09-21T11:45:00.441405+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:29.416946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.416946+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":230,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.201841+0000","last_change":"2024-09-21T11:41:39.527210+0000","last_active":"2024-09-21T11:45:03.201841+0000","last_peered":"2024-09-21T11:45:03.201841+0000","last_clean":"2024-09-21T11:45:03.201841+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:45:03.201841+0000","last_undegraded":"2024-09-21T11:45:03.201841+0000","last_fullsized":"2024-09-21T11:45:03.201841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.527151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.527151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:53.785197+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":25,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239275+0000","last_change":"2024-09-21T11:44:51.412955+0000","last_active":"2024-09-21T11:45:01.239275+0000","last_peered":"2024-09-21T11:45:01.239275+0000","last_clean":"2024-09-21T11:45:01.239275+0000","last_became_active":"2024-09-21T11:44:51.412662+0000","last_became_peered":"2024-09-21T11:44:51.412662+0000","last_unstale":"2024-09-21T11:45:01.239275+0000","last_undegraded":"2024-09-21T11:45:01.239275+0000","last_fullsized":"2024-09-21T11:45:01.239275+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:19.318609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":179,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.196959+0000","last_change":"2024-09-21T11:44:43.309806+0000","last_active":"2024-09-21T11:45:03.196959+0000","last_peered":"2024-09-21T11:45:03.196959+0000","last_clean":"2024-09-21T11:45:03.196959+0000","last_became_active":"2024-09-21T11:42:37.165120+0000","last_became_peered":"2024-09-21T11:42:37.165120+0000","last_unstale":"2024-09-21T11:45:03.196959+0000","last_undegraded":"2024-09-21T11:45:03.196959+0000","last_fullsized":"2024-09-21T11:45:03.196959+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:44:43.309439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:43.309439+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:06.560680+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":247,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197613+0000","last_change":"2024-09-21T11:44:09.209354+0000","last_active":"2024-09-21T11:45:03.197613+0000","last_peered":"2024-09-21T11:45:03.197613+0000","last_clean":"2024-09-21T11:45:03.197613+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:45:03.197613+0000","last_undegraded":"2024-09-21T11:45:03.197613+0000","last_fullsized":"2024-09-21T11:45:03.197613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.209310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.209310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:13.005605+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":301,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239089+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:45:01.239089+0000","last_peered":"2024-09-21T11:45:01.239089+0000","last_clean":"2024-09-21T11:45:01.239089+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:45:01.239089+0000","last_undegraded":"2024-09-21T11:45:01.239089+0000","last_fullsized":"2024-09-21T11:45:01.239089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:32.348874+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":225,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.238571+0000","last_change":"2024-09-21T11:41:40.278404+0000","last_active":"2024-09-21T11:45:01.238571+0000","last_peered":"2024-09-21T11:45:01.238571+0000","last_clean":"2024-09-21T11:45:01.238571+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:45:01.238571+0000","last_undegraded":"2024-09-21T11:45:01.238571+0000","last_fullsized":"2024-09-21T11:45:01.238571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.278315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.278315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:23.477004+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":25,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.444537+0000","last_change":"2024-09-21T11:44:51.410839+0000","last_active":"2024-09-21T11:45:00.444537+0000","last_peered":"2024-09-21T11:45:00.444537+0000","last_clean":"2024-09-21T11:45:00.444537+0000","last_became_active":"2024-09-21T11:44:51.410302+0000","last_became_peered":"2024-09-21T11:44:51.410302+0000","last_unstale":"2024-09-21T11:45:00.444537+0000","last_undegraded":"2024-09-21T11:45:00.444537+0000","last_fullsized":"2024-09-21T11:45:00.444537+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:59.553918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":269,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239160+0000","last_change":"2024-09-21T11:41:07.065583+0000","last_active":"2024-09-21T11:45:01.239160+0000","last_peered":"2024-09-21T11:45:01.239160+0000","last_clean":"2024-09-21T11:45:01.239160+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:45:01.239160+0000","last_undegraded":"2024-09-21T11:45:01.239160+0000","last_fullsized":"2024-09-21T11:45:01.239160+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:41:07.065507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.065507+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:59.176459+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":138,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.442612+0000","last_change":"2024-09-21T11:44:44.619532+0000","last_active":"2024-09-21T11:45:00.442612+0000","last_peered":"2024-09-21T11:45:00.442612+0000","last_clean":"2024-09-21T11:45:00.442612+0000","last_became_active":"2024-09-21T11:42:37.157562+0000","last_became_peered":"2024-09-21T11:42:37.157562+0000","last_unstale":"2024-09-21T11:45:00.442612+0000","last_undegraded":"2024-09-21T11:45:00.442612+0000","last_fullsized":"2024-09-21T11:45:00.442612+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:44.619493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.619493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:03.953460+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":227,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.921674+0000","last_change":"2024-09-21T11:43:26.704117+0000","last_active":"2024-09-21T11:44:59.921674+0000","last_peered":"2024-09-21T11:44:59.921674+0000","last_clean":"2024-09-21T11:44:59.921674+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:44:59.921674+0000","last_undegraded":"2024-09-21T11:44:59.921674+0000","last_fullsized":"2024-09-21T11:44:59.921674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.704073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.704073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":128,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.921261+0000","last_change":"2024-09-21T11:43:37.819912+0000","last_active":"2024-09-21T11:44:59.921261+0000","last_peered":"2024-09-21T11:44:59.921261+0000","last_clean":"2024-09-21T11:44:59.921261+0000","last_became_active":"2024-09-21T11:42:37.155437+0000","last_became_peered":"2024-09-21T11:42:37.155437+0000","last_unstale":"2024-09-21T11:44:59.921261+0000","last_undegraded":"2024-09-21T11:44:59.921261+0000","last_fullsized":"2024-09-21T11:44:59.921261+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:52.417609+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"168'647","reported_seq":674,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:05.778539+0000","last_change":"2024-09-21T11:44:51.745506+0000","last_active":"2024-09-21T11:45:05.778539+0000","last_peered":"2024-09-21T11:45:05.778539+0000","last_clean":"2024-09-21T11:45:05.778539+0000","last_became_active":"2024-09-21T11:44:51.745200+0000","last_became_peered":"2024-09-21T11:44:51.745200+0000","last_unstale":"2024-09-21T11:45:05.778539+0000","last_undegraded":"2024-09-21T11:45:05.778539+0000","last_fullsized":"2024-09-21T11:45:05.778539+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:50.538040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42401792,"num_objects":647,"num_object_clones":0,"num_object_copies":1941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":647,"num_write_kb":41408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"140'30","reported_seq":281,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.442427+0000","last_change":"2024-09-21T11:43:37.416987+0000","last_active":"2024-09-21T11:45:00.442427+0000","last_peered":"2024-09-21T11:45:00.442427+0000","last_clean":"2024-09-21T11:45:00.442427+0000","last_became_active":"2024-09-21T11:43:32.003036+0000","last_became_peered":"2024-09-21T11:43:32.003036+0000","last_unstale":"2024-09-21T11:45:00.442427+0000","last_undegraded":"2024-09-21T11:45:00.442427+0000","last_fullsized":"2024-09-21T11:45:00.442427+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:51.735328+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":246,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193888+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:45:03.193888+0000","last_peered":"2024-09-21T11:45:03.193888+0000","last_clean":"2024-09-21T11:45:03.193888+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:45:03.193888+0000","last_undegraded":"2024-09-21T11:45:03.193888+0000","last_fullsized":"2024-09-21T11:45:03.193888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:26.875471+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":260,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197094+0000","last_change":"2024-09-21T11:42:37.193243+0000","last_active":"2024-09-21T11:45:03.197094+0000","last_peered":"2024-09-21T11:45:03.197094+0000","last_clean":"2024-09-21T11:45:03.197094+0000","last_became_active":"2024-09-21T11:42:37.164723+0000","last_became_peered":"2024-09-21T11:42:37.164723+0000","last_unstale":"2024-09-21T11:45:03.197094+0000","last_undegraded":"2024-09-21T11:45:03.197094+0000","last_fullsized":"2024-09-21T11:45:03.197094+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:42:17.142454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.142454+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:32.482483+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":244,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.237777+0000","last_change":"2024-09-21T11:44:41.300665+0000","last_active":"2024-09-21T11:45:01.237777+0000","last_peered":"2024-09-21T11:45:01.237777+0000","last_clean":"2024-09-21T11:45:01.237777+0000","last_became_active":"2024-09-21T11:42:37.163682+0000","last_became_peered":"2024-09-21T11:42:37.163682+0000","last_unstale":"2024-09-21T11:45:01.237777+0000","last_undegraded":"2024-09-21T11:45:01.237777+0000","last_fullsized":"2024-09-21T11:45:01.237777+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:54.165925+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":226,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443341+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:45:00.443341+0000","last_peered":"2024-09-21T11:45:00.443341+0000","last_clean":"2024-09-21T11:45:00.443341+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:45:00.443341+0000","last_undegraded":"2024-09-21T11:45:00.443341+0000","last_fullsized":"2024-09-21T11:45:00.443341+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:55.779491+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":25,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443397+0000","last_change":"2024-09-21T11:44:51.410845+0000","last_active":"2024-09-21T11:45:00.443397+0000","last_peered":"2024-09-21T11:45:00.443397+0000","last_clean":"2024-09-21T11:45:00.443397+0000","last_became_active":"2024-09-21T11:44:51.410553+0000","last_became_peered":"2024-09-21T11:44:51.410553+0000","last_unstale":"2024-09-21T11:45:00.443397+0000","last_undegraded":"2024-09-21T11:45:00.443397+0000","last_fullsized":"2024-09-21T11:45:00.443397+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:09.683550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":183,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197797+0000","last_change":"2024-09-21T11:42:37.188687+0000","last_active":"2024-09-21T11:45:03.197797+0000","last_peered":"2024-09-21T11:45:03.197797+0000","last_clean":"2024-09-21T11:45:03.197797+0000","last_became_active":"2024-09-21T11:42:37.161797+0000","last_became_peered":"2024-09-21T11:42:37.161797+0000","last_unstale":"2024-09-21T11:45:03.197797+0000","last_undegraded":"2024-09-21T11:45:03.197797+0000","last_fullsized":"2024-09-21T11:45:03.197797+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:00.246582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":135,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443338+0000","last_change":"2024-09-21T11:42:37.182044+0000","last_active":"2024-09-21T11:45:00.443338+0000","last_peered":"2024-09-21T11:45:00.443338+0000","last_clean":"2024-09-21T11:45:00.443338+0000","last_became_active":"2024-09-21T11:42:37.156191+0000","last_became_peered":"2024-09-21T11:42:37.156191+0000","last_unstale":"2024-09-21T11:45:00.443338+0000","last_undegraded":"2024-09-21T11:45:00.443338+0000","last_fullsized":"2024-09-21T11:45:00.443338+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":255,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.440702+0000","last_change":"2024-09-21T11:43:36.446391+0000","last_active":"2024-09-21T11:45:00.440702+0000","last_peered":"2024-09-21T11:45:00.440702+0000","last_clean":"2024-09-21T11:45:00.440702+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:45:00.440702+0000","last_undegraded":"2024-09-21T11:45:00.440702+0000","last_fullsized":"2024-09-21T11:45:00.440702+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:43:36.446341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.446341+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:50.690926+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":238,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.441688+0000","last_change":"2024-09-21T11:43:28.505134+0000","last_active":"2024-09-21T11:45:00.441688+0000","last_peered":"2024-09-21T11:45:00.441688+0000","last_clean":"2024-09-21T11:45:00.441688+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:45:00.441688+0000","last_undegraded":"2024-09-21T11:45:00.441688+0000","last_fullsized":"2024-09-21T11:45:00.441688+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.504894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.504894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:30.487435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":227,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.202317+0000","last_change":"2024-09-21T11:42:25.245624+0000","last_active":"2024-09-21T11:45:03.202317+0000","last_peered":"2024-09-21T11:45:03.202317+0000","last_clean":"2024-09-21T11:45:03.202317+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:45:03.202317+0000","last_undegraded":"2024-09-21T11:45:03.202317+0000","last_fullsized":"2024-09-21T11:45:03.202317+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.245565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.245565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.670245+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":25,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.440425+0000","last_change":"2024-09-21T11:44:51.410741+0000","last_active":"2024-09-21T11:45:00.440425+0000","last_peered":"2024-09-21T11:45:00.440425+0000","last_clean":"2024-09-21T11:45:00.440425+0000","last_became_active":"2024-09-21T11:44:51.410454+0000","last_became_peered":"2024-09-21T11:44:51.410454+0000","last_unstale":"2024-09-21T11:45:00.440425+0000","last_undegraded":"2024-09-21T11:45:00.440425+0000","last_fullsized":"2024-09-21T11:45:00.440425+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:56.841724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":272,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.238030+0000","last_change":"2024-09-21T11:43:40.607996+0000","last_active":"2024-09-21T11:45:01.238030+0000","last_peered":"2024-09-21T11:45:01.238030+0000","last_clean":"2024-09-21T11:45:01.238030+0000","last_became_active":"2024-09-21T11:42:37.164059+0000","last_became_peered":"2024-09-21T11:42:37.164059+0000","last_unstale":"2024-09-21T11:45:01.238030+0000","last_undegraded":"2024-09-21T11:45:01.238030+0000","last_fullsized":"2024-09-21T11:45:01.238030+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:43:40.607801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.607801+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":245,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.238857+0000","last_change":"2024-09-21T11:44:45.187728+0000","last_active":"2024-09-21T11:45:01.238857+0000","last_peered":"2024-09-21T11:45:01.238857+0000","last_clean":"2024-09-21T11:45:01.238857+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:45:01.238857+0000","last_undegraded":"2024-09-21T11:45:01.238857+0000","last_fullsized":"2024-09-21T11:45:01.238857+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.187650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.187650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:12.532021+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":569,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:04.433918+0000","last_change":"2024-09-21T11:43:47.494999+0000","last_active":"2024-09-21T11:45:04.433918+0000","last_peered":"2024-09-21T11:45:04.433918+0000","last_clean":"2024-09-21T11:45:04.433918+0000","last_became_active":"2024-09-21T11:42:37.166141+0000","last_became_peered":"2024-09-21T11:42:37.166141+0000","last_unstale":"2024-09-21T11:45:04.433918+0000","last_undegraded":"2024-09-21T11:45:04.433918+0000","last_fullsized":"2024-09-21T11:45:04.433918+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:47.494865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:47.494865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":299,"num_read_kb":294,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":238,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197898+0000","last_change":"2024-09-21T11:43:45.046195+0000","last_active":"2024-09-21T11:45:03.197898+0000","last_peered":"2024-09-21T11:45:03.197898+0000","last_clean":"2024-09-21T11:45:03.197898+0000","last_became_active":"2024-09-21T11:42:37.172270+0000","last_became_peered":"2024-09-21T11:42:37.172270+0000","last_unstale":"2024-09-21T11:45:03.197898+0000","last_undegraded":"2024-09-21T11:45:03.197898+0000","last_fullsized":"2024-09-21T11:45:03.197898+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.046128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.046128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:01.258149+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":132,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197549+0000","last_change":"2024-09-21T11:44:10.289926+0000","last_active":"2024-09-21T11:45:03.197549+0000","last_peered":"2024-09-21T11:45:03.197549+0000","last_clean":"2024-09-21T11:45:03.197549+0000","last_became_active":"2024-09-21T11:42:37.172645+0000","last_became_peered":"2024-09-21T11:42:37.172645+0000","last_unstale":"2024-09-21T11:45:03.197549+0000","last_undegraded":"2024-09-21T11:45:03.197549+0000","last_fullsized":"2024-09-21T11:45:03.197549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:17.731384+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":23,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.920882+0000","last_change":"2024-09-21T11:44:51.410998+0000","last_active":"2024-09-21T11:44:59.920882+0000","last_peered":"2024-09-21T11:44:59.920882+0000","last_clean":"2024-09-21T11:44:59.920882+0000","last_became_active":"2024-09-21T11:44:51.410701+0000","last_became_peered":"2024-09-21T11:44:51.410701+0000","last_unstale":"2024-09-21T11:44:59.920882+0000","last_undegraded":"2024-09-21T11:44:59.920882+0000","last_fullsized":"2024-09-21T11:44:59.920882+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:09.087938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":314,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.923642+0000","last_change":"2024-09-21T11:42:19.703071+0000","last_active":"2024-09-21T11:44:59.923642+0000","last_peered":"2024-09-21T11:44:59.923642+0000","last_clean":"2024-09-21T11:44:59.923642+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:44:59.923642+0000","last_undegraded":"2024-09-21T11:44:59.923642+0000","last_fullsized":"2024-09-21T11:44:59.923642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:42:19.703007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.703007+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:50.426220+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":238,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443959+0000","last_change":"2024-09-21T11:43:30.437283+0000","last_active":"2024-09-21T11:45:00.443959+0000","last_peered":"2024-09-21T11:45:00.443959+0000","last_clean":"2024-09-21T11:45:00.443959+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:45:00.443959+0000","last_undegraded":"2024-09-21T11:45:00.443959+0000","last_fullsized":"2024-09-21T11:45:00.443959+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:30.437080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:30.437080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":293,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.201806+0000","last_change":"2024-09-21T11:44:10.558635+0000","last_active":"2024-09-21T11:45:03.201806+0000","last_peered":"2024-09-21T11:45:03.201806+0000","last_clean":"2024-09-21T11:45:03.201806+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:45:03.201806+0000","last_undegraded":"2024-09-21T11:45:03.201806+0000","last_fullsized":"2024-09-21T11:45:03.201806+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:44:10.558032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.558032+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:29.068228+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":231,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.202180+0000","last_change":"2024-09-21T11:43:31.630737+0000","last_active":"2024-09-21T11:45:03.202180+0000","last_peered":"2024-09-21T11:45:03.202180+0000","last_clean":"2024-09-21T11:45:03.202180+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:45:03.202180+0000","last_undegraded":"2024-09-21T11:45:03.202180+0000","last_fullsized":"2024-09-21T11:45:03.202180+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:31.630670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.630670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":227,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197827+0000","last_change":"2024-09-21T11:41:44.929146+0000","last_active":"2024-09-21T11:45:03.197827+0000","last_peered":"2024-09-21T11:45:03.197827+0000","last_clean":"2024-09-21T11:45:03.197827+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:45:03.197827+0000","last_undegraded":"2024-09-21T11:45:03.197827+0000","last_fullsized":"2024-09-21T11:45:03.197827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:44.929079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:44.929079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:05.159956+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"168'634","reported_seq":661,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:05.743901+0000","last_change":"2024-09-21T11:44:51.410732+0000","last_active":"2024-09-21T11:45:05.743901+0000","last_peered":"2024-09-21T11:45:05.743901+0000","last_clean":"2024-09-21T11:45:05.743901+0000","last_became_active":"2024-09-21T11:44:51.410197+0000","last_became_peered":"2024-09-21T11:44:51.410197+0000","last_unstale":"2024-09-21T11:45:05.743901+0000","last_undegraded":"2024-09-21T11:45:05.743901+0000","last_fullsized":"2024-09-21T11:45:05.743901+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:56.633363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41549824,"num_objects":634,"num_object_clones":0,"num_object_copies":1902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":634,"num_write_kb":40576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":274,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193871+0000","last_change":"2024-09-21T11:42:37.179956+0000","last_active":"2024-09-21T11:45:03.193871+0000","last_peered":"2024-09-21T11:45:03.193871+0000","last_clean":"2024-09-21T11:45:03.193871+0000","last_became_active":"2024-09-21T11:42:37.158158+0000","last_became_peered":"2024-09-21T11:42:37.158158+0000","last_unstale":"2024-09-21T11:45:03.193871+0000","last_undegraded":"2024-09-21T11:45:03.193871+0000","last_fullsized":"2024-09-21T11:45:03.193871+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:22.041174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.041174+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:55.877734+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":248,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193483+0000","last_change":"2024-09-21T11:44:08.947090+0000","last_active":"2024-09-21T11:45:03.193483+0000","last_peered":"2024-09-21T11:45:03.193483+0000","last_clean":"2024-09-21T11:45:03.193483+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:45:03.193483+0000","last_undegraded":"2024-09-21T11:45:03.193483+0000","last_fullsized":"2024-09-21T11:45:03.193483+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.946783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.946783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:11.772867+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":231,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197717+0000","last_change":"2024-09-21T11:43:44.049478+0000","last_active":"2024-09-21T11:45:03.197717+0000","last_peered":"2024-09-21T11:45:03.197717+0000","last_clean":"2024-09-21T11:45:03.197717+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:45:03.197717+0000","last_undegraded":"2024-09-21T11:45:03.197717+0000","last_fullsized":"2024-09-21T11:45:03.197717+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:44.049424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.049424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:05.726834+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":226,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443789+0000","last_change":"2024-09-21T11:41:38.453843+0000","last_active":"2024-09-21T11:45:00.443789+0000","last_peered":"2024-09-21T11:45:00.443789+0000","last_clean":"2024-09-21T11:45:00.443789+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:45:00.443789+0000","last_undegraded":"2024-09-21T11:45:00.443789+0000","last_fullsized":"2024-09-21T11:45:00.443789+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:38.453579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:38.453579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:29.742637+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":25,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.238767+0000","last_change":"2024-09-21T11:44:51.412888+0000","last_active":"2024-09-21T11:45:01.238767+0000","last_peered":"2024-09-21T11:45:01.238767+0000","last_clean":"2024-09-21T11:45:01.238767+0000","last_became_active":"2024-09-21T11:44:51.412596+0000","last_became_peered":"2024-09-21T11:44:51.412596+0000","last_unstale":"2024-09-21T11:45:01.238767+0000","last_undegraded":"2024-09-21T11:45:01.238767+0000","last_fullsized":"2024-09-21T11:45:01.238767+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:17.836059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":289,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193936+0000","last_change":"2024-09-21T11:43:39.233466+0000","last_active":"2024-09-21T11:45:03.193936+0000","last_peered":"2024-09-21T11:45:03.193936+0000","last_clean":"2024-09-21T11:45:03.193936+0000","last_became_active":"2024-09-21T11:42:37.160867+0000","last_became_peered":"2024-09-21T11:42:37.160867+0000","last_unstale":"2024-09-21T11:45:03.193936+0000","last_undegraded":"2024-09-21T11:45:03.193936+0000","last_fullsized":"2024-09-21T11:45:03.193936+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:43:39.233386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.233386+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.064116+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":258,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.922090+0000","last_change":"2024-09-21T11:44:40.478955+0000","last_active":"2024-09-21T11:44:59.922090+0000","last_peered":"2024-09-21T11:44:59.922090+0000","last_clean":"2024-09-21T11:44:59.922090+0000","last_became_active":"2024-09-21T11:42:37.167043+0000","last_became_peered":"2024-09-21T11:42:37.167043+0000","last_unstale":"2024-09-21T11:44:59.922090+0000","last_undegraded":"2024-09-21T11:44:59.922090+0000","last_fullsized":"2024-09-21T11:44:59.922090+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:40.478885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:40.478885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:56.899757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":141,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193365+0000","last_change":"2024-09-21T11:44:45.815953+0000","last_active":"2024-09-21T11:45:03.193365+0000","last_peered":"2024-09-21T11:45:03.193365+0000","last_clean":"2024-09-21T11:45:03.193365+0000","last_became_active":"2024-09-21T11:42:37.160659+0000","last_became_peered":"2024-09-21T11:42:37.160659+0000","last_unstale":"2024-09-21T11:45:03.193365+0000","last_undegraded":"2024-09-21T11:45:03.193365+0000","last_fullsized":"2024-09-21T11:45:03.193365+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.815895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.815895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:47.947745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":234,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.441210+0000","last_change":"2024-09-21T11:43:46.516054+0000","last_active":"2024-09-21T11:45:00.441210+0000","last_peered":"2024-09-21T11:45:00.441210+0000","last_clean":"2024-09-21T11:45:00.441210+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:45:00.441210+0000","last_undegraded":"2024-09-21T11:45:00.441210+0000","last_fullsized":"2024-09-21T11:45:00.441210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:46.515782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:46.515782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":25,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.440788+0000","last_change":"2024-09-21T11:44:51.409705+0000","last_active":"2024-09-21T11:45:00.440788+0000","last_peered":"2024-09-21T11:45:00.440788+0000","last_clean":"2024-09-21T11:45:00.440788+0000","last_became_active":"2024-09-21T11:44:51.409405+0000","last_became_peered":"2024-09-21T11:44:51.409405+0000","last_unstale":"2024-09-21T11:45:00.440788+0000","last_undegraded":"2024-09-21T11:45:00.440788+0000","last_fullsized":"2024-09-21T11:45:00.440788+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.088046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":267,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193613+0000","last_change":"2024-09-21T11:44:41.809293+0000","last_active":"2024-09-21T11:45:03.193613+0000","last_peered":"2024-09-21T11:45:03.193613+0000","last_clean":"2024-09-21T11:45:03.193613+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:45:03.193613+0000","last_undegraded":"2024-09-21T11:45:03.193613+0000","last_fullsized":"2024-09-21T11:45:03.193613+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:44:41.809215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.809215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:51.601377+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":238,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443871+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:45:00.443871+0000","last_peered":"2024-09-21T11:45:00.443871+0000","last_clean":"2024-09-21T11:45:00.443871+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:45:00.443871+0000","last_undegraded":"2024-09-21T11:45:00.443871+0000","last_fullsized":"2024-09-21T11:45:00.443871+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:51.767539+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":307,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.441180+0000","last_change":"2024-09-21T11:43:38.421538+0000","last_active":"2024-09-21T11:45:00.441180+0000","last_peered":"2024-09-21T11:45:00.441180+0000","last_clean":"2024-09-21T11:45:00.441180+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:45:00.441180+0000","last_undegraded":"2024-09-21T11:45:00.441180+0000","last_fullsized":"2024-09-21T11:45:00.441180+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:43:38.421251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.421251+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:49.651159+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":235,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193689+0000","last_change":"2024-09-21T11:42:50.131895+0000","last_active":"2024-09-21T11:45:03.193689+0000","last_peered":"2024-09-21T11:45:03.193689+0000","last_clean":"2024-09-21T11:45:03.193689+0000","last_became_active":"2024-09-21T11:42:37.157732+0000","last_became_peered":"2024-09-21T11:42:37.157732+0000","last_unstale":"2024-09-21T11:45:03.193689+0000","last_undegraded":"2024-09-21T11:45:03.193689+0000","last_fullsized":"2024-09-21T11:45:03.193689+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:50.131830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:50.131830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:57.882609+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":25,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.444220+0000","last_change":"2024-09-21T11:44:51.410552+0000","last_active":"2024-09-21T11:45:00.444220+0000","last_peered":"2024-09-21T11:45:00.444220+0000","last_clean":"2024-09-21T11:45:00.444220+0000","last_became_active":"2024-09-21T11:44:51.410164+0000","last_became_peered":"2024-09-21T11:44:51.410164+0000","last_unstale":"2024-09-21T11:45:00.444220+0000","last_undegraded":"2024-09-21T11:45:00.444220+0000","last_fullsized":"2024-09-21T11:45:00.444220+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:07.178214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":266,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.237728+0000","last_change":"2024-09-21T11:42:37.192718+0000","last_active":"2024-09-21T11:45:01.237728+0000","last_peered":"2024-09-21T11:45:01.237728+0000","last_clean":"2024-09-21T11:45:01.237728+0000","last_became_active":"2024-09-21T11:42:37.168751+0000","last_became_peered":"2024-09-21T11:42:37.168751+0000","last_unstale":"2024-09-21T11:45:01.237728+0000","last_undegraded":"2024-09-21T11:45:01.237728+0000","last_fullsized":"2024-09-21T11:45:01.237728+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:15.417817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.417817+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:25.527041+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":239,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197606+0000","last_change":"2024-09-21T11:41:28.936695+0000","last_active":"2024-09-21T11:45:03.197606+0000","last_peered":"2024-09-21T11:45:03.197606+0000","last_clean":"2024-09-21T11:45:03.197606+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:45:03.197606+0000","last_undegraded":"2024-09-21T11:45:03.197606+0000","last_fullsized":"2024-09-21T11:45:03.197606+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:28.936645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:28.936645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:04.154692+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":230,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.441834+0000","last_change":"2024-09-21T11:42:18.265457+0000","last_active":"2024-09-21T11:45:00.441834+0000","last_peered":"2024-09-21T11:45:00.441834+0000","last_clean":"2024-09-21T11:45:00.441834+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:45:00.441834+0000","last_undegraded":"2024-09-21T11:45:00.441834+0000","last_fullsized":"2024-09-21T11:45:00.441834+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.265392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.265392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:54.778191+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":217,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.238814+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:45:01.238814+0000","last_peered":"2024-09-21T11:45:01.238814+0000","last_clean":"2024-09-21T11:45:01.238814+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:45:01.238814+0000","last_undegraded":"2024-09-21T11:45:01.238814+0000","last_fullsized":"2024-09-21T11:45:01.238814+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:52.126040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":25,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239390+0000","last_change":"2024-09-21T11:44:51.744832+0000","last_active":"2024-09-21T11:45:01.239390+0000","last_peered":"2024-09-21T11:45:01.239390+0000","last_clean":"2024-09-21T11:45:01.239390+0000","last_became_active":"2024-09-21T11:44:51.744226+0000","last_became_peered":"2024-09-21T11:44:51.744226+0000","last_unstale":"2024-09-21T11:45:01.239390+0000","last_undegraded":"2024-09-21T11:45:01.239390+0000","last_fullsized":"2024-09-21T11:45:01.239390+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.608182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":297,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.921837+0000","last_change":"2024-09-21T11:44:38.463592+0000","last_active":"2024-09-21T11:44:59.921837+0000","last_peered":"2024-09-21T11:44:59.921837+0000","last_clean":"2024-09-21T11:44:59.921837+0000","last_became_active":"2024-09-21T11:42:37.166373+0000","last_became_peered":"2024-09-21T11:42:37.166373+0000","last_unstale":"2024-09-21T11:44:59.921837+0000","last_undegraded":"2024-09-21T11:44:59.921837+0000","last_fullsized":"2024-09-21T11:44:59.921837+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:44:38.463535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:38.463535+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:07.934116+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":239,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.202608+0000","last_change":"2024-09-21T11:41:46.471940+0000","last_active":"2024-09-21T11:45:03.202608+0000","last_peered":"2024-09-21T11:45:03.202608+0000","last_clean":"2024-09-21T11:45:03.202608+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:45:03.202608+0000","last_undegraded":"2024-09-21T11:45:03.202608+0000","last_fullsized":"2024-09-21T11:45:03.202608+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:46.471872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:46.471872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:57.784944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":246,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.444051+0000","last_change":"2024-09-21T11:44:38.629525+0000","last_active":"2024-09-21T11:45:00.444051+0000","last_peered":"2024-09-21T11:45:00.444051+0000","last_clean":"2024-09-21T11:45:00.444051+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:45:00.444051+0000","last_undegraded":"2024-09-21T11:45:00.444051+0000","last_fullsized":"2024-09-21T11:45:00.444051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:38.629466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:38.629466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:04.842945+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":227,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.201732+0000","last_change":"2024-09-21T11:43:29.650485+0000","last_active":"2024-09-21T11:45:03.201732+0000","last_peered":"2024-09-21T11:45:03.201732+0000","last_clean":"2024-09-21T11:45:03.201732+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:45:03.201732+0000","last_undegraded":"2024-09-21T11:45:03.201732+0000","last_fullsized":"2024-09-21T11:45:03.201732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:29.650427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.650427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:35.655216+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":23,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.920574+0000","last_change":"2024-09-21T11:44:51.746053+0000","last_active":"2024-09-21T11:44:59.920574+0000","last_peered":"2024-09-21T11:44:59.920574+0000","last_clean":"2024-09-21T11:44:59.920574+0000","last_became_active":"2024-09-21T11:44:51.745609+0000","last_became_peered":"2024-09-21T11:44:51.745609+0000","last_unstale":"2024-09-21T11:44:59.920574+0000","last_undegraded":"2024-09-21T11:44:59.920574+0000","last_fullsized":"2024-09-21T11:44:59.920574+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.492338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":294,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193296+0000","last_change":"2024-09-21T11:44:43.806283+0000","last_active":"2024-09-21T11:45:03.193296+0000","last_peered":"2024-09-21T11:45:03.193296+0000","last_clean":"2024-09-21T11:45:03.193296+0000","last_became_active":"2024-09-21T11:42:37.165364+0000","last_became_peered":"2024-09-21T11:42:37.165364+0000","last_unstale":"2024-09-21T11:45:03.193296+0000","last_undegraded":"2024-09-21T11:45:03.193296+0000","last_fullsized":"2024-09-21T11:45:03.193296+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:44:43.806186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:43.806186+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:11.084871+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":262,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.202603+0000","last_change":"2024-09-21T11:44:47.741161+0000","last_active":"2024-09-21T11:45:03.202603+0000","last_peered":"2024-09-21T11:45:03.202603+0000","last_clean":"2024-09-21T11:45:03.202603+0000","last_became_active":"2024-09-21T11:42:37.165470+0000","last_became_peered":"2024-09-21T11:42:37.165470+0000","last_unstale":"2024-09-21T11:45:03.202603+0000","last_undegraded":"2024-09-21T11:45:03.202603+0000","last_fullsized":"2024-09-21T11:45:03.202603+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:47.741095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:47.741095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:51.889263+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":245,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443190+0000","last_change":"2024-09-21T11:43:40.502911+0000","last_active":"2024-09-21T11:45:00.443190+0000","last_peered":"2024-09-21T11:45:00.443190+0000","last_clean":"2024-09-21T11:45:00.443190+0000","last_became_active":"2024-09-21T11:42:37.155675+0000","last_became_peered":"2024-09-21T11:42:37.155675+0000","last_unstale":"2024-09-21T11:45:00.443190+0000","last_undegraded":"2024-09-21T11:45:00.443190+0000","last_fullsized":"2024-09-21T11:45:00.443190+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:40.502747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:40.502747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":233,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443574+0000","last_change":"2024-09-21T11:42:37.170697+0000","last_active":"2024-09-21T11:45:00.443574+0000","last_peered":"2024-09-21T11:45:00.443574+0000","last_clean":"2024-09-21T11:45:00.443574+0000","last_became_active":"2024-09-21T11:42:37.168686+0000","last_became_peered":"2024-09-21T11:42:37.168686+0000","last_unstale":"2024-09-21T11:45:00.443574+0000","last_undegraded":"2024-09-21T11:45:00.443574+0000","last_fullsized":"2024-09-21T11:45:00.443574+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:27.327920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:27.327920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:26.709081+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"168'632","reported_seq":659,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:05.790096+0000","last_change":"2024-09-21T11:44:51.410102+0000","last_active":"2024-09-21T11:45:05.790096+0000","last_peered":"2024-09-21T11:45:05.790096+0000","last_clean":"2024-09-21T11:45:05.790096+0000","last_became_active":"2024-09-21T11:44:51.409710+0000","last_became_peered":"2024-09-21T11:44:51.409710+0000","last_unstale":"2024-09-21T11:45:05.790096+0000","last_undegraded":"2024-09-21T11:45:05.790096+0000","last_fullsized":"2024-09-21T11:45:05.790096+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:05.274072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41418752,"num_objects":632,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":40448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":252,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.196728+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:45:03.196728+0000","last_peered":"2024-09-21T11:45:03.196728+0000","last_clean":"2024-09-21T11:45:03.196728+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:45:03.196728+0000","last_undegraded":"2024-09-21T11:45:03.196728+0000","last_fullsized":"2024-09-21T11:45:03.196728+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:50.273362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":251,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.202715+0000","last_change":"2024-09-21T11:42:19.352990+0000","last_active":"2024-09-21T11:45:03.202715+0000","last_peered":"2024-09-21T11:45:03.202715+0000","last_clean":"2024-09-21T11:45:03.202715+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:45:03.202715+0000","last_undegraded":"2024-09-21T11:45:03.202715+0000","last_fullsized":"2024-09-21T11:45:03.202715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:42:19.352895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.352895+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":301,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.201857+0000","last_change":"2024-09-21T11:41:43.495040+0000","last_active":"2024-09-21T11:45:03.201857+0000","last_peered":"2024-09-21T11:45:03.201857+0000","last_clean":"2024-09-21T11:45:03.201857+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:45:03.201857+0000","last_undegraded":"2024-09-21T11:45:03.201857+0000","last_fullsized":"2024-09-21T11:45:03.201857+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:41:43.494960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:43.494960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:56.764304+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":235,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.196844+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:45:03.196844+0000","last_peered":"2024-09-21T11:45:03.196844+0000","last_clean":"2024-09-21T11:45:03.196844+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:45:03.196844+0000","last_undegraded":"2024-09-21T11:45:03.196844+0000","last_fullsized":"2024-09-21T11:45:03.196844+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:22.245021+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":243,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.922524+0000","last_change":"2024-09-21T11:44:36.483068+0000","last_active":"2024-09-21T11:44:59.922524+0000","last_peered":"2024-09-21T11:44:59.922524+0000","last_clean":"2024-09-21T11:44:59.922524+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:44:59.922524+0000","last_undegraded":"2024-09-21T11:44:59.922524+0000","last_fullsized":"2024-09-21T11:44:59.922524+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:36.483023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:36.483023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:44.759897+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":258,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.444505+0000","last_change":"2024-09-21T11:42:37.201056+0000","last_active":"2024-09-21T11:45:00.444505+0000","last_peered":"2024-09-21T11:45:00.444505+0000","last_clean":"2024-09-21T11:45:00.444505+0000","last_became_active":"2024-09-21T11:42:37.171563+0000","last_became_peered":"2024-09-21T11:42:37.171563+0000","last_unstale":"2024-09-21T11:45:00.444505+0000","last_undegraded":"2024-09-21T11:45:00.444505+0000","last_fullsized":"2024-09-21T11:45:00.444505+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:41:25.310957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:25.310957+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:22.716397+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":246,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197342+0000","last_change":"2024-09-21T11:44:37.519803+0000","last_active":"2024-09-21T11:45:03.197342+0000","last_peered":"2024-09-21T11:45:03.197342+0000","last_clean":"2024-09-21T11:45:03.197342+0000","last_became_active":"2024-09-21T11:42:37.170365+0000","last_became_peered":"2024-09-21T11:42:37.170365+0000","last_unstale":"2024-09-21T11:45:03.197342+0000","last_undegraded":"2024-09-21T11:45:03.197342+0000","last_fullsized":"2024-09-21T11:45:03.197342+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:37.519750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:37.519750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:47.718785+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":235,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.202264+0000","last_change":"2024-09-21T11:43:45.551832+0000","last_active":"2024-09-21T11:45:03.202264+0000","last_peered":"2024-09-21T11:45:03.202264+0000","last_clean":"2024-09-21T11:45:03.202264+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:45:03.202264+0000","last_undegraded":"2024-09-21T11:45:03.202264+0000","last_fullsized":"2024-09-21T11:45:03.202264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.551792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.551792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:58.758423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":266,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239573+0000","last_change":"2024-09-21T11:43:32.077598+0000","last_active":"2024-09-21T11:45:01.239573+0000","last_peered":"2024-09-21T11:45:01.239573+0000","last_clean":"2024-09-21T11:45:01.239573+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:45:01.239573+0000","last_undegraded":"2024-09-21T11:45:01.239573+0000","last_fullsized":"2024-09-21T11:45:01.239573+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:11.469922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.469922+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:24.521103+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":247,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.202098+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:45:03.202098+0000","last_peered":"2024-09-21T11:45:03.202098+0000","last_clean":"2024-09-21T11:45:03.202098+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:45:03.202098+0000","last_undegraded":"2024-09-21T11:45:03.202098+0000","last_fullsized":"2024-09-21T11:45:03.202098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:25.473570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":237,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.444412+0000","last_change":"2024-09-21T11:43:38.461076+0000","last_active":"2024-09-21T11:45:00.444412+0000","last_peered":"2024-09-21T11:45:00.444412+0000","last_clean":"2024-09-21T11:45:00.444412+0000","last_became_active":"2024-09-21T11:42:37.172423+0000","last_became_peered":"2024-09-21T11:42:37.172423+0000","last_unstale":"2024-09-21T11:45:00.444412+0000","last_undegraded":"2024-09-21T11:45:00.444412+0000","last_fullsized":"2024-09-21T11:45:00.444412+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.461014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.461014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":242,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.442080+0000","last_change":"2024-09-21T11:44:45.577521+0000","last_active":"2024-09-21T11:45:00.442080+0000","last_peered":"2024-09-21T11:45:00.442080+0000","last_clean":"2024-09-21T11:45:00.442080+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:45:00.442080+0000","last_undegraded":"2024-09-21T11:45:00.442080+0000","last_fullsized":"2024-09-21T11:45:00.442080+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.577451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.577451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:54.180657+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":251,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197517+0000","last_change":"2024-09-21T11:41:33.917474+0000","last_active":"2024-09-21T11:45:03.197517+0000","last_peered":"2024-09-21T11:45:03.197517+0000","last_clean":"2024-09-21T11:45:03.197517+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:45:03.197517+0000","last_undegraded":"2024-09-21T11:45:03.197517+0000","last_fullsized":"2024-09-21T11:45:03.197517+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:33.917389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:33.917389+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:20.341726+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":245,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.238948+0000","last_change":"2024-09-21T11:44:39.310122+0000","last_active":"2024-09-21T11:45:01.238948+0000","last_peered":"2024-09-21T11:45:01.238948+0000","last_clean":"2024-09-21T11:45:01.238948+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:45:01.238948+0000","last_undegraded":"2024-09-21T11:45:01.238948+0000","last_fullsized":"2024-09-21T11:45:01.238948+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:39.310075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:39.310075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:07.017104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":291,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443991+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:45:00.443991+0000","last_peered":"2024-09-21T11:45:00.443991+0000","last_clean":"2024-09-21T11:45:00.443991+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:45:00.443991+0000","last_undegraded":"2024-09-21T11:45:00.443991+0000","last_fullsized":"2024-09-21T11:45:00.443991+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:58.796955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":236,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.194033+0000","last_change":"2024-09-21T11:43:24.300706+0000","last_active":"2024-09-21T11:45:03.194033+0000","last_peered":"2024-09-21T11:45:03.194033+0000","last_clean":"2024-09-21T11:45:03.194033+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:45:03.194033+0000","last_undegraded":"2024-09-21T11:45:03.194033+0000","last_fullsized":"2024-09-21T11:45:03.194033+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:24.300642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:24.300642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:33.811314+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":268,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.444092+0000","last_change":"2024-09-21T11:42:37.207434+0000","last_active":"2024-09-21T11:45:00.444092+0000","last_peered":"2024-09-21T11:45:00.444092+0000","last_clean":"2024-09-21T11:45:00.444092+0000","last_became_active":"2024-09-21T11:42:37.171345+0000","last_became_peered":"2024-09-21T11:42:37.171345+0000","last_unstale":"2024-09-21T11:45:00.444092+0000","last_undegraded":"2024-09-21T11:45:00.444092+0000","last_fullsized":"2024-09-21T11:45:00.444092+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:59.769469+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":247,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.202155+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:45:03.202155+0000","last_peered":"2024-09-21T11:45:03.202155+0000","last_clean":"2024-09-21T11:45:03.202155+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:45:03.202155+0000","last_undegraded":"2024-09-21T11:45:03.202155+0000","last_fullsized":"2024-09-21T11:45:03.202155+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:12.955314+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"114'9","reported_seq":321,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197342+0000","last_change":"2024-09-21T11:44:46.282850+0000","last_active":"2024-09-21T11:45:03.197342+0000","last_peered":"2024-09-21T11:45:03.197342+0000","last_clean":"2024-09-21T11:45:03.197342+0000","last_became_active":"2024-09-21T11:42:37.163771+0000","last_became_peered":"2024-09-21T11:42:37.163771+0000","last_unstale":"2024-09-21T11:45:03.197342+0000","last_undegraded":"2024-09-21T11:45:03.197342+0000","last_fullsized":"2024-09-21T11:45:03.197342+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:44:46.282790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:46.282790+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.465256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":233,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239332+0000","last_change":"2024-09-21T11:44:13.516300+0000","last_active":"2024-09-21T11:45:01.239332+0000","last_peered":"2024-09-21T11:45:01.239332+0000","last_clean":"2024-09-21T11:45:01.239332+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:45:01.239332+0000","last_undegraded":"2024-09-21T11:45:01.239332+0000","last_fullsized":"2024-09-21T11:45:01.239332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.516196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.516196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:34.796604+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":290,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193575+0000","last_change":"2024-09-21T11:44:44.791676+0000","last_active":"2024-09-21T11:45:03.193575+0000","last_peered":"2024-09-21T11:45:03.193575+0000","last_clean":"2024-09-21T11:45:03.193575+0000","last_became_active":"2024-09-21T11:42:37.154835+0000","last_became_peered":"2024-09-21T11:42:37.154835+0000","last_unstale":"2024-09-21T11:45:03.193575+0000","last_undegraded":"2024-09-21T11:45:03.193575+0000","last_fullsized":"2024-09-21T11:45:03.193575+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:44:44.791621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.791621+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:11.895258+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":240,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.193818+0000","last_change":"2024-09-21T11:43:00.245058+0000","last_active":"2024-09-21T11:45:03.193818+0000","last_peered":"2024-09-21T11:45:03.193818+0000","last_clean":"2024-09-21T11:45:03.193818+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:45:03.193818+0000","last_undegraded":"2024-09-21T11:45:03.193818+0000","last_fullsized":"2024-09-21T11:45:03.193818+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:00.244985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:00.244985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:28.823028+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":133,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.196945+0000","last_change":"2024-09-21T11:44:45.269658+0000","last_active":"2024-09-21T11:45:03.196945+0000","last_peered":"2024-09-21T11:45:03.196945+0000","last_clean":"2024-09-21T11:45:03.196945+0000","last_became_active":"2024-09-21T11:42:37.163261+0000","last_became_peered":"2024-09-21T11:42:37.163261+0000","last_unstale":"2024-09-21T11:45:03.196945+0000","last_undegraded":"2024-09-21T11:45:03.196945+0000","last_fullsized":"2024-09-21T11:45:03.196945+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.269610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.269610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:47.910495+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":218,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.440916+0000","last_change":"2024-09-21T11:40:06.353577+0000","last_active":"2024-09-21T11:45:00.440916+0000","last_peered":"2024-09-21T11:45:00.440916+0000","last_clean":"2024-09-21T11:45:00.440916+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:45:00.440916+0000","last_undegraded":"2024-09-21T11:45:00.440916+0000","last_fullsized":"2024-09-21T11:45:00.440916+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:04.613207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":174,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.922040+0000","last_change":"2024-09-21T11:43:41.808987+0000","last_active":"2024-09-21T11:44:59.922040+0000","last_peered":"2024-09-21T11:44:59.922040+0000","last_clean":"2024-09-21T11:44:59.922040+0000","last_became_active":"2024-09-21T11:42:37.167265+0000","last_became_peered":"2024-09-21T11:42:37.167265+0000","last_unstale":"2024-09-21T11:44:59.922040+0000","last_undegraded":"2024-09-21T11:44:59.922040+0000","last_fullsized":"2024-09-21T11:44:59.922040+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:41.808900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:41.808900+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:07.057274+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":239,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.202445+0000","last_change":"2024-09-21T11:43:26.586603+0000","last_active":"2024-09-21T11:45:03.202445+0000","last_peered":"2024-09-21T11:45:03.202445+0000","last_clean":"2024-09-21T11:45:03.202445+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:45:03.202445+0000","last_undegraded":"2024-09-21T11:45:03.202445+0000","last_fullsized":"2024-09-21T11:45:03.202445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.586562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.586562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":245,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443656+0000","last_change":"2024-09-21T11:44:43.803593+0000","last_active":"2024-09-21T11:45:00.443656+0000","last_peered":"2024-09-21T11:45:00.443656+0000","last_clean":"2024-09-21T11:45:00.443656+0000","last_became_active":"2024-09-21T11:42:37.166917+0000","last_became_peered":"2024-09-21T11:42:37.166917+0000","last_unstale":"2024-09-21T11:45:00.443656+0000","last_undegraded":"2024-09-21T11:45:00.443656+0000","last_fullsized":"2024-09-21T11:45:00.443656+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:43.803530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:43.803530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:46.034655+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":245,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.238030+0000","last_change":"2024-09-21T11:44:12.486771+0000","last_active":"2024-09-21T11:45:01.238030+0000","last_peered":"2024-09-21T11:45:01.238030+0000","last_clean":"2024-09-21T11:45:01.238030+0000","last_became_active":"2024-09-21T11:42:37.161589+0000","last_became_peered":"2024-09-21T11:42:37.161589+0000","last_unstale":"2024-09-21T11:45:01.238030+0000","last_undegraded":"2024-09-21T11:45:01.238030+0000","last_fullsized":"2024-09-21T11:45:01.238030+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:44:12.486698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.486698+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:33.453220+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":253,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.442839+0000","last_change":"2024-09-21T11:42:15.470041+0000","last_active":"2024-09-21T11:45:00.442839+0000","last_peered":"2024-09-21T11:45:00.442839+0000","last_clean":"2024-09-21T11:45:00.442839+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:45:00.442839+0000","last_undegraded":"2024-09-21T11:45:00.442839+0000","last_fullsized":"2024-09-21T11:45:00.442839+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:15.469421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.469421+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:35.850864+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":246,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197485+0000","last_change":"2024-09-21T11:42:37.173431+0000","last_active":"2024-09-21T11:45:03.197485+0000","last_peered":"2024-09-21T11:45:03.197485+0000","last_clean":"2024-09-21T11:45:03.197485+0000","last_became_active":"2024-09-21T11:42:37.172859+0000","last_became_peered":"2024-09-21T11:42:37.172859+0000","last_unstale":"2024-09-21T11:45:03.197485+0000","last_undegraded":"2024-09-21T11:45:03.197485+0000","last_fullsized":"2024-09-21T11:45:03.197485+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.952335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.952335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:26.406977+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":231,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.201551+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:45:03.201551+0000","last_peered":"2024-09-21T11:45:03.201551+0000","last_clean":"2024-09-21T11:45:03.201551+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:45:03.201551+0000","last_undegraded":"2024-09-21T11:45:03.201551+0000","last_fullsized":"2024-09-21T11:45:03.201551+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:31.662430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":226,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.443771+0000","last_change":"2024-09-21T11:42:12.409717+0000","last_active":"2024-09-21T11:45:00.443771+0000","last_peered":"2024-09-21T11:45:00.443771+0000","last_clean":"2024-09-21T11:45:00.443771+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:45:00.443771+0000","last_undegraded":"2024-09-21T11:45:00.443771+0000","last_fullsized":"2024-09-21T11:45:00.443771+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:12.409641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:12.409641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:33.828910+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":274,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.440968+0000","last_change":"2024-09-21T11:44:19.651059+0000","last_active":"2024-09-21T11:45:00.440968+0000","last_peered":"2024-09-21T11:45:00.440968+0000","last_clean":"2024-09-21T11:45:00.440968+0000","last_became_active":"2024-09-21T11:42:37.169482+0000","last_became_peered":"2024-09-21T11:42:37.169482+0000","last_unstale":"2024-09-21T11:45:00.440968+0000","last_undegraded":"2024-09-21T11:45:00.440968+0000","last_fullsized":"2024-09-21T11:45:00.440968+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:44:19.649203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:19.649203+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:23.681633+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":245,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239492+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:45:01.239492+0000","last_peered":"2024-09-21T11:45:01.239492+0000","last_clean":"2024-09-21T11:45:01.239492+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:45:01.239492+0000","last_undegraded":"2024-09-21T11:45:01.239492+0000","last_fullsized":"2024-09-21T11:45:01.239492+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:31.371889+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":235,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.921352+0000","last_change":"2024-09-21T11:43:39.838067+0000","last_active":"2024-09-21T11:44:59.921352+0000","last_peered":"2024-09-21T11:44:59.921352+0000","last_clean":"2024-09-21T11:44:59.921352+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:44:59.921352+0000","last_undegraded":"2024-09-21T11:44:59.921352+0000","last_fullsized":"2024-09-21T11:44:59.921352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:39.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.838002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:55.379035+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":235,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.201829+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:45:03.201829+0000","last_peered":"2024-09-21T11:45:03.201829+0000","last_clean":"2024-09-21T11:45:03.201829+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:45:03.201829+0000","last_undegraded":"2024-09-21T11:45:03.201829+0000","last_fullsized":"2024-09-21T11:45:03.201829+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:27.321443+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":131,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T11:45:03.197372+0000","last_change":"2024-09-21T11:44:11.214596+0000","last_active":"2024-09-21T11:45:03.197372+0000","last_peered":"2024-09-21T11:45:03.197372+0000","last_clean":"2024-09-21T11:45:03.197372+0000","last_became_active":"2024-09-21T11:42:37.155620+0000","last_became_peered":"2024-09-21T11:42:37.155620+0000","last_unstale":"2024-09-21T11:45:03.197372+0000","last_undegraded":"2024-09-21T11:45:03.197372+0000","last_fullsized":"2024-09-21T11:45:03.197372+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:11.214534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:11.214534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:16.652208+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":245,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:00.441201+0000","last_change":"2024-09-21T11:43:36.413767+0000","last_active":"2024-09-21T11:45:00.441201+0000","last_peered":"2024-09-21T11:45:00.441201+0000","last_clean":"2024-09-21T11:45:00.441201+0000","last_became_active":"2024-09-21T11:42:37.167248+0000","last_became_peered":"2024-09-21T11:42:37.167248+0000","last_unstale":"2024-09-21T11:45:00.441201+0000","last_undegraded":"2024-09-21T11:45:00.441201+0000","last_fullsized":"2024-09-21T11:45:00.441201+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.413725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.413725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":285,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-21T11:44:59.923660+0000","last_change":"2024-09-21T11:44:44.412386+0000","last_active":"2024-09-21T11:44:59.923660+0000","last_peered":"2024-09-21T11:44:59.923660+0000","last_clean":"2024-09-21T11:44:59.923660+0000","last_became_active":"2024-09-21T11:42:37.171253+0000","last_became_peered":"2024-09-21T11:42:37.171253+0000","last_unstale":"2024-09-21T11:44:59.923660+0000","last_undegraded":"2024-09-21T11:44:59.923660+0000","last_fullsized":"2024-09-21T11:44:59.923660+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:44:44.412304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.412304+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:55.908350+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":237,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-21T11:45:01.239447+0000","last_change":"2024-09-21T11:44:09.512919+0000","last_active":"2024-09-21T11:45:01.239447+0000","last_peered":"2024-09-21T11:45:01.239447+0000","last_clean":"2024-09-21T11:45:01.239447+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:45:01.239447+0000","last_undegraded":"2024-09-21T11:45:01.239447+0000","last_fullsized":"2024-09-21T11:45:01.239447+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.512741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.512741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:11.430500+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":216924160,"num_objects":3310,"num_object_clones":0,"num_object_copies":9930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3310,"num_write_kb":211840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":896532480,"data_stored":896532480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3310,"ondisk_log_size":3310,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":5367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":299,"num_read_kb":294,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234050,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":400048,"kb_used_data":198112,"kb_used_omap":1,"kb_used_meta":201918,"kb_avail":93345104,"statfs":{"total":95995035648,"available":95585386496,"internally_reserved":0,"allocated":202866688,"data_stored":202428124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":206764490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":163116,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":161662,"kb_avail":93582036,"statfs":{"total":95995035648,"available":95828004864,"internally_reserved":0,"allocated":1462272,"data_stored":1023051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165542342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":414120,"kb_used_data":324628,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":93331032,"statfs":{"total":95995035648,"available":95570976768,"internally_reserved":0,"allocated":332419072,"data_stored":331991631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":91617734},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,6,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920850,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":334564,"kb_used_data":208980,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":93410588,"statfs":{"total":95995035648,"available":95652442112,"internally_reserved":0,"allocated":213995520,"data_stored":213568727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128580037},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117079,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":198620,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":197758,"kb_avail":93546532,"statfs":{"total":95995035648,"available":95791648768,"internally_reserved":0,"allocated":860160,"data_stored":438339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":202504647},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":198600,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":197758,"kb_avail":93546552,"statfs":{"total":95995035648,"available":95791669248,"internally_reserved":0,"allocated":835584,"data_stored":438001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":202504645},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":55834574949,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":136748,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":135294,"kb_avail":93608404,"statfs":{"total":95995035648,"available":95855005696,"internally_reserved":0,"allocated":1462272,"data_stored":1023240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138541509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":292159488,"data_stored":292159488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":392495104,"data_stored":392495104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":211877888,"data_stored":211877888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:45:07.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool set unique_pool_1 pgp_num 16 2024-09-21T11:45:07.190 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:07.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:07.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:07.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:07.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:07.338 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:07.338 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:07.439 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:07.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:07.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:07.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:07.531 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:45:07.531 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:45:07.531 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:45:07.531 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:45:07.531 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:45:07.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:07.591 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 5 16 15631 15615 195.114 181 0.00833806 0.00511754 2024-09-21T11:45:07.606 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:07.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:07.607 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:07.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:45:07.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:07 smithi043 ceph-mon[29710]: 6.18 scrub starts 2024-09-21T11:45:07.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:07 smithi043 ceph-mon[29710]: 6.18 scrub ok 2024-09-21T11:45:07.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:07 smithi043 ceph-mon[36454]: 6.18 scrub starts 2024-09-21T11:45:07.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:07 smithi043 ceph-mon[36454]: 6.18 scrub ok 2024-09-21T11:45:07.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:07.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:07.748 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:07.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:07.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:07 smithi175 ceph-mon[32613]: 6.18 scrub starts 2024-09-21T11:45:07.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:07 smithi175 ceph-mon[32613]: 6.18 scrub ok 2024-09-21T11:45:07.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:07.889 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:07.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:07.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:07.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:08.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:08.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:08.132 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:08.166 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:08.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:08.167 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:08.267 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:08.301 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:08.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:08.302 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:08.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:45:08.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:08.472 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:08.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:08.488 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:08.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:08.492 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:08.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:08.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T11:45:08.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T11:45:08.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[29710]: 5.1f scrub starts 2024-09-21T11:45:08.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[29710]: 5.1f scrub ok 2024-09-21T11:45:08.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[29710]: pgmap v237: 148 pgs: 148 active+clean; 709 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.42k op/s 2024-09-21T11:45:08.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[29710]: 6.1a scrub starts 2024-09-21T11:45:08.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[29710]: 6.1a scrub ok 2024-09-21T11:45:08.594 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T11:45:08.594 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T11:45:08.594 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[36454]: 5.1f scrub starts 2024-09-21T11:45:08.594 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[36454]: 5.1f scrub ok 2024-09-21T11:45:08.594 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[36454]: pgmap v237: 148 pgs: 148 active+clean; 709 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.42k op/s 2024-09-21T11:45:08.594 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[36454]: 6.1a scrub starts 2024-09-21T11:45:08.594 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:08 smithi043 ceph-mon[36454]: 6.1a scrub ok 2024-09-21T11:45:08.594 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 6 15 18298 18283 190.386 166.75 0.00828694 0.00524571 2024-09-21T11:45:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:08.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:08.634 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:08.735 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:08 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T11:45:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:08 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T11:45:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:08 smithi175 ceph-mon[32613]: 5.1f scrub starts 2024-09-21T11:45:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:08 smithi175 ceph-mon[32613]: 5.1f scrub ok 2024-09-21T11:45:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:08 smithi175 ceph-mon[32613]: pgmap v237: 148 pgs: 148 active+clean; 709 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.42k op/s 2024-09-21T11:45:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:08 smithi175 ceph-mon[32613]: 6.1a scrub starts 2024-09-21T11:45:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:08 smithi175 ceph-mon[32613]: 6.1a scrub ok 2024-09-21T11:45:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:08.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:08.778 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:08.879 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:08.914 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:08.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:08.915 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:09.016 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:09.049 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:09.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:09.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:09.151 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:09.183 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:09.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:09.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:09.285 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:09.326 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:09.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:09.327 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:09.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:09.464 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:09.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:09.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:09.566 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:09.591 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 7 16 20630 20614 184 145.688 0.0102538 0.00542844 2024-09-21T11:45:09.598 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:09.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:09.599 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:09.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:09.736 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:09.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:09.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:09 smithi043 ceph-mon[36454]: 4.19 scrub starts 2024-09-21T11:45:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:09 smithi043 ceph-mon[36454]: 4.19 scrub ok 2024-09-21T11:45:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:09 smithi043 ceph-mon[36454]: 5.18 scrub starts 2024-09-21T11:45:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:09 smithi043 ceph-mon[36454]: 5.18 scrub ok 2024-09-21T11:45:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:09 smithi043 ceph-mon[29710]: 4.19 scrub starts 2024-09-21T11:45:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:09 smithi043 ceph-mon[29710]: 4.19 scrub ok 2024-09-21T11:45:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:09 smithi043 ceph-mon[29710]: 5.18 scrub starts 2024-09-21T11:45:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:09 smithi043 ceph-mon[29710]: 5.18 scrub ok 2024-09-21T11:45:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:09 smithi175 ceph-mon[32613]: 4.19 scrub starts 2024-09-21T11:45:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:09 smithi175 ceph-mon[32613]: 4.19 scrub ok 2024-09-21T11:45:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:09 smithi175 ceph-mon[32613]: 5.18 scrub starts 2024-09-21T11:45:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:09 smithi175 ceph-mon[32613]: 5.18 scrub ok 2024-09-21T11:45:09.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:09.867 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:09.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:09.868 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:09.969 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:09.999 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:10.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:10.000 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:10.031 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:45:10.031 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:45:10.070 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:45:10.070 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:45:10.101 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:10.104 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:45:10.105 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:45:10.134 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:45:10.135 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:45:10.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:10.136 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:10.136 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:10.140 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:10.170 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:45:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:45:10.208 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:45:10.208 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:45:10.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:10.242 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:45:10.242 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:45:10.279 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:45:10.279 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:45:10.280 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:10.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:10.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:10.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:10.413 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:10.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:10.414 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:10.498 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:45:10.505 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 8 pgp_num to 16 2024-09-21T11:45:10.515 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:10.543 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:10.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:10.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:10.592 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 8 16 22850 22834 178.343 138.75 0.00388007 0.00560146 2024-09-21T11:45:10.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:10.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:10.679 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:10.679 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:10.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: 6.7 scrub starts 2024-09-21T11:45:10.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: 6.7 scrub ok 2024-09-21T11:45:10.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: 4.12 scrub starts 2024-09-21T11:45:10.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: 4.12 scrub ok 2024-09-21T11:45:10.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: 4.6 scrub starts 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: 4.6 scrub ok 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: 3.f scrub starts 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: 3.f scrub ok 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: 4.d scrub starts 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: 4.d scrub ok 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: pgmap v238: 148 pgs: 148 active+clean; 709 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.1 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/670821442' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1534867417' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2809298175' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: 6.7 scrub starts 2024-09-21T11:45:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: 6.7 scrub ok 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: 4.12 scrub starts 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: 4.12 scrub ok 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: 4.6 scrub starts 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: 4.6 scrub ok 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: 3.f scrub starts 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: 3.f scrub ok 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: 4.d scrub starts 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: 4.d scrub ok 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: pgmap v238: 148 pgs: 148 active+clean; 709 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.1 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/670821442' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:45:10.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1534867417' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:10.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2809298175' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: 6.7 scrub starts 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: 6.7 scrub ok 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: 4.12 scrub starts 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: 4.12 scrub ok 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: 4.6 scrub starts 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: 4.6 scrub ok 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: 3.f scrub starts 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: 3.f scrub ok 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: 4.d scrub starts 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: 4.d scrub ok 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: pgmap v238: 148 pgs: 148 active+clean; 709 MiB data, 2.9 GiB used, 623 GiB / 626 GiB avail; 1.1 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-21T11:45:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/670821442' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:45:10.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:45:10.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1534867417' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:10.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2809298175' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:10.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:10.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:10.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:10.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:10.957 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:10.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:10.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:11.059 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:11.091 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:11.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:11.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:11.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:11.214 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:11.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:11.215 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:11.315 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:11.349 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:11.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:11.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:11.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:11.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:11.490 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:11.495 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:45:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:45:11.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:11.592 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 9 16 25195 25179 174.811 146.562 0.00318214 0.00571445 2024-09-21T11:45:11.628 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:11.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:11.630 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:11.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:45:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:45:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]': finished 2024-09-21T11:45:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2809298175' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:45:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[36454]: osdmap e169: 8 total, 8 up, 7 in 2024-09-21T11:45:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:45:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:45:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]': finished 2024-09-21T11:45:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2809298175' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:45:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[29710]: osdmap e169: 8 total, 8 up, 7 in 2024-09-21T11:45:11.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:11 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:11.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:11 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:45:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:11 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:45:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:11 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]': finished 2024-09-21T11:45:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:11 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2809298175' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:45:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:11 smithi175 ceph-mon[32613]: osdmap e169: 8 total, 8 up, 7 in 2024-09-21T11:45:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:11 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:11.769 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:11.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:11.770 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:11.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:11.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:11.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:12.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:12.053 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:12.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:12.055 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:12.136 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:45:12.136 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:45:12.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:12.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:12.193 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:12.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:12.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:12.332 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:12.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:12.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:12.433 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:12.467 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:12.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:12.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:12.569 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:12.592 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 10 16 27662 27646 172.748 154.188 0.00838218 0.00578319 2024-09-21T11:45:12.602 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:12.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:12.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:12.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:12.743 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:12.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:12.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:12 smithi175 ceph-mon[32613]: 5.17 scrub starts 2024-09-21T11:45:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:12 smithi175 ceph-mon[32613]: 5.17 scrub ok 2024-09-21T11:45:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:12 smithi175 ceph-mon[32613]: 5.f scrub starts 2024-09-21T11:45:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:12 smithi175 ceph-mon[32613]: 5.f scrub ok 2024-09-21T11:45:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:12 smithi175 ceph-mon[32613]: pgmap v240: 148 pgs: 148 active+clean; 1.5 GiB data, 5.3 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 174 MiB/s wr, 2.78k op/s 2024-09-21T11:45:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:12 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T11:45:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:12 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T11:45:12.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:12.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[36454]: 5.17 scrub starts 2024-09-21T11:45:12.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[36454]: 5.17 scrub ok 2024-09-21T11:45:12.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[36454]: 5.f scrub starts 2024-09-21T11:45:12.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[36454]: 5.f scrub ok 2024-09-21T11:45:12.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[36454]: pgmap v240: 148 pgs: 148 active+clean; 1.5 GiB data, 5.3 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 174 MiB/s wr, 2.78k op/s 2024-09-21T11:45:12.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T11:45:12.851 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T11:45:12.851 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[29710]: 5.17 scrub starts 2024-09-21T11:45:12.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[29710]: 5.17 scrub ok 2024-09-21T11:45:12.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[29710]: 5.f scrub starts 2024-09-21T11:45:12.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[29710]: 5.f scrub ok 2024-09-21T11:45:12.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[29710]: pgmap v240: 148 pgs: 148 active+clean; 1.5 GiB data, 5.3 GiB used, 621 GiB / 626 GiB avail; 1.2 KiB/s rd, 174 MiB/s wr, 2.78k op/s 2024-09-21T11:45:12.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T11:45:12.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:12 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T11:45:12.891 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:12.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:12.892 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:12.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:13.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:13.031 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:13.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:13.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:13.171 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:13.171 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:13.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:13.315 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:13.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:13.316 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:13.417 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:13.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:13.455 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:13.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:13.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:13.592 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 11 16 30166 30150 171.268 156.5 0.00437383 0.0058335 2024-09-21T11:45:13.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:13.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:13.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:13.667 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:45:13.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:13.736 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:13.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:13.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:13 smithi043 ceph-mon[29710]: 5.1e scrub starts 2024-09-21T11:45:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:13 smithi043 ceph-mon[29710]: 5.1e scrub ok 2024-09-21T11:45:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:13 smithi043 ceph-mon[36454]: 5.1e scrub starts 2024-09-21T11:45:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:13 smithi043 ceph-mon[36454]: 5.1e scrub ok 2024-09-21T11:45:13.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:13 smithi175 ceph-mon[32613]: 5.1e scrub starts 2024-09-21T11:45:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:13 smithi175 ceph-mon[32613]: 5.1e scrub ok 2024-09-21T11:45:13.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:13.872 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:13.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:13.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:13.974 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:14.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:14.018 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:14.018 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:14.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:45:14.107 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:14.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:14.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:14.178 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:14.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:14.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:14.313 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:14.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:14.414 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:14.451 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:14.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:14.452 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:14.547 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:14.553 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:14.592 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:14.592 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 12 16 32380 32364 168.526 138.375 0.00501355 0.00592847 2024-09-21T11:45:14.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:14.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:14.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:14.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:14.733 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:14.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:14.771 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:14 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T11:45:14.771 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:14 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T11:45:14.771 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:14 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:45:14.771 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:14 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:45:14.771 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:14 smithi175 ceph-mon[32613]: pgmap v241: 148 pgs: 148 active+clean; 1.5 GiB data, 5.3 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 150 MiB/s wr, 2.39k op/s 2024-09-21T11:45:14.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:14.864 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:14 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T11:45:14.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:14 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T11:45:14.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:14 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:45:14.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:14 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:45:14.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:14 smithi043 ceph-mon[36454]: pgmap v241: 148 pgs: 148 active+clean; 1.5 GiB data, 5.3 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 150 MiB/s wr, 2.39k op/s 2024-09-21T11:45:14.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:14 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T11:45:14.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:14 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T11:45:14.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:14 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:45:14.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:14 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:45:14.866 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:14 smithi043 ceph-mon[29710]: pgmap v241: 148 pgs: 148 active+clean; 1.5 GiB data, 5.3 GiB used, 621 GiB / 626 GiB avail; 1023 B/s rd, 150 MiB/s wr, 2.39k op/s 2024-09-21T11:45:14.879 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:14.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:14.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:14.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:45:14.928 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:45:14.928 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:45:14.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:15.024 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:15.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:15.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:15.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:15.163 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:15.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:15.164 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:15.236 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:45:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:45:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:45:15.264 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:15.300 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:15.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:15.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:15.402 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:15.439 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:15.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:15.440 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:15.538 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:45:15.538 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:45:15.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:15.572 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:45:15.572 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:45:15.575 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:15.575 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:15.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:15.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:15.592 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 13 16 34204 34188 164.332 114 0.00755416 0.00607996 2024-09-21T11:45:15.606 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:45:15.606 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:45:15.640 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:45:15.640 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:45:15.673 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:45:15.673 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:45:15.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:15.712 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:45:15.712 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:45:15.715 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:15.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:15.716 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:15.746 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:45:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:45:15.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:15 smithi175 ceph-mon[32613]: osdmap e170: 8 total, 8 up, 7 in 2024-09-21T11:45:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:15 smithi175 ceph-mon[32613]: 6.1c scrub starts 2024-09-21T11:45:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:15 smithi175 ceph-mon[32613]: 6.1c scrub ok 2024-09-21T11:45:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1587162880' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:15.765 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:45:15.785 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:45:15.785 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:45:15.785 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:15 smithi043 ceph-mon[36454]: osdmap e170: 8 total, 8 up, 7 in 2024-09-21T11:45:15.785 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:15 smithi043 ceph-mon[36454]: 6.1c scrub starts 2024-09-21T11:45:15.785 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:15 smithi043 ceph-mon[36454]: 6.1c scrub ok 2024-09-21T11:45:15.785 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1587162880' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:15.786 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:15 smithi043 ceph-mon[29710]: osdmap e170: 8 total, 8 up, 7 in 2024-09-21T11:45:15.786 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:15 smithi043 ceph-mon[29710]: 6.1c scrub starts 2024-09-21T11:45:15.786 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:15 smithi043 ceph-mon[29710]: 6.1c scrub ok 2024-09-21T11:45:15.786 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1587162880' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:15.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:15.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:15.858 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:15.858 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:15.959 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:16.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:16.007 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:16.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:16.108 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:16.145 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:16.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:16.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:16.209 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:16.247 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:16.279 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:16.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:16.280 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:16.381 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:16.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:16.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:16.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:16.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:16.537 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:45:16.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 0 1 2024-09-21T11:45:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:16.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:16.562 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:16.592 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 14 16 35961 35945 160.437 109.812 0.0191193 0.00622602 2024-09-21T11:45:16.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:16.709 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:16.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:16.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:16.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:16.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:16.858 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:16.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:16.889 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:16 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:16.890 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:16 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:16.890 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:16 smithi043 ceph-mon[29710]: pgmap v243: 148 pgs: 148 active+clean; 1.7 GiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 125 MiB/s wr, 2.01k op/s 2024-09-21T11:45:16.890 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:16 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:16.890 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:16 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:16.890 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:16 smithi043 ceph-mon[36454]: pgmap v243: 148 pgs: 148 active+clean; 1.7 GiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 125 MiB/s wr, 2.01k op/s 2024-09-21T11:45:16.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:16 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:16.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:16 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:16.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:16 smithi175 ceph-mon[32613]: pgmap v243: 148 pgs: 148 active+clean; 1.7 GiB data, 6.2 GiB used, 620 GiB / 626 GiB avail; 1.2 KiB/s rd, 125 MiB/s wr, 2.01k op/s 2024-09-21T11:45:16.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:17.006 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:17.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:17.007 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:17.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:17.155 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:17.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:17.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:17.196 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:45:17.258 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:17.282 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:45:17.282 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:45:17.282 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:45:17.282 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:45:17.282 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:45:17.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:17.298 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:17.298 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:17.399 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:17.432 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:17.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:17.433 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:17.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:17.570 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:17.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:17.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:17.592 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 15 16 37772 37756 157.287 113.188 0.00505769 0.0063527 2024-09-21T11:45:17.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:17.710 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:17.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:17.712 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:17.812 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:17.847 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:17.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:17.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:17.949 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:17.983 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:17.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:17.984 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[29710]: 3.2 scrub starts 2024-09-21T11:45:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[29710]: 3.2 scrub ok 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3093880675' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[29710]: osdmap e171: 8 total, 8 up, 7 in 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[36454]: 3.2 scrub starts 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[36454]: 3.2 scrub ok 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3093880675' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:45:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:17 smithi043 ceph-mon[36454]: osdmap e171: 8 total, 8 up, 7 in 2024-09-21T11:45:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:17 smithi175 ceph-mon[32613]: 3.2 scrub starts 2024-09-21T11:45:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:17 smithi175 ceph-mon[32613]: 3.2 scrub ok 2024-09-21T11:45:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:17 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3093880675' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:45:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:17 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:45:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:17 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:17 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:45:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:17 smithi175 ceph-mon[32613]: osdmap e171: 8 total, 8 up, 7 in 2024-09-21T11:45:18.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:18.156 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:18.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:18.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:18.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:45:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:18.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:18.293 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:18.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:18.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:18.395 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:18.435 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:18.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:18.436 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:18.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:18.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:18.576 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:18.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:18.592 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 16 16 39789 39773 155.335 126.062 0.00905989 0.00643156 2024-09-21T11:45:18.641 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:18.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:18.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:18.715 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:18.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:18.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:18.839 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:18.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:18.841 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:18.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:18 smithi175 ceph-mon[32613]: 4.0 scrub starts 2024-09-21T11:45:18.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:18 smithi175 ceph-mon[32613]: 4.0 scrub ok 2024-09-21T11:45:18.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:18 smithi175 ceph-mon[32613]: pgmap v245: 148 pgs: 148 active+clean; 2.1 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.4 KiB/s rd, 202 MiB/s wr, 3.23k op/s 2024-09-21T11:45:18.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:18.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:18.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:18.975 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:18.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:18 smithi043 ceph-mon[29710]: 4.0 scrub starts 2024-09-21T11:45:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:18 smithi043 ceph-mon[29710]: 4.0 scrub ok 2024-09-21T11:45:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:18 smithi043 ceph-mon[29710]: pgmap v245: 148 pgs: 148 active+clean; 2.1 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.4 KiB/s rd, 202 MiB/s wr, 3.23k op/s 2024-09-21T11:45:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:18 smithi043 ceph-mon[36454]: 4.0 scrub starts 2024-09-21T11:45:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:18 smithi043 ceph-mon[36454]: 4.0 scrub ok 2024-09-21T11:45:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:18 smithi043 ceph-mon[36454]: pgmap v245: 148 pgs: 148 active+clean; 2.1 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 1.4 KiB/s rd, 202 MiB/s wr, 3.23k op/s 2024-09-21T11:45:19.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:19.110 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:19.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:19.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:19.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:19.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:19.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:19.245 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:45:19.245 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:45:19.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:45:19.245 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:101051 7.1 7:86930eb5:::benchmark_data_smithi043_70646_object101050:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:28.760976+0000", 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 170.479306775, 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.058579606999999999, 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 101051 2024-09-21T11:45:19.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.760976+0000", 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.760976+0000", 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.760977+0000", 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.328e-06 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761027+0000", 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9629000000000002e-05 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761039+0000", 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2199e-05 2024-09-21T11:45:19.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761044+0000", 2024-09-21T11:45:19.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T11:45:19.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761093+0000", 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.871e-05 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761199+0000", 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105896 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.761295+0000", 2024-09-21T11:45:19.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5965999999999996e-05 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.764871+0000", 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035761209999999998 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.764946+0000", 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4849000000000001e-05 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.819492+0000", 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054546866999999999 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.819528+0000", 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5142000000000003e-05 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.819555+0000", 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7747999999999999e-05 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.252 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:109597 7.1 7:80c96037:::benchmark_data_smithi043_70646_object109596:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:39.660809+0000", 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 159.579473396, 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.062593842999999996, 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.253 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 109597 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660809+0000", 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660809+0000", 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660811+0000", 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8390000000000001e-06 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660854+0000", 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3409999999999999e-05 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660873+0000", 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8734000000000001e-05 2024-09-21T11:45:19.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660882+0000", 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1509999999999993e-06 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660963+0000", 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0988000000000005e-05 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.661083+0000", 2024-09-21T11:45:19.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000119543 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.661218+0000", 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013496299999999999 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.663870+0000", 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002652387 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.663883+0000", 2024-09-21T11:45:19.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2553e-05 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.723331+0000", 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.059448190999999997 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.723373+0000", 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1659999999999998e-05 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.723403+0000", 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0423999999999999e-05 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:124246 7.1 7:8f040438:::benchmark_data_smithi043_70646_object6838:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:57.176009+0000", 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 142.064273781, 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.060312221999999999, 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.259 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 124246 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.176009+0000", 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.176009+0000", 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.176010+0000", 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4799999999999997e-07 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.176016+0000", 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.703e-06 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.176025+0000", 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1499999999999999e-06 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.176031+0000", 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3609999999999997e-06 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.178433+0000", 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024022739999999998 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.178537+0000", 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103708 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.178638+0000", 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100806 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.183070+0000", 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0044325229999999998 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.183684+0000", 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00061387300000000002 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.236222+0000", 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.052537672000000001 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.236278+0000", 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5795999999999999e-05 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:57.236321+0000", 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3408000000000002e-05 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.264 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126040 7.1 7:819ca1b9:::benchmark_data_smithi043_70646_object8632:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:58.958218+0000", 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.28206440899999, 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.060993535000000002, 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126040 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.265 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.958218+0000", 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.958218+0000", 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.958219+0000", 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.178e-06 2024-09-21T11:45:19.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.958224+0000", 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6890000000000003e-06 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.958235+0000", 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1032999999999999e-05 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.958242+0000", 2024-09-21T11:45:19.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0559999999999999e-06 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.960139+0000", 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018968100000000001 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.960253+0000", 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011366099999999999 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.960614+0000", 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000361944 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.963235+0000", 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026204100000000001 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:58.964471+0000", 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001236202 2024-09-21T11:45:19.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.019111+0000", 2024-09-21T11:45:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054639508000000003 2024-09-21T11:45:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.019166+0000", 2024-09-21T11:45:19.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5726999999999997e-05 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.019212+0000", 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5317e-05 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126212 7.1 7:86c8b838:::benchmark_data_smithi043_70646_object8804:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:45:19.271 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.194683+0000", 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 140.04559923400001, 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.097383589000000007, 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126212 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194683+0000", 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194683+0000", 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194685+0000", 2024-09-21T11:45:19.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9920000000000002e-06 2024-09-21T11:45:19.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194691+0000", 2024-09-21T11:45:19.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4809999999999997e-06 2024-09-21T11:45:19.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194702+0000", 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1559999999999999e-05 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194711+0000", 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5250000000000005e-06 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194786+0000", 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4846999999999997e-05 2024-09-21T11:45:19.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194894+0000", 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010852400000000001 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.195001+0000", 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106406 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197371+0000", 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002370367 2024-09-21T11:45:19.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197897+0000", 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052575600000000005 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291967+0000", 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.094070592999999994 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.292026+0000", 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8671000000000001e-05 2024-09-21T11:45:19.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.292067+0000", 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0867e-05 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:142176 7.1 7:8da4a2c6:::benchmark_data_smithi043_70646_object24768:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:17.045340+0000", 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 122.194942285, 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.064293720999999998, 2024-09-21T11:45:19.278 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 142176 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.045340+0000", 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.045340+0000", 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.045342+0000", 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4449999999999999e-06 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.045347+0000", 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3480000000000003e-06 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.045357+0000", 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.007e-05 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.045362+0000", 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1889999999999998e-06 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.045404+0000", 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2021999999999998e-05 2024-09-21T11:45:19.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.045483+0000", 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8738000000000004e-05 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.045551+0000", 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8034999999999995e-05 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.047866+0000", 2024-09-21T11:45:19.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023149860000000002 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.048026+0000", 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000160171 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.109529+0000", 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.061503100999999998 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.109587+0000", 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7331999999999999e-05 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.109634+0000", 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7284000000000003e-05 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:142181 7.1 7:86628d3a:::benchmark_data_smithi043_70646_object24773:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:17.049349+0000", 2024-09-21T11:45:19.284 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 122.19093327500001, 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.060420884000000001, 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 142181 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.049349+0000", 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.049349+0000", 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.049351+0000", 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.454e-06 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.049356+0000", 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1760000000000001e-06 2024-09-21T11:45:19.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.049368+0000", 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2038000000000001e-05 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.049375+0000", 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4800000000000004e-06 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.049438+0000", 2024-09-21T11:45:19.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2490000000000006e-05 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.049549+0000", 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111319 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.049664+0000", 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115071 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.052371+0000", 2024-09-21T11:45:19.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027065660000000001 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.053192+0000", 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00082151700000000002 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.109676+0000", 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.056484120999999998 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.109732+0000", 2024-09-21T11:45:19.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6001000000000001e-05 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:17.109770+0000", 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7651000000000003e-05 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:184847 7.1 7:83a2e03d:::benchmark_data_smithi043_70646_object67439:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:57.522550+0000", 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 81.717732728000001, 2024-09-21T11:45:19.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.073875239999999995, 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 184847 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.522550+0000", 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.522550+0000", 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.522552+0000", 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2309999999999999e-06 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.522557+0000", 2024-09-21T11:45:19.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0209999999999999e-06 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.522570+0000", 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2497000000000001e-05 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.522577+0000", 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3339999999999996e-06 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.522648+0000", 2024-09-21T11:45:19.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1217000000000006e-05 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.522769+0000", 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000120654 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.522886+0000", 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011728 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.529363+0000", 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0064767920000000003 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.529869+0000", 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00050569600000000003 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.596353+0000", 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.066484669999999996 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.596388+0000", 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4814000000000002e-05 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.596425+0000", 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7033999999999998e-05 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:185199 7.1 7:848115c2:::benchmark_data_smithi043_70646_object67791:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:45:19.296 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:57.851664+0000", 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 81.388618063999999, 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092637500999999997, 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 185199 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851664+0000", 2024-09-21T11:45:19.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851664+0000", 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851666+0000", 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2249999999999999e-06 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851671+0000", 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8919999999999999e-06 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851686+0000", 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5675000000000002e-05 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851694+0000", 2024-09-21T11:45:19.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9489999999999996e-06 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851811+0000", 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116335 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851954+0000", 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143941 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.852069+0000", 2024-09-21T11:45:19.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011415499999999999 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.854468+0000", 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023996629999999998 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.854814+0000", 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00034581899999999998 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.944168+0000", 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.089353558999999999 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.944259+0000", 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0932000000000003e-05 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.944302+0000", 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3356000000000003e-05 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.302 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221222 7.1 7:8e904c52:::benchmark_data_smithi043_70646_object103814:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.183001+0000", 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.057282016000002, 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090847370999999996, 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.303 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221222 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.183001+0000", 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.183001+0000", 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.183002+0000", 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.539e-06 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.183008+0000", 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7019999999999998e-06 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.183022+0000", 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4419999999999999e-05 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.183030+0000", 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6499999999999996e-06 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.183092+0000", 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2014999999999995e-05 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.183205+0000", 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011357999999999999 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.183311+0000", 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105665 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.188336+0000", 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0050250399999999997 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.189064+0000", 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072764699999999999 2024-09-21T11:45:19.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.273728+0000", 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.084663745999999998 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.273793+0000", 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5089e-05 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.273848+0000", 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5278000000000002e-05 2024-09-21T11:45:19.308 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221225 7.1 7:81c1cf62:::benchmark_data_smithi043_70646_object103817:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.185689+0000", 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.054593527000002, 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092272482000000003, 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.309 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221225 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185689+0000", 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185689+0000", 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185690+0000", 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4079999999999999e-06 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185696+0000", 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3789999999999997e-06 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185708+0000", 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2383e-05 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185714+0000", 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5709999999999997e-06 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185775+0000", 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1477999999999997e-05 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185891+0000", 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011618800000000001 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186010+0000", 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118808 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.188391+0000", 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023805369999999998 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.189115+0000", 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072425500000000004 2024-09-21T11:45:19.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.277871+0000", 2024-09-21T11:45:19.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.088755954999999997 2024-09-21T11:45:19.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.277921+0000", 2024-09-21T11:45:19.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9827999999999998e-05 2024-09-21T11:45:19.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.277961+0000", 2024-09-21T11:45:19.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0692000000000003e-05 2024-09-21T11:45:19.315 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.315 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221228 7.1 7:8b8c499f:::benchmark_data_smithi043_70646_object103820:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.190021+0000", 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 52.050261231, 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.088087843999999998, 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.316 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221228 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.190021+0000", 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.190021+0000", 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.190022+0000", 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0529999999999999e-06 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.190026+0000", 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0040000000000001e-06 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.190036+0000", 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5289999999999997e-06 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.190041+0000", 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9200000000000003e-06 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.190086+0000", 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5225000000000002e-05 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.190165+0000", 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9095000000000001e-05 2024-09-21T11:45:19.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.190236+0000", 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0908999999999995e-05 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.192214+0000", 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019784540000000002 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.192502+0000", 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028794200000000002 2024-09-21T11:45:19.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.278010+0000", 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.085507591999999993 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.278068+0000", 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8001999999999998e-05 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.278109+0000", 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1118999999999997e-05 2024-09-21T11:45:19.321 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221757 7.1 7:81ca14e2:::benchmark_data_smithi043_70646_object104349:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.700876+0000", 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.539406841999998, 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072947736999999999, 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.322 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221757 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700876+0000", 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700876+0000", 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700877+0000", 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.697e-06 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700882+0000", 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0100000000000003e-06 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700898+0000", 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5318000000000001e-05 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700905+0000", 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1439999999999997e-06 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700966+0000", 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1518000000000003e-05 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.701066+0000", 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9611000000000006e-05 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.701182+0000", 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011577 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.707127+0000", 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00594481 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.708962+0000", 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001835873 2024-09-21T11:45:19.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.773738+0000", 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.064775349999999995 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.773779+0000", 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1254000000000002e-05 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.773823+0000", 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4382000000000003e-05 2024-09-21T11:45:19.327 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221759 7.1 7:867423fb:::benchmark_data_smithi043_70646_object104351:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.701179+0000", 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 51.539103584000003, 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072957455000000004, 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.328 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221759 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.701179+0000", 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.701179+0000", 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.701180+0000", 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1230000000000001e-06 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.701185+0000", 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6739999999999996e-06 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.701196+0000", 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0978999999999999e-05 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.701203+0000", 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6260000000000003e-06 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.703543+0000", 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023395120000000002 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.703649+0000", 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106405 2024-09-21T11:45:19.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.703763+0000", 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114042 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.708997+0000", 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0052334570000000004 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.709006+0000", 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7049999999999994e-06 2024-09-21T11:45:19.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.774079+0000", 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.065072029000000003 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.774106+0000", 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7141e-05 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.774136+0000", 2024-09-21T11:45:19.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0762000000000001e-05 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:227381 7.1 7:8c6f9997:::benchmark_data_smithi043_70646_object109973:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:32.322186+0000", 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.918096886000001, 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.070874351000000002, 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.334 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 227381 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.322186+0000", 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.322186+0000", 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.322187+0000", 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6649999999999999e-06 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.322192+0000", 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3800000000000004e-06 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.322201+0000", 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4830000000000005e-06 2024-09-21T11:45:19.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.322208+0000", 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.238e-06 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.322252+0000", 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3883e-05 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.322356+0000", 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103933 2024-09-21T11:45:19.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.322439+0000", 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3176000000000005e-05 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324342+0000", 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001903065 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385805+0000", 2024-09-21T11:45:19.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.061462343000000003 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.392904+0000", 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0070993860000000001 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.393010+0000", 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105784 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.393060+0000", 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0015000000000001e-05 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.14805.0:227375 7.9 e153/130 7:93700504:::benchmark_data_smithi043_70646_object109967:head v 153'14075, mlcod=153'14069)", 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:32.324420+0000", 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.915862824000001, 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.06143614, 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324420+0000", 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324420+0000", 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324420+0000", 2024-09-21T11:45:19.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4199999999999995e-07 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324434+0000", 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3893999999999999e-05 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324442+0000", 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5939999999999996e-06 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324447+0000", 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6750000000000001e-06 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324480+0000", 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3949000000000003e-05 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324498+0000", 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7965000000000002e-05 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385814+0000", 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.061315863999999998 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385856+0000", 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1557000000000002e-05 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.14805.0:227379 7.3 e153/82 7:c1473e8d:::benchmark_data_smithi043_70646_object109971:head v 153'14109, mlcod=153'14106)", 2024-09-21T11:45:19.344 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:32.324623+0000", 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.915659351999999, 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.061002894000000002, 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324623+0000", 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324623+0000", 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324624+0000", 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2799999999999996e-07 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324627+0000", 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.551e-06 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324633+0000", 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2660000000000001e-06 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324636+0000", 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.501e-06 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324676+0000", 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0122e-05 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.324688+0000", 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.151e-05 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.326273+0000", 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001585709 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385626+0000", 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.059352706999999998 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.348 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.14805.0:227386 7.8 e153/82 7:1030955b:::benchmark_data_smithi043_70646_object109978:head v 153'14297, mlcod=153'14295)", 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:32.325775+0000", 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.914507983999997, 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.059914582000000001, 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.325775+0000", 2024-09-21T11:45:19.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.325775+0000", 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.325775+0000", 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2099999999999996e-07 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.325781+0000", 2024-09-21T11:45:19.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0290000000000002e-06 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.325791+0000", 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4450000000000002e-06 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.325797+0000", 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2550000000000003e-06 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.325843+0000", 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5605000000000001e-05 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.325859+0000", 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6816999999999998e-05 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385642+0000", 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.059782881000000003 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385689+0000", 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6829000000000002e-05 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:227390 7.1 7:80222cc6:::benchmark_data_smithi043_70646_object109982:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:32.327298+0000", 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.912984891999997, 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.065953014000000004, 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.353 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 227390 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.327298+0000", 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.327298+0000", 2024-09-21T11:45:19.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.327299+0000", 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.282e-06 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.327303+0000", 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0559999999999998e-06 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385239+0000", 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057936399 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385276+0000", 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6409000000000003e-05 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385953+0000", 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00067716399999999996 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.386077+0000", 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012384599999999999 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.386188+0000", 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111172 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.389332+0000", 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0031437909999999999 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:19.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.389793+0000", 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000461156 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.393131+0000", 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003338471 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.393210+0000", 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8453000000000003e-05 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.393251+0000", 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0815000000000001e-05 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.14805.0:227383 7.9 e153/130 7:9066583c:::benchmark_data_smithi043_70646_object109975:head v 153'14076, mlcod=153'14073)", 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:32.327302+0000", 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 46.912980802999996, 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.063158221000000001, 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:19.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.327302+0000", 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.327302+0000", 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.327302+0000", 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2699999999999999e-07 2024-09-21T11:45:19.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.327307+0000", 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8450000000000002e-06 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385429+0000", 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.058121233000000001 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385437+0000", 2024-09-21T11:45:19.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2029999999999992e-06 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385904+0000", 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046757999999999999 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.385927+0000", 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2382999999999999e-05 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.390428+0000", 2024-09-21T11:45:19.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045010329999999998 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:32.390460+0000", 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2217e-05 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:19.363 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:45:19.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:19.400 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:19.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:19.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:19.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:45:19.502 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:19.534 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:19.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:19.535 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:19.592 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 17 16 41543 41527 152.645 109.625 0.00528492 0.00654536 2024-09-21T11:45:19.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:19.677 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:19.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:19.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:19.778 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:19.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:19.801 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:19.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:19.942 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:19.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:19.943 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:20.044 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:20.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:45:20.079 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:20.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:20.080 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:20.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:20.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:20.227 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:20.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:20.259 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:45:20.328 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:20.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:20.375 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:20.375 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:20.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:20.513 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:20.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:20.526 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:20.526 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:20.593 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 18 15 43244 43229 150.074 106.375 0.00324832 0.00665728 2024-09-21T11:45:20.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:20.660 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:20.666 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:20.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:20.668 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:20.768 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:20.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:20.791 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:20.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T11:45:20.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:20.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:20 smithi043 ceph-mon[29710]: pgmap v246: 148 pgs: 148 active+clean; 2.1 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-21T11:45:20.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1221483250' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:45:20.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:20 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:45:20.902 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:20 smithi043 ceph-mon[36454]: pgmap v246: 148 pgs: 148 active+clean; 2.1 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-21T11:45:20.902 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1221483250' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:45:20.902 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:20 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:45:20.928 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:20.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:20.929 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:21.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:21.054 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:21.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:21.055 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:21.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:20 smithi175 ceph-mon[32613]: pgmap v246: 148 pgs: 148 active+clean; 2.1 GiB data, 7.4 GiB used, 618 GiB / 626 GiB avail; 639 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-21T11:45:21.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1221483250' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:45:21.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:20 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:45:21.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:45:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:45:21.156 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:21.192 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:21.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:21.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:21.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:21.295 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:45:21.295 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:45:21.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.295 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.15150.0:44194 8.d e172/160 ondisk, result = 0)", 2024-09-21T11:45:21.295 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.289884+0000", 2024-09-21T11:45:21.295 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.00016194399999999999, 2024-09-21T11:45:21.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000183041, 2024-09-21T11:45:21.295 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289884+0000", 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289884+0000", 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289885+0000", 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9400000000000004e-07 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289889+0000", 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7419999999999999e-06 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289897+0000", 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6130000000000004e-06 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289907+0000", 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3060000000000003e-06 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289996+0000", 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9159999999999993e-05 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.290004+0000", 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.072e-06 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.298 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:44194 8.d 8:b6f43447:::benchmark_data_smithi043_97042_object44193:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.276046+0000", 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.013999987, 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.014079714, 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:21.299 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 44194 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.276046+0000", 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.276046+0000", 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.276048+0000", 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5090000000000001e-06 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.276087+0000", 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9087e-05 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.276099+0000", 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2523e-05 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.276107+0000", 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9410000000000007e-06 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.278104+0000", 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019969160000000001 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.278221+0000", 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116597 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.278345+0000", 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012385400000000001 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.280740+0000", 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023957689999999999 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.280872+0000", 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013137399999999999 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.303 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.290010+0000", 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091379589999999993 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.290071+0000", 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1656999999999996e-05 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.290128+0000", 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6360000000000002e-05 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.304 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:44199 8.8 8:19b822eb:::benchmark_data_smithi043_97042_object44198:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.278399+0000", 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.011647411, 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011816172999999999, 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 44199 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.305 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.278399+0000", 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.278399+0000", 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.278399+0000", 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9200000000000001e-07 2024-09-21T11:45:21.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.278441+0000", 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2196000000000001e-05 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.278449+0000", 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6869999999999994e-06 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.278456+0000", 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.477e-06 2024-09-21T11:45:21.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.280194+0000", 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017380919999999999 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.280306+0000", 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000112414 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.280427+0000", 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000120418 2024-09-21T11:45:21.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.283217+0000", 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027909570000000002 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.283410+0000", 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000192535 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.309 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:44200 8.2 e172/160 8:4d982d58:::benchmark_data_smithi043_97042_object44199:head v 172'2749, mlcod=172'2747)", 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.282405+0000", 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0076413660000000001, 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0078884660000000002, 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.282405+0000", 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.282405+0000", 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.282406+0000", 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5600000000000004e-07 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.282435+0000", 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9031e-05 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.282444+0000", 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6020000000000006e-06 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.282450+0000", 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2759999999999997e-06 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.283267+0000", 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00081689800000000004 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.283287+0000", 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9211e-05 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:44207 8.0 e172/160 8:0d72a645:::benchmark_data_smithi043_97042_object44206:head v 172'2761, mlcod=172'2759)", 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.284124+0000", 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0059221209999999998, 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062148180000000004, 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.313 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284124+0000", 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284124+0000", 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284125+0000", 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0800000000000004e-07 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284169+0000", 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3992999999999997e-05 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284176+0000", 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7479999999999993e-06 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284183+0000", 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3489999999999997e-06 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284263+0000", 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0085000000000003e-05 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284376+0000", 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113618 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.316 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:44208 8.8 8:12c53c49:::benchmark_data_smithi043_97042_object44207:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.284575+0000", 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0054706390000000002, 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0058124350000000003, 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 44208 2024-09-21T11:45:21.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284575+0000", 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284575+0000", 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284577+0000", 2024-09-21T11:45:21.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1000000000000001e-06 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284608+0000", 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1489999999999998e-05 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284615+0000", 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3270000000000003e-06 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284622+0000", 2024-09-21T11:45:21.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1600000000000003e-06 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.284998+0000", 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037679800000000002 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285113+0000", 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115074 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285240+0000", 2024-09-21T11:45:21.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000126842 2024-09-21T11:45:21.321 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.321 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.321 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.321 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:44211 8.2 e172/160 8:413c81b3:::benchmark_data_smithi043_97042_object44210:head v 172'2750, mlcod=172'2747)", 2024-09-21T11:45:21.321 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.285283+0000", 2024-09-21T11:45:21.322 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0047635330000000004, 2024-09-21T11:45:21.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0051625960000000002, 2024-09-21T11:45:21.322 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.322 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.322 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:45:21.322 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285283+0000", 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285283+0000", 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285283+0000", 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3300000000000002e-07 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285305+0000", 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1671e-05 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285311+0000", 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4189999999999999e-06 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285317+0000", 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5090000000000001e-06 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285733+0000", 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00041629699999999999 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285752+0000", 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9304e-05 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:44216 8.8 8:18a9d491:::benchmark_data_smithi043_97042_object44215:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.285367+0000", 2024-09-21T11:45:21.325 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0046785999999999998, 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0051255129999999999, 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 44216 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285367+0000", 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285367+0000", 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285368+0000", 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3699999999999999e-07 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285407+0000", 2024-09-21T11:45:21.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8349999999999997e-05 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285416+0000", 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9819999999999997e-06 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285421+0000", 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6790000000000002e-06 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287529+0000", 2024-09-21T11:45:21.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021077410000000002 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287676+0000", 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014658600000000001 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289412+0000", 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017360380000000001 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:44217 8.3 8:c91e094c:::benchmark_data_smithi043_97042_object44216:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.285431+0000", 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0046149989999999998, 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0051225089999999999, 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 44217 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285431+0000", 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285431+0000", 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285432+0000", 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2600000000000002e-07 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285461+0000", 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9252999999999999e-05 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285467+0000", 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1040000000000003e-06 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285472+0000", 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7450000000000003e-06 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285527+0000", 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5167999999999998e-05 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285665+0000", 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000137668 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.285790+0000", 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012554100000000001 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.290493+0000", 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047029680000000001 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:44215 8.a e172/160 8:5d9b1c23:::benchmark_data_smithi043_97042_object44214:head v 172'2751, mlcod=172'2749)", 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.287112+0000", 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.002933565, 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035032779999999999, 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.334 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287112+0000", 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287112+0000", 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287113+0000", 2024-09-21T11:45:21.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9800000000000002e-07 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287147+0000", 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3364000000000001e-05 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287157+0000", 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8379999999999997e-06 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287164+0000", 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1049999999999997e-06 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.288113+0000", 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00094894 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.288129+0000", 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6477000000000002e-05 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:44209 8.9 e172/160 8:90f10777:::benchmark_data_smithi043_97042_object44208:head v 172'2761, mlcod=172'2759)", 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.287414+0000", 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0026325039999999999, 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0032502960000000002, 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287414+0000", 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287414+0000", 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287414+0000", 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5499999999999997e-07 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287441+0000", 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6630000000000001e-05 2024-09-21T11:45:21.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287453+0000", 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1917e-05 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287459+0000", 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1109999999999996e-06 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287608+0000", 2024-09-21T11:45:21.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000148751 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287623+0000", 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5313000000000001e-05 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:44213 8.0 e172/160 8:073ef0cf:::benchmark_data_smithi043_97042_object44212:head v 172'2762, mlcod=172'2759)", 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.287480+0000", 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0025659789999999999, 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0032314570000000001, 2024-09-21T11:45:21.341 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287480+0000", 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287480+0000", 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287481+0000", 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1799999999999995e-07 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287514+0000", 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3435000000000001e-05 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287520+0000", 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1840000000000004e-06 2024-09-21T11:45:21.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.287525+0000", 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7929999999999997e-06 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.288374+0000", 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00084885500000000003 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.288392+0000", 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7708000000000001e-05 2024-09-21T11:45:21.344 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.15150.0:44208 8.8 e172/160)", 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:21.289279+0000", 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.00076721500000000002, 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014786339999999999, 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "queued for pg", 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:21.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289279+0000", 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289279+0000", 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289280+0000", 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.78e-07 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:21.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289287+0000", 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0500000000000003e-06 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289295+0000", 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1769999999999998e-06 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.289301+0000", 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3840000000000002e-06 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.347 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:45:21.348 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 13 2024-09-21T11:45:21.348 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:45:21.354 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:21.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:21.355 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:21.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:21.494 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:21.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:21.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:21.593 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 19 16 44734 44718 147.073 93.0625 0.00813745 0.00679378 2024-09-21T11:45:21.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:21.631 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:21.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:21.632 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:21.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:21.770 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:21.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:21.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:21.871 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:21.909 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:21.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:21.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:22.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:22.049 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:22.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:22.050 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:22.151 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:21 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:45:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:21 smithi175 ceph-mon[32613]: osdmap e172: 8 total, 8 up, 7 in 2024-09-21T11:45:22.175 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:21 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:22.187 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:22.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:22.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:22.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:45:22.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:21 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:45:22.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:21 smithi043 ceph-mon[36454]: osdmap e172: 8 total, 8 up, 7 in 2024-09-21T11:45:22.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:21 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:22.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:21 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:45:22.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:21 smithi043 ceph-mon[29710]: osdmap e172: 8 total, 8 up, 7 in 2024-09-21T11:45:22.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:21 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:22.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:22.333 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:22.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:22.334 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:22.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:22.472 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:22.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:22.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:22.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:22.593 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:45:22.592339+0000 min lat: 0.00112969 max lat: 0.146256 avg lat: 0.0069181 2024-09-21T11:45:22.593 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:45:22.593 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 20 16 46246 46230 144.444 94.5 0.0133402 0.0069181 2024-09-21T11:45:22.603 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:22.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:22.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:22.610 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:22.704 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:22.727 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:22.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:22.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:22.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:22.865 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:22.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:22.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:22.966 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:22.987 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:22.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:22.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:23.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:23.126 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:23.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:23.127 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:23.192 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:45:23.193 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:45:23.193 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:45:23.193 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:45:23.193 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:45:23.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:23.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:23.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:23.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:23.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:23.391 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:23.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:23.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:23 smithi043 ceph-mon[29710]: pgmap v248: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 617 GiB / 626 GiB avail; 1.4 KiB/s rd, 171 MiB/s wr, 2.74k op/s 2024-09-21T11:45:23.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:23 smithi043 ceph-mon[29710]: osdmap e173: 8 total, 8 up, 7 in 2024-09-21T11:45:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:23 smithi043 ceph-mon[36454]: pgmap v248: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 617 GiB / 626 GiB avail; 1.4 KiB/s rd, 171 MiB/s wr, 2.74k op/s 2024-09-21T11:45:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:23 smithi043 ceph-mon[36454]: osdmap e173: 8 total, 8 up, 7 in 2024-09-21T11:45:23.491 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:23.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:23 smithi175 ceph-mon[32613]: pgmap v248: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 617 GiB / 626 GiB avail; 1.4 KiB/s rd, 171 MiB/s wr, 2.74k op/s 2024-09-21T11:45:23.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:23 smithi175 ceph-mon[32613]: osdmap e173: 8 total, 8 up, 7 in 2024-09-21T11:45:23.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:23.514 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:23.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:23.593 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 21 16 47555 47539 141.462 81.8125 0.0137898 0.00706346 2024-09-21T11:45:23.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:23.650 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:23.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:23.651 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:23.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:23.774 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:23.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:23.775 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:23.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:23.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:23.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:23.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:45:24.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:24.050 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:24.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:24.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:24.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:24.192 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:24.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:24.193 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:24.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:24.331 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:24.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:24.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:24.332 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:24.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:24.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:24.472 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:24.472 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:24.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:24.593 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 22 16 49109 49093 139.446 97.125 0.00689613 0.00716623 2024-09-21T11:45:24.607 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:24.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:24.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:24.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:24.748 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:24.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:24.749 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:24.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:24.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:24.872 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:24.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:24.899 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:45:24.899 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:45:24.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:45:24.899 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:45:24.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:109589 7.0 7:0a40998a:::benchmark_data_smithi043_70646_object109588:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:39.656674+0000", 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 165.23934478000001, 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.076169328999999994, 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 109589 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.656674+0000", 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.656674+0000", 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.656675+0000", 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6759999999999999e-06 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.656713+0000", 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.79e-05 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.656732+0000", 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8834e-05 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.656742+0000", 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5259999999999999e-06 2024-09-21T11:45:24.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.657486+0000", 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00074463700000000003 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.657612+0000", 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012536900000000001 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,2,3", 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.657779+0000", 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016722899999999999 2024-09-21T11:45:24.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660623+0000", 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002843894 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.663602+0000", 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0029792799999999999 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.664142+0000", 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054042999999999995 2024-09-21T11:45:24.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.732665+0000", 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.068522727000000005 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.732707+0000", 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2033999999999998e-05 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.732843+0000", 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013579299999999999 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:109596 7.0 7:02c5e065:::benchmark_data_smithi043_70646_object109595:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:39.660587+0000", 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 165.235431304, 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072938314000000004, 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.906 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 109596 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660587+0000", 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660587+0000", 2024-09-21T11:45:24.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660588+0000", 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7999999999999993e-07 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660626+0000", 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7475000000000003e-05 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660637+0000", 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1001e-05 2024-09-21T11:45:24.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660651+0000", 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4453e-05 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660696+0000", 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5342000000000001e-05 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.660811+0000", 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011435200000000001 2024-09-21T11:45:24.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,2,3", 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.661007+0000", 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019616600000000001 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.664200+0000", 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0031932530000000001 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.664262+0000", 2024-09-21T11:45:24.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1803000000000004e-05 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.664279+0000", 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6932999999999999e-05 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.733423+0000", 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.069144250000000004 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.733461+0000", 2024-09-21T11:45:24.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7576000000000001e-05 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:39.733525+0000", 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4729999999999999e-05 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123060 7.0 7:07a51697:::benchmark_data_smithi043_70646_object5652:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.887750+0000", 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 149.00826799199999, 2024-09-21T11:45:24.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.083149179000000004, 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123060 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.887750+0000", 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.887750+0000", 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.887752+0000", 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.381e-06 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.887760+0000", 2024-09-21T11:45:24.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2779999999999993e-06 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.887773+0000", 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3045e-05 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.887780+0000", 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.233e-06 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.887846+0000", 2024-09-21T11:45:24.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5537000000000006e-05 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.887999+0000", 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015286100000000001 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,2,3", 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.888134+0000", 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013537799999999999 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.890659+0000", 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025243689999999998 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.892329+0000", 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016701769999999999 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.892359+0000", 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0196e-05 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.970727+0000", 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.078368236999999993 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.970788+0000", 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0406999999999999e-05 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.970900+0000", 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011208 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126429 7.0 7:0ab6ed09:::benchmark_data_smithi043_70646_object9021:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.480864+0000", 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 145.41515398600001, 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.086014851000000003, 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126429 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.919 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.480864+0000", 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.480864+0000", 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.480866+0000", 2024-09-21T11:45:24.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.688e-06 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.480871+0000", 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0849999999999996e-06 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.480884+0000", 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.226e-05 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.480891+0000", 2024-09-21T11:45:24.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5259999999999996e-06 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.482010+0000", 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011188649999999999 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.482122+0000", 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111729 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,2,3", 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.482245+0000", 2024-09-21T11:45:24.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012344 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.486411+0000", 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0041660009999999999 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.487168+0000", 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075650299999999995 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.488181+0000", 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001013042 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.566694+0000", 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.078513870999999999 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.566751+0000", 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6486000000000001e-05 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.566879+0000", 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128355 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126434 7.0 7:04054769:::benchmark_data_smithi043_70646_object9026:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.483020+0000", 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 145.412998617, 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.085840196999999993, 2024-09-21T11:45:24.925 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.926 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.926 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.926 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.926 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126434 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.483020+0000", 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.483020+0000", 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.483021+0000", 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1209999999999999e-06 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.483026+0000", 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9089999999999999e-06 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.483038+0000", 2024-09-21T11:45:24.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2140000000000001e-05 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.483045+0000", 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0659999999999998e-06 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.484810+0000", 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001764746 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.484884+0000", 2024-09-21T11:45:24.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3994999999999999e-05 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,2,3", 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.484974+0000", 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0329999999999997e-05 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.486543+0000", 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001569031 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.487221+0000", 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00067802699999999999 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.488209+0000", 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0009879489999999999 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.568749+0000", 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.080539830000000007 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.568787+0000", 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8371000000000002e-05 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.568860+0000", 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2682000000000006e-05 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:180788 7.0 7:0ce476d4:::benchmark_data_smithi043_70646_object63380:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:54.466188+0000", 2024-09-21T11:45:24.932 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 90.429830484000007, 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.078653706000000004, 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 180788 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.466188+0000", 2024-09-21T11:45:24.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.466188+0000", 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.466189+0000", 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.466194+0000", 2024-09-21T11:45:24.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8570000000000003e-06 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.466204+0000", 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0098999999999999e-05 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.466210+0000", 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9710000000000001e-06 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.466255+0000", 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4802000000000002e-05 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.466335+0000", 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0409999999999998e-05 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.466410+0000", 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4808999999999995e-05 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.468852+0000", 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024416770000000002 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.507944+0000", 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.039092572999999999 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.544748+0000", 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.036804075999999998 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.544799+0000", 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0977999999999999e-05 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:54.544842+0000", 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2224000000000001e-05 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:185190 7.0 7:0d032549:::benchmark_data_smithi043_70646_object67782:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:57.846047+0000", 2024-09-21T11:45:24.938 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 87.049971110000001, 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.088081193000000002, 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 185190 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846047+0000", 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846047+0000", 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846048+0000", 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1370000000000001e-06 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846053+0000", 2024-09-21T11:45:24.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1899999999999997e-06 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846062+0000", 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7289999999999995e-06 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846069+0000", 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0949999999999998e-06 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846118+0000", 2024-09-21T11:45:24.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8365999999999997e-05 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846241+0000", 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122798 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846371+0000", 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013066299999999999 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851702+0000", 2024-09-21T11:45:24.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0053307160000000001 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851775+0000", 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2871000000000001e-05 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.934022+0000", 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.082247023000000002 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.934081+0000", 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8879000000000003e-05 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.934129+0000", 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7725999999999997e-05 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:185192 7.0 7:0bfda4b9:::benchmark_data_smithi043_70646_object67784:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:57.846089+0000", 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 87.049929051999996, 2024-09-21T11:45:24.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.088152759999999997, 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 185192 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846089+0000", 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846089+0000", 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846090+0000", 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1099999999999996e-07 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846093+0000", 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6919999999999998e-06 2024-09-21T11:45:24.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846098+0000", 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7289999999999997e-06 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.846104+0000", 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6350000000000003e-06 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.848873+0000", 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002768885 2024-09-21T11:45:24.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.848983+0000", 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011027399999999999 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.849082+0000", 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9048000000000004e-05 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.852385+0000", 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0033025720000000001 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.854905+0000", 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002519973 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.934169+0000", 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.079264662 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.934208+0000", 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8364000000000001e-05 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.934242+0000", 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4415000000000002e-05 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220182 7.0 7:0ab20e26:::benchmark_data_smithi043_70646_object102774:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.198969+0000", 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 58.697049235999998, 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10672838, 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.950 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220182 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198969+0000", 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198969+0000", 2024-09-21T11:45:24.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198971+0000", 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8509999999999999e-06 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198977+0000", 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7699999999999998e-06 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198988+0000", 2024-09-21T11:45:24.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0808000000000001e-05 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198993+0000", 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4940000000000003e-06 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.199102+0000", 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109261 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.199301+0000", 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000198913 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.199428+0000", 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012705600000000001 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.202312+0000", 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028836199999999999 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.249560+0000", 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047248342999999998 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.305560+0000", 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.05599954 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.305641+0000", 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0918e-05 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.305698+0000", 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6805999999999998e-05 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220190 7.0 7:030c2ffe:::benchmark_data_smithi043_70646_object102782:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.205396+0000", 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 58.690622404999999, 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10103582, 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.956 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220190 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205396+0000", 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205396+0000", 2024-09-21T11:45:24.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205398+0000", 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.46e-06 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205403+0000", 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3900000000000001e-06 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205414+0000", 2024-09-21T11:45:24.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0966e-05 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205420+0000", 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9800000000000003e-06 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205466+0000", 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5797000000000002e-05 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205542+0000", 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6149999999999994e-05 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205612+0000", 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0216999999999995e-05 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.207655+0000", 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020429710000000002 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.251909+0000", 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.044254454999999998 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306258+0000", 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.054348354000000001 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306334+0000", 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5940999999999997e-05 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306432+0000", 2024-09-21T11:45:24.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8139000000000003e-05 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221551 7.b 7:d9f868e4:::benchmark_data_smithi043_70646_object104143:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.510996+0000", 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 57.385022110999998, 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.073229817000000003, 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221551 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.510996+0000", 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.510996+0000", 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.510998+0000", 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.579e-06 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.511003+0000", 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0649999999999998e-06 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.511026+0000", 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2531000000000002e-05 2024-09-21T11:45:24.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.511034+0000", 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.439e-06 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.511113+0000", 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9322999999999999e-05 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.511222+0000", 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108372 2024-09-21T11:45:24.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.511349+0000", 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127744 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.517484+0000", 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0061342130000000003 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.519713+0000", 2024-09-21T11:45:24.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022293980000000001 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.584132+0000", 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.064418955 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.584184+0000", 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1681000000000002e-05 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.584226+0000", 2024-09-21T11:45:24.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2517e-05 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221756 7.0 7:0e91c63f:::benchmark_data_smithi043_70646_object104348:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.700708+0000", 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 57.195310341999999, 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.078624942000000003, 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:45:24.968 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221756 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700708+0000", 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700708+0000", 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700710+0000", 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2230000000000001e-06 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700715+0000", 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1490000000000002e-06 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700729+0000", 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3489000000000001e-05 2024-09-21T11:45:24.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700736+0000", 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2359999999999998e-06 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700811+0000", 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4970999999999998e-05 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.700913+0000", 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101708 2024-09-21T11:45:24.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.701021+0000", 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108009 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.703835+0000", 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028136599999999999 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.707126+0000", 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0032914979999999999 2024-09-21T11:45:24.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.779271+0000", 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.072144802999999993 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.779302+0000", 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1349000000000001e-05 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.779333+0000", 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0846999999999998e-05 2024-09-21T11:45:24.973 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43896 8.3 8:c3bd4fd6:::benchmark_data_smithi043_97042_object43895:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.982177+0000", 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.9138411280000001, 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14573064599999999, 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:24.974 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43896 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982177+0000", 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982177+0000", 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982178+0000", 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5600000000000005e-07 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982209+0000", 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0809000000000003e-05 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982218+0000", 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0089999999999996e-06 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982224+0000", 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5350000000000001e-06 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.984992+0000", 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027677230000000001 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.985115+0000", 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012332000000000001 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:45:24.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.985242+0000", 2024-09-21T11:45:24.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012655500000000001 2024-09-21T11:45:24.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989565+0000", 2024-09-21T11:45:24.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043232679999999999 2024-09-21T11:45:24.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.978 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992180+0000", 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002614487 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127774+0000", 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13559412700000001 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127839+0000", 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4784999999999997e-05 2024-09-21T11:45:24.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127908+0000", 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9271999999999997e-05 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43898 8.3 8:cb8873a2:::benchmark_data_smithi043_97042_object43897:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.986596+0000", 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.909422261, 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.1414368, 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.981 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43898 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986596+0000", 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986596+0000", 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986598+0000", 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.844e-06 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986632+0000", 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.413e-05 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986644+0000", 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1443e-05 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986653+0000", 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5550000000000008e-06 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987082+0000", 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042925400000000001 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987555+0000", 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047269100000000002 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987683+0000", 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128329 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992217+0000", 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045337889999999999 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992269+0000", 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1941999999999998e-05 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127950+0000", 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13568037199999999 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127999+0000", 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9444999999999998e-05 2024-09-21T11:45:24.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.128033+0000", 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4005999999999999e-05 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43901 8.3 8:c3170c6a:::benchmark_data_smithi043_97042_object43900:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.986960+0000", 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.909057995, 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.144105175, 2024-09-21T11:45:24.987 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43901 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986960+0000", 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986960+0000", 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986961+0000", 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1800000000000004e-07 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987010+0000", 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8708999999999997e-05 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987021+0000", 2024-09-21T11:45:24.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0548e-05 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987028+0000", 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9949999999999999e-06 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989603+0000", 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025754509999999999 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989731+0000", 2024-09-21T11:45:24.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012751 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989866+0000", 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000135105 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992250+0000", 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023844780000000002 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992368+0000", 2024-09-21T11:45:24.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117924 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.130962+0000", 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13859406799999999 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.131025+0000", 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2992999999999998e-05 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.131066+0000", 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0475999999999997e-05 2024-09-21T11:45:24.992 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43907 8.8 8:131facf7:::benchmark_data_smithi043_97042_object43906:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.989559+0000", 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 3.9064596069999999, 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12934364800000001, 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:24.993 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43907 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989559+0000", 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989559+0000", 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989560+0000", 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989621+0000", 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0619999999999999e-05 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989629+0000", 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3459999999999994e-06 2024-09-21T11:45:24.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989634+0000", 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9540000000000003e-06 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989707+0000", 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3330000000000004e-05 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989835+0000", 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127687 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989962+0000", 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127179 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.991540+0000", 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015774160000000001 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.991814+0000", 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027401599999999997 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118806+0000", 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.126992523 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118858+0000", 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1774999999999998e-05 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118902+0000", 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4571999999999999e-05 2024-09-21T11:45:24.998 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:45873 8.8 8:1a0feefe:::benchmark_data_smithi043_97042_object45872:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.277880+0000", 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 2.6181389140000002, 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11236278299999999, 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:24.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 45873 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277880+0000", 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277880+0000", 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277881+0000", 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277944+0000", 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3847e-05 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277953+0000", 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3729999999999993e-06 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277958+0000", 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5380000000000002e-06 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.278005+0000", 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6214000000000002e-05 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.278086+0000", 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1856999999999999e-05 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.278164+0000", 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7516999999999997e-05 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.279569+0000", 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014045819999999999 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.279975+0000", 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040607499999999999 2024-09-21T11:45:25.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.390144+0000", 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.110169824 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.390197+0000", 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2070000000000001e-05 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.390242+0000", 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5816000000000003e-05 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:25.004 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47786 8.d 8:b7d985cc:::benchmark_data_smithi043_97042_object47785:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.712413+0000", 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.1836056180000001, 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.133025376, 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:25.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47786 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712413+0000", 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712413+0000", 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712414+0000", 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9500000000000001e-07 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712445+0000", 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0970000000000003e-05 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712657+0000", 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021223800000000001 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712664+0000", 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4159999999999998e-06 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712753+0000", 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8609999999999994e-05 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712865+0000", 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011187400000000001 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713024+0000", 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000158954 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716097+0000", 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030732699999999999 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716114+0000", 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6693000000000001e-05 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845317+0000", 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12920314499999999 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845388+0000", 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0706999999999999e-05 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845438+0000", 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0603999999999999e-05 2024-09-21T11:45:25.010 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47788 8.8 8:1016d613:::benchmark_data_smithi043_97042_object47787:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.712688+0000", 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.18332997, 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13416687399999999, 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:25.011 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47788 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712688+0000", 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712688+0000", 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712689+0000", 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8899999999999999e-07 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712720+0000", 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0641e-05 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712826+0000", 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712837+0000", 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0925e-05 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712917+0000", 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9839999999999995e-05 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713082+0000", 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000164947 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713214+0000", 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000132829 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.715666+0000", 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024515679999999999 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716616+0000", 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00095051100000000002 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.846760+0000", 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13014347900000001 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.846809+0000", 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9141999999999998e-05 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.846855+0000", 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6303e-05 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:25.016 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47793 8.d 8:b20302a8:::benchmark_data_smithi043_97042_object47792:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.716282+0000", 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.179736873, 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.133436156, 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47793 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.017 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716282+0000", 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716282+0000", 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:45:25.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716283+0000", 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2979999999999999e-06 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716340+0000", 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7364999999999997e-05 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716443+0000", 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010284400000000001 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716449+0000", 2024-09-21T11:45:25.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3910000000000003e-06 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716490+0000", 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0630000000000002e-05 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716621+0000", 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013114600000000001 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:45:25.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716748+0000", 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012651000000000001 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.717993+0000", 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001245265 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.718461+0000", 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046752700000000003 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:25.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.849624+0000", 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.131163909 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.849675+0000", 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0621000000000002e-05 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.849718+0000", 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.265e-05 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:25.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:25.023 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:45:25.023 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:45:25.023 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:45:25.055 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:25.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:25.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:25.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:25.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:25.196 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:25.296 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:25.336 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:25.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:25.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:25.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:25.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:25.466 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:25.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:25.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:25 smithi043 ceph-mon[29710]: pgmap v250: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 617 GiB / 626 GiB avail; 773 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-21T11:45:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:45:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:45:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:45:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:25 smithi043 ceph-mon[36454]: pgmap v250: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 617 GiB / 626 GiB avail; 773 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-21T11:45:25.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:25 smithi175 ceph-mon[32613]: pgmap v250: 148 pgs: 148 active+clean; 2.7 GiB data, 9.2 GiB used, 617 GiB / 626 GiB avail; 773 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-21T11:45:25.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:25.594 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 23 16 50652 50636 137.572 96.4375 0.00798498 0.00726397 2024-09-21T11:45:25.607 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:25.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:25.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:25.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:25.749 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:25.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:25.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:25.850 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:25.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:25.891 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:25.891 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:25.959 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:45:25.992 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:26.031 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:26.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:26.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:26.133 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:26.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:26.178 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:26.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:26.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:26.325 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:26.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:26.378 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:26.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:26.464 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:26.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:26.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:26.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:26.594 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 24 16 52247 52231 135.993 99.6875 0.00831601 0.00734849 2024-09-21T11:45:26.603 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:26.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:26.604 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:26.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:26.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:26.736 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:26.736 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:26.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:26.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:26.872 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:26.873 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:26.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:26.983 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:45:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:45:26.983 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:45:26.984 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:45:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:27.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:27.019 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:27.120 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:27.149 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:27.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:27.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:27.195 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:45:27.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:27 smithi043 ceph-mon[36454]: pgmap v251: 148 pgs: 148 active+clean; 2.7 GiB data, 9.6 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-21T11:45:27.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:27 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/113922038' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:27.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:27 smithi043 ceph-mon[29710]: pgmap v251: 148 pgs: 148 active+clean; 2.7 GiB data, 9.6 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-21T11:45:27.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:27 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/113922038' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:27.251 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:27.285 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:27.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:27.286 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:27.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:27.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:27.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:27.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:27 smithi175 ceph-mon[32613]: pgmap v251: 148 pgs: 148 active+clean; 2.7 GiB data, 9.6 GiB used, 616 GiB / 626 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-21T11:45:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:27 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/113922038' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:27 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:27 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/113922038' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:45:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:27 smithi175 ceph-mon[32613]: osdmap e174: 8 total, 8 up, 7 in 2024-09-21T11:45:27.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:27.563 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:27.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:27.564 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:27.594 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 25 16 53844 53828 134.546 99.8125 0.00604444 0.00742732 2024-09-21T11:45:27.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:27 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:27.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:27 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/113922038' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:45:27.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:27 smithi043 ceph-mon[36454]: osdmap e174: 8 total, 8 up, 7 in 2024-09-21T11:45:27.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:27 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:27.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:27 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/113922038' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:45:27.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:27 smithi043 ceph-mon[29710]: osdmap e174: 8 total, 8 up, 7 in 2024-09-21T11:45:27.665 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:27.703 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:27.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:27.704 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:27.805 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:27.836 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:27.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:27.837 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:27.930 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:45:27.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:27.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:27.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:28.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:28.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:28.114 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:28.114 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:28.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 2 1 2024-09-21T11:45:28.215 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:28.255 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:28.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:28.256 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:28.357 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:28.362 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:28.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:28.407 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:28.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:28.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:28.547 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:28.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:28.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:28.594 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 26 16 55443 55427 133.214 99.9375 0.0103951 0.00750118 2024-09-21T11:45:28.645 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:45:28.646 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:45:28.649 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:28.677 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:45:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:45:28.682 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:28.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:28.683 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:28.697 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:45:28.697 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:45:28.720 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:45:28.720 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:45:28.757 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:45:28.757 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:45:28.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:28.791 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:45:28.791 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:45:28.822 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:28.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:28.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:28.828 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:45:28.828 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:45:28.862 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:45:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:45:28.924 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:28.955 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:45:28.955 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:45:28.963 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:28.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:28.965 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:29.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:29.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:29.105 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:29.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:29.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:29.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:29.243 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:29.344 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:29.387 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:29.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:29.388 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:29.489 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:29.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:29 smithi043 ceph-mon[36454]: pgmap v253: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 737 B/s rd, 112 MiB/s wr, 1.80k op/s 2024-09-21T11:45:29.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1254620588' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:29 smithi043 ceph-mon[29710]: pgmap v253: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 737 B/s rd, 112 MiB/s wr, 1.80k op/s 2024-09-21T11:45:29.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1254620588' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:29.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:29 smithi175 ceph-mon[32613]: pgmap v253: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 737 B/s rd, 112 MiB/s wr, 1.80k op/s 2024-09-21T11:45:29.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1254620588' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:29.525 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:29.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:29.527 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:29.594 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 27 16 57029 57013 131.952 99.125 0.0037307 0.00757324 2024-09-21T11:45:29.627 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:29.661 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:29.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:29.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:29.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:29.805 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:29.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:29.806 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:29.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:29.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:29.945 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:29.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:45:30.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:30.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:30.085 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:30.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:30.224 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:30.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:30.225 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:30.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:30.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:30.359 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:30.387 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:30.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:30.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:30.499 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:30.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:30.594 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 28 16 58618 58602 130.786 99.3125 0.00631725 0.00764037 2024-09-21T11:45:30.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:30.632 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:30.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:30.651 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:30.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:30.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:30.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:30.886 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:30.925 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:30.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:30.926 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:31.008 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:45:31.008 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:45:31.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:31.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:31.066 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:31.166 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:31.200 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:31.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:31.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:31.234 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:45:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:45:31.302 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:31.336 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:31.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:31.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:31.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:31.474 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:31.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:31.475 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:31.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:31 smithi043 ceph-mon[29710]: pgmap v254: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 58 MiB/s wr, 935 op/s 2024-09-21T11:45:31.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:31.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:31.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2700577610' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:45:31.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:31 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:45:31.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:31 smithi043 ceph-mon[36454]: pgmap v254: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 58 MiB/s wr, 935 op/s 2024-09-21T11:45:31.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:31.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:31.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2700577610' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:45:31.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:31 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:45:31.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:31 smithi175 ceph-mon[32613]: pgmap v254: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 615 GiB / 626 GiB avail; 639 B/s rd, 58 MiB/s wr, 935 op/s 2024-09-21T11:45:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2700577610' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:45:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:31 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:45:31.576 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:31.594 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 29 16 60212 60196 129.711 99.625 0.00803944 0.00770411 2024-09-21T11:45:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:31.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:31.614 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:31.656 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T11:45:31.714 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:31.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:45:31.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:31.761 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:31.762 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:31.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:45:31.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:31.888 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:31.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:31.889 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:31.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:32.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:32.028 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:32.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:32.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:32.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:32.165 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:32.194 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:32.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:32.305 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:32.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:32.306 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:32.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:45:32.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:32.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:32.447 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:32.447 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:32.488 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:32.488 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:32 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:45:32.488 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:32 smithi175 ceph-mon[32613]: osdmap e175: 8 total, 8 up, 7 in 2024-09-21T11:45:32.488 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:32 smithi175 ceph-mon[32613]: osdmap e176: 8 total, 8 up, 7 in 2024-09-21T11:45:32.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:32.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:32 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:45:32.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:32 smithi043 ceph-mon[36454]: osdmap e175: 8 total, 8 up, 7 in 2024-09-21T11:45:32.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:32 smithi043 ceph-mon[36454]: osdmap e176: 8 total, 8 up, 7 in 2024-09-21T11:45:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:32 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:45:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:32 smithi043 ceph-mon[29710]: osdmap e175: 8 total, 8 up, 7 in 2024-09-21T11:45:32.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:32 smithi043 ceph-mon[29710]: osdmap e176: 8 total, 8 up, 7 in 2024-09-21T11:45:32.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:32.591 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:32.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:32.592 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:32.594 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 30 16 61630 61614 128.341 88.625 0.0157271 0.00778681 2024-09-21T11:45:32.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:32.715 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:32.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:32.716 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:32.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:32.854 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:32.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:32.856 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:32.956 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:32.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:32.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:33.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 3 1 2024-09-21T11:45:33.074 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:45:33.074 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:45:33.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:33.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:33.166 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:33.166 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:33.201 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:33.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:33.269 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:33.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:33.328 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:33.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:33 smithi043 ceph-mon[29710]: pgmap v256: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.2 KiB/s rd, 121 MiB/s wr, 1.93k op/s 2024-09-21T11:45:33.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:33 smithi043 ceph-mon[36454]: pgmap v256: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.2 KiB/s rd, 121 MiB/s wr, 1.93k op/s 2024-09-21T11:45:33.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:33.463 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:33.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:33.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:33.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:33 smithi175 ceph-mon[32613]: pgmap v256: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 1.2 KiB/s rd, 121 MiB/s wr, 1.93k op/s 2024-09-21T11:45:33.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:33.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:33.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:33.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:33.595 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 31 16 63022 63006 127.007 87 0.00921617 0.0078671 2024-09-21T11:45:33.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:33.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:33.734 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:33.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:33.859 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:33.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:33.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:33.960 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:33.992 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:33.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:33.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:34.093 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:34.117 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:34.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:34.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:34.219 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:34.252 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:34.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:34.253 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:34.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:34.360 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:45:34.360 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:45:34.360 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:45:34.360 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:45:34.360 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:45:34.360 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:45:34.360 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:45:34.360 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:45:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:45:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:45:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:45:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:45:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:45:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:45:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:45:34.361 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:45:34.398 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:34.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:34.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:34.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1699047635' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:34.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1699047635' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:34.501 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:34.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1699047635' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:34.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:34.532 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:34.532 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:34.597 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 32 16 64355 64339 125.631 83.3125 0.019413 0.00795456 2024-09-21T11:45:34.633 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:34.665 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:34.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:34.666 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:34.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:34.807 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:34.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:34.808 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:34.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:34.943 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:34.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:34.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:35.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:35.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:35.084 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:35.084 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:35.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:35.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:35.223 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:35.223 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:35.266 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:45:35.324 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:35.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:35.347 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:35.348 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:35.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:35.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:35 smithi043 ceph-mon[29710]: pgmap v258: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 773 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:45:35.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:35 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3247769461' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:45:35.468 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:35 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:45:35.469 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:45:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:45:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:45:35.469 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:35 smithi043 ceph-mon[36454]: pgmap v258: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 773 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:45:35.469 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:35 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3247769461' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:45:35.469 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:35 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:45:35.491 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:35.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:35.492 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:35.593 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:35.597 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 33 16 65692 65676 124.356 83.5625 0.00695673 0.00803637 2024-09-21T11:45:35.629 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:35.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:35.630 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:35.730 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:35.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:35 smithi175 ceph-mon[32613]: pgmap v258: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 614 GiB / 626 GiB avail; 773 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:45:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:35 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3247769461' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:45:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:35 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:45:35.764 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:35.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:35.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:35.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:35.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:35.908 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:36.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:36.030 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:36.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:36.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:36.132 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:36.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:36.165 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:36.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:36.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T11:45:36.300 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:36.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:36.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:36.401 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:36.436 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:36.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:36.437 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:36.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:36 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:36.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:36 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:45:36.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:36 smithi043 ceph-mon[29710]: osdmap e177: 8 total, 8 up, 7 in 2024-09-21T11:45:36.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:36 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3591013124' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:45:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:36 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:36 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:45:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:36 smithi043 ceph-mon[36454]: osdmap e177: 8 total, 8 up, 7 in 2024-09-21T11:45:36.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:36 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3591013124' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:45:36.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:36 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:36.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:36 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:45:36.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:36 smithi175 ceph-mon[32613]: osdmap e177: 8 total, 8 up, 7 in 2024-09-21T11:45:36.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:36 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3591013124' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:45:36.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:36.588 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:36.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:36.589 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:36.598 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 34 16 66855 66839 122.836 72.6875 0.00734714 0.00813563 2024-09-21T11:45:36.689 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:36.730 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:36.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:36.731 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:36.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:36.856 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:36.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:36.857 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:36.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:36.992 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:36.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:36.993 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:37.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:37.132 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:37.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:37.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:37.234 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:37.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:45:37.268 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:37.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:37.269 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:37.369 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:37.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:37.407 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:37.407 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:37.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 4 1 2024-09-21T11:45:37.509 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:37.537 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:37.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:37.539 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:37.598 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 35 16 68185 68169 121.701 83.125 0.00523148 0.00821097 2024-09-21T11:45:37.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:37.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:37.692 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:37.709 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[36454]: pgmap v260: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 1.7 KiB/s rd, 87 MiB/s wr, 1.40k op/s 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[36454]: 3.1c scrub starts 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[36454]: 3.1c scrub ok 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3591013124' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[36454]: osdmap e178: 8 total, 8 up, 7 in 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[36454]: 3.14 scrub starts 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[36454]: 3.14 scrub ok 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[29710]: pgmap v260: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 1.7 KiB/s rd, 87 MiB/s wr, 1.40k op/s 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[29710]: 3.1c scrub starts 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[29710]: 3.1c scrub ok 2024-09-21T11:45:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3591013124' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:45:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[29710]: osdmap e178: 8 total, 8 up, 7 in 2024-09-21T11:45:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[29710]: 3.14 scrub starts 2024-09-21T11:45:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:37 smithi043 ceph-mon[29710]: 3.14 scrub ok 2024-09-21T11:45:37.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:37 smithi175 ceph-mon[32613]: pgmap v260: 148 pgs: 148 active+clean; 3.6 GiB data, 12 GiB used, 613 GiB / 626 GiB avail; 1.7 KiB/s rd, 87 MiB/s wr, 1.40k op/s 2024-09-21T11:45:37.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:37 smithi175 ceph-mon[32613]: 3.1c scrub starts 2024-09-21T11:45:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:37 smithi175 ceph-mon[32613]: 3.1c scrub ok 2024-09-21T11:45:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:37 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3591013124' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:45:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:37 smithi175 ceph-mon[32613]: osdmap e178: 8 total, 8 up, 7 in 2024-09-21T11:45:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:37 smithi175 ceph-mon[32613]: 3.14 scrub starts 2024-09-21T11:45:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:37 smithi175 ceph-mon[32613]: 3.14 scrub ok 2024-09-21T11:45:37.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:37.833 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:37.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:37.834 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:37.934 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:37.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:37.969 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:37.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:38.070 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:38.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:38.110 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:38.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:38.157 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:45:38.211 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:38.267 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:38.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:38.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:38.359 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:38.366 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:45:38.366 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:45:38.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:38.405 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:45:38.405 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:45:38.421 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:38.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:38.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:38.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[29710]: 6.8 scrub starts 2024-09-21T11:45:38.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[29710]: 6.8 scrub ok 2024-09-21T11:45:38.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[29710]: 5.1c scrub starts 2024-09-21T11:45:38.440 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[29710]: 5.1c scrub ok 2024-09-21T11:45:38.441 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:45:38.441 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:45:38.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[36454]: 6.8 scrub starts 2024-09-21T11:45:38.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[36454]: 6.8 scrub ok 2024-09-21T11:45:38.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[36454]: 5.1c scrub starts 2024-09-21T11:45:38.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[36454]: 5.1c scrub ok 2024-09-21T11:45:38.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:45:38.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:38 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:45:38.441 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:45:38.442 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:45:38.473 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:45:38.474 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:45:38.507 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:45:38.507 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:45:38.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:38 smithi175 ceph-mon[32613]: 6.8 scrub starts 2024-09-21T11:45:38.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:38 smithi175 ceph-mon[32613]: 6.8 scrub ok 2024-09-21T11:45:38.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:38 smithi175 ceph-mon[32613]: 5.1c scrub starts 2024-09-21T11:45:38.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:38 smithi175 ceph-mon[32613]: 5.1c scrub ok 2024-09-21T11:45:38.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:38 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:45:38.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:38 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:45:38.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:38.539 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:45:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:45:38.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:38.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:38.573 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:38.576 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:45:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:45:38.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:45:38.598 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 36 16 69393 69377 120.418 75.5 0.0119503 0.00829919 2024-09-21T11:45:38.612 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:45:38.612 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:45:38.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:38.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:38.748 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:38.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:38.849 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:38.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:38.883 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:38.883 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:38.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:39.028 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:39.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:39.029 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:39.130 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:39.167 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:39.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:39.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:39.269 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:39.302 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:39.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:39.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:39.404 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:39.440 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:39.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:39.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:39.542 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:39.574 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:45:39.574 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:45:39.575 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:39 smithi175 ceph-mon[32613]: pgmap v262: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:45:39.575 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:39 smithi175 ceph-mon[32613]: osdmap e179: 8 total, 8 up, 7 in 2024-09-21T11:45:39.575 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3778872173' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:39.575 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:39 smithi175 ceph-mon[32613]: 6.17 scrub starts 2024-09-21T11:45:39.575 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:39 smithi175 ceph-mon[32613]: 6.17 scrub ok 2024-09-21T11:45:39.578 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:39.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:39.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:39.598 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 37 16 70712 70696 119.391 82.4375 0.00656208 0.00837017 2024-09-21T11:45:39.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:39.717 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:39.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:39.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:39 smithi043 ceph-mon[36454]: pgmap v262: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:45:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:39 smithi043 ceph-mon[36454]: osdmap e179: 8 total, 8 up, 7 in 2024-09-21T11:45:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3778872173' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:39 smithi043 ceph-mon[36454]: 6.17 scrub starts 2024-09-21T11:45:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:39 smithi043 ceph-mon[36454]: 6.17 scrub ok 2024-09-21T11:45:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:39 smithi043 ceph-mon[29710]: pgmap v262: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 852 B/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:45:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:39 smithi043 ceph-mon[29710]: osdmap e179: 8 total, 8 up, 7 in 2024-09-21T11:45:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3778872173' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:39 smithi043 ceph-mon[29710]: 6.17 scrub starts 2024-09-21T11:45:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:39 smithi043 ceph-mon[29710]: 6.17 scrub ok 2024-09-21T11:45:39.819 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:39.853 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:39.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:39.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:39.907 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:39.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:39.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:39.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:40.097 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:40.137 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:40.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:40.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:40.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:40.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:40.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:40.307 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T11:45:40.331 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:45:40.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:40.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:45:40.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:45:40.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:45:40.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:45:40.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[29710]: osdmap e180: 8 total, 8 up, 7 in 2024-09-21T11:45:40.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2215530840' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:45:40.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[29710]: 6.1f scrub starts 2024-09-21T11:45:40.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[29710]: 6.1f scrub ok 2024-09-21T11:45:40.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:45:40.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:45:40.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:45:40.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:45:40.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[36454]: osdmap e180: 8 total, 8 up, 7 in 2024-09-21T11:45:40.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2215530840' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:45:40.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[36454]: 6.1f scrub starts 2024-09-21T11:45:40.389 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:40 smithi043 ceph-mon[36454]: 6.1f scrub ok 2024-09-21T11:45:40.426 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:40.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:40.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:40.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:40.564 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:40.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:40.565 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:40.598 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 38 16 72101 72085 118.534 86.8125 0.00981377 0.00843105 2024-09-21T11:45:40.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:40.716 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:40.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:40.717 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:40 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:45:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:40 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:45:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:40 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:45:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:40 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:45:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:40 smithi175 ceph-mon[32613]: osdmap e180: 8 total, 8 up, 7 in 2024-09-21T11:45:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:40 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2215530840' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:45:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:40 smithi175 ceph-mon[32613]: 6.1f scrub starts 2024-09-21T11:45:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:40 smithi175 ceph-mon[32613]: 6.1f scrub ok 2024-09-21T11:45:40.773 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:45:40.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:40.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:40.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:40.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:40.999 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:41.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:41.000 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:41.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:41.123 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:41.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:41.124 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:41.219 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:45:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:45:41.225 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:41.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:41.268 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:41.268 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:41.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 5 1 2024-09-21T11:45:41.323 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:45:41.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:41.389 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:45:41.389 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:45:41.407 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:41.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:41.408 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[36454]: pgmap v265: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 1.1 KiB/s rd, 95 MiB/s wr, 1.51k op/s 2024-09-21T11:45:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2215530840' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:45:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[36454]: osdmap e181: 8 total, 8 up, 7 in 2024-09-21T11:45:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[36454]: 4.1c scrub starts 2024-09-21T11:45:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[36454]: 4.1c scrub ok 2024-09-21T11:45:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2532182494' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[29710]: pgmap v265: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 1.1 KiB/s rd, 95 MiB/s wr, 1.51k op/s 2024-09-21T11:45:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2215530840' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:45:41.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[29710]: osdmap e181: 8 total, 8 up, 7 in 2024-09-21T11:45:41.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[29710]: 4.1c scrub starts 2024-09-21T11:45:41.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[29710]: 4.1c scrub ok 2024-09-21T11:45:41.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2532182494' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:41.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:41 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:41.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:41 smithi175 ceph-mon[32613]: pgmap v265: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 612 GiB / 626 GiB avail; 1.1 KiB/s rd, 95 MiB/s wr, 1.51k op/s 2024-09-21T11:45:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:41 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2215530840' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:45:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:41 smithi175 ceph-mon[32613]: osdmap e181: 8 total, 8 up, 7 in 2024-09-21T11:45:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:41 smithi175 ceph-mon[32613]: 4.1c scrub starts 2024-09-21T11:45:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:41 smithi175 ceph-mon[32613]: 4.1c scrub ok 2024-09-21T11:45:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:41 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2532182494' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:41 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:45:41.554 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:41.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:41.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:41.599 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 39 16 73399 73383 117.573 81.125 0.00746258 0.00850046 2024-09-21T11:45:41.655 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:41.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:41.698 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:41.698 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:41.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:41.833 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:41.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:41.834 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:41.878 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:45:41.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:41.989 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:41.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:41.990 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:42.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:42.124 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:42.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:42.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:42.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:45:42.226 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:42.259 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:42.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:42.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:42.320 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:42.361 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:42.393 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:42.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:42.394 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:42.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:42.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:45:42.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:45:42.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:45:42.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[29710]: osdmap e182: 8 total, 8 up, 7 in 2024-09-21T11:45:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[29710]: 4.1 scrub starts 2024-09-21T11:45:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[29710]: 4.1 scrub ok 2024-09-21T11:45:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:45:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:45:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:45:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[36454]: osdmap e182: 8 total, 8 up, 7 in 2024-09-21T11:45:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[36454]: 4.1 scrub starts 2024-09-21T11:45:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:42 smithi043 ceph-mon[36454]: 4.1 scrub ok 2024-09-21T11:45:42.495 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:42.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:42.541 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:42.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:42.599 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:45:42.598247+0000 min lat: 0.00112969 max lat: 0.147407 avg lat: 0.00856557 2024-09-21T11:45:42.599 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:45:42.599 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 40 16 74712 74696 116.685 82.0625 0.0101251 0.00856557 2024-09-21T11:45:42.642 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:42.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:42.682 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:42.682 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:42 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:45:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:42 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:45:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:42 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:45:42.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:42 smithi175 ceph-mon[32613]: osdmap e182: 8 total, 8 up, 7 in 2024-09-21T11:45:42.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:42 smithi175 ceph-mon[32613]: 4.1 scrub starts 2024-09-21T11:45:42.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:42 smithi175 ceph-mon[32613]: 4.1 scrub ok 2024-09-21T11:45:42.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:42.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:42.821 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:42.821 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:42.922 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:42.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:42.959 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:42.959 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:43.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:43.097 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:43.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:43.098 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:43.199 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:43.228 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:43.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:43.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:43.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:43.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:43.362 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:43.363 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:43.395 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:45:43.395 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:45:43.427 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:45:43.427 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:45:43.463 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:43.469 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:45:43.469 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:45:43.509 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:43.510 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:45:43.510 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:45:43.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:43.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:43.553 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:45:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:45:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:43.594 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:45:43.595 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:45:43.595 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:43 smithi043 ceph-mon[29710]: pgmap v268: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-21T11:45:43.595 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:43 smithi043 ceph-mon[29710]: 3.8 scrub starts 2024-09-21T11:45:43.595 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:43 smithi043 ceph-mon[29710]: 3.8 scrub ok 2024-09-21T11:45:43.595 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:43 smithi043 ceph-mon[36454]: pgmap v268: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-21T11:45:43.596 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:43 smithi043 ceph-mon[36454]: 3.8 scrub starts 2024-09-21T11:45:43.596 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:43 smithi043 ceph-mon[36454]: 3.8 scrub ok 2024-09-21T11:45:43.601 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 41 16 75991 75975 115.784 79.9375 0.0143372 0.00863185 2024-09-21T11:45:43.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:43.641 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:45:43.642 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:45:43.670 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:43.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:43.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:43.678 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:45:43.679 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:45:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:43 smithi175 ceph-mon[32613]: pgmap v268: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-21T11:45:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:43 smithi175 ceph-mon[32613]: 3.8 scrub starts 2024-09-21T11:45:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:43 smithi175 ceph-mon[32613]: 3.8 scrub ok 2024-09-21T11:45:43.772 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:43.794 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:43.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:43.796 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:43.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:43.932 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:43.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:43.933 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:44.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:44.070 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:44.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:44.071 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:44.172 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:44.206 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:44.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:44.207 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:44.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:45:44.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:44.342 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:44.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:44.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:44.357 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T11:45:44.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:44.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:44.490 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:44.590 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:44.601 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 42 16 77268 77252 114.927 79.8125 0.0134779 0.00869626 2024-09-21T11:45:44.627 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:44.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:44.628 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:44.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:44 smithi043 ceph-mon[29710]: 6.10 scrub starts 2024-09-21T11:45:44.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:44 smithi043 ceph-mon[29710]: 6.10 scrub ok 2024-09-21T11:45:44.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/282590275' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:44.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3101342623' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:45:44.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:44 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:45:44.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:44 smithi043 ceph-mon[36454]: 6.10 scrub starts 2024-09-21T11:45:44.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:44 smithi043 ceph-mon[36454]: 6.10 scrub ok 2024-09-21T11:45:44.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/282590275' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:44.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3101342623' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:45:44.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:44 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:45:44.729 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:44.759 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:44.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:44.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:44.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:44 smithi175 ceph-mon[32613]: 6.10 scrub starts 2024-09-21T11:45:44.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:44 smithi175 ceph-mon[32613]: 6.10 scrub ok 2024-09-21T11:45:44.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/282590275' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:44.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3101342623' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:45:44.763 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:44 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:45:44.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:44.870 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:77614 8.c 8:3e27b486:::benchmark_data_smithi043_97042_object77613:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:44.849565+0000", 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018586602000000001, 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018621334, 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:44.871 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 77614 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849565+0000", 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849565+0000", 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849566+0000", 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0999999999999998e-07 2024-09-21T11:45:44.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849586+0000", 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0191e-05 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849593+0000", 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1250000000000004e-06 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849598+0000", 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5289999999999999e-06 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.873 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849919+0000", 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032043700000000002 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.850038+0000", 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118915 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.850168+0000", 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013063199999999999 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.862150+0000", 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011981986 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:77615 8.c 8:34af53ce:::benchmark_data_smithi043_97042_object77614:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:44.849609+0000", 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.018542629000000001, 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018661733, 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:44.875 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 77615 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849609+0000", 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849609+0000", 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:44.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849609+0000", 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.37e-07 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849660+0000", 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0365000000000003e-05 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849667+0000", 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1369999999999996e-06 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.849673+0000", 2024-09-21T11:45:44.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6840000000000001e-06 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.858454+0000", 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0087816100000000005 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.858592+0000", 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000138188 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:45:44.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.858736+0000", 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014372899999999999 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:77622 8.e 8:7ca27767:::benchmark_data_smithi043_97042_object77621:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:44.855572+0000", 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012579672, 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012765576000000001, 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:44.879 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 77622 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.855572+0000", 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.855572+0000", 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.855574+0000", 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7069999999999999e-06 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.855596+0000", 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2192e-05 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.855610+0000", 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3835000000000001e-05 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.855619+0000", 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0669999999999998e-06 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.856512+0000", 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00089295300000000004 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.856676+0000", 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016452 2024-09-21T11:45:44.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:45:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.856855+0000", 2024-09-21T11:45:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000178927 2024-09-21T11:45:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:45:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.859782+0000", 2024-09-21T11:45:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029267469999999999 2024-09-21T11:45:44.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861128+0000", 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001346187 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:77626 8.c 8:3c07996b:::benchmark_data_smithi043_97042_object77625:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:44.861358+0000", 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0067934689999999999, 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070518730000000002, 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:44.884 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 77626 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861358+0000", 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:44.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861358+0000", 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861360+0000", 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7349999999999999e-06 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861380+0000", 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0509e-05 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:44.886 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861395+0000", 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5139e-05 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861403+0000", 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.515e-06 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.862186+0000", 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00078328800000000004 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:44.887 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.862314+0000", 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012741400000000001 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.862436+0000", 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012279000000000001 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:77627 8.c 8.9e1a271c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:44.861421+0000", 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0067303290000000002, 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070449919999999999, 2024-09-21T11:45:44.888 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 77627 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861421+0000", 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:44.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861421+0000", 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861422+0000", 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0800000000000005e-07 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861441+0000", 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9080999999999999e-05 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:44.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861447+0000", 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7610000000000004e-06 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861452+0000", 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2220000000000001e-06 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:77631 8.e 8:7204e285:::benchmark_data_smithi043_97042_object77630:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e183)", 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:44.861459+0000", 2024-09-21T11:45:44.891 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0066920820000000002, 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070547350000000003, 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 77631 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:44.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861459+0000", 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861459+0000", 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861460+0000", 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.03e-07 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:44.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861478+0000", 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8508e-05 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861484+0000", 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5910000000000003e-06 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861489+0000", 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6600000000000003e-06 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861541+0000", 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2012999999999998e-05 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861671+0000", 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013010300000000001 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861937+0000", 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000266591 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.895 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:77628 8.9 e183/160 8:986adab0:::benchmark_data_smithi043_97042_object77627:head v 183'4853, mlcod=183'4850)", 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:44.861695+0000", 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0064568079999999996, 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0068784830000000003, 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:44.896 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861695+0000", 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861695+0000", 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861696+0000", 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8599999999999997e-07 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861761+0000", 2024-09-21T11:45:44.897 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5444000000000006e-05 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861768+0000", 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.861774+0000", 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8889999999999999e-06 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:45:44.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.864863+0000", 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003088712 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.864881+0000", 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8745000000000001e-05 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:77615 8.c e183/160)", 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:44.863551+0000", 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0046006479999999997, 2024-09-21T11:45:44.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050665390000000001, 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.863551+0000", 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.863551+0000", 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.863553+0000", 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.714e-06 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.863564+0000", 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1456e-05 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.863576+0000", 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.154e-05 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:45:44.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:45:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:45:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:44.863584+0000", 2024-09-21T11:45:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5380000000000002e-06 2024-09-21T11:45:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:45:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:45:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:45:44.902 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 8 2024-09-21T11:45:44.902 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:45:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:44.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:44.914 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:45.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:45.051 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:45.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:45.052 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:45.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:45.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:45.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:45.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:45.326 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:45.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:45.327 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:45.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:45.466 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:45.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:45.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:45.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:45:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:45:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[29710]: pgmap v269: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 927 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:45:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[29710]: 6.16 scrub starts 2024-09-21T11:45:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[29710]: 6.16 scrub ok 2024-09-21T11:45:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:45:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[29710]: osdmap e183: 8 total, 8 up, 7 in 2024-09-21T11:45:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[29710]: 4.18 scrub starts 2024-09-21T11:45:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[29710]: 4.18 scrub ok 2024-09-21T11:45:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[36454]: pgmap v269: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 927 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:45:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[36454]: 6.16 scrub starts 2024-09-21T11:45:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[36454]: 6.16 scrub ok 2024-09-21T11:45:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:45:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[36454]: osdmap e183: 8 total, 8 up, 7 in 2024-09-21T11:45:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[36454]: 4.18 scrub starts 2024-09-21T11:45:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:45 smithi043 ceph-mon[36454]: 4.18 scrub ok 2024-09-21T11:45:45.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:45.601 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 43 16 78678 78662 114.304 88.125 0.0161473 0.00874385 2024-09-21T11:45:45.604 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:45.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:45.605 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:45.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:45.743 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:45.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:45.745 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:45 smithi175 ceph-mon[32613]: pgmap v269: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 927 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:45:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:45 smithi175 ceph-mon[32613]: 6.16 scrub starts 2024-09-21T11:45:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:45 smithi175 ceph-mon[32613]: 6.16 scrub ok 2024-09-21T11:45:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:45 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:45:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:45 smithi175 ceph-mon[32613]: osdmap e183: 8 total, 8 up, 7 in 2024-09-21T11:45:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:45 smithi175 ceph-mon[32613]: 4.18 scrub starts 2024-09-21T11:45:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:45 smithi175 ceph-mon[32613]: 4.18 scrub ok 2024-09-21T11:45:45.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:45.883 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:45.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:45.884 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:45.984 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:46.024 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:46.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:46.025 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:46.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:46.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:46.161 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:46.161 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:46.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:46.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:46.302 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:46.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:46.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:46.446 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:46.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:46.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:46.548 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:46.590 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:46.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:46.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:46.604 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 44 16 80021 80005 113.608 83.9375 0.013398 0.00879722 2024-09-21T11:45:46.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:46.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 6 1 2024-09-21T11:45:46.750 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:46.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:46.751 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:46.791 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:46.791 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:46.791 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: 6.13 scrub starts 2024-09-21T11:45:46.791 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: 6.13 scrub ok 2024-09-21T11:45:46.791 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: pgmap v271: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.7 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:45:46.791 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: 3.10 scrub starts 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: 3.10 scrub ok 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: 6.13 scrub starts 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: 6.13 scrub ok 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: pgmap v271: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.7 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:45:46.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:45:46.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: 3.10 scrub starts 2024-09-21T11:45:46.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: 3.10 scrub ok 2024-09-21T11:45:46.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:46.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:46.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:46.793 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:46 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:46.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:46.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:46.916 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:45:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: 6.13 scrub starts 2024-09-21T11:45:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: 6.13 scrub ok 2024-09-21T11:45:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: pgmap v271: 148 pgs: 148 active+clean; 4.5 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 1.7 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:45:47.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:45:47.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: 3.10 scrub starts 2024-09-21T11:45:47.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: 3.10 scrub ok 2024-09-21T11:45:47.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:47.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:47.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:47.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:46 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:47.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:47.055 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:47.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:47.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:47.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:47.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:47.194 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:47.194 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:47.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:47.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:45:47.330 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:47.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:47.331 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:47.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:47.473 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:47.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:47.474 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:47.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:47.604 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 45 16 81216 81200 112.742 74.6875 0.0378493 0.00886361 2024-09-21T11:45:47.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:47.622 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:47.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:47.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:47.806 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:47.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:47.808 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:47.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:47.955 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:47.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:47.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:47 smithi043 ceph-mon[29710]: 4.11 scrub starts 2024-09-21T11:45:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:47 smithi043 ceph-mon[29710]: 4.11 scrub ok 2024-09-21T11:45:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:47 smithi043 ceph-mon[29710]: osdmap e184: 8 total, 8 up, 7 in 2024-09-21T11:45:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:47 smithi043 ceph-mon[36454]: 4.11 scrub starts 2024-09-21T11:45:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:47 smithi043 ceph-mon[36454]: 4.11 scrub ok 2024-09-21T11:45:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:47 smithi043 ceph-mon[36454]: osdmap e184: 8 total, 8 up, 7 in 2024-09-21T11:45:48.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:47 smithi175 ceph-mon[32613]: 4.11 scrub starts 2024-09-21T11:45:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:47 smithi175 ceph-mon[32613]: 4.11 scrub ok 2024-09-21T11:45:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:47 smithi175 ceph-mon[32613]: osdmap e184: 8 total, 8 up, 7 in 2024-09-21T11:45:48.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:48.094 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:48.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:48.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:48.108 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:48.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:48.251 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:48.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:48.253 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:48.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:48.392 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:48.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:48.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:48.494 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:48.523 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:48.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:48.524 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:48.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:45:48.604 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 46 16 82323 82307 111.795 69.1875 0.00934544 0.00894007 2024-09-21T11:45:48.625 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:48.657 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:48.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:48.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:48 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:45:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:48 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:45:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:48 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:48 smithi043 ceph-mon[29710]: pgmap v273: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 787 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-21T11:45:48.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:48 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2927276569' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:45:48.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:48 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:45:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:48 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:45:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:48 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:48 smithi043 ceph-mon[36454]: pgmap v273: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 787 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-21T11:45:48.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:48 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2927276569' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:45:48.758 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:48.784 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:48.785 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:48.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:48.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:48.839 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:45:48.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:48.915 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:48.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:48.916 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:48 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:45:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:48 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:45:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:48 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:45:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:48 smithi175 ceph-mon[32613]: pgmap v273: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 787 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-21T11:45:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:48 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2927276569' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:45:49.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:49.047 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:49.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:49.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:49.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:49.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:49.191 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:49.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:49.317 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:49.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:49.319 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:49.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:49.463 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:49.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:49.464 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:49.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:49.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:49.599 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:49.599 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:49.604 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 47 16 83378 83362 110.819 65.9375 0.00420694 0.00901848 2024-09-21T11:45:49.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:49.736 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:49.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:49.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:49.837 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T11:45:49.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:49.875 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:49.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:49.876 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:49.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[29710]: 4.1e scrub starts 2024-09-21T11:45:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[29710]: 4.1e scrub ok 2024-09-21T11:45:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2927276569' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:45:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[29710]: osdmap e185: 8 total, 8 up, 7 in 2024-09-21T11:45:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3115455240' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:45:49.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[36454]: 4.1e scrub starts 2024-09-21T11:45:49.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[36454]: 4.1e scrub ok 2024-09-21T11:45:49.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2927276569' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:45:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[36454]: osdmap e185: 8 total, 8 up, 7 in 2024-09-21T11:45:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3115455240' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:45:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:49 smithi175 ceph-mon[32613]: 4.1e scrub starts 2024-09-21T11:45:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:49 smithi175 ceph-mon[32613]: 4.1e scrub ok 2024-09-21T11:45:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:49 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:45:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2927276569' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:45:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:49 smithi175 ceph-mon[32613]: osdmap e185: 8 total, 8 up, 7 in 2024-09-21T11:45:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3115455240' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:45:50.019 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:50.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:50.020 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:50.121 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:50.161 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:50.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:50.162 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:50.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:50.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:50.302 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:50.302 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:50.403 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:50.445 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:50.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:50.446 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:50.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:50.578 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:50.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:50.579 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:50.604 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 48 15 84638 84623 110.153 78.8125 0.00648662 0.009073 2024-09-21T11:45:50.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:50.716 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:50.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:50.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:50.817 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:50.850 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:50.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:50.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:50.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:50.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:50.991 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:50.991 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:51.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:50 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:45:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:50 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:45:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:50 smithi175 ceph-mon[32613]: pgmap v275: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 853 B/s rd, 70 MiB/s wr, 1.11k op/s 2024-09-21T11:45:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3115455240' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:45:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:50 smithi175 ceph-mon[32613]: osdmap e186: 8 total, 8 up, 7 in 2024-09-21T11:45:51.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:51.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:51.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:51.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:50 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:45:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:50 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:45:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:50 smithi043 ceph-mon[36454]: pgmap v275: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 853 B/s rd, 70 MiB/s wr, 1.11k op/s 2024-09-21T11:45:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3115455240' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:45:51.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:50 smithi043 ceph-mon[36454]: osdmap e186: 8 total, 8 up, 7 in 2024-09-21T11:45:51.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:50 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:45:51.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:50 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:45:51.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:50 smithi043 ceph-mon[29710]: pgmap v275: 148 pgs: 148 active+clean; 4.8 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 853 B/s rd, 70 MiB/s wr, 1.11k op/s 2024-09-21T11:45:51.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3115455240' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:45:51.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:50 smithi043 ceph-mon[29710]: osdmap e186: 8 total, 8 up, 7 in 2024-09-21T11:45:51.264 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:51.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:51.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:51.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:51.401 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:51.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:51.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:51.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:51.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:45:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:45:51.534 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:51.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:51.535 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:51.604 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 49 16 85933 85917 109.555 80.875 0.00669327 0.0091229 2024-09-21T11:45:51.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:51.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:51.658 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:51.759 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:51.794 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:51.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:51.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:51.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:51.938 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:51.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:51.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:52.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:52.078 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:52.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:52.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:52.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:52.217 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:52.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:52.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:51 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:52.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:51 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:52.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:51 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:45:52.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:52.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:52.346 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:52.346 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:52.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:52.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:52.484 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:52.585 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:52.604 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 50 16 87100 87084 108.823 72.9375 0.0110145 0.00918421 2024-09-21T11:45:52.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:52.627 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:52.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:52.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:52.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:52.765 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:52.766 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:52.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:52.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:52.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:52.949 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:45:52.949 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:45:52.949 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:45:52.949 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:45:52.949 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:45:53.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:53.046 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:53.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:53.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:53.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:53.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:53.186 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:53.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:53.286 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:53.327 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:53.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:53.328 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:53.429 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:53.449 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:53.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:53.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:53.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:53 smithi043 ceph-mon[36454]: pgmap v277: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 853 B/s rd, 127 MiB/s wr, 2.03k op/s 2024-09-21T11:45:53.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:53 smithi043 ceph-mon[29710]: pgmap v277: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 853 B/s rd, 127 MiB/s wr, 2.03k op/s 2024-09-21T11:45:53.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:53 smithi175 ceph-mon[32613]: pgmap v277: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 853 B/s rd, 127 MiB/s wr, 2.03k op/s 2024-09-21T11:45:53.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:53.587 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:53.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:53.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:53.604 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 51 16 88477 88461 108.376 86.0625 0.00915375 0.00922271 2024-09-21T11:45:53.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:53.729 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:53.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:53.730 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:53.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:53.864 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:53.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:53.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:53.966 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:54.002 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:54.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:54.003 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:54.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:54.140 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:54.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:54.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:54.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:54.277 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:54.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:54.278 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:54.378 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:54.415 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:54.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:54.416 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:54.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:54.553 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:54.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:54.554 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:54.604 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 52 16 89823 89807 107.91 84.125 0.0107843 0.00926198 2024-09-21T11:45:54.655 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:54.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:54.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:54.797 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:54.836 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:54.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:54.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:54.937 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:54.973 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:54.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:54.974 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:55.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:55.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:55.112 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:55.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:55.254 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:55.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:55.255 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:55.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:55.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:55.397 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:55.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 7 1 2024-09-21T11:45:55.480 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:45:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:45:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:45:55.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:55 smithi043 ceph-mon[29710]: pgmap v278: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 773 B/s rd, 56 MiB/s wr, 890 op/s 2024-09-21T11:45:55.481 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:55 smithi043 ceph-mon[36454]: pgmap v278: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 773 B/s rd, 56 MiB/s wr, 890 op/s 2024-09-21T11:45:55.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:55.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:55 smithi175 ceph-mon[32613]: pgmap v278: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 773 B/s rd, 56 MiB/s wr, 890 op/s 2024-09-21T11:45:55.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:55.551 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:55.551 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:55.605 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 53 16 91223 91207 107.524 87.5 0.00937846 0.00929549 2024-09-21T11:45:55.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:55.693 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:55.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:55.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:55.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:55.836 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:55.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:55.837 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:55.938 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:55.976 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:55.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:55.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:56.039 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:56.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:45:56.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:56.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:56.145 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:56.145 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:56.246 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:56.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:56.282 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:56.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:56.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:56.411 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:56.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:56.412 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:56.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:56.549 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:56.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:56.550 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:56.605 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 54 15 92493 92478 107.004 79.4375 0.0211711 0.00934094 2024-09-21T11:45:56.644 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:56.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:56.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:56.717 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:56.718 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:45:56.718 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:45:56.755 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:45:56.755 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:45:56.786 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:45:56.786 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:45:56.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:56.822 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:45:56.822 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:45:56.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:56.850 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:56.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:56.856 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:45:56.856 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:45:56.899 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:45:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:45:56.930 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:45:56.930 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:45:56.950 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:56.966 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:45:56.966 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:45:56.982 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:56.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:56.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:57.083 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:57.121 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:57.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:57.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:57.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:57.267 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:57.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:57.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:57.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:57.412 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:57.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:57.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:57.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:45:57.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:45:57.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[36454]: pgmap v279: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 790 op/s 2024-09-21T11:45:57.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[36454]: osdmap e187: 8 total, 8 up, 7 in 2024-09-21T11:45:57.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4039079713' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:57.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[36454]: osdmap e188: 8 total, 8 up, 7 in 2024-09-21T11:45:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:45:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:45:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[29710]: pgmap v279: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 790 op/s 2024-09-21T11:45:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[29710]: osdmap e187: 8 total, 8 up, 7 in 2024-09-21T11:45:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4039079713' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:57.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:57 smithi043 ceph-mon[29710]: osdmap e188: 8 total, 8 up, 7 in 2024-09-21T11:45:57.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:57 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:45:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:57 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:45:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:57 smithi175 ceph-mon[32613]: pgmap v279: 148 pgs: 148 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 790 op/s 2024-09-21T11:45:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:57 smithi175 ceph-mon[32613]: osdmap e187: 8 total, 8 up, 7 in 2024-09-21T11:45:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:57 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4039079713' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:45:57.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:57 smithi175 ceph-mon[32613]: osdmap e188: 8 total, 8 up, 7 in 2024-09-21T11:45:57.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:45:57.552 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:57.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:57.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:45:57.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:45:57.605 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 55 16 93794 93778 106.536 81.25 0.0100182 0.00938189 2024-09-21T11:45:57.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:57.699 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:57.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:57.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:57.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:45:57.835 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:57.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:57.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:45:57.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:57.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:57.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:58.082 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:58.101 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:58.107 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:58.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:58.108 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:58.209 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:45:58.240 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:58.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:58.241 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:45:58.341 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:45:58.389 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:58.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:58.390 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:45:58.491 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:58.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:58.521 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:58.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:58.605 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 56 16 95083 95067 106.072 80.5625 0.00813766 0.00942349 2024-09-21T11:45:58.622 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:58.661 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:58.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:58.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:58.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:58.798 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:58.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:58.799 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:58.817 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:45:58.838 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:45:58.838 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:45:58.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:58.941 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:58.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:58.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:59.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:59.084 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:59.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:59.085 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:59.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:59.219 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:59.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:59.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T11:45:59.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:59.356 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:59.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:59.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:45:59.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:45:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:59 smithi043 ceph-mon[29710]: pgmap v282: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.3 KiB/s rd, 98 MiB/s wr, 1.56k op/s 2024-09-21T11:45:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:45:59 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2934611859' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:45:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:59 smithi043 ceph-mon[36454]: pgmap v282: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.3 KiB/s rd, 98 MiB/s wr, 1.56k op/s 2024-09-21T11:45:59.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:45:59 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2934611859' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:45:59.494 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:59.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:59.495 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:45:59.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:59 smithi175 ceph-mon[32613]: pgmap v282: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 1.3 KiB/s rd, 98 MiB/s wr, 1.56k op/s 2024-09-21T11:45:59.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:45:59 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2934611859' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:45:59.596 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:45:59.605 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 57 16 96451 96435 105.711 85.5 0.0161259 0.00945481 2024-09-21T11:45:59.627 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:45:59.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:59.629 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:45:59.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:45:59.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:45:59.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:59.772 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:59.772 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:45:59.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:45:59.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:45:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:45:59.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:45:59.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:00.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:00.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:00.075 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:00.075 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:00.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:00.216 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:00.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:00.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:00.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:00.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:00.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:00.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:00 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:46:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:00 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:46:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2934611859' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:46:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:00 smithi043 ceph-mon[36454]: osdmap e189: 8 total, 8 up, 7 in 2024-09-21T11:46:00.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:00 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:46:00.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:00 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:46:00.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2934611859' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:46:00.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:00 smithi043 ceph-mon[29710]: osdmap e189: 8 total, 8 up, 7 in 2024-09-21T11:46:00.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:00.507 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:00.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:00.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:00 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:46:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:00 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:46:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2934611859' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:46:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:00 smithi175 ceph-mon[32613]: osdmap e189: 8 total, 8 up, 7 in 2024-09-21T11:46:00.605 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 58 16 97738 97722 105.275 80.4375 0.0215704 0.00949436 2024-09-21T11:46:00.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:00.648 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:00.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:00.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:00.750 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:00.786 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:00.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:00.787 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:00.888 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:00.900 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T11:46:00.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:46:00.917 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:00.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:00.918 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:00.931 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:01.002 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:01.019 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:01.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:01.086 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:01.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:01.234 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:01.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:01.235 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:01.336 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:01.385 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:01.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:01.386 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:01.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:01 smithi043 ceph-mon[36454]: pgmap v284: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-21T11:46:01.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:01 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:46:01.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:01.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3604549372' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:01.400 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:01 smithi043 ceph-mon[29710]: pgmap v284: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-21T11:46:01.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:01 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:46:01.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:01.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3604549372' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:01.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:01 smithi175 ceph-mon[32613]: pgmap v284: 148 pgs: 148 active+clean; 5.6 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 853 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-21T11:46:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:01 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:46:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3604549372' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:01.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:46:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:46:01.521 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:01.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:01.522 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:01.580 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:46:01.581 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:46:01.605 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 59 16 98982 98966 104.809 77.75 0.00591891 0.00953591 2024-09-21T11:46:01.615 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:46:01.615 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:46:01.623 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:01.647 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:46:01.648 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:46:01.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:01.665 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:01.666 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:01.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:46:01.685 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:46:01.703 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:46:01.720 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:46:01.720 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:46:01.759 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:46:01.759 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:46:01.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:01.800 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:46:01.800 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:46:01.815 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:01.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:01.817 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:01.839 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:46:01.839 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:46:01.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:01.956 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:01.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:01.957 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:02.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:02.100 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:02.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:02.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:02.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:02.236 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:02.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:02.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:02.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:02.375 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:02.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:02.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:02.477 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3158744257' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:02 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3604549372' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:02.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:02 smithi043 ceph-mon[29710]: osdmap e190: 8 total, 8 up, 7 in 2024-09-21T11:46:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3158744257' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:02.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:02 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3604549372' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:02.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:02 smithi043 ceph-mon[36454]: osdmap e190: 8 total, 8 up, 7 in 2024-09-21T11:46:02.515 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:02.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:02.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:02.605 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:46:02.604871+0000 min lat: 0.00112969 max lat: 0.147407 avg lat: 0.00957174 2024-09-21T11:46:02.605 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:46:02.606 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 60 16 100289 100273 104.423 81.6875 0.0130929 0.00957174 2024-09-21T11:46:02.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:02.654 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:02.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:02.655 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:02.756 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3158744257' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:02 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3604549372' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:02 smithi175 ceph-mon[32613]: osdmap e190: 8 total, 8 up, 7 in 2024-09-21T11:46:02.795 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:02.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:02.796 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:02.857 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:46:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:46:02.857 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-21T11:46:02.857 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:46:02.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:02.931 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:02.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:02.932 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:03.033 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:03.061 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:03.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:03.062 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:03.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:03.199 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:03.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:03.200 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:03.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:03.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:03.339 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:03.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:03.441 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:03 smithi043 ceph-mon[36454]: pgmap v286: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 912 B/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-21T11:46:03.442 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:03 smithi043 ceph-mon[29710]: pgmap v286: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 912 B/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-21T11:46:03.495 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:03.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:03.496 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:03.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:03.606 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 61 16 101693 101677 104.149 87.75 0.0115168 0.00959729 2024-09-21T11:46:03.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:03.630 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:03.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:03.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:03.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:03 smithi175 ceph-mon[32613]: pgmap v286: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 912 B/s rd, 109 MiB/s wr, 1.75k op/s 2024-09-21T11:46:03.771 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:03.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:03.773 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:03.873 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:03.911 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:03.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:03.912 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:04.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:04.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:04.046 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:04.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:04.148 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:04.187 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:04.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:04.188 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:04.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:46:04.251 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:04.289 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:04.321 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:04.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:04.322 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:04.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:04.471 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:04.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:04.472 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:04.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:46:04.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:04.605 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:04.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:04.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:04.606 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 62 16 103072 103056 103.859 86.1875 0.0089943 0.00962331 2024-09-21T11:46:04.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:04.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:04.757 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:04.757 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:04.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:04.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:04.898 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:04.898 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:04.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:05.002 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:05.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:05.071 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:05.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:05.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:05.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:05.213 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:05.213 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:05.314 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:05.334 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:46:05.341 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:46:05.351 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:05.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:05.352 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:05.453 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:05.489 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:05.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:05.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:46:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:46:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:46:05.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:05 smithi043 ceph-mon[36454]: pgmap v287: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 771 B/s rd, 61 MiB/s wr, 980 op/s 2024-09-21T11:46:05.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:05 smithi043 ceph-mon[29710]: pgmap v287: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 771 B/s rd, 61 MiB/s wr, 980 op/s 2024-09-21T11:46:05.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:46:05.591 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:05.607 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 63 16 104290 104274 103.418 76.125 0.00635283 0.00966487 2024-09-21T11:46:05.622 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:05.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:05.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:05.726 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:05.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:05 smithi175 ceph-mon[32613]: pgmap v287: 148 pgs: 148 active+clean; 6.0 GiB data, 19 GiB used, 606 GiB / 626 GiB avail; 771 B/s rd, 61 MiB/s wr, 980 op/s 2024-09-21T11:46:05.769 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:05.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:05.770 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:05.871 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:05.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:05.906 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:05.907 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:06.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:06.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:06.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:06.080 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-21T11:46:03.845323+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6445865499,"num_objects":98546,"num_object_clones":0,"num_object_copies":295638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98546,"num_whiteouts":0,"num_read":1190,"num_read_kb":1047,"num_write":98896,"num_write_kb":6295881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":594233,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76603,"ondisk_log_size":76603,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":20409240,"kb_used_data":18770708,"kb_used_omap":10,"kb_used_meta":1638389,"kb_avail":635806824,"statfs":{"total":671965249536,"available":651066187776,"internally_reserved":0,"allocated":19221204992,"data_stored":19217389025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11156,"internal_metadata":1677710444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":425787392,"num_objects":6497,"num_object_clones":0,"num_object_copies":19491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6497,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6497,"num_write_kb":415808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.634642"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":281,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.254947+0000","last_change":"2024-09-21T11:41:42.291880+0000","last_active":"2024-09-21T11:45:59.254947+0000","last_peered":"2024-09-21T11:45:59.254947+0000","last_clean":"2024-09-21T11:45:59.254947+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:45:59.254947+0000","last_undegraded":"2024-09-21T11:45:59.254947+0000","last_fullsized":"2024-09-21T11:45:59.254947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:42.290188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:42.290188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.279318+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":327,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.237826+0000","last_change":"2024-09-21T11:45:44.553786+0000","last_active":"2024-09-21T11:45:59.237826+0000","last_peered":"2024-09-21T11:45:59.237826+0000","last_clean":"2024-09-21T11:45:59.237826+0000","last_became_active":"2024-09-21T11:42:37.164858+0000","last_became_peered":"2024-09-21T11:42:37.164858+0000","last_unstale":"2024-09-21T11:45:59.237826+0000","last_undegraded":"2024-09-21T11:45:59.237826+0000","last_fullsized":"2024-09-21T11:45:59.237826+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:44.553561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.553561+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:10.417329+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":175,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247620+0000","last_change":"2024-09-21T11:43:37.050216+0000","last_active":"2024-09-21T11:45:59.247620+0000","last_peered":"2024-09-21T11:45:59.247620+0000","last_clean":"2024-09-21T11:45:59.247620+0000","last_became_active":"2024-09-21T11:42:37.161713+0000","last_became_peered":"2024-09-21T11:42:37.161713+0000","last_unstale":"2024-09-21T11:45:59.247620+0000","last_undegraded":"2024-09-21T11:45:59.247620+0000","last_fullsized":"2024-09-21T11:45:59.247620+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:57.113849+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":285,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.237837+0000","last_change":"2024-09-21T11:45:07.831102+0000","last_active":"2024-09-21T11:45:59.237837+0000","last_peered":"2024-09-21T11:45:59.237837+0000","last_clean":"2024-09-21T11:45:59.237837+0000","last_became_active":"2024-09-21T11:42:37.158907+0000","last_became_peered":"2024-09-21T11:42:37.158907+0000","last_unstale":"2024-09-21T11:45:59.237837+0000","last_undegraded":"2024-09-21T11:45:59.237837+0000","last_fullsized":"2024-09-21T11:45:59.237837+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:07.831036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.831036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:24.100624+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.244551+0000","last_change":"2024-09-21T11:43:28.461458+0000","last_active":"2024-09-21T11:45:59.244551+0000","last_peered":"2024-09-21T11:45:59.244551+0000","last_clean":"2024-09-21T11:45:59.244551+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:45:59.244551+0000","last_undegraded":"2024-09-21T11:45:59.244551+0000","last_fullsized":"2024-09-21T11:45:59.244551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.461378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.461378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:01.775665+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":274,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241227+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:45:59.241227+0000","last_peered":"2024-09-21T11:45:59.241227+0000","last_clean":"2024-09-21T11:45:59.241227+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:45:59.241227+0000","last_undegraded":"2024-09-21T11:45:59.241227+0000","last_fullsized":"2024-09-21T11:45:59.241227+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:55.591623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":303,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.240644+0000","last_change":"2024-09-21T11:45:07.788590+0000","last_active":"2024-09-21T11:45:59.240644+0000","last_peered":"2024-09-21T11:45:59.240644+0000","last_clean":"2024-09-21T11:45:59.240644+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:45:59.240644+0000","last_undegraded":"2024-09-21T11:45:59.240644+0000","last_fullsized":"2024-09-21T11:45:59.240644+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:07.788512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.788512+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:22.182413+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.237747+0000","last_change":"2024-09-21T11:45:08.781900+0000","last_active":"2024-09-21T11:45:59.237747+0000","last_peered":"2024-09-21T11:45:59.237747+0000","last_clean":"2024-09-21T11:45:59.237747+0000","last_became_active":"2024-09-21T11:42:37.154511+0000","last_became_peered":"2024-09-21T11:42:37.154511+0000","last_unstale":"2024-09-21T11:45:59.237747+0000","last_undegraded":"2024-09-21T11:45:59.237747+0000","last_fullsized":"2024-09-21T11:45:59.237747+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:21.829040+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":281,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247900+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:45:59.247900+0000","last_peered":"2024-09-21T11:45:59.247900+0000","last_clean":"2024-09-21T11:45:59.247900+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:45:59.247900+0000","last_undegraded":"2024-09-21T11:45:59.247900+0000","last_fullsized":"2024-09-21T11:45:59.247900+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:31.264231+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":302,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238178+0000","last_change":"2024-09-21T11:43:26.258023+0000","last_active":"2024-09-21T11:45:59.238178+0000","last_peered":"2024-09-21T11:45:59.238178+0000","last_clean":"2024-09-21T11:45:59.238178+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:45:59.238178+0000","last_undegraded":"2024-09-21T11:45:59.238178+0000","last_fullsized":"2024-09-21T11:45:59.238178+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:26.257967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.257967+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":281,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247644+0000","last_change":"2024-09-21T11:44:12.257513+0000","last_active":"2024-09-21T11:45:59.247644+0000","last_peered":"2024-09-21T11:45:59.247644+0000","last_clean":"2024-09-21T11:45:59.247644+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:45:59.247644+0000","last_undegraded":"2024-09-21T11:45:59.247644+0000","last_fullsized":"2024-09-21T11:45:59.247644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.257454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.257454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:18.340172+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":284,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.248584+0000","last_change":"2024-09-21T11:45:06.385208+0000","last_active":"2024-09-21T11:45:59.248584+0000","last_peered":"2024-09-21T11:45:59.248584+0000","last_clean":"2024-09-21T11:45:59.248584+0000","last_became_active":"2024-09-21T11:42:37.155879+0000","last_became_peered":"2024-09-21T11:42:37.155879+0000","last_unstale":"2024-09-21T11:45:59.248584+0000","last_undegraded":"2024-09-21T11:45:59.248584+0000","last_fullsized":"2024-09-21T11:45:59.248584+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:06.385125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:06.385125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:25.118118+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":304,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246815+0000","last_change":"2024-09-21T11:45:36.172885+0000","last_active":"2024-09-21T11:45:59.246815+0000","last_peered":"2024-09-21T11:45:59.246815+0000","last_clean":"2024-09-21T11:45:59.246815+0000","last_became_active":"2024-09-21T11:42:37.156575+0000","last_became_peered":"2024-09-21T11:42:37.156575+0000","last_unstale":"2024-09-21T11:45:59.246815+0000","last_undegraded":"2024-09-21T11:45:59.246815+0000","last_fullsized":"2024-09-21T11:45:59.246815+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:02.715601+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":309,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238367+0000","last_change":"2024-09-21T11:43:08.326923+0000","last_active":"2024-09-21T11:45:59.238367+0000","last_peered":"2024-09-21T11:45:59.238367+0000","last_clean":"2024-09-21T11:45:59.238367+0000","last_became_active":"2024-09-21T11:41:31.634863+0000","last_became_peered":"2024-09-21T11:41:31.634863+0000","last_unstale":"2024-09-21T11:45:59.238367+0000","last_undegraded":"2024-09-21T11:45:59.238367+0000","last_fullsized":"2024-09-21T11:45:59.238367+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:08.326846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:08.326846+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:27.597564+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":280,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249162+0000","last_change":"2024-09-21T11:43:38.567194+0000","last_active":"2024-09-21T11:45:59.249162+0000","last_peered":"2024-09-21T11:45:59.249162+0000","last_clean":"2024-09-21T11:45:59.249162+0000","last_became_active":"2024-09-21T11:42:37.164285+0000","last_became_peered":"2024-09-21T11:42:37.164285+0000","last_unstale":"2024-09-21T11:45:59.249162+0000","last_undegraded":"2024-09-21T11:45:59.249162+0000","last_fullsized":"2024-09-21T11:45:59.249162+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.567131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.567131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:24.764953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":284,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247711+0000","last_change":"2024-09-21T11:44:42.350677+0000","last_active":"2024-09-21T11:45:59.247711+0000","last_peered":"2024-09-21T11:45:59.247711+0000","last_clean":"2024-09-21T11:45:59.247711+0000","last_became_active":"2024-09-21T11:42:37.162912+0000","last_became_peered":"2024-09-21T11:42:37.162912+0000","last_unstale":"2024-09-21T11:45:59.247711+0000","last_undegraded":"2024-09-21T11:45:59.247711+0000","last_fullsized":"2024-09-21T11:45:59.247711+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:42.350621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:42.350621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:07.179165+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238590+0000","last_change":"2024-09-21T11:44:38.861427+0000","last_active":"2024-09-21T11:45:59.238590+0000","last_peered":"2024-09-21T11:45:59.238590+0000","last_clean":"2024-09-21T11:45:59.238590+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:45:59.238590+0000","last_undegraded":"2024-09-21T11:45:59.238590+0000","last_fullsized":"2024-09-21T11:45:59.238590+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:38.861369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:38.861369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:51.513431+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":281,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.251656+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:45:59.251656+0000","last_peered":"2024-09-21T11:45:59.251656+0000","last_clean":"2024-09-21T11:45:59.251656+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:45:59.251656+0000","last_undegraded":"2024-09-21T11:45:59.251656+0000","last_fullsized":"2024-09-21T11:45:59.251656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":365,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246149+0000","last_change":"2024-09-21T11:45:40.530176+0000","last_active":"2024-09-21T11:45:59.246149+0000","last_peered":"2024-09-21T11:45:59.246149+0000","last_clean":"2024-09-21T11:45:59.246149+0000","last_became_active":"2024-09-21T11:42:37.166901+0000","last_became_peered":"2024-09-21T11:42:37.166901+0000","last_unstale":"2024-09-21T11:45:59.246149+0000","last_undegraded":"2024-09-21T11:45:59.246149+0000","last_fullsized":"2024-09-21T11:45:59.246149+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:40.530099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:40.530099+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:41.983312+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":183,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238305+0000","last_change":"2024-09-21T11:44:46.822178+0000","last_active":"2024-09-21T11:45:59.238305+0000","last_peered":"2024-09-21T11:45:59.238305+0000","last_clean":"2024-09-21T11:45:59.238305+0000","last_became_active":"2024-09-21T11:42:37.153799+0000","last_became_peered":"2024-09-21T11:42:37.153799+0000","last_unstale":"2024-09-21T11:45:59.238305+0000","last_undegraded":"2024-09-21T11:45:59.238305+0000","last_fullsized":"2024-09-21T11:45:59.238305+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:46.822132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:46.822132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.312064+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":278,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241490+0000","last_change":"2024-09-21T11:45:39.826902+0000","last_active":"2024-09-21T11:45:59.241490+0000","last_peered":"2024-09-21T11:45:59.241490+0000","last_clean":"2024-09-21T11:45:59.241490+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:45:59.241490+0000","last_undegraded":"2024-09-21T11:45:59.241490+0000","last_fullsized":"2024-09-21T11:45:59.241490+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:39.826701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:39.826701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:58.518406+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238306+0000","last_change":"2024-09-21T11:43:45.093396+0000","last_active":"2024-09-21T11:45:59.238306+0000","last_peered":"2024-09-21T11:45:59.238306+0000","last_clean":"2024-09-21T11:45:59.238306+0000","last_became_active":"2024-09-21T11:42:37.153694+0000","last_became_peered":"2024-09-21T11:42:37.153694+0000","last_unstale":"2024-09-21T11:45:59.238306+0000","last_undegraded":"2024-09-21T11:45:59.238306+0000","last_fullsized":"2024-09-21T11:45:59.238306+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.093325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.093325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:23.661922+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":320,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.240751+0000","last_change":"2024-09-21T11:42:37.189830+0000","last_active":"2024-09-21T11:45:59.240751+0000","last_peered":"2024-09-21T11:45:59.240751+0000","last_clean":"2024-09-21T11:45:59.240751+0000","last_became_active":"2024-09-21T11:42:37.165964+0000","last_became_peered":"2024-09-21T11:42:37.165964+0000","last_unstale":"2024-09-21T11:45:59.240751+0000","last_undegraded":"2024-09-21T11:45:59.240751+0000","last_fullsized":"2024-09-21T11:45:59.240751+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:41:35.427210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:35.427210+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:32.829672+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":282,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.239204+0000","last_change":"2024-09-21T11:45:37.670176+0000","last_active":"2024-09-21T11:45:59.239204+0000","last_peered":"2024-09-21T11:45:59.239204+0000","last_clean":"2024-09-21T11:45:59.239204+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:45:59.239204+0000","last_undegraded":"2024-09-21T11:45:59.239204+0000","last_fullsized":"2024-09-21T11:45:59.239204+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:37.670117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:37.670117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:07.566687+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249368+0000","last_change":"2024-09-21T11:45:14.660863+0000","last_active":"2024-09-21T11:45:59.249368+0000","last_peered":"2024-09-21T11:45:59.249368+0000","last_clean":"2024-09-21T11:45:59.249368+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:45:59.249368+0000","last_undegraded":"2024-09-21T11:45:59.249368+0000","last_fullsized":"2024-09-21T11:45:59.249368+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:45:14.660727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:14.660727+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:35.024056+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":187,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.240607+0000","last_change":"2024-09-21T11:44:44.796081+0000","last_active":"2024-09-21T11:45:59.240607+0000","last_peered":"2024-09-21T11:45:59.240607+0000","last_clean":"2024-09-21T11:45:59.240607+0000","last_became_active":"2024-09-21T11:42:37.166144+0000","last_became_peered":"2024-09-21T11:42:37.166144+0000","last_unstale":"2024-09-21T11:45:59.240607+0000","last_undegraded":"2024-09-21T11:45:59.240607+0000","last_fullsized":"2024-09-21T11:45:59.240607+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:44:44.796015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.796015+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":308,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.254913+0000","last_change":"2024-09-21T11:45:48.385970+0000","last_active":"2024-09-21T11:45:59.254913+0000","last_peered":"2024-09-21T11:45:59.254913+0000","last_clean":"2024-09-21T11:45:59.254913+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:45:59.254913+0000","last_undegraded":"2024-09-21T11:45:59.254913+0000","last_fullsized":"2024-09-21T11:45:59.254913+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:45:48.385895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:48.385895+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:08.226832+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":381,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.245115+0000","last_change":"2024-09-21T11:45:07.351708+0000","last_active":"2024-09-21T11:45:59.245115+0000","last_peered":"2024-09-21T11:45:59.245115+0000","last_clean":"2024-09-21T11:45:59.245115+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:45:59.245115+0000","last_undegraded":"2024-09-21T11:45:59.245115+0000","last_fullsized":"2024-09-21T11:45:59.245115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:45:07.351662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.351662+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.953626+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"189'6139","reported_seq":6209,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:00.822122+0000","last_change":"2024-09-21T11:44:51.412099+0000","last_active":"2024-09-21T11:46:00.822122+0000","last_peered":"2024-09-21T11:46:00.822122+0000","last_clean":"2024-09-21T11:46:00.822122+0000","last_became_active":"2024-09-21T11:44:51.411794+0000","last_became_peered":"2024-09-21T11:44:51.411794+0000","last_unstale":"2024-09-21T11:46:00.822122+0000","last_undegraded":"2024-09-21T11:46:00.822122+0000","last_fullsized":"2024-09-21T11:46:00.822122+0000","mapping_epoch":160,"log_start":"168'1300","ondisk_log_start":"168'1300","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":402325504,"num_objects":6139,"num_object_clones":0,"num_object_copies":18417,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6139,"num_write_kb":392896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":235,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241583+0000","last_change":"2024-09-21T11:43:44.440472+0000","last_active":"2024-09-21T11:45:59.241583+0000","last_peered":"2024-09-21T11:45:59.241583+0000","last_clean":"2024-09-21T11:45:59.241583+0000","last_became_active":"2024-09-21T11:42:37.169939+0000","last_became_peered":"2024-09-21T11:42:37.169939+0000","last_unstale":"2024-09-21T11:45:59.241583+0000","last_undegraded":"2024-09-21T11:45:59.241583+0000","last_fullsized":"2024-09-21T11:45:59.241583+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:44.440374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.440374+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:17.887245+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241383+0000","last_change":"2024-09-21T11:45:42.816788+0000","last_active":"2024-09-21T11:45:59.241383+0000","last_peered":"2024-09-21T11:45:59.241383+0000","last_clean":"2024-09-21T11:45:59.241383+0000","last_became_active":"2024-09-21T11:42:37.171365+0000","last_became_peered":"2024-09-21T11:42:37.171365+0000","last_unstale":"2024-09-21T11:45:59.241383+0000","last_undegraded":"2024-09-21T11:45:59.241383+0000","last_fullsized":"2024-09-21T11:45:59.241383+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:42.816725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:42.816725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:43.651183+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":355,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238696+0000","last_change":"2024-09-21T11:42:26.008760+0000","last_active":"2024-09-21T11:45:59.238696+0000","last_peered":"2024-09-21T11:45:59.238696+0000","last_clean":"2024-09-21T11:45:59.238696+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:45:59.238696+0000","last_undegraded":"2024-09-21T11:45:59.238696+0000","last_fullsized":"2024-09-21T11:45:59.238696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:26.008646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:26.008646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":276,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247229+0000","last_change":"2024-09-21T11:42:37.157278+0000","last_active":"2024-09-21T11:45:59.247229+0000","last_peered":"2024-09-21T11:45:59.247229+0000","last_clean":"2024-09-21T11:45:59.247229+0000","last_became_active":"2024-09-21T11:42:37.156952+0000","last_became_peered":"2024-09-21T11:42:37.156952+0000","last_unstale":"2024-09-21T11:45:59.247229+0000","last_undegraded":"2024-09-21T11:45:59.247229+0000","last_fullsized":"2024-09-21T11:45:59.247229+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.066324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.066324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:32.220706+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"189'6163","reported_seq":6233,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.334207+0000","last_change":"2024-09-21T11:44:51.744583+0000","last_active":"2024-09-21T11:46:01.334207+0000","last_peered":"2024-09-21T11:46:01.334207+0000","last_clean":"2024-09-21T11:46:01.334207+0000","last_became_active":"2024-09-21T11:44:51.744224+0000","last_became_peered":"2024-09-21T11:44:51.744224+0000","last_unstale":"2024-09-21T11:46:01.334207+0000","last_undegraded":"2024-09-21T11:46:01.334207+0000","last_fullsized":"2024-09-21T11:46:01.334207+0000","mapping_epoch":160,"log_start":"169'1400","ondisk_log_start":"169'1400","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4763,"log_dups_size":0,"ondisk_log_size":4763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:05.760922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403898368,"num_objects":6163,"num_object_clones":0,"num_object_copies":18489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6163,"num_write_kb":394432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":318,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241140+0000","last_change":"2024-09-21T11:45:16.866668+0000","last_active":"2024-09-21T11:45:59.241140+0000","last_peered":"2024-09-21T11:45:59.241140+0000","last_clean":"2024-09-21T11:45:59.241140+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:45:59.241140+0000","last_undegraded":"2024-09-21T11:45:59.241140+0000","last_fullsized":"2024-09-21T11:45:59.241140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:45:16.866479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:16.866479+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:38.175618+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":290,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241762+0000","last_change":"2024-09-21T11:44:18.614485+0000","last_active":"2024-09-21T11:45:59.241762+0000","last_peered":"2024-09-21T11:45:59.241762+0000","last_clean":"2024-09-21T11:45:59.241762+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:45:59.241762+0000","last_undegraded":"2024-09-21T11:45:59.241762+0000","last_fullsized":"2024-09-21T11:45:59.241762+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:18.614293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:18.614293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:50.610585+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":282,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.237695+0000","last_change":"2024-09-21T11:44:13.946705+0000","last_active":"2024-09-21T11:45:59.237695+0000","last_peered":"2024-09-21T11:45:59.237695+0000","last_clean":"2024-09-21T11:45:59.237695+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:45:59.237695+0000","last_undegraded":"2024-09-21T11:45:59.237695+0000","last_fullsized":"2024-09-21T11:45:59.237695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:22.686173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238098+0000","last_change":"2024-09-21T11:44:11.930363+0000","last_active":"2024-09-21T11:45:59.238098+0000","last_peered":"2024-09-21T11:45:59.238098+0000","last_clean":"2024-09-21T11:45:59.238098+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:45:59.238098+0000","last_undegraded":"2024-09-21T11:45:59.238098+0000","last_fullsized":"2024-09-21T11:45:59.238098+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:11.930295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:11.930295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:20.670677+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"189'6109","reported_seq":6179,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:00.844697+0000","last_change":"2024-09-21T11:44:51.744967+0000","last_active":"2024-09-21T11:46:00.844697+0000","last_peered":"2024-09-21T11:46:00.844697+0000","last_clean":"2024-09-21T11:46:00.844697+0000","last_became_active":"2024-09-21T11:44:51.744633+0000","last_became_peered":"2024-09-21T11:44:51.744633+0000","last_unstale":"2024-09-21T11:46:00.844697+0000","last_undegraded":"2024-09-21T11:46:00.844697+0000","last_fullsized":"2024-09-21T11:46:00.844697+0000","mapping_epoch":160,"log_start":"168'1300","ondisk_log_start":"168'1300","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.094633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400359424,"num_objects":6109,"num_object_clones":0,"num_object_copies":18327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6109,"num_write_kb":390976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":315,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238026+0000","last_change":"2024-09-21T11:45:41.588559+0000","last_active":"2024-09-21T11:45:59.238026+0000","last_peered":"2024-09-21T11:45:59.238026+0000","last_clean":"2024-09-21T11:45:59.238026+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:45:59.238026+0000","last_undegraded":"2024-09-21T11:45:59.238026+0000","last_fullsized":"2024-09-21T11:45:59.238026+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:45:41.588470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:41.588470+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:47.875537+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":301,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.255187+0000","last_change":"2024-09-21T11:45:05.343516+0000","last_active":"2024-09-21T11:45:59.255187+0000","last_peered":"2024-09-21T11:45:59.255187+0000","last_clean":"2024-09-21T11:45:59.255187+0000","last_became_active":"2024-09-21T11:42:37.159390+0000","last_became_peered":"2024-09-21T11:42:37.159390+0000","last_unstale":"2024-09-21T11:45:59.255187+0000","last_undegraded":"2024-09-21T11:45:59.255187+0000","last_fullsized":"2024-09-21T11:45:59.255187+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:45:05.343450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:05.343450+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.902794+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"114'9","reported_seq":367,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241000+0000","last_change":"2024-09-21T11:43:29.417021+0000","last_active":"2024-09-21T11:45:59.241000+0000","last_peered":"2024-09-21T11:45:59.241000+0000","last_clean":"2024-09-21T11:45:59.241000+0000","last_became_active":"2024-09-21T11:42:37.169838+0000","last_became_peered":"2024-09-21T11:42:37.169838+0000","last_unstale":"2024-09-21T11:45:59.241000+0000","last_undegraded":"2024-09-21T11:45:59.241000+0000","last_fullsized":"2024-09-21T11:45:59.241000+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:29.416946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.416946+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:00.760179+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":272,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.248959+0000","last_change":"2024-09-21T11:41:39.527210+0000","last_active":"2024-09-21T11:45:59.248959+0000","last_peered":"2024-09-21T11:45:59.248959+0000","last_clean":"2024-09-21T11:45:59.248959+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:45:59.248959+0000","last_undegraded":"2024-09-21T11:45:59.248959+0000","last_fullsized":"2024-09-21T11:45:59.248959+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.527151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.527151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"189'6266","reported_seq":6336,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.348609+0000","last_change":"2024-09-21T11:44:51.412955+0000","last_active":"2024-09-21T11:46:01.348609+0000","last_peered":"2024-09-21T11:46:01.348609+0000","last_clean":"2024-09-21T11:46:01.348609+0000","last_became_active":"2024-09-21T11:44:51.412662+0000","last_became_peered":"2024-09-21T11:44:51.412662+0000","last_unstale":"2024-09-21T11:46:01.348609+0000","last_undegraded":"2024-09-21T11:46:01.348609+0000","last_fullsized":"2024-09-21T11:46:01.348609+0000","mapping_epoch":160,"log_start":"169'1500","ondisk_log_start":"169'1500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4766,"log_dups_size":0,"ondisk_log_size":4766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:06.806791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410648576,"num_objects":6266,"num_object_clones":0,"num_object_copies":18798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6266,"num_write_kb":401024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":221,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247176+0000","last_change":"2024-09-21T11:44:43.309806+0000","last_active":"2024-09-21T11:45:59.247176+0000","last_peered":"2024-09-21T11:45:59.247176+0000","last_clean":"2024-09-21T11:45:59.247176+0000","last_became_active":"2024-09-21T11:42:37.165120+0000","last_became_peered":"2024-09-21T11:42:37.165120+0000","last_unstale":"2024-09-21T11:45:59.247176+0000","last_undegraded":"2024-09-21T11:45:59.247176+0000","last_fullsized":"2024-09-21T11:45:59.247176+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:44:43.309439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:43.309439+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:06.560680+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247926+0000","last_change":"2024-09-21T11:45:13.309329+0000","last_active":"2024-09-21T11:45:59.247926+0000","last_peered":"2024-09-21T11:45:59.247926+0000","last_clean":"2024-09-21T11:45:59.247926+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:45:59.247926+0000","last_undegraded":"2024-09-21T11:45:59.247926+0000","last_fullsized":"2024-09-21T11:45:59.247926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:13.309257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:13.309257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:16.178713+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":357,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249736+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:45:59.249736+0000","last_peered":"2024-09-21T11:45:59.249736+0000","last_clean":"2024-09-21T11:45:59.249736+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:45:59.249736+0000","last_undegraded":"2024-09-21T11:45:59.249736+0000","last_fullsized":"2024-09-21T11:45:59.249736+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:35.166408+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":269,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249008+0000","last_change":"2024-09-21T11:41:40.278404+0000","last_active":"2024-09-21T11:45:59.249008+0000","last_peered":"2024-09-21T11:45:59.249008+0000","last_clean":"2024-09-21T11:45:59.249008+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:45:59.249008+0000","last_undegraded":"2024-09-21T11:45:59.249008+0000","last_fullsized":"2024-09-21T11:45:59.249008+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.278315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.278315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:25.297484+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"189'6110","reported_seq":6180,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.273265+0000","last_change":"2024-09-21T11:44:51.410839+0000","last_active":"2024-09-21T11:46:01.273265+0000","last_peered":"2024-09-21T11:46:01.273265+0000","last_clean":"2024-09-21T11:46:01.273265+0000","last_became_active":"2024-09-21T11:44:51.410302+0000","last_became_peered":"2024-09-21T11:44:51.410302+0000","last_unstale":"2024-09-21T11:46:01.273265+0000","last_undegraded":"2024-09-21T11:46:01.273265+0000","last_fullsized":"2024-09-21T11:46:01.273265+0000","mapping_epoch":160,"log_start":"169'1600","ondisk_log_start":"169'1600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4510,"log_dups_size":0,"ondisk_log_size":4510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400424960,"num_objects":6110,"num_object_clones":0,"num_object_copies":18330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6110,"num_write_kb":391040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":313,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.251366+0000","last_change":"2024-09-21T11:41:07.065583+0000","last_active":"2024-09-21T11:45:59.251366+0000","last_peered":"2024-09-21T11:45:59.251366+0000","last_clean":"2024-09-21T11:45:59.251366+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:45:59.251366+0000","last_undegraded":"2024-09-21T11:45:59.251366+0000","last_fullsized":"2024-09-21T11:45:59.251366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:41:07.065507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.065507+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":182,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246233+0000","last_change":"2024-09-21T11:44:44.619532+0000","last_active":"2024-09-21T11:45:59.246233+0000","last_peered":"2024-09-21T11:45:59.246233+0000","last_clean":"2024-09-21T11:45:59.246233+0000","last_became_active":"2024-09-21T11:42:37.157562+0000","last_became_peered":"2024-09-21T11:42:37.157562+0000","last_unstale":"2024-09-21T11:45:59.246233+0000","last_undegraded":"2024-09-21T11:45:59.246233+0000","last_fullsized":"2024-09-21T11:45:59.246233+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:44.619493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.619493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:03.953460+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":273,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247242+0000","last_change":"2024-09-21T11:43:26.704117+0000","last_active":"2024-09-21T11:45:59.247242+0000","last_peered":"2024-09-21T11:45:59.247242+0000","last_clean":"2024-09-21T11:45:59.247242+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:45:59.247242+0000","last_undegraded":"2024-09-21T11:45:59.247242+0000","last_fullsized":"2024-09-21T11:45:59.247242+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.704073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.704073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:01.391667+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":174,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.245718+0000","last_change":"2024-09-21T11:43:37.819912+0000","last_active":"2024-09-21T11:45:59.245718+0000","last_peered":"2024-09-21T11:45:59.245718+0000","last_clean":"2024-09-21T11:45:59.245718+0000","last_became_active":"2024-09-21T11:42:37.155437+0000","last_became_peered":"2024-09-21T11:42:37.155437+0000","last_unstale":"2024-09-21T11:45:59.245718+0000","last_undegraded":"2024-09-21T11:45:59.245718+0000","last_fullsized":"2024-09-21T11:45:59.245718+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"189'6179","reported_seq":6249,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:00.831546+0000","last_change":"2024-09-21T11:44:51.745506+0000","last_active":"2024-09-21T11:46:00.831546+0000","last_peered":"2024-09-21T11:46:00.831546+0000","last_clean":"2024-09-21T11:46:00.831546+0000","last_became_active":"2024-09-21T11:44:51.745200+0000","last_became_peered":"2024-09-21T11:44:51.745200+0000","last_unstale":"2024-09-21T11:46:00.831546+0000","last_undegraded":"2024-09-21T11:46:00.831546+0000","last_fullsized":"2024-09-21T11:46:00.831546+0000","mapping_epoch":160,"log_start":"168'1400","ondisk_log_start":"168'1400","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4779,"log_dups_size":0,"ondisk_log_size":4779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:00.084252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":404946944,"num_objects":6179,"num_object_clones":0,"num_object_copies":18537,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6179,"num_write_kb":395456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"140'30","reported_seq":337,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246103+0000","last_change":"2024-09-21T11:43:37.416987+0000","last_active":"2024-09-21T11:45:59.246103+0000","last_peered":"2024-09-21T11:45:59.246103+0000","last_clean":"2024-09-21T11:45:59.246103+0000","last_became_active":"2024-09-21T11:43:32.003036+0000","last_became_peered":"2024-09-21T11:43:32.003036+0000","last_unstale":"2024-09-21T11:45:59.246103+0000","last_undegraded":"2024-09-21T11:45:59.246103+0000","last_fullsized":"2024-09-21T11:45:59.246103+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:58.675676+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":288,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238618+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:45:59.238618+0000","last_peered":"2024-09-21T11:45:59.238618+0000","last_clean":"2024-09-21T11:45:59.238618+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:45:59.238618+0000","last_undegraded":"2024-09-21T11:45:59.238618+0000","last_fullsized":"2024-09-21T11:45:59.238618+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:33.611132+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":302,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247366+0000","last_change":"2024-09-21T11:42:37.193243+0000","last_active":"2024-09-21T11:45:59.247366+0000","last_peered":"2024-09-21T11:45:59.247366+0000","last_clean":"2024-09-21T11:45:59.247366+0000","last_became_active":"2024-09-21T11:42:37.164723+0000","last_became_peered":"2024-09-21T11:42:37.164723+0000","last_unstale":"2024-09-21T11:45:59.247366+0000","last_undegraded":"2024-09-21T11:45:59.247366+0000","last_fullsized":"2024-09-21T11:45:59.247366+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:42:17.142454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.142454+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:34.146418+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":288,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.251304+0000","last_change":"2024-09-21T11:44:41.300665+0000","last_active":"2024-09-21T11:45:59.251304+0000","last_peered":"2024-09-21T11:45:59.251304+0000","last_clean":"2024-09-21T11:45:59.251304+0000","last_became_active":"2024-09-21T11:42:37.163682+0000","last_became_peered":"2024-09-21T11:42:37.163682+0000","last_unstale":"2024-09-21T11:45:59.251304+0000","last_undegraded":"2024-09-21T11:45:59.251304+0000","last_fullsized":"2024-09-21T11:45:59.251304+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241098+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:45:59.241098+0000","last_peered":"2024-09-21T11:45:59.241098+0000","last_clean":"2024-09-21T11:45:59.241098+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:45:59.241098+0000","last_undegraded":"2024-09-21T11:45:59.241098+0000","last_fullsized":"2024-09-21T11:45:59.241098+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"189'6044","reported_seq":6114,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.244571+0000","last_change":"2024-09-21T11:44:51.410845+0000","last_active":"2024-09-21T11:46:01.244571+0000","last_peered":"2024-09-21T11:46:01.244571+0000","last_clean":"2024-09-21T11:46:01.244571+0000","last_became_active":"2024-09-21T11:44:51.410553+0000","last_became_peered":"2024-09-21T11:44:51.410553+0000","last_unstale":"2024-09-21T11:46:01.244571+0000","last_undegraded":"2024-09-21T11:46:01.244571+0000","last_fullsized":"2024-09-21T11:46:01.244571+0000","mapping_epoch":160,"log_start":"169'1600","ondisk_log_start":"169'1600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4444,"log_dups_size":0,"ondisk_log_size":4444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:09.920894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396099584,"num_objects":6044,"num_object_clones":0,"num_object_copies":18132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6044,"num_write_kb":386816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":225,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247221+0000","last_change":"2024-09-21T11:42:37.188687+0000","last_active":"2024-09-21T11:45:59.247221+0000","last_peered":"2024-09-21T11:45:59.247221+0000","last_clean":"2024-09-21T11:45:59.247221+0000","last_became_active":"2024-09-21T11:42:37.161797+0000","last_became_peered":"2024-09-21T11:42:37.161797+0000","last_unstale":"2024-09-21T11:45:59.247221+0000","last_undegraded":"2024-09-21T11:45:59.247221+0000","last_fullsized":"2024-09-21T11:45:59.247221+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:00.246582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":179,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.245501+0000","last_change":"2024-09-21T11:42:37.182044+0000","last_active":"2024-09-21T11:45:59.245501+0000","last_peered":"2024-09-21T11:45:59.245501+0000","last_clean":"2024-09-21T11:45:59.245501+0000","last_became_active":"2024-09-21T11:42:37.156191+0000","last_became_peered":"2024-09-21T11:42:37.156191+0000","last_unstale":"2024-09-21T11:45:59.245501+0000","last_undegraded":"2024-09-21T11:45:59.245501+0000","last_fullsized":"2024-09-21T11:45:59.245501+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.704463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":299,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.248228+0000","last_change":"2024-09-21T11:43:36.446391+0000","last_active":"2024-09-21T11:45:59.248228+0000","last_peered":"2024-09-21T11:45:59.248228+0000","last_clean":"2024-09-21T11:45:59.248228+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:45:59.248228+0000","last_undegraded":"2024-09-21T11:45:59.248228+0000","last_fullsized":"2024-09-21T11:45:59.248228+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:43:36.446341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.446341+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:53.554424+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":282,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246679+0000","last_change":"2024-09-21T11:43:28.505134+0000","last_active":"2024-09-21T11:45:59.246679+0000","last_peered":"2024-09-21T11:45:59.246679+0000","last_clean":"2024-09-21T11:45:59.246679+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:45:59.246679+0000","last_undegraded":"2024-09-21T11:45:59.246679+0000","last_fullsized":"2024-09-21T11:45:59.246679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.504894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.504894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:30.585866+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":269,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249551+0000","last_change":"2024-09-21T11:42:25.245624+0000","last_active":"2024-09-21T11:45:59.249551+0000","last_peered":"2024-09-21T11:45:59.249551+0000","last_clean":"2024-09-21T11:45:59.249551+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:45:59.249551+0000","last_undegraded":"2024-09-21T11:45:59.249551+0000","last_fullsized":"2024-09-21T11:45:59.249551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.245565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.245565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.670245+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"189'6045","reported_seq":6115,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:00.982844+0000","last_change":"2024-09-21T11:44:51.410741+0000","last_active":"2024-09-21T11:46:00.982844+0000","last_peered":"2024-09-21T11:46:00.982844+0000","last_clean":"2024-09-21T11:46:00.982844+0000","last_became_active":"2024-09-21T11:44:51.410454+0000","last_became_peered":"2024-09-21T11:44:51.410454+0000","last_unstale":"2024-09-21T11:46:00.982844+0000","last_undegraded":"2024-09-21T11:46:00.982844+0000","last_fullsized":"2024-09-21T11:46:00.982844+0000","mapping_epoch":160,"log_start":"168'600","ondisk_log_start":"168'600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5445,"log_dups_size":0,"ondisk_log_size":5445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.918401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396165120,"num_objects":6045,"num_object_clones":0,"num_object_copies":18135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6045,"num_write_kb":386880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":324,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.255007+0000","last_change":"2024-09-21T11:45:09.370313+0000","last_active":"2024-09-21T11:45:59.255007+0000","last_peered":"2024-09-21T11:45:59.255007+0000","last_clean":"2024-09-21T11:45:59.255007+0000","last_became_active":"2024-09-21T11:42:37.164059+0000","last_became_peered":"2024-09-21T11:42:37.164059+0000","last_unstale":"2024-09-21T11:45:59.255007+0000","last_undegraded":"2024-09-21T11:45:59.255007+0000","last_fullsized":"2024-09-21T11:45:59.255007+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:27.315480+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.248956+0000","last_change":"2024-09-21T11:44:45.187728+0000","last_active":"2024-09-21T11:45:59.248956+0000","last_peered":"2024-09-21T11:45:59.248956+0000","last_clean":"2024-09-21T11:45:59.248956+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:45:59.248956+0000","last_undegraded":"2024-09-21T11:45:59.248956+0000","last_fullsized":"2024-09-21T11:45:59.248956+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.187650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.187650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:12.532021+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":667,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:00.556432+0000","last_change":"2024-09-21T11:43:47.494999+0000","last_active":"2024-09-21T11:46:00.556432+0000","last_peered":"2024-09-21T11:46:00.556432+0000","last_clean":"2024-09-21T11:46:00.556432+0000","last_became_active":"2024-09-21T11:42:37.166141+0000","last_became_peered":"2024-09-21T11:42:37.166141+0000","last_unstale":"2024-09-21T11:46:00.556432+0000","last_undegraded":"2024-09-21T11:46:00.556432+0000","last_fullsized":"2024-09-21T11:46:00.556432+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:47.494865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:47.494865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:23.715570+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":354,"num_read_kb":349,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":280,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247717+0000","last_change":"2024-09-21T11:43:45.046195+0000","last_active":"2024-09-21T11:45:59.247717+0000","last_peered":"2024-09-21T11:45:59.247717+0000","last_clean":"2024-09-21T11:45:59.247717+0000","last_became_active":"2024-09-21T11:42:37.172270+0000","last_became_peered":"2024-09-21T11:42:37.172270+0000","last_unstale":"2024-09-21T11:45:59.247717+0000","last_undegraded":"2024-09-21T11:45:59.247717+0000","last_fullsized":"2024-09-21T11:45:59.247717+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.046128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.046128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:01.258149+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":174,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246902+0000","last_change":"2024-09-21T11:44:10.289926+0000","last_active":"2024-09-21T11:45:59.246902+0000","last_peered":"2024-09-21T11:45:59.246902+0000","last_clean":"2024-09-21T11:45:59.246902+0000","last_became_active":"2024-09-21T11:42:37.172645+0000","last_became_peered":"2024-09-21T11:42:37.172645+0000","last_unstale":"2024-09-21T11:45:59.246902+0000","last_undegraded":"2024-09-21T11:45:59.246902+0000","last_fullsized":"2024-09-21T11:45:59.246902+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:23.376344+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"189'6126","reported_seq":6196,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:00.842961+0000","last_change":"2024-09-21T11:44:51.410998+0000","last_active":"2024-09-21T11:46:00.842961+0000","last_peered":"2024-09-21T11:46:00.842961+0000","last_clean":"2024-09-21T11:46:00.842961+0000","last_became_active":"2024-09-21T11:44:51.410701+0000","last_became_peered":"2024-09-21T11:44:51.410701+0000","last_unstale":"2024-09-21T11:46:00.842961+0000","last_undegraded":"2024-09-21T11:46:00.842961+0000","last_fullsized":"2024-09-21T11:46:00.842961+0000","mapping_epoch":160,"log_start":"168'1100","ondisk_log_start":"168'1100","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5026,"log_dups_size":0,"ondisk_log_size":5026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401473536,"num_objects":6126,"num_object_clones":0,"num_object_copies":18378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6126,"num_write_kb":392064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":372,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.245604+0000","last_change":"2024-09-21T11:42:19.703071+0000","last_active":"2024-09-21T11:45:59.245604+0000","last_peered":"2024-09-21T11:45:59.245604+0000","last_clean":"2024-09-21T11:45:59.245604+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:45:59.245604+0000","last_undegraded":"2024-09-21T11:45:59.245604+0000","last_fullsized":"2024-09-21T11:45:59.245604+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:42:19.703007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.703007+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":290,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.242424+0000","last_change":"2024-09-21T11:45:15.850112+0000","last_active":"2024-09-21T11:45:59.242424+0000","last_peered":"2024-09-21T11:45:59.242424+0000","last_clean":"2024-09-21T11:45:59.242424+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:45:59.242424+0000","last_undegraded":"2024-09-21T11:45:59.242424+0000","last_fullsized":"2024-09-21T11:45:59.242424+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:15.850021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:15.850021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:34.453388+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":344,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249044+0000","last_change":"2024-09-21T11:45:38.787162+0000","last_active":"2024-09-21T11:45:59.249044+0000","last_peered":"2024-09-21T11:45:59.249044+0000","last_clean":"2024-09-21T11:45:59.249044+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:45:59.249044+0000","last_undegraded":"2024-09-21T11:45:59.249044+0000","last_fullsized":"2024-09-21T11:45:59.249044+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:45:38.786968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:45:38.786968+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:53.105012+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":273,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249666+0000","last_change":"2024-09-21T11:43:31.630737+0000","last_active":"2024-09-21T11:45:59.249666+0000","last_peered":"2024-09-21T11:45:59.249666+0000","last_clean":"2024-09-21T11:45:59.249666+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:45:59.249666+0000","last_undegraded":"2024-09-21T11:45:59.249666+0000","last_fullsized":"2024-09-21T11:45:59.249666+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:31.630670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.630670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:17.693993+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247309+0000","last_change":"2024-09-21T11:45:09.222611+0000","last_active":"2024-09-21T11:45:59.247309+0000","last_peered":"2024-09-21T11:45:59.247309+0000","last_clean":"2024-09-21T11:45:59.247309+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:45:59.247309+0000","last_undegraded":"2024-09-21T11:45:59.247309+0000","last_fullsized":"2024-09-21T11:45:59.247309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:09.222544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.222544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:12.846344+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"189'6082","reported_seq":6152,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:00.768667+0000","last_change":"2024-09-21T11:44:51.410732+0000","last_active":"2024-09-21T11:46:00.768667+0000","last_peered":"2024-09-21T11:46:00.768667+0000","last_clean":"2024-09-21T11:46:00.768667+0000","last_became_active":"2024-09-21T11:44:51.410197+0000","last_became_peered":"2024-09-21T11:44:51.410197+0000","last_unstale":"2024-09-21T11:46:00.768667+0000","last_undegraded":"2024-09-21T11:46:00.768667+0000","last_fullsized":"2024-09-21T11:46:00.768667+0000","mapping_epoch":160,"log_start":"169'1600","ondisk_log_start":"169'1600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398589952,"num_objects":6082,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6082,"num_write_kb":389248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":316,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238825+0000","last_change":"2024-09-21T11:42:37.179956+0000","last_active":"2024-09-21T11:45:59.238825+0000","last_peered":"2024-09-21T11:45:59.238825+0000","last_clean":"2024-09-21T11:45:59.238825+0000","last_became_active":"2024-09-21T11:42:37.158158+0000","last_became_peered":"2024-09-21T11:42:37.158158+0000","last_unstale":"2024-09-21T11:45:59.238825+0000","last_undegraded":"2024-09-21T11:45:59.238825+0000","last_fullsized":"2024-09-21T11:45:59.238825+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:22.041174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.041174+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":298,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238159+0000","last_change":"2024-09-21T11:45:11.834001+0000","last_active":"2024-09-21T11:45:59.238159+0000","last_peered":"2024-09-21T11:45:59.238159+0000","last_clean":"2024-09-21T11:45:59.238159+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:45:59.238159+0000","last_undegraded":"2024-09-21T11:45:59.238159+0000","last_fullsized":"2024-09-21T11:45:59.238159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:11.833924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:11.833924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:13.592689+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":281,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247141+0000","last_change":"2024-09-21T11:45:11.312119+0000","last_active":"2024-09-21T11:45:59.247141+0000","last_peered":"2024-09-21T11:45:59.247141+0000","last_clean":"2024-09-21T11:45:59.247141+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:45:59.247141+0000","last_undegraded":"2024-09-21T11:45:59.247141+0000","last_fullsized":"2024-09-21T11:45:59.247141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:11.312058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:11.312058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:32.826661+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.240318+0000","last_change":"2024-09-21T11:41:38.453843+0000","last_active":"2024-09-21T11:45:59.240318+0000","last_peered":"2024-09-21T11:45:59.240318+0000","last_clean":"2024-09-21T11:45:59.240318+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:45:59.240318+0000","last_undegraded":"2024-09-21T11:45:59.240318+0000","last_fullsized":"2024-09-21T11:45:59.240318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:38.453579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:38.453579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:33.866050+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"189'6289","reported_seq":6359,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.258193+0000","last_change":"2024-09-21T11:44:51.412888+0000","last_active":"2024-09-21T11:46:01.258193+0000","last_peered":"2024-09-21T11:46:01.258193+0000","last_clean":"2024-09-21T11:46:01.258193+0000","last_became_active":"2024-09-21T11:44:51.412596+0000","last_became_peered":"2024-09-21T11:44:51.412596+0000","last_unstale":"2024-09-21T11:46:01.258193+0000","last_undegraded":"2024-09-21T11:46:01.258193+0000","last_fullsized":"2024-09-21T11:46:01.258193+0000","mapping_epoch":160,"log_start":"169'1600","ondisk_log_start":"169'1600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4689,"log_dups_size":0,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:13.580236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":412155904,"num_objects":6289,"num_object_clones":0,"num_object_copies":18867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6289,"num_write_kb":402496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":339,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238759+0000","last_change":"2024-09-21T11:45:09.804803+0000","last_active":"2024-09-21T11:45:59.238759+0000","last_peered":"2024-09-21T11:45:59.238759+0000","last_clean":"2024-09-21T11:45:59.238759+0000","last_became_active":"2024-09-21T11:42:37.160867+0000","last_became_peered":"2024-09-21T11:42:37.160867+0000","last_unstale":"2024-09-21T11:45:59.238759+0000","last_undegraded":"2024-09-21T11:45:59.238759+0000","last_fullsized":"2024-09-21T11:45:59.238759+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:45:09.804703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.804703+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:17.840842+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":304,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247634+0000","last_change":"2024-09-21T11:44:40.478955+0000","last_active":"2024-09-21T11:45:59.247634+0000","last_peered":"2024-09-21T11:45:59.247634+0000","last_clean":"2024-09-21T11:45:59.247634+0000","last_became_active":"2024-09-21T11:42:37.167043+0000","last_became_peered":"2024-09-21T11:42:37.167043+0000","last_unstale":"2024-09-21T11:45:59.247634+0000","last_undegraded":"2024-09-21T11:45:59.247634+0000","last_fullsized":"2024-09-21T11:45:59.247634+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:40.478885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:40.478885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":183,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238832+0000","last_change":"2024-09-21T11:44:45.815953+0000","last_active":"2024-09-21T11:45:59.238832+0000","last_peered":"2024-09-21T11:45:59.238832+0000","last_clean":"2024-09-21T11:45:59.238832+0000","last_became_active":"2024-09-21T11:42:37.160659+0000","last_became_peered":"2024-09-21T11:42:37.160659+0000","last_unstale":"2024-09-21T11:45:59.238832+0000","last_undegraded":"2024-09-21T11:45:59.238832+0000","last_fullsized":"2024-09-21T11:45:59.238832+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.815895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.815895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:49.516167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246975+0000","last_change":"2024-09-21T11:45:03.661699+0000","last_active":"2024-09-21T11:45:59.246975+0000","last_peered":"2024-09-21T11:45:59.246975+0000","last_clean":"2024-09-21T11:45:59.246975+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:45:59.246975+0000","last_undegraded":"2024-09-21T11:45:59.246975+0000","last_fullsized":"2024-09-21T11:45:59.246975+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:03.661625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:03.661625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:15.894877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"189'6159","reported_seq":6229,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.273696+0000","last_change":"2024-09-21T11:44:51.409705+0000","last_active":"2024-09-21T11:46:01.273696+0000","last_peered":"2024-09-21T11:46:01.273696+0000","last_clean":"2024-09-21T11:46:01.273696+0000","last_became_active":"2024-09-21T11:44:51.409405+0000","last_became_peered":"2024-09-21T11:44:51.409405+0000","last_unstale":"2024-09-21T11:46:01.273696+0000","last_undegraded":"2024-09-21T11:46:01.273696+0000","last_fullsized":"2024-09-21T11:46:01.273696+0000","mapping_epoch":160,"log_start":"169'1700","ondisk_log_start":"169'1700","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4459,"log_dups_size":0,"ondisk_log_size":4459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:17.726404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403636224,"num_objects":6159,"num_object_clones":0,"num_object_copies":18477,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6159,"num_write_kb":394176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":309,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.239250+0000","last_change":"2024-09-21T11:44:41.809293+0000","last_active":"2024-09-21T11:45:59.239250+0000","last_peered":"2024-09-21T11:45:59.239250+0000","last_clean":"2024-09-21T11:45:59.239250+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:45:59.239250+0000","last_undegraded":"2024-09-21T11:45:59.239250+0000","last_fullsized":"2024-09-21T11:45:59.239250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:44:41.809215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.809215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":282,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246295+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:45:59.246295+0000","last_peered":"2024-09-21T11:45:59.246295+0000","last_clean":"2024-09-21T11:45:59.246295+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:45:59.246295+0000","last_undegraded":"2024-09-21T11:45:59.246295+0000","last_fullsized":"2024-09-21T11:45:59.246295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":363,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246699+0000","last_change":"2024-09-21T11:43:38.421538+0000","last_active":"2024-09-21T11:45:59.246699+0000","last_peered":"2024-09-21T11:45:59.246699+0000","last_clean":"2024-09-21T11:45:59.246699+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:45:59.246699+0000","last_undegraded":"2024-09-21T11:45:59.246699+0000","last_fullsized":"2024-09-21T11:45:59.246699+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:43:38.421251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.421251+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:52.556125+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238518+0000","last_change":"2024-09-21T11:42:50.131895+0000","last_active":"2024-09-21T11:45:59.238518+0000","last_peered":"2024-09-21T11:45:59.238518+0000","last_clean":"2024-09-21T11:45:59.238518+0000","last_became_active":"2024-09-21T11:42:37.157732+0000","last_became_peered":"2024-09-21T11:42:37.157732+0000","last_unstale":"2024-09-21T11:45:59.238518+0000","last_undegraded":"2024-09-21T11:45:59.238518+0000","last_fullsized":"2024-09-21T11:45:59.238518+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:50.131830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:50.131830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"189'6146","reported_seq":6216,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.212896+0000","last_change":"2024-09-21T11:44:51.410552+0000","last_active":"2024-09-21T11:46:01.212896+0000","last_peered":"2024-09-21T11:46:01.212896+0000","last_clean":"2024-09-21T11:46:01.212896+0000","last_became_active":"2024-09-21T11:44:51.410164+0000","last_became_peered":"2024-09-21T11:44:51.410164+0000","last_unstale":"2024-09-21T11:46:01.212896+0000","last_undegraded":"2024-09-21T11:46:01.212896+0000","last_fullsized":"2024-09-21T11:46:01.212896+0000","mapping_epoch":160,"log_start":"169'1700","ondisk_log_start":"169'1700","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4446,"log_dups_size":0,"ondisk_log_size":4446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":402784256,"num_objects":6146,"num_object_clones":0,"num_object_copies":18438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6146,"num_write_kb":393344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":310,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.251215+0000","last_change":"2024-09-21T11:42:37.192718+0000","last_active":"2024-09-21T11:45:59.251215+0000","last_peered":"2024-09-21T11:45:59.251215+0000","last_clean":"2024-09-21T11:45:59.251215+0000","last_became_active":"2024-09-21T11:42:37.168751+0000","last_became_peered":"2024-09-21T11:42:37.168751+0000","last_unstale":"2024-09-21T11:45:59.251215+0000","last_undegraded":"2024-09-21T11:45:59.251215+0000","last_fullsized":"2024-09-21T11:45:59.251215+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:15.417817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.417817+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:31.193831+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246993+0000","last_change":"2024-09-21T11:45:07.249883+0000","last_active":"2024-09-21T11:45:59.246993+0000","last_peered":"2024-09-21T11:45:59.246993+0000","last_clean":"2024-09-21T11:45:59.246993+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:45:59.246993+0000","last_undegraded":"2024-09-21T11:45:59.246993+0000","last_fullsized":"2024-09-21T11:45:59.246993+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:07.249826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.249826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:26.836645+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":274,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247598+0000","last_change":"2024-09-21T11:42:18.265457+0000","last_active":"2024-09-21T11:45:59.247598+0000","last_peered":"2024-09-21T11:45:59.247598+0000","last_clean":"2024-09-21T11:45:59.247598+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:45:59.247598+0000","last_undegraded":"2024-09-21T11:45:59.247598+0000","last_fullsized":"2024-09-21T11:45:59.247598+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.265392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.265392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:55.578456+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":261,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.248874+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:45:59.248874+0000","last_peered":"2024-09-21T11:45:59.248874+0000","last_clean":"2024-09-21T11:45:59.248874+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:45:59.248874+0000","last_undegraded":"2024-09-21T11:45:59.248874+0000","last_fullsized":"2024-09-21T11:45:59.248874+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:53.248979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"189'6218","reported_seq":6288,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.327183+0000","last_change":"2024-09-21T11:44:51.744832+0000","last_active":"2024-09-21T11:46:01.327183+0000","last_peered":"2024-09-21T11:46:01.327183+0000","last_clean":"2024-09-21T11:46:01.327183+0000","last_became_active":"2024-09-21T11:44:51.744226+0000","last_became_peered":"2024-09-21T11:44:51.744226+0000","last_unstale":"2024-09-21T11:46:01.327183+0000","last_undegraded":"2024-09-21T11:46:01.327183+0000","last_fullsized":"2024-09-21T11:46:01.327183+0000","mapping_epoch":160,"log_start":"169'1500","ondisk_log_start":"169'1500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4718,"log_dups_size":0,"ondisk_log_size":4718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:04.388235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":407502848,"num_objects":6218,"num_object_clones":0,"num_object_copies":18654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6218,"num_write_kb":397952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":343,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247393+0000","last_change":"2024-09-21T11:44:38.463592+0000","last_active":"2024-09-21T11:45:59.247393+0000","last_peered":"2024-09-21T11:45:59.247393+0000","last_clean":"2024-09-21T11:45:59.247393+0000","last_became_active":"2024-09-21T11:42:37.166373+0000","last_became_peered":"2024-09-21T11:42:37.166373+0000","last_unstale":"2024-09-21T11:45:59.247393+0000","last_undegraded":"2024-09-21T11:45:59.247393+0000","last_fullsized":"2024-09-21T11:45:59.247393+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:44:38.463535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:38.463535+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:07.934116+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":281,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249960+0000","last_change":"2024-09-21T11:41:46.471940+0000","last_active":"2024-09-21T11:45:59.249960+0000","last_peered":"2024-09-21T11:45:59.249960+0000","last_clean":"2024-09-21T11:45:59.249960+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:45:59.249960+0000","last_undegraded":"2024-09-21T11:45:59.249960+0000","last_fullsized":"2024-09-21T11:45:59.249960+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:46.471872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:46.471872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":290,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.245891+0000","last_change":"2024-09-21T11:44:38.629525+0000","last_active":"2024-09-21T11:45:59.245891+0000","last_peered":"2024-09-21T11:45:59.245891+0000","last_clean":"2024-09-21T11:45:59.245891+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:45:59.245891+0000","last_undegraded":"2024-09-21T11:45:59.245891+0000","last_fullsized":"2024-09-21T11:45:59.245891+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:38.629466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:38.629466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:04.842945+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249015+0000","last_change":"2024-09-21T11:45:36.779034+0000","last_active":"2024-09-21T11:45:59.249015+0000","last_peered":"2024-09-21T11:45:59.249015+0000","last_clean":"2024-09-21T11:45:59.249015+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:45:59.249015+0000","last_undegraded":"2024-09-21T11:45:59.249015+0000","last_fullsized":"2024-09-21T11:45:59.249015+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.778909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.778909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:57.501118+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"189'6142","reported_seq":6212,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:00.832182+0000","last_change":"2024-09-21T11:44:51.746053+0000","last_active":"2024-09-21T11:46:00.832182+0000","last_peered":"2024-09-21T11:46:00.832182+0000","last_clean":"2024-09-21T11:46:00.832182+0000","last_became_active":"2024-09-21T11:44:51.745609+0000","last_became_peered":"2024-09-21T11:44:51.745609+0000","last_unstale":"2024-09-21T11:46:00.832182+0000","last_undegraded":"2024-09-21T11:46:00.832182+0000","last_fullsized":"2024-09-21T11:46:00.832182+0000","mapping_epoch":160,"log_start":"168'1100","ondisk_log_start":"168'1100","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5042,"log_dups_size":0,"ondisk_log_size":5042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:15.803134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":402522112,"num_objects":6142,"num_object_clones":0,"num_object_copies":18426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6142,"num_write_kb":393088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":336,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.237866+0000","last_change":"2024-09-21T11:44:43.806283+0000","last_active":"2024-09-21T11:45:59.237866+0000","last_peered":"2024-09-21T11:45:59.237866+0000","last_clean":"2024-09-21T11:45:59.237866+0000","last_became_active":"2024-09-21T11:42:37.165364+0000","last_became_peered":"2024-09-21T11:42:37.165364+0000","last_unstale":"2024-09-21T11:45:59.237866+0000","last_undegraded":"2024-09-21T11:45:59.237866+0000","last_fullsized":"2024-09-21T11:45:59.237866+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:44:43.806186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:43.806186+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:11.084871+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":304,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.250644+0000","last_change":"2024-09-21T11:44:47.741161+0000","last_active":"2024-09-21T11:45:59.250644+0000","last_peered":"2024-09-21T11:45:59.250644+0000","last_clean":"2024-09-21T11:45:59.250644+0000","last_became_active":"2024-09-21T11:42:37.165470+0000","last_became_peered":"2024-09-21T11:42:37.165470+0000","last_unstale":"2024-09-21T11:45:59.250644+0000","last_undegraded":"2024-09-21T11:45:59.250644+0000","last_fullsized":"2024-09-21T11:45:59.250644+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:47.741095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:47.741095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246323+0000","last_change":"2024-09-21T11:45:05.622676+0000","last_active":"2024-09-21T11:45:59.246323+0000","last_peered":"2024-09-21T11:45:59.246323+0000","last_clean":"2024-09-21T11:45:59.246323+0000","last_became_active":"2024-09-21T11:42:37.155675+0000","last_became_peered":"2024-09-21T11:42:37.155675+0000","last_unstale":"2024-09-21T11:45:59.246323+0000","last_undegraded":"2024-09-21T11:45:59.246323+0000","last_fullsized":"2024-09-21T11:45:59.246323+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:05.622608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:05.622608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.685022+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.244658+0000","last_change":"2024-09-21T11:42:37.170697+0000","last_active":"2024-09-21T11:45:59.244658+0000","last_peered":"2024-09-21T11:45:59.244658+0000","last_clean":"2024-09-21T11:45:59.244658+0000","last_became_active":"2024-09-21T11:42:37.168686+0000","last_became_peered":"2024-09-21T11:42:37.168686+0000","last_unstale":"2024-09-21T11:45:59.244658+0000","last_undegraded":"2024-09-21T11:45:59.244658+0000","last_fullsized":"2024-09-21T11:45:59.244658+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:27.327920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:27.327920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:27.806052+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"189'6130","reported_seq":6200,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:46:00.830941+0000","last_change":"2024-09-21T11:44:51.410102+0000","last_active":"2024-09-21T11:46:00.830941+0000","last_peered":"2024-09-21T11:46:00.830941+0000","last_clean":"2024-09-21T11:46:00.830941+0000","last_became_active":"2024-09-21T11:44:51.409710+0000","last_became_peered":"2024-09-21T11:44:51.409710+0000","last_unstale":"2024-09-21T11:46:00.830941+0000","last_undegraded":"2024-09-21T11:46:00.830941+0000","last_fullsized":"2024-09-21T11:46:00.830941+0000","mapping_epoch":160,"log_start":"168'1300","ondisk_log_start":"168'1300","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4830,"log_dups_size":0,"ondisk_log_size":4830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.594152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401735680,"num_objects":6130,"num_object_clones":0,"num_object_copies":18390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6130,"num_write_kb":392320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":294,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246569+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:45:59.246569+0000","last_peered":"2024-09-21T11:45:59.246569+0000","last_clean":"2024-09-21T11:45:59.246569+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:45:59.246569+0000","last_undegraded":"2024-09-21T11:45:59.246569+0000","last_fullsized":"2024-09-21T11:45:59.246569+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:53.034642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":301,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249917+0000","last_change":"2024-09-21T11:45:09.619839+0000","last_active":"2024-09-21T11:45:59.249917+0000","last_peered":"2024-09-21T11:45:59.249917+0000","last_clean":"2024-09-21T11:45:59.249917+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:45:59.249917+0000","last_undegraded":"2024-09-21T11:45:59.249917+0000","last_fullsized":"2024-09-21T11:45:59.249917+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:45:09.619759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.619759+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:18.758338+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":354,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249174+0000","last_change":"2024-09-21T11:41:43.495040+0000","last_active":"2024-09-21T11:45:59.249174+0000","last_peered":"2024-09-21T11:45:59.249174+0000","last_clean":"2024-09-21T11:45:59.249174+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:45:59.249174+0000","last_undegraded":"2024-09-21T11:45:59.249174+0000","last_fullsized":"2024-09-21T11:45:59.249174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:41:43.494960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:43.494960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246876+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:45:59.246876+0000","last_peered":"2024-09-21T11:45:59.246876+0000","last_clean":"2024-09-21T11:45:59.246876+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:45:59.246876+0000","last_undegraded":"2024-09-21T11:45:59.246876+0000","last_fullsized":"2024-09-21T11:45:59.246876+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:25.320855+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.248228+0000","last_change":"2024-09-21T11:45:46.470043+0000","last_active":"2024-09-21T11:45:59.248228+0000","last_peered":"2024-09-21T11:45:59.248228+0000","last_clean":"2024-09-21T11:45:59.248228+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:45:59.248228+0000","last_undegraded":"2024-09-21T11:45:59.248228+0000","last_fullsized":"2024-09-21T11:45:59.248228+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:46.469951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.469951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:14.031400+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":302,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241621+0000","last_change":"2024-09-21T11:42:37.201056+0000","last_active":"2024-09-21T11:45:59.241621+0000","last_peered":"2024-09-21T11:45:59.241621+0000","last_clean":"2024-09-21T11:45:59.241621+0000","last_became_active":"2024-09-21T11:42:37.171563+0000","last_became_peered":"2024-09-21T11:42:37.171563+0000","last_unstale":"2024-09-21T11:45:59.241621+0000","last_undegraded":"2024-09-21T11:45:59.241621+0000","last_fullsized":"2024-09-21T11:45:59.241621+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:41:25.310957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:25.310957+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:23.843504+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":288,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246797+0000","last_change":"2024-09-21T11:44:37.519803+0000","last_active":"2024-09-21T11:45:59.246797+0000","last_peered":"2024-09-21T11:45:59.246797+0000","last_clean":"2024-09-21T11:45:59.246797+0000","last_became_active":"2024-09-21T11:42:37.170365+0000","last_became_peered":"2024-09-21T11:42:37.170365+0000","last_unstale":"2024-09-21T11:45:59.246797+0000","last_undegraded":"2024-09-21T11:45:59.246797+0000","last_fullsized":"2024-09-21T11:45:59.246797+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:37.519750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:37.519750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:58.075070+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249460+0000","last_change":"2024-09-21T11:43:45.551832+0000","last_active":"2024-09-21T11:45:59.249460+0000","last_peered":"2024-09-21T11:45:59.249460+0000","last_clean":"2024-09-21T11:45:59.249460+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:45:59.249460+0000","last_undegraded":"2024-09-21T11:45:59.249460+0000","last_fullsized":"2024-09-21T11:45:59.249460+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.551792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.551792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":310,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.255098+0000","last_change":"2024-09-21T11:43:32.077598+0000","last_active":"2024-09-21T11:45:59.255098+0000","last_peered":"2024-09-21T11:45:59.255098+0000","last_clean":"2024-09-21T11:45:59.255098+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:45:59.255098+0000","last_undegraded":"2024-09-21T11:45:59.255098+0000","last_fullsized":"2024-09-21T11:45:59.255098+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:11.469922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.469922+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:28.225788+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249234+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:45:59.249234+0000","last_peered":"2024-09-21T11:45:59.249234+0000","last_clean":"2024-09-21T11:45:59.249234+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:45:59.249234+0000","last_undegraded":"2024-09-21T11:45:59.249234+0000","last_fullsized":"2024-09-21T11:45:59.249234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:26.713672+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241535+0000","last_change":"2024-09-21T11:45:10.860594+0000","last_active":"2024-09-21T11:45:59.241535+0000","last_peered":"2024-09-21T11:45:59.241535+0000","last_clean":"2024-09-21T11:45:59.241535+0000","last_became_active":"2024-09-21T11:42:37.172423+0000","last_became_peered":"2024-09-21T11:42:37.172423+0000","last_unstale":"2024-09-21T11:45:59.241535+0000","last_undegraded":"2024-09-21T11:45:59.241535+0000","last_fullsized":"2024-09-21T11:45:59.241535+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:10.860526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:10.860526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:22.630126+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.245674+0000","last_change":"2024-09-21T11:44:45.577521+0000","last_active":"2024-09-21T11:45:59.245674+0000","last_peered":"2024-09-21T11:45:59.245674+0000","last_clean":"2024-09-21T11:45:59.245674+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:45:59.245674+0000","last_undegraded":"2024-09-21T11:45:59.245674+0000","last_fullsized":"2024-09-21T11:45:59.245674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.577451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.577451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":293,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247806+0000","last_change":"2024-09-21T11:41:33.917474+0000","last_active":"2024-09-21T11:45:59.247806+0000","last_peered":"2024-09-21T11:45:59.247806+0000","last_clean":"2024-09-21T11:45:59.247806+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:45:59.247806+0000","last_undegraded":"2024-09-21T11:45:59.247806+0000","last_fullsized":"2024-09-21T11:45:59.247806+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:33.917389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:33.917389+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:20.368193+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.251511+0000","last_change":"2024-09-21T11:44:39.310122+0000","last_active":"2024-09-21T11:45:59.251511+0000","last_peered":"2024-09-21T11:45:59.251511+0000","last_clean":"2024-09-21T11:45:59.251511+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:45:59.251511+0000","last_undegraded":"2024-09-21T11:45:59.251511+0000","last_fullsized":"2024-09-21T11:45:59.251511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:39.310075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:39.310075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:07.017104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":347,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.240549+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:45:59.240549+0000","last_peered":"2024-09-21T11:45:59.240549+0000","last_clean":"2024-09-21T11:45:59.240549+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:45:59.240549+0000","last_undegraded":"2024-09-21T11:45:59.240549+0000","last_fullsized":"2024-09-21T11:45:59.240549+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":286,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.239672+0000","last_change":"2024-09-21T11:45:38.652432+0000","last_active":"2024-09-21T11:45:59.239672+0000","last_peered":"2024-09-21T11:45:59.239672+0000","last_clean":"2024-09-21T11:45:59.239672+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:45:59.239672+0000","last_undegraded":"2024-09-21T11:45:59.239672+0000","last_fullsized":"2024-09-21T11:45:59.239672+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:38.652380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:38.652380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:06.653019+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":312,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.240636+0000","last_change":"2024-09-21T11:42:37.207434+0000","last_active":"2024-09-21T11:45:59.240636+0000","last_peered":"2024-09-21T11:45:59.240636+0000","last_clean":"2024-09-21T11:45:59.240636+0000","last_became_active":"2024-09-21T11:42:37.171345+0000","last_became_peered":"2024-09-21T11:42:37.171345+0000","last_unstale":"2024-09-21T11:45:59.240636+0000","last_undegraded":"2024-09-21T11:45:59.240636+0000","last_fullsized":"2024-09-21T11:45:59.240636+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:45:59.769469+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249428+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:45:59.249428+0000","last_peered":"2024-09-21T11:45:59.249428+0000","last_clean":"2024-09-21T11:45:59.249428+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:45:59.249428+0000","last_undegraded":"2024-09-21T11:45:59.249428+0000","last_fullsized":"2024-09-21T11:45:59.249428+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:25.725488+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"114'9","reported_seq":374,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247429+0000","last_change":"2024-09-21T11:44:46.282850+0000","last_active":"2024-09-21T11:45:59.247429+0000","last_peered":"2024-09-21T11:45:59.247429+0000","last_clean":"2024-09-21T11:45:59.247429+0000","last_became_active":"2024-09-21T11:42:37.163771+0000","last_became_peered":"2024-09-21T11:42:37.163771+0000","last_unstale":"2024-09-21T11:45:59.247429+0000","last_undegraded":"2024-09-21T11:45:59.247429+0000","last_fullsized":"2024-09-21T11:45:59.247429+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:44:46.282790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:46.282790+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.465256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":285,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.254402+0000","last_change":"2024-09-21T11:45:44.314664+0000","last_active":"2024-09-21T11:45:59.254402+0000","last_peered":"2024-09-21T11:45:59.254402+0000","last_clean":"2024-09-21T11:45:59.254402+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:45:59.254402+0000","last_undegraded":"2024-09-21T11:45:59.254402+0000","last_fullsized":"2024-09-21T11:45:59.254402+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:44.314461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.314461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:52.882063+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":332,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.238458+0000","last_change":"2024-09-21T11:44:44.791676+0000","last_active":"2024-09-21T11:45:59.238458+0000","last_peered":"2024-09-21T11:45:59.238458+0000","last_clean":"2024-09-21T11:45:59.238458+0000","last_became_active":"2024-09-21T11:42:37.154835+0000","last_became_peered":"2024-09-21T11:42:37.154835+0000","last_unstale":"2024-09-21T11:45:59.238458+0000","last_undegraded":"2024-09-21T11:45:59.238458+0000","last_fullsized":"2024-09-21T11:45:59.238458+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:44:44.791621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.791621+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:11.895258+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":290,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.239384+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:45:59.239384+0000","last_peered":"2024-09-21T11:45:59.239384+0000","last_clean":"2024-09-21T11:45:59.239384+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:45:59.239384+0000","last_undegraded":"2024-09-21T11:45:59.239384+0000","last_fullsized":"2024-09-21T11:45:59.239384+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:55.103919+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":175,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247095+0000","last_change":"2024-09-21T11:44:45.269658+0000","last_active":"2024-09-21T11:45:59.247095+0000","last_peered":"2024-09-21T11:45:59.247095+0000","last_clean":"2024-09-21T11:45:59.247095+0000","last_became_active":"2024-09-21T11:42:37.163261+0000","last_became_peered":"2024-09-21T11:42:37.163261+0000","last_unstale":"2024-09-21T11:45:59.247095+0000","last_undegraded":"2024-09-21T11:45:59.247095+0000","last_fullsized":"2024-09-21T11:45:59.247095+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.269610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.269610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:59.105495+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":270,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.240238+0000","last_change":"2024-09-21T11:45:04.770055+0000","last_active":"2024-09-21T11:45:59.240238+0000","last_peered":"2024-09-21T11:45:59.240238+0000","last_clean":"2024-09-21T11:45:59.240238+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:45:59.240238+0000","last_undegraded":"2024-09-21T11:45:59.240238+0000","last_fullsized":"2024-09-21T11:45:59.240238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:04.769963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:04.769963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:16.912717+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":228,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247532+0000","last_change":"2024-09-21T11:45:09.366566+0000","last_active":"2024-09-21T11:45:59.247532+0000","last_peered":"2024-09-21T11:45:59.247532+0000","last_clean":"2024-09-21T11:45:59.247532+0000","last_became_active":"2024-09-21T11:42:37.167265+0000","last_became_peered":"2024-09-21T11:42:37.167265+0000","last_unstale":"2024-09-21T11:45:59.247532+0000","last_undegraded":"2024-09-21T11:45:59.247532+0000","last_fullsized":"2024-09-21T11:45:59.247532+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:09.366483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.366483+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:21.649277+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":281,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249757+0000","last_change":"2024-09-21T11:43:26.586603+0000","last_active":"2024-09-21T11:45:59.249757+0000","last_peered":"2024-09-21T11:45:59.249757+0000","last_clean":"2024-09-21T11:45:59.249757+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:45:59.249757+0000","last_undegraded":"2024-09-21T11:45:59.249757+0000","last_fullsized":"2024-09-21T11:45:59.249757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.586562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.586562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:22.725895+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.244736+0000","last_change":"2024-09-21T11:44:43.803593+0000","last_active":"2024-09-21T11:45:59.244736+0000","last_peered":"2024-09-21T11:45:59.244736+0000","last_clean":"2024-09-21T11:45:59.244736+0000","last_became_active":"2024-09-21T11:42:37.166917+0000","last_became_peered":"2024-09-21T11:42:37.166917+0000","last_unstale":"2024-09-21T11:45:59.244736+0000","last_undegraded":"2024-09-21T11:45:59.244736+0000","last_fullsized":"2024-09-21T11:45:59.244736+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:43.803530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:43.803530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:51.565596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":297,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.248407+0000","last_change":"2024-09-21T11:45:43.314879+0000","last_active":"2024-09-21T11:45:59.248407+0000","last_peered":"2024-09-21T11:45:59.248407+0000","last_clean":"2024-09-21T11:45:59.248407+0000","last_became_active":"2024-09-21T11:42:37.161589+0000","last_became_peered":"2024-09-21T11:42:37.161589+0000","last_unstale":"2024-09-21T11:45:59.248407+0000","last_undegraded":"2024-09-21T11:45:59.248407+0000","last_fullsized":"2024-09-21T11:45:59.248407+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:45:43.314803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:43.314803+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:02.487936+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":305,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.240423+0000","last_change":"2024-09-21T11:45:46.706221+0000","last_active":"2024-09-21T11:45:59.240423+0000","last_peered":"2024-09-21T11:45:59.240423+0000","last_clean":"2024-09-21T11:45:59.240423+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:45:59.240423+0000","last_undegraded":"2024-09-21T11:45:59.240423+0000","last_fullsized":"2024-09-21T11:45:59.240423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:45:46.706130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.706130+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:06.652208+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":288,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.246941+0000","last_change":"2024-09-21T11:42:37.173431+0000","last_active":"2024-09-21T11:45:59.246941+0000","last_peered":"2024-09-21T11:45:59.246941+0000","last_clean":"2024-09-21T11:45:59.246941+0000","last_became_active":"2024-09-21T11:42:37.172859+0000","last_became_peered":"2024-09-21T11:42:37.172859+0000","last_unstale":"2024-09-21T11:45:59.246941+0000","last_undegraded":"2024-09-21T11:45:59.246941+0000","last_fullsized":"2024-09-21T11:45:59.246941+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.952335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.952335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:28.274100+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":273,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.248732+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:45:59.248732+0000","last_peered":"2024-09-21T11:45:59.248732+0000","last_clean":"2024-09-21T11:45:59.248732+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:45:59.248732+0000","last_undegraded":"2024-09-21T11:45:59.248732+0000","last_fullsized":"2024-09-21T11:45:59.248732+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:34.784746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":278,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.245545+0000","last_change":"2024-09-21T11:45:45.730374+0000","last_active":"2024-09-21T11:45:59.245545+0000","last_peered":"2024-09-21T11:45:59.245545+0000","last_clean":"2024-09-21T11:45:59.245545+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:45:59.245545+0000","last_undegraded":"2024-09-21T11:45:59.245545+0000","last_fullsized":"2024-09-21T11:45:59.245545+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:45.730169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:45.730169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:06.852393+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":318,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241050+0000","last_change":"2024-09-21T11:44:19.651059+0000","last_active":"2024-09-21T11:45:59.241050+0000","last_peered":"2024-09-21T11:45:59.241050+0000","last_clean":"2024-09-21T11:45:59.241050+0000","last_became_active":"2024-09-21T11:42:37.169482+0000","last_became_peered":"2024-09-21T11:42:37.169482+0000","last_unstale":"2024-09-21T11:45:59.241050+0000","last_undegraded":"2024-09-21T11:45:59.241050+0000","last_fullsized":"2024-09-21T11:45:59.241050+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:44:19.649203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:19.649203+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:49.638098+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.255028+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:45:59.255028+0000","last_peered":"2024-09-21T11:45:59.255028+0000","last_clean":"2024-09-21T11:45:59.255028+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:45:59.255028+0000","last_undegraded":"2024-09-21T11:45:59.255028+0000","last_fullsized":"2024-09-21T11:45:59.255028+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:34.215479+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":281,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.245823+0000","last_change":"2024-09-21T11:43:39.838067+0000","last_active":"2024-09-21T11:45:59.245823+0000","last_peered":"2024-09-21T11:45:59.245823+0000","last_clean":"2024-09-21T11:45:59.245823+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:45:59.245823+0000","last_undegraded":"2024-09-21T11:45:59.245823+0000","last_fullsized":"2024-09-21T11:45:59.245823+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:39.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.838002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":277,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.249120+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:45:59.249120+0000","last_peered":"2024-09-21T11:45:59.249120+0000","last_clean":"2024-09-21T11:45:59.249120+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:45:59.249120+0000","last_undegraded":"2024-09-21T11:45:59.249120+0000","last_fullsized":"2024-09-21T11:45:59.249120+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:27.696704+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":174,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.247478+0000","last_change":"2024-09-21T11:44:11.214596+0000","last_active":"2024-09-21T11:45:59.247478+0000","last_peered":"2024-09-21T11:45:59.247478+0000","last_clean":"2024-09-21T11:45:59.247478+0000","last_became_active":"2024-09-21T11:42:37.155620+0000","last_became_peered":"2024-09-21T11:42:37.155620+0000","last_unstale":"2024-09-21T11:45:59.247478+0000","last_undegraded":"2024-09-21T11:45:59.247478+0000","last_fullsized":"2024-09-21T11:45:59.247478+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:11.214534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:11.214534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:22.389504+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.241301+0000","last_change":"2024-09-21T11:43:36.413767+0000","last_active":"2024-09-21T11:45:59.241301+0000","last_peered":"2024-09-21T11:45:59.241301+0000","last_clean":"2024-09-21T11:45:59.241301+0000","last_became_active":"2024-09-21T11:42:37.167248+0000","last_became_peered":"2024-09-21T11:42:37.167248+0000","last_unstale":"2024-09-21T11:45:59.241301+0000","last_undegraded":"2024-09-21T11:45:59.241301+0000","last_fullsized":"2024-09-21T11:45:59.241301+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.413725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.413725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.803112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":331,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.245160+0000","last_change":"2024-09-21T11:44:44.412386+0000","last_active":"2024-09-21T11:45:59.245160+0000","last_peered":"2024-09-21T11:45:59.245160+0000","last_clean":"2024-09-21T11:45:59.245160+0000","last_became_active":"2024-09-21T11:42:37.171253+0000","last_became_peered":"2024-09-21T11:42:37.171253+0000","last_unstale":"2024-09-21T11:45:59.245160+0000","last_undegraded":"2024-09-21T11:45:59.245160+0000","last_fullsized":"2024-09-21T11:45:59.245160+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:44:44.412304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.412304+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":289,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-21T11:45:59.254780+0000","last_change":"2024-09-21T11:45:12.377954+0000","last_active":"2024-09-21T11:45:59.254780+0000","last_peered":"2024-09-21T11:45:59.254780+0000","last_clean":"2024-09-21T11:45:59.254780+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:45:59.254780+0000","last_undegraded":"2024-09-21T11:45:59.254780+0000","last_fullsized":"2024-09-21T11:45:59.254780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:12.377880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:12.377880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:34.634197+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6445268992,"num_objects":98347,"num_object_clones":0,"num_object_copies":295041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98347,"num_write_kb":6294208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19310379008,"data_stored":19310379008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76047,"ondisk_log_size":76047,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":5367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":354,"num_read_kb":349,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234061,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2227760,"kb_used_data":1966108,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":91517392,"statfs":{"total":95995035648,"available":93713809408,"internally_reserved":0,"allocated":2013294592,"data_stored":2012742736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":267909578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462994,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2155836,"kb_used_data":1958056,"kb_used_omap":1,"kb_used_meta":197758,"kb_avail":91589316,"statfs":{"total":95995035648,"available":93787459584,"internally_reserved":0,"allocated":2005049344,"data_stored":2004486025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":202504646},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724631,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3681192,"kb_used_data":3497296,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":90063960,"statfs":{"total":95995035648,"available":92225495040,"internally_reserved":0,"allocated":3581231104,"data_stored":3580690371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188283334},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920861,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2879972,"kb_used_data":2726608,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":90865180,"statfs":{"total":95995035648,"available":93045944320,"internally_reserved":0,"allocated":2792046592,"data_stored":2791506507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":157022661},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117091,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3009568,"kb_used_data":2725712,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":90735584,"statfs":{"total":95995035648,"available":92913238016,"internally_reserved":0,"allocated":2791129088,"data_stored":2790588547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":290650567},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3844488,"kb_used_data":3516216,"kb_used_omap":1,"kb_used_meta":328254,"kb_avail":89900664,"statfs":{"total":95995035648,"available":92058279936,"internally_reserved":0,"allocated":3600605184,"data_stored":3600088881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":336132549},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":55834574960,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2610424,"kb_used_data":2380712,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":91134728,"statfs":{"total":95995035648,"available":93321961472,"internally_reserved":0,"allocated":2437849088,"data_stored":2437285958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":235207109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2439380992,"data_stored":2439380992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3616407552,"data_stored":3616407552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2811166720,"data_stored":2811166720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2802253824,"data_stored":2802253824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3616342016,"data_stored":3616342016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2012151808,"data_stored":2012151808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2012676096,"data_stored":2012676096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:46:06.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:46:06.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:06.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:06.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:06.327 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:06.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:06.369 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:46:06.369 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:46:06.369 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:46:06.369 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:46:06.369 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:46:06.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:06.391 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:06.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:06.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:06.527 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:06.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:06.528 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:06.607 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 64 16 105564 105548 103.046 79.625 0.00956266 0.00970005 2024-09-21T11:46:06.630 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:06.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:06.672 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:06.672 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:06 smithi043 ceph-mon[36454]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:46:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:06 smithi043 ceph-mon[29710]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:46:06.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:06 smithi175 ceph-mon[32613]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:46:06.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:06.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:06.808 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:06.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:06.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:06.948 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:06.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:06.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:07.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:07.086 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:07.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:07.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:07.188 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:07.223 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:07.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:07.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:07.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:07.365 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:07.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:07.367 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:07.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:07.524 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:07.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:07.525 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:07.607 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 65 16 106783 106767 102.633 76.1875 0.00731654 0.0097391 2024-09-21T11:46:07.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:07.663 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:07.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:07.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:07 smithi043 ceph-mon[36454]: pgmap v288: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 863 op/s 2024-09-21T11:46:07.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:07 smithi043 ceph-mon[29710]: pgmap v288: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 863 op/s 2024-09-21T11:46:07.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:07 smithi175 ceph-mon[32613]: pgmap v288: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 863 op/s 2024-09-21T11:46:07.765 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:07.809 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:07.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:07.810 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:07.910 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:07.948 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:07.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:07.949 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:07.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:46:07.989 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:08.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:08.083 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:08.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:08.121 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:08.122 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:08.222 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:08.260 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:08.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:08.261 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:08.327 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:46:08.361 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:08.391 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:08.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:08.391 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:08.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:08 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1795727' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:08.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:08 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:08.471 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:08 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1795727' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:08.471 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:08 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:08.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:08.527 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:08.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:08.529 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:08.607 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 66 16 108046 108030 102.273 78.9375 0.0125682 0.00977311 2024-09-21T11:46:08.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:08.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:08.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:08.760 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:08 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1795727' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:08 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:08.796 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:08.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:08.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:08.870 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:46:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:46:08.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:08.903 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:46:08.903 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:46:08.937 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:46:08.937 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:46:08.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:08.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:08.969 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:46:08.970 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:46:09.005 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:46:09.005 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:46:09.044 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:09.045 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:46:09.046 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:46:09.078 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:09.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:09.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:09.082 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:46:09.082 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:46:09.121 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:46:09.122 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:46:09.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:09.213 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:09.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:09.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:09.314 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:09.350 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:09.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:09.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:09.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:09.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:09.494 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:09.494 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:09.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[36454]: pgmap v289: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-21T11:46:09.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:09.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:46:09.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[36454]: osdmap e191: 8 total, 8 up, 7 in 2024-09-21T11:46:09.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[36454]: 5.0 scrub starts 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[36454]: 5.0 scrub ok 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[36454]: 3.4 scrub starts 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[36454]: 3.4 scrub ok 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2546868584' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[29710]: pgmap v289: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[29710]: osdmap e191: 8 total, 8 up, 7 in 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[29710]: 5.0 scrub starts 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[29710]: 5.0 scrub ok 2024-09-21T11:46:09.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[29710]: 3.4 scrub starts 2024-09-21T11:46:09.571 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[29710]: 3.4 scrub ok 2024-09-21T11:46:09.571 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2546868584' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:09.596 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:09.607 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 67 16 109395 109379 102.005 84.3125 0.0118433 0.0097992 2024-09-21T11:46:09.629 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:09.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:09.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:09.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:09 smithi175 ceph-mon[32613]: pgmap v289: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-21T11:46:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:09 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:09 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:46:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:09 smithi175 ceph-mon[32613]: osdmap e191: 8 total, 8 up, 7 in 2024-09-21T11:46:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:09 smithi175 ceph-mon[32613]: 5.0 scrub starts 2024-09-21T11:46:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:09 smithi175 ceph-mon[32613]: 5.0 scrub ok 2024-09-21T11:46:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:09 smithi175 ceph-mon[32613]: 3.4 scrub starts 2024-09-21T11:46:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:09 smithi175 ceph-mon[32613]: 3.4 scrub ok 2024-09-21T11:46:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2546868584' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:09.773 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:09.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:09.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:09.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:09.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:09.919 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:09.919 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:10.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:10.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:10.062 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:10.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:10.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:10.201 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:10.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:10.202 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:10.223 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:46:10.225 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:46:10.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:10.341 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:10.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:10.342 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:10.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:10.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:10.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:10 smithi043 ceph-mon[36454]: 6.1b scrub starts 2024-09-21T11:46:10.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:10 smithi043 ceph-mon[36454]: 6.1b scrub ok 2024-09-21T11:46:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:10 smithi043 ceph-mon[29710]: 6.1b scrub starts 2024-09-21T11:46:10.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:10 smithi043 ceph-mon[29710]: 6.1b scrub ok 2024-09-21T11:46:10.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:10.607 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 68 15 110746 110731 101.748 84.5 0.00881671 0.00982395 2024-09-21T11:46:10.618 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:10.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:10.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:10.719 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:10.753 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:10.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:10.754 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:10.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:10 smithi175 ceph-mon[32613]: 6.1b scrub starts 2024-09-21T11:46:10.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:10 smithi175 ceph-mon[32613]: 6.1b scrub ok 2024-09-21T11:46:10.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:10.893 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:10.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:10.894 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:10.994 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:11.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:11.031 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:11.031 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:11.132 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:11.167 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:11.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:11.168 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:11.268 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:11.303 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:11.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:11.304 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:11.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:11.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:11.443 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:11.469 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-21T11:46:09.847017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6878009883,"num_objects":105140,"num_object_clones":0,"num_object_copies":315420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105140,"num_whiteouts":0,"num_read":1195,"num_read_kb":1052,"num_write":105490,"num_write_kb":6717897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":594233,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76597,"ondisk_log_size":76597,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":656216064,"kb_used":21785048,"kb_used_data":20036904,"kb_used_omap":10,"kb_used_meta":1748021,"kb_avail":634431016,"statfs":{"total":671965249536,"available":649657360384,"internally_reserved":0,"allocated":20517789696,"data_stored":20513935733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11156,"internal_metadata":1789973612},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,8,12,9,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":432144384,"num_objects":6594,"num_object_clones":0,"num_object_copies":19782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6594,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6594,"num_write_kb":422016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.141684"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.710316+0000","last_change":"2024-09-21T11:41:42.291880+0000","last_active":"2024-09-21T11:46:01.710316+0000","last_peered":"2024-09-21T11:46:01.710316+0000","last_clean":"2024-09-21T11:46:01.710316+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:46:01.710316+0000","last_undegraded":"2024-09-21T11:46:01.710316+0000","last_fullsized":"2024-09-21T11:46:01.710316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:42.290188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:42.290188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.279318+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":329,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708021+0000","last_change":"2024-09-21T11:45:44.553786+0000","last_active":"2024-09-21T11:46:01.708021+0000","last_peered":"2024-09-21T11:46:01.708021+0000","last_clean":"2024-09-21T11:46:01.708021+0000","last_became_active":"2024-09-21T11:42:37.164858+0000","last_became_peered":"2024-09-21T11:42:37.164858+0000","last_unstale":"2024-09-21T11:46:01.708021+0000","last_undegraded":"2024-09-21T11:46:01.708021+0000","last_fullsized":"2024-09-21T11:46:01.708021+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:44.553561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.553561+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:10.417329+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":177,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717173+0000","last_change":"2024-09-21T11:43:37.050216+0000","last_active":"2024-09-21T11:46:01.717173+0000","last_peered":"2024-09-21T11:46:01.717173+0000","last_clean":"2024-09-21T11:46:01.717173+0000","last_became_active":"2024-09-21T11:42:37.161713+0000","last_became_peered":"2024-09-21T11:42:37.161713+0000","last_unstale":"2024-09-21T11:46:01.717173+0000","last_undegraded":"2024-09-21T11:46:01.717173+0000","last_fullsized":"2024-09-21T11:46:01.717173+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:57.113849+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":287,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708033+0000","last_change":"2024-09-21T11:45:07.831102+0000","last_active":"2024-09-21T11:46:01.708033+0000","last_peered":"2024-09-21T11:46:01.708033+0000","last_clean":"2024-09-21T11:46:01.708033+0000","last_became_active":"2024-09-21T11:42:37.158907+0000","last_became_peered":"2024-09-21T11:42:37.158907+0000","last_unstale":"2024-09-21T11:46:01.708033+0000","last_undegraded":"2024-09-21T11:46:01.708033+0000","last_fullsized":"2024-09-21T11:46:01.708033+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:07.831036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.831036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:24.100624+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,7],"acting":[4,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":272,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715082+0000","last_change":"2024-09-21T11:43:28.461458+0000","last_active":"2024-09-21T11:46:01.715082+0000","last_peered":"2024-09-21T11:46:01.715082+0000","last_clean":"2024-09-21T11:46:01.715082+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:46:01.715082+0000","last_undegraded":"2024-09-21T11:46:01.715082+0000","last_fullsized":"2024-09-21T11:46:01.715082+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.461378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.461378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:01.775665+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":277,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717518+0000","last_change":"2024-09-21T11:40:00.307381+0000","last_active":"2024-09-21T11:46:01.717518+0000","last_peered":"2024-09-21T11:46:01.717518+0000","last_clean":"2024-09-21T11:46:01.717518+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:46:01.717518+0000","last_undegraded":"2024-09-21T11:46:01.717518+0000","last_fullsized":"2024-09-21T11:46:01.717518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:39:59.294953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:55.591623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":305,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715031+0000","last_change":"2024-09-21T11:45:07.788590+0000","last_active":"2024-09-21T11:46:01.715031+0000","last_peered":"2024-09-21T11:46:01.715031+0000","last_clean":"2024-09-21T11:46:01.715031+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:46:01.715031+0000","last_undegraded":"2024-09-21T11:46:01.715031+0000","last_fullsized":"2024-09-21T11:46:01.715031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:07.788512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.788512+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:22.182413+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":299,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.707922+0000","last_change":"2024-09-21T11:45:08.781900+0000","last_active":"2024-09-21T11:46:01.707922+0000","last_peered":"2024-09-21T11:46:01.707922+0000","last_clean":"2024-09-21T11:46:01.707922+0000","last_became_active":"2024-09-21T11:42:37.154511+0000","last_became_peered":"2024-09-21T11:42:37.154511+0000","last_unstale":"2024-09-21T11:46:01.707922+0000","last_undegraded":"2024-09-21T11:46:01.707922+0000","last_fullsized":"2024-09-21T11:46:01.707922+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:21.829040+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.720047+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:46:01.720047+0000","last_peered":"2024-09-21T11:46:01.720047+0000","last_clean":"2024-09-21T11:46:01.720047+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:46:01.720047+0000","last_undegraded":"2024-09-21T11:46:01.720047+0000","last_fullsized":"2024-09-21T11:46:01.720047+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:31.264231+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":304,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708308+0000","last_change":"2024-09-21T11:43:26.258023+0000","last_active":"2024-09-21T11:46:01.708308+0000","last_peered":"2024-09-21T11:46:01.708308+0000","last_clean":"2024-09-21T11:46:01.708308+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:46:01.708308+0000","last_undegraded":"2024-09-21T11:46:01.708308+0000","last_fullsized":"2024-09-21T11:46:01.708308+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:26.257967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.257967+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718184+0000","last_change":"2024-09-21T11:44:12.257513+0000","last_active":"2024-09-21T11:46:01.718184+0000","last_peered":"2024-09-21T11:46:01.718184+0000","last_clean":"2024-09-21T11:46:01.718184+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:46:01.718184+0000","last_undegraded":"2024-09-21T11:46:01.718184+0000","last_fullsized":"2024-09-21T11:46:01.718184+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.257454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.257454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:18.340172+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":286,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.707443+0000","last_change":"2024-09-21T11:45:06.385208+0000","last_active":"2024-09-21T11:46:01.707443+0000","last_peered":"2024-09-21T11:46:01.707443+0000","last_clean":"2024-09-21T11:46:01.707443+0000","last_became_active":"2024-09-21T11:42:37.155879+0000","last_became_peered":"2024-09-21T11:42:37.155879+0000","last_unstale":"2024-09-21T11:46:01.707443+0000","last_undegraded":"2024-09-21T11:46:01.707443+0000","last_fullsized":"2024-09-21T11:46:01.707443+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:06.385125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:06.385125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:25.118118+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":306,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717994+0000","last_change":"2024-09-21T11:45:36.172885+0000","last_active":"2024-09-21T11:46:01.717994+0000","last_peered":"2024-09-21T11:46:01.717994+0000","last_clean":"2024-09-21T11:46:01.717994+0000","last_became_active":"2024-09-21T11:42:37.156575+0000","last_became_peered":"2024-09-21T11:42:37.156575+0000","last_unstale":"2024-09-21T11:46:01.717994+0000","last_undegraded":"2024-09-21T11:46:01.717994+0000","last_fullsized":"2024-09-21T11:46:01.717994+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:02.715601+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":311,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708254+0000","last_change":"2024-09-21T11:43:08.326923+0000","last_active":"2024-09-21T11:46:01.708254+0000","last_peered":"2024-09-21T11:46:01.708254+0000","last_clean":"2024-09-21T11:46:01.708254+0000","last_became_active":"2024-09-21T11:41:31.634863+0000","last_became_peered":"2024-09-21T11:41:31.634863+0000","last_unstale":"2024-09-21T11:46:01.708254+0000","last_undegraded":"2024-09-21T11:46:01.708254+0000","last_fullsized":"2024-09-21T11:46:01.708254+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:08.326846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:08.326846+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:27.597564+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":282,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714898+0000","last_change":"2024-09-21T11:43:38.567194+0000","last_active":"2024-09-21T11:46:01.714898+0000","last_peered":"2024-09-21T11:46:01.714898+0000","last_clean":"2024-09-21T11:46:01.714898+0000","last_became_active":"2024-09-21T11:42:37.164285+0000","last_became_peered":"2024-09-21T11:42:37.164285+0000","last_unstale":"2024-09-21T11:46:01.714898+0000","last_undegraded":"2024-09-21T11:46:01.714898+0000","last_fullsized":"2024-09-21T11:46:01.714898+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.567131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.567131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:24.764953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":286,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717264+0000","last_change":"2024-09-21T11:44:42.350677+0000","last_active":"2024-09-21T11:46:01.717264+0000","last_peered":"2024-09-21T11:46:01.717264+0000","last_clean":"2024-09-21T11:46:01.717264+0000","last_became_active":"2024-09-21T11:42:37.162912+0000","last_became_peered":"2024-09-21T11:42:37.162912+0000","last_unstale":"2024-09-21T11:46:01.717264+0000","last_undegraded":"2024-09-21T11:46:01.717264+0000","last_fullsized":"2024-09-21T11:46:01.717264+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:42.350621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:42.350621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:07.179165+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":288,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708421+0000","last_change":"2024-09-21T11:44:38.861427+0000","last_active":"2024-09-21T11:46:01.708421+0000","last_peered":"2024-09-21T11:46:01.708421+0000","last_clean":"2024-09-21T11:46:01.708421+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:46:01.708421+0000","last_undegraded":"2024-09-21T11:46:01.708421+0000","last_fullsized":"2024-09-21T11:46:01.708421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:38.861369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:38.861369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:51.513431+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708745+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:46:01.708745+0000","last_peered":"2024-09-21T11:46:01.708745+0000","last_clean":"2024-09-21T11:46:01.708745+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:46:01.708745+0000","last_undegraded":"2024-09-21T11:46:01.708745+0000","last_fullsized":"2024-09-21T11:46:01.708745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":367,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.716164+0000","last_change":"2024-09-21T11:45:40.530176+0000","last_active":"2024-09-21T11:46:01.716164+0000","last_peered":"2024-09-21T11:46:01.716164+0000","last_clean":"2024-09-21T11:46:01.716164+0000","last_became_active":"2024-09-21T11:42:37.166901+0000","last_became_peered":"2024-09-21T11:42:37.166901+0000","last_unstale":"2024-09-21T11:46:01.716164+0000","last_undegraded":"2024-09-21T11:46:01.716164+0000","last_fullsized":"2024-09-21T11:46:01.716164+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:40.530099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:40.530099+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:41.983312+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":185,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708983+0000","last_change":"2024-09-21T11:44:46.822178+0000","last_active":"2024-09-21T11:46:01.708983+0000","last_peered":"2024-09-21T11:46:01.708983+0000","last_clean":"2024-09-21T11:46:01.708983+0000","last_became_active":"2024-09-21T11:42:37.153799+0000","last_became_peered":"2024-09-21T11:42:37.153799+0000","last_unstale":"2024-09-21T11:46:01.708983+0000","last_undegraded":"2024-09-21T11:46:01.708983+0000","last_fullsized":"2024-09-21T11:46:01.708983+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:46.822132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:46.822132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.312064+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.719024+0000","last_change":"2024-09-21T11:45:39.826902+0000","last_active":"2024-09-21T11:46:01.719024+0000","last_peered":"2024-09-21T11:46:01.719024+0000","last_clean":"2024-09-21T11:46:01.719024+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:46:01.719024+0000","last_undegraded":"2024-09-21T11:46:01.719024+0000","last_fullsized":"2024-09-21T11:46:01.719024+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:39.826701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:39.826701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:58.518406+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708432+0000","last_change":"2024-09-21T11:43:45.093396+0000","last_active":"2024-09-21T11:46:01.708432+0000","last_peered":"2024-09-21T11:46:01.708432+0000","last_clean":"2024-09-21T11:46:01.708432+0000","last_became_active":"2024-09-21T11:42:37.153694+0000","last_became_peered":"2024-09-21T11:42:37.153694+0000","last_unstale":"2024-09-21T11:46:01.708432+0000","last_undegraded":"2024-09-21T11:46:01.708432+0000","last_fullsized":"2024-09-21T11:46:01.708432+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.093325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.093325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:23.661922+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,0],"acting":[4,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":322,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714677+0000","last_change":"2024-09-21T11:42:37.189830+0000","last_active":"2024-09-21T11:46:01.714677+0000","last_peered":"2024-09-21T11:46:01.714677+0000","last_clean":"2024-09-21T11:46:01.714677+0000","last_became_active":"2024-09-21T11:42:37.165964+0000","last_became_peered":"2024-09-21T11:42:37.165964+0000","last_unstale":"2024-09-21T11:46:01.714677+0000","last_undegraded":"2024-09-21T11:46:01.714677+0000","last_fullsized":"2024-09-21T11:46:01.714677+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:41:35.427210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:35.427210+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:32.829672+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":284,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.709482+0000","last_change":"2024-09-21T11:45:37.670176+0000","last_active":"2024-09-21T11:46:01.709482+0000","last_peered":"2024-09-21T11:46:01.709482+0000","last_clean":"2024-09-21T11:46:01.709482+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:46:01.709482+0000","last_undegraded":"2024-09-21T11:46:01.709482+0000","last_fullsized":"2024-09-21T11:46:01.709482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:37.670117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:37.670117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:07.566687+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":288,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715408+0000","last_change":"2024-09-21T11:45:14.660863+0000","last_active":"2024-09-21T11:46:01.715408+0000","last_peered":"2024-09-21T11:46:01.715408+0000","last_clean":"2024-09-21T11:46:01.715408+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:46:01.715408+0000","last_undegraded":"2024-09-21T11:46:01.715408+0000","last_fullsized":"2024-09-21T11:46:01.715408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:45:14.660727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:14.660727+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:35.024056+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":189,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714936+0000","last_change":"2024-09-21T11:44:44.796081+0000","last_active":"2024-09-21T11:46:01.714936+0000","last_peered":"2024-09-21T11:46:01.714936+0000","last_clean":"2024-09-21T11:46:01.714936+0000","last_became_active":"2024-09-21T11:42:37.166144+0000","last_became_peered":"2024-09-21T11:42:37.166144+0000","last_unstale":"2024-09-21T11:46:01.714936+0000","last_undegraded":"2024-09-21T11:46:01.714936+0000","last_fullsized":"2024-09-21T11:46:01.714936+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:44:44.796015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.796015+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":311,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711663+0000","last_change":"2024-09-21T11:45:48.385970+0000","last_active":"2024-09-21T11:46:01.711663+0000","last_peered":"2024-09-21T11:46:01.711663+0000","last_clean":"2024-09-21T11:46:01.711663+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:46:01.711663+0000","last_undegraded":"2024-09-21T11:46:01.711663+0000","last_fullsized":"2024-09-21T11:46:01.711663+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:45:48.385895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:48.385895+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:08.226832+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":384,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.712505+0000","last_change":"2024-09-21T11:45:07.351708+0000","last_active":"2024-09-21T11:46:01.712505+0000","last_peered":"2024-09-21T11:46:01.712505+0000","last_clean":"2024-09-21T11:46:01.712505+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:46:01.712505+0000","last_undegraded":"2024-09-21T11:46:01.712505+0000","last_fullsized":"2024-09-21T11:46:01.712505+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:45:07.351662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.351662+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.953626+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"190'6547","reported_seq":6619,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:05.850664+0000","last_change":"2024-09-21T11:44:51.412099+0000","last_active":"2024-09-21T11:46:05.850664+0000","last_peered":"2024-09-21T11:46:05.850664+0000","last_clean":"2024-09-21T11:46:05.850664+0000","last_became_active":"2024-09-21T11:44:51.411794+0000","last_became_peered":"2024-09-21T11:44:51.411794+0000","last_unstale":"2024-09-21T11:46:05.850664+0000","last_undegraded":"2024-09-21T11:46:05.850664+0000","last_fullsized":"2024-09-21T11:46:05.850664+0000","mapping_epoch":160,"log_start":"169'1700","ondisk_log_start":"169'1700","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4847,"log_dups_size":0,"ondisk_log_size":4847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429064192,"num_objects":6547,"num_object_clones":0,"num_object_copies":19641,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6547,"num_write_kb":419008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":237,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.719520+0000","last_change":"2024-09-21T11:43:44.440472+0000","last_active":"2024-09-21T11:46:01.719520+0000","last_peered":"2024-09-21T11:46:01.719520+0000","last_clean":"2024-09-21T11:46:01.719520+0000","last_became_active":"2024-09-21T11:42:37.169939+0000","last_became_peered":"2024-09-21T11:42:37.169939+0000","last_unstale":"2024-09-21T11:46:01.719520+0000","last_undegraded":"2024-09-21T11:46:01.719520+0000","last_fullsized":"2024-09-21T11:46:01.719520+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:43:44.440374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:44.440374+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:17.887245+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":299,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718714+0000","last_change":"2024-09-21T11:45:42.816788+0000","last_active":"2024-09-21T11:46:01.718714+0000","last_peered":"2024-09-21T11:46:01.718714+0000","last_clean":"2024-09-21T11:46:01.718714+0000","last_became_active":"2024-09-21T11:42:37.171365+0000","last_became_peered":"2024-09-21T11:42:37.171365+0000","last_unstale":"2024-09-21T11:46:01.718714+0000","last_undegraded":"2024-09-21T11:46:01.718714+0000","last_fullsized":"2024-09-21T11:46:01.718714+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:42.816725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:42.816725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:43.651183+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":358,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708535+0000","last_change":"2024-09-21T11:42:26.008760+0000","last_active":"2024-09-21T11:46:01.708535+0000","last_peered":"2024-09-21T11:46:01.708535+0000","last_clean":"2024-09-21T11:46:01.708535+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:46:01.708535+0000","last_undegraded":"2024-09-21T11:46:01.708535+0000","last_fullsized":"2024-09-21T11:46:01.708535+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:26.008646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:26.008646+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:01.551959+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":278,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.719482+0000","last_change":"2024-09-21T11:42:37.157278+0000","last_active":"2024-09-21T11:46:01.719482+0000","last_peered":"2024-09-21T11:46:01.719482+0000","last_clean":"2024-09-21T11:46:01.719482+0000","last_became_active":"2024-09-21T11:42:37.156952+0000","last_became_peered":"2024-09-21T11:42:37.156952+0000","last_unstale":"2024-09-21T11:46:01.719482+0000","last_undegraded":"2024-09-21T11:46:01.719482+0000","last_fullsized":"2024-09-21T11:46:01.719482+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.066324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.066324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:32.220706+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"190'6603","reported_seq":6675,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:06.330535+0000","last_change":"2024-09-21T11:44:51.744583+0000","last_active":"2024-09-21T11:46:06.330535+0000","last_peered":"2024-09-21T11:46:06.330535+0000","last_clean":"2024-09-21T11:46:06.330535+0000","last_became_active":"2024-09-21T11:44:51.744224+0000","last_became_peered":"2024-09-21T11:44:51.744224+0000","last_unstale":"2024-09-21T11:46:06.330535+0000","last_undegraded":"2024-09-21T11:46:06.330535+0000","last_fullsized":"2024-09-21T11:46:06.330535+0000","mapping_epoch":160,"log_start":"169'1900","ondisk_log_start":"169'1900","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4703,"log_dups_size":0,"ondisk_log_size":4703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432734208,"num_objects":6603,"num_object_clones":0,"num_object_copies":19809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6603,"num_write_kb":422592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":320,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718177+0000","last_change":"2024-09-21T11:45:16.866668+0000","last_active":"2024-09-21T11:46:01.718177+0000","last_peered":"2024-09-21T11:46:01.718177+0000","last_clean":"2024-09-21T11:46:01.718177+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:46:01.718177+0000","last_undegraded":"2024-09-21T11:46:01.718177+0000","last_fullsized":"2024-09-21T11:46:01.718177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:45:16.866479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:16.866479+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:38.175618+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":292,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.720671+0000","last_change":"2024-09-21T11:44:18.614485+0000","last_active":"2024-09-21T11:46:01.720671+0000","last_peered":"2024-09-21T11:46:01.720671+0000","last_clean":"2024-09-21T11:46:01.720671+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:46:01.720671+0000","last_undegraded":"2024-09-21T11:46:01.720671+0000","last_fullsized":"2024-09-21T11:46:01.720671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:18.614293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:18.614293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:50.610585+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":284,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.707706+0000","last_change":"2024-09-21T11:44:13.946705+0000","last_active":"2024-09-21T11:46:01.707706+0000","last_peered":"2024-09-21T11:46:01.707706+0000","last_clean":"2024-09-21T11:46:01.707706+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:46:01.707706+0000","last_undegraded":"2024-09-21T11:46:01.707706+0000","last_fullsized":"2024-09-21T11:46:01.707706+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:22.686173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":272,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708225+0000","last_change":"2024-09-21T11:44:11.930363+0000","last_active":"2024-09-21T11:46:01.708225+0000","last_peered":"2024-09-21T11:46:01.708225+0000","last_clean":"2024-09-21T11:46:01.708225+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:46:01.708225+0000","last_undegraded":"2024-09-21T11:46:01.708225+0000","last_fullsized":"2024-09-21T11:46:01.708225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:11.930295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:11.930295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:20.670677+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"190'6514","reported_seq":6584,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:05.841532+0000","last_change":"2024-09-21T11:44:51.744967+0000","last_active":"2024-09-21T11:46:05.841532+0000","last_peered":"2024-09-21T11:46:05.841532+0000","last_clean":"2024-09-21T11:46:05.841532+0000","last_became_active":"2024-09-21T11:44:51.744633+0000","last_became_peered":"2024-09-21T11:44:51.744633+0000","last_unstale":"2024-09-21T11:46:05.841532+0000","last_undegraded":"2024-09-21T11:46:05.841532+0000","last_fullsized":"2024-09-21T11:46:05.841532+0000","mapping_epoch":160,"log_start":"169'1700","ondisk_log_start":"169'1700","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4814,"log_dups_size":0,"ondisk_log_size":4814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.094633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426901504,"num_objects":6514,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6514,"num_write_kb":416896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":317,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.707973+0000","last_change":"2024-09-21T11:45:41.588559+0000","last_active":"2024-09-21T11:46:01.707973+0000","last_peered":"2024-09-21T11:46:01.707973+0000","last_clean":"2024-09-21T11:46:01.707973+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:46:01.707973+0000","last_undegraded":"2024-09-21T11:46:01.707973+0000","last_fullsized":"2024-09-21T11:46:01.707973+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:45:41.588470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:41.588470+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:47.875537+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":303,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.712450+0000","last_change":"2024-09-21T11:45:05.343516+0000","last_active":"2024-09-21T11:46:01.712450+0000","last_peered":"2024-09-21T11:46:01.712450+0000","last_clean":"2024-09-21T11:46:01.712450+0000","last_became_active":"2024-09-21T11:42:37.159390+0000","last_became_peered":"2024-09-21T11:42:37.159390+0000","last_unstale":"2024-09-21T11:46:01.712450+0000","last_undegraded":"2024-09-21T11:46:01.712450+0000","last_fullsized":"2024-09-21T11:46:01.712450+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:45:05.343450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:05.343450+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.902794+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"114'9","reported_seq":370,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718020+0000","last_change":"2024-09-21T11:43:29.417021+0000","last_active":"2024-09-21T11:46:01.718020+0000","last_peered":"2024-09-21T11:46:01.718020+0000","last_clean":"2024-09-21T11:46:01.718020+0000","last_became_active":"2024-09-21T11:42:37.169838+0000","last_became_peered":"2024-09-21T11:42:37.169838+0000","last_unstale":"2024-09-21T11:46:01.718020+0000","last_undegraded":"2024-09-21T11:46:01.718020+0000","last_fullsized":"2024-09-21T11:46:01.718020+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:43:29.416946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:29.416946+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":274,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715040+0000","last_change":"2024-09-21T11:41:39.527210+0000","last_active":"2024-09-21T11:46:01.715040+0000","last_peered":"2024-09-21T11:46:01.715040+0000","last_clean":"2024-09-21T11:46:01.715040+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:46:01.715040+0000","last_undegraded":"2024-09-21T11:46:01.715040+0000","last_fullsized":"2024-09-21T11:46:01.715040+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.527151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.527151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"190'6678","reported_seq":6750,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:06.343397+0000","last_change":"2024-09-21T11:44:51.412955+0000","last_active":"2024-09-21T11:46:06.343397+0000","last_peered":"2024-09-21T11:46:06.343397+0000","last_clean":"2024-09-21T11:46:06.343397+0000","last_became_active":"2024-09-21T11:44:51.412662+0000","last_became_peered":"2024-09-21T11:44:51.412662+0000","last_unstale":"2024-09-21T11:46:06.343397+0000","last_undegraded":"2024-09-21T11:46:06.343397+0000","last_fullsized":"2024-09-21T11:46:06.343397+0000","mapping_epoch":160,"log_start":"169'1900","ondisk_log_start":"169'1900","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4778,"log_dups_size":0,"ondisk_log_size":4778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:06.806791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437649408,"num_objects":6678,"num_object_clones":0,"num_object_copies":20034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6678,"num_write_kb":427392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":223,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717544+0000","last_change":"2024-09-21T11:44:43.309806+0000","last_active":"2024-09-21T11:46:01.717544+0000","last_peered":"2024-09-21T11:46:01.717544+0000","last_clean":"2024-09-21T11:46:01.717544+0000","last_became_active":"2024-09-21T11:42:37.165120+0000","last_became_peered":"2024-09-21T11:42:37.165120+0000","last_unstale":"2024-09-21T11:46:01.717544+0000","last_undegraded":"2024-09-21T11:46:01.717544+0000","last_fullsized":"2024-09-21T11:46:01.717544+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:44:43.309439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:43.309439+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:06.560680+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":299,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.720122+0000","last_change":"2024-09-21T11:45:13.309329+0000","last_active":"2024-09-21T11:46:01.720122+0000","last_peered":"2024-09-21T11:46:01.720122+0000","last_clean":"2024-09-21T11:46:01.720122+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:46:01.720122+0000","last_undegraded":"2024-09-21T11:46:01.720122+0000","last_fullsized":"2024-09-21T11:46:01.720122+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:13.309257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:13.309257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:16.178713+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":360,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708346+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:46:01.708346+0000","last_peered":"2024-09-21T11:46:01.708346+0000","last_clean":"2024-09-21T11:46:01.708346+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:46:01.708346+0000","last_undegraded":"2024-09-21T11:46:01.708346+0000","last_fullsized":"2024-09-21T11:46:01.708346+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:35.166408+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":271,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.707582+0000","last_change":"2024-09-21T11:41:40.278404+0000","last_active":"2024-09-21T11:46:01.707582+0000","last_peered":"2024-09-21T11:46:01.707582+0000","last_clean":"2024-09-21T11:46:01.707582+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:46:01.707582+0000","last_undegraded":"2024-09-21T11:46:01.707582+0000","last_fullsized":"2024-09-21T11:46:01.707582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:40.278315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:40.278315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:25.297484+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"190'6552","reported_seq":6624,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:06.293813+0000","last_change":"2024-09-21T11:44:51.410839+0000","last_active":"2024-09-21T11:46:06.293813+0000","last_peered":"2024-09-21T11:46:06.293813+0000","last_clean":"2024-09-21T11:46:06.293813+0000","last_became_active":"2024-09-21T11:44:51.410302+0000","last_became_peered":"2024-09-21T11:44:51.410302+0000","last_unstale":"2024-09-21T11:46:06.293813+0000","last_undegraded":"2024-09-21T11:46:06.293813+0000","last_fullsized":"2024-09-21T11:46:06.293813+0000","mapping_epoch":160,"log_start":"170'2100","ondisk_log_start":"170'2100","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4452,"log_dups_size":0,"ondisk_log_size":4452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429391872,"num_objects":6552,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6552,"num_write_kb":419328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":315,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708477+0000","last_change":"2024-09-21T11:41:07.065583+0000","last_active":"2024-09-21T11:46:01.708477+0000","last_peered":"2024-09-21T11:46:01.708477+0000","last_clean":"2024-09-21T11:46:01.708477+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:46:01.708477+0000","last_undegraded":"2024-09-21T11:46:01.708477+0000","last_fullsized":"2024-09-21T11:46:01.708477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:41:07.065507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.065507+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":184,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.716248+0000","last_change":"2024-09-21T11:44:44.619532+0000","last_active":"2024-09-21T11:46:01.716248+0000","last_peered":"2024-09-21T11:46:01.716248+0000","last_clean":"2024-09-21T11:46:01.716248+0000","last_became_active":"2024-09-21T11:42:37.157562+0000","last_became_peered":"2024-09-21T11:42:37.157562+0000","last_unstale":"2024-09-21T11:46:01.716248+0000","last_undegraded":"2024-09-21T11:46:01.716248+0000","last_fullsized":"2024-09-21T11:46:01.716248+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:44.619493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.619493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:03.953460+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":275,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711106+0000","last_change":"2024-09-21T11:43:26.704117+0000","last_active":"2024-09-21T11:46:01.711106+0000","last_peered":"2024-09-21T11:46:01.711106+0000","last_clean":"2024-09-21T11:46:01.711106+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:46:01.711106+0000","last_undegraded":"2024-09-21T11:46:01.711106+0000","last_fullsized":"2024-09-21T11:46:01.711106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.704073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.704073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":176,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711619+0000","last_change":"2024-09-21T11:43:37.819912+0000","last_active":"2024-09-21T11:46:01.711619+0000","last_peered":"2024-09-21T11:46:01.711619+0000","last_clean":"2024-09-21T11:46:01.711619+0000","last_became_active":"2024-09-21T11:42:37.155437+0000","last_became_peered":"2024-09-21T11:42:37.155437+0000","last_unstale":"2024-09-21T11:46:01.711619+0000","last_undegraded":"2024-09-21T11:46:01.711619+0000","last_fullsized":"2024-09-21T11:46:01.711619+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"190'6575","reported_seq":6647,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:05.835827+0000","last_change":"2024-09-21T11:44:51.745506+0000","last_active":"2024-09-21T11:46:05.835827+0000","last_peered":"2024-09-21T11:46:05.835827+0000","last_clean":"2024-09-21T11:46:05.835827+0000","last_became_active":"2024-09-21T11:44:51.745200+0000","last_became_peered":"2024-09-21T11:44:51.745200+0000","last_unstale":"2024-09-21T11:46:05.835827+0000","last_undegraded":"2024-09-21T11:46:05.835827+0000","last_fullsized":"2024-09-21T11:46:05.835827+0000","mapping_epoch":160,"log_start":"169'1800","ondisk_log_start":"169'1800","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:00.084252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430899200,"num_objects":6575,"num_object_clones":0,"num_object_copies":19725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6575,"num_write_kb":420800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"140'30","reported_seq":340,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.716117+0000","last_change":"2024-09-21T11:43:37.416987+0000","last_active":"2024-09-21T11:46:01.716117+0000","last_peered":"2024-09-21T11:46:01.716117+0000","last_clean":"2024-09-21T11:46:01.716117+0000","last_became_active":"2024-09-21T11:43:32.003036+0000","last_became_peered":"2024-09-21T11:43:32.003036+0000","last_unstale":"2024-09-21T11:46:01.716117+0000","last_undegraded":"2024-09-21T11:46:01.716117+0000","last_fullsized":"2024-09-21T11:46:01.716117+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:58.675676+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":290,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708574+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:46:01.708574+0000","last_peered":"2024-09-21T11:46:01.708574+0000","last_clean":"2024-09-21T11:46:01.708574+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:46:01.708574+0000","last_undegraded":"2024-09-21T11:46:01.708574+0000","last_fullsized":"2024-09-21T11:46:01.708574+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:33.611132+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":304,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717753+0000","last_change":"2024-09-21T11:42:37.193243+0000","last_active":"2024-09-21T11:46:01.717753+0000","last_peered":"2024-09-21T11:46:01.717753+0000","last_clean":"2024-09-21T11:46:01.717753+0000","last_became_active":"2024-09-21T11:42:37.164723+0000","last_became_peered":"2024-09-21T11:42:37.164723+0000","last_unstale":"2024-09-21T11:46:01.717753+0000","last_undegraded":"2024-09-21T11:46:01.717753+0000","last_fullsized":"2024-09-21T11:46:01.717753+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:42:17.142454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:42:17.142454+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:34.146418+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":290,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708987+0000","last_change":"2024-09-21T11:44:41.300665+0000","last_active":"2024-09-21T11:46:01.708987+0000","last_peered":"2024-09-21T11:46:01.708987+0000","last_clean":"2024-09-21T11:46:01.708987+0000","last_became_active":"2024-09-21T11:42:37.163682+0000","last_became_peered":"2024-09-21T11:42:37.163682+0000","last_unstale":"2024-09-21T11:46:01.708987+0000","last_undegraded":"2024-09-21T11:46:01.708987+0000","last_fullsized":"2024-09-21T11:46:01.708987+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":272,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717216+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:46:01.717216+0000","last_peered":"2024-09-21T11:46:01.717216+0000","last_clean":"2024-09-21T11:46:01.717216+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:46:01.717216+0000","last_undegraded":"2024-09-21T11:46:01.717216+0000","last_fullsized":"2024-09-21T11:46:01.717216+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"190'6472","reported_seq":6543,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:06.295601+0000","last_change":"2024-09-21T11:44:51.410845+0000","last_active":"2024-09-21T11:46:06.295601+0000","last_peered":"2024-09-21T11:46:06.295601+0000","last_clean":"2024-09-21T11:46:06.295601+0000","last_became_active":"2024-09-21T11:44:51.410553+0000","last_became_peered":"2024-09-21T11:44:51.410553+0000","last_unstale":"2024-09-21T11:46:06.295601+0000","last_undegraded":"2024-09-21T11:46:06.295601+0000","last_fullsized":"2024-09-21T11:46:06.295601+0000","mapping_epoch":160,"log_start":"170'2000","ondisk_log_start":"170'2000","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4472,"log_dups_size":0,"ondisk_log_size":4472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:09.920894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":424148992,"num_objects":6472,"num_object_clones":0,"num_object_copies":19416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6472,"num_write_kb":414208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":227,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717812+0000","last_change":"2024-09-21T11:42:37.188687+0000","last_active":"2024-09-21T11:46:01.717812+0000","last_peered":"2024-09-21T11:46:01.717812+0000","last_clean":"2024-09-21T11:46:01.717812+0000","last_became_active":"2024-09-21T11:42:37.161797+0000","last_became_peered":"2024-09-21T11:42:37.161797+0000","last_unstale":"2024-09-21T11:46:01.717812+0000","last_undegraded":"2024-09-21T11:46:01.717812+0000","last_fullsized":"2024-09-21T11:46:01.717812+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":181,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.713660+0000","last_change":"2024-09-21T11:42:37.182044+0000","last_active":"2024-09-21T11:46:01.713660+0000","last_peered":"2024-09-21T11:46:01.713660+0000","last_clean":"2024-09-21T11:46:01.713660+0000","last_became_active":"2024-09-21T11:42:37.156191+0000","last_became_peered":"2024-09-21T11:42:37.156191+0000","last_unstale":"2024-09-21T11:46:01.713660+0000","last_undegraded":"2024-09-21T11:46:01.713660+0000","last_fullsized":"2024-09-21T11:46:01.713660+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.704463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":301,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714103+0000","last_change":"2024-09-21T11:43:36.446391+0000","last_active":"2024-09-21T11:46:01.714103+0000","last_peered":"2024-09-21T11:46:01.714103+0000","last_clean":"2024-09-21T11:46:01.714103+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:46:01.714103+0000","last_undegraded":"2024-09-21T11:46:01.714103+0000","last_fullsized":"2024-09-21T11:46:01.714103+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:43:36.446341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.446341+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:53.554424+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":284,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714971+0000","last_change":"2024-09-21T11:43:28.505134+0000","last_active":"2024-09-21T11:46:01.714971+0000","last_peered":"2024-09-21T11:46:01.714971+0000","last_clean":"2024-09-21T11:46:01.714971+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:46:01.714971+0000","last_undegraded":"2024-09-21T11:46:01.714971+0000","last_fullsized":"2024-09-21T11:46:01.714971+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.504894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.504894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:30.585866+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":271,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.716187+0000","last_change":"2024-09-21T11:42:25.245624+0000","last_active":"2024-09-21T11:46:01.716187+0000","last_peered":"2024-09-21T11:46:01.716187+0000","last_clean":"2024-09-21T11:46:01.716187+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:46:01.716187+0000","last_undegraded":"2024-09-21T11:46:01.716187+0000","last_fullsized":"2024-09-21T11:46:01.716187+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.245565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.245565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.670245+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"190'6448","reported_seq":6520,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:05.974184+0000","last_change":"2024-09-21T11:44:51.410741+0000","last_active":"2024-09-21T11:46:05.974184+0000","last_peered":"2024-09-21T11:46:05.974184+0000","last_clean":"2024-09-21T11:46:05.974184+0000","last_became_active":"2024-09-21T11:44:51.410454+0000","last_became_peered":"2024-09-21T11:44:51.410454+0000","last_unstale":"2024-09-21T11:46:05.974184+0000","last_undegraded":"2024-09-21T11:46:05.974184+0000","last_fullsized":"2024-09-21T11:46:05.974184+0000","mapping_epoch":160,"log_start":"168'1000","ondisk_log_start":"168'1000","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5448,"log_dups_size":0,"ondisk_log_size":5448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":422576128,"num_objects":6448,"num_object_clones":0,"num_object_copies":19344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6448,"num_write_kb":412672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":326,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711707+0000","last_change":"2024-09-21T11:45:09.370313+0000","last_active":"2024-09-21T11:46:01.711707+0000","last_peered":"2024-09-21T11:46:01.711707+0000","last_clean":"2024-09-21T11:46:01.711707+0000","last_became_active":"2024-09-21T11:42:37.164059+0000","last_became_peered":"2024-09-21T11:42:37.164059+0000","last_unstale":"2024-09-21T11:46:01.711707+0000","last_undegraded":"2024-09-21T11:46:01.711707+0000","last_fullsized":"2024-09-21T11:46:01.711707+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:27.315480+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.707744+0000","last_change":"2024-09-21T11:44:45.187728+0000","last_active":"2024-09-21T11:46:01.707744+0000","last_peered":"2024-09-21T11:46:01.707744+0000","last_clean":"2024-09-21T11:46:01.707744+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:46:01.707744+0000","last_undegraded":"2024-09-21T11:46:01.707744+0000","last_fullsized":"2024-09-21T11:46:01.707744+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.187650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.187650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:12.532021+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":674,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:04.563347+0000","last_change":"2024-09-21T11:43:47.494999+0000","last_active":"2024-09-21T11:46:04.563347+0000","last_peered":"2024-09-21T11:46:04.563347+0000","last_clean":"2024-09-21T11:46:04.563347+0000","last_became_active":"2024-09-21T11:42:37.166141+0000","last_became_peered":"2024-09-21T11:42:37.166141+0000","last_unstale":"2024-09-21T11:46:04.563347+0000","last_undegraded":"2024-09-21T11:46:04.563347+0000","last_fullsized":"2024-09-21T11:46:04.563347+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:47.494865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:43:47.494865+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:23.715570+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":359,"num_read_kb":354,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":389,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":282,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718056+0000","last_change":"2024-09-21T11:43:45.046195+0000","last_active":"2024-09-21T11:46:01.718056+0000","last_peered":"2024-09-21T11:46:01.718056+0000","last_clean":"2024-09-21T11:46:01.718056+0000","last_became_active":"2024-09-21T11:42:37.172270+0000","last_became_peered":"2024-09-21T11:42:37.172270+0000","last_unstale":"2024-09-21T11:46:01.718056+0000","last_undegraded":"2024-09-21T11:46:01.718056+0000","last_fullsized":"2024-09-21T11:46:01.718056+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.046128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.046128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":176,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718073+0000","last_change":"2024-09-21T11:44:10.289926+0000","last_active":"2024-09-21T11:46:01.718073+0000","last_peered":"2024-09-21T11:46:01.718073+0000","last_clean":"2024-09-21T11:46:01.718073+0000","last_became_active":"2024-09-21T11:42:37.172645+0000","last_became_peered":"2024-09-21T11:42:37.172645+0000","last_unstale":"2024-09-21T11:46:01.718073+0000","last_undegraded":"2024-09-21T11:46:01.718073+0000","last_fullsized":"2024-09-21T11:46:01.718073+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:23.376344+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"190'6543","reported_seq":6615,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:05.812936+0000","last_change":"2024-09-21T11:44:51.410998+0000","last_active":"2024-09-21T11:46:05.812936+0000","last_peered":"2024-09-21T11:46:05.812936+0000","last_clean":"2024-09-21T11:46:05.812936+0000","last_became_active":"2024-09-21T11:44:51.410701+0000","last_became_peered":"2024-09-21T11:44:51.410701+0000","last_unstale":"2024-09-21T11:46:05.812936+0000","last_undegraded":"2024-09-21T11:46:05.812936+0000","last_fullsized":"2024-09-21T11:46:05.812936+0000","mapping_epoch":160,"log_start":"169'1500","ondisk_log_start":"169'1500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5043,"log_dups_size":0,"ondisk_log_size":5043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":428802048,"num_objects":6543,"num_object_clones":0,"num_object_copies":19629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6543,"num_write_kb":418752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":375,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.712585+0000","last_change":"2024-09-21T11:42:19.703071+0000","last_active":"2024-09-21T11:46:01.712585+0000","last_peered":"2024-09-21T11:46:01.712585+0000","last_clean":"2024-09-21T11:46:01.712585+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:46:01.712585+0000","last_undegraded":"2024-09-21T11:46:01.712585+0000","last_fullsized":"2024-09-21T11:46:01.712585+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:42:19.703007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.703007+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:03.500986+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":292,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718540+0000","last_change":"2024-09-21T11:45:15.850112+0000","last_active":"2024-09-21T11:46:01.718540+0000","last_peered":"2024-09-21T11:46:01.718540+0000","last_clean":"2024-09-21T11:46:01.718540+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:46:01.718540+0000","last_undegraded":"2024-09-21T11:46:01.718540+0000","last_fullsized":"2024-09-21T11:46:01.718540+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:15.850021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:15.850021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:34.453388+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":346,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.716469+0000","last_change":"2024-09-21T11:45:38.787162+0000","last_active":"2024-09-21T11:46:01.716469+0000","last_peered":"2024-09-21T11:46:01.716469+0000","last_clean":"2024-09-21T11:46:01.716469+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:46:01.716469+0000","last_undegraded":"2024-09-21T11:46:01.716469+0000","last_fullsized":"2024-09-21T11:46:01.716469+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:45:38.786968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:45:38.786968+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:53.105012+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":275,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715592+0000","last_change":"2024-09-21T11:43:31.630737+0000","last_active":"2024-09-21T11:46:01.715592+0000","last_peered":"2024-09-21T11:46:01.715592+0000","last_clean":"2024-09-21T11:46:01.715592+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:46:01.715592+0000","last_undegraded":"2024-09-21T11:46:01.715592+0000","last_fullsized":"2024-09-21T11:46:01.715592+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:31.630670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:31.630670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:17.693993+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":279,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717903+0000","last_change":"2024-09-21T11:45:09.222611+0000","last_active":"2024-09-21T11:46:01.717903+0000","last_peered":"2024-09-21T11:46:01.717903+0000","last_clean":"2024-09-21T11:46:01.717903+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:46:01.717903+0000","last_undegraded":"2024-09-21T11:46:01.717903+0000","last_fullsized":"2024-09-21T11:46:01.717903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:09.222544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.222544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:12.846344+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"190'6501","reported_seq":6573,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:05.773569+0000","last_change":"2024-09-21T11:44:51.410732+0000","last_active":"2024-09-21T11:46:05.773569+0000","last_peered":"2024-09-21T11:46:05.773569+0000","last_clean":"2024-09-21T11:46:05.773569+0000","last_became_active":"2024-09-21T11:44:51.410197+0000","last_became_peered":"2024-09-21T11:44:51.410197+0000","last_unstale":"2024-09-21T11:46:05.773569+0000","last_undegraded":"2024-09-21T11:46:05.773569+0000","last_fullsized":"2024-09-21T11:46:05.773569+0000","mapping_epoch":160,"log_start":"169'2000","ondisk_log_start":"169'2000","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4501,"log_dups_size":0,"ondisk_log_size":4501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426049536,"num_objects":6501,"num_object_clones":0,"num_object_copies":19503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6501,"num_write_kb":416064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":318,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708583+0000","last_change":"2024-09-21T11:42:37.179956+0000","last_active":"2024-09-21T11:46:01.708583+0000","last_peered":"2024-09-21T11:46:01.708583+0000","last_clean":"2024-09-21T11:46:01.708583+0000","last_became_active":"2024-09-21T11:42:37.158158+0000","last_became_peered":"2024-09-21T11:42:37.158158+0000","last_unstale":"2024-09-21T11:46:01.708583+0000","last_undegraded":"2024-09-21T11:46:01.708583+0000","last_fullsized":"2024-09-21T11:46:01.708583+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:42:22.041174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:22.041174+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":300,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708197+0000","last_change":"2024-09-21T11:45:11.834001+0000","last_active":"2024-09-21T11:46:01.708197+0000","last_peered":"2024-09-21T11:46:01.708197+0000","last_clean":"2024-09-21T11:46:01.708197+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:46:01.708197+0000","last_undegraded":"2024-09-21T11:46:01.708197+0000","last_fullsized":"2024-09-21T11:46:01.708197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:11.833924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:11.833924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:13.592689+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717723+0000","last_change":"2024-09-21T11:45:11.312119+0000","last_active":"2024-09-21T11:46:01.717723+0000","last_peered":"2024-09-21T11:46:01.717723+0000","last_clean":"2024-09-21T11:46:01.717723+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:46:01.717723+0000","last_undegraded":"2024-09-21T11:46:01.717723+0000","last_fullsized":"2024-09-21T11:46:01.717723+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:11.312058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:11.312058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:32.826661+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":272,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714523+0000","last_change":"2024-09-21T11:41:38.453843+0000","last_active":"2024-09-21T11:46:01.714523+0000","last_peered":"2024-09-21T11:46:01.714523+0000","last_clean":"2024-09-21T11:46:01.714523+0000","last_became_active":"2024-09-21T11:40:06.487541+0000","last_became_peered":"2024-09-21T11:40:06.487541+0000","last_unstale":"2024-09-21T11:46:01.714523+0000","last_undegraded":"2024-09-21T11:46:01.714523+0000","last_fullsized":"2024-09-21T11:46:01.714523+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:38.453579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:38.453579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:33.866050+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"190'6723","reported_seq":6795,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:06.281344+0000","last_change":"2024-09-21T11:44:51.412888+0000","last_active":"2024-09-21T11:46:06.281344+0000","last_peered":"2024-09-21T11:46:06.281344+0000","last_clean":"2024-09-21T11:46:06.281344+0000","last_became_active":"2024-09-21T11:44:51.412596+0000","last_became_peered":"2024-09-21T11:44:51.412596+0000","last_unstale":"2024-09-21T11:46:06.281344+0000","last_undegraded":"2024-09-21T11:46:06.281344+0000","last_fullsized":"2024-09-21T11:46:06.281344+0000","mapping_epoch":160,"log_start":"169'2000","ondisk_log_start":"169'2000","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4723,"log_dups_size":0,"ondisk_log_size":4723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:13.580236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":440598528,"num_objects":6723,"num_object_clones":0,"num_object_copies":20169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6723,"num_write_kb":430272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":341,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.709146+0000","last_change":"2024-09-21T11:45:09.804803+0000","last_active":"2024-09-21T11:46:01.709146+0000","last_peered":"2024-09-21T11:46:01.709146+0000","last_clean":"2024-09-21T11:46:01.709146+0000","last_became_active":"2024-09-21T11:42:37.160867+0000","last_became_peered":"2024-09-21T11:42:37.160867+0000","last_unstale":"2024-09-21T11:46:01.709146+0000","last_undegraded":"2024-09-21T11:46:01.709146+0000","last_fullsized":"2024-09-21T11:46:01.709146+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:45:09.804703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.804703+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:17.840842+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":306,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711461+0000","last_change":"2024-09-21T11:44:40.478955+0000","last_active":"2024-09-21T11:46:01.711461+0000","last_peered":"2024-09-21T11:46:01.711461+0000","last_clean":"2024-09-21T11:46:01.711461+0000","last_became_active":"2024-09-21T11:42:37.167043+0000","last_became_peered":"2024-09-21T11:42:37.167043+0000","last_unstale":"2024-09-21T11:46:01.711461+0000","last_undegraded":"2024-09-21T11:46:01.711461+0000","last_fullsized":"2024-09-21T11:46:01.711461+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:40.478885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:40.478885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3],"acting":[6,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":185,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708661+0000","last_change":"2024-09-21T11:44:45.815953+0000","last_active":"2024-09-21T11:46:01.708661+0000","last_peered":"2024-09-21T11:46:01.708661+0000","last_clean":"2024-09-21T11:46:01.708661+0000","last_became_active":"2024-09-21T11:42:37.160659+0000","last_became_peered":"2024-09-21T11:42:37.160659+0000","last_unstale":"2024-09-21T11:46:01.708661+0000","last_undegraded":"2024-09-21T11:46:01.708661+0000","last_fullsized":"2024-09-21T11:46:01.708661+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.815895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.815895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:49.516167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":288,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717381+0000","last_change":"2024-09-21T11:45:03.661699+0000","last_active":"2024-09-21T11:46:01.717381+0000","last_peered":"2024-09-21T11:46:01.717381+0000","last_clean":"2024-09-21T11:46:01.717381+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:46:01.717381+0000","last_undegraded":"2024-09-21T11:46:01.717381+0000","last_fullsized":"2024-09-21T11:46:01.717381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:03.661625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:03.661625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:15.894877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"190'6518","reported_seq":6589,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:06.262671+0000","last_change":"2024-09-21T11:44:51.409705+0000","last_active":"2024-09-21T11:46:06.262671+0000","last_peered":"2024-09-21T11:46:06.262671+0000","last_clean":"2024-09-21T11:46:06.262671+0000","last_became_active":"2024-09-21T11:44:51.409405+0000","last_became_peered":"2024-09-21T11:44:51.409405+0000","last_unstale":"2024-09-21T11:46:06.262671+0000","last_undegraded":"2024-09-21T11:46:06.262671+0000","last_fullsized":"2024-09-21T11:46:06.262671+0000","mapping_epoch":160,"log_start":"170'2100","ondisk_log_start":"170'2100","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4418,"log_dups_size":0,"ondisk_log_size":4418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:17.726404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427163648,"num_objects":6518,"num_object_clones":0,"num_object_copies":19554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6518,"num_write_kb":417152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,4],"acting":[3,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":311,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.709573+0000","last_change":"2024-09-21T11:44:41.809293+0000","last_active":"2024-09-21T11:46:01.709573+0000","last_peered":"2024-09-21T11:46:01.709573+0000","last_clean":"2024-09-21T11:46:01.709573+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:46:01.709573+0000","last_undegraded":"2024-09-21T11:46:01.709573+0000","last_fullsized":"2024-09-21T11:46:01.709573+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:44:41.809215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.809215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":284,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718001+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:46:01.718001+0000","last_peered":"2024-09-21T11:46:01.718001+0000","last_clean":"2024-09-21T11:46:01.718001+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:46:01.718001+0000","last_undegraded":"2024-09-21T11:46:01.718001+0000","last_fullsized":"2024-09-21T11:46:01.718001+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":366,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.720144+0000","last_change":"2024-09-21T11:43:38.421538+0000","last_active":"2024-09-21T11:46:01.720144+0000","last_peered":"2024-09-21T11:46:01.720144+0000","last_clean":"2024-09-21T11:46:01.720144+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:46:01.720144+0000","last_undegraded":"2024-09-21T11:46:01.720144+0000","last_fullsized":"2024-09-21T11:46:01.720144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:43:38.421251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.421251+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:52.556125+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":279,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708335+0000","last_change":"2024-09-21T11:42:50.131895+0000","last_active":"2024-09-21T11:46:01.708335+0000","last_peered":"2024-09-21T11:46:01.708335+0000","last_clean":"2024-09-21T11:46:01.708335+0000","last_became_active":"2024-09-21T11:42:37.157732+0000","last_became_peered":"2024-09-21T11:42:37.157732+0000","last_unstale":"2024-09-21T11:46:01.708335+0000","last_undegraded":"2024-09-21T11:46:01.708335+0000","last_fullsized":"2024-09-21T11:46:01.708335+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:50.131830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:50.131830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"190'6527","reported_seq":6599,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:06.253972+0000","last_change":"2024-09-21T11:44:51.410552+0000","last_active":"2024-09-21T11:46:06.253972+0000","last_peered":"2024-09-21T11:46:06.253972+0000","last_clean":"2024-09-21T11:46:06.253972+0000","last_became_active":"2024-09-21T11:44:51.410164+0000","last_became_peered":"2024-09-21T11:44:51.410164+0000","last_unstale":"2024-09-21T11:46:06.253972+0000","last_undegraded":"2024-09-21T11:46:06.253972+0000","last_fullsized":"2024-09-21T11:46:06.253972+0000","mapping_epoch":160,"log_start":"170'2100","ondisk_log_start":"170'2100","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4427,"log_dups_size":0,"ondisk_log_size":4427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427753472,"num_objects":6527,"num_object_clones":0,"num_object_copies":19581,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6527,"num_write_kb":417728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":312,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708393+0000","last_change":"2024-09-21T11:42:37.192718+0000","last_active":"2024-09-21T11:46:01.708393+0000","last_peered":"2024-09-21T11:46:01.708393+0000","last_clean":"2024-09-21T11:46:01.708393+0000","last_became_active":"2024-09-21T11:42:37.168751+0000","last_became_peered":"2024-09-21T11:42:37.168751+0000","last_unstale":"2024-09-21T11:46:01.708393+0000","last_undegraded":"2024-09-21T11:46:01.708393+0000","last_fullsized":"2024-09-21T11:46:01.708393+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:15.417817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.417817+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:31.193831+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718146+0000","last_change":"2024-09-21T11:45:07.249883+0000","last_active":"2024-09-21T11:46:01.718146+0000","last_peered":"2024-09-21T11:46:01.718146+0000","last_clean":"2024-09-21T11:46:01.718146+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:46:01.718146+0000","last_undegraded":"2024-09-21T11:46:01.718146+0000","last_fullsized":"2024-09-21T11:46:01.718146+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:07.249826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.249826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:26.836645+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":276,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717573+0000","last_change":"2024-09-21T11:42:18.265457+0000","last_active":"2024-09-21T11:46:01.717573+0000","last_peered":"2024-09-21T11:46:01.717573+0000","last_clean":"2024-09-21T11:46:01.717573+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:46:01.717573+0000","last_undegraded":"2024-09-21T11:46:01.717573+0000","last_fullsized":"2024-09-21T11:46:01.717573+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:18.265392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:18.265392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:55.578456+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":263,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.707657+0000","last_change":"2024-09-21T11:40:06.341948+0000","last_active":"2024-09-21T11:46:01.707657+0000","last_peered":"2024-09-21T11:46:01.707657+0000","last_clean":"2024-09-21T11:46:01.707657+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:46:01.707657+0000","last_undegraded":"2024-09-21T11:46:01.707657+0000","last_fullsized":"2024-09-21T11:46:01.707657+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:40:05.327009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:53.248979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"190'6627","reported_seq":6699,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:06.337174+0000","last_change":"2024-09-21T11:44:51.744832+0000","last_active":"2024-09-21T11:46:06.337174+0000","last_peered":"2024-09-21T11:46:06.337174+0000","last_clean":"2024-09-21T11:46:06.337174+0000","last_became_active":"2024-09-21T11:44:51.744226+0000","last_became_peered":"2024-09-21T11:44:51.744226+0000","last_unstale":"2024-09-21T11:46:06.337174+0000","last_undegraded":"2024-09-21T11:46:06.337174+0000","last_fullsized":"2024-09-21T11:46:06.337174+0000","mapping_epoch":160,"log_start":"169'1900","ondisk_log_start":"169'1900","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4727,"log_dups_size":0,"ondisk_log_size":4727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":434307072,"num_objects":6627,"num_object_clones":0,"num_object_copies":19881,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6627,"num_write_kb":424128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":345,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711243+0000","last_change":"2024-09-21T11:44:38.463592+0000","last_active":"2024-09-21T11:46:01.711243+0000","last_peered":"2024-09-21T11:46:01.711243+0000","last_clean":"2024-09-21T11:46:01.711243+0000","last_became_active":"2024-09-21T11:42:37.166373+0000","last_became_peered":"2024-09-21T11:42:37.166373+0000","last_unstale":"2024-09-21T11:46:01.711243+0000","last_undegraded":"2024-09-21T11:46:01.711243+0000","last_fullsized":"2024-09-21T11:46:01.711243+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:44:38.463535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:38.463535+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:07.934116+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.716463+0000","last_change":"2024-09-21T11:41:46.471940+0000","last_active":"2024-09-21T11:46:01.716463+0000","last_peered":"2024-09-21T11:46:01.716463+0000","last_clean":"2024-09-21T11:46:01.716463+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:46:01.716463+0000","last_undegraded":"2024-09-21T11:46:01.716463+0000","last_fullsized":"2024-09-21T11:46:01.716463+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:46.471872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:46.471872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":292,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715353+0000","last_change":"2024-09-21T11:44:38.629525+0000","last_active":"2024-09-21T11:46:01.715353+0000","last_peered":"2024-09-21T11:46:01.715353+0000","last_clean":"2024-09-21T11:46:01.715353+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:46:01.715353+0000","last_undegraded":"2024-09-21T11:46:01.715353+0000","last_fullsized":"2024-09-21T11:46:01.715353+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:38.629466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:38.629466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:04.842945+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":279,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.716392+0000","last_change":"2024-09-21T11:45:36.779034+0000","last_active":"2024-09-21T11:46:01.716392+0000","last_peered":"2024-09-21T11:46:01.716392+0000","last_clean":"2024-09-21T11:46:01.716392+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:46:01.716392+0000","last_undegraded":"2024-09-21T11:46:01.716392+0000","last_fullsized":"2024-09-21T11:46:01.716392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.778909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.778909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:57.501118+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"190'6584","reported_seq":6656,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:05.833672+0000","last_change":"2024-09-21T11:44:51.746053+0000","last_active":"2024-09-21T11:46:05.833672+0000","last_peered":"2024-09-21T11:46:05.833672+0000","last_clean":"2024-09-21T11:46:05.833672+0000","last_became_active":"2024-09-21T11:44:51.745609+0000","last_became_peered":"2024-09-21T11:44:51.745609+0000","last_unstale":"2024-09-21T11:46:05.833672+0000","last_undegraded":"2024-09-21T11:46:05.833672+0000","last_fullsized":"2024-09-21T11:46:05.833672+0000","mapping_epoch":160,"log_start":"169'1500","ondisk_log_start":"169'1500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5084,"log_dups_size":0,"ondisk_log_size":5084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:15.803134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":431489024,"num_objects":6584,"num_object_clones":0,"num_object_copies":19752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6584,"num_write_kb":421376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":338,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.707866+0000","last_change":"2024-09-21T11:44:43.806283+0000","last_active":"2024-09-21T11:46:01.707866+0000","last_peered":"2024-09-21T11:46:01.707866+0000","last_clean":"2024-09-21T11:46:01.707866+0000","last_became_active":"2024-09-21T11:42:37.165364+0000","last_became_peered":"2024-09-21T11:42:37.165364+0000","last_unstale":"2024-09-21T11:46:01.707866+0000","last_undegraded":"2024-09-21T11:46:01.707866+0000","last_fullsized":"2024-09-21T11:46:01.707866+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:44:43.806186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:43.806186+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:11.084871+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":306,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718110+0000","last_change":"2024-09-21T11:44:47.741161+0000","last_active":"2024-09-21T11:46:01.718110+0000","last_peered":"2024-09-21T11:46:01.718110+0000","last_clean":"2024-09-21T11:46:01.718110+0000","last_became_active":"2024-09-21T11:42:37.165470+0000","last_became_peered":"2024-09-21T11:42:37.165470+0000","last_unstale":"2024-09-21T11:46:01.718110+0000","last_undegraded":"2024-09-21T11:46:01.718110+0000","last_fullsized":"2024-09-21T11:46:01.718110+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:47.741095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:47.741095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":299,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.713524+0000","last_change":"2024-09-21T11:45:05.622676+0000","last_active":"2024-09-21T11:46:01.713524+0000","last_peered":"2024-09-21T11:46:01.713524+0000","last_clean":"2024-09-21T11:46:01.713524+0000","last_became_active":"2024-09-21T11:42:37.155675+0000","last_became_peered":"2024-09-21T11:42:37.155675+0000","last_unstale":"2024-09-21T11:46:01.713524+0000","last_undegraded":"2024-09-21T11:46:01.713524+0000","last_fullsized":"2024-09-21T11:46:01.713524+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:05.622608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:05.622608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:14.685022+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":279,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715176+0000","last_change":"2024-09-21T11:42:37.170697+0000","last_active":"2024-09-21T11:46:01.715176+0000","last_peered":"2024-09-21T11:46:01.715176+0000","last_clean":"2024-09-21T11:46:01.715176+0000","last_became_active":"2024-09-21T11:42:37.168686+0000","last_became_peered":"2024-09-21T11:42:37.168686+0000","last_unstale":"2024-09-21T11:46:01.715176+0000","last_undegraded":"2024-09-21T11:46:01.715176+0000","last_fullsized":"2024-09-21T11:46:01.715176+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:27.327920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:27.327920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:27.806052+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"190'6529","reported_seq":6601,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:05.864081+0000","last_change":"2024-09-21T11:44:51.410102+0000","last_active":"2024-09-21T11:46:05.864081+0000","last_peered":"2024-09-21T11:46:05.864081+0000","last_clean":"2024-09-21T11:46:05.864081+0000","last_became_active":"2024-09-21T11:44:51.409710+0000","last_became_peered":"2024-09-21T11:44:51.409710+0000","last_unstale":"2024-09-21T11:46:05.864081+0000","last_undegraded":"2024-09-21T11:46:05.864081+0000","last_fullsized":"2024-09-21T11:46:05.864081+0000","mapping_epoch":160,"log_start":"169'1700","ondisk_log_start":"169'1700","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4829,"log_dups_size":0,"ondisk_log_size":4829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":427884544,"num_objects":6529,"num_object_clones":0,"num_object_copies":19587,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6529,"num_write_kb":417856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":296,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.721146+0000","last_change":"2024-09-21T11:40:02.399575+0000","last_active":"2024-09-21T11:46:01.721146+0000","last_peered":"2024-09-21T11:46:01.721146+0000","last_clean":"2024-09-21T11:46:01.721146+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:46:01.721146+0000","last_undegraded":"2024-09-21T11:46:01.721146+0000","last_fullsized":"2024-09-21T11:46:01.721146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:40:01.298098+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:53.034642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":303,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715645+0000","last_change":"2024-09-21T11:45:09.619839+0000","last_active":"2024-09-21T11:46:01.715645+0000","last_peered":"2024-09-21T11:46:01.715645+0000","last_clean":"2024-09-21T11:46:01.715645+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:46:01.715645+0000","last_undegraded":"2024-09-21T11:46:01.715645+0000","last_fullsized":"2024-09-21T11:46:01.715645+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:45:09.619759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.619759+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:18.758338+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":357,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714481+0000","last_change":"2024-09-21T11:41:43.495040+0000","last_active":"2024-09-21T11:46:01.714481+0000","last_peered":"2024-09-21T11:46:01.714481+0000","last_clean":"2024-09-21T11:46:01.714481+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:46:01.714481+0000","last_undegraded":"2024-09-21T11:46:01.714481+0000","last_fullsized":"2024-09-21T11:46:01.714481+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:41:43.494960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:41:43.494960+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":279,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717250+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:46:01.717250+0000","last_peered":"2024-09-21T11:46:01.717250+0000","last_clean":"2024-09-21T11:46:01.717250+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:46:01.717250+0000","last_undegraded":"2024-09-21T11:46:01.717250+0000","last_fullsized":"2024-09-21T11:46:01.717250+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:25.320855+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":299,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711031+0000","last_change":"2024-09-21T11:45:46.470043+0000","last_active":"2024-09-21T11:46:01.711031+0000","last_peered":"2024-09-21T11:46:01.711031+0000","last_clean":"2024-09-21T11:46:01.711031+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:46:01.711031+0000","last_undegraded":"2024-09-21T11:46:01.711031+0000","last_fullsized":"2024-09-21T11:46:01.711031+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:46.469951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.469951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:14.031400+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":304,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.719385+0000","last_change":"2024-09-21T11:42:37.201056+0000","last_active":"2024-09-21T11:46:01.719385+0000","last_peered":"2024-09-21T11:46:01.719385+0000","last_clean":"2024-09-21T11:46:01.719385+0000","last_became_active":"2024-09-21T11:42:37.171563+0000","last_became_peered":"2024-09-21T11:42:37.171563+0000","last_unstale":"2024-09-21T11:46:01.719385+0000","last_undegraded":"2024-09-21T11:46:01.719385+0000","last_fullsized":"2024-09-21T11:46:01.719385+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:41:25.310957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:25.310957+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:23.843504+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":290,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717467+0000","last_change":"2024-09-21T11:44:37.519803+0000","last_active":"2024-09-21T11:46:01.717467+0000","last_peered":"2024-09-21T11:46:01.717467+0000","last_clean":"2024-09-21T11:46:01.717467+0000","last_became_active":"2024-09-21T11:42:37.170365+0000","last_became_peered":"2024-09-21T11:42:37.170365+0000","last_unstale":"2024-09-21T11:46:01.717467+0000","last_undegraded":"2024-09-21T11:46:01.717467+0000","last_fullsized":"2024-09-21T11:46:01.717467+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:37.519750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:37.519750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:58.075070+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":279,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.716095+0000","last_change":"2024-09-21T11:43:45.551832+0000","last_active":"2024-09-21T11:46:01.716095+0000","last_peered":"2024-09-21T11:46:01.716095+0000","last_clean":"2024-09-21T11:46:01.716095+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:46:01.716095+0000","last_undegraded":"2024-09-21T11:46:01.716095+0000","last_fullsized":"2024-09-21T11:46:01.716095+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.551792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.551792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":312,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711788+0000","last_change":"2024-09-21T11:43:32.077598+0000","last_active":"2024-09-21T11:46:01.711788+0000","last_peered":"2024-09-21T11:46:01.711788+0000","last_clean":"2024-09-21T11:46:01.711788+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:46:01.711788+0000","last_undegraded":"2024-09-21T11:46:01.711788+0000","last_fullsized":"2024-09-21T11:46:01.711788+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:11.469922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.469922+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:28.225788+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714551+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:46:01.714551+0000","last_peered":"2024-09-21T11:46:01.714551+0000","last_clean":"2024-09-21T11:46:01.714551+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:46:01.714551+0000","last_undegraded":"2024-09-21T11:46:01.714551+0000","last_fullsized":"2024-09-21T11:46:01.714551+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:26.713672+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.719129+0000","last_change":"2024-09-21T11:45:10.860594+0000","last_active":"2024-09-21T11:46:01.719129+0000","last_peered":"2024-09-21T11:46:01.719129+0000","last_clean":"2024-09-21T11:46:01.719129+0000","last_became_active":"2024-09-21T11:42:37.172423+0000","last_became_peered":"2024-09-21T11:42:37.172423+0000","last_unstale":"2024-09-21T11:46:01.719129+0000","last_undegraded":"2024-09-21T11:46:01.719129+0000","last_fullsized":"2024-09-21T11:46:01.719129+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:10.860526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:10.860526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:22.630126+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":288,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715673+0000","last_change":"2024-09-21T11:44:45.577521+0000","last_active":"2024-09-21T11:46:01.715673+0000","last_peered":"2024-09-21T11:46:01.715673+0000","last_clean":"2024-09-21T11:46:01.715673+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:46:01.715673+0000","last_undegraded":"2024-09-21T11:46:01.715673+0000","last_fullsized":"2024-09-21T11:46:01.715673+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.577451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.577451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":295,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.719884+0000","last_change":"2024-09-21T11:41:33.917474+0000","last_active":"2024-09-21T11:46:01.719884+0000","last_peered":"2024-09-21T11:46:01.719884+0000","last_clean":"2024-09-21T11:46:01.719884+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:46:01.719884+0000","last_undegraded":"2024-09-21T11:46:01.719884+0000","last_fullsized":"2024-09-21T11:46:01.719884+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:41:33.917389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:33.917389+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:20.368193+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708234+0000","last_change":"2024-09-21T11:44:39.310122+0000","last_active":"2024-09-21T11:46:01.708234+0000","last_peered":"2024-09-21T11:46:01.708234+0000","last_clean":"2024-09-21T11:46:01.708234+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:46:01.708234+0000","last_undegraded":"2024-09-21T11:46:01.708234+0000","last_fullsized":"2024-09-21T11:46:01.708234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:39.310075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:39.310075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:07.017104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":350,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718165+0000","last_change":"2024-09-21T11:40:04.328283+0000","last_active":"2024-09-21T11:46:01.718165+0000","last_peered":"2024-09-21T11:46:01.718165+0000","last_clean":"2024-09-21T11:46:01.718165+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:46:01.718165+0000","last_undegraded":"2024-09-21T11:46:01.718165+0000","last_fullsized":"2024-09-21T11:46:01.718165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:40:03.313144+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:00.712206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":288,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708435+0000","last_change":"2024-09-21T11:45:38.652432+0000","last_active":"2024-09-21T11:46:01.708435+0000","last_peered":"2024-09-21T11:46:01.708435+0000","last_clean":"2024-09-21T11:46:01.708435+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:46:01.708435+0000","last_undegraded":"2024-09-21T11:46:01.708435+0000","last_fullsized":"2024-09-21T11:46:01.708435+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:38.652380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:38.652380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:06.653019+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":314,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.719190+0000","last_change":"2024-09-21T11:42:37.207434+0000","last_active":"2024-09-21T11:46:01.719190+0000","last_peered":"2024-09-21T11:46:01.719190+0000","last_clean":"2024-09-21T11:46:01.719190+0000","last_became_active":"2024-09-21T11:42:37.171345+0000","last_became_peered":"2024-09-21T11:42:37.171345+0000","last_unstale":"2024-09-21T11:46:01.719190+0000","last_undegraded":"2024-09-21T11:46:01.719190+0000","last_fullsized":"2024-09-21T11:46:01.719190+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715290+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:46:01.715290+0000","last_peered":"2024-09-21T11:46:01.715290+0000","last_clean":"2024-09-21T11:46:01.715290+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:46:01.715290+0000","last_undegraded":"2024-09-21T11:46:01.715290+0000","last_fullsized":"2024-09-21T11:46:01.715290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:25.725488+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"114'9","reported_seq":377,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.719688+0000","last_change":"2024-09-21T11:44:46.282850+0000","last_active":"2024-09-21T11:46:01.719688+0000","last_peered":"2024-09-21T11:46:01.719688+0000","last_clean":"2024-09-21T11:46:01.719688+0000","last_became_active":"2024-09-21T11:42:37.163771+0000","last_became_peered":"2024-09-21T11:42:37.163771+0000","last_unstale":"2024-09-21T11:46:01.719688+0000","last_undegraded":"2024-09-21T11:46:01.719688+0000","last_fullsized":"2024-09-21T11:46:01.719688+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:44:46.282790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:46.282790+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":287,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708238+0000","last_change":"2024-09-21T11:45:44.314664+0000","last_active":"2024-09-21T11:46:01.708238+0000","last_peered":"2024-09-21T11:46:01.708238+0000","last_clean":"2024-09-21T11:46:01.708238+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:46:01.708238+0000","last_undegraded":"2024-09-21T11:46:01.708238+0000","last_fullsized":"2024-09-21T11:46:01.708238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:44.314461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.314461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:52.882063+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":334,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.708341+0000","last_change":"2024-09-21T11:44:44.791676+0000","last_active":"2024-09-21T11:46:01.708341+0000","last_peered":"2024-09-21T11:46:01.708341+0000","last_clean":"2024-09-21T11:46:01.708341+0000","last_became_active":"2024-09-21T11:42:37.154835+0000","last_became_peered":"2024-09-21T11:42:37.154835+0000","last_unstale":"2024-09-21T11:46:01.708341+0000","last_undegraded":"2024-09-21T11:46:01.708341+0000","last_fullsized":"2024-09-21T11:46:01.708341+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:44:44.791621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.791621+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:11.895258+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":292,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.709657+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:46:01.709657+0000","last_peered":"2024-09-21T11:46:01.709657+0000","last_clean":"2024-09-21T11:46:01.709657+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:46:01.709657+0000","last_undegraded":"2024-09-21T11:46:01.709657+0000","last_fullsized":"2024-09-21T11:46:01.709657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:55.103919+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":177,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717457+0000","last_change":"2024-09-21T11:44:45.269658+0000","last_active":"2024-09-21T11:46:01.717457+0000","last_peered":"2024-09-21T11:46:01.717457+0000","last_clean":"2024-09-21T11:46:01.717457+0000","last_became_active":"2024-09-21T11:42:37.163261+0000","last_became_peered":"2024-09-21T11:42:37.163261+0000","last_unstale":"2024-09-21T11:46:01.717457+0000","last_undegraded":"2024-09-21T11:46:01.717457+0000","last_fullsized":"2024-09-21T11:46:01.717457+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.269610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.269610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:59.105495+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":272,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714608+0000","last_change":"2024-09-21T11:45:04.770055+0000","last_active":"2024-09-21T11:46:01.714608+0000","last_peered":"2024-09-21T11:46:01.714608+0000","last_clean":"2024-09-21T11:46:01.714608+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:46:01.714608+0000","last_undegraded":"2024-09-21T11:46:01.714608+0000","last_fullsized":"2024-09-21T11:46:01.714608+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:04.769963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:04.769963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:16.912717+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":230,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711375+0000","last_change":"2024-09-21T11:45:09.366566+0000","last_active":"2024-09-21T11:46:01.711375+0000","last_peered":"2024-09-21T11:46:01.711375+0000","last_clean":"2024-09-21T11:46:01.711375+0000","last_became_active":"2024-09-21T11:42:37.167265+0000","last_became_peered":"2024-09-21T11:42:37.167265+0000","last_unstale":"2024-09-21T11:46:01.711375+0000","last_undegraded":"2024-09-21T11:46:01.711375+0000","last_fullsized":"2024-09-21T11:46:01.711375+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:09.366483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.366483+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:21.649277+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.716428+0000","last_change":"2024-09-21T11:43:26.586603+0000","last_active":"2024-09-21T11:46:01.716428+0000","last_peered":"2024-09-21T11:46:01.716428+0000","last_clean":"2024-09-21T11:46:01.716428+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:46:01.716428+0000","last_undegraded":"2024-09-21T11:46:01.716428+0000","last_fullsized":"2024-09-21T11:46:01.716428+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.586562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.586562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:22.725895+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715273+0000","last_change":"2024-09-21T11:44:43.803593+0000","last_active":"2024-09-21T11:46:01.715273+0000","last_peered":"2024-09-21T11:46:01.715273+0000","last_clean":"2024-09-21T11:46:01.715273+0000","last_became_active":"2024-09-21T11:42:37.166917+0000","last_became_peered":"2024-09-21T11:42:37.166917+0000","last_unstale":"2024-09-21T11:46:01.715273+0000","last_undegraded":"2024-09-21T11:46:01.715273+0000","last_fullsized":"2024-09-21T11:46:01.715273+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:43.803530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:43.803530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:51.565596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":299,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.707274+0000","last_change":"2024-09-21T11:45:43.314879+0000","last_active":"2024-09-21T11:46:01.707274+0000","last_peered":"2024-09-21T11:46:01.707274+0000","last_clean":"2024-09-21T11:46:01.707274+0000","last_became_active":"2024-09-21T11:42:37.161589+0000","last_became_peered":"2024-09-21T11:42:37.161589+0000","last_unstale":"2024-09-21T11:46:01.707274+0000","last_undegraded":"2024-09-21T11:46:01.707274+0000","last_fullsized":"2024-09-21T11:46:01.707274+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:45:43.314803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:43.314803+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:02.487936+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":307,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714745+0000","last_change":"2024-09-21T11:45:46.706221+0000","last_active":"2024-09-21T11:46:01.714745+0000","last_peered":"2024-09-21T11:46:01.714745+0000","last_clean":"2024-09-21T11:46:01.714745+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:46:01.714745+0000","last_undegraded":"2024-09-21T11:46:01.714745+0000","last_fullsized":"2024-09-21T11:46:01.714745+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:45:46.706130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.706130+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:06.652208+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":290,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.717557+0000","last_change":"2024-09-21T11:42:37.173431+0000","last_active":"2024-09-21T11:46:01.717557+0000","last_peered":"2024-09-21T11:46:01.717557+0000","last_clean":"2024-09-21T11:46:01.717557+0000","last_became_active":"2024-09-21T11:42:37.172859+0000","last_became_peered":"2024-09-21T11:42:37.172859+0000","last_unstale":"2024-09-21T11:46:01.717557+0000","last_undegraded":"2024-09-21T11:46:01.717557+0000","last_fullsized":"2024-09-21T11:46:01.717557+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.952335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.952335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:28.274100+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":275,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.716024+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:46:01.716024+0000","last_peered":"2024-09-21T11:46:01.716024+0000","last_clean":"2024-09-21T11:46:01.716024+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:46:01.716024+0000","last_undegraded":"2024-09-21T11:46:01.716024+0000","last_fullsized":"2024-09-21T11:46:01.716024+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:34.784746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.715423+0000","last_change":"2024-09-21T11:45:45.730374+0000","last_active":"2024-09-21T11:46:01.715423+0000","last_peered":"2024-09-21T11:46:01.715423+0000","last_clean":"2024-09-21T11:46:01.715423+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:46:01.715423+0000","last_undegraded":"2024-09-21T11:46:01.715423+0000","last_fullsized":"2024-09-21T11:46:01.715423+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:45.730169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:45.730169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:06.852393+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":320,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718075+0000","last_change":"2024-09-21T11:44:19.651059+0000","last_active":"2024-09-21T11:46:01.718075+0000","last_peered":"2024-09-21T11:46:01.718075+0000","last_clean":"2024-09-21T11:46:01.718075+0000","last_became_active":"2024-09-21T11:42:37.169482+0000","last_became_peered":"2024-09-21T11:42:37.169482+0000","last_unstale":"2024-09-21T11:46:01.718075+0000","last_undegraded":"2024-09-21T11:46:01.718075+0000","last_fullsized":"2024-09-21T11:46:01.718075+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:44:19.649203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:19.649203+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:49.638098+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,4],"acting":[3,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.710390+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:46:01.710390+0000","last_peered":"2024-09-21T11:46:01.710390+0000","last_clean":"2024-09-21T11:46:01.710390+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:46:01.710390+0000","last_undegraded":"2024-09-21T11:46:01.710390+0000","last_fullsized":"2024-09-21T11:46:01.710390+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:34.215479+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711704+0000","last_change":"2024-09-21T11:43:39.838067+0000","last_active":"2024-09-21T11:46:01.711704+0000","last_peered":"2024-09-21T11:46:01.711704+0000","last_clean":"2024-09-21T11:46:01.711704+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:46:01.711704+0000","last_undegraded":"2024-09-21T11:46:01.711704+0000","last_fullsized":"2024-09-21T11:46:01.711704+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:39.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.838002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":279,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.714814+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:46:01.714814+0000","last_peered":"2024-09-21T11:46:01.714814+0000","last_clean":"2024-09-21T11:46:01.714814+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:46:01.714814+0000","last_undegraded":"2024-09-21T11:46:01.714814+0000","last_fullsized":"2024-09-21T11:46:01.714814+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:27.696704+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":176,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.719744+0000","last_change":"2024-09-21T11:44:11.214596+0000","last_active":"2024-09-21T11:46:01.719744+0000","last_peered":"2024-09-21T11:46:01.719744+0000","last_clean":"2024-09-21T11:46:01.719744+0000","last_became_active":"2024-09-21T11:42:37.155620+0000","last_became_peered":"2024-09-21T11:42:37.155620+0000","last_unstale":"2024-09-21T11:46:01.719744+0000","last_undegraded":"2024-09-21T11:46:01.719744+0000","last_fullsized":"2024-09-21T11:46:01.719744+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:11.214534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:11.214534+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:22.389504+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":291,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.718635+0000","last_change":"2024-09-21T11:43:36.413767+0000","last_active":"2024-09-21T11:46:01.718635+0000","last_peered":"2024-09-21T11:46:01.718635+0000","last_clean":"2024-09-21T11:46:01.718635+0000","last_became_active":"2024-09-21T11:42:37.167248+0000","last_became_peered":"2024-09-21T11:42:37.167248+0000","last_unstale":"2024-09-21T11:46:01.718635+0000","last_undegraded":"2024-09-21T11:46:01.718635+0000","last_fullsized":"2024-09-21T11:46:01.718635+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:36.413725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:36.413725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:02.803112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":333,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.712550+0000","last_change":"2024-09-21T11:44:44.412386+0000","last_active":"2024-09-21T11:46:01.712550+0000","last_peered":"2024-09-21T11:46:01.712550+0000","last_clean":"2024-09-21T11:46:01.712550+0000","last_became_active":"2024-09-21T11:42:37.171253+0000","last_became_peered":"2024-09-21T11:42:37.171253+0000","last_unstale":"2024-09-21T11:46:01.712550+0000","last_undegraded":"2024-09-21T11:46:01.712550+0000","last_fullsized":"2024-09-21T11:46:01.712550+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:44:44.412304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:44:44.412304+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":292,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-21T11:46:01.711541+0000","last_change":"2024-09-21T11:45:12.377954+0000","last_active":"2024-09-21T11:46:01.711541+0000","last_peered":"2024-09-21T11:46:01.711541+0000","last_clean":"2024-09-21T11:46:01.711541+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:46:01.711541+0000","last_undegraded":"2024-09-21T11:46:01.711541+0000","last_fullsized":"2024-09-21T11:46:01.711541+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:12.377880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:12.377880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:46:34.634197+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6877413376,"num_objects":104941,"num_object_clones":0,"num_object_copies":314823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104941,"num_write_kb":6716224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20604387328,"data_stored":20604387328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76041,"ondisk_log_size":76041,"up":48,"acting":48,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":13,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":499712,"data_stored":13224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":5367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":359,"num_read_kb":354,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234062,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2385400,"kb_used_data":2097384,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":91359752,"statfs":{"total":95995035648,"available":93552386048,"internally_reserved":0,"allocated":2147721216,"data_stored":2147163932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":294910410},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462995,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2291972,"kb_used_data":2090036,"kb_used_omap":1,"kb_used_meta":201918,"kb_avail":91453180,"statfs":{"total":95995035648,"available":93648056320,"internally_reserved":0,"allocated":2140196864,"data_stored":2139628117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":206764486},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724632,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3947820,"kb_used_data":3731996,"kb_used_omap":1,"kb_used_meta":215806,"kb_avail":89797332,"statfs":{"total":95995035648,"available":91952467968,"internally_reserved":0,"allocated":3821563904,"data_stored":3821017743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":220985798},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920862,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3082860,"kb_used_data":2917020,"kb_used_omap":1,"kb_used_meta":165822,"kb_avail":90662292,"statfs":{"total":95995035648,"available":92838187008,"internally_reserved":0,"allocated":2987028480,"data_stored":2986482967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169802181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117092,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3200108,"kb_used_data":2903772,"kb_used_omap":1,"kb_used_meta":296318,"kb_avail":90545044,"statfs":{"total":95995035648,"available":92718125056,"internally_reserved":0,"allocated":2973462528,"data_stored":2972916559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":303430087},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4079956,"kb_used_data":3755844,"kb_used_omap":1,"kb_used_meta":324094,"kb_avail":89665196,"statfs":{"total":95995035648,"available":91817160704,"internally_reserved":0,"allocated":3845984256,"data_stored":3845462525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":331872709},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":55834574961,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738487,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2796932,"kb_used_data":2540852,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":90948220,"statfs":{"total":95995035648,"available":93130977280,"internally_reserved":0,"allocated":2601832448,"data_stored":2601263890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":262207941},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2605383680,"data_stored":2605383680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3859611648,"data_stored":3859611648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2997747712,"data_stored":2997747712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2991652864,"data_stored":2991652864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3855876096,"data_stored":3855876096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2145845248,"data_stored":2145845248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2148270080,"data_stored":2148270080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:46:11.475 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-21T11:46:11.475 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:46:11.475 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-21T11:46:11.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.0 'trim stale osdmaps' 2024-09-21T11:46:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:11.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:11 smithi175 ceph-mon[32613]: pgmap v291: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 628 B/s rd, 51 MiB/s wr, 810 op/s 2024-09-21T11:46:11.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:11 smithi175 ceph-mon[32613]: 5.15 scrub starts 2024-09-21T11:46:11.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:11 smithi175 ceph-mon[32613]: 5.15 scrub ok 2024-09-21T11:46:11.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:11 smithi175 ceph-mon[32613]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:46:11.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:46:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:46:11.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:11.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:46:11.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:11.598 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:11.599 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:11.609 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 69 16 112107 112091 101.502 85 0.0162061 0.009847 2024-09-21T11:46:11.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:11 smithi043 ceph-mon[29710]: pgmap v291: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 628 B/s rd, 51 MiB/s wr, 810 op/s 2024-09-21T11:46:11.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:11 smithi043 ceph-mon[29710]: 5.15 scrub starts 2024-09-21T11:46:11.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:11 smithi043 ceph-mon[29710]: 5.15 scrub ok 2024-09-21T11:46:11.626 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:11 smithi043 ceph-mon[29710]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:46:11.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:11 smithi043 ceph-mon[36454]: pgmap v291: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 628 B/s rd, 51 MiB/s wr, 810 op/s 2024-09-21T11:46:11.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:11 smithi043 ceph-mon[36454]: 5.15 scrub starts 2024-09-21T11:46:11.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:11 smithi043 ceph-mon[36454]: 5.15 scrub ok 2024-09-21T11:46:11.626 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:11 smithi043 ceph-mon[36454]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:46:11.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:11.735 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:11.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:11.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:11.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:11.865 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:11.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:11.866 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:11.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:12.011 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:12.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:12.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:12.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:12.161 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:46:12.161 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:46:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:12.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:12.172 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:12.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:12.311 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:12.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:12.313 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:12.413 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:12.440 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:12.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:12.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:12.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:12.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:12.578 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:12.579 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:12.609 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 70 16 113229 113213 101.054 70.125 0.0414405 0.0098872 2024-09-21T11:46:12.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:12.714 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:12.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:12.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:12.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:46:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: 5.9 scrub starts 2024-09-21T11:46:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: 5.9 scrub ok 2024-09-21T11:46:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:46:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:46:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: 8.4 scrub starts 2024-09-21T11:46:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: osdmap e192: 8 total, 8 up, 7 in 2024-09-21T11:46:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: 3.18 scrub starts 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: 3.18 scrub ok 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: 4.c scrub starts 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[36454]: 4.c scrub ok 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: 5.9 scrub starts 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: 5.9 scrub ok 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: 8.4 scrub starts 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: osdmap e192: 8 total, 8 up, 7 in 2024-09-21T11:46:12.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: 3.18 scrub starts 2024-09-21T11:46:12.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: 3.18 scrub ok 2024-09-21T11:46:12.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: 4.c scrub starts 2024-09-21T11:46:12.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:12 smithi043 ceph-mon[29710]: 4.c scrub ok 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: 5.9 scrub starts 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: 5.9 scrub ok 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: 8.4 scrub starts 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: osdmap e192: 8 total, 8 up, 7 in 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: 3.18 scrub starts 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: 3.18 scrub ok 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: 4.c scrub starts 2024-09-21T11:46:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:12 smithi175 ceph-mon[32613]: 4.c scrub ok 2024-09-21T11:46:12.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:12.851 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:12.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:12.853 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:12.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:13.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:13.003 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:13.003 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:13.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:13.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:13.143 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:13.144 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:13.244 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:13.265 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:13.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:13.266 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:13.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:13.400 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:13.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:13.401 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:13.472 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:13.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:13.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:13.545 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:13.545 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:13.611 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 71 15 114356 114341 100.621 70.5 0.0205845 0.00993365 2024-09-21T11:46:13.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:13.682 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:13.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:13.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[36454]: pgmap v293: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s 2024-09-21T11:46:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[36454]: 4.2 scrub starts 2024-09-21T11:46:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[36454]: 4.2 scrub ok 2024-09-21T11:46:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[36454]: 3.19 scrub starts 2024-09-21T11:46:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[36454]: 3.19 scrub ok 2024-09-21T11:46:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[36454]: 8.6 scrub starts 2024-09-21T11:46:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[29710]: pgmap v293: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s 2024-09-21T11:46:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[29710]: 4.2 scrub starts 2024-09-21T11:46:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[29710]: 4.2 scrub ok 2024-09-21T11:46:13.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[29710]: 3.19 scrub starts 2024-09-21T11:46:13.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[29710]: 3.19 scrub ok 2024-09-21T11:46:13.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:13 smithi043 ceph-mon[29710]: 8.6 scrub starts 2024-09-21T11:46:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:13 smithi175 ceph-mon[32613]: pgmap v293: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 102 MiB/s wr, 1.63k op/s 2024-09-21T11:46:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:13 smithi175 ceph-mon[32613]: 4.2 scrub starts 2024-09-21T11:46:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:13 smithi175 ceph-mon[32613]: 4.2 scrub ok 2024-09-21T11:46:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:13 smithi175 ceph-mon[32613]: 3.19 scrub starts 2024-09-21T11:46:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:13 smithi175 ceph-mon[32613]: 3.19 scrub ok 2024-09-21T11:46:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:13 smithi175 ceph-mon[32613]: 8.6 scrub starts 2024-09-21T11:46:13.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:13.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:13.808 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:13.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:13.945 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:13.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:13.946 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:14.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:14.082 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:14.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:14.083 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:14.184 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:14.218 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:14.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:14.219 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:14.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:14.340 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:14.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:14.341 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:14.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:14.483 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:14.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:14.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:14.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:14.611 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 72 16 115417 115401 100.144 66.25 0.0204112 0.00998106 2024-09-21T11:46:14.625 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:14.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:14.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:14.727 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:14.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[36454]: 3.d scrub starts 2024-09-21T11:46:14.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[36454]: 3.d scrub ok 2024-09-21T11:46:14.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[36454]: 8.4 scrub ok 2024-09-21T11:46:14.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[36454]: 6.19 scrub starts 2024-09-21T11:46:14.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[36454]: 6.19 scrub ok 2024-09-21T11:46:14.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[36454]: 8.f scrub starts 2024-09-21T11:46:14.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[29710]: 3.d scrub starts 2024-09-21T11:46:14.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[29710]: 3.d scrub ok 2024-09-21T11:46:14.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[29710]: 8.4 scrub ok 2024-09-21T11:46:14.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[29710]: 6.19 scrub starts 2024-09-21T11:46:14.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[29710]: 6.19 scrub ok 2024-09-21T11:46:14.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:14 smithi043 ceph-mon[29710]: 8.f scrub starts 2024-09-21T11:46:14.753 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:14.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:14.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:14.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:14 smithi175 ceph-mon[32613]: 3.d scrub starts 2024-09-21T11:46:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:14 smithi175 ceph-mon[32613]: 3.d scrub ok 2024-09-21T11:46:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:14 smithi175 ceph-mon[32613]: 8.4 scrub ok 2024-09-21T11:46:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:14 smithi175 ceph-mon[32613]: 6.19 scrub starts 2024-09-21T11:46:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:14 smithi175 ceph-mon[32613]: 6.19 scrub ok 2024-09-21T11:46:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:14 smithi175 ceph-mon[32613]: 8.f scrub starts 2024-09-21T11:46:14.856 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:14.892 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:14.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:14.892 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:14.993 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:15.027 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:15.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:15.028 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:15.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:15.168 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:15.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:15.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:15.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:15.283 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:15.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:46:15.332 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:15.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:15.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:15.436 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:15.460 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:15.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:15.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:15.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:46:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:46:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:46:15.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:15 smithi043 ceph-mon[29710]: pgmap v294: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 638 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-21T11:46:15.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:15 smithi043 ceph-mon[29710]: 6.7 scrub starts 2024-09-21T11:46:15.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:15 smithi043 ceph-mon[29710]: 6.7 scrub ok 2024-09-21T11:46:15.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:15 smithi043 ceph-mon[29710]: 8.7 scrub starts 2024-09-21T11:46:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:15 smithi043 ceph-mon[36454]: pgmap v294: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 638 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-21T11:46:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:15 smithi043 ceph-mon[36454]: 6.7 scrub starts 2024-09-21T11:46:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:15 smithi043 ceph-mon[36454]: 6.7 scrub ok 2024-09-21T11:46:15.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:15 smithi043 ceph-mon[36454]: 8.7 scrub starts 2024-09-21T11:46:15.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:15.615 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 73 16 116487 116471 99.6865 66.875 0.013723 0.0100267 2024-09-21T11:46:15.619 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:15.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:15.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:15.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:15 smithi175 ceph-mon[32613]: pgmap v294: 148 pgs: 148 active+clean; 6.8 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 638 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-21T11:46:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:15 smithi175 ceph-mon[32613]: 6.7 scrub starts 2024-09-21T11:46:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:15 smithi175 ceph-mon[32613]: 6.7 scrub ok 2024-09-21T11:46:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:15 smithi175 ceph-mon[32613]: 8.7 scrub starts 2024-09-21T11:46:15.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:15.769 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:15.870 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:15.898 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:15.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:15.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:16.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:16.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:16.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:16.144 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:16.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:16.190 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:16.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:16.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:16.332 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:16.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:16.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:16.435 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:16.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:16.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:16.470 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:16.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:46:16.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:46:16.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[36454]: 8.9 scrub starts 2024-09-21T11:46:16.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[36454]: 8.6 scrub ok 2024-09-21T11:46:16.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:16.570 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[36454]: 8.2 scrub starts 2024-09-21T11:46:16.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:46:16.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:46:16.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[29710]: 8.9 scrub starts 2024-09-21T11:46:16.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[29710]: 8.6 scrub ok 2024-09-21T11:46:16.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:16.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:16 smithi043 ceph-mon[29710]: 8.2 scrub starts 2024-09-21T11:46:16.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:16.601 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:16.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:16.602 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:16.613 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 74 16 117506 117490 99.2001 63.6875 0.00775253 0.0100756 2024-09-21T11:46:16.702 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:16.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:16.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:16 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:46:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:16 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:46:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:16 smithi175 ceph-mon[32613]: 8.9 scrub starts 2024-09-21T11:46:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:16 smithi175 ceph-mon[32613]: 8.6 scrub ok 2024-09-21T11:46:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:16 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:16 smithi175 ceph-mon[32613]: 8.2 scrub starts 2024-09-21T11:46:16.805 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:46:16.805 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:46:16.805 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:46:16.805 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:46:16.805 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:46:16.805 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:46:16.805 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:46:16.805 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:46:16.805 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:46:16.806 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:46:16.806 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:46:16.806 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:46:16.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:16.888 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:16.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:16.889 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:16.989 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:17.042 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:17.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:17.043 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:17.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:17.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:17.196 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:17.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:17.330 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:17.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:17.330 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:46:17.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:17.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:17.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[29710]: 6.5 scrub starts 2024-09-21T11:46:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[29710]: 6.5 scrub ok 2024-09-21T11:46:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[29710]: pgmap v295: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 939 op/s 2024-09-21T11:46:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[29710]: osdmap e193: 8 total, 8 up, 7 in 2024-09-21T11:46:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/8631439' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[29710]: 5.b scrub starts 2024-09-21T11:46:17.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[29710]: 5.b scrub ok 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2221856448' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[29710]: 8.c scrub starts 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[36454]: 6.5 scrub starts 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[36454]: 6.5 scrub ok 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[36454]: pgmap v295: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 939 op/s 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[36454]: osdmap e193: 8 total, 8 up, 7 in 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/8631439' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[36454]: 5.b scrub starts 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[36454]: 5.b scrub ok 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2221856448' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:17.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:17 smithi043 ceph-mon[36454]: 8.c scrub starts 2024-09-21T11:46:17.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:17.605 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:17.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:17.606 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:17.613 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 75 16 118507 118491 98.7116 62.5625 0.0101125 0.0101258 2024-09-21T11:46:17.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:17.749 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:17.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:17.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:17 smithi175 ceph-mon[32613]: 6.5 scrub starts 2024-09-21T11:46:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:17 smithi175 ceph-mon[32613]: 6.5 scrub ok 2024-09-21T11:46:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:17 smithi175 ceph-mon[32613]: pgmap v295: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 6.9 GiB data, 22 GiB used, 603 GiB / 626 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 939 op/s 2024-09-21T11:46:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:17 smithi175 ceph-mon[32613]: osdmap e193: 8 total, 8 up, 7 in 2024-09-21T11:46:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:17 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/8631439' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:17 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:17 smithi175 ceph-mon[32613]: 5.b scrub starts 2024-09-21T11:46:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:17 smithi175 ceph-mon[32613]: 5.b scrub ok 2024-09-21T11:46:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:17 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2221856448' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:17 smithi175 ceph-mon[32613]: 8.c scrub starts 2024-09-21T11:46:17.785 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:46:17.786 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:46:17.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.2 'trim stale osdmaps' 2024-09-21T11:46:17.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:17.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:17.937 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:17.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:18.038 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:18.065 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:46:18.065 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:46:18.066 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:46:18.066 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:46:18.066 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:46:18.066 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:46:18.066 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:46:18.066 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:46:18.066 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:46:18.067 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:46:18.067 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:46:18.067 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:46:18.067 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:46:18.067 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:46:18.067 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:46:18.067 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:46:18.083 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:18.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:18.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:18.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:18.224 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:18.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:18.225 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:18.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:18.383 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:46:18.383 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:18.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:18.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:18.485 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:18.517 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:18.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:18.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:18.613 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 76 16 119730 119714 98.4185 76.4375 0.0157286 0.0101562 2024-09-21T11:46:18.618 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:18.663 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:18.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:18.664 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:46:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: 8.f scrub ok 2024-09-21T11:46:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:46:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:46:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: 8.2 scrub ok 2024-09-21T11:46:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: osdmap e194: 8 total, 8 up, 7 in 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: 8.9 scrub ok 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: 5.1d scrub starts 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: 5.1d scrub ok 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: 5.8 scrub starts 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[29710]: 5.8 scrub ok 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: 8.f scrub ok 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: 8.2 scrub ok 2024-09-21T11:46:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:18.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: osdmap e194: 8 total, 8 up, 7 in 2024-09-21T11:46:18.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: 8.9 scrub ok 2024-09-21T11:46:18.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: 5.1d scrub starts 2024-09-21T11:46:18.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: 5.1d scrub ok 2024-09-21T11:46:18.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: 5.8 scrub starts 2024-09-21T11:46:18.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:18 smithi043 ceph-mon[36454]: 5.8 scrub ok 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: 8.f scrub ok 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: 8.2 scrub ok 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: osdmap e194: 8 total, 8 up, 7 in 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: 8.9 scrub ok 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: 5.1d scrub starts 2024-09-21T11:46:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: 5.1d scrub ok 2024-09-21T11:46:18.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: 5.8 scrub starts 2024-09-21T11:46:18.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:18 smithi175 ceph-mon[32613]: 5.8 scrub ok 2024-09-21T11:46:18.765 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:18.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:18.807 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:18.905 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:18.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:18.942 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:18.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:18.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:19.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:19.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:19.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:19.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:19.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:19.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:19.331 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:19.366 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:19.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:19.367 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:19.467 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:19.505 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:19.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:19.506 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:19.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:19.613 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 77 16 121009 120993 98.1784 79.9375 0.014302 0.0101811 2024-09-21T11:46:19.642 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:19.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:19.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:19.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:19 smithi175 ceph-mon[32613]: 3.a scrub starts 2024-09-21T11:46:19.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:19 smithi175 ceph-mon[32613]: 3.a scrub ok 2024-09-21T11:46:19.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:19 smithi175 ceph-mon[32613]: 6.15 scrub starts 2024-09-21T11:46:19.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:19 smithi175 ceph-mon[32613]: 6.15 scrub ok 2024-09-21T11:46:19.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:19 smithi175 ceph-mon[32613]: pgmap v298: 148 pgs: 4 active+clean+scrubbing, 144 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 782 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-21T11:46:19.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:19 smithi175 ceph-mon[32613]: 8.7 scrub ok 2024-09-21T11:46:19.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:19 smithi175 ceph-mon[32613]: 8.e scrub starts 2024-09-21T11:46:19.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[36454]: 3.a scrub starts 2024-09-21T11:46:19.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[36454]: 3.a scrub ok 2024-09-21T11:46:19.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[36454]: 6.15 scrub starts 2024-09-21T11:46:19.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[36454]: 6.15 scrub ok 2024-09-21T11:46:19.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[36454]: pgmap v298: 148 pgs: 4 active+clean+scrubbing, 144 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 782 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-21T11:46:19.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[36454]: 8.7 scrub ok 2024-09-21T11:46:19.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[36454]: 8.e scrub starts 2024-09-21T11:46:19.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[29710]: 3.a scrub starts 2024-09-21T11:46:19.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[29710]: 3.a scrub ok 2024-09-21T11:46:19.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[29710]: 6.15 scrub starts 2024-09-21T11:46:19.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[29710]: 6.15 scrub ok 2024-09-21T11:46:19.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[29710]: pgmap v298: 148 pgs: 4 active+clean+scrubbing, 144 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 782 B/s rd, 69 MiB/s wr, 1.10k op/s 2024-09-21T11:46:19.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[29710]: 8.7 scrub ok 2024-09-21T11:46:19.671 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:19 smithi043 ceph-mon[29710]: 8.e scrub starts 2024-09-21T11:46:19.688 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:19.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:19.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:19.795 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:19.896 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:19.933 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:19.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:19.934 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:20.035 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:20.069 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:20.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:20.070 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:20.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:20.206 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:20.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:20.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:20.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:20.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:20.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:20.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:20.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:20.485 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:20.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:20.613 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 78 16 122242 122226 97.9077 77.0625 0.0199438 0.010209 2024-09-21T11:46:20.621 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:20.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:20.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:20.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:20 smithi043 ceph-mon[36454]: 5.1b scrub starts 2024-09-21T11:46:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:20 smithi043 ceph-mon[36454]: 5.1b scrub ok 2024-09-21T11:46:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:20 smithi043 ceph-mon[36454]: 8.c scrub ok 2024-09-21T11:46:20.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:20 smithi043 ceph-mon[36454]: 4.9 scrub starts 2024-09-21T11:46:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:20 smithi043 ceph-mon[36454]: 4.9 scrub ok 2024-09-21T11:46:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:20 smithi043 ceph-mon[29710]: 5.1b scrub starts 2024-09-21T11:46:20.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:20 smithi043 ceph-mon[29710]: 5.1b scrub ok 2024-09-21T11:46:20.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:20 smithi043 ceph-mon[29710]: 8.c scrub ok 2024-09-21T11:46:20.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:20 smithi043 ceph-mon[29710]: 4.9 scrub starts 2024-09-21T11:46:20.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:20 smithi043 ceph-mon[29710]: 4.9 scrub ok 2024-09-21T11:46:20.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:20 smithi175 ceph-mon[32613]: 5.1b scrub starts 2024-09-21T11:46:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:20 smithi175 ceph-mon[32613]: 5.1b scrub ok 2024-09-21T11:46:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:20 smithi175 ceph-mon[32613]: 8.c scrub ok 2024-09-21T11:46:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:20 smithi175 ceph-mon[32613]: 4.9 scrub starts 2024-09-21T11:46:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:20 smithi175 ceph-mon[32613]: 4.9 scrub ok 2024-09-21T11:46:20.763 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:20.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:20.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:20.839 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:46:20.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:20.938 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:20.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:20.939 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:21.040 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:21.068 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:21.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:21.069 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:21.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:21.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:21.208 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:21.208 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:21.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:21.345 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:21.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:21.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:21.415 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:46:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:46:21.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:21.487 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:21.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:21.488 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:21.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:46:21.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:21.614 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 79 16 123405 123389 97.5879 72.6875 0.00550665 0.0102426 2024-09-21T11:46:21.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:21.659 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:21.659 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:21.695 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[36454]: 5.4 scrub starts 2024-09-21T11:46:21.696 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[36454]: 5.4 scrub ok 2024-09-21T11:46:21.696 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[36454]: pgmap v299: 148 pgs: 4 active+clean+scrubbing, 144 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 639 B/s rd, 43 MiB/s wr, 696 op/s 2024-09-21T11:46:21.696 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[36454]: 4.a scrub starts 2024-09-21T11:46:21.696 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[36454]: 4.a scrub ok 2024-09-21T11:46:21.696 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[36454]: 6.f scrub starts 2024-09-21T11:46:21.696 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[36454]: 6.f scrub ok 2024-09-21T11:46:21.696 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[29710]: 5.4 scrub starts 2024-09-21T11:46:21.696 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[29710]: 5.4 scrub ok 2024-09-21T11:46:21.696 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[29710]: pgmap v299: 148 pgs: 4 active+clean+scrubbing, 144 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 639 B/s rd, 43 MiB/s wr, 696 op/s 2024-09-21T11:46:21.696 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[29710]: 4.a scrub starts 2024-09-21T11:46:21.697 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[29710]: 4.a scrub ok 2024-09-21T11:46:21.697 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[29710]: 6.f scrub starts 2024-09-21T11:46:21.697 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:21 smithi043 ceph-mon[29710]: 6.f scrub ok 2024-09-21T11:46:21.760 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:21 smithi175 ceph-mon[32613]: 5.4 scrub starts 2024-09-21T11:46:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:21 smithi175 ceph-mon[32613]: 5.4 scrub ok 2024-09-21T11:46:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:21 smithi175 ceph-mon[32613]: pgmap v299: 148 pgs: 4 active+clean+scrubbing, 144 active+clean; 7.1 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 639 B/s rd, 43 MiB/s wr, 696 op/s 2024-09-21T11:46:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:21 smithi175 ceph-mon[32613]: 4.a scrub starts 2024-09-21T11:46:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:21 smithi175 ceph-mon[32613]: 4.a scrub ok 2024-09-21T11:46:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:21 smithi175 ceph-mon[32613]: 6.f scrub starts 2024-09-21T11:46:21.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:21 smithi175 ceph-mon[32613]: 6.f scrub ok 2024-09-21T11:46:21.785 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:21.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:21.786 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:21.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:21.910 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:21.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:21.911 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:22.012 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:22.047 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:22.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:22.048 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:22.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:22.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:22.189 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:22.189 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:22.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:22.326 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:22.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:22.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:22.427 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:22.457 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:22.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:22.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:22.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:22.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:22.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:22.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:22.614 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:46:22.613360+0000 min lat: 0.00112969 max lat: 0.147407 avg lat: 0.0102775 2024-09-21T11:46:22.614 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:46:22.614 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 80 16 124545 124529 97.2586 71.25 0.011733 0.0102775 2024-09-21T11:46:22.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:22.736 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:22.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:22.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:22 smithi043 ceph-mon[36454]: 8.e scrub ok 2024-09-21T11:46:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:22 smithi043 ceph-mon[29710]: 8.e scrub ok 2024-09-21T11:46:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:22 smithi175 ceph-mon[32613]: 8.e scrub ok 2024-09-21T11:46:22.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:22.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:22.869 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:22.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:22.970 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:22.991 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:22.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:22.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:23.093 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:23.127 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:23.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:23.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:23.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:23.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:23.263 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:23.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:23.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:23.403 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:23.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:23.404 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:23.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:23.539 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:23.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:23.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:23.614 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 81 16 125649 125633 96.9097 69 0.00987246 0.0103145 2024-09-21T11:46:23.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:23.679 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:23.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:23.680 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[29710]: 3.e scrub starts 2024-09-21T11:46:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[29710]: 3.e scrub ok 2024-09-21T11:46:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[29710]: pgmap v300: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.5 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-21T11:46:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[29710]: 5.1f scrub starts 2024-09-21T11:46:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[29710]: 5.1f scrub ok 2024-09-21T11:46:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[29710]: 4.f scrub starts 2024-09-21T11:46:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[29710]: 4.f scrub ok 2024-09-21T11:46:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[29710]: 3.f scrub starts 2024-09-21T11:46:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[29710]: 3.f scrub ok 2024-09-21T11:46:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[36454]: 3.e scrub starts 2024-09-21T11:46:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[36454]: 3.e scrub ok 2024-09-21T11:46:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[36454]: pgmap v300: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.5 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-21T11:46:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[36454]: 5.1f scrub starts 2024-09-21T11:46:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[36454]: 5.1f scrub ok 2024-09-21T11:46:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[36454]: 4.f scrub starts 2024-09-21T11:46:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[36454]: 4.f scrub ok 2024-09-21T11:46:23.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[36454]: 3.f scrub starts 2024-09-21T11:46:23.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:23 smithi043 ceph-mon[36454]: 3.f scrub ok 2024-09-21T11:46:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:23 smithi175 ceph-mon[32613]: 3.e scrub starts 2024-09-21T11:46:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:23 smithi175 ceph-mon[32613]: 3.e scrub ok 2024-09-21T11:46:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:23 smithi175 ceph-mon[32613]: pgmap v300: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.5 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-21T11:46:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:23 smithi175 ceph-mon[32613]: 5.1f scrub starts 2024-09-21T11:46:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:23 smithi175 ceph-mon[32613]: 5.1f scrub ok 2024-09-21T11:46:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:23 smithi175 ceph-mon[32613]: 4.f scrub starts 2024-09-21T11:46:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:23 smithi175 ceph-mon[32613]: 4.f scrub ok 2024-09-21T11:46:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:23 smithi175 ceph-mon[32613]: 3.f scrub starts 2024-09-21T11:46:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:23 smithi175 ceph-mon[32613]: 3.f scrub ok 2024-09-21T11:46:23.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:23.817 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:23.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:23.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:23.897 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:46:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:46:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.897 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:124872 8.f e194/160 8:f43f0681:::benchmark_data_smithi043_97042_object124871:head v 194'7760, mlcod=194'7756)", 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:22.945077+0000", 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0029273250000000002, 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002953724, 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945077+0000", 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945077+0000", 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945078+0000", 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0070000000000001e-06 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945103+0000", 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4725000000000001e-05 2024-09-21T11:46:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945114+0000", 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1234e-05 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945121+0000", 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8600000000000004e-06 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945175+0000", 2024-09-21T11:46:23.900 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3844000000000002e-05 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945194+0000", 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8584999999999999e-05 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:124873 8.1 e194/160 8:8826b5b6:::benchmark_data_smithi043_97042_object124872:head v 194'7947, mlcod=194'7943)", 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:22.945245+0000", 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0027599629999999998, 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028455479999999998, 2024-09-21T11:46:23.901 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945245+0000", 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945245+0000", 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.902 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945245+0000", 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3000000000000002e-07 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945273+0000", 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7877000000000001e-05 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945280+0000", 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.64e-06 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945284+0000", 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3290000000000004e-06 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945553+0000", 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026921900000000001 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945570+0000", 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6704000000000001e-05 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:23.904 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:124881 8.a 8:527a5f21:::benchmark_data_smithi043_97042_object124880:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:22.945948+0000", 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0020572699999999999, 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022035079999999999, 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 124881 2024-09-21T11:46:23.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945948+0000", 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945948+0000", 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945949+0000", 2024-09-21T11:46:23.906 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.302e-06 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945970+0000", 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1427999999999999e-05 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945980+0000", 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0188e-05 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.945988+0000", 2024-09-21T11:46:23.907 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3599999999999998e-06 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.946053+0000", 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5140000000000003e-05 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.946181+0000", 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012768100000000001 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:22.946331+0000", 2024-09-21T11:46:23.908 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000150586 2024-09-21T11:46:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:46:23.909 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 3 2024-09-21T11:46:23.909 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:46:23.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:23.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:23.959 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:23.959 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:24.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:24.096 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:24.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:24.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:24.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:24.233 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:24.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:24.234 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:24.335 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:24.367 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:24.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:24.368 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:24.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:24.510 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:24.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:24.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:24.611 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:24.614 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 82 16 126986 126970 96.7469 83.5625 0.00963781 0.010332 2024-09-21T11:46:24.648 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:24.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:24.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:24.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:24 smithi043 ceph-mon[36454]: 3.12 scrub starts 2024-09-21T11:46:24.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:24 smithi043 ceph-mon[36454]: 3.12 scrub ok 2024-09-21T11:46:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:24 smithi043 ceph-mon[29710]: 3.12 scrub starts 2024-09-21T11:46:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:24 smithi043 ceph-mon[29710]: 3.12 scrub ok 2024-09-21T11:46:24.749 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:24 smithi175 ceph-mon[32613]: 3.12 scrub starts 2024-09-21T11:46:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:24 smithi175 ceph-mon[32613]: 3.12 scrub ok 2024-09-21T11:46:24.789 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:24.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:24.790 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:24.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:24.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:24.927 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:24.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:25.027 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:25.064 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:25.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:25.065 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:25.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:25.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:25.205 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:25.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:25.344 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:25.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:25.345 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:25.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:25.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:25.484 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:25.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:46:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:46:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-mon[36454]: pgmap v301: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.5 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 639 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-21T11:46:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-mon[36454]: 4.15 scrub starts 2024-09-21T11:46:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-mon[36454]: 4.15 scrub ok 2024-09-21T11:46:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-mon[36454]: 6.2 scrub starts 2024-09-21T11:46:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-mon[36454]: 6.2 scrub ok 2024-09-21T11:46:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-mon[29710]: pgmap v301: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.5 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 639 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-21T11:46:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-mon[29710]: 4.15 scrub starts 2024-09-21T11:46:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-mon[29710]: 4.15 scrub ok 2024-09-21T11:46:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-mon[29710]: 6.2 scrub starts 2024-09-21T11:46:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:25 smithi043 ceph-mon[29710]: 6.2 scrub ok 2024-09-21T11:46:25.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:25.614 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 83 16 128333 128317 96.5954 84.1875 0.0089739 0.010348 2024-09-21T11:46:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:25.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:25.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:25.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:25 smithi175 ceph-mon[32613]: pgmap v301: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.5 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 639 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-21T11:46:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:25 smithi175 ceph-mon[32613]: 4.15 scrub starts 2024-09-21T11:46:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:25 smithi175 ceph-mon[32613]: 4.15 scrub ok 2024-09-21T11:46:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:25 smithi175 ceph-mon[32613]: 6.2 scrub starts 2024-09-21T11:46:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:25 smithi175 ceph-mon[32613]: 6.2 scrub ok 2024-09-21T11:46:25.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:25.763 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:25.763 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:25.864 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:25.908 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:25.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:25.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:26.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:26.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:26.037 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:26.037 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:26.138 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:26.170 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:26.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:26.171 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:26.271 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:26.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:26.312 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:26.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:26.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:26.437 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:26.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:26.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:26.539 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:26.582 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:26.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:26.583 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:26.615 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 84 16 129477 129461 96.2967 71.5 0.00518803 0.0103793 2024-09-21T11:46:26.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:26.722 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:26.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:26.724 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:26 smithi043 ceph-mon[36454]: 8.5 scrub starts 2024-09-21T11:46:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:26 smithi043 ceph-mon[36454]: 3.15 scrub starts 2024-09-21T11:46:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:26 smithi043 ceph-mon[36454]: 3.15 scrub ok 2024-09-21T11:46:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:26 smithi043 ceph-mon[29710]: 8.5 scrub starts 2024-09-21T11:46:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:26 smithi043 ceph-mon[29710]: 3.15 scrub starts 2024-09-21T11:46:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:26 smithi043 ceph-mon[29710]: 3.15 scrub ok 2024-09-21T11:46:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:26 smithi175 ceph-mon[32613]: 8.5 scrub starts 2024-09-21T11:46:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:26 smithi175 ceph-mon[32613]: 3.15 scrub starts 2024-09-21T11:46:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:26 smithi175 ceph-mon[32613]: 3.15 scrub ok 2024-09-21T11:46:26.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:26.866 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:26.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:26.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:26.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:27.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:27.004 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:27.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:27.009 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:46:27.009 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:46:27.009 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:46:27.009 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:46:27.009 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:46:27.009 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:46:27.009 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:46:27.009 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:46:27.009 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:46:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:46:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:46:27.010 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:46:27.105 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:27.142 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:27.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:27.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:27.245 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:27.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:27.288 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:27.288 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:27.389 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:27.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:27.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:27.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:27.529 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:27.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:27.562 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:27.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:27.615 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 85 16 130656 130640 96.0306 73.6875 0.00645697 0.0104083 2024-09-21T11:46:27.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:27.697 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:27.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:27.699 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:27 smithi043 ceph-mon[29710]: pgmap v302: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.5 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 50 MiB/s wr, 793 op/s 2024-09-21T11:46:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:27 smithi043 ceph-mon[29710]: 4.12 scrub starts 2024-09-21T11:46:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:27 smithi043 ceph-mon[29710]: 4.12 scrub ok 2024-09-21T11:46:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:27 smithi043 ceph-mon[36454]: pgmap v302: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.5 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 50 MiB/s wr, 793 op/s 2024-09-21T11:46:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:27 smithi043 ceph-mon[36454]: 4.12 scrub starts 2024-09-21T11:46:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:27 smithi043 ceph-mon[36454]: 4.12 scrub ok 2024-09-21T11:46:27.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:27 smithi175 ceph-mon[32613]: pgmap v302: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.5 GiB data, 24 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 50 MiB/s wr, 793 op/s 2024-09-21T11:46:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:27 smithi175 ceph-mon[32613]: 4.12 scrub starts 2024-09-21T11:46:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:27 smithi175 ceph-mon[32613]: 4.12 scrub ok 2024-09-21T11:46:27.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:27.839 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:27.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:27.839 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:27.926 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:46:27.926 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:46:27.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.3 'trim stale osdmaps' 2024-09-21T11:46:27.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:28.009 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:28.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:28.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:28.111 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:28.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:28.151 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:28.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:28.252 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:28.287 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:28.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:28.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:28.389 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:28.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:28.424 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:28.425 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:28.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:46:28.516 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:28.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:28.558 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:28.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:28.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:28.617 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 86 16 131955 131939 95.8555 81.1875 0.0172177 0.0104281 2024-09-21T11:46:28.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:28.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:28 smithi043 ceph-mon[36454]: 6.1d scrub starts 2024-09-21T11:46:28.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:28 smithi043 ceph-mon[36454]: 6.1d scrub ok 2024-09-21T11:46:28.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:28 smithi043 ceph-mon[36454]: 8.5 scrub ok 2024-09-21T11:46:28.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:28 smithi043 ceph-mon[36454]: 6.0 scrub starts 2024-09-21T11:46:28.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:28 smithi043 ceph-mon[36454]: 6.0 scrub ok 2024-09-21T11:46:28.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:28 smithi043 ceph-mon[29710]: 6.1d scrub starts 2024-09-21T11:46:28.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:28 smithi043 ceph-mon[29710]: 6.1d scrub ok 2024-09-21T11:46:28.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:28 smithi043 ceph-mon[29710]: 8.5 scrub ok 2024-09-21T11:46:28.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:28 smithi043 ceph-mon[29710]: 6.0 scrub starts 2024-09-21T11:46:28.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:28 smithi043 ceph-mon[29710]: 6.0 scrub ok 2024-09-21T11:46:28.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:28.730 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:28.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:28 smithi175 ceph-mon[32613]: 6.1d scrub starts 2024-09-21T11:46:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:28 smithi175 ceph-mon[32613]: 6.1d scrub ok 2024-09-21T11:46:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:28 smithi175 ceph-mon[32613]: 8.5 scrub ok 2024-09-21T11:46:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:28 smithi175 ceph-mon[32613]: 6.0 scrub starts 2024-09-21T11:46:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:28 smithi175 ceph-mon[32613]: 6.0 scrub ok 2024-09-21T11:46:28.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:28.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:28.871 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:28.871 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:28.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:29.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:29.014 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:29.015 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:29.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:29.154 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:29.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:29.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:29.233 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:29.256 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:29.282 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:29.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:29.283 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:29.384 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:29.416 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:29.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:29.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:29.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:29.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:29.576 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:29.576 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:29.617 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 87 15 133151 133136 95.6136 74.8125 0.0192745 0.0104541 2024-09-21T11:46:29.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:46:29.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:46:29.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[36454]: pgmap v303: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.9 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 977 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-21T11:46:29.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[36454]: osdmap e195: 8 total, 8 up, 7 in 2024-09-21T11:46:29.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[36454]: 4.17 scrub starts 2024-09-21T11:46:29.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[36454]: 4.17 scrub ok 2024-09-21T11:46:29.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[36454]: 4.16 scrub starts 2024-09-21T11:46:29.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[36454]: 4.16 scrub ok 2024-09-21T11:46:29.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:46:29.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:46:29.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[29710]: pgmap v303: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.9 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 977 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-21T11:46:29.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[29710]: osdmap e195: 8 total, 8 up, 7 in 2024-09-21T11:46:29.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[29710]: 4.17 scrub starts 2024-09-21T11:46:29.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[29710]: 4.17 scrub ok 2024-09-21T11:46:29.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[29710]: 4.16 scrub starts 2024-09-21T11:46:29.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:29 smithi043 ceph-mon[29710]: 4.16 scrub ok 2024-09-21T11:46:29.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:29.714 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:29.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:29.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:29 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:46:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:29 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:46:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:29 smithi175 ceph-mon[32613]: pgmap v303: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.9 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 977 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-21T11:46:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:29 smithi175 ceph-mon[32613]: osdmap e195: 8 total, 8 up, 7 in 2024-09-21T11:46:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:29 smithi175 ceph-mon[32613]: 4.17 scrub starts 2024-09-21T11:46:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:29 smithi175 ceph-mon[32613]: 4.17 scrub ok 2024-09-21T11:46:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:29 smithi175 ceph-mon[32613]: 4.16 scrub starts 2024-09-21T11:46:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:29 smithi175 ceph-mon[32613]: 4.16 scrub ok 2024-09-21T11:46:29.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:29.847 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:29.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:29.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:29.949 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:29.966 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:29.979 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:29.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:29.980 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:30.051 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:46:30.051 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:46:30.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:30.086 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:46:30.086 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:46:30.127 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:46:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:46:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:30.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:30.138 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:30.176 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:46:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:46:30.211 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:46:30.211 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:46:30.239 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:30.249 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:46:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:46:30.276 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:30.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:30.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:30.291 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:46:30.291 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:46:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:46:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:46:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:46:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:46:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:46:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:46:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:46:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:46:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:46:30.311 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:46:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:46:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:46:30.325 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:46:30.325 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:46:30.379 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:30.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:30.416 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:30.416 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:46:30.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:30.553 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:30.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:30.554 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:30.617 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 88 16 134387 134371 95.4042 77.1875 0.0101655 0.0104775 2024-09-21T11:46:30.655 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:30.691 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:30.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:30.692 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:30.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2138190198' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:46:30.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:46:30.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[36454]: 6.b scrub starts 2024-09-21T11:46:30.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[36454]: 6.b scrub ok 2024-09-21T11:46:30.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[36454]: 2.0 scrub starts 2024-09-21T11:46:30.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[36454]: 2.0 scrub ok 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[36454]: 3.1 scrub starts 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[36454]: 3.1 scrub ok 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2799828729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2138190198' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[29710]: 6.b scrub starts 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[29710]: 6.b scrub ok 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[29710]: 2.0 scrub starts 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[29710]: 2.0 scrub ok 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[29710]: 3.1 scrub starts 2024-09-21T11:46:30.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[29710]: 3.1 scrub ok 2024-09-21T11:46:30.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:30 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2799828729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:30 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2138190198' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:46:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:30 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:46:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:30 smithi175 ceph-mon[32613]: 6.b scrub starts 2024-09-21T11:46:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:30 smithi175 ceph-mon[32613]: 6.b scrub ok 2024-09-21T11:46:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:30 smithi175 ceph-mon[32613]: 2.0 scrub starts 2024-09-21T11:46:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:30 smithi175 ceph-mon[32613]: 2.0 scrub ok 2024-09-21T11:46:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:30 smithi175 ceph-mon[32613]: 3.1 scrub starts 2024-09-21T11:46:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:30 smithi175 ceph-mon[32613]: 3.1 scrub ok 2024-09-21T11:46:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:30 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2799828729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:30.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:30.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:30.834 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:30.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:30.972 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:30.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:30.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:31.074 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:31.112 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:31.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:31.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:31.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:31.255 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:31.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:31.256 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:31.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:31.395 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:31.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:31.396 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:31.424 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:46:31.424 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:46:31.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.4 'trim stale osdmaps' 2024-09-21T11:46:31.472 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:46:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:46:31.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:31.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:31.546 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:31.547 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:31.620 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 89 16 135767 135751 95.2985 86.25 0.0153134 0.0104887 2024-09-21T11:46:31.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:31.676 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:31.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:31.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:31.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:31 smithi043 ceph-mon[29710]: pgmap v305: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.9 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:46:31.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:31 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:31.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:31 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:46:31.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:31 smithi043 ceph-mon[29710]: osdmap e196: 8 total, 8 up, 7 in 2024-09-21T11:46:31.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:31 smithi043 ceph-mon[36454]: pgmap v305: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.9 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:46:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:31 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:31 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:46:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:31 smithi043 ceph-mon[36454]: osdmap e196: 8 total, 8 up, 7 in 2024-09-21T11:46:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:31 smithi175 ceph-mon[32613]: pgmap v305: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.9 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1023 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:46:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:31 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:31 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:46:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:31 smithi175 ceph-mon[32613]: osdmap e196: 8 total, 8 up, 7 in 2024-09-21T11:46:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:31.778 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:31.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:31.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:31.843 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:46:31.843 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:46:31.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:31.961 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:31.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:31.962 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:32.063 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:32.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:32.114 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:32.215 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:32.254 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:32.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:32.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:32.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:32.393 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:32.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:32.394 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:32.495 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:32.529 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:32.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:32.530 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:32.620 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Total time run: 90.0108 2024-09-21T11:46:32.620 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Total writes made: 137091 2024-09-21T11:46:32.620 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Write size: 65536 2024-09-21T11:46:32.620 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Object size: 65536 2024-09-21T11:46:32.620 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Bandwidth (MB/sec): 95.1907 2024-09-21T11:46:32.620 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev Bandwidth: 32.7653 2024-09-21T11:46:32.620 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max bandwidth (MB/sec): 225.438 2024-09-21T11:46:32.620 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min bandwidth (MB/sec): 62.5625 2024-09-21T11:46:32.620 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Average IOPS: 1523 2024-09-21T11:46:32.621 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev IOPS: 524.245 2024-09-21T11:46:32.621 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max IOPS: 3607 2024-09-21T11:46:32.621 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min IOPS: 1001 2024-09-21T11:46:32.621 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Average Latency(s): 0.010501 2024-09-21T11:46:32.621 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev Latency(s): 0.00788935 2024-09-21T11:46:32.621 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max latency(s): 0.147407 2024-09-21T11:46:32.621 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min latency(s): 0.00112969 2024-09-21T11:46:32.627 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Cleaning up (deleting benchmark objects) 2024-09-21T11:46:32.630 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:32.663 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:32.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:32.664 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:32.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:32.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:32.765 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:32.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:32.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:32.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:32.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:32.939 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:33.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:33.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:33.086 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:33.187 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:33.222 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:33.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:33.223 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:33.323 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:33.359 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:33.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:33.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:33.461 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:33.495 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:33.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:33.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:33.596 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:33.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:46:33.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:33.630 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:33.630 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:33.731 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:33 smithi043 ceph-mon[36454]: pgmap v307: 148 pgs: 148 active+clean; 8.2 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-21T11:46:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:33 smithi043 ceph-mon[29710]: pgmap v307: 148 pgs: 148 active+clean; 8.2 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-21T11:46:33.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:33 smithi175 ceph-mon[32613]: pgmap v307: 148 pgs: 148 active+clean; 8.2 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-21T11:46:33.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:33.762 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:33.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:33.862 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:33.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:33.895 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:33.895 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:33.996 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:34.029 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:34.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:34.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:34.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:34.182 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:34.188 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:34.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:34.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:34.207 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:34.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:34.327 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:34.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:34.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:34.429 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:34.464 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:34.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:34.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:34.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:34.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:34.627 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:34.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:34.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:34.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:34.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:34.933 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:34.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:34.935 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:35.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:46:35.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:35.072 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:46:35.072 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:46:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:46:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:46:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:46:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:46:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:46:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:46:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:46:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:46:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:46:35.073 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:46:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:35.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:35.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:35.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:35.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:35.283 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:35.283 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:35.384 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:35.419 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:35.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:35.420 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:35.487 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:46:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:46:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:46:35.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:35.552 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:35.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:35.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:35.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:35.688 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:35.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:35.689 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:35.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:35 smithi043 ceph-mon[36454]: pgmap v308: 148 pgs: 148 active+clean; 8.2 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 639 B/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-21T11:46:35.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:35 smithi043 ceph-mon[29710]: pgmap v308: 148 pgs: 148 active+clean; 8.2 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 639 B/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-21T11:46:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:35 smithi175 ceph-mon[32613]: pgmap v308: 148 pgs: 148 active+clean; 8.2 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 639 B/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-21T11:46:35.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:35.826 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:35.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:35.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:35.915 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:46:35.916 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:46:35.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.5 'trim stale osdmaps' 2024-09-21T11:46:35.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:35.957 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:35.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:35.958 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:36.033 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:46:36.033 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:46:36.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:36.114 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:36.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:36.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:36.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:46:36.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:36.272 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:36.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:36.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:36.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:36.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:36.426 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:36.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:36.562 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:36.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:36.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:36.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:36.701 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:36.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:36.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:36.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:36.839 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:36.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:36.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:36.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:36.977 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:36.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:36.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:37.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:46:37.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:37.104 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:37.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:37.105 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:37.117 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:37.127 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:37.205 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:37.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:37.236 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:37.237 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:37.337 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:37.370 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:37.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:37.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:37.472 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:37.498 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:37.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:37.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:37.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:37.603 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:37 smithi043 ceph-mon[29710]: pgmap v309: 148 pgs: 148 active+clean; 8.2 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 826 op/s 2024-09-21T11:46:37.603 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:37 smithi043 ceph-mon[36454]: pgmap v309: 148 pgs: 148 active+clean; 8.2 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 826 op/s 2024-09-21T11:46:37.639 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:37.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:37.640 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:37.740 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:37 smithi175 ceph-mon[32613]: pgmap v309: 148 pgs: 148 active+clean; 8.2 GiB data, 26 GiB used, 599 GiB / 626 GiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 826 op/s 2024-09-21T11:46:37.772 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:37.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:37.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:37.874 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:37.904 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:37.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:37.905 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:38.006 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:38.040 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:38.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:38.043 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:38.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:38.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:38.180 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:38.180 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:38.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:38.309 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:38.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:38.310 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:38.411 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:38.426 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:46:38.426 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:46:38.426 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:46:38.426 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:46:38.426 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:46:38.426 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:46:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:46:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:46:38.444 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:38.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:38.445 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:38.462 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:46:38.462 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:46:38.500 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:46:38.501 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:46:38.507 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:46:38.539 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:46:38.539 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:46:38.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:38.577 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:38.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:38.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:38.581 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:46:38.581 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:46:38.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:38.716 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:38.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:38.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:38 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/162601512' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:38.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:38 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/844953742' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:38.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:38 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:38 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/162601512' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:38 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/844953742' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:38 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:38 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/162601512' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:38 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/844953742' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:38 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:38.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:38.846 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:46:38.846 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:46:38.846 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:46:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:46:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:46:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:46:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:46:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:46:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:46:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:46:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:46:38.847 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:46:38.859 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:38.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:38.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:38.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:38.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:38.999 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:38.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:39.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:39.152 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:39.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:39.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:39.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:39.284 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:39.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:39.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:39.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:39.422 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:39.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:39.423 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:39.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:39.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:39.558 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:39.558 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:39.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:39.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:39.699 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:39.699 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:39 smithi175 ceph-mon[32613]: pgmap v310: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 1.40k op/s 2024-09-21T11:46:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:39 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:39 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:39 smithi175 ceph-mon[32613]: osdmap e197: 8 total, 8 up, 7 in 2024-09-21T11:46:39.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:39.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:39.839 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:39.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:39.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:39 smithi043 ceph-mon[36454]: pgmap v310: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 1.40k op/s 2024-09-21T11:46:39.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:39 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:39.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:39 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:39.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:39 smithi043 ceph-mon[36454]: osdmap e197: 8 total, 8 up, 7 in 2024-09-21T11:46:39.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:39 smithi043 ceph-mon[29710]: pgmap v310: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 1.40k op/s 2024-09-21T11:46:39.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:39 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:39.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:39 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:39.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:39 smithi043 ceph-mon[29710]: osdmap e197: 8 total, 8 up, 7 in 2024-09-21T11:46:39.980 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:39.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:39.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:40.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:40.116 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:40.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:40.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:40.218 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:40.253 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:40.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:40.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:40.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:40.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:40.389 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:40.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:40.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:40.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:40.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:40.628 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:40.661 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:40.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:40.662 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:40 smithi043 ceph-mon[36454]: pgmap v312: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 1.40k op/s 2024-09-21T11:46:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:40 smithi043 ceph-mon[29710]: pgmap v312: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 1.40k op/s 2024-09-21T11:46:40.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:40.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:40.799 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:40.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:40.837 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:46:40.838 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:46:40.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.6 'trim stale osdmaps' 2024-09-21T11:46:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:40.900 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:40.935 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:40.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:40.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:40 smithi175 ceph-mon[32613]: pgmap v312: 148 pgs: 148 active+clean; 8.1 GiB data, 26 GiB used, 600 GiB / 626 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 1.40k op/s 2024-09-21T11:46:41.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:41.076 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:41.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:41.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:41.177 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:41.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:41.207 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:41.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:41.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:41.344 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:41.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:41.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:41.446 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:41.479 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:41.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:41.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:41.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:46:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:46:41.560 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:149722 8.0 8:0e797ccd:::benchmark_data_smithi043_97042_object12628:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e197)", 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:41.546189+0000", 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010199685, 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010229685000000001, 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:41.561 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 149722 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.546189+0000", 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.546189+0000", 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:41.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.546191+0000", 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.546197+0000", 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4679999999999997e-06 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.546211+0000", 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3711e-05 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.546219+0000", 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9540000000000004e-06 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.546329+0000", 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010956200000000001 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.546444+0000", 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011509400000000001 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,5", 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.546554+0000", 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109793 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.549832+0000", 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032785100000000001 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.550675+0000", 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00084298100000000003 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:41.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:149728 8.a e197/160 8:55c4e442:::benchmark_data_smithi043_97042_object12634:head v 197'9273, mlcod=197'9271)", 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:41.549697+0000", 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0066919630000000004, 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0068013960000000004, 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.549697+0000", 2024-09-21T11:46:41.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.549697+0000", 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.549698+0000", 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.24e-06 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.549705+0000", 2024-09-21T11:46:41.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759999999999998e-06 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.549715+0000", 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.995e-06 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.549721+0000", 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6159999999999998e-06 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:41.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552417+0000", 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026958640000000001 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552622+0000", 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020509599999999999 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:149724 8.0 8:0c7b262d:::benchmark_data_smithi043_97042_object12630:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e197)", 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:41.552358+0000", 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0040315469999999999, 2024-09-21T11:46:41.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041963039999999997, 2024-09-21T11:46:41.570 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:41.570 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:41.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:46:41.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:41.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:41.570 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:41.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 149724 2024-09-21T11:46:41.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.570 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:41.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:41.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552358+0000", 2024-09-21T11:46:41.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:41.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552358+0000", 2024-09-21T11:46:41.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:41.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552359+0000", 2024-09-21T11:46:41.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6249999999999999e-06 2024-09-21T11:46:41.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:41.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552367+0000", 2024-09-21T11:46:41.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9200000000000004e-06 2024-09-21T11:46:41.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:41.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552388+0000", 2024-09-21T11:46:41.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0579999999999999e-05 2024-09-21T11:46:41.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:41.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552398+0000", 2024-09-21T11:46:41.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0061000000000001e-05 2024-09-21T11:46:41.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:41.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552486+0000", 2024-09-21T11:46:41.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8344000000000007e-05 2024-09-21T11:46:41.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:41.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552615+0000", 2024-09-21T11:46:41.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012840099999999999 2024-09-21T11:46:41.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,5", 2024-09-21T11:46:41.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552743+0000", 2024-09-21T11:46:41.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000128677 2024-09-21T11:46:41.576 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.576 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:41.576 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.577 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:149729 8.0 8:002be4b2:::benchmark_data_smithi043_97042_object12635:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e197)", 2024-09-21T11:46:41.577 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:41.552423+0000", 2024-09-21T11:46:41.577 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0039659439999999999, 2024-09-21T11:46:41.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041966929999999996, 2024-09-21T11:46:41.577 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 149729 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552423+0000", 2024-09-21T11:46:41.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552423+0000", 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552424+0000", 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9100000000000004e-07 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552427+0000", 2024-09-21T11:46:41.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5619999999999999e-06 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552433+0000", 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9580000000000004e-06 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552439+0000", 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4670000000000003e-06 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555465+0000", 2024-09-21T11:46:41.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030263579999999998 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555578+0000", 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011317200000000001 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:149730 8.f 8:f58a42c3:::benchmark_data_smithi043_97042_object12636:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e197)", 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:41.552446+0000", 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0039430209999999997, 2024-09-21T11:46:41.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042325870000000003, 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 149730 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552446+0000", 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552446+0000", 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552447+0000", 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6699999999999999e-07 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552449+0000", 2024-09-21T11:46:41.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.018e-06 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552462+0000", 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3828e-05 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552471+0000", 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3720000000000005e-06 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552544+0000", 2024-09-21T11:46:41.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3662e-05 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552651+0000", 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106984 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552757+0000", 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010577999999999999 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:41.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555500+0000", 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027424609999999999 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.556661+0000", 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011618679999999999 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:149727 8.8 e197/160 8:176d4c30:::benchmark_data_smithi043_97042_object12633:head v 197'9404, mlcod=197'9400)", 2024-09-21T11:46:41.586 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:41.552512+0000", 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0038770950000000001, 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042411769999999996, 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552512+0000", 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552512+0000", 2024-09-21T11:46:41.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552513+0000", 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0800000000000004e-07 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552519+0000", 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1449999999999996e-06 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552527+0000", 2024-09-21T11:46:41.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7270000000000007e-06 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552532+0000", 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.101e-06 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555397+0000", 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028651409999999999 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:41.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555417+0000", 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0378e-05 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:149722 8.0 e197/160)", 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:41.552541+0000", 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.003847764, 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042589419999999999, 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:41.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552541+0000", 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552541+0000", 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552542+0000", 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1800000000000002e-07 2024-09-21T11:46:41.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552544+0000", 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4399999999999999e-06 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552549+0000", 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6770000000000003e-06 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.552553+0000", 2024-09-21T11:46:41.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1540000000000004e-06 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:149735 8.a e197/160 8:55c4c08d:::benchmark_data_smithi043_97042_object12641:head v 197'9274, mlcod=197'9271)", 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:41.554959+0000", 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0014297940000000001, 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018849489999999999, 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.554959+0000", 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.554959+0000", 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.554960+0000", 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.23e-07 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.554966+0000", 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5509999999999999e-06 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.554981+0000", 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5305e-05 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.554989+0000", 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7919999999999993e-06 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555743+0000", 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000754516 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555762+0000", 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8185999999999998e-05 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.596 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:149724 8.0 e197/160)", 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:46:41.555363+0000", 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001026484, 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015259990000000001, 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555363+0000", 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:41.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555363+0000", 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555363+0000", 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2399999999999997e-07 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555368+0000", 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5750000000000002e-06 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555378+0000", 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0049e-05 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:46:41.555385+0000", 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7179999999999999e-06 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 9 2024-09-21T11:46:41.599 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:46:41.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:41.600 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:41.646 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:41.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:41.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:41.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:41 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:41 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:41.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:41.783 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:41.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:41.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:41.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:41.921 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:41.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:41.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:41 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:42.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:46:42.023 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:42.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:42.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:42.158 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:42.191 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:42.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:42.192 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:42.293 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:42.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:42.319 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:42.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:42.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:46:42.420 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:42.453 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:42.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:42.454 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:42.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:42.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:42.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:42.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:42.696 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:42.731 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:42.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:42.732 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:42.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:42.871 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:42.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:42.872 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:42.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:42.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:42.996 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:42.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:43.097 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:43.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:43.129 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:43.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:43.264 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:43.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:43.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:43.366 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:43.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:43.398 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:43.399 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:43.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:43 smithi043 ceph-mon[36454]: pgmap v313: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-09-21T11:46:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:43 smithi043 ceph-mon[29710]: pgmap v313: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-09-21T11:46:43.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:43.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:43 smithi175 ceph-mon[32613]: pgmap v313: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-09-21T11:46:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:43.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:43.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:43.639 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:43.678 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:43.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:43.679 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:43.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:46:43.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:43.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:43.864 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:43.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:43.965 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:44.001 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:44.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:44.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:44.103 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:44.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:44.135 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:44.136 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:44.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:44.272 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:44.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:44.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:44.374 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:44.407 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:44.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:44.408 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:44.446 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:44.454 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:46:44.454 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:46:44.454 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:46:44.455 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:46:44.455 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:46:44.455 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:46:44.455 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:46:44.455 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:46:44.455 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:46:44.455 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:46:44.455 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:46:44.455 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:46:44.458 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:44.509 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:44.543 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:44.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:44.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:44.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:44.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:44.690 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:44.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:44.704 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:44.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:44.831 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:44.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:44.832 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:44.933 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:44.971 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:44.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:44.972 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:45.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:45.106 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:45.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:45.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:45.208 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:45.252 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:45.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:45.253 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:45.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:45.390 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:45.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:45.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:45.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:45 smithi043 ceph-mon[29710]: pgmap v314: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-09-21T11:46:45.487 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:46:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:46:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:46:45.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:45 smithi043 ceph-mon[36454]: pgmap v314: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-09-21T11:46:45.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:45.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:45 smithi175 ceph-mon[32613]: pgmap v314: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 12 MiB/s wr, 1.41k op/s 2024-09-21T11:46:45.522 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:45.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:45.523 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:45.541 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:46:45.542 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:46:45.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.7 'trim stale osdmaps' 2024-09-21T11:46:45.624 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:45.661 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:45.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:45.662 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:45.763 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:45.799 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:45.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:45.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:45.900 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:45.937 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:45.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:45.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:46.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:46.098 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:46.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:46.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:46.200 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:46.235 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:46.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:46.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:46.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:46.377 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:46.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:46.378 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:46.422 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:46.422 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:46.479 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:46.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:46:46.513 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:46.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:46.514 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:46.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:46.618 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:46:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:46:46.655 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:46.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:46.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:46.664 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:46:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:46:46.698 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:46:46.698 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:46:46.737 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:46:46.737 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:46:46.757 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:46.774 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:46:46.774 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:46:46.788 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:46.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:46.789 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:46.810 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:46:46.811 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:46:46.849 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:46:46.849 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:46:46.881 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:46:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:46:46.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:46.939 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:46.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:46.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:47.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:47.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:47.078 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:47.079 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:47.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:47.198 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:46:47.198 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:46:47.214 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:47.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:47.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:47.256 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:46:47.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:47 smithi043 ceph-mon[29710]: pgmap v315: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 11 MiB/s wr, 1.41k op/s 2024-09-21T11:46:47.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4085619085' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:47.283 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3914430638' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:47.284 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:47 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:47.284 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:47 smithi043 ceph-mon[36454]: pgmap v315: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 11 MiB/s wr, 1.41k op/s 2024-09-21T11:46:47.284 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4085619085' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:47.284 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3914430638' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:47.284 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:47 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:47.315 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:47.344 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:47.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:47.345 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:47.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:47.479 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:47.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:47.480 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:47 smithi175 ceph-mon[32613]: pgmap v315: 148 pgs: 148 active+clean; 7.6 GiB data, 25 GiB used, 601 GiB / 626 GiB avail; 1.1 KiB/s rd, 11 MiB/s wr, 1.41k op/s 2024-09-21T11:46:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4085619085' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3914430638' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:47 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:46:47.581 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:47.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:47.615 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:47.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:47.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:47.752 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:47.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:47.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:47.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:47.887 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:47.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:47.888 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:47.989 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:48.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:48.027 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:48.027 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:48.128 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:48.157 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:48.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:48.158 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:48.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:48.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:48.302 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:48.303 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:48.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:48.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:48.438 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:48 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:48 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:46:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:48 smithi043 ceph-mon[36454]: osdmap e198: 8 total, 8 up, 7 in 2024-09-21T11:46:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:48 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:48 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:46:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:48 smithi043 ceph-mon[29710]: osdmap e198: 8 total, 8 up, 7 in 2024-09-21T11:46:48.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:48.574 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:48.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:48.575 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:48.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:48.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:48.718 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:48.718 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:48 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:46:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:48 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:46:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:48 smithi175 ceph-mon[32613]: osdmap e198: 8 total, 8 up, 7 in 2024-09-21T11:46:48.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:48.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:48.864 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:48.964 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:49.020 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:49.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:49.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:49.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:49.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:49.161 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:49.261 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:49.296 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:49.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:49.297 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:49.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:49.440 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:49.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:49.443 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:49.544 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:49.593 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:49.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:49.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:49 smithi175 ceph-mon[32613]: 6.1c scrub starts 2024-09-21T11:46:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:49 smithi175 ceph-mon[32613]: 6.1c scrub ok 2024-09-21T11:46:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:49 smithi175 ceph-mon[32613]: pgmap v317: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.49k op/s 2024-09-21T11:46:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:49 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:46:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:49 smithi175 ceph-mon[32613]: 5.17 scrub starts 2024-09-21T11:46:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:49 smithi175 ceph-mon[32613]: 5.17 scrub ok 2024-09-21T11:46:49.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:49.734 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:49.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:49.735 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:49.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[36454]: 6.1c scrub starts 2024-09-21T11:46:49.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[36454]: 6.1c scrub ok 2024-09-21T11:46:49.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[36454]: pgmap v317: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.49k op/s 2024-09-21T11:46:49.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:46:49.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[36454]: 5.17 scrub starts 2024-09-21T11:46:49.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[36454]: 5.17 scrub ok 2024-09-21T11:46:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[29710]: 6.1c scrub starts 2024-09-21T11:46:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[29710]: 6.1c scrub ok 2024-09-21T11:46:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[29710]: pgmap v317: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.49k op/s 2024-09-21T11:46:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:46:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[29710]: 5.17 scrub starts 2024-09-21T11:46:49.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:49 smithi043 ceph-mon[29710]: 5.17 scrub ok 2024-09-21T11:46:49.836 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:49.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:49.875 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:49.875 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:49.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:50.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:50.014 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:50.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:50.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:50.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:50.146 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:50.147 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:50.163 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:50.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:46:50.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:50.275 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:50.275 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:50 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:46:50.275 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:50 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:46:50.275 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:50 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:46:50.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:50.276 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:50.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:50.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:50.422 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:50.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:50.564 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:50.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:50.565 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:50.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:50 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:46:50.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:50 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:46:50.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:50 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:46:50.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:50 smithi043 ceph-mon[36454]: 5.c scrub starts 2024-09-21T11:46:50.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:50 smithi043 ceph-mon[36454]: 5.c scrub ok 2024-09-21T11:46:50.660 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:50 smithi043 ceph-mon[29710]: 5.c scrub starts 2024-09-21T11:46:50.660 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:50 smithi043 ceph-mon[29710]: 5.c scrub ok 2024-09-21T11:46:50.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:50.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:50.698 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:50.698 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:50.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:46:50.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:50 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:46:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:50 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:46:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:50 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:46:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:50 smithi175 ceph-mon[32613]: 5.c scrub starts 2024-09-21T11:46:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:50 smithi175 ceph-mon[32613]: 5.c scrub ok 2024-09-21T11:46:50.799 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:50.829 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:50.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:50.830 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:50.930 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:50.951 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:50.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:50.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:51.053 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:51.086 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:51.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:51.087 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:51.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:51.226 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:51.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:51.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:51.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:51.366 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:51.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:51.367 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:51.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:46:51.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:51 smithi175 ceph-mon[32613]: 8.1 scrub starts 2024-09-21T11:46:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:51 smithi175 ceph-mon[32613]: pgmap v318: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-21T11:46:51.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:46:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:46:51.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:51.514 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:51.514 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:51.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:51.646 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:51.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:51.647 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:51 smithi043 ceph-mon[29710]: 8.1 scrub starts 2024-09-21T11:46:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:51 smithi043 ceph-mon[29710]: pgmap v318: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-21T11:46:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:51 smithi043 ceph-mon[36454]: 8.1 scrub starts 2024-09-21T11:46:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:51 smithi043 ceph-mon[36454]: pgmap v318: 148 pgs: 148 active+clean; 7.2 GiB data, 24 GiB used, 602 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-21T11:46:51.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:51.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:51.788 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:51.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:51.889 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:51.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:51.924 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:51.925 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:52.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:52.056 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:52.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:52.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:52.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:52.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:52.186 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:52.287 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:52.322 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:52.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:52.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:52.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:52.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:52.463 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:52.463 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:52.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:52.598 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:52.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:52.599 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:52.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:52.734 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:52.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:52.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:52 smithi043 ceph-mon[36454]: 4.10 scrub starts 2024-09-21T11:46:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:52 smithi043 ceph-mon[36454]: 4.10 scrub ok 2024-09-21T11:46:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:52 smithi043 ceph-mon[36454]: 6.1e scrub starts 2024-09-21T11:46:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:52 smithi043 ceph-mon[36454]: 6.1e scrub ok 2024-09-21T11:46:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:52 smithi043 ceph-mon[29710]: 4.10 scrub starts 2024-09-21T11:46:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:52 smithi043 ceph-mon[29710]: 4.10 scrub ok 2024-09-21T11:46:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:52 smithi043 ceph-mon[29710]: 6.1e scrub starts 2024-09-21T11:46:52.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:52 smithi043 ceph-mon[29710]: 6.1e scrub ok 2024-09-21T11:46:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:46:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:52 smithi175 ceph-mon[32613]: 4.10 scrub starts 2024-09-21T11:46:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:52 smithi175 ceph-mon[32613]: 4.10 scrub ok 2024-09-21T11:46:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:52 smithi175 ceph-mon[32613]: 6.1e scrub starts 2024-09-21T11:46:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:52 smithi175 ceph-mon[32613]: 6.1e scrub ok 2024-09-21T11:46:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:52.836 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:52.867 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:52.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:52.868 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:52.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:53.008 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:53.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:53.009 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:53.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:53.138 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:53.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:53.139 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:53.239 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:53.274 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:53.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:53.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:53.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:53.411 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:53.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:53.411 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:53.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:53.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:53.537 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:53.576 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:53 smithi043 ceph-mon[29710]: pgmap v319: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.8 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-21T11:46:53.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:53 smithi043 ceph-mon[29710]: 5.d scrub starts 2024-09-21T11:46:53.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:53 smithi043 ceph-mon[29710]: 5.d scrub ok 2024-09-21T11:46:53.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:53 smithi043 ceph-mon[29710]: 3.7 scrub starts 2024-09-21T11:46:53.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:53 smithi043 ceph-mon[29710]: 3.7 scrub ok 2024-09-21T11:46:53.577 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:53 smithi043 ceph-mon[36454]: pgmap v319: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.8 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-21T11:46:53.577 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:53 smithi043 ceph-mon[36454]: 5.d scrub starts 2024-09-21T11:46:53.577 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:53 smithi043 ceph-mon[36454]: 5.d scrub ok 2024-09-21T11:46:53.577 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:53 smithi043 ceph-mon[36454]: 3.7 scrub starts 2024-09-21T11:46:53.577 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:53 smithi043 ceph-mon[36454]: 3.7 scrub ok 2024-09-21T11:46:53.638 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:53.670 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:53.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:53.671 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:53.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:53 smithi175 ceph-mon[32613]: pgmap v319: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.8 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-21T11:46:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:53 smithi175 ceph-mon[32613]: 5.d scrub starts 2024-09-21T11:46:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:53 smithi175 ceph-mon[32613]: 5.d scrub ok 2024-09-21T11:46:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:53 smithi175 ceph-mon[32613]: 3.7 scrub starts 2024-09-21T11:46:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:53 smithi175 ceph-mon[32613]: 3.7 scrub ok 2024-09-21T11:46:53.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:53.807 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:53.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:53.809 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:53.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:53.934 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:53.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:53.935 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:54.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:54.066 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:54.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:54.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:no valid command found; 10 closest matches: 2024-09-21T11:46:54.103 INFO:teuthology.orchestra.run.smithi043.stderr:0 2024-09-21T11:46:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:1 2024-09-21T11:46:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:2 2024-09-21T11:46:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:abort 2024-09-21T11:46:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:assert 2024-09-21T11:46:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:bench [] [] [] [] 2024-09-21T11:46:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs debug_inject_read_zeros 2024-09-21T11:46:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs files list 2024-09-21T11:46:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:bluefs stats 2024-09-21T11:46:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:bluestore allocator dump block 2024-09-21T11:46:54.104 INFO:teuthology.orchestra.run.smithi043.stderr:Error EINVAL: invalid command 2024-09-21T11:46:54.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:54.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:54.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:54.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:54.336 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:54.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:54.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:54.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:46:54.476 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[36454]: 8.1 scrub ok 2024-09-21T11:46:54.476 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:46:54.476 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:46:54.476 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[36454]: 5.13 scrub starts 2024-09-21T11:46:54.476 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[36454]: 5.13 scrub ok 2024-09-21T11:46:54.476 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[36454]: 4.13 scrub starts 2024-09-21T11:46:54.476 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[36454]: 4.13 scrub ok 2024-09-21T11:46:54.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[29710]: 8.1 scrub ok 2024-09-21T11:46:54.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:46:54.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:46:54.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[29710]: 5.13 scrub starts 2024-09-21T11:46:54.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[29710]: 5.13 scrub ok 2024-09-21T11:46:54.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[29710]: 4.13 scrub starts 2024-09-21T11:46:54.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:54 smithi043 ceph-mon[29710]: 4.13 scrub ok 2024-09-21T11:46:54.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:54.478 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:54.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:46:54.579 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:54.616 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:54.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:54.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:54.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:54.749 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:54.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:54.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:54 smithi175 ceph-mon[32613]: 8.1 scrub ok 2024-09-21T11:46:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:54 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:46:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:54 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:46:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:54 smithi175 ceph-mon[32613]: 5.13 scrub starts 2024-09-21T11:46:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:54 smithi175 ceph-mon[32613]: 5.13 scrub ok 2024-09-21T11:46:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:54 smithi175 ceph-mon[32613]: 4.13 scrub starts 2024-09-21T11:46:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:54 smithi175 ceph-mon[32613]: 4.13 scrub ok 2024-09-21T11:46:54.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:54.864 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T11:46:54.864 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-21T11:46:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:54.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:54.893 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:54.915 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:46:54.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:55.031 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:55.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:55.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:55.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:55.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:55.176 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:55.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:55.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:55.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:55.314 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:55.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:55.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:55.448 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:46:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:46:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:46:55.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:55 smithi043 ceph-mon[29710]: pgmap v320: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.8 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-21T11:46:55.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:55 smithi043 ceph-mon[29710]: osdmap e199: 8 total, 8 up, 7 in 2024-09-21T11:46:55.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:55 smithi043 ceph-mon[29710]: 2.1 scrub starts 2024-09-21T11:46:55.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:55 smithi043 ceph-mon[29710]: 2.1 scrub ok 2024-09-21T11:46:55.451 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:55 smithi043 ceph-mon[36454]: pgmap v320: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.8 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-21T11:46:55.451 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:55 smithi043 ceph-mon[36454]: osdmap e199: 8 total, 8 up, 7 in 2024-09-21T11:46:55.451 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:55 smithi043 ceph-mon[36454]: 2.1 scrub starts 2024-09-21T11:46:55.451 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:55 smithi043 ceph-mon[36454]: 2.1 scrub ok 2024-09-21T11:46:55.456 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:55.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:55.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:55.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:55.589 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:55.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:55.590 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:55.684 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:46:55.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:46:55.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:46:55.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:46:55.685 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:46:55.685 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:46:55.686 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:46:55.686 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:46:55.686 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:46:55.686 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:46:55.689 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:46:55.689 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:46:55.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:55.726 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:46:55.727 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:46:55.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:55.730 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:55.730 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:55.760 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:46:55.760 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:46:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:55 smithi175 ceph-mon[32613]: pgmap v320: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.8 GiB data, 23 GiB used, 603 GiB / 626 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s 2024-09-21T11:46:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:55 smithi175 ceph-mon[32613]: osdmap e199: 8 total, 8 up, 7 in 2024-09-21T11:46:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:55 smithi175 ceph-mon[32613]: 2.1 scrub starts 2024-09-21T11:46:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:55 smithi175 ceph-mon[32613]: 2.1 scrub ok 2024-09-21T11:46:55.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:55.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:55.875 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:55.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:55.976 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:56.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:56.009 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:56.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:56.110 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:56.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:56.140 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:56.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:56.174 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:46:56.174 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:220961 7.a 7:5daa3d65:::benchmark_data_smithi043_70646_object103553:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:26.879010+0000", 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 149.12418718999999, 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13911041299999999, 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:46:56.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 220961 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.879010+0000", 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.879010+0000", 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.879011+0000", 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.09e-07 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.879015+0000", 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6129999999999997e-06 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.879029+0000", 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3811999999999999e-05 2024-09-21T11:46:56.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.879037+0000", 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.627e-06 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.886053+0000", 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0070159749999999998 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.886153+0000", 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100581 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.886264+0000", 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111128 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888635+0000", 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023705330000000002 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.896993+0000", 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083579280000000006 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.018022+0000", 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.121028897 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.018074+0000", 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1822e-05 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.018120+0000", 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6588000000000002e-05 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:220962 7.2 7:48169232:::benchmark_data_smithi043_70646_object103554:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:26.879999+0000", 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 149.12319760099999, 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.140615238, 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.181 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 220962 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.879999+0000", 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.879999+0000", 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.880006+0000", 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7299999999999999e-06 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.880010+0000", 2024-09-21T11:46:56.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.335e-06 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.880031+0000", 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0244000000000001e-05 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.880038+0000", 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888597+0000", 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0085591680000000007 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888715+0000", 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118546 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888835+0000", 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011908699999999999 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.893906+0000", 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050718559999999996 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020400+0000", 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.126493631 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020515+0000", 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011487799999999999 2024-09-21T11:46:56.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020564+0000", 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9011000000000002e-05 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020615+0000", 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0596000000000001e-05 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:220970 7.a 7:57d28474:::benchmark_data_smithi043_70646_object103562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:26.884101+0000", 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 149.11909636300001, 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.136630007, 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 220970 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884101+0000", 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884101+0000", 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884102+0000", 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8029999999999999e-06 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884107+0000", 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0810000000000001e-06 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884118+0000", 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0387000000000001e-05 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884125+0000", 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7669999999999998e-06 2024-09-21T11:46:56.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888657+0000", 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004532566 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888727+0000", 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9921999999999999e-05 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888835+0000", 2024-09-21T11:46:56.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107524 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.893861+0000", 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050264840000000003 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020395+0000", 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12653352900000001 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020658+0000", 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026389600000000001 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020698+0000", 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9789999999999997e-05 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020731+0000", 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2258000000000002e-05 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.193 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:220972 7.a 7:53cd9682:::benchmark_data_smithi043_70646_object103564:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:26.884397+0000", 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 149.11880038800001, 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.136461901, 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:46:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 220972 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884397+0000", 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884397+0000", 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884397+0000", 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4499999999999996e-07 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884401+0000", 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1080000000000003e-06 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884412+0000", 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.083e-05 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.884420+0000", 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4869999999999996e-06 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.891061+0000", 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0066414689999999997 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.891158+0000", 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6389000000000002e-05 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.891272+0000", 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114151 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.894107+0000", 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002835134 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020410+0000", 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12630258699999999 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020779+0000", 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000369852 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020821+0000", 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1189000000000002e-05 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020858+0000", 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.786e-05 2024-09-21T11:46:56.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:220974 7.2 7:422758c1:::benchmark_data_smithi043_70646_object103566:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:26.888527+0000", 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 149.114670313, 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13244075599999999, 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:46:56.200 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 220974 2024-09-21T11:46:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888527+0000", 2024-09-21T11:46:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888527+0000", 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888528+0000", 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3319999999999999e-06 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888533+0000", 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1970000000000004e-06 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888551+0000", 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.785e-05 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.888558+0000", 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3239999999999997e-06 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.891162+0000", 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00260414 2024-09-21T11:46:56.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.891277+0000", 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114877 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.891387+0000", 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.896676+0000", 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052888370000000002 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020405+0000", 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12372888999999999 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020893+0000", 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000488011 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020931+0000", 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8279999999999999e-05 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020967+0000", 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6018e-05 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:220976 7.a 7:559ee579:::benchmark_data_smithi043_70646_object103568:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:26.889391+0000", 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 149.113805837, 2024-09-21T11:46:56.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13166521, 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 220976 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.889391+0000", 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.889391+0000", 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.889392+0000", 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8899999999999998e-07 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.889396+0000", 2024-09-21T11:46:56.208 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6899999999999998e-06 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.889405+0000", 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8899999999999996e-06 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.889410+0000", 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2279999999999998e-06 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.894139+0000", 2024-09-21T11:46:56.209 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047291740000000001 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.894241+0000", 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102279 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.894354+0000", 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112415 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.210 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.017982+0000", 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.123628213 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020475+0000", 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024927719999999999 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.020972+0000", 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00049691199999999996 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.211 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.021019+0000", 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7508e-05 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.021056+0000", 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7240000000000003e-05 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43891 8.f 8:f55fc01e:::benchmark_data_smithi043_97042_object43890:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:46:56.212 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.981481+0000", 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.021715841000002, 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.137559443, 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43891 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.213 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981481+0000", 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981481+0000", 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981483+0000", 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4789999999999999e-06 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.214 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981503+0000", 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0564000000000002e-05 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981514+0000", 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.098e-05 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981520+0000", 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9059999999999999e-06 2024-09-21T11:46:56.215 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981576+0000", 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6280000000000003e-05 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981712+0000", 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013535399999999999 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:46:56.216 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981848+0000", 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136415 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.984930+0000", 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003081669 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.984942+0000", 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1868999999999999e-05 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.217 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.118941+0000", 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133998961 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.118996+0000", 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5417000000000002e-05 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.119041+0000", 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4549000000000003e-05 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.218 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43894 8.f 8:ffe19373:::benchmark_data_smithi043_97042_object43893:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.981775+0000", 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.02142182, 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13738866199999999, 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:56.219 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43894 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981775+0000", 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981775+0000", 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.220 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981776+0000", 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.031e-06 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981804+0000", 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7484999999999999e-05 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981815+0000", 2024-09-21T11:46:56.221 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1003e-05 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981822+0000", 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3000000000000004e-06 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983389+0000", 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015668050000000001 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.222 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983472+0000", 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3373000000000003e-05 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983564+0000", 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2097999999999998e-05 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.985072+0000", 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015080899999999999 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.223 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.985187+0000", 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114425 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.119084+0000", 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13389731899999999 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.119132+0000", 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7864000000000001e-05 2024-09-21T11:46:56.224 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.119164+0000", 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1869000000000003e-05 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43895 8.b 8:df4b4431:::benchmark_data_smithi043_97042_object43894:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.981835+0000", 2024-09-21T11:46:56.225 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.021361591000002, 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13339694399999999, 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43895 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981835+0000", 2024-09-21T11:46:56.226 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981835+0000", 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981836+0000", 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9100000000000004e-07 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981858+0000", 2024-09-21T11:46:56.227 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1940999999999999e-05 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981863+0000", 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1760000000000001e-06 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981867+0000", 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0929999999999996e-06 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.228 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981913+0000", 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6247e-05 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.982043+0000", 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012943699999999999 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.982157+0000", 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114067 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.229 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983779+0000", 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001622376 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.984039+0000", 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00025932800000000002 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115100+0000", 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13106125399999999 2024-09-21T11:46:56.230 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115190+0000", 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0469000000000004e-05 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115232+0000", 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1965000000000002e-05 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.231 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:43891 8.f e172/160 ondisk, result = 0)", 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.986280+0000", 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.016917358000001, 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13276740100000001, 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.986280+0000", 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.232 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.986280+0000", 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.986280+0000", 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.46e-07 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.986282+0000", 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4030000000000001e-06 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.233 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115370+0000", 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12908721100000001 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115379+0000", 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3819999999999993e-06 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.118926+0000", 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035474730000000002 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.234 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.118935+0000", 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2539999999999992e-06 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.119047+0000", 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011223200000000001 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43904 8.f 8:fd29ade4:::benchmark_data_smithi043_97042_object43903:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:46:56.235 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.987249+0000", 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 95.015947718999996, 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13401378, 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43904 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.987249+0000", 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.236 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.987249+0000", 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.987250+0000", 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5700000000000001e-07 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.987276+0000", 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6077000000000001e-05 2024-09-21T11:46:56.237 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114870+0000", 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12759351399999999 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114882+0000", 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2388000000000001e-05 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115296+0000", 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00041409299999999998 2024-09-21T11:46:56.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115456+0000", 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015947199999999999 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115613+0000", 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000157045 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.121114+0000", 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055009400000000002 2024-09-21T11:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.121156+0000", 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2441000000000002e-05 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.121169+0000", 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2924e-05 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.240 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.121217+0000", 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7960000000000002e-05 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.121263+0000", 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6069000000000002e-05 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:46257 8.b 8:d72ba8ac:::benchmark_data_smithi043_97042_object46256:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:22.599589+0000", 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 93.403607718999993, 2024-09-21T11:46:56.241 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14588980200000001, 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46257 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.599589+0000", 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.242 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.599589+0000", 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.599590+0000", 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.034e-06 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.599616+0000", 2024-09-21T11:46:56.243 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5446e-05 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.599623+0000", 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.294e-06 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.599628+0000", 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5920000000000002e-06 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.244 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.599667+0000", 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9338999999999997e-05 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.732542+0000", 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13287468899999999 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.732670+0000", 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012844700000000001 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.736538+0000", 2024-09-21T11:46:56.245 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038684240000000001 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.738331+0000", 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001792119 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.745235+0000", 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0069040330000000004 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.246 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.745433+0000", 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019824099999999999 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.745479+0000", 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6143999999999997e-05 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:46260 8.0 8:07d649a8:::benchmark_data_smithi043_97042_object46259:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:46:56.247 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:22.601973+0000", 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 93.401224284999998, 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.138519955, 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46260 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.601973+0000", 2024-09-21T11:46:56.248 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.601973+0000", 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.601973+0000", 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4600000000000003e-07 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.249 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.601993+0000", 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9809999999999998e-05 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.732424+0000", 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13043035 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.732437+0000", 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3419e-05 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.250 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.733196+0000", 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000758951 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.733321+0000", 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012515000000000001 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,5", 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.733622+0000", 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030075600000000001 2024-09-21T11:46:56.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.738241+0000", 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046186279999999996 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.738293+0000", 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2791999999999999e-05 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.252 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.740391+0000", 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020978849999999999 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.740438+0000", 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7107000000000001e-05 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.740493+0000", 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4261000000000001e-05 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.253 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47787 8.0 8:0e217459:::benchmark_data_smithi043_97042_object47786:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.712644+0000", 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 92.290552749, 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134000338, 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:56.254 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47787 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712644+0000", 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712644+0000", 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.255 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712646+0000", 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.576e-06 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712669+0000", 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3013000000000001e-05 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712678+0000", 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1120000000000002e-06 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.256 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712685+0000", 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4549999999999998e-06 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712739+0000", 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3868e-05 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712822+0000", 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2945e-05 2024-09-21T11:46:56.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,5", 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712928+0000", 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106277 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846503+0000", 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13357476400000001 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846519+0000", 2024-09-21T11:46:56.258 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5503e-05 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846556+0000", 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6838000000000002e-05 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846606+0000", 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0299000000000001e-05 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.259 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846644+0000", 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8688e-05 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47789 8.0 8:0fd5bd62:::benchmark_data_smithi043_97042_object47788:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.712784+0000", 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 92.290412549999999, 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134301267, 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.260 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47789 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712784+0000", 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712784+0000", 2024-09-21T11:46:56.261 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712785+0000", 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9500000000000001e-07 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712806+0000", 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1152e-05 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.262 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712816+0000", 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9380000000000004e-06 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712823+0000", 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0999999999999998e-06 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.714288+0000", 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014645789999999999 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.263 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.844401+0000", 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13011305200000001 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,5", 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.844570+0000", 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016908 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846968+0000", 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002398257 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.264 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846979+0000", 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0424999999999999e-05 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.847004+0000", 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5259e-05 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.847051+0000", 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6946000000000001e-05 2024-09-21T11:46:56.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.847086+0000", 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4684000000000002e-05 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47791 8.0 8:0faf2dad:::benchmark_data_smithi043_97042_object47790:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.713104+0000", 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 92.290093093999999, 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.138408964, 2024-09-21T11:46:56.266 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47791 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713104+0000", 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713104+0000", 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713105+0000", 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0599999999999999e-07 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713123+0000", 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8629000000000001e-05 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.268 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713131+0000", 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7239999999999992e-06 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713136+0000", 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5859999999999998e-06 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.847112+0000", 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13397615500000001 2024-09-21T11:46:56.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.847230+0000", 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011821599999999999 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,5", 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.847604+0000", 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037389500000000001 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.270 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.849549+0000", 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019445930000000001 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.850247+0000", 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000698809 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.851042+0000", 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00079433300000000004 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.271 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.851138+0000", 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6337999999999998e-05 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.851513+0000", 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037478000000000001 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:47788 8.8 e173/160 8:1016d613:::benchmark_data_smithi043_97042_object47787:head v 173'3003, mlcod=173'3001)", 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.714068+0000", 2024-09-21T11:46:56.272 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 92.289128495, 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13285760199999999, 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.714068+0000", 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.714068+0000", 2024-09-21T11:46:56.273 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.714069+0000", 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7800000000000001e-07 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.714141+0000", 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1778000000000003e-05 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.274 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.714148+0000", 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.199e-06 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.714153+0000", 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5669999999999996e-06 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.714272+0000", 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119298 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.275 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.714283+0000", 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287000000000001e-05 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846520+0000", 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.132236717 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846926+0000", 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00040607799999999998 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.276 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47787 8.0 e173/160 ondisk, result = 0)", 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.715437+0000", 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 92.287759668999996, 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13121374799999999, 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715437+0000", 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.277 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715437+0000", 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715438+0000", 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.82e-07 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715441+0000", 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1640000000000001e-06 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.278 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.843572+0000", 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12813081800000001 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.843581+0000", 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.346e-06 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846541+0000", 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002960004 2024-09-21T11:46:56.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846550+0000", 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1400000000000006e-06 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846651+0000", 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100694 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.280 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47787 8.0 e173/160 ondisk, result = 0)", 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.715457+0000", 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 92.287740163999999, 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.131476485, 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715457+0000", 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.281 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715457+0000", 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715457+0000", 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.37e-07 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715460+0000", 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7599999999999998e-06 2024-09-21T11:46:56.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842749+0000", 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12728932100000001 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842767+0000", 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7456e-05 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846488+0000", 2024-09-21T11:46:56.283 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003721636 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846497+0000", 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6540000000000005e-06 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846933+0000", 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00043612099999999998 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:66137 8.0 8:0f2cb91d:::benchmark_data_smithi043_97042_object66136:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:35.923371+0000", 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 80.079826073000007, 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13187506099999999, 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 66137 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.285 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.923371+0000", 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.923371+0000", 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.923373+0000", 2024-09-21T11:46:56.286 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.996e-06 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.923398+0000", 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5072999999999999e-05 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.923409+0000", 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1358e-05 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:46:56.287 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.923416+0000", 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2320000000000004e-06 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.924949+0000", 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00153242 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.925090+0000", 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014132700000000001 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.288 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,5", 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.925225+0000", 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000134618 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.927649+0000", 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024238100000000002 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.927746+0000", 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7097999999999998e-05 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.289 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.055051+0000", 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.127305108 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.055162+0000", 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011092200000000001 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.055246+0000", 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4098999999999996e-05 2024-09-21T11:46:56.290 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:46:56.291 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:46:56.291 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:46:56.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:56.326 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:56.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:56.327 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:56.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:56.459 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:56.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:56.460 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:56.561 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:56.595 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:56.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:56.596 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:56.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[36454]: 6.9 scrub starts 2024-09-21T11:46:56.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[36454]: 6.9 scrub ok 2024-09-21T11:46:56.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2404987409' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:56.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[36454]: 3.1e scrub starts 2024-09-21T11:46:56.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[36454]: 3.1e scrub ok 2024-09-21T11:46:56.619 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[36454]: 5.a scrub starts 2024-09-21T11:46:56.620 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[36454]: 5.a scrub ok 2024-09-21T11:46:56.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[29710]: 6.9 scrub starts 2024-09-21T11:46:56.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[29710]: 6.9 scrub ok 2024-09-21T11:46:56.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2404987409' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:56.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[29710]: 3.1e scrub starts 2024-09-21T11:46:56.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[29710]: 3.1e scrub ok 2024-09-21T11:46:56.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[29710]: 5.a scrub starts 2024-09-21T11:46:56.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:56 smithi043 ceph-mon[29710]: 5.a scrub ok 2024-09-21T11:46:56.697 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:56.726 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:56.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:56.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:56.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:56 smithi175 ceph-mon[32613]: 6.9 scrub starts 2024-09-21T11:46:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:56 smithi175 ceph-mon[32613]: 6.9 scrub ok 2024-09-21T11:46:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:56 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2404987409' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:46:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:56 smithi175 ceph-mon[32613]: 3.1e scrub starts 2024-09-21T11:46:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:56 smithi175 ceph-mon[32613]: 3.1e scrub ok 2024-09-21T11:46:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:56 smithi175 ceph-mon[32613]: 5.a scrub starts 2024-09-21T11:46:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:56 smithi175 ceph-mon[32613]: 5.a scrub ok 2024-09-21T11:46:56.828 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:56.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:56.867 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:56.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:57.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:57.007 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:57.008 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:57.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:57.149 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:57.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:57.150 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:57.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:57.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:57.292 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:46:57.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:46:57.429 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:57.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:57.430 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:46:57.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:57.568 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:57.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:57.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:57.670 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:57.706 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:57.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:57.707 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:57.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:57 smithi043 ceph-mon[29710]: pgmap v322: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.8 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 822 op/s 2024-09-21T11:46:57.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/595065798' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:57.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:57 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:57.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:57 smithi043 ceph-mon[36454]: pgmap v322: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.8 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 822 op/s 2024-09-21T11:46:57.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:57 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/595065798' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:57.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:57 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:57 smithi175 ceph-mon[32613]: pgmap v322: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.8 GiB data, 22 GiB used, 604 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 822 op/s 2024-09-21T11:46:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:57 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/595065798' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:57 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:46:57.808 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:57.846 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:57.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:57.847 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:57.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:46:57.987 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:57.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:57.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:46:58.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:58.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:58.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:58.191 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:46:58.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:58.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:58.267 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:46:58.368 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:58.413 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:58.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:58.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[29710]: 4.8 scrub starts 2024-09-21T11:46:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[29710]: 4.8 scrub ok 2024-09-21T11:46:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:46:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:46:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[29710]: osdmap e200: 8 total, 8 up, 7 in 2024-09-21T11:46:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[29710]: 3.2 scrub starts 2024-09-21T11:46:58.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[29710]: 3.2 scrub ok 2024-09-21T11:46:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[36454]: 4.8 scrub starts 2024-09-21T11:46:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[36454]: 4.8 scrub ok 2024-09-21T11:46:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:46:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:46:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[36454]: osdmap e200: 8 total, 8 up, 7 in 2024-09-21T11:46:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[36454]: 3.2 scrub starts 2024-09-21T11:46:58.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:58 smithi043 ceph-mon[36454]: 3.2 scrub ok 2024-09-21T11:46:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:58 smithi175 ceph-mon[32613]: 4.8 scrub starts 2024-09-21T11:46:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:58 smithi175 ceph-mon[32613]: 4.8 scrub ok 2024-09-21T11:46:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:58 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:46:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:58 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:46:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:58 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:46:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:58 smithi175 ceph-mon[32613]: osdmap e200: 8 total, 8 up, 7 in 2024-09-21T11:46:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:58 smithi175 ceph-mon[32613]: 3.2 scrub starts 2024-09-21T11:46:58.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:58 smithi175 ceph-mon[32613]: 3.2 scrub ok 2024-09-21T11:46:58.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:58.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:58.571 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:58.571 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:58.630 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:46:58.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:58.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:58.712 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:58.713 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:58.813 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:58.849 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:58.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:58.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:58.951 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:46:58.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:58.984 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:58.984 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:46:59.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:59.122 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:59.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:59.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:59.224 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:59.261 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:59.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:59.262 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:59.359 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:46:59.359 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:46:59.359 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.359 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:173909 8.2 8:4d589ae8:::benchmark_data_smithi043_97042_object36815:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e200)", 2024-09-21T11:46:59.359 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:59.345226+0000", 2024-09-21T11:46:59.359 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.01042656, 2024-09-21T11:46:59.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010454921000000001, 2024-09-21T11:46:59.359 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:46:59.359 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 173909 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.345226+0000", 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.360 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.345226+0000", 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.345228+0000", 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.494e-06 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.345233+0000", 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4020000000000001e-06 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.361 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.345244+0000", 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0654e-05 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.345261+0000", 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6793e-05 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.345320+0000", 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9168e-05 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.362 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.345429+0000", 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010875899999999999 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.345533+0000", 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010411799999999999 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.349933+0000", 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004400709 2024-09-21T11:46:59.363 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.349946+0000", 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2988e-05 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:173912 8.5 e200/160 8:aef3af78:::benchmark_data_smithi043_97042_object36818:head v 200'10851, mlcod=200'10847)", 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:59.347477+0000", 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0081760989999999992, 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0082790290000000002, 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:46:59.364 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.347477+0000", 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.347477+0000", 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:46:59.365 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.347478+0000", 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.92e-07 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.347483+0000", 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4770000000000002e-06 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.347676+0000", 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019310099999999999 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.366 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.347684+0000", 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8690000000000004e-06 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.350145+0000", 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024607909999999999 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.350162+0000", 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7404000000000001e-05 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:46:59.367 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:173917 8.5 e200/160 8:aa6d81ed:::benchmark_data_smithi043_97042_object36823:head v 200'10852, mlcod=200'10847)", 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:59.352756+0000", 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0028967120000000001, 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030459829999999999, 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.352756+0000", 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:46:59.368 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.352756+0000", 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.352757+0000", 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4900000000000004e-07 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.352763+0000", 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7089999999999999e-06 2024-09-21T11:46:59.369 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.352770+0000", 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7909999999999999e-06 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.352776+0000", 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0310000000000004e-06 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.352848+0000", 2024-09-21T11:46:59.370 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2435999999999996e-05 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.352866+0000", 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7711999999999999e-05 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:173921 8.4 e200/160 8:2393d53c:::benchmark_data_smithi043_97042_object36827:head v 200'10911, mlcod=200'10907)", 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:59.354308+0000", 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.001345026, 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001540504, 2024-09-21T11:46:59.371 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354308+0000", 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354308+0000", 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.372 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354308+0000", 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3e-07 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354313+0000", 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.95e-06 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354324+0000", 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0512e-05 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.373 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354330+0000", 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0680000000000002e-06 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354415+0000", 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5177000000000001e-05 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354432+0000", 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6957e-05 2024-09-21T11:46:59.374 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:173920 8.5 e200/160)", 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:59.354347+0000", 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.001305957, 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015437579999999999, 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "queued for pg", 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354347+0000", 2024-09-21T11:46:59.375 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354347+0000", 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354348+0000", 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1799999999999995e-07 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:46:59.376 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354352+0000", 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2490000000000003e-06 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354361+0000", 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3859999999999996e-06 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:59.354367+0000", 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.925e-06 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.377 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:46:59.378 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:46:59.378 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 5 2024-09-21T11:46:59.378 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:46:59.378 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:46:59.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:59.425 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:59.426 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:46:59.526 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:59.559 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:59.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:59.560 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:59.661 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:46:59.696 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:59.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:59.697 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:46:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:59 smithi043 ceph-mon[36454]: pgmap v324: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.68k op/s 2024-09-21T11:46:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:59 smithi043 ceph-mon[36454]: 4.0 scrub starts 2024-09-21T11:46:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:46:59 smithi043 ceph-mon[36454]: 4.0 scrub ok 2024-09-21T11:46:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:59 smithi043 ceph-mon[29710]: pgmap v324: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.68k op/s 2024-09-21T11:46:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:59 smithi043 ceph-mon[29710]: 4.0 scrub starts 2024-09-21T11:46:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:46:59 smithi043 ceph-mon[29710]: 4.0 scrub ok 2024-09-21T11:46:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:59 smithi175 ceph-mon[32613]: pgmap v324: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.68k op/s 2024-09-21T11:46:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:59 smithi175 ceph-mon[32613]: 4.0 scrub starts 2024-09-21T11:46:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:46:59 smithi175 ceph-mon[32613]: 4.0 scrub ok 2024-09-21T11:46:59.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:46:59.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:46:59.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:59.834 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:46:59.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:46:59.866 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:46:59.866 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:46:59.866 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-21T11:46:59.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd in 1 2024-09-21T11:46:59.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:46:59.990 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:46:59.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:46:59.992 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:00.092 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:00.114 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:00.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:00.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:00.208 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:00.215 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:00.253 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:00.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:00.254 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:00.354 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:00.393 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:00.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:00.394 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:00.495 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:00 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:47:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:00 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:47:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:00 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:47:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:00 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:47:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:00 smithi175 ceph-mon[32613]: 3.8 scrub starts 2024-09-21T11:47:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:00 smithi175 ceph-mon[32613]: 3.8 scrub ok 2024-09-21T11:47:00.529 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:00.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:00.530 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:00.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:00.665 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:00.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:00.666 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:00.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:47:00.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:47:00.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:47:00.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:47:00.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[29710]: 3.8 scrub starts 2024-09-21T11:47:00.694 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[29710]: 3.8 scrub ok 2024-09-21T11:47:00.695 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:47:00.695 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:47:00.695 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:47:00.695 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:47:00.695 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[36454]: 3.8 scrub starts 2024-09-21T11:47:00.695 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:00 smithi043 ceph-mon[36454]: 3.8 scrub ok 2024-09-21T11:47:00.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:00.802 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:00.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:00.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:00.807 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:00.839 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:47:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:47:00.863 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:47:00.863 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:47:00.893 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:47:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:47:00.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:00.927 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:47:00.927 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:47:00.950 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:00.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:00.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:00.961 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:47:00.962 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:47:00.996 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:47:00.996 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:47:01.033 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:47:01.033 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:47:01.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:01.068 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:47:01.068 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:47:01.088 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:01.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:01.090 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:01.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:01.224 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:01.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:01.225 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:01.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:01.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:01.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:01.378 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: 3.6 scrub starts 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: 6.8 scrub starts 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: 3.6 scrub ok 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: 6.8 scrub ok 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: pgmap v325: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 847 op/s 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: osdmap e201: 8 total, 8 up, 7 in 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3194549639' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T11:47:01.419 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T11:47:01.420 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:47:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:47:01.460 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:01.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:47:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:01.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:01.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:01.598 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: 3.6 scrub starts 2024-09-21T11:47:01.598 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: 6.8 scrub starts 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: 3.6 scrub ok 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: 6.8 scrub ok 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: pgmap v325: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 847 op/s 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: osdmap e201: 8 total, 8 up, 7 in 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3194549639' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: 3.6 scrub starts 2024-09-21T11:47:01.599 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: 6.8 scrub starts 2024-09-21T11:47:01.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: 3.6 scrub ok 2024-09-21T11:47:01.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: 6.8 scrub ok 2024-09-21T11:47:01.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: pgmap v325: 148 pgs: 148 active+clean; 6.4 GiB data, 21 GiB used, 605 GiB / 626 GiB avail; 639 B/s rd, 0 B/s wr, 847 op/s 2024-09-21T11:47:01.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: osdmap e201: 8 total, 8 up, 7 in 2024-09-21T11:47:01.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:01.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:01.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3194549639' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:01.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T11:47:01.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:01 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T11:47:01.603 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:01.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:01.637 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:01.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:01.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:01.772 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:01.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:01.794 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:01.794 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:01.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:01.939 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:01.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:01.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:02.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:02.072 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:02.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:02.073 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:02.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:02.217 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:02.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:02.218 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:02.284 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:02.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:02.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a[29706]: 2024-09-21T11:47:02.020+0000 7f5b548dc640 -1 mon.a@0(leader).osd e201 definitely_dead 0 2024-09-21T11:47:02.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:02.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-mon[29710]: 5.e scrub starts 2024-09-21T11:47:02.348 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-mon[29710]: 5.e scrub ok 2024-09-21T11:47:02.348 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4069113787' entity='client.admin' cmd={"prefix": "osd in", "ids": ["1"]} : dispatch 2024-09-21T11:47:02.348 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd in", "ids": ["1"]} : dispatch 2024-09-21T11:47:02.357 INFO:teuthology.orchestra.run.smithi043.stderr:marked in osd.1. 2024-09-21T11:47:02.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:02.360 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:02.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:02.387 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:47:02.387 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:47:02.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:02.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:02.486 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:02.486 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:02 smithi175 ceph-mon[32613]: 5.e scrub starts 2024-09-21T11:47:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:02 smithi175 ceph-mon[32613]: 5.e scrub ok 2024-09-21T11:47:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4069113787' entity='client.admin' cmd={"prefix": "osd in", "ids": ["1"]} : dispatch 2024-09-21T11:47:02.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:02 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd in", "ids": ["1"]} : dispatch 2024-09-21T11:47:02.587 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:02.610 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:02.610 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-mon[36454]: 5.e scrub starts 2024-09-21T11:47:02.610 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-mon[36454]: 5.e scrub ok 2024-09-21T11:47:02.610 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4069113787' entity='client.admin' cmd={"prefix": "osd in", "ids": ["1"]} : dispatch 2024-09-21T11:47:02.610 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:02 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd in", "ids": ["1"]} : dispatch 2024-09-21T11:47:02.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:02.611 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:02.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:02.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:02.735 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:02.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:02.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:02.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:02.859 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:02.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:02.860 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:02.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:02.969 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:47:02.983 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:02.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:02.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:03.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:03.107 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:03.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:03.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:03.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:03.213 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:03.231 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:03.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:03.232 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:03.330 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-21T11:47:03.333 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:03.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:03.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:03.464 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:03.487 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:03.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:03.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:03.588 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:03.609 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:03.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:03.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:03.630 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:03 smithi175 ceph-mon[32613]: pgmap v327: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.77k op/s 2024-09-21T11:47:03.630 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:03 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-09-21T11:47:03.630 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:03 smithi175 ceph-mon[32613]: osdmap e202: 8 total, 8 up, 8 in 2024-09-21T11:47:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:03 smithi175 ceph-mon[32613]: 5.14 scrub starts 2024-09-21T11:47:03.631 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:03 smithi175 ceph-mon[32613]: 5.14 scrub ok 2024-09-21T11:47:03.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:03.729 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:47:03.729 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:47:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:03.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:03.734 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:03.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:03 smithi043 ceph-mon[36454]: pgmap v327: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.77k op/s 2024-09-21T11:47:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:03 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-09-21T11:47:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:03 smithi043 ceph-mon[36454]: osdmap e202: 8 total, 8 up, 8 in 2024-09-21T11:47:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:03 smithi043 ceph-mon[36454]: 5.14 scrub starts 2024-09-21T11:47:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:03 smithi043 ceph-mon[36454]: 5.14 scrub ok 2024-09-21T11:47:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:03 smithi043 ceph-mon[29710]: pgmap v327: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.77k op/s 2024-09-21T11:47:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:03 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-09-21T11:47:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:03 smithi043 ceph-mon[29710]: osdmap e202: 8 total, 8 up, 8 in 2024-09-21T11:47:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:03 smithi043 ceph-mon[29710]: 5.14 scrub starts 2024-09-21T11:47:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:03 smithi043 ceph-mon[29710]: 5.14 scrub ok 2024-09-21T11:47:03.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:03.857 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:03.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:03.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:03.958 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:03.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:03.982 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:04.083 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:04.106 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:04.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:04.107 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:04.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:47:04.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:04.233 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:04.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:04.234 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:04.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:04.357 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:04.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:04.358 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:04.359 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:47:04.460 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:04.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[29710]: osdmap e203: 8 total, 8 up, 8 in 2024-09-21T11:47:04.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/329823790' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:47:04.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[29710]: 4.3 scrub starts 2024-09-21T11:47:04.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[29710]: 4.3 scrub ok 2024-09-21T11:47:04.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[29710]: 4.1d scrub starts 2024-09-21T11:47:04.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[29710]: 4.1d scrub ok 2024-09-21T11:47:04.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[36454]: osdmap e203: 8 total, 8 up, 8 in 2024-09-21T11:47:04.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/329823790' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:47:04.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[36454]: 4.3 scrub starts 2024-09-21T11:47:04.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[36454]: 4.3 scrub ok 2024-09-21T11:47:04.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[36454]: 4.1d scrub starts 2024-09-21T11:47:04.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:04 smithi043 ceph-mon[36454]: 4.1d scrub ok 2024-09-21T11:47:04.502 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:04.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:04.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:04.604 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:04.637 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:04.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:04.638 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:04.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:04.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:04 smithi175 ceph-mon[32613]: osdmap e203: 8 total, 8 up, 8 in 2024-09-21T11:47:04.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:04 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/329823790' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:47:04.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:04 smithi175 ceph-mon[32613]: 4.3 scrub starts 2024-09-21T11:47:04.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:04 smithi175 ceph-mon[32613]: 4.3 scrub ok 2024-09-21T11:47:04.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:04 smithi175 ceph-mon[32613]: 4.1d scrub starts 2024-09-21T11:47:04.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:04 smithi175 ceph-mon[32613]: 4.1d scrub ok 2024-09-21T11:47:04.793 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:04.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:04.794 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:04.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:04.931 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:04.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:04.932 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:05.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:47:05.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:05.063 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:05.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:05.064 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:05.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:05.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:05.206 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:05.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:05.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:05.346 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:05.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:05.347 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:05.403 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:05.425 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:47:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: pgmap v330: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/329823790' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: 8.b starting backfill to osd.1 from (0'0,0'0] MAX to 201'10928 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: 8.0 starting backfill to osd.1 from (0'0,0'0] MAX to 201'10980 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: 8.a starting backfill to osd.1 from (0'0,0'0] MAX to 201'10978 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: osdmap e204: 8 total, 8 up, 8 in 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: 8.7 starting backfill to osd.1 from (0'0,0'0] MAX to 201'11121 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: 8.f starting backfill to osd.1 from (0'0,0'0] MAX to 201'11019 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: 8.1 starting backfill to osd.1 from (0'0,0'0] MAX to 201'11346 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: 8.3 starting backfill to osd.1 from (0'0,0'0] MAX to 201'11166 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:47:05.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: pgmap v330: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/329823790' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: 8.b starting backfill to osd.1 from (0'0,0'0] MAX to 201'10928 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: 8.0 starting backfill to osd.1 from (0'0,0'0] MAX to 201'10980 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: 8.a starting backfill to osd.1 from (0'0,0'0] MAX to 201'10978 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: osdmap e204: 8 total, 8 up, 8 in 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: 8.7 starting backfill to osd.1 from (0'0,0'0] MAX to 201'11121 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: 8.f starting backfill to osd.1 from (0'0,0'0] MAX to 201'11019 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: 8.1 starting backfill to osd.1 from (0'0,0'0] MAX to 201'11346 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: 8.3 starting backfill to osd.1 from (0'0,0'0] MAX to 201'11166 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:47:05.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:05 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:47:05.448 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:05.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:05.484 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:05.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:05.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:05.618 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:05.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:05.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:05.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:05.747 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:05.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:05.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: pgmap v330: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 606 GiB / 626 GiB avail; 853 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/329823790' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: 8.b starting backfill to osd.1 from (0'0,0'0] MAX to 201'10928 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: 8.0 starting backfill to osd.1 from (0'0,0'0] MAX to 201'10980 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: 8.a starting backfill to osd.1 from (0'0,0'0] MAX to 201'10978 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: osdmap e204: 8 total, 8 up, 8 in 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: 8.7 starting backfill to osd.1 from (0'0,0'0] MAX to 201'11121 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: 8.f starting backfill to osd.1 from (0'0,0'0] MAX to 201'11019 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: 8.1 starting backfill to osd.1 from (0'0,0'0] MAX to 201'11346 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: 8.3 starting backfill to osd.1 from (0'0,0'0] MAX to 201'11166 2024-09-21T11:47:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:47:05.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:05 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:47:05.849 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:05.880 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:05.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:05.881 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:05.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:06.020 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:06.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:06.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:06.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:06.162 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:06.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:06.163 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:06.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:06.301 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:06.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:06.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:06.387 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:47:06.387 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:47:06.402 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:06.435 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:06.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:06.436 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:06 smithi175 ceph-mon[32613]: osdmap e205: 8 total, 8 up, 8 in 2024-09-21T11:47:06.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:06.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:06.575 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:06.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:06.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:06.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:06.714 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:06.714 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:06 smithi043 ceph-mon[36454]: osdmap e205: 8 total, 8 up, 8 in 2024-09-21T11:47:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:06 smithi043 ceph-mon[29710]: osdmap e205: 8 total, 8 up, 8 in 2024-09-21T11:47:06.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:06.853 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:06.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:06.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:06.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:06.998 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:06.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:06.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:07.100 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:07.122 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:07.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:07.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:07.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:47:07.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:47:07.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:07.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:07.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:07.396 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:47:07.396 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:47:07.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:07.437 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:47:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:47:07.461 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:07.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:07.462 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:07.482 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:07 smithi043 ceph-mon[36454]: pgmap v333: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 48 op/s; 418 B/s, 11 objects/s recovering 2024-09-21T11:47:07.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/363015707' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:07.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:07 smithi043 ceph-mon[29710]: pgmap v333: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 48 op/s; 418 B/s, 11 objects/s recovering 2024-09-21T11:47:07.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/363015707' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:07.483 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:47:07.484 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:47:07.509 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:07.520 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:47:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:47:07.559 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:47:07.559 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:47:07.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:07.592 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:07.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:07.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:07.597 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:47:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:47:07.629 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:47:07.629 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:47:07.636 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:07 smithi175 ceph-mon[32613]: pgmap v333: 148 pgs: 148 active+clean; 6.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 48 op/s; 418 B/s, 11 objects/s recovering 2024-09-21T11:47:07.636 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/363015707' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:07.672 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:47:07.672 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:47:07.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:07.720 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:07.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:07.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:07.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:07.857 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:07.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:07.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:07.959 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:07.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:07.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:07.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:08.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:08.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:08.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:08.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:08.265 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:08.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:08.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:08.331 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:47:08.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 0 1 2024-09-21T11:47:08.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:08.419 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:08.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:08.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:08.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:08.565 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:08.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:08.603 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:08.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:08.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:08 smithi043 ceph-mon[29710]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:47:08.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:08 smithi043 ceph-mon[36454]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:47:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:08.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:08.756 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:08.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:08 smithi175 ceph-mon[32613]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:47:08.783 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:47:08.783 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:47:08.783 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:47:08.783 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:47:08.783 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:47:08.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:08.900 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:08.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:08.902 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:09.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:09.036 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:09.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:09.037 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:09.138 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:09.166 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:09.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:09.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:09.267 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:09.289 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:09.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:09.293 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:09.293 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:09.394 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:09.419 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:09.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:09.420 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:09.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:09.548 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:09.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:09.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:09.644 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:47:09.650 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:09.733 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:09.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:09.734 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:09 smithi043 ceph-mon[29710]: pgmap v334: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 554 B/s rd, 0 B/s wr, 563 op/s; 40596/285360 objects misplaced (14.226%); 7.7 MiB/s, 141 objects/s recovering 2024-09-21T11:47:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:09 smithi043 ceph-mon[36454]: pgmap v334: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 554 B/s rd, 0 B/s wr, 563 op/s; 40596/285360 objects misplaced (14.226%); 7.7 MiB/s, 141 objects/s recovering 2024-09-21T11:47:09.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:09 smithi175 ceph-mon[32613]: pgmap v334: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 554 B/s rd, 0 B/s wr, 563 op/s; 40596/285360 objects misplaced (14.226%); 7.7 MiB/s, 141 objects/s recovering 2024-09-21T11:47:09.835 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:09.860 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:09.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:09.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:09.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:10.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:10.002 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:10.002 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:10.019 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:10.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:10.144 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:10.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:10.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:10.246 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:10.275 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:10.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:10.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:10.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:10.414 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:10.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:10.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:10.492 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:47:10.516 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:10.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:47:10.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:10.551 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:10.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:10.599 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:47:10.600 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:47:10.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:47:10.600 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:47:10.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.600 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220182 7.0 7:0ab20e26:::benchmark_data_smithi043_70646_object102774:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:47:10.600 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.198969+0000", 2024-09-21T11:47:10.600 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 164.395384815, 2024-09-21T11:47:10.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10672838, 2024-09-21T11:47:10.600 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.600 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220182 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198969+0000", 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198969+0000", 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198971+0000", 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8509999999999999e-06 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198977+0000", 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7699999999999998e-06 2024-09-21T11:47:10.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198988+0000", 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0808000000000001e-05 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.198993+0000", 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4940000000000003e-06 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.199102+0000", 2024-09-21T11:47:10.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109261 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.199301+0000", 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000198913 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.199428+0000", 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012705600000000001 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.202312+0000", 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028836199999999999 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.249560+0000", 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.047248342999999998 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.305560+0000", 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.05599954 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.305641+0000", 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0918e-05 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.305698+0000", 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6805999999999998e-05 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43896 8.3 8:c3bd4fd6:::benchmark_data_smithi043_97042_object43895:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:47:10.606 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.982177+0000", 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 109.612176707, 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14573064599999999, 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43896 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982177+0000", 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982177+0000", 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982178+0000", 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5600000000000005e-07 2024-09-21T11:47:10.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982209+0000", 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0809000000000003e-05 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982218+0000", 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0089999999999996e-06 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982224+0000", 2024-09-21T11:47:10.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5350000000000001e-06 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.984992+0000", 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027677230000000001 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.985115+0000", 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012332000000000001 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.985242+0000", 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012655500000000001 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989565+0000", 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043232679999999999 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992180+0000", 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002614487 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127774+0000", 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13559412700000001 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127839+0000", 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4784999999999997e-05 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127908+0000", 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9271999999999997e-05 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43898 8.3 8:cb8873a2:::benchmark_data_smithi043_97042_object43897:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.986596+0000", 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 109.60775784, 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.1414368, 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43898 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986596+0000", 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986596+0000", 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986598+0000", 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.844e-06 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986632+0000", 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.413e-05 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986644+0000", 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1443e-05 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986653+0000", 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5550000000000008e-06 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987082+0000", 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042925400000000001 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987555+0000", 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047269100000000002 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987683+0000", 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128329 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992217+0000", 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045337889999999999 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992269+0000", 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1941999999999998e-05 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127950+0000", 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13568037199999999 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127999+0000", 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9444999999999998e-05 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.128033+0000", 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4005999999999999e-05 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43901 8.3 8:c3170c6a:::benchmark_data_smithi043_97042_object43900:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.986960+0000", 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 109.607393574, 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.144105175, 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.619 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43901 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986960+0000", 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986960+0000", 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986961+0000", 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1800000000000004e-07 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987010+0000", 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8708999999999997e-05 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987021+0000", 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0548e-05 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987028+0000", 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9949999999999999e-06 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989603+0000", 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025754509999999999 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989731+0000", 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012751 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989866+0000", 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000135105 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992250+0000", 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023844780000000002 2024-09-21T11:47:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992368+0000", 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117924 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.130962+0000", 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13859406799999999 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.131025+0000", 2024-09-21T11:47:10.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2992999999999998e-05 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.131066+0000", 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0475999999999997e-05 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43907 8.8 8:131facf7:::benchmark_data_smithi043_97042_object43906:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.989559+0000", 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 109.604795186, 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12934364800000001, 2024-09-21T11:47:10.625 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43907 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989559+0000", 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989559+0000", 2024-09-21T11:47:10.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989560+0000", 2024-09-21T11:47:10.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T11:47:10.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989621+0000", 2024-09-21T11:47:10.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0619999999999999e-05 2024-09-21T11:47:10.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989629+0000", 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3459999999999994e-06 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989634+0000", 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9540000000000003e-06 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989707+0000", 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3330000000000004e-05 2024-09-21T11:47:10.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989835+0000", 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127687 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989962+0000", 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127179 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.991540+0000", 2024-09-21T11:47:10.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015774160000000001 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.991814+0000", 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027401599999999997 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118806+0000", 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.126992523 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118858+0000", 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1774999999999998e-05 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118902+0000", 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4571999999999999e-05 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:45873 8.8 8:1a0feefe:::benchmark_data_smithi043_97042_object45872:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.277880+0000", 2024-09-21T11:47:10.632 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 108.316474493, 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11236278299999999, 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 45873 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277880+0000", 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277880+0000", 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277881+0000", 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0699999999999999e-06 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277944+0000", 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3847e-05 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277953+0000", 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3729999999999993e-06 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.277958+0000", 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5380000000000002e-06 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.278005+0000", 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6214000000000002e-05 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.278086+0000", 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1856999999999999e-05 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.278164+0000", 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7516999999999997e-05 2024-09-21T11:47:10.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.279569+0000", 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014045819999999999 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.279975+0000", 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040607499999999999 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.390144+0000", 2024-09-21T11:47:10.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.110169824 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.390197+0000", 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2070000000000001e-05 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.390242+0000", 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5816000000000003e-05 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.638 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47786 8.d 8:b7d985cc:::benchmark_data_smithi043_97042_object47785:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.712413+0000", 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.881941197, 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.133025376, 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47786 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712413+0000", 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712413+0000", 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712414+0000", 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9500000000000001e-07 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712445+0000", 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0970000000000003e-05 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712657+0000", 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021223800000000001 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712664+0000", 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4159999999999998e-06 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712753+0000", 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8609999999999994e-05 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712865+0000", 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011187400000000001 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713024+0000", 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000158954 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716097+0000", 2024-09-21T11:47:10.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030732699999999999 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716114+0000", 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6693000000000001e-05 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845317+0000", 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12920314499999999 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845388+0000", 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0706999999999999e-05 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845438+0000", 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0603999999999999e-05 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.645 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47788 8.8 8:1016d613:::benchmark_data_smithi043_97042_object47787:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.712688+0000", 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.881665549, 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13416687399999999, 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47788 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712688+0000", 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712688+0000", 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712689+0000", 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8899999999999999e-07 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712720+0000", 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0641e-05 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712826+0000", 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712837+0000", 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0925e-05 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712917+0000", 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9839999999999995e-05 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713082+0000", 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000164947 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713214+0000", 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000132829 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.715666+0000", 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024515679999999999 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716616+0000", 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00095051100000000002 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.846760+0000", 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13014347900000001 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.846809+0000", 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9141999999999998e-05 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.846855+0000", 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6303e-05 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47793 8.d 8:b20302a8:::benchmark_data_smithi043_97042_object47792:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.716282+0000", 2024-09-21T11:47:10.651 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 106.878072452, 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.133436156, 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47793 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716282+0000", 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716282+0000", 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716283+0000", 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2979999999999999e-06 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716340+0000", 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7364999999999997e-05 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716443+0000", 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010284400000000001 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716449+0000", 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3910000000000003e-06 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716490+0000", 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0630000000000002e-05 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716621+0000", 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013114600000000001 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.716748+0000", 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012651000000000001 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.717993+0000", 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001245265 2024-09-21T11:47:10.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.718461+0000", 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046752700000000003 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.849624+0000", 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.131163909 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.849675+0000", 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0621000000000002e-05 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.849718+0000", 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.265e-05 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66125 8.4 8:2b051868:::benchmark_data_smithi043_97042_object66124:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.915339+0000", 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.679014875000007, 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.133401625, 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66125 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.915339+0000", 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.915339+0000", 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.915341+0000", 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0449999999999999e-06 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.915416+0000", 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5067999999999993e-05 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.915426+0000", 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0122999999999999e-05 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.915433+0000", 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9e-06 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.915484+0000", 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0526999999999999e-05 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.915559+0000", 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5302999999999994e-05 2024-09-21T11:47:10.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.915638+0000", 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9049999999999997e-05 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920276+0000", 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0046378189999999996 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920309+0000", 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2938999999999998e-05 2024-09-21T11:47:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048649+0000", 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12833990100000001 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048696+0000", 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7157000000000003e-05 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048741+0000", 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4793000000000003e-05 2024-09-21T11:47:10.662 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66127 8.4 8:2a1dd9a8:::benchmark_data_smithi043_97042_object66126:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.916244+0000", 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.678109962999997, 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13852240499999999, 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66127 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.663 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916244+0000", 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916244+0000", 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916246+0000", 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.542e-06 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916321+0000", 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5754999999999996e-05 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916332+0000", 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0617e-05 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916339+0000", 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9040000000000003e-06 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.917528+0000", 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001189486 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.917639+0000", 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110455 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.917760+0000", 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012136 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920338+0000", 2024-09-21T11:47:10.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002577916 2024-09-21T11:47:10.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048623+0000", 2024-09-21T11:47:10.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12828446499999999 2024-09-21T11:47:10.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.667 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:10.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.054693+0000", 2024-09-21T11:47:10.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0060701349999999999 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.054727+0000", 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4706000000000003e-05 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.054766+0000", 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9063999999999997e-05 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.669 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66129 8.d 8:b79d813a:::benchmark_data_smithi043_97042_object66128:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.918517+0000", 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.675837510999997, 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.127661054, 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66129 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918517+0000", 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918517+0000", 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918518+0000", 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.207e-06 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918559+0000", 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1538000000000001e-05 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918569+0000", 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3209999999999994e-06 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918574+0000", 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7089999999999999e-06 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918639+0000", 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4851999999999995e-05 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918744+0000", 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104586 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918860+0000", 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116192 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046020+0000", 2024-09-21T11:47:10.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.127160312 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046046+0000", 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5633999999999999e-05 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046079+0000", 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3352999999999997e-05 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046130+0000", 2024-09-21T11:47:10.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0315000000000002e-05 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046178+0000", 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8034999999999997e-05 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66130 8.4 8:216cc0c6:::benchmark_data_smithi043_97042_object66129:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.919278+0000", 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.675076403999995, 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.138675363, 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.675 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66130 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919278+0000", 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919278+0000", 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919279+0000", 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8700000000000004e-07 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919324+0000", 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5812000000000001e-05 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919334+0000", 2024-09-21T11:47:10.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9499999999999996e-06 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919341+0000", 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920363+0000", 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010227229999999999 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.045883+0000", 2024-09-21T11:47:10.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12551943400000001 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046054+0000", 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017078800000000001 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.051620+0000", 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005566718 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.054778+0000", 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00315745 2024-09-21T11:47:10.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.057852+0000", 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030744449999999999 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.057909+0000", 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6851000000000002e-05 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.057953+0000", 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3908e-05 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.680 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:66128 8.2 e177/160 8:4c9d601c:::benchmark_data_smithi043_97042_object66127:head v 177'4126, mlcod=177'4123)", 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.919757+0000", 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.674597234000004, 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12873114399999999, 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919757+0000", 2024-09-21T11:47:10.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919757+0000", 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919758+0000", 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3300000000000001e-07 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919778+0000", 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0418999999999999e-05 2024-09-21T11:47:10.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919788+0000", 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5780000000000005e-06 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919794+0000", 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1870000000000002e-06 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919835+0000", 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1575000000000001e-05 2024-09-21T11:47:10.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919852+0000", 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7056999999999999e-05 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048450+0000", 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12859727900000001 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048488+0000", 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8315999999999997e-05 2024-09-21T11:47:10.684 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66132 8.8 8:11c54109:::benchmark_data_smithi043_97042_object66131:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.920105+0000", 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.674248887999994, 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12918839700000001, 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.685 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66132 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920105+0000", 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920105+0000", 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920106+0000", 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0270000000000001e-06 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920166+0000", 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0223999999999998e-05 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920203+0000", 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6418000000000003e-05 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920210+0000", 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3520000000000001e-06 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920269+0000", 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8627e-05 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920383+0000", 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011428 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920507+0000", 2024-09-21T11:47:10.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124327 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048578+0000", 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12807105399999999 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.049139+0000", 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00056081700000000002 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.049161+0000", 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1280999999999998e-05 2024-09-21T11:47:10.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.049202+0000", 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1610000000000003e-05 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.049294+0000", 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1379999999999996e-05 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66136 8.4 8:2cd25696:::benchmark_data_smithi043_97042_object66135:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.920996+0000", 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.673358324000006, 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.137070152, 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66136 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920996+0000", 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920996+0000", 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920997+0000", 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6500000000000008e-07 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.921088+0000", 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1389000000000002e-05 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.045913+0000", 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.124825011 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.045924+0000", 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1165000000000001e-05 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048778+0000", 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002853707 2024-09-21T11:47:10.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048896+0000", 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011762800000000001 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.049029+0000", 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013383399999999999 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.051658+0000", 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002628306 2024-09-21T11:47:10.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.054782+0000", 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0031247150000000001 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.057989+0000", 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0032060830000000002 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.058034+0000", 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5878999999999999e-05 2024-09-21T11:47:10.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.058066+0000", 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1470000000000002e-05 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.15150.0:66129 8.d e177/160 ondisk, result = 0)", 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.921493+0000", 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.672861181000002, 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.124537785, 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.696 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.921493+0000", 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.921493+0000", 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.921494+0000", 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8299999999999996e-07 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.921497+0000", 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5990000000000002e-06 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.045580+0000", 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.124083043 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.045597+0000", 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6750000000000001e-05 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046006+0000", 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040933899999999999 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046014+0000", 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6529999999999994e-06 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046031+0000", 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6718000000000002e-05 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.15150.0:66129 8.d e177/160 ondisk, result = 0)", 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.923095+0000", 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 94.671259113000005, 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.123089195, 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.923095+0000", 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.923095+0000", 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.923096+0000", 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7400000000000002e-07 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.923102+0000", 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8579999999999996e-06 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.045918+0000", 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12281661100000001 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.045940+0000", 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1759e-05 2024-09-21T11:47:10.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046068+0000", 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012855700000000001 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046075+0000", 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2890000000000003e-06 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046184+0000", 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109347 2024-09-21T11:47:10.703 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:68259 8.8 8:11d86798:::benchmark_data_smithi043_97042_object68258:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:37.656579+0000", 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 92.937775418000001, 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11699565300000001, 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.704 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 68259 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.656579+0000", 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.656579+0000", 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.656579+0000", 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9999999999999997e-07 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.656606+0000", 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7013e-05 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.656612+0000", 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7300000000000002e-06 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.656617+0000", 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6380000000000003e-06 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.661821+0000", 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0052041709999999996 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.661941+0000", 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000119999 2024-09-21T11:47:10.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.662024+0000", 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.365e-05 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.664555+0000", 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025304210000000001 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.664806+0000", 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025121699999999998 2024-09-21T11:47:10.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773463+0000", 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.108656709 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773523+0000", 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0037999999999997e-05 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773574+0000", 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1467e-05 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:68281 8.8 8:115670a8:::benchmark_data_smithi043_97042_object68280:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:37.675390+0000", 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 92.918963660000003, 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.106143504, 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:10.710 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 68281 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675390+0000", 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675390+0000", 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:10.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675392+0000", 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.457e-06 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675451+0000", 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9398000000000003e-05 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675461+0000", 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0002e-05 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675468+0000", 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2459999999999997e-06 2024-09-21T11:47:10.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675529+0000", 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0038999999999999e-05 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675636+0000", 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107635 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.771071+0000", 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.095434354999999998 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.775892+0000", 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0048215089999999999 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.776838+0000", 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00094589200000000004 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.781349+0000", 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045111689999999998 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.781483+0000", 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013414300000000001 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.781534+0000", 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0658999999999997e-05 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:10.715 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:47:10.741 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:10.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:10 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:47:10.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:10 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:47:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:10 smithi043 ceph-mon[29710]: osdmap e206: 8 total, 8 up, 8 in 2024-09-21T11:47:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4062721116' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:10.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:10 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:10 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:47:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:10 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:47:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:10 smithi043 ceph-mon[36454]: osdmap e206: 8 total, 8 up, 8 in 2024-09-21T11:47:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4062721116' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:10 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:10.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:10.743 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:10.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:10 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:47:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:10 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:47:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:10 smithi175 ceph-mon[32613]: osdmap e206: 8 total, 8 up, 8 in 2024-09-21T11:47:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4062721116' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:10 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:10.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:10.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:10.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:10.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:11.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:11.044 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:11.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:11.045 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:11.146 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:11.175 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:11.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:11.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:11.276 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:11.312 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:11.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:11.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:11.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:11.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:11.468 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:11.468 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:11.478 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T11:47:11.489 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:47:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:47:11.569 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:11.600 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:11.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:11.601 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:11.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:11.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:47:11.736 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:11.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:11.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:11 smithi043 ceph-mon[36454]: pgmap v336: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 519 op/s; 40596/285360 objects misplaced (14.226%); 7.1 MiB/s, 130 objects/s recovering 2024-09-21T11:47:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:11 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:11 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:47:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:11 smithi043 ceph-mon[36454]: osdmap e207: 8 total, 8 up, 8 in 2024-09-21T11:47:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:11 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/987186567' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:47:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:11 smithi043 ceph-mon[29710]: pgmap v336: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 519 op/s; 40596/285360 objects misplaced (14.226%); 7.1 MiB/s, 130 objects/s recovering 2024-09-21T11:47:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:11 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:11 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:47:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:11 smithi043 ceph-mon[29710]: osdmap e207: 8 total, 8 up, 8 in 2024-09-21T11:47:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:11 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/987186567' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:47:11.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:11 smithi175 ceph-mon[32613]: pgmap v336: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 519 op/s; 40596/285360 objects misplaced (14.226%); 7.1 MiB/s, 130 objects/s recovering 2024-09-21T11:47:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:11 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:11 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:47:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:11 smithi175 ceph-mon[32613]: osdmap e207: 8 total, 8 up, 8 in 2024-09-21T11:47:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:11 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/987186567' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:47:11.764 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:11.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:11.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:11.890 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:11.890 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:11.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:12.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:12.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:12.107 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:12.132 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:12.172 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:12.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:12.173 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:12.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:12.318 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:12.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:12.319 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:12.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 2 1 2024-09-21T11:47:12.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:12.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:12.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:12.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:12.616 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:12.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:12.617 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:12.685 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:47:12.685 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:47:12.685 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:47:12.685 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:47:12.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:12.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:12.745 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:12 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:12 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/987186567' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:47:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:12 smithi175 ceph-mon[32613]: osdmap e208: 8 total, 8 up, 8 in 2024-09-21T11:47:12.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:12.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:12.874 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:12.875 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:12.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:12.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:12 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:12.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:12 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/987186567' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:47:12.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:12 smithi043 ceph-mon[36454]: osdmap e208: 8 total, 8 up, 8 in 2024-09-21T11:47:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:12 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:12 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/987186567' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:47:12.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:12 smithi043 ceph-mon[29710]: osdmap e208: 8 total, 8 up, 8 in 2024-09-21T11:47:13.009 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:13.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:13.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:13.110 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:13.141 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:13.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:13.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:13.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:13.294 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:13.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:13.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:13.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:47:13.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:13.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:47:13.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:13.501 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:13.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:13.522 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:13 smithi043 ceph-mon[36454]: pgmap v339: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.31k op/s; 36939/270612 objects misplaced (13.650%); 24 MiB/s, 390 objects/s recovering 2024-09-21T11:47:13.522 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:13 smithi043 ceph-mon[29710]: pgmap v339: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.31k op/s; 36939/270612 objects misplaced (13.650%); 24 MiB/s, 390 objects/s recovering 2024-09-21T11:47:13.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:13.631 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:13.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:13.632 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:13.731 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:13.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:13.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:13 smithi175 ceph-mon[32613]: pgmap v339: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.31k op/s; 36939/270612 objects misplaced (13.650%); 24 MiB/s, 390 objects/s recovering 2024-09-21T11:47:13.782 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:13.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:13.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:13.868 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:13.872 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:47:13.872 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:47:13.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:13.908 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:47:13.908 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:47:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:13.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:13.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:13.973 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:47:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:47:14.018 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:47:14.019 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:47:14.048 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:14.051 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:47:14.051 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:47:14.076 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:14.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:14.077 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:14.093 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:47:14.094 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:47:14.128 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:47:14.129 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:47:14.159 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:47:14.160 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:47:14.178 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:14.206 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:14.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:14.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:14.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:14.338 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:14.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:14.339 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:14.440 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:14.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:14.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:14.470 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:14.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:14.611 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:14.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:14.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:14.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:14.740 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:14.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:14.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:14.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2044671983' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:14.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2044671983' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:14.805 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2044671983' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:14.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:14.898 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:14.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:14.899 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:14.937 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:47:14.937 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:47:15.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:15.028 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:15.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:15.029 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:15.130 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:15.155 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:15.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:15.156 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:15.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:15.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:15.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:15.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:15.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:15.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:15.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:47:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:47:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:47:15.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:15.537 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T11:47:15.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:15.577 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:15.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:15.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:47:15.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:15.757 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:15.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:15.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:15.859 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:15.896 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:15.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:15.897 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:15 smithi043 ceph-mon[29710]: pgmap v340: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 819 op/s; 36939/270612 objects misplaced (13.650%); 17 MiB/s, 268 objects/s recovering 2024-09-21T11:47:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1523123847' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:47:15.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:15 smithi043 ceph-mon[36454]: pgmap v340: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 819 op/s; 36939/270612 objects misplaced (13.650%); 17 MiB/s, 268 objects/s recovering 2024-09-21T11:47:15.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1523123847' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:47:15.998 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:16.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:15 smithi175 ceph-mon[32613]: pgmap v340: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 819 op/s; 36939/270612 objects misplaced (13.650%); 17 MiB/s, 268 objects/s recovering 2024-09-21T11:47:16.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1523123847' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:47:16.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:16.034 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:16.034 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:16.046 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:16.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:16.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:16.164 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:16.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:16.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:16.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:16.305 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:16.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:16.453 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:16.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:16.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:16.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:16.576 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:16.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:16.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:16.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:16.708 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:16.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:16.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:16.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:16.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:16.852 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:16.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:16.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:16.982 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:16.982 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:16 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1523123847' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:47:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:16 smithi043 ceph-mon[29710]: osdmap e209: 8 total, 8 up, 8 in 2024-09-21T11:47:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:16 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:16 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:16 smithi043 ceph-mon[29710]: pgmap v342: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 865 op/s; 36939/269787 objects misplaced (13.692%); 17 MiB/s, 268 objects/s recovering 2024-09-21T11:47:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:16 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1523123847' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:47:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:16 smithi043 ceph-mon[36454]: osdmap e209: 8 total, 8 up, 8 in 2024-09-21T11:47:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:16 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:16 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:16 smithi043 ceph-mon[36454]: pgmap v342: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 865 op/s; 36939/269787 objects misplaced (13.692%); 17 MiB/s, 268 objects/s recovering 2024-09-21T11:47:17.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:16 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1523123847' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:47:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:16 smithi175 ceph-mon[32613]: osdmap e209: 8 total, 8 up, 8 in 2024-09-21T11:47:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:16 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:16 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:16 smithi175 ceph-mon[32613]: pgmap v342: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.5 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 865 op/s; 36939/269787 objects misplaced (13.692%); 17 MiB/s, 268 objects/s recovering 2024-09-21T11:47:17.083 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:17.111 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:17.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:17.112 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:17.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:17.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:17.261 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:17.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:17.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:17.409 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:17.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:17.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:17.462 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:47:17.462 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:47:17.512 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:17.542 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:17.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:17.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:17.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:17.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:17.686 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:17.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:17.827 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:17.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:17.828 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:17.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:17.965 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:17.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:17.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:18.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:18.100 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:18.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:18.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:18.202 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:18.235 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:18.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:18.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:18.245 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:47:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:47:18.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:18.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:18.380 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:18.381 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:18.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:18.519 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:18.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:18.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:18.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:18.652 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:18.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:18.666 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:18.666 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:18.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:18 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2110678147' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:47:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:18 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2110678147' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:47:18.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:18 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2110678147' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:47:18.767 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:18.799 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:18.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:18.800 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:18.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:18.944 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:18.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:18.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:19.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 3 1 2024-09-21T11:47:19.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:19.153 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:19.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:19.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:19.252 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:47:19.253 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:47:19.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:19.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:19.292 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:19.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:19.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:19.437 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:19.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:19.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:19.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:19 smithi175 ceph-mon[32613]: pgmap v343: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.26k op/s; 33915/257382 objects misplaced (13.177%); 24 MiB/s, 390 objects/s recovering 2024-09-21T11:47:19.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:19 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:19.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:19 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2110678147' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:47:19.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:19 smithi175 ceph-mon[32613]: osdmap e210: 8 total, 8 up, 8 in 2024-09-21T11:47:19.538 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:19.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:19.575 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:19.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:19.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:19.714 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:19.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:19.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:19 smithi043 ceph-mon[36454]: pgmap v343: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.26k op/s; 33915/257382 objects misplaced (13.177%); 24 MiB/s, 390 objects/s recovering 2024-09-21T11:47:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:19 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:19 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2110678147' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:47:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:19 smithi043 ceph-mon[36454]: osdmap e210: 8 total, 8 up, 8 in 2024-09-21T11:47:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:19 smithi043 ceph-mon[29710]: pgmap v343: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.26k op/s; 33915/257382 objects misplaced (13.177%); 24 MiB/s, 390 objects/s recovering 2024-09-21T11:47:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:19 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:19 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2110678147' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:47:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:19 smithi043 ceph-mon[29710]: osdmap e210: 8 total, 8 up, 8 in 2024-09-21T11:47:19.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:19.852 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:19.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:19.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:19.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:19.970 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:47:20.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:20.001 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:20.001 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:20.102 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:20.124 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:20.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:20.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:20.131 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:20.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:20.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:20.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:20.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:20.371 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:20.409 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:20.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:20.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:20.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:20.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:20.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:20.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:20 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:47:20.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:20 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:47:20.572 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:20 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:47:20.573 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:20 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:47:20.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:20.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:20 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:47:20.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:20 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:47:20.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:20.690 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:20.690 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:20.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:20.823 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:20.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:20.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:20.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:20.967 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:20.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:20.969 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:20.974 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:47:20.975 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:47:21.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:21.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:21.107 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:21.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:47:21.207 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:21.239 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:21.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:21.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:21.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:47:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T11:47:21.341 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:21.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:21.445 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:21.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[36454]: pgmap v345: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 551 op/s; 33915/257382 objects misplaced (13.177%); 10 MiB/s, 160 objects/s recovering 2024-09-21T11:47:21.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[36454]: 2.2 scrub starts 2024-09-21T11:47:21.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[36454]: 2.2 scrub ok 2024-09-21T11:47:21.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[36454]: osdmap e211: 8 total, 8 up, 8 in 2024-09-21T11:47:21.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[36454]: 5.9 scrub starts 2024-09-21T11:47:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[36454]: 5.9 scrub ok 2024-09-21T11:47:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[36454]: 6.1a scrub starts 2024-09-21T11:47:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[36454]: 6.1a scrub ok 2024-09-21T11:47:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4159921011' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:47:21.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:47:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[29710]: pgmap v345: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 551 op/s; 33915/257382 objects misplaced (13.177%); 10 MiB/s, 160 objects/s recovering 2024-09-21T11:47:21.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[29710]: 2.2 scrub starts 2024-09-21T11:47:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[29710]: 2.2 scrub ok 2024-09-21T11:47:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[29710]: osdmap e211: 8 total, 8 up, 8 in 2024-09-21T11:47:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[29710]: 5.9 scrub starts 2024-09-21T11:47:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[29710]: 5.9 scrub ok 2024-09-21T11:47:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[29710]: 6.1a scrub starts 2024-09-21T11:47:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[29710]: 6.1a scrub ok 2024-09-21T11:47:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4159921011' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:47:21.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:21 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:47:21.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:21.551 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-mon[32613]: pgmap v345: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 5.2 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 551 op/s; 33915/257382 objects misplaced (13.177%); 10 MiB/s, 160 objects/s recovering 2024-09-21T11:47:21.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-mon[32613]: 2.2 scrub starts 2024-09-21T11:47:21.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-mon[32613]: 2.2 scrub ok 2024-09-21T11:47:21.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-mon[32613]: osdmap e211: 8 total, 8 up, 8 in 2024-09-21T11:47:21.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-mon[32613]: 5.9 scrub starts 2024-09-21T11:47:21.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-mon[32613]: 5.9 scrub ok 2024-09-21T11:47:21.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-mon[32613]: 6.1a scrub starts 2024-09-21T11:47:21.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-mon[32613]: 6.1a scrub ok 2024-09-21T11:47:21.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4159921011' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:47:21.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:21 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:47:21.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:21.593 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:21.593 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:21.665 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:47:21.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:21.747 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:21.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:21.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:21.849 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:21.884 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:21.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:21.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:21.986 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:22.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:47:22.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:22.032 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:22.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:22.101 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:47:22.134 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:22.165 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:22.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:22.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:22.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:22.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 4 1 2024-09-21T11:47:22.320 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:22.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:22.341 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:22.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:22.495 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:22.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:22.496 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:22.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:22.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:22 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:47:22.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:22 smithi175 ceph-mon[32613]: osdmap e212: 8 total, 8 up, 8 in 2024-09-21T11:47:22.596 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:22.625 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:22.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:22.626 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:22.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:22.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:22 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:47:22.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:22 smithi043 ceph-mon[29710]: osdmap e212: 8 total, 8 up, 8 in 2024-09-21T11:47:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:22 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:47:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:22 smithi043 ceph-mon[36454]: osdmap e212: 8 total, 8 up, 8 in 2024-09-21T11:47:22.727 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:22.754 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:22.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:22.755 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:22.856 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:22.886 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:22.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:22.887 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:22.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:23.025 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:23.048 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:47:23.054 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:47:23.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:23.085 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:23.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:23.101 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:47:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:47:23.146 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:47:23.146 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:47:23.185 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:47:23.185 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:47:23.186 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:23.213 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:23.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:23.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:23.220 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:47:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:47:23.252 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:47:23.252 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:47:23.291 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:47:23.292 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:47:23.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:23.326 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:47:23.327 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:47:23.362 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:23.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:23.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:23.451 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:23 smithi175 ceph-mon[32613]: pgmap v348: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.47k op/s; 30492/243408 objects misplaced (12.527%); 30 MiB/s, 486 objects/s recovering 2024-09-21T11:47:23.451 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:23 smithi175 ceph-mon[32613]: 5.7 scrub starts 2024-09-21T11:47:23.452 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:23 smithi175 ceph-mon[32613]: 5.7 scrub ok 2024-09-21T11:47:23.452 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:23 smithi175 ceph-mon[32613]: 4.d scrub starts 2024-09-21T11:47:23.452 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:23 smithi175 ceph-mon[32613]: 4.d scrub ok 2024-09-21T11:47:23.452 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2029913563' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:23.463 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:23.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:23.512 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:23.512 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:23.595 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:47:23.595 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:47:23.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:47:23.595 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:79209 7.5 7:a7c68b32:::benchmark_data_smithi043_70646_object79208:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:06.636157+0000", 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 316.95465014000001, 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.073249726000000001, 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 79209 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.596 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636157+0000", 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636157+0000", 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636158+0000", 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4799999999999997e-07 2024-09-21T11:47:23.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636238+0000", 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0576999999999998e-05 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636248+0000", 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1290000000000002e-06 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.636253+0000", 2024-09-21T11:47:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7579999999999998e-06 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.638949+0000", 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026954660000000001 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.639072+0000", 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123707 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.639222+0000", 2024-09-21T11:47:23.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014934799999999999 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.641482+0000", 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022601159999999999 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.643463+0000", 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019809939999999998 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.709301+0000", 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.065838240000000006 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.709361+0000", 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9478000000000002e-05 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:06.709407+0000", 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6165000000000003e-05 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:80911 7.5 7:a71a7e8a:::benchmark_data_smithi043_70646_object80910:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:08.206845+0000", 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 315.38396261999998, 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.089039726, 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 80911 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.602 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206845+0000", 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206845+0000", 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206849+0000", 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7400000000000004e-06 2024-09-21T11:47:23.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206891+0000", 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1918000000000001e-05 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206908+0000", 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6589999999999999e-05 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206917+0000", 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7169999999999997e-06 2024-09-21T11:47:23.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.206987+0000", 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0568999999999995e-05 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.207072+0000", 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4778999999999996e-05 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.207155+0000", 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3567999999999997e-05 2024-09-21T11:47:23.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.209349+0000", 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021931419999999999 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.211318+0000", 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019697680000000002 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.295819+0000", 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.084500668000000001 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.295854+0000", 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4601000000000003e-05 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:08.295884+0000", 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0666000000000001e-05 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:99033 7.5 7:a309294f:::benchmark_data_smithi043_70646_object99032:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:26.337954+0000", 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 297.25285358299999, 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10807878899999999, 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99033 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.608 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337954+0000", 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337954+0000", 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337955+0000", 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-21T11:47:23.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337993+0000", 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7715000000000001e-05 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338011+0000", 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7676000000000002e-05 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338020+0000", 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9209999999999998e-06 2024-09-21T11:47:23.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338081+0000", 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0928999999999999e-05 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338172+0000", 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0969000000000003e-05 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338260+0000", 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8417000000000004e-05 2024-09-21T11:47:23.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.340226+0000", 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001966069 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.433908+0000", 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093681581999999999 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.445962+0000", 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012053895 2024-09-21T11:47:23.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.446001+0000", 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9795000000000001e-05 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.446032+0000", 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0970999999999999e-05 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.613 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:100915 7.5 7:a66f7709:::benchmark_data_smithi043_70646_object100914:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:28.566552+0000", 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.024255665, 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.076657827999999997, 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100915 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566552+0000", 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566552+0000", 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566554+0000", 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8329999999999998e-06 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566593+0000", 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8578999999999998e-05 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566612+0000", 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9029999999999999e-05 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566622+0000", 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.009e-05 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566690+0000", 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8045000000000003e-05 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566822+0000", 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013180700000000001 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.566973+0000", 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015138099999999999 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.569472+0000", 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002498564 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.570043+0000", 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057143099999999996 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.643130+0000", 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.073087066000000006 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.643175+0000", 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4249999999999998e-05 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.643209+0000", 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4752000000000002e-05 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:100922 7.5 7:a128daba:::benchmark_data_smithi043_70646_object100921:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:47:23.619 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:28.573462+0000", 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.017345234, 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.074314127999999993, 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100922 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573462+0000", 2024-09-21T11:47:23.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.621 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:23.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573462+0000", 2024-09-21T11:47:23.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573464+0000", 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0269999999999998e-06 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573512+0000", 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7676000000000002e-05 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573525+0000", 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3594000000000001e-05 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573534+0000", 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3110000000000006e-06 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573601+0000", 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7235000000000003e-05 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573732+0000", 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000130872 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.573835+0000", 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102905 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.576309+0000", 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024744039999999999 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.576330+0000", 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0550000000000001e-05 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.647672+0000", 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.071342384999999994 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.647729+0000", 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7358000000000002e-05 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:28.647776+0000", 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6811000000000003e-05 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.626 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123054 7.5 7:a4098129:::benchmark_data_smithi043_70646_object5646:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.880861+0000", 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 267.70994623500002, 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.098889820000000003, 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123054 2024-09-21T11:47:23.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880861+0000", 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880861+0000", 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880863+0000", 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6300000000000001e-06 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880867+0000", 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3239999999999996e-06 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880875+0000", 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0639999999999994e-06 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880882+0000", 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2139999999999999e-06 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880946+0000", 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3366000000000003e-05 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.881054+0000", 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010853000000000001 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.881162+0000", 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010778 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.885528+0000", 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043657339999999996 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889186+0000", 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036579680000000002 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979646+0000", 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090460503999999997 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979705+0000", 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9070999999999997e-05 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979751+0000", 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5634999999999999e-05 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.632 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123057 7.5 7:a56ed89a:::benchmark_data_smithi043_70646_object5649:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.882509+0000", 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 267.70829838999998, 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.097395657999999996, 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123057 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.633 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882509+0000", 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882509+0000", 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882510+0000", 2024-09-21T11:47:23.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5099999999999998e-07 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882513+0000", 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7680000000000002e-06 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882522+0000", 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9549999999999998e-06 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882529+0000", 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2079999999999997e-06 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.883324+0000", 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079580500000000001 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.883429+0000", 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104654 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.883536+0000", 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107166 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.885724+0000", 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021879960000000002 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889619+0000", 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0038952510000000002 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979813+0000", 2024-09-21T11:47:23.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090193465 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979862+0000", 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9344000000000001e-05 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979904+0000", 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2194999999999998e-05 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123064 7.5 7:ae62c878:::benchmark_data_smithi043_70646_object5656:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.889753+0000", 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 267.70105420700003, 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090290819999999994, 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123064 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.639 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889753+0000", 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889753+0000", 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889755+0000", 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.968e-06 2024-09-21T11:47:23.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889760+0000", 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.113e-06 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889771+0000", 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1070000000000001e-05 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889778+0000", 2024-09-21T11:47:23.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0979999999999996e-06 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889826+0000", 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8152999999999998e-05 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889903+0000", 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6737999999999996e-05 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889980+0000", 2024-09-21T11:47:23.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7373000000000006e-05 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.891776+0000", 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001795984 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.892127+0000", 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035003399999999999 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979962+0000", 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.087835977999999995 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.980007+0000", 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4619000000000001e-05 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.980044+0000", 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6692e-05 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126210 7.5 7:a6687ff4:::benchmark_data_smithi043_70646_object8802:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.192178+0000", 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 264.39862931800002, 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.099158476999999995, 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126210 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.645 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192178+0000", 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192178+0000", 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192180+0000", 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.807e-06 2024-09-21T11:47:23.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192185+0000", 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1250000000000001e-06 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192197+0000", 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2527000000000001e-05 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192205+0000", 2024-09-21T11:47:23.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0709999999999995e-06 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192287+0000", 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1976000000000003e-05 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192415+0000", 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127317 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192525+0000", 2024-09-21T11:47:23.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110297 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194639+0000", 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021140999999999998 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200841+0000", 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062023790000000001 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291238+0000", 2024-09-21T11:47:23.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090396746 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291292+0000", 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3792000000000003e-05 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291336+0000", 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4339999999999999e-05 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126211 7.5 7:a0cb9a25:::benchmark_data_smithi043_70646_object8803:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.194824+0000", 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 264.39598334900001, 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096639920000000004, 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126211 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194824+0000", 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194824+0000", 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194825+0000", 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.609e-06 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194829+0000", 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9539999999999998e-06 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194838+0000", 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4400000000000005e-06 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194844+0000", 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2040000000000002e-06 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194897+0000", 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3405000000000001e-05 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.195017+0000", 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011910599999999999 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.195137+0000", 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012094 2024-09-21T11:47:23.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200887+0000", 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057499279999999996 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200960+0000", 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2243999999999995e-05 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291381+0000", 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090421027000000001 2024-09-21T11:47:23.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291429+0000", 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8605e-05 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291464+0000", 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4458000000000001e-05 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.656 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126214 7.5 7:a1d8d5c0:::benchmark_data_smithi043_70646_object8806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.194864+0000", 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 264.39594336800002, 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096713571999999998, 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126214 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194864+0000", 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194864+0000", 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194864+0000", 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0100000000000005e-07 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194867+0000", 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3700000000000002e-06 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194874+0000", 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2779999999999996e-06 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194879+0000", 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4720000000000003e-06 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197151+0000", 2024-09-21T11:47:23.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022713999999999998 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197270+0000", 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000119391 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197386+0000", 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011619400000000001 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200949+0000", 2024-09-21T11:47:23.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035628579999999999 2024-09-21T11:47:23.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200962+0000", 2024-09-21T11:47:23.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3125000000000001e-05 2024-09-21T11:47:23.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291499+0000", 2024-09-21T11:47:23.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090537054000000006 2024-09-21T11:47:23.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291545+0000", 2024-09-21T11:47:23.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5392000000000002e-05 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291577+0000", 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2536999999999997e-05 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126216 7.5 7:a039bdc0:::benchmark_data_smithi043_70646_object8808:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.196576+0000", 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 264.394231504, 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.095402531999999998, 2024-09-21T11:47:23.663 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126216 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196576+0000", 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196576+0000", 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196577+0000", 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.71e-07 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196581+0000", 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6360000000000002e-06 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196595+0000", 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3653e-05 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.196602+0000", 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0439999999999998e-06 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.199091+0000", 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024891990000000001 2024-09-21T11:47:23.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.199198+0000", 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107163 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.199296+0000", 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8194000000000002e-05 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200996+0000", 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017000190000000001 2024-09-21T11:47:23.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.201042+0000", 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5439000000000003e-05 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291891+0000", 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090849471000000001 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291941+0000", 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0096000000000003e-05 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291978+0000", 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6746999999999999e-05 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:143066 7.5 7:af12805e:::benchmark_data_smithi043_70646_object25658:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128)", 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:18.046672+0000", 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 245.54413542099999, 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.073440914999999996, 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.669 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 143066 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046672+0000", 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046672+0000", 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046673+0000", 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7000000000000003e-07 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046677+0000", 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5519999999999998e-06 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046689+0000", 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1496e-05 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.046696+0000", 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0740000000000004e-06 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.048689+0000", 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019935370000000001 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.048794+0000", 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104295 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.049018+0000", 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000224699 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.053787+0000", 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004768881 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.062786+0000", 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089982719999999999 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.120028+0000", 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.057242807 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.120077+0000", 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8798999999999998e-05 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:18.120113+0000", 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5533e-05 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:185206 7.5 7:a5331398:::benchmark_data_smithi043_70646_object67798:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:57.859097+0000", 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 205.731710639, 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.082023416000000002, 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.675 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 185206 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859097+0000", 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859097+0000", 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859098+0000", 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2219999999999999e-06 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859102+0000", 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5979999999999999e-06 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859113+0000", 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0317e-05 2024-09-21T11:47:23.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859120+0000", 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4340000000000003e-06 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859165+0000", 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5327000000000002e-05 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859239+0000", 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3697999999999998e-05 2024-09-21T11:47:23.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.859307+0000", 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7611e-05 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.861065+0000", 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001758461 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.940968+0000", 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.079902619999999994 2024-09-21T11:47:23.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.941021+0000", 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3060999999999999e-05 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.941072+0000", 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1301999999999998e-05 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.941120+0000", 2024-09-21T11:47:23.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7765000000000001e-05 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:219297 7.5 7:a035a6a1:::benchmark_data_smithi043_70646_object101889:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:25.434259+0000", 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 178.15654801599999, 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.095706954999999996, 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.681 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 219297 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434259+0000", 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434259+0000", 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434261+0000", 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.514e-06 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434264+0000", 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5329999999999999e-06 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434272+0000", 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5429999999999996e-06 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434277+0000", 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.344e-06 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434363+0000", 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6327000000000002e-05 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434438+0000", 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4917000000000001e-05 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.434508+0000", 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9884999999999999e-05 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.436738+0000", 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022296730000000002 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.437766+0000", 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010284580000000001 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.529759+0000", 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.091992554000000004 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.529890+0000", 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013153899999999999 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.529966+0000", 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5667999999999994e-05 2024-09-21T11:47:23.686 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:219307 7.5 7:aa109419:::benchmark_data_smithi043_70646_object101899:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:25.442393+0000", 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 178.14841411099999, 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.087742547000000004, 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.687 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 219307 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442393+0000", 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442393+0000", 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442395+0000", 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.615e-06 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442401+0000", 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6760000000000001e-06 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442413+0000", 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1758e-05 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442419+0000", 2024-09-21T11:47:23.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0390000000000001e-06 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442465+0000", 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5738000000000002e-05 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442543+0000", 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7843000000000007e-05 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.442613+0000", 2024-09-21T11:47:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0632999999999999e-05 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.444495+0000", 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018818229999999999 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.445347+0000", 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00085143100000000004 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.530026+0000", 2024-09-21T11:47:23.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.084679373000000002 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.530092+0000", 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6237999999999998e-05 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:25.530136+0000", 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3380000000000001e-05 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220187 7.5 7:af65a55d:::benchmark_data_smithi043_70646_object102779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.200942+0000", 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 177.389865203, 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.095579969000000001, 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220187 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200942+0000", 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200942+0000", 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200944+0000", 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5349999999999999e-06 2024-09-21T11:47:23.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200949+0000", 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.113e-06 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200963+0000", 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4837999999999999e-05 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.200971+0000", 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7249999999999997e-06 2024-09-21T11:47:23.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.202084+0000", 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011125530000000001 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.202191+0000", 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107434 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.202297+0000", 2024-09-21T11:47:23.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106078 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204798+0000", 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025004239999999998 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.206727+0000", 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001928864 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.296449+0000", 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.089722510000000005 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.296488+0000", 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9178000000000003e-05 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.296522+0000", 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3717000000000003e-05 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220189 7.5 7:acec303d:::benchmark_data_smithi043_70646_object102781:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.204953+0000", 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 177.385853948, 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.102016905, 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220189 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204953+0000", 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204953+0000", 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204954+0000", 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204959+0000", 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.296e-06 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204968+0000", 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7329999999999998e-06 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204974+0000", 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.276e-06 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205021+0000", 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7154000000000003e-05 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205112+0000", 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1334000000000003e-05 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205186+0000", 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4011999999999995e-05 2024-09-21T11:47:23.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.206522+0000", 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001336292 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.206787+0000", 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026466100000000002 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306842+0000", 2024-09-21T11:47:23.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.100054927 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306920+0000", 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8430999999999995e-05 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306970+0000", 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9784000000000003e-05 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221210 7.5 7:a52e1ee7:::benchmark_data_smithi043_70646_object103802:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.174153+0000", 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 176.41665461900001, 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.110515855, 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221210 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174153+0000", 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174153+0000", 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174154+0000", 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3820000000000001e-06 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174159+0000", 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.746e-06 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174171+0000", 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2434e-05 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174178+0000", 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8700000000000003e-06 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174742+0000", 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00056385099999999998 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174850+0000", 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108383 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174961+0000", 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110818 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.177355+0000", 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002393825 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.178344+0000", 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00098920699999999998 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.284555+0000", 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10621093 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.284625+0000", 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9937000000000005e-05 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.284668+0000", 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3472e-05 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.710 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221223 7.5 7:a91eb7cb:::benchmark_data_smithi043_70646_object103815:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.184471+0000", 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 176.40633584599999, 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10880833400000001, 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221223 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184471+0000", 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184471+0000", 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184473+0000", 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3999999999999999e-06 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184477+0000", 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.33e-06 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184487+0000", 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7510000000000003e-06 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184494+0000", 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9059999999999996e-06 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:47:23.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184572+0000", 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8236000000000001e-05 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184735+0000", 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016326399999999999 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184852+0000", 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117142 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.187351+0000", 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024981640000000002 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.189227+0000", 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018760070000000001 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293189+0000", 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.103962165 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293240+0000", 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1685999999999999e-05 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293280+0000", 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9283000000000003e-05 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:47:23.716 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:47:23.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:23.718 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:23.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[36454]: pgmap v348: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.47k op/s; 30492/243408 objects misplaced (12.527%); 30 MiB/s, 486 objects/s recovering 2024-09-21T11:47:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[36454]: 5.7 scrub starts 2024-09-21T11:47:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[36454]: 5.7 scrub ok 2024-09-21T11:47:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[36454]: 4.d scrub starts 2024-09-21T11:47:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[36454]: 4.d scrub ok 2024-09-21T11:47:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2029913563' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[29710]: pgmap v348: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.47k op/s; 30492/243408 objects misplaced (12.527%); 30 MiB/s, 486 objects/s recovering 2024-09-21T11:47:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[29710]: 5.7 scrub starts 2024-09-21T11:47:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[29710]: 5.7 scrub ok 2024-09-21T11:47:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[29710]: 4.d scrub starts 2024-09-21T11:47:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[29710]: 4.d scrub ok 2024-09-21T11:47:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2029913563' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:23.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:23.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:23.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:23.966 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:23.998 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:23.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:23.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:24.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:24.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:24.153 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:24.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:24.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:24.286 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:24.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:24.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:24.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:24.433 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:24.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:24.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:24.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:24.569 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:24.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:24.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:24.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:47:24.670 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:24.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:24.695 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:24.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:24.796 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:24.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:24.851 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:24.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:24.982 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:24.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:24.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:25.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:25.140 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:25.141 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:25.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:25.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:25.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:25.298 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:25.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:25.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:25.362 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T11:47:25.365 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:47:25.400 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:25.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:25.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:25.428 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:25.490 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:47:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:47:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:47:25.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:25 smithi043 ceph-mon[36454]: pgmap v349: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 776 op/s; 30492/243408 objects misplaced (12.527%); 17 MiB/s, 272 objects/s recovering 2024-09-21T11:47:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:25 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4150155004' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:25 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2683723392' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:47:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:25 smithi043 ceph-mon[29710]: pgmap v349: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 776 op/s; 30492/243408 objects misplaced (12.527%); 17 MiB/s, 272 objects/s recovering 2024-09-21T11:47:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:25 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4150155004' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:25 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2683723392' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:47:25.529 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:25.558 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:25.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:25.559 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:25.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:25.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:25.686 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:25 smithi175 ceph-mon[32613]: pgmap v349: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 776 op/s; 30492/243408 objects misplaced (12.527%); 17 MiB/s, 272 objects/s recovering 2024-09-21T11:47:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:25 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4150155004' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:25 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2683723392' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:47:25.786 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:25.816 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:25.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:25.817 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:25.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:25.945 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:25.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:25.946 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:26.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:26.084 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:26.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:26.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:26.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:26.228 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:26.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:26.229 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:26.330 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:26.369 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:26.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:26.370 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:26.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:26.500 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:26.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:26.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:26.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:26.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:26.649 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:26.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:47:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4150155004' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:47:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2683723392' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:47:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[36454]: osdmap e213: 8 total, 8 up, 8 in 2024-09-21T11:47:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[36454]: 5.4 scrub starts 2024-09-21T11:47:26.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[36454]: 5.4 scrub ok 2024-09-21T11:47:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:47:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4150155004' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:47:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2683723392' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:47:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[29710]: osdmap e213: 8 total, 8 up, 8 in 2024-09-21T11:47:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[29710]: 5.4 scrub starts 2024-09-21T11:47:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:26 smithi043 ceph-mon[29710]: 5.4 scrub ok 2024-09-21T11:47:26.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:26 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:47:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:26 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4150155004' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:47:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:26 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2683723392' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:47:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:26 smithi175 ceph-mon[32613]: osdmap e213: 8 total, 8 up, 8 in 2024-09-21T11:47:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:26 smithi175 ceph-mon[32613]: 5.4 scrub starts 2024-09-21T11:47:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:26 smithi175 ceph-mon[32613]: 5.4 scrub ok 2024-09-21T11:47:26.791 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:26.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:26.792 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:26.892 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:26.922 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:26.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:26.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:26.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:47:27.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:27.050 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:27.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:27.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:27.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:27.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:27.207 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:27.208 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:27.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:27.335 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:27.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:27.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:27.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:27.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:27.488 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:27.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[36454]: pgmap v351: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 823 op/s; 30492/242571 objects misplaced (12.570%); 17 MiB/s, 272 objects/s recovering 2024-09-21T11:47:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T11:47:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T11:47:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[36454]: 3.e scrub starts 2024-09-21T11:47:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[36454]: 3.e scrub ok 2024-09-21T11:47:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:47:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:47:27.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[29710]: pgmap v351: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 823 op/s; 30492/242571 objects misplaced (12.570%); 17 MiB/s, 272 objects/s recovering 2024-09-21T11:47:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T11:47:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T11:47:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[29710]: 3.e scrub starts 2024-09-21T11:47:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[29710]: 3.e scrub ok 2024-09-21T11:47:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:47:27.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:27 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:47:27.503 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:198805 8.e 8:7c3d8e7a:::benchmark_data_smithi043_97042_object61711:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:27.493684+0000", 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0054493629999999996, 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054769099999999998, 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 198805 2024-09-21T11:47:27.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493684+0000", 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493684+0000", 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:27.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493685+0000", 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2589999999999999e-06 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493690+0000", 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2789999999999998e-06 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493701+0000", 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0951000000000001e-05 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493709+0000", 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6739999999999997e-06 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.498342+0000", 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046334590000000004 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.498445+0000", 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102237 2024-09-21T11:47:27.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.498546+0000", 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101728 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:198790 8.6 8:63902cc6:::benchmark_data_smithi043_97042_object61696:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:27.477715+0000", 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.021418054999999998, 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021526454, 2024-09-21T11:47:27.508 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 198790 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.477715+0000", 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.477715+0000", 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.477717+0000", 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.46e-06 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.477722+0000", 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.378e-06 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.477734+0000", 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1681e-05 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.477742+0000", 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8669999999999994e-06 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493169+0000", 2024-09-21T11:47:27.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015427885000000001 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.494512+0000", 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013421450000000001 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.494626+0000", 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011457 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:198794 8.6 8:615aa49e:::benchmark_data_smithi043_97042_object61700:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:27.479740+0000", 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.019393513000000001, 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019578221999999999, 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 198794 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:27.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.479740+0000", 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.479740+0000", 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.479741+0000", 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6300000000000001e-06 2024-09-21T11:47:27.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.479746+0000", 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6399999999999996e-06 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.479761+0000", 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4873e-05 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.479769+0000", 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9200000000000004e-06 2024-09-21T11:47:27.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.497973+0000", 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018204000000000001 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.498091+0000", 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011848799999999999 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.498226+0000", 2024-09-21T11:47:27.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013478399999999999 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:198804 8.9 e214/160 8:906f1794:::benchmark_data_smithi043_97042_object61710:head v 214'12342, mlcod=214'12340)", 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:27.493368+0000", 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0057647740000000003, 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0060184000000000001, 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:27.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493368+0000", 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493368+0000", 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493369+0000", 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7400000000000003e-07 2024-09-21T11:47:27.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493374+0000", 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2089999999999996e-06 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493387+0000", 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2785e-05 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493393+0000", 2024-09-21T11:47:27.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2759999999999997e-06 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493468+0000", 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4289000000000007e-05 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:27.493488+0000", 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0630000000000001e-05 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:27.520 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:47:27.521 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 4 2024-09-21T11:47:27.521 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:47:27.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:27.618 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:27.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:27.619 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:27.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:27 smithi175 ceph-mon[32613]: pgmap v351: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 823 op/s; 30492/242571 objects misplaced (12.570%); 17 MiB/s, 272 objects/s recovering 2024-09-21T11:47:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:27 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T11:47:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:27 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T11:47:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:27 smithi175 ceph-mon[32613]: 3.e scrub starts 2024-09-21T11:47:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:27 smithi175 ceph-mon[32613]: 3.e scrub ok 2024-09-21T11:47:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:27 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:47:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:27 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:47:27.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:27.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:27.764 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:27.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:27.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:27.896 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:27.896 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:27.997 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:28.025 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:28.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:28.026 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:28.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:28.156 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:28.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:28.157 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:28.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:28.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:28.302 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:28.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:28.403 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:28.430 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:28.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:28.431 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:28.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:47:28.476 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:28.532 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:28.562 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:28.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:28.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:28.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:28.703 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:28.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:28.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:28.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:28 smithi043 ceph-mon[29710]: osdmap e214: 8 total, 8 up, 8 in 2024-09-21T11:47:28.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:28 smithi043 ceph-mon[36454]: osdmap e214: 8 total, 8 up, 8 in 2024-09-21T11:47:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:28 smithi175 ceph-mon[32613]: osdmap e214: 8 total, 8 up, 8 in 2024-09-21T11:47:28.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:28.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:28.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:28.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:28.995 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:28.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:28.996 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:29.097 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:29.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:29.131 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:29.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:29.232 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:29.256 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:29.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:29.257 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:29.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:47:29.357 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:29.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:29.386 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:29.386 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:29.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:29.508 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:29.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:29.509 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:29.610 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:29.641 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:29.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:29.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:47:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:47:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[36454]: pgmap v353: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.7 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 777 B/s rd, 0 B/s wr, 979 op/s; 27518/230091 objects misplaced (11.960%); 23 MiB/s, 366 objects/s recovering 2024-09-21T11:47:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[36454]: 6.15 scrub starts 2024-09-21T11:47:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[36454]: 6.15 scrub ok 2024-09-21T11:47:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[36454]: osdmap e215: 8 total, 8 up, 8 in 2024-09-21T11:47:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:47:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:47:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[29710]: pgmap v353: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.7 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 777 B/s rd, 0 B/s wr, 979 op/s; 27518/230091 objects misplaced (11.960%); 23 MiB/s, 366 objects/s recovering 2024-09-21T11:47:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[29710]: 6.15 scrub starts 2024-09-21T11:47:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[29710]: 6.15 scrub ok 2024-09-21T11:47:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:29 smithi043 ceph-mon[29710]: osdmap e215: 8 total, 8 up, 8 in 2024-09-21T11:47:29.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:29.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:29 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:47:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:29 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:47:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:29 smithi175 ceph-mon[32613]: pgmap v353: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.7 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 777 B/s rd, 0 B/s wr, 979 op/s; 27518/230091 objects misplaced (11.960%); 23 MiB/s, 366 objects/s recovering 2024-09-21T11:47:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:29 smithi175 ceph-mon[32613]: 6.15 scrub starts 2024-09-21T11:47:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:29 smithi175 ceph-mon[32613]: 6.15 scrub ok 2024-09-21T11:47:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:29 smithi175 ceph-mon[32613]: osdmap e215: 8 total, 8 up, 8 in 2024-09-21T11:47:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:29.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:29.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:29.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:29.939 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:29.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:29.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:30.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:30.081 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:30.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:30.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:30.183 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:30.213 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:30.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:30.214 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:30.314 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:30.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:30.352 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:30.352 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:30.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:30.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:30.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:30.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:30.614 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:30.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:30.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:30.715 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:30.746 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:30.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:30.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:30.848 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:30.878 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:30.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:30.879 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:30.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 5 1 2024-09-21T11:47:30.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:31.038 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:31.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:31.039 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:31.140 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:31.171 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:31.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:31.172 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:31.273 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:31.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:31.299 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:31.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:31.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:31.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:31.453 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:31.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:31 smithi043 ceph-mon[29710]: pgmap v355: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.7 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 740 op/s; 27518/230091 objects misplaced (11.960%); 14 MiB/s, 226 objects/s recovering 2024-09-21T11:47:31.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:31 smithi043 ceph-mon[29710]: 8.9 scrub starts 2024-09-21T11:47:31.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:31.502 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:31.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:31 smithi043 ceph-mon[36454]: pgmap v355: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.7 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 740 op/s; 27518/230091 objects misplaced (11.960%); 14 MiB/s, 226 objects/s recovering 2024-09-21T11:47:31.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:31 smithi043 ceph-mon[36454]: 8.9 scrub starts 2024-09-21T11:47:31.502 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:31.503 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:31.510 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:47:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:47:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:31 smithi175 ceph-mon[32613]: pgmap v355: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 141 active+clean; 4.7 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 740 op/s; 27518/230091 objects misplaced (11.960%); 14 MiB/s, 226 objects/s recovering 2024-09-21T11:47:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:31 smithi175 ceph-mon[32613]: 8.9 scrub starts 2024-09-21T11:47:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:31.512 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:31.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:31.578 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:31.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:31.579 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:31.621 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:47:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:47:31.675 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:47:31.675 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:47:31.680 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:31.720 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:47:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:47:31.733 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:31.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:31.734 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:31.755 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:47:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:47:31.797 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:47:31.797 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:47:31.831 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:47:31.831 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:47:31.835 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:31.860 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:31.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:31.861 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:31.872 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:47:31.873 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:47:31.914 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:47:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:47:31.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:32.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:32.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:32.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:32.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:32.153 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:32.153 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:32.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:32.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:32.298 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:32.370 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:32.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:32.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:32.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:32 smithi043 ceph-mon[29710]: 8.9 scrub ok 2024-09-21T11:47:32.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:32 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3489624821' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:32.421 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:32.421 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:32 smithi043 ceph-mon[36454]: 8.9 scrub ok 2024-09-21T11:47:32.447 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:32.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:32.448 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:32.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:32.576 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:32.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:32.577 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:32.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:32.720 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:32.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:32.721 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:32.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:32 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3489624821' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:32.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:32.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:32 smithi175 ceph-mon[32613]: 8.9 scrub ok 2024-09-21T11:47:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:32 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3489624821' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:32.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:32.861 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:32.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:32.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:32.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:33.005 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:33.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:33.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:33.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:33.135 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:33.151 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:33.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:33.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:33.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:33.303 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:33.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:33.304 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:33.405 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:33.426 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:47:33.439 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:33.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:33.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:33.540 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:33.576 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:33.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:33.577 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:33.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:33.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:33.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:33 smithi043 ceph-mon[36454]: pgmap v356: 148 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.39k op/s; 24524/216186 objects misplaced (11.344%); 26 MiB/s, 418 objects/s recovering 2024-09-21T11:47:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:33 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/813692017' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:47:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:33 smithi043 ceph-mon[36454]: 4.1 scrub starts 2024-09-21T11:47:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:33 smithi043 ceph-mon[36454]: 4.1 scrub ok 2024-09-21T11:47:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:33 smithi043 ceph-mon[29710]: pgmap v356: 148 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.39k op/s; 24524/216186 objects misplaced (11.344%); 26 MiB/s, 418 objects/s recovering 2024-09-21T11:47:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:33 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/813692017' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:47:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:33 smithi043 ceph-mon[29710]: 4.1 scrub starts 2024-09-21T11:47:33.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:33 smithi043 ceph-mon[29710]: 4.1 scrub ok 2024-09-21T11:47:33.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:33 smithi175 ceph-mon[32613]: pgmap v356: 148 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.39k op/s; 24524/216186 objects misplaced (11.344%); 26 MiB/s, 418 objects/s recovering 2024-09-21T11:47:33.774 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:33 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/813692017' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:47:33.775 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:33 smithi175 ceph-mon[32613]: 4.1 scrub starts 2024-09-21T11:47:33.775 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:33 smithi175 ceph-mon[32613]: 4.1 scrub ok 2024-09-21T11:47:33.829 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:33.856 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:33.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:33.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:33.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:33.985 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:33.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:33.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:34.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:34.127 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:34.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:34.128 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:34.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:34.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:34.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:34.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:34.417 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:34.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:34.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:34.479 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T11:47:34.518 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:34.548 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:34.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:34.549 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:34.650 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:34.675 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:34.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:34.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:34.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:34 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:34.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/813692017' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:47:34.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:34 smithi043 ceph-mon[36454]: osdmap e216: 8 total, 8 up, 8 in 2024-09-21T11:47:34.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/8902765' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:47:34.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:34 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:47:34.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:34 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:34.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/813692017' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:47:34.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:34 smithi043 ceph-mon[29710]: osdmap e216: 8 total, 8 up, 8 in 2024-09-21T11:47:34.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/8902765' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:47:34.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:34 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:47:34.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:34 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/813692017' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:47:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:34 smithi175 ceph-mon[32613]: osdmap e216: 8 total, 8 up, 8 in 2024-09-21T11:47:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/8902765' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:47:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:34 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:47:34.777 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:34.807 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:34.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:34.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:34.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:34.932 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:34.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:34.933 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:35.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:35.071 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:35.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:35.072 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:35.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:35.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:47:35.214 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:35.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:35.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:35.315 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:35.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:35.342 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:35.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:35.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:35.488 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:35.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:35.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:47:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:47:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:47:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:35 smithi043 ceph-mon[36454]: pgmap v358: 148 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 784 B/s rd, 0 B/s wr, 711 op/s; 24524/216186 objects misplaced (11.344%); 13 MiB/s, 211 objects/s recovering 2024-09-21T11:47:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:35 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:47:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:35 smithi043 ceph-mon[36454]: osdmap e217: 8 total, 8 up, 8 in 2024-09-21T11:47:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:35 smithi043 ceph-mon[29710]: pgmap v358: 148 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 784 B/s rd, 0 B/s wr, 711 op/s; 24524/216186 objects misplaced (11.344%); 13 MiB/s, 211 objects/s recovering 2024-09-21T11:47:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:35 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:47:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:35 smithi043 ceph-mon[29710]: osdmap e217: 8 total, 8 up, 8 in 2024-09-21T11:47:35.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:35.619 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:35.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:35.620 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:35.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:35 smithi175 ceph-mon[32613]: pgmap v358: 148 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 784 B/s rd, 0 B/s wr, 711 op/s; 24524/216186 objects misplaced (11.344%); 13 MiB/s, 211 objects/s recovering 2024-09-21T11:47:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:35 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:47:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:35 smithi175 ceph-mon[32613]: osdmap e217: 8 total, 8 up, 8 in 2024-09-21T11:47:35.770 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:35.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:35.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:35.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:35.894 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:35.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:35.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:35.996 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:36.025 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:36.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:36.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:36.127 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:36.150 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:36.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:36.151 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:36.252 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:36.284 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:36.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:36.285 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:36.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:36.428 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:36.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:36.429 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:36.530 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:36.560 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:36.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:36.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:36.603 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:47:36.603 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:47:36.662 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:36.691 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:36.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:36.692 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:36.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:36.833 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:36.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:36.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:36.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:36.974 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:36.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:36.975 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:37.075 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:37.103 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:37.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:37.104 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:37.205 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:37.242 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:37.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:37.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:37.344 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:37.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:37.379 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:37.380 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:37.480 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:37.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:37.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:37.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:37.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:37.641 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:37.641 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:37 smithi043 ceph-mon[29710]: pgmap v360: 148 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 653 op/s; 24524/215409 objects misplaced (11.385%); 11 MiB/s, 183 objects/s recovering 2024-09-21T11:47:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:37 smithi043 ceph-mon[36454]: pgmap v360: 148 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 653 op/s; 24524/215409 objects misplaced (11.385%); 11 MiB/s, 183 objects/s recovering 2024-09-21T11:47:37.742 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:37.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:37 smithi175 ceph-mon[32613]: pgmap v360: 148 pgs: 1 active+clean+scrubbing, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 140 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 653 op/s; 24524/215409 objects misplaced (11.385%); 11 MiB/s, 183 objects/s recovering 2024-09-21T11:47:37.777 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:37.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:37.778 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:37.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:37.902 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:37.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:37.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:38.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:38.044 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:38.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:38.045 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:38.146 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:38.168 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:38.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:38.169 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:38.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:38.302 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:38.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:38.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:38.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:38.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:38.445 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:38.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:38.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:38.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:38.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:38.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:38.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:38.734 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:38.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:38.735 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:38.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:38.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:38.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:38.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:39.027 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:39.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:39.028 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:39.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:39.175 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:39.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:39.176 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:39.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:39.316 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:39.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:39.317 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:39.418 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:39.446 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:39.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:39.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:39.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:39.589 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:39.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:39.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:39.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:39.720 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:39.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:39.721 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:39 smithi043 ceph-mon[29710]: pgmap v361: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 141 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.13k op/s; 21690/202950 objects misplaced (10.687%); 21 MiB/s, 335 objects/s recovering 2024-09-21T11:47:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:39 smithi043 ceph-mon[36454]: pgmap v361: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 141 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.13k op/s; 21690/202950 objects misplaced (10.687%); 21 MiB/s, 335 objects/s recovering 2024-09-21T11:47:39.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:39 smithi175 ceph-mon[32613]: pgmap v361: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 141 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.13k op/s; 21690/202950 objects misplaced (10.687%); 21 MiB/s, 335 objects/s recovering 2024-09-21T11:47:39.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:39.865 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:39.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:39.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:39.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:40.000 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:40.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:40.001 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:40.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:47:40.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:40.183 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:40.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:40.184 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:40.285 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:40.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:40.311 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:40.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:40.412 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:40.438 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:40.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:40.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:40.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:40.587 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:40.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:40.588 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:40.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:40.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:40.730 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:40.730 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:40.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 6 1 2024-09-21T11:47:40.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:40.888 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:40.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:40.889 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:40.990 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:41.012 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:41.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:41.013 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:41.065 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:41.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:41.123 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:41.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:41.170 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:41.271 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:41.305 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:41.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:41.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:41.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:41.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:41.456 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:41.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:41.477 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:47:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:47:41.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:41.592 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:41.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:41.593 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:41.606 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:41 smithi043 ceph-mon[36454]: pgmap v362: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 141 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 551 op/s; 21690/202950 objects misplaced (10.687%); 10 MiB/s, 163 objects/s recovering 2024-09-21T11:47:41.607 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:41 smithi043 ceph-mon[29710]: pgmap v362: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 141 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 551 op/s; 21690/202950 objects misplaced (10.687%); 10 MiB/s, 163 objects/s recovering 2024-09-21T11:47:41.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:41.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:41.752 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:41.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:41.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:41 smithi175 ceph-mon[32613]: pgmap v362: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfill_wait, 3 active+remapped+backfilling, 141 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 551 op/s; 21690/202950 objects misplaced (10.687%); 10 MiB/s, 163 objects/s recovering 2024-09-21T11:47:41.804 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:47:41.804 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:47:41.850 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:47:41.850 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:47:41.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:41.881 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:41.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:41.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:41.891 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:47:41.891 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:47:41.927 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:47:41.928 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:47:41.962 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:47:41.962 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:47:41.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:41.999 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:47:41.999 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:47:42.011 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:42.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:42.012 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:42.036 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:47:42.036 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:47:42.082 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:47:42.083 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:47:42.112 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:42.141 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:42.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:42.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:42.243 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:42.268 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:42.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:42.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:42.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:42.408 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:42.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:42.409 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:42.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:42.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:42.552 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:42.582 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:47:42.582 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:47:42.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:47:42.582 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:47:42.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.583 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:220988 7.f 7:fa5453fd:::benchmark_data_smithi043_70646_object103580:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:47:42.583 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:26.898041+0000", 2024-09-21T11:47:42.583 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 195.45004359399999, 2024-09-21T11:47:42.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.125302578, 2024-09-21T11:47:42.583 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.583 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:47:42.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:47:42.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 220988 2024-09-21T11:47:42.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.584 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898041+0000", 2024-09-21T11:47:42.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898041+0000", 2024-09-21T11:47:42.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898042+0000", 2024-09-21T11:47:42.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0729999999999999e-06 2024-09-21T11:47:42.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898046+0000", 2024-09-21T11:47:42.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.506e-06 2024-09-21T11:47:42.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898053+0000", 2024-09-21T11:47:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-21T11:47:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898059+0000", 2024-09-21T11:47:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4459999999999998e-06 2024-09-21T11:47:42.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898124+0000", 2024-09-21T11:47:42.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5030999999999995e-05 2024-09-21T11:47:42.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898228+0000", 2024-09-21T11:47:42.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103584 2024-09-21T11:47:42.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:47:42.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898341+0000", 2024-09-21T11:47:42.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113522 2024-09-21T11:47:42.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.900967+0000", 2024-09-21T11:47:42.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002625432 2024-09-21T11:47:42.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.901062+0000", 2024-09-21T11:47:42.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5518000000000003e-05 2024-09-21T11:47:42.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.023195+0000", 2024-09-21T11:47:42.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.122133055 2024-09-21T11:47:42.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.023295+0000", 2024-09-21T11:47:42.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9841999999999997e-05 2024-09-21T11:47:42.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.023344+0000", 2024-09-21T11:47:42.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8531e-05 2024-09-21T11:47:42.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.594 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.595 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43882 8.c 8:35a2a8fe:::benchmark_data_smithi043_97042_object43881:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:47:42.595 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:47:42.595 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 141.37412176000001, 2024-09-21T11:47:42.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14113139399999999, 2024-09-21T11:47:42.595 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43882 2024-09-21T11:47:42.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.597 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:47:42.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:47:42.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973964+0000", 2024-09-21T11:47:42.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.28e-06 2024-09-21T11:47:42.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973985+0000", 2024-09-21T11:47:42.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1178e-05 2024-09-21T11:47:42.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973994+0000", 2024-09-21T11:47:42.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0119999999999994e-06 2024-09-21T11:47:42.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974001+0000", 2024-09-21T11:47:42.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4570000000000002e-06 2024-09-21T11:47:42.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974047+0000", 2024-09-21T11:47:42.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6393000000000001e-05 2024-09-21T11:47:42.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974162+0000", 2024-09-21T11:47:42.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001149 2024-09-21T11:47:42.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:47:42.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974245+0000", 2024-09-21T11:47:42.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2937999999999998e-05 2024-09-21T11:47:42.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.975846+0000", 2024-09-21T11:47:42.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001600944 2024-09-21T11:47:42.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976044+0000", 2024-09-21T11:47:42.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019846299999999999 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114984+0000", 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13893909600000001 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115046+0000", 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2924999999999998e-05 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115094+0000", 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7808e-05 2024-09-21T11:47:42.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43886 8.e 8:7c60ce26:::benchmark_data_smithi043_97042_object43885:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 141.37120923099999, 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13826519300000001, 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.605 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43886 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976877+0000", 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976898+0000", 2024-09-21T11:47:42.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1528000000000001e-05 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976910+0000", 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1565e-05 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976918+0000", 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7090000000000002e-06 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976966+0000", 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8168999999999999e-05 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.977045+0000", 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9594999999999999e-05 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.977134+0000", 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8233999999999996e-05 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.978395+0000", 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012616459999999999 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.978816+0000", 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000420522 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981299+0000", 2024-09-21T11:47:42.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024827949999999999 2024-09-21T11:47:42.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981332+0000", 2024-09-21T11:47:42.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3321999999999997e-05 2024-09-21T11:47:42.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115141+0000", 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133808599 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43890 8.6 8:68dae88a:::benchmark_data_smithi043_97042_object43889:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 141.368081195, 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13554245000000001, 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.612 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43890 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:47:42.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980005+0000", 2024-09-21T11:47:42.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2470000000000001e-06 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980022+0000", 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7380999999999998e-05 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980030+0000", 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999995e-06 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980035+0000", 2024-09-21T11:47:42.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7090000000000001e-06 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980083+0000", 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7709e-05 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980191+0000", 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010809799999999999 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980327+0000", 2024-09-21T11:47:42.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013611600000000001 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115211+0000", 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13488383300000001 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115245+0000", 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4292000000000003e-05 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115423+0000", 2024-09-21T11:47:42.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017810499999999999 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115476+0000", 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2747000000000002e-05 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115546+0000", 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0159000000000004e-05 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:43886 8.e e172/160 ondisk, result = 0)", 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 141.36686123000001, 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133923551, 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981224+0000", 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5899999999999996e-07 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981227+0000", 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8609999999999998e-06 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981234+0000", 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4039999999999998e-06 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981239+0000", 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3939999999999998e-06 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981291+0000", 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2352000000000003e-05 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981296+0000", 2024-09-21T11:47:42.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6079999999999998e-06 2024-09-21T11:47:42.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115147+0000", 2024-09-21T11:47:42.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133851373 2024-09-21T11:47:42.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.622 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:43882 8.c e172/160 ondisk, result = 0)", 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.982128+0000", 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 141.365956939, 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13297520400000001, 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.982128+0000", 2024-09-21T11:47:42.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.982128+0000", 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.982128+0000", 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2900000000000003e-07 2024-09-21T11:47:42.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.982131+0000", 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1439999999999999e-06 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114868+0000", 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13273664199999999 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114884+0000", 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5569e-05 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114967+0000", 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3372000000000001e-05 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114977+0000", 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6779999999999995e-06 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115103+0000", 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012616999999999999 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:43890 8.6 e172/160 ondisk, result = 0)", 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.983212+0000", 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 141.364872891, 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13234341699999999, 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.627 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983212+0000", 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983212+0000", 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983213+0000", 2024-09-21T11:47:42.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8000000000000005e-07 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983217+0000", 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5879999999999999e-06 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115347+0000", 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.132130321 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115357+0000", 2024-09-21T11:47:42.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0016e-05 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115409+0000", 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1205000000000002e-05 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115417+0000", 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5730000000000008e-06 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115555+0000", 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013793400000000001 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:45844 8.c 8:3052ae9b:::benchmark_data_smithi043_97042_object45843:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:22.263170+0000", 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 140.08491425099999, 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12402154, 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 45844 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.263170+0000", 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.263170+0000", 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.263171+0000", 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599999999999995e-07 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.263193+0000", 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1633000000000001e-05 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.263200+0000", 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5979999999999999e-06 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.263207+0000", 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5980000000000002e-06 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.265399+0000", 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021920809999999998 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.265509+0000", 2024-09-21T11:47:42.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110633 2024-09-21T11:47:42.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:47:42.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.265644+0000", 2024-09-21T11:47:42.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000134828 2024-09-21T11:47:42.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.267779+0000", 2024-09-21T11:47:42.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021343130000000001 2024-09-21T11:47:42.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.269106+0000", 2024-09-21T11:47:42.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001327761 2024-09-21T11:47:42.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.387069+0000", 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11796259000000001 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.387129+0000", 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9799000000000001e-05 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.387192+0000", 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3150000000000004e-05 2024-09-21T11:47:42.637 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:46235 8.2 e173/160 8:4e6f27b7:::benchmark_data_smithi043_97042_object46234:head v 173'2880, mlcod=173'2878)", 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:22.590295+0000", 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 139.75779002900001, 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12607759099999999, 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.638 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590295+0000", 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590295+0000", 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590295+0000", 2024-09-21T11:47:42.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7199999999999998e-07 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590327+0000", 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1486000000000003e-05 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590336+0000", 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9830000000000002e-06 2024-09-21T11:47:42.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590342+0000", 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5169999999999996e-06 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590412+0000", 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9840999999999998e-05 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711199+0000", 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.120787109 2024-09-21T11:47:42.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.716291+0000", 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050917089999999998 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.716372+0000", 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1273999999999995e-05 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.643 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:46239 8.1 8:8e86b97b:::benchmark_data_smithi043_97042_object46238:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:42.643 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:47:42.643 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 139.75732952499999, 2024-09-21T11:47:42.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.144299073, 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46239 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590756+0000", 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590781+0000", 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5000000000000001e-05 2024-09-21T11:47:42.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711205+0000", 2024-09-21T11:47:42.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.120423856 2024-09-21T11:47:42.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711223+0000", 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.751e-05 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711326+0000", 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103274 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711483+0000", 2024-09-21T11:47:42.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015652 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711616+0000", 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000133518 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.715084+0000", 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034678579999999999 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.715128+0000", 2024-09-21T11:47:42.648 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3659000000000003e-05 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.734947+0000", 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019819593999999999 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.735006+0000", 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8415000000000002e-05 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.735054+0000", 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8513000000000001e-05 2024-09-21T11:47:42.649 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.650 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.650 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.650 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:46237 8.2 e173/160 8:4ddff5f6:::benchmark_data_smithi043_97042_object46236:head v 173'2881, mlcod=173'2878)", 2024-09-21T11:47:42.650 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:22.592056+0000", 2024-09-21T11:47:42.650 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 139.75602902700001, 2024-09-21T11:47:42.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124370488, 2024-09-21T11:47:42.650 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.650 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.592056+0000", 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.592056+0000", 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.592057+0000", 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7199999999999997e-07 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.592112+0000", 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5395000000000001e-05 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711204+0000", 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.119092293 2024-09-21T11:47:42.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711221+0000", 2024-09-21T11:47:42.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6863000000000001e-05 2024-09-21T11:47:42.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.712878+0000", 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00165643 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.712896+0000", 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8510000000000001e-05 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.716375+0000", 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034790329999999999 2024-09-21T11:47:42.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.716426+0000", 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0992000000000002e-05 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47773 8.6 8:6c5bd339:::benchmark_data_smithi043_97042_object47772:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 138.64404880199999, 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.141128856, 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47773 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:47:42.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704037+0000", 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-06 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704057+0000", 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0608999999999999e-05 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704068+0000", 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0611000000000001e-05 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704077+0000", 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1710000000000008e-06 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704131+0000", 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3525e-05 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704244+0000", 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113169 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:47:42.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704377+0000", 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013258899999999999 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.706707+0000", 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023301260000000001 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.706973+0000", 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026666 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845071+0000", 2024-09-21T11:47:42.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13809718700000001 2024-09-21T11:47:42.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845116+0000", 2024-09-21T11:47:42.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5581999999999998e-05 2024-09-21T11:47:42.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845165+0000", 2024-09-21T11:47:42.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8603999999999997e-05 2024-09-21T11:47:42.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.662 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.663 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47778 8.e 8:7e5bf474:::benchmark_data_smithi043_97042_object47777:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:42.663 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:47:42.663 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 138.64060385900001, 2024-09-21T11:47:42.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13783190200000001, 2024-09-21T11:47:42.663 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.664 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.664 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.664 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.664 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47778 2024-09-21T11:47:42.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.665 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.665 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.665 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:47:42.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.665 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:47:42.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707482+0000", 2024-09-21T11:47:42.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2750000000000001e-06 2024-09-21T11:47:42.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707504+0000", 2024-09-21T11:47:42.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1684000000000001e-05 2024-09-21T11:47:42.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707515+0000", 2024-09-21T11:47:42.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0844e-05 2024-09-21T11:47:42.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707522+0000", 2024-09-21T11:47:42.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6090000000000003e-06 2024-09-21T11:47:42.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707577+0000", 2024-09-21T11:47:42.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4703000000000001e-05 2024-09-21T11:47:42.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707687+0000", 2024-09-21T11:47:42.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110474 2024-09-21T11:47:42.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:47:42.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707808+0000", 2024-09-21T11:47:42.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012113299999999999 2024-09-21T11:47:42.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845066+0000", 2024-09-21T11:47:42.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.137257564 2024-09-21T11:47:42.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845181+0000", 2024-09-21T11:47:42.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011519 2024-09-21T11:47:42.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845218+0000", 2024-09-21T11:47:42.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6356000000000002e-05 2024-09-21T11:47:42.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845277+0000", 2024-09-21T11:47:42.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9357000000000001e-05 2024-09-21T11:47:42.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845313+0000", 2024-09-21T11:47:42.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5713000000000002e-05 2024-09-21T11:47:42.675 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.675 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.676 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47781 8.6 8:69458834:::benchmark_data_smithi043_97042_object47780:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:42.676 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:47:42.676 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 138.63979336200001, 2024-09-21T11:47:42.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13727709099999999, 2024-09-21T11:47:42.676 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.677 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.677 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47781 2024-09-21T11:47:42.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.678 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:47:42.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:47:42.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708293+0000", 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2440000000000001e-06 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708313+0000", 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0846e-05 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708324+0000", 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0563e-05 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708331+0000", 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0820000000000001e-06 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708421+0000", 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0422999999999998e-05 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708530+0000", 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108286 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708651+0000", 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120821 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845324+0000", 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.136673447 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845358+0000", 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3760999999999997e-05 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845478+0000", 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120512 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845531+0000", 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2714999999999999e-05 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845568+0000", 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7391000000000002e-05 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47784 8.6 8:65eeb18a:::benchmark_data_smithi043_97042_object47783:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 138.63790898299999, 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13626728099999999, 2024-09-21T11:47:42.683 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47784 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:47:42.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710177+0000", 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.229e-06 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.685 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710199+0000", 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469000000000001e-05 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710210+0000", 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0387000000000001e-05 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710217+0000", 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8709999999999999e-06 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710448+0000", 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023145800000000001 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710539+0000", 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0539999999999997e-05 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710618+0000", 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9515e-05 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845369+0000", 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134750694 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845618+0000", 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024913600000000001 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846364+0000", 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074579699999999997 2024-09-21T11:47:42.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846408+0000", 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4425000000000002e-05 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846443+0000", 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4759999999999999e-05 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47773 8.6 e173/160 ondisk, result = 0)", 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 138.637845344, 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134942961, 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710240+0000", 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5600000000000001e-07 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710242+0000", 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2239999999999998e-06 2024-09-21T11:47:42.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710248+0000", 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5949999999999998e-06 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710252+0000", 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3950000000000003e-06 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845062+0000", 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13480961599999999 2024-09-21T11:47:42.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845067+0000", 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1019999999999996e-06 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845182+0000", 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115573 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47778 8.e e173/160 ondisk, result = 0)", 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 138.63779755799999, 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134785869, 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.694 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710290+0000", 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5339999999999998e-06 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710296+0000", 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0970000000000002e-06 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710301+0000", 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7809999999999996e-06 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845053+0000", 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13475179800000001 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845061+0000", 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845073+0000", 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2425999999999999e-05 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47781 8.6 e173/160 ondisk, result = 0)", 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 138.63712711700001, 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13441215100000001, 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710962+0000", 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.692e-06 2024-09-21T11:47:42.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842829+0000", 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13186700100000001 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842844+0000", 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5281000000000002e-05 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845347+0000", 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002502659 2024-09-21T11:47:42.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845353+0000", 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6510000000000004e-06 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845370+0000", 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6214000000000001e-05 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47792 8.e 8:71619300:::benchmark_data_smithi043_97042_object47791:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:47:42.701 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 138.63489958700001, 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14195813500000001, 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47792 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713186+0000", 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.703 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713211+0000", 2024-09-21T11:47:42.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4343999999999999e-05 2024-09-21T11:47:42.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.843341+0000", 2024-09-21T11:47:42.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13012984499999999 2024-09-21T11:47:42.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.843353+0000", 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2554e-05 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845654+0000", 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023008730000000002 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845787+0000", 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013336700000000001 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845921+0000", 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000134031 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.853059+0000", 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071370510000000002 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.853133+0000", 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4765000000000007e-05 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855050+0000", 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019165880000000001 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855099+0000", 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8680000000000001e-05 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855143+0000", 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4626000000000003e-05 2024-09-21T11:47:42.707 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.708 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.708 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.708 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:66142 8.1 8:87e3d573:::benchmark_data_smithi043_97042_object66141:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:47:42.708 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:47:42.708 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 126.421859197, 2024-09-21T11:47:42.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13618391299999999, 2024-09-21T11:47:42.708 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 66142 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926226+0000", 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3500000000000005e-07 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926267+0000", 2024-09-21T11:47:42.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0932e-05 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926277+0000", 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8609999999999993e-06 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926283+0000", 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4010000000000002e-06 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926387+0000", 2024-09-21T11:47:42.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103999 2024-09-21T11:47:42.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:42.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926630+0000", 2024-09-21T11:47:42.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000242938 2024-09-21T11:47:42.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.712 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:47:42.712 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926754+0000", 2024-09-21T11:47:42.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123433 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.930232+0000", 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034784540000000002 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.930829+0000", 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059640099999999998 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:47:42.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.059677+0000", 2024-09-21T11:47:42.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12884799699999999 2024-09-21T11:47:42.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:47:42.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.059734+0000", 2024-09-21T11:47:42.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7191999999999997e-05 2024-09-21T11:47:42.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:42.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:42.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:47:42.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.062409+0000", 2024-09-21T11:47:42.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026754700000000001 2024-09-21T11:47:42.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.715 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:42.715 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:42.715 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:47:42.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:42.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/569145233' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:42.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:42.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/569145233' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:42.760 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:42.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:42.761 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/569145233' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:42.861 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:42.892 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:42.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:42.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:42.994 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:43.020 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:43.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:43.021 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:43.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:47:43.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:43.172 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:43.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:43.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:43.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:43.300 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:43.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:43.302 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:43.403 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:43.431 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:43.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:43.432 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:43.532 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:43.558 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:43.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:43.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:43.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:43.681 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:43.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:43.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:43 smithi043 ceph-mon[36454]: pgmap v363: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 18826/188889 objects misplaced (9.967%); 20 MiB/s, 319 objects/s recovering 2024-09-21T11:47:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:43 smithi043 ceph-mon[29710]: pgmap v363: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 18826/188889 objects misplaced (9.967%); 20 MiB/s, 319 objects/s recovering 2024-09-21T11:47:43.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:43 smithi175 ceph-mon[32613]: pgmap v363: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 18826/188889 objects misplaced (9.967%); 20 MiB/s, 319 objects/s recovering 2024-09-21T11:47:43.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:43.826 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:43.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:43.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:43.927 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:43.956 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:43.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:43.957 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:44.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:44.078 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:44.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:44.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:44.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:44.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:44.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:44.321 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:44.346 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:44.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:44.347 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:44.448 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:44.473 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:44.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:44.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:44.574 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:44.598 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:44.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:44.599 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:44.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:44.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:44.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:44.844 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:44.874 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:44.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:44.875 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:44.975 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:45.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:45.003 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:45.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:45.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:45.126 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:45.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:45.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:45.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:45.274 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:45.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:45.275 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:45.376 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:45.397 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:45.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:45.398 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:45.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:47:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:47:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:47:45.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:45.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:45.555 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:45.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:45.693 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:45.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:45.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:45 smithi175 ceph-mon[32613]: pgmap v364: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 910 op/s; 18826/188889 objects misplaced (9.967%); 17 MiB/s, 270 objects/s recovering 2024-09-21T11:47:45.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:45.825 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:45.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:45.826 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:45.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:45.948 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:45 smithi043 ceph-mon[29710]: pgmap v364: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 910 op/s; 18826/188889 objects misplaced (9.967%); 17 MiB/s, 270 objects/s recovering 2024-09-21T11:47:45.949 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:45 smithi043 ceph-mon[36454]: pgmap v364: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 910 op/s; 18826/188889 objects misplaced (9.967%); 17 MiB/s, 270 objects/s recovering 2024-09-21T11:47:45.955 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:45.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:45.990 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:46.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:46.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:47:46.186 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:46.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:46.187 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:46.288 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:46.321 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:46.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:46.322 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:46.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:46.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:46.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:46.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:46.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:46.622 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:46.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:46.723 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:46.744 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:46.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:46.745 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:46.845 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:46.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:46.883 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:46.883 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:46.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:47.005 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:47.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:47.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:47.107 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:47.138 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:47.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:47.139 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:47.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:47.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:46 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:47.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:47.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:46 smithi043 ceph-mon[29710]: pgmap v365: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 818 op/s; 18826/188142 objects misplaced (10.006%); 15 MiB/s, 236 objects/s recovering 2024-09-21T11:47:47.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:46 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:47.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:47.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:46 smithi043 ceph-mon[36454]: pgmap v365: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 818 op/s; 18826/188142 objects misplaced (10.006%); 15 MiB/s, 236 objects/s recovering 2024-09-21T11:47:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:46 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:47:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:46 smithi175 ceph-mon[32613]: pgmap v365: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 818 op/s; 18826/188142 objects misplaced (10.006%); 15 MiB/s, 236 objects/s recovering 2024-09-21T11:47:47.291 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:47.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:47.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:47.393 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:47.425 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:47.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:47.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:47.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:47.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:47.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:47.674 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:47.702 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:47.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:47.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:47.804 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:47.831 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:47.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:47.831 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:47.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:47.962 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:47.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:47.963 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:48.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:47:48.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:48.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:48.130 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:48.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:48.231 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:48.257 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:48.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:48.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:48.259 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:48.277 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T11:47:48.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:48.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:48.420 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:48 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1190428936' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:47:48.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:48 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:47:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:48 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1190428936' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:47:48.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:48 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:47:48.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:48.548 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:48.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:48.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:48.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:48.699 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:48.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:48.700 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:48.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:48 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1190428936' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:47:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:48 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:47:48.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:48.826 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:48.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:48.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:48.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:48.954 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:48.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:48.955 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:49.056 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:49.084 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:49.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:49.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:49.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 7 1 2024-09-21T11:47:49.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:49.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:49.267 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:49.267 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:49.351 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:49 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:47:49.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:49 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:47:49.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:49 smithi043 ceph-mon[29710]: pgmap v366: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.08k op/s; 16479/176514 objects misplaced (9.336%); 20 MiB/s, 319 objects/s recovering 2024-09-21T11:47:49.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:49 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:47:49.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:49 smithi043 ceph-mon[29710]: osdmap e218: 8 total, 8 up, 8 in 2024-09-21T11:47:49.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:49 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:47:49.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:49 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:47:49.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:49 smithi043 ceph-mon[36454]: pgmap v366: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.08k op/s; 16479/176514 objects misplaced (9.336%); 20 MiB/s, 319 objects/s recovering 2024-09-21T11:47:49.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:49 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:47:49.353 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:49 smithi043 ceph-mon[36454]: osdmap e218: 8 total, 8 up, 8 in 2024-09-21T11:47:49.368 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:49.391 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:49.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:49.392 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:49.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:49.517 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:49.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:49.517 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:49.618 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:49.626 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:49.644 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:49.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:49.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:49 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:47:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:49 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:47:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:49 smithi175 ceph-mon[32613]: pgmap v366: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.08k op/s; 16479/176514 objects misplaced (9.336%); 20 MiB/s, 319 objects/s recovering 2024-09-21T11:47:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:49 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:47:49.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:49 smithi175 ceph-mon[32613]: osdmap e218: 8 total, 8 up, 8 in 2024-09-21T11:47:49.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:49.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:49.789 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:49.789 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:49.890 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:49.923 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:49.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:49.924 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:49.961 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:50.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:50.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:50.086 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:50.087 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:50.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:50.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:50.227 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:50.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:47:50.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:50.374 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:50.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:50.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:50.476 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:50.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:50.511 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:50.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:50.530 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4216426524' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:50.530 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4216426524' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:50.532 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:47:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:47:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:219143 8.f e219/203 8:f2e85789:::benchmark_data_smithi043_97042_object82049:head v 219'13608, mlcod=219'13605)", 2024-09-21T11:47:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:50.520749+0000", 2024-09-21T11:47:50.532 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0078668430000000001, 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0078955380000000006, 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.520749+0000", 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.520749+0000", 2024-09-21T11:47:50.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.520751+0000", 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.327e-06 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.520758+0000", 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9310000000000002e-06 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:50.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.520770+0000", 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2741e-05 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.520778+0000", 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1500000000000002e-06 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.520926+0000", 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014837499999999999 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:50.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.520953+0000", 2024-09-21T11:47:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7327000000000001e-05 2024-09-21T11:47:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:50.536 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:219148 8.3 e219/203 8:cf939b88:::benchmark_data_smithi043_97042_object82054:head v 219'13786, mlcod=219'13784)", 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:50.523420+0000", 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.005195987, 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052816679999999998, 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.523420+0000", 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.523420+0000", 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.523421+0000", 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1200000000000002e-07 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.523427+0000", 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8429999999999998e-06 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.523436+0000", 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4490000000000005e-06 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.523443+0000", 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5679999999999996e-06 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.523523+0000", 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9801000000000004e-05 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:47:50.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:47:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.523542+0000", 2024-09-21T11:47:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9689000000000001e-05 2024-09-21T11:47:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:47:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:47:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:47:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 2 2024-09-21T11:47:50.541 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:47:50.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:50.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:50.675 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:50.675 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4216426524' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:47:50.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:50.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:50.811 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:50.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:50.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:50.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:50.963 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:50.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:51.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:51.131 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:51.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:51.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:51.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:51.260 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:51.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:51.261 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:51.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[29710]: pgmap v368: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 881 op/s; 16479/176514 objects misplaced (9.336%); 16 MiB/s, 252 objects/s recovering 2024-09-21T11:47:51.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:51.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4216426524' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:47:51.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[29710]: osdmap e219: 8 total, 8 up, 8 in 2024-09-21T11:47:51.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:47:51.318 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/740428835' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[36454]: pgmap v368: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 881 op/s; 16479/176514 objects misplaced (9.336%); 16 MiB/s, 252 objects/s recovering 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4216426524' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[36454]: osdmap e219: 8 total, 8 up, 8 in 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:51.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:51 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/740428835' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:51.362 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:51.390 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:51.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:51.391 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:51.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:51 smithi175 ceph-mon[32613]: pgmap v368: 148 pgs: 1 active+remapped+backfill_wait, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.6 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 881 op/s; 16479/176514 objects misplaced (9.336%); 16 MiB/s, 252 objects/s recovering 2024-09-21T11:47:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:51 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:51 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4216426524' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:47:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:51 smithi175 ceph-mon[32613]: osdmap e219: 8 total, 8 up, 8 in 2024-09-21T11:47:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:51 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:47:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:51 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:47:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:51 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:47:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:51 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:47:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:51 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/740428835' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:51.512 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:47:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:47:51.516 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:51.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:51.517 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:51.618 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:51.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:51.651 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:51.751 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:51.773 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:51.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:51.773 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:51.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:51.904 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:51.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:51.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:52.006 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:52.030 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:52.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:52.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:52.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:52.161 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:52.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:52.162 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:52.262 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:52.286 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:52.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:52.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:52.296 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:52.347 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:47:52.388 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:52.400 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:47:52.400 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:47:52.400 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:47:52.400 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:47:52.400 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:47:52.401 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:47:52.414 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:52.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:52.415 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:52.516 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:52.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:52.538 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:52.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:52.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:52.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:52.672 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:52 smithi043 ceph-mon[29710]: osdmap e220: 8 total, 8 up, 8 in 2024-09-21T11:47:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:52 smithi043 ceph-mon[29710]: osdmap e221: 8 total, 8 up, 8 in 2024-09-21T11:47:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:52.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:52 smithi043 ceph-mon[36454]: osdmap e220: 8 total, 8 up, 8 in 2024-09-21T11:47:52.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:52 smithi043 ceph-mon[36454]: osdmap e221: 8 total, 8 up, 8 in 2024-09-21T11:47:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:47:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:52 smithi175 ceph-mon[32613]: osdmap e220: 8 total, 8 up, 8 in 2024-09-21T11:47:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:52 smithi175 ceph-mon[32613]: osdmap e221: 8 total, 8 up, 8 in 2024-09-21T11:47:52.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:52.801 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:52.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:52.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:52.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:52.924 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:52.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:52.925 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:53.026 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:53.057 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:53.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:53.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:53.159 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:53.181 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:53.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:53.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:53.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:53.314 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:53.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:53.315 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:53.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:53.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:53.451 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:53.451 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:53.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:53.585 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:53.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:53.586 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:53.609 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:53 smithi043 ceph-mon[36454]: pgmap v371: 148 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.34k op/s; 13957/163932 objects misplaced (8.514%); 27 MiB/s, 428 objects/s recovering 2024-09-21T11:47:53.609 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:53 smithi043 ceph-mon[36454]: osdmap e222: 8 total, 8 up, 8 in 2024-09-21T11:47:53.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:53 smithi043 ceph-mon[29710]: pgmap v371: 148 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.34k op/s; 13957/163932 objects misplaced (8.514%); 27 MiB/s, 428 objects/s recovering 2024-09-21T11:47:53.610 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:53 smithi043 ceph-mon[29710]: osdmap e222: 8 total, 8 up, 8 in 2024-09-21T11:47:53.687 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:53.709 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:53.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:53.709 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:53 smithi175 ceph-mon[32613]: pgmap v371: 148 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.34k op/s; 13957/163932 objects misplaced (8.514%); 27 MiB/s, 428 objects/s recovering 2024-09-21T11:47:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:53 smithi175 ceph-mon[32613]: osdmap e222: 8 total, 8 up, 8 in 2024-09-21T11:47:53.775 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:53.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:53.846 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:53.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:53.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:53.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:47:53.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:53.980 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:53.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:53.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:54.082 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:54.103 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:54.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:54.104 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:54.205 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:54.226 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:54.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:54.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:54.259 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T11:47:54.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:54.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:54.367 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:54.407 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:47:54.407 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:47:54.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:54.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:54 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4051069004' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:47:54.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:54 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4051069004' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:47:54.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:54 smithi043 ceph-mon[36454]: osdmap e223: 8 total, 8 up, 8 in 2024-09-21T11:47:54.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:54 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4051069004' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:47:54.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:54 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4051069004' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:47:54.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:54 smithi043 ceph-mon[29710]: osdmap e223: 8 total, 8 up, 8 in 2024-09-21T11:47:54.512 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:54.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:54.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:54.614 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:54.640 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:54.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:54.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:54.742 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:54.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:54 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4051069004' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:47:54.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:54 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4051069004' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:47:54.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:54 smithi175 ceph-mon[32613]: osdmap e223: 8 total, 8 up, 8 in 2024-09-21T11:47:54.768 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:54.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:54.769 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:54.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:54.910 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:54.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:54.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:55.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:55.050 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:55.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:55.051 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:55.152 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:55.178 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:55.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:55.179 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:55.280 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:55.312 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:55.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:55.313 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:55.413 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:55.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:55.441 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:55.441 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:55.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:47:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:47:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:47:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:55 smithi043 ceph-mon[29710]: pgmap v374: 148 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 13957/163932 objects misplaced (8.514%); 23 MiB/s, 360 objects/s recovering 2024-09-21T11:47:55.495 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:47:55.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:55 smithi043 ceph-mon[36454]: pgmap v374: 148 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 13957/163932 objects misplaced (8.514%); 23 MiB/s, 360 objects/s recovering 2024-09-21T11:47:55.543 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:55.544 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 1 1 2024-09-21T11:47:55.574 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:55.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:55.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:55.675 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:55.702 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:55.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:55.703 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:55.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:55 smithi175 ceph-mon[32613]: pgmap v374: 148 pgs: 1 active+clean+remapped, 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 141 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 13957/163932 objects misplaced (8.514%); 23 MiB/s, 360 objects/s recovering 2024-09-21T11:47:55.804 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:55.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:55.879 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:55.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:55.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:56.007 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:56.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:56.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:56.109 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:56.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:47:56.134 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:56.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:56.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:56.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:56.257 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:56.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:56.258 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:56.262 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:56.337 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:47:56.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:56.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:56.411 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:56.411 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:56.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:56.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:56.586 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:56.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:56.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:56.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:56.727 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:56.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:56.864 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:56.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:56.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:56.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:57.012 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:57.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:57.013 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:57.114 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:57.156 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:57.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:57.157 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:57.258 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:57.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:47:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:47:57.283 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:57.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:57.284 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:57.317 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:47:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:47:57.356 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:47:57.356 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:47:57.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:57.401 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:47:57.402 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:47:57.436 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:47:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:47:57.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:57.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:57.469 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:47:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:47:57.508 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:47:57.509 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:47:57.538 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:57.549 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:47:57.550 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:47:57.563 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:57.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:57.564 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:57.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:57 smithi043 ceph-mon[29710]: pgmap v376: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 142 active+clean; 3.3 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 464 op/s; 13957/163029 objects misplaced (8.561%); 15 MiB/s, 235 objects/s recovering 2024-09-21T11:47:57.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3455358486' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:57.635 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:57 smithi043 ceph-mon[36454]: pgmap v376: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 142 active+clean; 3.3 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 464 op/s; 13957/163029 objects misplaced (8.561%); 15 MiB/s, 235 objects/s recovering 2024-09-21T11:47:57.635 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:57 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3455358486' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:57.664 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:57.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:57.692 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:57.692 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:57.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:57 smithi175 ceph-mon[32613]: pgmap v376: 148 pgs: 3 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 142 active+clean; 3.3 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 464 op/s; 13957/163029 objects misplaced (8.561%); 15 MiB/s, 235 objects/s recovering 2024-09-21T11:47:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:57 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3455358486' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:47:57.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:47:57.838 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:57.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:57.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:47:57.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:57.964 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:57.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:57.965 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:58.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:58.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:58.108 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:58.108 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:58.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:58.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:58.256 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:58.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:58.362 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:47:58.363 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T11:47:58.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:58 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3105900705' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:47:58.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:58 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:47:58.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:58 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/194061573' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:47:58.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:58 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:47:58.382 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:58 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3105900705' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:47:58.382 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:58 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:47:58.382 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:58 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/194061573' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:47:58.382 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:58 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:47:58.383 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:58.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:58.385 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:47:58.485 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:47:58.512 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:58.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:58.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:47:58.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:58.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:58.654 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:58.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:58.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:58.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:58 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3105900705' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:47:58.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:58 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:47:58.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:58 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/194061573' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:47:58.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:58 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:47:58.799 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:58.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:58.800 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:58.901 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:47:58.930 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:58.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:58.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:47:59.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:59.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:59.075 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:59.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:47:59.220 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:59.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:59.221 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:47:59.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:59.375 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:59.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:59.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:59.477 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:47:59.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:59.518 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:59.519 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:47:59.620 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:47:59.648 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:59.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:59.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:47:59.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[29710]: pgmap v377: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 424 op/s; 9232/156303 objects misplaced (5.906%); 44 MiB/s, 709 objects/s recovering 2024-09-21T11:47:59.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[29710]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:47:59.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:59.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:47:59.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:47:59.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[29710]: osdmap e224: 8 total, 8 up, 8 in 2024-09-21T11:47:59.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[36454]: pgmap v377: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 424 op/s; 9232/156303 objects misplaced (5.906%); 44 MiB/s, 709 objects/s recovering 2024-09-21T11:47:59.656 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[36454]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:47:59.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:59.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:47:59.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:47:59.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:47:59 smithi043 ceph-mon[36454]: osdmap e224: 8 total, 8 up, 8 in 2024-09-21T11:47:59.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:47:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:59 smithi175 ceph-mon[32613]: pgmap v377: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 424 op/s; 9232/156303 objects misplaced (5.906%); 44 MiB/s, 709 objects/s recovering 2024-09-21T11:47:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:59 smithi175 ceph-mon[32613]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:47:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:59 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:47:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:59 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:47:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:59 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:47:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:47:59 smithi175 ceph-mon[32613]: osdmap e224: 8 total, 8 up, 8 in 2024-09-21T11:47:59.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:59.795 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:47:59.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:47:59.896 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:47:59.922 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:47:59.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:47:59.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:00.024 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:00.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:00.052 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:00.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:00.192 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:00.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:00.194 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:00.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:00.339 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:00.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:00.340 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:00.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:00.468 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:00.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:00.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:00.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:00.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:00.618 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:00.618 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:00.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:48:00.719 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:00.762 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:00.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:00.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:00.864 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:00.889 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:00.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:00.890 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:00.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:01.014 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:01.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:01.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:01.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:01.161 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:01.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:01.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:01.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:01.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:01.286 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:01.286 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:01.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:48:01.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:01.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:01.424 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:01.425 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:01.459 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:01 smithi175 ceph-mon[32613]: pgmap v379: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 382 op/s; 9232/156303 objects misplaced (5.906%); 40 MiB/s, 640 objects/s recovering 2024-09-21T11:48:01.459 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:01.459 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:48:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:48:01.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:01.551 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:01.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:01.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:01.652 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:01.676 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:01.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:01.677 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:01.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:01 smithi043 ceph-mon[29710]: pgmap v379: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 382 op/s; 9232/156303 objects misplaced (5.906%); 40 MiB/s, 640 objects/s recovering 2024-09-21T11:48:01.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:01 smithi043 ceph-mon[36454]: pgmap v379: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 382 op/s; 9232/156303 objects misplaced (5.906%); 40 MiB/s, 640 objects/s recovering 2024-09-21T11:48:01.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:01.778 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:01.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:01.808 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:01.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:01.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:01.935 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:01.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:01.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:02.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:02.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:02.061 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:02.061 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:02.073 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T11:48:02.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:48:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:02.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:02.186 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:02.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:02.187 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:02.288 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:02.316 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:02.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:02.317 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:02.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:02.474 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:02.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:02.475 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:02.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:02.620 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:02.622 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:02.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:02.641 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:02.641 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:02.742 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:02.747 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:02.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:02.769 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:02.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:02.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:02.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:02.916 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:48:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:48:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:48:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:48:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:37277 7.b 7:d13dff0d:::benchmark_data_smithi043_70646_object37276:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-21T11:48:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:41:35.480704+0000", 2024-09-21T11:48:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 387.42945689499999, 2024-09-21T11:48:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036890579, 2024-09-21T11:48:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:48:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:02.918 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 37277 2024-09-21T11:48:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480704+0000", 2024-09-21T11:48:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:02.919 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480704+0000", 2024-09-21T11:48:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480706+0000", 2024-09-21T11:48:02.920 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0339999999999999e-06 2024-09-21T11:48:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480728+0000", 2024-09-21T11:48:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1526e-05 2024-09-21T11:48:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480738+0000", 2024-09-21T11:48:02.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0083e-05 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.480745+0000", 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9970000000000001e-06 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481292+0000", 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00054686700000000003 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481399+0000", 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107378 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.481563+0000", 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000163833 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484047+0000", 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024838680000000002 2024-09-21T11:48:02.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.484084+0000", 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7339000000000003e-05 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517265+0000", 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.033180236000000002 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517535+0000", 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00027041599999999999 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:41:35.517595+0000", 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0001999999999999e-05 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.925 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:92195 7.b 7:dccd3640:::benchmark_data_smithi043_70646_object92194:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:48:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:19.376248+0000", 2024-09-21T11:48:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 343.53391334499997, 2024-09-21T11:48:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043184081999999999, 2024-09-21T11:48:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:02.926 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:48:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 92195 2024-09-21T11:48:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:02.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376248+0000", 2024-09-21T11:48:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:02.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376248+0000", 2024-09-21T11:48:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376250+0000", 2024-09-21T11:48:02.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.809e-06 2024-09-21T11:48:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376279+0000", 2024-09-21T11:48:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9685000000000001e-05 2024-09-21T11:48:02.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376294+0000", 2024-09-21T11:48:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4382000000000001e-05 2024-09-21T11:48:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.376302+0000", 2024-09-21T11:48:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7489999999999998e-06 2024-09-21T11:48:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:02.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.381875+0000", 2024-09-21T11:48:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055737729999999997 2024-09-21T11:48:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.382223+0000", 2024-09-21T11:48:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00034789100000000002 2024-09-21T11:48:02.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:48:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.382499+0000", 2024-09-21T11:48:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00027602900000000001 2024-09-21T11:48:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:02.934 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394336+0000", 2024-09-21T11:48:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011837089 2024-09-21T11:48:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394380+0000", 2024-09-21T11:48:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3791999999999997e-05 2024-09-21T11:48:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419127+0000", 2024-09-21T11:48:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.024747149 2024-09-21T11:48:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419380+0000", 2024-09-21T11:48:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00025246999999999997 2024-09-21T11:48:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419432+0000", 2024-09-21T11:48:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2264e-05 2024-09-21T11:48:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:92197 7.b 7:d6014428:::benchmark_data_smithi043_70646_object92196:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:48:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:19.377823+0000", 2024-09-21T11:48:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 343.53233806200001, 2024-09-21T11:48:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041873795999999998, 2024-09-21T11:48:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:02.938 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 92197 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377823+0000", 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:02.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377823+0000", 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377825+0000", 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.316e-06 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377847+0000", 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2942000000000002e-05 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:02.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377863+0000", 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5571999999999999e-05 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377871+0000", 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6830000000000008e-06 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.386479+0000", 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0086077989999999993 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.386599+0000", 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012036699999999999 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.389012+0000", 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024130559999999998 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394424+0000", 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054119509999999999 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394464+0000", 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0262000000000002e-05 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419493+0000", 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.025028741 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419655+0000", 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000162349 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419697+0000", 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1758000000000002e-05 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:92198 7.b 7:de413604:::benchmark_data_smithi043_70646_object92197:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:19.377883+0000", 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 343.53227802700002, 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042019793999999999, 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:02.944 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 92198 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377883+0000", 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377883+0000", 2024-09-21T11:48:02.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377884+0000", 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1399999999999997e-07 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377904+0000", 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0386000000000001e-05 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377910+0000", 2024-09-21T11:48:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3219999999999998e-06 2024-09-21T11:48:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:02.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.377916+0000", 2024-09-21T11:48:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1950000000000002e-06 2024-09-21T11:48:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394496+0000", 2024-09-21T11:48:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016580785000000001 2024-09-21T11:48:02.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394620+0000", 2024-09-21T11:48:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123736 2024-09-21T11:48:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:48:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.394768+0000", 2024-09-21T11:48:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147365 2024-09-21T11:48:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.410416+0000", 2024-09-21T11:48:02.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015648695000000001 2024-09-21T11:48:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.410469+0000", 2024-09-21T11:48:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2683999999999999e-05 2024-09-21T11:48:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419710+0000", 2024-09-21T11:48:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092409359999999999 2024-09-21T11:48:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:02.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419862+0000", 2024-09-21T11:48:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015258599999999999 2024-09-21T11:48:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:19.419903+0000", 2024-09-21T11:48:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0590000000000003e-05 2024-09-21T11:48:02.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:94692 7.9 e105/82 7:92d687ee:::benchmark_data_smithi043_70646_object94691:head v 105'5840, mlcod=105'5838)", 2024-09-21T11:48:02.954 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:21.710418+0000", 2024-09-21T11:48:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 341.19974295600002, 2024-09-21T11:48:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043836149999999997, 2024-09-21T11:48:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:02.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710418+0000", 2024-09-21T11:48:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710418+0000", 2024-09-21T11:48:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710419+0000", 2024-09-21T11:48:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.84e-07 2024-09-21T11:48:02.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710450+0000", 2024-09-21T11:48:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1259e-05 2024-09-21T11:48:02.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710457+0000", 2024-09-21T11:48:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2540000000000003e-06 2024-09-21T11:48:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.710463+0000", 2024-09-21T11:48:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.451e-06 2024-09-21T11:48:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.712306+0000", 2024-09-21T11:48:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001843021 2024-09-21T11:48:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:02.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.712326+0000", 2024-09-21T11:48:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0322999999999998e-05 2024-09-21T11:48:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.754204+0000", 2024-09-21T11:48:02.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041877978000000003 2024-09-21T11:48:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.754254+0000", 2024-09-21T11:48:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0479999999999998e-05 2024-09-21T11:48:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.964 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:94698 7.9 e105/82 7:9acdce21:::benchmark_data_smithi043_70646_object94697:head v 105'5841, mlcod=105'5838)", 2024-09-21T11:48:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:21.717113+0000", 2024-09-21T11:48:02.965 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 341.19304782699999, 2024-09-21T11:48:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.038293739, 2024-09-21T11:48:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:02.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717113+0000", 2024-09-21T11:48:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:02.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717113+0000", 2024-09-21T11:48:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717115+0000", 2024-09-21T11:48:02.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.181e-06 2024-09-21T11:48:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717141+0000", 2024-09-21T11:48:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6837000000000001e-05 2024-09-21T11:48:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717153+0000", 2024-09-21T11:48:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1262e-05 2024-09-21T11:48:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:02.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.717160+0000", 2024-09-21T11:48:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3669999999999999e-06 2024-09-21T11:48:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.719727+0000", 2024-09-21T11:48:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025664730000000001 2024-09-21T11:48:02.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.719850+0000", 2024-09-21T11:48:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012304899999999999 2024-09-21T11:48:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.754262+0000", 2024-09-21T11:48:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034412700999999997 2024-09-21T11:48:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:21.755407+0000", 2024-09-21T11:48:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001144869 2024-09-21T11:48:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:96882 7.0 e106/82 7:0b4ad799:::benchmark_data_smithi043_70646_object96881:head v 106'6094, mlcod=106'6090)", 2024-09-21T11:48:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:23.906882+0000", 2024-09-21T11:48:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 339.003279572, 2024-09-21T11:48:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.046078277000000001, 2024-09-21T11:48:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:02.975 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906882+0000", 2024-09-21T11:48:02.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906882+0000", 2024-09-21T11:48:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906883+0000", 2024-09-21T11:48:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2999999999999999e-07 2024-09-21T11:48:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906918+0000", 2024-09-21T11:48:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5533e-05 2024-09-21T11:48:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906935+0000", 2024-09-21T11:48:02.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6473999999999998e-05 2024-09-21T11:48:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.906943+0000", 2024-09-21T11:48:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0430000000000008e-06 2024-09-21T11:48:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.911336+0000", 2024-09-21T11:48:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004393798 2024-09-21T11:48:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.911401+0000", 2024-09-21T11:48:02.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4611999999999997e-05 2024-09-21T11:48:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.952912+0000", 2024-09-21T11:48:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.041511406000000001 2024-09-21T11:48:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.952960+0000", 2024-09-21T11:48:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7481000000000002e-05 2024-09-21T11:48:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.14805.0:96890 7.0 e106/82 7:0e02357d:::benchmark_data_smithi043_70646_object96889:head v 106'6095, mlcod=106'6090)", 2024-09-21T11:48:02.983 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:23.916357+0000", 2024-09-21T11:48:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 338.99380464299998, 2024-09-21T11:48:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036631786999999999, 2024-09-21T11:48:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:02.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916357+0000", 2024-09-21T11:48:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916357+0000", 2024-09-21T11:48:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:02.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916358+0000", 2024-09-21T11:48:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6500000000000008e-07 2024-09-21T11:48:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916454+0000", 2024-09-21T11:48:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6772000000000001e-05 2024-09-21T11:48:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916465+0000", 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0536999999999999e-05 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.916473+0000", 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7389999999999999e-06 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.919650+0000", 2024-09-21T11:48:02.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031771780000000001 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.919713+0000", 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3551000000000002e-05 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.952967+0000", 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.033253222999999998 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:23.952988+0000", 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1821999999999999e-05 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:100678 7.b 7:d782bdfb:::benchmark_data_smithi043_70646_object100677:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:28.332750+0000", 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 334.57741103299998, 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.044281528000000001, 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:02.989 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:48:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 100678 2024-09-21T11:48:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332750+0000", 2024-09-21T11:48:02.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332750+0000", 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332751+0000", 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7199999999999999e-07 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332772+0000", 2024-09-21T11:48:02.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1446000000000001e-05 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332779+0000", 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.17e-06 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.332785+0000", 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0240000000000003e-06 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.335754+0000", 2024-09-21T11:48:02.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002968498 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.335876+0000", 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012237399999999999 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.336024+0000", 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014732999999999999 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.341309+0000", 2024-09-21T11:48:02.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052852929999999999 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.342298+0000", 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098862199999999998 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.376908+0000", 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034610502000000001 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.376980+0000", 2024-09-21T11:48:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1679000000000003e-05 2024-09-21T11:48:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:02.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:28.377032+0000", 2024-09-21T11:48:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2018000000000002e-05 2024-09-21T11:48:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:101832 7.b 7:dcbd601b:::benchmark_data_smithi043_70646_object101831:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:48:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:29.648649+0000", 2024-09-21T11:48:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 333.26151276399997, 2024-09-21T11:48:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.052835252999999999, 2024-09-21T11:48:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:02.997 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 101832 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648649+0000", 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648649+0000", 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648650+0000", 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.778e-06 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648675+0000", 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4752e-05 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:02.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648690+0000", 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4932e-05 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.648697+0000", 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5070000000000003e-06 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.649977+0000", 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012800159999999999 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.650104+0000", 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000126236 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.650252+0000", 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014790499999999999 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.655289+0000", 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050370090000000003 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.655495+0000", 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020639200000000001 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.701323+0000", 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045828277000000001 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.701429+0000", 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010541 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.701484+0000", 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5038999999999999e-05 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:102078 7.b 7:df4f0289:::benchmark_data_smithi043_70646_object102077:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:29.972683+0000", 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 332.93747783100002, 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.040813940999999999, 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:03.003 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 102078 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972683+0000", 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972683+0000", 2024-09-21T11:48:03.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972685+0000", 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.756e-06 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972708+0000", 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2775000000000001e-05 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972720+0000", 2024-09-21T11:48:03.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2408999999999999e-05 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.972728+0000", 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8790000000000003e-06 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.978889+0000", 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061604269999999996 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.979014+0000", 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012558 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.979160+0000", 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014582899999999999 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.991453+0000", 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012293282000000001 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:29.994408+0000", 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029544100000000002 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.013365+0000", 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01895728 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.013432+0000", 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6841999999999995e-05 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.013497+0000", 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5471999999999999e-05 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:102215 7.b 7:d049ffb3:::benchmark_data_smithi043_70646_object102214:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:30.137277+0000", 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 332.77288381, 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.039707500999999999, 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:03.009 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 102215 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137277+0000", 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137277+0000", 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137279+0000", 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9970000000000001e-06 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137306+0000", 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6928e-05 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137324+0000", 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7448e-05 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137333+0000", 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.414e-06 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.137998+0000", 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00066516999999999998 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.138109+0000", 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110862 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.138265+0000", 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000155384 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.169637+0000", 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.031372672999999997 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.169671+0000", 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4156000000000003e-05 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.176870+0000", 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071988649999999996 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.176924+0000", 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3944999999999999e-05 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.176985+0000", 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0659000000000003e-05 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.014 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:03.016 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:03.016 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:102266 7.b 7:d1444c05:::benchmark_data_smithi043_70646_object102265:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108)", 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:30.193280+0000", 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 332.71688165799998, 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.037717675999999999, 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 102266 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:03.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193280+0000", 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193280+0000", 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193282+0000", 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.153e-06 2024-09-21T11:48:03.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193309+0000", 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6948e-05 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193327+0000", 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8445999999999999e-05 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.193337+0000", 2024-09-21T11:48:03.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6840000000000008e-06 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.204924+0000", 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011586905999999999 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.205037+0000", 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011285099999999999 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:48:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.205177+0000", 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000140803 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.211726+0000", 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0065485650000000001 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.230657+0000", 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018930939000000001 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.230719+0000", 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1834000000000005e-05 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.230907+0000", 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000188121 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:30.230997+0000", 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0426000000000005e-05 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:218712 8.1 e218/203 8:8c7de8f4:::benchmark_data_smithi043_97042_object81618:head v 218'13887, mlcod=218'13882)", 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:49.993664+0000", 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 12.916497583, 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.043691104000000001, 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:49.993664+0000", 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:49.993664+0000", 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:49.993664+0000", 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4999999999999998e-07 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:49.993668+0000", 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5250000000000001e-06 2024-09-21T11:48:03.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:49.993673+0000", 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7529999999999998e-06 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:49.993678+0000", 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8999999999999997e-06 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.010796+0000", 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.017117963999999999 2024-09-21T11:48:03.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.010851+0000", 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4348999999999998e-05 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.037125+0000", 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.026274226000000001 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:50.037355+0000", 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023003699999999999 2024-09-21T11:48:03.026 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:224289 8.0 e224/221 8:05748f85:::benchmark_data_smithi043_97042_object87195:head v 224'13940, mlcod=224'13937)", 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:58.593022+0000", 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4.31713916, 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.038483313999999998, 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:03.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:58.593022+0000", 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:58.593022+0000", 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:58.593023+0000", 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9999999999999995e-07 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:03.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:58.593028+0000", 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3550000000000004e-06 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:58.593038+0000", 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4350000000000003e-06 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:58.593044+0000", 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3169999999999998e-06 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:58.596726+0000", 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036821699999999998 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:58.596745+0000", 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8695999999999999e-05 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:58.631414+0000", 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034668830999999997 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:58.631505+0000", 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1509999999999996e-05 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:225744 8.a 8:55d467c9:::benchmark_data_smithi043_97042_object88650:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e224)", 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:00.055272+0000", 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8548895619999999, 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.049233443000000002, 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:03.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 225744 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.055272+0000", 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.055272+0000", 2024-09-21T11:48:03.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.055272+0000", 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6499999999999999e-07 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.055275+0000", 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9560000000000002e-06 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.055282+0000", 2024-09-21T11:48:03.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7989999999999996e-06 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.055289+0000", 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5470000000000002e-06 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.058367+0000", 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030780249999999999 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.058493+0000", 2024-09-21T11:48:03.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012613500000000001 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.058612+0000", 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011968000000000001 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.064409+0000", 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057965409999999997 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.067058+0000", 2024-09-21T11:48:03.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026489259999999998 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.104394+0000", 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.037336309999999998 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.104455+0000", 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1080000000000005e-05 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.104505+0000", 2024-09-21T11:48:03.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9879000000000001e-05 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:225753 8.a 8:55c2383a:::benchmark_data_smithi043_97042_object88659:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e224)", 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:00.064279+0000", 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.8458826049999999, 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.040352329999999999, 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:48:03.037 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 225753 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.064279+0000", 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.064279+0000", 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.064280+0000", 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.72e-06 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.064285+0000", 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0599999999999998e-06 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.064296+0000", 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0512e-05 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.064303+0000", 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1430000000000001e-06 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.064434+0000", 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000130999 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.064547+0000", 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112565 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:48:03.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.064661+0000", 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113923 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.067802+0000", 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031411989999999999 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.067838+0000", 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6198000000000001e-05 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.104547+0000", 2024-09-21T11:48:03.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.036709449999999998 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.104591+0000", 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3489999999999999e-05 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.104631+0000", 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0071000000000003e-05 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:225761 8.a 8:536ccfee:::benchmark_data_smithi043_97042_object88667:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e224)", 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:00.069372+0000", 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 2.84078878, 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.040803323000000002, 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 225761 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.069372+0000", 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.069372+0000", 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.069374+0000", 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.531e-06 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.069380+0000", 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0700000000000003e-06 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.069392+0000", 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1939e-05 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.069399+0000", 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2169999999999997e-06 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.069457+0000", 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7503999999999997e-05 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.069574+0000", 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117478 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.069689+0000", 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011448899999999999 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.071583+0000", 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001893806 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.075950+0000", 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043675470000000003 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.110083+0000", 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.034133016000000002 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.110135+0000", 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2123000000000002e-05 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:00.110176+0000", 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0602999999999997e-05 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:226882 8.a 8:5a269417:::benchmark_data_smithi043_97042_object89788:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e224)", 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:01.234460+0000", 2024-09-21T11:48:03.048 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.675701538, 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.079911359000000001, 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 226882 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.234460+0000", 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.234460+0000", 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.234461+0000", 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.234467+0000", 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9680000000000003e-06 2024-09-21T11:48:03.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.234479+0000", 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1484000000000001e-05 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.234486+0000", 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5959999999999998e-06 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.234541+0000", 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4632000000000001e-05 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.234658+0000", 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116989 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.234772+0000", 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113467 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.237481+0000", 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027099149999999998 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.240116+0000", 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002635025 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.313954+0000", 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.073837621000000006 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.314204+0000", 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000249662 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.314371+0000", 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016735200000000001 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:226884 8.a 8:5cb868a6:::benchmark_data_smithi043_97042_object89790:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e224)", 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:01.235725+0000", 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 1.6744362860000002, 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.088517609999999997, 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:03.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 226884 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.235725+0000", 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.235725+0000", 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:03.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.235726+0000", 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.486e-06 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.235731+0000", 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.464e-06 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.235740+0000", 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4290000000000007e-06 2024-09-21T11:48:03.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.235747+0000", 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8959999999999997e-06 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.237507+0000", 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017601509999999999 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.237614+0000", 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106499 2024-09-21T11:48:03.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.237725+0000", 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011069300000000001 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.240131+0000", 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024065190000000002 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.241191+0000", 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001060213 2024-09-21T11:48:03.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.324091+0000", 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.082899263000000001 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.324194+0000", 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103638 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:01.324243+0000", 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8359000000000002e-05 2024-09-21T11:48:03.059 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:03.060 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:03.060 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:48:03.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:03.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:03.163 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:03.164 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:03.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:03.309 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:03.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:03.310 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:03.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:03.460 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:03.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:03.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:03.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:03.596 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:03.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:03.597 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:03.698 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:03.728 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:03.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:03.729 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:03 smithi043 ceph-mon[29710]: pgmap v380: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 893 op/s; 7299/142500 objects misplaced (5.122%); 42 MiB/s, 676 objects/s recovering 2024-09-21T11:48:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:03 smithi043 ceph-mon[36454]: pgmap v380: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 893 op/s; 7299/142500 objects misplaced (5.122%); 42 MiB/s, 676 objects/s recovering 2024-09-21T11:48:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:03 smithi175 ceph-mon[32613]: pgmap v380: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 893 op/s; 7299/142500 objects misplaced (5.122%); 42 MiB/s, 676 objects/s recovering 2024-09-21T11:48:03.830 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:03.859 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:03.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:03.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:03.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:04.007 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:04.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:04.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:04.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:04.156 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:04.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:04.157 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:04.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:04.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:04.317 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:04.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:04.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:04.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:04.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:04.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:04 smithi043 ceph-mon[29710]: 4.9 scrub starts 2024-09-21T11:48:04.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:04 smithi043 ceph-mon[29710]: 4.9 scrub ok 2024-09-21T11:48:04.542 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:04 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2825175952' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:04.542 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:04 smithi043 ceph-mon[36454]: 4.9 scrub starts 2024-09-21T11:48:04.542 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:04 smithi043 ceph-mon[36454]: 4.9 scrub ok 2024-09-21T11:48:04.542 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:04 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2825175952' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:04.542 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:48:04.543 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:48:04.543 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:48:04.543 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:48:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:48:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:48:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:48:04.543 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:48:04.544 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:48:04.544 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:48:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:48:04.544 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:48:04.553 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:48:04.553 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:48:04.594 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:48:04.594 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:48:04.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:04.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:04.684 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:04.684 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:04.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:04 smithi175 ceph-mon[32613]: 4.9 scrub starts 2024-09-21T11:48:04.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:04 smithi175 ceph-mon[32613]: 4.9 scrub ok 2024-09-21T11:48:04.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:04 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2825175952' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:04.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:04.825 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:04.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:04.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:04.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:04.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:04.967 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:04.967 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:05.068 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:05.106 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:05.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:05.107 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:05.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:05.247 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:05.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:05.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:05.350 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:05.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:48:05.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:05.395 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:05.397 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:05.398 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:48:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:48:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:48:05.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[36454]: 3.d scrub starts 2024-09-21T11:48:05.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[36454]: 3.d scrub ok 2024-09-21T11:48:05.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[36454]: pgmap v381: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 740 op/s; 7299/142500 objects misplaced (5.122%); 35 MiB/s, 561 objects/s recovering 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3521903615' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[36454]: 4.e scrub starts 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[36454]: 4.e scrub ok 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[29710]: 3.d scrub starts 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[29710]: 3.d scrub ok 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[29710]: pgmap v381: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 740 op/s; 7299/142500 objects misplaced (5.122%); 35 MiB/s, 561 objects/s recovering 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3521903615' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[29710]: 4.e scrub starts 2024-09-21T11:48:05.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:05 smithi043 ceph-mon[29710]: 4.e scrub ok 2024-09-21T11:48:05.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:05.544 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:05.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:05.545 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:05.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:05.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:05.697 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:05.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:05 smithi175 ceph-mon[32613]: 3.d scrub starts 2024-09-21T11:48:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:05 smithi175 ceph-mon[32613]: 3.d scrub ok 2024-09-21T11:48:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:05 smithi175 ceph-mon[32613]: pgmap v381: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 740 op/s; 7299/142500 objects misplaced (5.122%); 35 MiB/s, 561 objects/s recovering 2024-09-21T11:48:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:05 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3521903615' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:05 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:05 smithi175 ceph-mon[32613]: 4.e scrub starts 2024-09-21T11:48:05.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:05 smithi175 ceph-mon[32613]: 4.e scrub ok 2024-09-21T11:48:05.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:05.808 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:05.826 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:05.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:05.827 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:05.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:05.960 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:05.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:05.961 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:06.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:06.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:06.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:06.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:06.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:06.246 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:06.246 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:06.347 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:06.395 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:06.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:06.396 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:06.497 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:06.529 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:06.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:06.530 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:06.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:06.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[36454]: 3.15 scrub starts 2024-09-21T11:48:06.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[36454]: 3.15 scrub ok 2024-09-21T11:48:06.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[36454]: 8.d scrub starts 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[36454]: osdmap e225: 8 total, 8 up, 8 in 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[36454]: 8.d scrub ok 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[29710]: 3.15 scrub starts 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[29710]: 3.15 scrub ok 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[29710]: 8.d scrub starts 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[29710]: osdmap e225: 8 total, 8 up, 8 in 2024-09-21T11:48:06.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:06 smithi043 ceph-mon[29710]: 8.d scrub ok 2024-09-21T11:48:06.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:06.674 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:06.675 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:06 smithi175 ceph-mon[32613]: 3.15 scrub starts 2024-09-21T11:48:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:06 smithi175 ceph-mon[32613]: 3.15 scrub ok 2024-09-21T11:48:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:06 smithi175 ceph-mon[32613]: 8.d scrub starts 2024-09-21T11:48:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:06 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:48:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:06 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:48:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:06 smithi175 ceph-mon[32613]: osdmap e225: 8 total, 8 up, 8 in 2024-09-21T11:48:06.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:06 smithi175 ceph-mon[32613]: 8.d scrub ok 2024-09-21T11:48:06.776 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:06.804 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:06.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:06.805 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:06.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:06.932 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:06.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:06.933 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:07.034 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:07.057 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:07.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:07.059 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:07.159 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:07.188 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:07.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:07.189 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:07.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:07.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:07.333 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:07.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:07.361 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:48:07.365 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:48:07.434 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:07.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:07.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:07.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:07.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:07.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:07.596 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:07.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:07.727 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:07.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:07.728 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:07 smithi043 ceph-mon[36454]: pgmap v383: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 652 op/s; 7299/140661 objects misplaced (5.189%); 9.0 MiB/s, 144 objects/s recovering 2024-09-21T11:48:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:07 smithi043 ceph-mon[36454]: 4.5 scrub starts 2024-09-21T11:48:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:07 smithi043 ceph-mon[36454]: 4.5 scrub ok 2024-09-21T11:48:07.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:07 smithi043 ceph-mon[29710]: pgmap v383: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 652 op/s; 7299/140661 objects misplaced (5.189%); 9.0 MiB/s, 144 objects/s recovering 2024-09-21T11:48:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:07 smithi043 ceph-mon[29710]: 4.5 scrub starts 2024-09-21T11:48:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:07 smithi043 ceph-mon[29710]: 4.5 scrub ok 2024-09-21T11:48:07.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:07 smithi175 ceph-mon[32613]: pgmap v383: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 652 op/s; 7299/140661 objects misplaced (5.189%); 9.0 MiB/s, 144 objects/s recovering 2024-09-21T11:48:07.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:07 smithi175 ceph-mon[32613]: 4.5 scrub starts 2024-09-21T11:48:07.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:07 smithi175 ceph-mon[32613]: 4.5 scrub ok 2024-09-21T11:48:07.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:07.868 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:07.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:07.869 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:07.970 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:08.001 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:08.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:08.002 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:08.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:08.141 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:08.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:08.142 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:08.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:08.279 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:08.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:08.280 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:08.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:08.403 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:08.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:08.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:08.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:48:08.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:08.545 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:08.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:08.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:08.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:08.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:08.678 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:08.678 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[29710]: 5.1a scrub starts 2024-09-21T11:48:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[29710]: 5.1a scrub ok 2024-09-21T11:48:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T11:48:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T11:48:08.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[29710]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:08.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[29710]: 6.e scrub starts 2024-09-21T11:48:08.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[29710]: 6.e scrub ok 2024-09-21T11:48:08.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[36454]: 5.1a scrub starts 2024-09-21T11:48:08.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[36454]: 5.1a scrub ok 2024-09-21T11:48:08.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T11:48:08.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T11:48:08.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[36454]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:08.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[36454]: 6.e scrub starts 2024-09-21T11:48:08.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:08 smithi043 ceph-mon[36454]: 6.e scrub ok 2024-09-21T11:48:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:08 smithi175 ceph-mon[32613]: 5.1a scrub starts 2024-09-21T11:48:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:08 smithi175 ceph-mon[32613]: 5.1a scrub ok 2024-09-21T11:48:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:08 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T11:48:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:08 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T11:48:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:08 smithi175 ceph-mon[32613]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:08 smithi175 ceph-mon[32613]: 6.e scrub starts 2024-09-21T11:48:08.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:08 smithi175 ceph-mon[32613]: 6.e scrub ok 2024-09-21T11:48:08.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:08.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:08.803 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:08.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:08.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:08.932 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:08.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:08.933 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:09.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:09.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:09.064 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:09.164 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:09.192 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:09.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:09.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:09.294 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:09.321 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:09.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:09.322 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:09.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:09.464 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:09.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:09.465 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:09.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:09.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:09.592 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:09.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:09.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:09.719 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:09.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:09.720 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[29710]: 6.5 scrub starts 2024-09-21T11:48:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[29710]: 6.5 scrub ok 2024-09-21T11:48:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[29710]: pgmap v384: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 972 op/s; 5495/128427 objects misplaced (4.279%); 15 MiB/s, 243 objects/s recovering 2024-09-21T11:48:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T11:48:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T11:48:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[29710]: 8.2 scrub starts 2024-09-21T11:48:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[36454]: 6.5 scrub starts 2024-09-21T11:48:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[36454]: 6.5 scrub ok 2024-09-21T11:48:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[36454]: pgmap v384: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 972 op/s; 5495/128427 objects misplaced (4.279%); 15 MiB/s, 243 objects/s recovering 2024-09-21T11:48:09.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T11:48:09.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T11:48:09.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:09 smithi043 ceph-mon[36454]: 8.2 scrub starts 2024-09-21T11:48:09.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:09 smithi175 ceph-mon[32613]: 6.5 scrub starts 2024-09-21T11:48:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:09 smithi175 ceph-mon[32613]: 6.5 scrub ok 2024-09-21T11:48:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:09 smithi175 ceph-mon[32613]: pgmap v384: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 972 op/s; 5495/128427 objects misplaced (4.279%); 15 MiB/s, 243 objects/s recovering 2024-09-21T11:48:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:09 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T11:48:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:09 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T11:48:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:09 smithi175 ceph-mon[32613]: 8.2 scrub starts 2024-09-21T11:48:09.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:09.863 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:09.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:09.865 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:09.965 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:09.991 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:09.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:09.993 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:10.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:10.116 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:10.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:10.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:10.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:10.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:10.262 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:10.262 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:10.363 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:10.391 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:10.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:10.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:10.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:10.515 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:10.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:10.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:10.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:10.647 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:10.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:10.648 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:10.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[29710]: 5.11 scrub starts 2024-09-21T11:48:10.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[29710]: 5.11 scrub ok 2024-09-21T11:48:10.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:48:10.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:48:10.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[29710]: osdmap e226: 8 total, 8 up, 8 in 2024-09-21T11:48:10.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[29710]: 8.2 scrub ok 2024-09-21T11:48:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[36454]: 5.11 scrub starts 2024-09-21T11:48:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[36454]: 5.11 scrub ok 2024-09-21T11:48:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:48:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:48:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[36454]: osdmap e226: 8 total, 8 up, 8 in 2024-09-21T11:48:10.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:10 smithi043 ceph-mon[36454]: 8.2 scrub ok 2024-09-21T11:48:10.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:10.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:10 smithi175 ceph-mon[32613]: 5.11 scrub starts 2024-09-21T11:48:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:10 smithi175 ceph-mon[32613]: 5.11 scrub ok 2024-09-21T11:48:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:10 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:48:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:10 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:48:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:10 smithi175 ceph-mon[32613]: osdmap e226: 8 total, 8 up, 8 in 2024-09-21T11:48:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:10 smithi175 ceph-mon[32613]: 8.2 scrub ok 2024-09-21T11:48:10.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:10.788 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:10.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:10.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:10.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:10.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:10.912 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:11.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:11.049 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:11.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:11.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:11.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:11.172 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:11.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:11.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:11.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:11.325 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:11.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:11.326 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:11.427 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:11.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:11.462 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:11.462 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:11.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:48:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:48:11.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:11 smithi175 ceph-mon[32613]: 3.f scrub starts 2024-09-21T11:48:11.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:11 smithi175 ceph-mon[32613]: 3.f scrub ok 2024-09-21T11:48:11.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:11 smithi175 ceph-mon[32613]: pgmap v386: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 586 op/s; 5495/128427 objects misplaced (4.279%); 9.2 MiB/s, 146 objects/s recovering 2024-09-21T11:48:11.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:11 smithi175 ceph-mon[32613]: 3.16 scrub starts 2024-09-21T11:48:11.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:11 smithi175 ceph-mon[32613]: 3.16 scrub ok 2024-09-21T11:48:11.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:11 smithi175 ceph-mon[32613]: 5.2 scrub starts 2024-09-21T11:48:11.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:11 smithi175 ceph-mon[32613]: 5.2 scrub ok 2024-09-21T11:48:11.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:11 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:48:11.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:11 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:48:11.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:11.586 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:11.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:11.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:11.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:11.716 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:11.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:11.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[29710]: 3.f scrub starts 2024-09-21T11:48:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[29710]: 3.f scrub ok 2024-09-21T11:48:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[29710]: pgmap v386: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 586 op/s; 5495/128427 objects misplaced (4.279%); 9.2 MiB/s, 146 objects/s recovering 2024-09-21T11:48:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[29710]: 3.16 scrub starts 2024-09-21T11:48:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[29710]: 3.16 scrub ok 2024-09-21T11:48:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[29710]: 5.2 scrub starts 2024-09-21T11:48:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[29710]: 5.2 scrub ok 2024-09-21T11:48:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:48:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:48:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[36454]: 3.f scrub starts 2024-09-21T11:48:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[36454]: 3.f scrub ok 2024-09-21T11:48:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[36454]: pgmap v386: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 586 op/s; 5495/128427 objects misplaced (4.279%); 9.2 MiB/s, 146 objects/s recovering 2024-09-21T11:48:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[36454]: 3.16 scrub starts 2024-09-21T11:48:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[36454]: 3.16 scrub ok 2024-09-21T11:48:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[36454]: 5.2 scrub starts 2024-09-21T11:48:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[36454]: 5.2 scrub ok 2024-09-21T11:48:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:48:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:11 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:48:11.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:11.861 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:11.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:11.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:11.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:11.984 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:11.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:11.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:12.086 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:12.122 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:12.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:12.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:12.224 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:12.255 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:12.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:12.256 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:12.357 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:12.387 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:12.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:12.388 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:12.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:12 smithi043 ceph-mon[29710]: 6.3 scrub starts 2024-09-21T11:48:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:12 smithi043 ceph-mon[29710]: 6.3 scrub ok 2024-09-21T11:48:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:12 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:12 smithi043 ceph-mon[29710]: osdmap e227: 8 total, 8 up, 8 in 2024-09-21T11:48:12.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:12 smithi043 ceph-mon[36454]: 6.3 scrub starts 2024-09-21T11:48:12.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:12 smithi043 ceph-mon[36454]: 6.3 scrub ok 2024-09-21T11:48:12.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:12 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:12.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:12 smithi043 ceph-mon[36454]: osdmap e227: 8 total, 8 up, 8 in 2024-09-21T11:48:12.526 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:12.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:12.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:12.628 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:12.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:12.658 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:12.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:12.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:12 smithi175 ceph-mon[32613]: 6.3 scrub starts 2024-09-21T11:48:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:12 smithi175 ceph-mon[32613]: 6.3 scrub ok 2024-09-21T11:48:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:12 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:12 smithi175 ceph-mon[32613]: osdmap e227: 8 total, 8 up, 8 in 2024-09-21T11:48:12.800 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:12.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:12.801 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:12.902 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:12.930 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:12.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:12.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:13.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:13.054 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:13.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:13.055 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:13.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:13.196 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:13.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:13.197 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:13.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:13.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:13.319 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:13.319 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:13.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:13.469 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:13.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:13.469 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:13.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:13.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:13.607 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:13.608 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:13.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:13 smithi043 ceph-mon[36454]: pgmap v388: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.33k op/s; 3959/116403 objects misplaced (3.401%); 22 MiB/s, 346 objects/s recovering 2024-09-21T11:48:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:13 smithi043 ceph-mon[36454]: 4.1b scrub starts 2024-09-21T11:48:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:13 smithi043 ceph-mon[36454]: 4.1b scrub ok 2024-09-21T11:48:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:13 smithi043 ceph-mon[29710]: pgmap v388: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.33k op/s; 3959/116403 objects misplaced (3.401%); 22 MiB/s, 346 objects/s recovering 2024-09-21T11:48:13.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:13 smithi043 ceph-mon[29710]: 4.1b scrub starts 2024-09-21T11:48:13.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:13 smithi043 ceph-mon[29710]: 4.1b scrub ok 2024-09-21T11:48:13.753 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:13.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:13.755 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:13.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:13 smithi175 ceph-mon[32613]: pgmap v388: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.33k op/s; 3959/116403 objects misplaced (3.401%); 22 MiB/s, 346 objects/s recovering 2024-09-21T11:48:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:13 smithi175 ceph-mon[32613]: 4.1b scrub starts 2024-09-21T11:48:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:13 smithi175 ceph-mon[32613]: 4.1b scrub ok 2024-09-21T11:48:13.855 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:13.881 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:13.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:13.882 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:13.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:14.020 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:14.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:14.021 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:14.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:14.158 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:14.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:14.159 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:14.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:14.288 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:14.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:14.289 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:14.390 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:14.417 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:14.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:14.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:14.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:14.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:14.560 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:14.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:14.695 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:14.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:14.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:14.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:14 smithi043 ceph-mon[29710]: 4.1a scrub starts 2024-09-21T11:48:14.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:14 smithi043 ceph-mon[29710]: 4.1a scrub ok 2024-09-21T11:48:14.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:14 smithi043 ceph-mon[36454]: 4.1a scrub starts 2024-09-21T11:48:14.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:14 smithi043 ceph-mon[36454]: 4.1a scrub ok 2024-09-21T11:48:14.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:14 smithi175 ceph-mon[32613]: 4.1a scrub starts 2024-09-21T11:48:14.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:14 smithi175 ceph-mon[32613]: 4.1a scrub ok 2024-09-21T11:48:14.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:14.819 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:14.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:14.820 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:14.920 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:14.941 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:14.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:14.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:15.043 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:15.064 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:15.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:15.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:15.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:15.196 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:15.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:15.197 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:15.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:15.319 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:15.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:15.320 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:15.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:15.448 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:15.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:15.449 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:15.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:48:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:48:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:48:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:15 smithi043 ceph-mon[36454]: pgmap v389: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.01k op/s; 3959/116403 objects misplaced (3.401%); 18 MiB/s, 282 objects/s recovering 2024-09-21T11:48:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:15 smithi043 ceph-mon[36454]: 3.1f scrub starts 2024-09-21T11:48:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:15 smithi043 ceph-mon[36454]: 3.1f scrub ok 2024-09-21T11:48:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:15 smithi043 ceph-mon[36454]: osdmap e228: 8 total, 8 up, 8 in 2024-09-21T11:48:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:15 smithi043 ceph-mon[29710]: pgmap v389: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.01k op/s; 3959/116403 objects misplaced (3.401%); 18 MiB/s, 282 objects/s recovering 2024-09-21T11:48:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:15 smithi043 ceph-mon[29710]: 3.1f scrub starts 2024-09-21T11:48:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:15 smithi043 ceph-mon[29710]: 3.1f scrub ok 2024-09-21T11:48:15.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:15 smithi043 ceph-mon[29710]: osdmap e228: 8 total, 8 up, 8 in 2024-09-21T11:48:15.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:15.589 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:15.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:15.590 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:15.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:15.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:15.725 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:15 smithi175 ceph-mon[32613]: pgmap v389: 148 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 3 active+remapped+backfilling, 144 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.01k op/s; 3959/116403 objects misplaced (3.401%); 18 MiB/s, 282 objects/s recovering 2024-09-21T11:48:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:15 smithi175 ceph-mon[32613]: 3.1f scrub starts 2024-09-21T11:48:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:15 smithi175 ceph-mon[32613]: 3.1f scrub ok 2024-09-21T11:48:15.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:15 smithi175 ceph-mon[32613]: osdmap e228: 8 total, 8 up, 8 in 2024-09-21T11:48:15.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:15.847 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:15.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:15.848 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:15.949 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:15.970 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:15.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:15.971 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:16.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:16.098 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:16.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:16.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:16.199 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:16.221 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:16.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:16.222 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:16.323 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:16.344 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:16.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:16.345 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:16.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:16.471 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:16.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:16.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:16.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:16.616 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:16.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:16.617 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:16.718 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T11:48:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T11:48:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[36454]: osdmap e229: 8 total, 8 up, 8 in 2024-09-21T11:48:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:48:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[36454]: 5.6 scrub starts 2024-09-21T11:48:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[36454]: 5.6 scrub ok 2024-09-21T11:48:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T11:48:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T11:48:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[29710]: osdmap e229: 8 total, 8 up, 8 in 2024-09-21T11:48:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:48:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[29710]: 5.6 scrub starts 2024-09-21T11:48:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:16 smithi043 ceph-mon[29710]: 5.6 scrub ok 2024-09-21T11:48:16.744 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:16.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:16.745 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:16.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:16 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T11:48:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:16 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T11:48:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:16 smithi175 ceph-mon[32613]: osdmap e229: 8 total, 8 up, 8 in 2024-09-21T11:48:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:16 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:48:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:16 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:16 smithi175 ceph-mon[32613]: 5.6 scrub starts 2024-09-21T11:48:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:16 smithi175 ceph-mon[32613]: 5.6 scrub ok 2024-09-21T11:48:16.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:16.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:16.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:16.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:16.999 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:16.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:17.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:17.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:17.125 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:17.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:17.125 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:17.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:17.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:17.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:17.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:17.378 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:17.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:17.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:17.385 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-21T11:48:05.914779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3060275763,"num_objects":46887,"num_object_clones":0,"num_object_copies":140661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7299,"num_objects_unfound":0,"num_objects_dirty":46887,"num_whiteouts":0,"num_read":1314,"num_read_kb":1171,"num_write":228045,"num_write_kb":8775498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18888,"num_bytes_recovered":1222777508,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90449,"ondisk_log_size":90449,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":448,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":11739248,"kb_used_data":9286304,"kb_used_omap":12,"kb_used_meta":2452531,"kb_avail":738221968,"statfs":{"total":767960285184,"available":755939295232,"internally_reserved":0,"allocated":9509175296,"data_stored":9506331705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2511392306},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,3,5,8,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-341704704,"num_objects":-5214,"num_object_clones":0,"num_object_copies":-15642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1933,"num_objects_unfound":0,"num_objects_dirty":-5214,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5214,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1156,"num_bytes_recovered":75759616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002579"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.365449+0000","last_change":"2024-09-21T11:41:42.291880+0000","last_active":"2024-09-21T11:47:58.365449+0000","last_peered":"2024-09-21T11:47:58.365449+0000","last_clean":"2024-09-21T11:47:58.365449+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:47:58.365449+0000","last_undegraded":"2024-09-21T11:47:58.365449+0000","last_fullsized":"2024-09-21T11:47:58.365449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:42.290188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:42.290188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":417,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.391892+0000","last_change":"2024-09-21T11:47:03.433310+0000","last_active":"2024-09-21T11:48:05.391892+0000","last_peered":"2024-09-21T11:48:05.391892+0000","last_clean":"2024-09-21T11:48:05.391892+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:48:05.391892+0000","last_undegraded":"2024-09-21T11:48:05.391892+0000","last_fullsized":"2024-09-21T11:48:05.391892+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:44.553561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.553561+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.027789+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":56,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.413639+0000","last_change":"2024-09-21T11:47:03.372676+0000","last_active":"2024-09-21T11:48:05.413639+0000","last_peered":"2024-09-21T11:48:05.413639+0000","last_clean":"2024-09-21T11:48:05.413639+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:48:05.413639+0000","last_undegraded":"2024-09-21T11:48:05.413639+0000","last_fullsized":"2024-09-21T11:48:05.413639+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":373,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.392215+0000","last_change":"2024-09-21T11:47:20.957722+0000","last_active":"2024-09-21T11:48:05.392215+0000","last_peered":"2024-09-21T11:48:05.392215+0000","last_clean":"2024-09-21T11:48:05.392215+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:48:05.392215+0000","last_undegraded":"2024-09-21T11:48:05.392215+0000","last_fullsized":"2024-09-21T11:48:05.392215+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:20.957679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.957679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:25.355154+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":349,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357696+0000","last_change":"2024-09-21T11:46:09.675727+0000","last_active":"2024-09-21T11:47:58.357696+0000","last_peered":"2024-09-21T11:47:58.357696+0000","last_clean":"2024-09-21T11:47:58.357696+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:47:58.357696+0000","last_undegraded":"2024-09-21T11:47:58.357696+0000","last_fullsized":"2024-09-21T11:47:58.357696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:09.675148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:09.675148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":353,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.359072+0000","last_change":"2024-09-21T11:46:55.804484+0000","last_active":"2024-09-21T11:47:58.359072+0000","last_peered":"2024-09-21T11:47:58.359072+0000","last_clean":"2024-09-21T11:47:58.359072+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:47:58.359072+0000","last_undegraded":"2024-09-21T11:47:58.359072+0000","last_fullsized":"2024-09-21T11:47:58.359072+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.804419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.804419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:02.552472+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":374,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357619+0000","last_change":"2024-09-21T11:45:07.788590+0000","last_active":"2024-09-21T11:47:58.357619+0000","last_peered":"2024-09-21T11:47:58.357619+0000","last_clean":"2024-09-21T11:47:58.357619+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:47:58.357619+0000","last_undegraded":"2024-09-21T11:47:58.357619+0000","last_fullsized":"2024-09-21T11:47:58.357619+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:07.788512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.788512+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":377,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.391881+0000","last_change":"2024-09-21T11:47:03.371710+0000","last_active":"2024-09-21T11:48:05.391881+0000","last_peered":"2024-09-21T11:48:05.391881+0000","last_clean":"2024-09-21T11:48:05.391881+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:48:05.391881+0000","last_undegraded":"2024-09-21T11:48:05.391881+0000","last_fullsized":"2024-09-21T11:48:05.391881+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:16.876126+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358369+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:47:58.358369+0000","last_peered":"2024-09-21T11:47:58.358369+0000","last_clean":"2024-09-21T11:47:58.358369+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:47:58.358369+0000","last_undegraded":"2024-09-21T11:47:58.358369+0000","last_fullsized":"2024-09-21T11:47:58.358369+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:43.925152+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":375,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.393632+0000","last_change":"2024-09-21T11:43:26.258023+0000","last_active":"2024-09-21T11:48:05.393632+0000","last_peered":"2024-09-21T11:48:05.393632+0000","last_clean":"2024-09-21T11:48:05.393632+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:48:05.393632+0000","last_undegraded":"2024-09-21T11:48:05.393632+0000","last_fullsized":"2024-09-21T11:48:05.393632+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:43:26.257967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.257967+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:13.822638+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357817+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:47:58.357817+0000","last_peered":"2024-09-21T11:47:58.357817+0000","last_clean":"2024-09-21T11:47:58.357817+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:47:58.357817+0000","last_undegraded":"2024-09-21T11:47:58.357817+0000","last_fullsized":"2024-09-21T11:47:58.357817+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":362,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.364008+0000","last_change":"2024-09-21T11:47:03.385157+0000","last_active":"2024-09-21T11:47:58.364008+0000","last_peered":"2024-09-21T11:47:58.364008+0000","last_clean":"2024-09-21T11:47:58.364008+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:47:58.364008+0000","last_undegraded":"2024-09-21T11:47:58.364008+0000","last_fullsized":"2024-09-21T11:47:58.364008+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:06.385125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:06.385125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":382,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358498+0000","last_change":"2024-09-21T11:47:03.385245+0000","last_active":"2024-09-21T11:47:58.358498+0000","last_peered":"2024-09-21T11:47:58.358498+0000","last_clean":"2024-09-21T11:47:58.358498+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:47:58.358498+0000","last_undegraded":"2024-09-21T11:47:58.358498+0000","last_fullsized":"2024-09-21T11:47:58.358498+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":395,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.392344+0000","last_change":"2024-09-21T11:47:03.432517+0000","last_active":"2024-09-21T11:48:05.392344+0000","last_peered":"2024-09-21T11:48:05.392344+0000","last_clean":"2024-09-21T11:48:05.392344+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:48:05.392344+0000","last_undegraded":"2024-09-21T11:48:05.392344+0000","last_fullsized":"2024-09-21T11:48:05.392344+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:43:08.326846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:08.326846+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:11.898806+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":360,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.417340+0000","last_change":"2024-09-21T11:47:03.368622+0000","last_active":"2024-09-21T11:48:05.417340+0000","last_peered":"2024-09-21T11:48:05.417340+0000","last_clean":"2024-09-21T11:48:05.417340+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:48:05.417340+0000","last_undegraded":"2024-09-21T11:48:05.417340+0000","last_fullsized":"2024-09-21T11:48:05.417340+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:38.567131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:38.567131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":370,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358348+0000","last_change":"2024-09-21T11:47:03.380856+0000","last_active":"2024-09-21T11:47:58.358348+0000","last_peered":"2024-09-21T11:47:58.358348+0000","last_clean":"2024-09-21T11:47:58.358348+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:47:58.358348+0000","last_undegraded":"2024-09-21T11:47:58.358348+0000","last_fullsized":"2024-09-21T11:47:58.358348+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:13.288522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:13.288522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":367,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.393400+0000","last_change":"2024-09-21T11:46:52.035524+0000","last_active":"2024-09-21T11:48:05.393400+0000","last_peered":"2024-09-21T11:48:05.393400+0000","last_clean":"2024-09-21T11:48:05.393400+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:48:05.393400+0000","last_undegraded":"2024-09-21T11:48:05.393400+0000","last_fullsized":"2024-09-21T11:48:05.393400+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.035461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.035461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.367746+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:47:58.367746+0000","last_peered":"2024-09-21T11:47:58.367746+0000","last_clean":"2024-09-21T11:47:58.367746+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:47:58.367746+0000","last_undegraded":"2024-09-21T11:47:58.367746+0000","last_fullsized":"2024-09-21T11:47:58.367746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":459,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.372662+0000","last_change":"2024-09-21T11:47:03.424753+0000","last_active":"2024-09-21T11:47:58.372662+0000","last_peered":"2024-09-21T11:47:58.372662+0000","last_clean":"2024-09-21T11:47:58.372662+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:47:58.372662+0000","last_undegraded":"2024-09-21T11:47:58.372662+0000","last_fullsized":"2024-09-21T11:47:58.372662+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:40.530099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:40.530099+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":56,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.414736+0000","last_change":"2024-09-21T11:47:03.377667+0000","last_active":"2024-09-21T11:48:05.414736+0000","last_peered":"2024-09-21T11:48:05.414736+0000","last_clean":"2024-09-21T11:48:05.414736+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:48:05.414736+0000","last_undegraded":"2024-09-21T11:48:05.414736+0000","last_fullsized":"2024-09-21T11:48:05.414736+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.718478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.718478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":349,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.370511+0000","last_change":"2024-09-21T11:45:39.826902+0000","last_active":"2024-09-21T11:47:58.370511+0000","last_peered":"2024-09-21T11:47:58.370511+0000","last_clean":"2024-09-21T11:47:58.370511+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:47:58.370511+0000","last_undegraded":"2024-09-21T11:47:58.370511+0000","last_fullsized":"2024-09-21T11:47:58.370511+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:39.826701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:39.826701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":369,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.393773+0000","last_change":"2024-09-21T11:47:03.371044+0000","last_active":"2024-09-21T11:48:05.393773+0000","last_peered":"2024-09-21T11:48:05.393773+0000","last_clean":"2024-09-21T11:48:05.393773+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:48:05.393773+0000","last_undegraded":"2024-09-21T11:48:05.393773+0000","last_fullsized":"2024-09-21T11:48:05.393773+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.093325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.093325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:15.837332+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":418,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357226+0000","last_change":"2024-09-21T11:47:03.873307+0000","last_active":"2024-09-21T11:47:58.357226+0000","last_peered":"2024-09-21T11:47:58.357226+0000","last_clean":"2024-09-21T11:47:58.357226+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:47:58.357226+0000","last_undegraded":"2024-09-21T11:47:58.357226+0000","last_fullsized":"2024-09-21T11:47:58.357226+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:47:03.873239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.873239+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:22.833198+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":355,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.392427+0000","last_change":"2024-09-21T11:45:37.670176+0000","last_active":"2024-09-21T11:48:05.392427+0000","last_peered":"2024-09-21T11:48:05.392427+0000","last_clean":"2024-09-21T11:48:05.392427+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:48:05.392427+0000","last_undegraded":"2024-09-21T11:48:05.392427+0000","last_fullsized":"2024-09-21T11:48:05.392427+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:37.670117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:37.670117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:50.969760+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":367,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.417684+0000","last_change":"2024-09-21T11:46:47.611118+0000","last_active":"2024-09-21T11:48:05.417684+0000","last_peered":"2024-09-21T11:48:05.417684+0000","last_clean":"2024-09-21T11:48:05.417684+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:48:05.417684+0000","last_undegraded":"2024-09-21T11:48:05.417684+0000","last_fullsized":"2024-09-21T11:48:05.417684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:46:47.611042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:47.611042+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":60,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.413852+0000","last_change":"2024-09-21T11:47:03.388975+0000","last_active":"2024-09-21T11:48:05.413852+0000","last_peered":"2024-09-21T11:48:05.413852+0000","last_clean":"2024-09-21T11:48:05.413852+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:48:05.413852+0000","last_undegraded":"2024-09-21T11:48:05.413852+0000","last_fullsized":"2024-09-21T11:48:05.413852+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:12.636105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.636105+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":379,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.368756+0000","last_change":"2024-09-21T11:45:48.385970+0000","last_active":"2024-09-21T11:47:58.368756+0000","last_peered":"2024-09-21T11:47:58.368756+0000","last_clean":"2024-09-21T11:47:58.368756+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:47:58.368756+0000","last_undegraded":"2024-09-21T11:47:58.368756+0000","last_fullsized":"2024-09-21T11:47:58.368756+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:45:48.385895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:48.385895+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":484,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.367999+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:47:58.367999+0000","last_peered":"2024-09-21T11:47:58.367999+0000","last_clean":"2024-09-21T11:47:58.367999+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:47:58.367999+0000","last_undegraded":"2024-09-21T11:47:58.367999+0000","last_fullsized":"2024-09-21T11:47:58.367999+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:38.235643+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"224'14259","reported_seq":17017,"reported_epoch":224,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:48:01.337666+0000","last_change":"2024-09-21T11:47:52.533445+0000","last_active":"2024-09-21T11:48:01.337666+0000","last_peered":"2024-09-21T11:48:01.337666+0000","last_clean":"2024-09-21T11:47:02.336849+0000","last_became_active":"2024-09-21T11:47:04.533306+0000","last_became_peered":"2024-09-21T11:47:04.533306+0000","last_unstale":"2024-09-21T11:48:01.337666+0000","last_undegraded":"2024-09-21T11:48:01.337666+0000","last_fullsized":"2024-09-21T11:48:01.337666+0000","mapping_epoch":203,"log_start":"194'7700","ondisk_log_start":"194'7700","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":6559,"log_dups_size":0,"ondisk_log_size":6559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191823872,"num_objects":2927,"num_object_clones":0,"num_object_copies":8781,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1771,"num_objects_unfound":0,"num_objects_dirty":2927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14259,"num_write_kb":549952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1305,"num_bytes_recovered":85524480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[2,5,6],"avail_no_missing":["2","5","6"],"object_location_counts":[{"shards":"2,5,6","objects":2927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":72,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.416325+0000","last_change":"2024-09-21T11:47:03.400465+0000","last_active":"2024-09-21T11:48:05.416325+0000","last_peered":"2024-09-21T11:48:05.416325+0000","last_clean":"2024-09-21T11:48:05.416325+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:48:05.416325+0000","last_undegraded":"2024-09-21T11:48:05.416325+0000","last_fullsized":"2024-09-21T11:48:05.416325+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:22.708839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.708839+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":383,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357848+0000","last_change":"2024-09-21T11:47:03.391541+0000","last_active":"2024-09-21T11:47:58.357848+0000","last_peered":"2024-09-21T11:47:58.357848+0000","last_clean":"2024-09-21T11:47:58.357848+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:47:58.357848+0000","last_undegraded":"2024-09-21T11:47:58.357848+0000","last_fullsized":"2024-09-21T11:47:58.357848+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.836903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.836903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:05.421205+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":461,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.393442+0000","last_change":"2024-09-21T11:47:01.933697+0000","last_active":"2024-09-21T11:48:05.393442+0000","last_peered":"2024-09-21T11:48:05.393442+0000","last_clean":"2024-09-21T11:48:05.393442+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:48:05.393442+0000","last_undegraded":"2024-09-21T11:48:05.393442+0000","last_fullsized":"2024-09-21T11:48:05.393442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:01.933648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:01.933648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:13.030516+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":354,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357737+0000","last_change":"2024-09-21T11:47:03.384895+0000","last_active":"2024-09-21T11:47:58.357737+0000","last_peered":"2024-09-21T11:47:58.357737+0000","last_clean":"2024-09-21T11:47:58.357737+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:47:58.357737+0000","last_undegraded":"2024-09-21T11:47:58.357737+0000","last_fullsized":"2024-09-21T11:47:58.357737+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:11.066324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:11.066324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:11.176260+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"224'14280","reported_seq":14503,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:48:01.613464+0000","last_change":"2024-09-21T11:46:19.360961+0000","last_active":"2024-09-21T11:48:01.613464+0000","last_peered":"2024-09-21T11:48:01.613464+0000","last_clean":"2024-09-21T11:48:01.613464+0000","last_became_active":"2024-09-21T11:44:51.744224+0000","last_became_peered":"2024-09-21T11:44:51.744224+0000","last_unstale":"2024-09-21T11:48:01.613464+0000","last_undegraded":"2024-09-21T11:48:01.613464+0000","last_fullsized":"2024-09-21T11:48:01.613464+0000","mapping_epoch":160,"log_start":"196'8500","ondisk_log_start":"196'8500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7561","last_scrub_stamp":"2024-09-21T11:46:19.360902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.360902+0000","objects_scrubbed":7459,"log_size":5780,"log_dups_size":0,"ondisk_log_size":5780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188612608,"num_objects":2878,"num_object_clones":0,"num_object_copies":8634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14280,"num_write_kb":549056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":397,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357104+0000","last_change":"2024-09-21T11:46:58.789719+0000","last_active":"2024-09-21T11:47:58.357104+0000","last_peered":"2024-09-21T11:47:58.357104+0000","last_clean":"2024-09-21T11:47:58.357104+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:47:58.357104+0000","last_undegraded":"2024-09-21T11:47:58.357104+0000","last_fullsized":"2024-09-21T11:47:58.357104+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:46:58.789625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.789625+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:19.133894+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":369,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.372474+0000","last_change":"2024-09-21T11:46:52.891978+0000","last_active":"2024-09-21T11:47:58.372474+0000","last_peered":"2024-09-21T11:47:58.372474+0000","last_clean":"2024-09-21T11:47:58.372474+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:47:58.372474+0000","last_undegraded":"2024-09-21T11:47:58.372474+0000","last_fullsized":"2024-09-21T11:47:58.372474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.891793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.891793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:19.681093+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":355,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.392023+0000","last_change":"2024-09-21T11:44:13.946705+0000","last_active":"2024-09-21T11:48:05.392023+0000","last_peered":"2024-09-21T11:48:05.392023+0000","last_clean":"2024-09-21T11:48:05.392023+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:48:05.392023+0000","last_undegraded":"2024-09-21T11:48:05.392023+0000","last_fullsized":"2024-09-21T11:48:05.392023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:42.015785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.392416+0000","last_change":"2024-09-21T11:46:24.717355+0000","last_active":"2024-09-21T11:48:05.392416+0000","last_peered":"2024-09-21T11:48:05.392416+0000","last_clean":"2024-09-21T11:48:05.392416+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:48:05.392416+0000","last_undegraded":"2024-09-21T11:48:05.392416+0000","last_fullsized":"2024-09-21T11:48:05.392416+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.717285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.717285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:58.045440+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"224'14092","reported_seq":14234,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:48:00.945672+0000","last_change":"2024-09-21T11:44:51.744967+0000","last_active":"2024-09-21T11:48:00.945672+0000","last_peered":"2024-09-21T11:48:00.945672+0000","last_clean":"2024-09-21T11:48:00.945672+0000","last_became_active":"2024-09-21T11:44:51.744633+0000","last_became_peered":"2024-09-21T11:44:51.744633+0000","last_unstale":"2024-09-21T11:48:00.945672+0000","last_undegraded":"2024-09-21T11:48:00.945672+0000","last_fullsized":"2024-09-21T11:48:00.945672+0000","mapping_epoch":160,"log_start":"196'8700","ondisk_log_start":"196'8700","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5392,"log_dups_size":0,"ondisk_log_size":5392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194183192,"num_objects":2964,"num_object_clones":0,"num_object_copies":8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14092,"num_write_kb":545729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":396,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.392169+0000","last_change":"2024-09-21T11:47:33.018713+0000","last_active":"2024-09-21T11:48:05.392169+0000","last_peered":"2024-09-21T11:48:05.392169+0000","last_clean":"2024-09-21T11:48:05.392169+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:48:05.392169+0000","last_undegraded":"2024-09-21T11:48:05.392169+0000","last_fullsized":"2024-09-21T11:48:05.392169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:47:33.018639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:33.018639+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:41.906194+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":391,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.369020+0000","last_change":"2024-09-21T11:47:03.429013+0000","last_active":"2024-09-21T11:47:58.369020+0000","last_peered":"2024-09-21T11:47:58.369020+0000","last_clean":"2024-09-21T11:47:58.369020+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:47:58.369020+0000","last_undegraded":"2024-09-21T11:47:58.369020+0000","last_fullsized":"2024-09-21T11:47:58.369020+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:59.702813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.702813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:00.033341+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":481,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.356948+0000","last_change":"2024-09-21T11:47:03.406087+0000","last_active":"2024-09-21T11:47:58.356948+0000","last_peered":"2024-09-21T11:47:58.356948+0000","last_clean":"2024-09-21T11:47:58.356948+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:47:58.356948+0000","last_undegraded":"2024-09-21T11:47:58.356948+0000","last_fullsized":"2024-09-21T11:47:58.356948+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:08.620519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.620519+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":345,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.417573+0000","last_change":"2024-09-21T11:41:39.527210+0000","last_active":"2024-09-21T11:48:05.417573+0000","last_peered":"2024-09-21T11:48:05.417573+0000","last_clean":"2024-09-21T11:48:05.417573+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:48:05.417573+0000","last_undegraded":"2024-09-21T11:48:05.417573+0000","last_fullsized":"2024-09-21T11:48:05.417573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:39.527151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:39.527151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:10.532359+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"224'14392","reported_seq":14618,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:48:01.615859+0000","last_change":"2024-09-21T11:46:22.226583+0000","last_active":"2024-09-21T11:48:01.615859+0000","last_peered":"2024-09-21T11:48:01.615859+0000","last_clean":"2024-09-21T11:48:01.615859+0000","last_became_active":"2024-09-21T11:44:51.412662+0000","last_became_peered":"2024-09-21T11:44:51.412662+0000","last_unstale":"2024-09-21T11:48:01.615859+0000","last_undegraded":"2024-09-21T11:48:01.615859+0000","last_fullsized":"2024-09-21T11:48:01.615859+0000","mapping_epoch":160,"log_start":"196'8600","ondisk_log_start":"196'8600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7855","last_scrub_stamp":"2024-09-21T11:46:22.226400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.226400+0000","objects_scrubbed":7719,"log_size":5792,"log_dups_size":0,"ondisk_log_size":5792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198574080,"num_objects":3030,"num_object_clones":0,"num_object_copies":9090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14392,"num_write_kb":557504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":68,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.414294+0000","last_change":"2024-09-21T11:47:03.398638+0000","last_active":"2024-09-21T11:48:05.414294+0000","last_peered":"2024-09-21T11:48:05.414294+0000","last_clean":"2024-09-21T11:48:05.414294+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:48:05.414294+0000","last_undegraded":"2024-09-21T11:48:05.414294+0000","last_fullsized":"2024-09-21T11:48:05.414294+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:12.227121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.227121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":368,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358448+0000","last_change":"2024-09-21T11:45:13.309329+0000","last_active":"2024-09-21T11:47:58.358448+0000","last_peered":"2024-09-21T11:47:58.358448+0000","last_clean":"2024-09-21T11:47:58.358448+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:47:58.358448+0000","last_undegraded":"2024-09-21T11:47:58.358448+0000","last_fullsized":"2024-09-21T11:47:58.358448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:13.309257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:13.309257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":452,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.367837+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:47:58.367837+0000","last_peered":"2024-09-21T11:47:58.367837+0000","last_clean":"2024-09-21T11:47:58.367837+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:47:58.367837+0000","last_undegraded":"2024-09-21T11:47:58.367837+0000","last_fullsized":"2024-09-21T11:47:58.367837+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":348,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.364415+0000","last_change":"2024-09-21T11:46:28.010676+0000","last_active":"2024-09-21T11:47:58.364415+0000","last_peered":"2024-09-21T11:47:58.364415+0000","last_clean":"2024-09-21T11:47:58.364415+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:47:58.364415+0000","last_undegraded":"2024-09-21T11:47:58.364415+0000","last_fullsized":"2024-09-21T11:47:58.364415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:28.010602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.010602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"224'14107","reported_seq":23227,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:48:01.689144+0000","last_change":"2024-09-21T11:47:54.309078+0000","last_active":"2024-09-21T11:48:01.689144+0000","last_peered":"2024-09-21T11:48:01.689144+0000","last_clean":"2024-09-21T11:48:01.689144+0000","last_became_active":"2024-09-21T11:47:54.270690+0000","last_became_peered":"2024-09-21T11:47:54.270690+0000","last_unstale":"2024-09-21T11:48:01.689144+0000","last_undegraded":"2024-09-21T11:48:01.689144+0000","last_fullsized":"2024-09-21T11:48:01.689144+0000","mapping_epoch":222,"log_start":"197'9100","ondisk_log_start":"197'9100","created":160,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'7360","last_scrub_stamp":"2024-09-21T11:46:17.038356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.038356+0000","objects_scrubbed":7242,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192741376,"num_objects":2941,"num_object_clones":0,"num_object_copies":8823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14107,"num_write_kb":545536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4441,"num_bytes_recovered":291045376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":392,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.367458+0000","last_change":"2024-09-21T11:47:03.750064+0000","last_active":"2024-09-21T11:47:58.367458+0000","last_peered":"2024-09-21T11:47:58.367458+0000","last_clean":"2024-09-21T11:47:58.367458+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:47:58.367458+0000","last_undegraded":"2024-09-21T11:47:58.367458+0000","last_fullsized":"2024-09-21T11:47:58.367458+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:47:03.749962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.749962+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:07.808517+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":56,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.413824+0000","last_change":"2024-09-21T11:47:03.382675+0000","last_active":"2024-09-21T11:48:05.413824+0000","last_peered":"2024-09-21T11:48:05.413824+0000","last_clean":"2024-09-21T11:48:05.413824+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:48:05.413824+0000","last_undegraded":"2024-09-21T11:48:05.413824+0000","last_fullsized":"2024-09-21T11:48:05.413824+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:08.723593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.723593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":344,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.365512+0000","last_change":"2024-09-21T11:43:26.704117+0000","last_active":"2024-09-21T11:47:58.365512+0000","last_peered":"2024-09-21T11:47:58.365512+0000","last_clean":"2024-09-21T11:47:58.365512+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:47:58.365512+0000","last_undegraded":"2024-09-21T11:47:58.365512+0000","last_fullsized":"2024-09-21T11:47:58.365512+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:26.704073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:26.704073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:09.276494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":56,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.413545+0000","last_change":"2024-09-21T11:47:03.379157+0000","last_active":"2024-09-21T11:48:05.413545+0000","last_peered":"2024-09-21T11:48:05.413545+0000","last_clean":"2024-09-21T11:48:05.413545+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:48:05.413545+0000","last_undegraded":"2024-09-21T11:48:05.413545+0000","last_fullsized":"2024-09-21T11:48:05.413545+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"224'14266","reported_seq":14407,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:48:00.952384+0000","last_change":"2024-09-21T11:44:51.745506+0000","last_active":"2024-09-21T11:48:00.952384+0000","last_peered":"2024-09-21T11:48:00.952384+0000","last_clean":"2024-09-21T11:48:00.952384+0000","last_became_active":"2024-09-21T11:44:51.745200+0000","last_became_peered":"2024-09-21T11:44:51.745200+0000","last_unstale":"2024-09-21T11:48:00.952384+0000","last_undegraded":"2024-09-21T11:48:00.952384+0000","last_fullsized":"2024-09-21T11:48:00.952384+0000","mapping_epoch":160,"log_start":"196'8900","ondisk_log_start":"196'8900","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5366,"log_dups_size":0,"ondisk_log_size":5366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195952640,"num_objects":2990,"num_object_clones":0,"num_object_copies":8970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14266,"num_write_kb":552192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"205'31","reported_seq":91,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.413778+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:48:05.413778+0000","last_peered":"2024-09-21T11:48:05.413778+0000","last_clean":"2024-09-21T11:48:05.413778+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:48:05.413778+0000","last_undegraded":"2024-09-21T11:48:05.413778+0000","last_fullsized":"2024-09-21T11:48:05.413778+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":361,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.394024+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:48:05.394024+0000","last_peered":"2024-09-21T11:48:05.394024+0000","last_clean":"2024-09-21T11:48:05.394024+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:48:05.394024+0000","last_undegraded":"2024-09-21T11:48:05.394024+0000","last_fullsized":"2024-09-21T11:48:05.394024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:39.997756+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":392,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357991+0000","last_change":"2024-09-21T11:47:20.000195+0000","last_active":"2024-09-21T11:47:58.357991+0000","last_peered":"2024-09-21T11:47:58.357991+0000","last_clean":"2024-09-21T11:47:58.357991+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:47:58.357991+0000","last_undegraded":"2024-09-21T11:47:58.357991+0000","last_fullsized":"2024-09-21T11:47:58.357991+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:47:20.000051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.000051+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:48.196772+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":366,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.365145+0000","last_change":"2024-09-21T11:47:03.385173+0000","last_active":"2024-09-21T11:47:58.365145+0000","last_peered":"2024-09-21T11:47:58.365145+0000","last_clean":"2024-09-21T11:47:58.365145+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:47:58.365145+0000","last_undegraded":"2024-09-21T11:47:58.365145+0000","last_fullsized":"2024-09-21T11:47:58.365145+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":341,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358877+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:47:58.358877+0000","last_peered":"2024-09-21T11:47:58.358877+0000","last_clean":"2024-09-21T11:47:58.358877+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:47:58.358877+0000","last_undegraded":"2024-09-21T11:47:58.358877+0000","last_fullsized":"2024-09-21T11:47:58.358877+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"224'14019","reported_seq":14169,"reported_epoch":224,"state":"active+remapped+backfill_wait+backfill_toofull","last_fresh":"2024-09-21T11:48:01.688497+0000","last_change":"2024-09-21T11:47:34.584968+0000","last_active":"2024-09-21T11:48:01.688497+0000","last_peered":"2024-09-21T11:48:01.688497+0000","last_clean":"2024-09-21T11:47:02.342762+0000","last_became_active":"2024-09-21T11:47:04.531689+0000","last_became_peered":"2024-09-21T11:47:04.531689+0000","last_unstale":"2024-09-21T11:48:01.688497+0000","last_undegraded":"2024-09-21T11:48:01.688497+0000","last_fullsized":"2024-09-21T11:48:01.688497+0000","mapping_epoch":203,"log_start":"197'9000","ondisk_log_start":"197'9000","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5019,"log_dups_size":0,"ondisk_log_size":5019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":193921024,"num_objects":2959,"num_object_clones":0,"num_object_copies":8877,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2959,"num_objects_unfound":0,"num_objects_dirty":2959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14019,"num_write_kb":543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,2],"avail_no_missing":["3","2","4"],"object_location_counts":[{"shards":"2,3,4","objects":2959}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":68,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.416422+0000","last_change":"2024-09-21T11:47:03.399466+0000","last_active":"2024-09-21T11:48:05.416422+0000","last_peered":"2024-09-21T11:48:05.416422+0000","last_clean":"2024-09-21T11:48:05.416422+0000","last_became_active":"2024-09-21T11:47:03.377155+0000","last_became_peered":"2024-09-21T11:47:03.377155+0000","last_unstale":"2024-09-21T11:48:05.416422+0000","last_undegraded":"2024-09-21T11:48:05.416422+0000","last_fullsized":"2024-09-21T11:48:05.416422+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":60,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.416061+0000","last_change":"2024-09-21T11:47:03.391283+0000","last_active":"2024-09-21T11:48:05.416061+0000","last_peered":"2024-09-21T11:48:05.416061+0000","last_clean":"2024-09-21T11:48:05.416061+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:48:05.416061+0000","last_undegraded":"2024-09-21T11:48:05.416061+0000","last_fullsized":"2024-09-21T11:48:05.416061+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":378,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.369254+0000","last_change":"2024-09-21T11:46:54.864288+0000","last_active":"2024-09-21T11:47:58.369254+0000","last_peered":"2024-09-21T11:47:58.369254+0000","last_clean":"2024-09-21T11:47:58.369254+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:47:58.369254+0000","last_undegraded":"2024-09-21T11:47:58.369254+0000","last_fullsized":"2024-09-21T11:47:58.369254+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:54.864074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.864074+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:47:59.857912+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":353,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.381460+0000","last_change":"2024-09-21T11:43:28.505134+0000","last_active":"2024-09-21T11:47:58.381460+0000","last_peered":"2024-09-21T11:47:58.381460+0000","last_clean":"2024-09-21T11:47:58.381460+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:47:58.381460+0000","last_undegraded":"2024-09-21T11:47:58.381460+0000","last_fullsized":"2024-09-21T11:47:58.381460+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:28.504894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:28.504894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":350,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.419970+0000","last_change":"2024-09-21T11:46:15.718117+0000","last_active":"2024-09-21T11:48:05.419970+0000","last_peered":"2024-09-21T11:48:05.419970+0000","last_clean":"2024-09-21T11:48:05.419970+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:48:05.419970+0000","last_undegraded":"2024-09-21T11:48:05.419970+0000","last_fullsized":"2024-09-21T11:48:05.419970+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:15.718010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:15.718010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"225'14238","reported_seq":23661,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.829841+0000","last_change":"2024-09-21T11:47:52.322057+0000","last_active":"2024-09-21T11:48:05.829841+0000","last_peered":"2024-09-21T11:48:05.829841+0000","last_clean":"2024-09-21T11:48:05.829841+0000","last_became_active":"2024-09-21T11:47:52.264063+0000","last_became_peered":"2024-09-21T11:47:52.264063+0000","last_unstale":"2024-09-21T11:48:05.829841+0000","last_undegraded":"2024-09-21T11:48:05.829841+0000","last_fullsized":"2024-09-21T11:48:05.829841+0000","mapping_epoch":220,"log_start":"197'9606","ondisk_log_start":"197'9606","created":160,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4632,"log_dups_size":0,"ondisk_log_size":4632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173277184,"num_objects":2644,"num_object_clones":0,"num_object_copies":7932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14238,"num_write_kb":540224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4633,"num_bytes_recovered":303628288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":416,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.368838+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:47:58.368838+0000","last_peered":"2024-09-21T11:47:58.368838+0000","last_clean":"2024-09-21T11:47:58.368838+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:47:58.368838+0000","last_undegraded":"2024-09-21T11:47:58.368838+0000","last_fullsized":"2024-09-21T11:47:58.368838+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":368,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.369906+0000","last_change":"2024-09-21T11:46:29.937866+0000","last_active":"2024-09-21T11:47:58.369906+0000","last_peered":"2024-09-21T11:47:58.369906+0000","last_clean":"2024-09-21T11:47:58.369906+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:47:58.369906+0000","last_undegraded":"2024-09-21T11:47:58.369906+0000","last_fullsized":"2024-09-21T11:47:58.369906+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.937799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.937799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":883,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.418760+0000","last_change":"2024-09-21T11:47:03.389526+0000","last_active":"2024-09-21T11:48:05.418760+0000","last_peered":"2024-09-21T11:48:05.418760+0000","last_clean":"2024-09-21T11:48:05.418760+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:48:05.418760+0000","last_undegraded":"2024-09-21T11:48:05.418760+0000","last_fullsized":"2024-09-21T11:48:05.418760+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:46:29.761453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.761453+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":477,"num_read_kb":472,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":366,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358220+0000","last_change":"2024-09-21T11:47:22.025919+0000","last_active":"2024-09-21T11:47:58.358220+0000","last_peered":"2024-09-21T11:47:58.358220+0000","last_clean":"2024-09-21T11:47:58.358220+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:47:58.358220+0000","last_undegraded":"2024-09-21T11:47:58.358220+0000","last_fullsized":"2024-09-21T11:47:58.358220+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:22.025842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.025842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:29.017093+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":56,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.413652+0000","last_change":"2024-09-21T11:47:03.374507+0000","last_active":"2024-09-21T11:48:05.413652+0000","last_peered":"2024-09-21T11:48:05.413652+0000","last_clean":"2024-09-21T11:48:05.413652+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:48:05.413652+0000","last_undegraded":"2024-09-21T11:48:05.413652+0000","last_fullsized":"2024-09-21T11:48:05.413652+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"224'14077","reported_seq":14351,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:48:00.884919+0000","last_change":"2024-09-21T11:47:31.508923+0000","last_active":"2024-09-21T11:48:00.884919+0000","last_peered":"2024-09-21T11:48:00.884919+0000","last_clean":"2024-09-21T11:48:00.884919+0000","last_became_active":"2024-09-21T11:44:51.410701+0000","last_became_peered":"2024-09-21T11:44:51.410701+0000","last_unstale":"2024-09-21T11:48:00.884919+0000","last_undegraded":"2024-09-21T11:48:00.884919+0000","last_fullsized":"2024-09-21T11:48:00.884919+0000","mapping_epoch":160,"log_start":"194'7600","ondisk_log_start":"194'7600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'12576","last_scrub_stamp":"2024-09-21T11:47:31.508822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:47:31.508822+0000","objects_scrubbed":4421,"log_size":6477,"log_dups_size":0,"ondisk_log_size":6477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.146813+0000","scrub_duration":1214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189595648,"num_objects":2893,"num_object_clones":0,"num_object_copies":8679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14077,"num_write_kb":543040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":467,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.367744+0000","last_change":"2024-09-21T11:42:19.703071+0000","last_active":"2024-09-21T11:47:58.367744+0000","last_peered":"2024-09-21T11:47:58.367744+0000","last_clean":"2024-09-21T11:47:58.367744+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:47:58.367744+0000","last_undegraded":"2024-09-21T11:47:58.367744+0000","last_fullsized":"2024-09-21T11:47:58.367744+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:42:19.703007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.703007+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:05.239060+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":369,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358135+0000","last_change":"2024-09-21T11:46:57.833589+0000","last_active":"2024-09-21T11:47:58.358135+0000","last_peered":"2024-09-21T11:47:58.358135+0000","last_clean":"2024-09-21T11:47:58.358135+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:47:58.358135+0000","last_undegraded":"2024-09-21T11:47:58.358135+0000","last_fullsized":"2024-09-21T11:47:58.358135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:57.833329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.833329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:02.226639+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":425,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.418757+0000","last_change":"2024-09-21T11:46:58.645189+0000","last_active":"2024-09-21T11:48:05.418757+0000","last_peered":"2024-09-21T11:48:05.418757+0000","last_clean":"2024-09-21T11:48:05.418757+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:48:05.418757+0000","last_undegraded":"2024-09-21T11:48:05.418757+0000","last_fullsized":"2024-09-21T11:48:05.418757+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:46:58.645122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.645122+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":362,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.417901+0000","last_change":"2024-09-21T11:47:25.395237+0000","last_active":"2024-09-21T11:48:05.417901+0000","last_peered":"2024-09-21T11:48:05.417901+0000","last_clean":"2024-09-21T11:48:05.417901+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:48:05.417901+0000","last_undegraded":"2024-09-21T11:48:05.417901+0000","last_fullsized":"2024-09-21T11:48:05.417901+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.395185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.395185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.204032+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":356,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358123+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:47:58.358123+0000","last_peered":"2024-09-21T11:47:58.358123+0000","last_clean":"2024-09-21T11:47:58.358123+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:47:58.358123+0000","last_undegraded":"2024-09-21T11:47:58.358123+0000","last_fullsized":"2024-09-21T11:47:58.358123+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"225'14324","reported_seq":14547,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.861738+0000","last_change":"2024-09-21T11:46:17.377588+0000","last_active":"2024-09-21T11:48:05.861738+0000","last_peered":"2024-09-21T11:48:05.861738+0000","last_clean":"2024-09-21T11:48:05.861738+0000","last_became_active":"2024-09-21T11:44:51.410197+0000","last_became_peered":"2024-09-21T11:44:51.410197+0000","last_unstale":"2024-09-21T11:48:05.861738+0000","last_undegraded":"2024-09-21T11:48:05.861738+0000","last_fullsized":"2024-09-21T11:48:05.861738+0000","mapping_epoch":160,"log_start":"197'9500","ondisk_log_start":"197'9500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'7333","last_scrub_stamp":"2024-09-21T11:46:17.377369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.377369+0000","objects_scrubbed":7286,"log_size":4824,"log_dups_size":0,"ondisk_log_size":4824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1653,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":175243264,"num_objects":2674,"num_object_clones":0,"num_object_copies":8022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14324,"num_write_kb":543936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":416,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.393497+0000","last_change":"2024-09-21T11:48:05.072998+0000","last_active":"2024-09-21T11:48:05.393497+0000","last_peered":"2024-09-21T11:48:05.393497+0000","last_clean":"2024-09-21T11:48:05.393497+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:48:05.393497+0000","last_undegraded":"2024-09-21T11:48:05.393497+0000","last_fullsized":"2024-09-21T11:48:05.393497+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:05.072910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.072910+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:33.824160+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":379,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.392265+0000","last_change":"2024-09-21T11:47:00.950268+0000","last_active":"2024-09-21T11:48:05.392265+0000","last_peered":"2024-09-21T11:48:05.392265+0000","last_clean":"2024-09-21T11:48:05.392265+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:48:05.392265+0000","last_undegraded":"2024-09-21T11:48:05.392265+0000","last_fullsized":"2024-09-21T11:48:05.392265+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:00.950208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:00.950208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:06.868808+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358025+0000","last_change":"2024-09-21T11:45:11.312119+0000","last_active":"2024-09-21T11:47:58.358025+0000","last_peered":"2024-09-21T11:47:58.358025+0000","last_clean":"2024-09-21T11:47:58.358025+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:47:58.358025+0000","last_undegraded":"2024-09-21T11:47:58.358025+0000","last_fullsized":"2024-09-21T11:47:58.358025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:11.312058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:11.312058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":348,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358068+0000","last_change":"2024-09-21T11:47:03.378923+0000","last_active":"2024-09-21T11:47:58.358068+0000","last_peered":"2024-09-21T11:47:58.358068+0000","last_clean":"2024-09-21T11:47:58.358068+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:47:58.358068+0000","last_undegraded":"2024-09-21T11:47:58.358068+0000","last_fullsized":"2024-09-21T11:47:58.358068+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:38.453579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:38.453579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"224'14386","reported_seq":18874,"reported_epoch":224,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:48:01.617091+0000","last_change":"2024-09-21T11:47:50.387672+0000","last_active":"2024-09-21T11:48:01.617091+0000","last_peered":"2024-09-21T11:48:01.617091+0000","last_clean":"2024-09-21T11:47:02.292468+0000","last_became_active":"2024-09-21T11:47:04.479633+0000","last_became_peered":"2024-09-21T11:47:04.479633+0000","last_unstale":"2024-09-21T11:48:01.617091+0000","last_undegraded":"2024-09-21T11:48:01.617091+0000","last_fullsized":"2024-09-21T11:48:01.617091+0000","mapping_epoch":203,"log_start":"196'8600","ondisk_log_start":"196'8600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'10449","last_scrub_stamp":"2024-09-21T11:46:52.316998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.316998+0000","objects_scrubbed":7011,"log_size":5786,"log_dups_size":0,"ondisk_log_size":5786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194641920,"num_objects":2970,"num_object_clones":0,"num_object_copies":8910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1064,"num_objects_unfound":0,"num_objects_dirty":2970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14386,"num_write_kb":555392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2130,"num_bytes_recovered":139591680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[0,2,5],"avail_no_missing":["0","2","5"],"object_location_counts":[{"shards":"0,2,5","objects":2970}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":445,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.393425+0000","last_change":"2024-09-21T11:47:22.974837+0000","last_active":"2024-09-21T11:48:05.393425+0000","last_peered":"2024-09-21T11:48:05.393425+0000","last_clean":"2024-09-21T11:48:05.393425+0000","last_became_active":"2024-09-21T11:47:03.369099+0000","last_became_peered":"2024-09-21T11:47:03.369099+0000","last_unstale":"2024-09-21T11:48:05.393425+0000","last_undegraded":"2024-09-21T11:48:05.393425+0000","last_fullsized":"2024-09-21T11:48:05.393425+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:47:22.974761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.974761+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:31.827386+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":390,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.366536+0000","last_change":"2024-09-21T11:47:03.365071+0000","last_active":"2024-09-21T11:47:58.366536+0000","last_peered":"2024-09-21T11:47:58.366536+0000","last_clean":"2024-09-21T11:47:58.366536+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:47:58.366536+0000","last_undegraded":"2024-09-21T11:47:58.366536+0000","last_fullsized":"2024-09-21T11:47:58.366536+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.692429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.692429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:54.181079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":56,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.413567+0000","last_change":"2024-09-21T11:47:03.375108+0000","last_active":"2024-09-21T11:48:05.413567+0000","last_peered":"2024-09-21T11:48:05.413567+0000","last_clean":"2024-09-21T11:48:05.413567+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:48:05.413567+0000","last_undegraded":"2024-09-21T11:48:05.413567+0000","last_fullsized":"2024-09-21T11:48:05.413567+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:50.008635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:50.008635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:08.701548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":365,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.372366+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:47:58.372366+0000","last_peered":"2024-09-21T11:47:58.372366+0000","last_clean":"2024-09-21T11:47:58.372366+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:47:58.372366+0000","last_undegraded":"2024-09-21T11:47:58.372366+0000","last_fullsized":"2024-09-21T11:47:58.372366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"224'14129","reported_seq":23227,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:48:01.635993+0000","last_change":"2024-09-21T11:47:53.304799+0000","last_active":"2024-09-21T11:48:01.635993+0000","last_peered":"2024-09-21T11:48:01.635993+0000","last_clean":"2024-09-21T11:48:01.635993+0000","last_became_active":"2024-09-21T11:47:53.264856+0000","last_became_peered":"2024-09-21T11:47:53.264856+0000","last_unstale":"2024-09-21T11:48:01.635993+0000","last_undegraded":"2024-09-21T11:48:01.635993+0000","last_fullsized":"2024-09-21T11:48:01.635993+0000","mapping_epoch":221,"log_start":"197'9100","ondisk_log_start":"197'9100","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5029,"log_dups_size":0,"ondisk_log_size":5029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183959552,"num_objects":2807,"num_object_clones":0,"num_object_copies":8421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14129,"num_write_kb":541952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4468,"num_bytes_recovered":292749312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":390,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.393691+0000","last_change":"2024-09-21T11:46:11.730053+0000","last_active":"2024-09-21T11:48:05.393691+0000","last_peered":"2024-09-21T11:48:05.393691+0000","last_clean":"2024-09-21T11:48:05.393691+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:48:05.393691+0000","last_undegraded":"2024-09-21T11:48:05.393691+0000","last_fullsized":"2024-09-21T11:48:05.393691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:11.729984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.729984+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.980319+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":353,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358123+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:47:58.358123+0000","last_peered":"2024-09-21T11:47:58.358123+0000","last_clean":"2024-09-21T11:47:58.358123+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:47:58.358123+0000","last_undegraded":"2024-09-21T11:47:58.358123+0000","last_fullsized":"2024-09-21T11:47:58.358123+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":466,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.371219+0000","last_change":"2024-09-21T11:46:52.834287+0000","last_active":"2024-09-21T11:47:58.371219+0000","last_peered":"2024-09-21T11:47:58.371219+0000","last_clean":"2024-09-21T11:47:58.371219+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:47:58.371219+0000","last_undegraded":"2024-09-21T11:47:58.371219+0000","last_fullsized":"2024-09-21T11:47:58.371219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:52.834206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.834206+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":357,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.393308+0000","last_change":"2024-09-21T11:47:03.369368+0000","last_active":"2024-09-21T11:48:05.393308+0000","last_peered":"2024-09-21T11:48:05.393308+0000","last_clean":"2024-09-21T11:48:05.393308+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:48:05.393308+0000","last_undegraded":"2024-09-21T11:48:05.393308+0000","last_fullsized":"2024-09-21T11:48:05.393308+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:50.131830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:50.131830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:06.848658+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"224'14188","reported_seq":17763,"reported_epoch":224,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:48:01.341490+0000","last_change":"2024-09-21T11:47:51.783220+0000","last_active":"2024-09-21T11:48:01.341490+0000","last_peered":"2024-09-21T11:48:01.341490+0000","last_clean":"2024-09-21T11:47:02.336531+0000","last_became_active":"2024-09-21T11:47:04.492509+0000","last_became_peered":"2024-09-21T11:47:04.492509+0000","last_unstale":"2024-09-21T11:48:01.341490+0000","last_undegraded":"2024-09-21T11:48:01.341490+0000","last_fullsized":"2024-09-21T11:48:01.341490+0000","mapping_epoch":203,"log_start":"194'7600","ondisk_log_start":"194'7600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7408","last_scrub_stamp":"2024-09-21T11:46:17.934135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.934135+0000","objects_scrubbed":7290,"log_size":6588,"log_dups_size":0,"ondisk_log_size":6588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"no scrub is scheduled","scrub_duration":3305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":3008,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1505,"num_objects_unfound":0,"num_objects_dirty":3008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14188,"num_write_kb":550272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1673,"num_bytes_recovered":109641728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[2,3,6],"avail_no_missing":["2","3","6"],"object_location_counts":[{"shards":"2,3,6","objects":3008}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":398,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.364234+0000","last_change":"2024-09-21T11:47:03.414060+0000","last_active":"2024-09-21T11:47:58.364234+0000","last_peered":"2024-09-21T11:47:58.364234+0000","last_clean":"2024-09-21T11:47:58.364234+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:47:58.364234+0000","last_undegraded":"2024-09-21T11:47:58.364234+0000","last_fullsized":"2024-09-21T11:47:58.364234+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:15.417817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.417817+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358574+0000","last_change":"2024-09-21T11:45:07.249883+0000","last_active":"2024-09-21T11:47:58.358574+0000","last_peered":"2024-09-21T11:47:58.358574+0000","last_clean":"2024-09-21T11:47:58.358574+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:47:58.358574+0000","last_undegraded":"2024-09-21T11:47:58.358574+0000","last_fullsized":"2024-09-21T11:47:58.358574+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:07.249826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.249826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":353,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.369221+0000","last_change":"2024-09-21T11:46:55.879988+0000","last_active":"2024-09-21T11:47:58.369221+0000","last_peered":"2024-09-21T11:47:58.369221+0000","last_clean":"2024-09-21T11:47:58.369221+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:47:58.369221+0000","last_undegraded":"2024-09-21T11:47:58.369221+0000","last_fullsized":"2024-09-21T11:47:58.369221+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.879761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.879761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:09.163806+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":340,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.363977+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:47:58.363977+0000","last_peered":"2024-09-21T11:47:58.363977+0000","last_clean":"2024-09-21T11:47:58.363977+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:47:58.363977+0000","last_undegraded":"2024-09-21T11:47:58.363977+0000","last_fullsized":"2024-09-21T11:47:58.363977+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:23.420693+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"224'14336","reported_seq":14558,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:48:01.605122+0000","last_change":"2024-09-21T11:46:15.665868+0000","last_active":"2024-09-21T11:48:01.605122+0000","last_peered":"2024-09-21T11:48:01.605122+0000","last_clean":"2024-09-21T11:48:01.605122+0000","last_became_active":"2024-09-21T11:44:51.744226+0000","last_became_peered":"2024-09-21T11:44:51.744226+0000","last_unstale":"2024-09-21T11:48:01.605122+0000","last_undegraded":"2024-09-21T11:48:01.605122+0000","last_fullsized":"2024-09-21T11:48:01.605122+0000","mapping_epoch":160,"log_start":"196'8500","ondisk_log_start":"196'8500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7351","last_scrub_stamp":"2024-09-21T11:46:15.665775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:15.665775+0000","objects_scrubbed":7260,"log_size":5836,"log_dups_size":0,"ondisk_log_size":5836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":193069056,"num_objects":2946,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14336,"num_write_kb":553024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":449,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.366350+0000","last_change":"2024-09-21T11:47:03.397050+0000","last_active":"2024-09-21T11:47:58.366350+0000","last_peered":"2024-09-21T11:47:58.366350+0000","last_clean":"2024-09-21T11:47:58.366350+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:47:58.366350+0000","last_undegraded":"2024-09-21T11:47:58.366350+0000","last_fullsized":"2024-09-21T11:47:58.366350+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:46:20.650403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.650403+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:51.167956+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":370,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.420177+0000","last_change":"2024-09-21T11:48:03.546909+0000","last_active":"2024-09-21T11:48:05.420177+0000","last_peered":"2024-09-21T11:48:05.420177+0000","last_clean":"2024-09-21T11:48:05.420177+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:48:05.420177+0000","last_undegraded":"2024-09-21T11:48:05.420177+0000","last_fullsized":"2024-09-21T11:48:05.420177+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:03.546843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:03.546843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.594742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":369,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.371795+0000","last_change":"2024-09-21T11:46:16.775013+0000","last_active":"2024-09-21T11:47:58.371795+0000","last_peered":"2024-09-21T11:47:58.371795+0000","last_clean":"2024-09-21T11:47:58.371795+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:47:58.371795+0000","last_undegraded":"2024-09-21T11:47:58.371795+0000","last_fullsized":"2024-09-21T11:47:58.371795+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:16.774939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.774939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":358,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.418674+0000","last_change":"2024-09-21T11:46:59.704981+0000","last_active":"2024-09-21T11:48:05.418674+0000","last_peered":"2024-09-21T11:48:05.418674+0000","last_clean":"2024-09-21T11:48:05.418674+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:48:05.418674+0000","last_undegraded":"2024-09-21T11:48:05.418674+0000","last_fullsized":"2024-09-21T11:48:05.418674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"224'14224","reported_seq":14454,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:48:00.952071+0000","last_change":"2024-09-21T11:46:27.656566+0000","last_active":"2024-09-21T11:48:00.952071+0000","last_peered":"2024-09-21T11:48:00.952071+0000","last_clean":"2024-09-21T11:48:00.952071+0000","last_became_active":"2024-09-21T11:44:51.745609+0000","last_became_peered":"2024-09-21T11:44:51.745609+0000","last_unstale":"2024-09-21T11:48:00.952071+0000","last_undegraded":"2024-09-21T11:48:00.952071+0000","last_fullsized":"2024-09-21T11:48:00.952071+0000","mapping_epoch":160,"log_start":"194'7800","ondisk_log_start":"194'7800","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'8199","last_scrub_stamp":"2024-09-21T11:46:27.656360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.656360+0000","objects_scrubbed":8107,"log_size":6424,"log_dups_size":0,"ondisk_log_size":6424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:46.172763+0000","scrub_duration":2017,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198311936,"num_objects":3026,"num_object_clones":0,"num_object_copies":9078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14224,"num_write_kb":552000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":444,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.392064+0000","last_change":"2024-09-21T11:48:04.111199+0000","last_active":"2024-09-21T11:48:05.392064+0000","last_peered":"2024-09-21T11:48:05.392064+0000","last_clean":"2024-09-21T11:48:05.392064+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:48:05.392064+0000","last_undegraded":"2024-09-21T11:48:05.392064+0000","last_fullsized":"2024-09-21T11:48:05.392064+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:48:04.111115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.111115+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:19.507697+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":400,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.418474+0000","last_change":"2024-09-21T11:47:26.411190+0000","last_active":"2024-09-21T11:48:05.418474+0000","last_peered":"2024-09-21T11:48:05.418474+0000","last_clean":"2024-09-21T11:48:05.418474+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:48:05.418474+0000","last_undegraded":"2024-09-21T11:48:05.418474+0000","last_fullsized":"2024-09-21T11:48:05.418474+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:26.411146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:26.411146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:48.131154+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":383,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.366655+0000","last_change":"2024-09-21T11:47:03.385382+0000","last_active":"2024-09-21T11:47:58.366655+0000","last_peered":"2024-09-21T11:47:58.366655+0000","last_clean":"2024-09-21T11:47:58.366655+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:47:58.366655+0000","last_undegraded":"2024-09-21T11:47:58.366655+0000","last_fullsized":"2024-09-21T11:47:58.366655+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.766819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.766819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":363,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357778+0000","last_change":"2024-09-21T11:47:03.376001+0000","last_active":"2024-09-21T11:47:58.357778+0000","last_peered":"2024-09-21T11:47:58.357778+0000","last_clean":"2024-09-21T11:47:58.357778+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:47:58.357778+0000","last_undegraded":"2024-09-21T11:47:58.357778+0000","last_fullsized":"2024-09-21T11:47:58.357778+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.703712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.703712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"224'14179","reported_seq":14399,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:48:00.975593+0000","last_change":"2024-09-21T11:46:12.974904+0000","last_active":"2024-09-21T11:48:00.975593+0000","last_peered":"2024-09-21T11:48:00.975593+0000","last_clean":"2024-09-21T11:48:00.975593+0000","last_became_active":"2024-09-21T11:44:51.409710+0000","last_became_peered":"2024-09-21T11:44:51.409710+0000","last_unstale":"2024-09-21T11:48:00.975593+0000","last_undegraded":"2024-09-21T11:48:00.975593+0000","last_fullsized":"2024-09-21T11:48:00.975593+0000","mapping_epoch":160,"log_start":"196'8800","ondisk_log_start":"196'8800","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7106","last_scrub_stamp":"2024-09-21T11:46:12.974687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.974687+0000","objects_scrubbed":7036,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1797,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198639616,"num_objects":3031,"num_object_clones":0,"num_object_copies":9093,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14179,"num_write_kb":550720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":373,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357586+0000","last_change":"2024-09-21T11:46:57.248649+0000","last_active":"2024-09-21T11:47:58.357586+0000","last_peered":"2024-09-21T11:47:58.357586+0000","last_clean":"2024-09-21T11:47:58.357586+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:47:58.357586+0000","last_undegraded":"2024-09-21T11:47:58.357586+0000","last_fullsized":"2024-09-21T11:47:58.357586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:57.248575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.248575+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:16.322877+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":382,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.418355+0000","last_change":"2024-09-21T11:46:22.731137+0000","last_active":"2024-09-21T11:48:05.418355+0000","last_peered":"2024-09-21T11:48:05.418355+0000","last_clean":"2024-09-21T11:48:05.418355+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:48:05.418355+0000","last_undegraded":"2024-09-21T11:48:05.418355+0000","last_fullsized":"2024-09-21T11:48:05.418355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:46:22.731054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.731054+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":468,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.417349+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:48:05.417349+0000","last_peered":"2024-09-21T11:48:05.417349+0000","last_clean":"2024-09-21T11:48:05.417349+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:48:05.417349+0000","last_undegraded":"2024-09-21T11:48:05.417349+0000","last_fullsized":"2024-09-21T11:48:05.417349+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:24.037103+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":348,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357779+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:47:58.357779+0000","last_peered":"2024-09-21T11:47:58.357779+0000","last_clean":"2024-09-21T11:47:58.357779+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:47:58.357779+0000","last_undegraded":"2024-09-21T11:47:58.357779+0000","last_fullsized":"2024-09-21T11:47:58.357779+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.006344+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":368,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.365254+0000","last_change":"2024-09-21T11:45:46.470043+0000","last_active":"2024-09-21T11:47:58.365254+0000","last_peered":"2024-09-21T11:47:58.365254+0000","last_clean":"2024-09-21T11:47:58.365254+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:47:58.365254+0000","last_undegraded":"2024-09-21T11:47:58.365254+0000","last_fullsized":"2024-09-21T11:47:58.365254+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:46.469951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.469951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:14.290613+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":396,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.371529+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:47:58.371529+0000","last_peered":"2024-09-21T11:47:58.371529+0000","last_clean":"2024-09-21T11:47:58.371529+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:47:58.371529+0000","last_undegraded":"2024-09-21T11:47:58.371529+0000","last_fullsized":"2024-09-21T11:47:58.371529+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":374,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357968+0000","last_change":"2024-09-21T11:47:25.905705+0000","last_active":"2024-09-21T11:47:58.357968+0000","last_peered":"2024-09-21T11:47:58.357968+0000","last_clean":"2024-09-21T11:47:58.357968+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:47:58.357968+0000","last_undegraded":"2024-09-21T11:47:58.357968+0000","last_fullsized":"2024-09-21T11:47:58.357968+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.905645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.905645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.034596+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":366,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.419851+0000","last_change":"2024-09-21T11:47:28.380378+0000","last_active":"2024-09-21T11:48:05.419851+0000","last_peered":"2024-09-21T11:48:05.419851+0000","last_clean":"2024-09-21T11:48:05.419851+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:48:05.419851+0000","last_undegraded":"2024-09-21T11:48:05.419851+0000","last_fullsized":"2024-09-21T11:48:05.419851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:28.380245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:28.380245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:34.745499+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":389,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.368937+0000","last_change":"2024-09-21T11:46:28.973700+0000","last_active":"2024-09-21T11:47:58.368937+0000","last_peered":"2024-09-21T11:47:58.368937+0000","last_clean":"2024-09-21T11:47:58.368937+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:47:58.368937+0000","last_undegraded":"2024-09-21T11:47:58.368937+0000","last_fullsized":"2024-09-21T11:47:58.368937+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:28.973606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.973606+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":362,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.417519+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:48:05.417519+0000","last_peered":"2024-09-21T11:48:05.417519+0000","last_clean":"2024-09-21T11:48:05.417519+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:48:05.417519+0000","last_undegraded":"2024-09-21T11:48:05.417519+0000","last_fullsized":"2024-09-21T11:48:05.417519+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:47.600096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":375,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.373313+0000","last_change":"2024-09-21T11:47:03.379216+0000","last_active":"2024-09-21T11:47:58.373313+0000","last_peered":"2024-09-21T11:47:58.373313+0000","last_clean":"2024-09-21T11:47:58.373313+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:47:58.373313+0000","last_undegraded":"2024-09-21T11:47:58.373313+0000","last_fullsized":"2024-09-21T11:47:58.373313+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:48.903980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:48.903980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:09.074040+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":357,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.369415+0000","last_change":"2024-09-21T11:44:45.577521+0000","last_active":"2024-09-21T11:47:58.369415+0000","last_peered":"2024-09-21T11:47:58.369415+0000","last_clean":"2024-09-21T11:47:58.369415+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:47:58.369415+0000","last_undegraded":"2024-09-21T11:47:58.369415+0000","last_fullsized":"2024-09-21T11:47:58.369415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.577451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.577451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":372,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358360+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:47:58.358360+0000","last_peered":"2024-09-21T11:47:58.358360+0000","last_clean":"2024-09-21T11:47:58.358360+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:47:58.358360+0000","last_undegraded":"2024-09-21T11:47:58.358360+0000","last_fullsized":"2024-09-21T11:47:58.358360+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":368,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.371842+0000","last_change":"2024-09-21T11:46:24.087039+0000","last_active":"2024-09-21T11:47:58.371842+0000","last_peered":"2024-09-21T11:47:58.371842+0000","last_clean":"2024-09-21T11:47:58.371842+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:47:58.371842+0000","last_undegraded":"2024-09-21T11:47:58.371842+0000","last_fullsized":"2024-09-21T11:47:58.371842+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.086918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.086918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":450,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358328+0000","last_change":"2024-09-21T11:47:02.849839+0000","last_active":"2024-09-21T11:47:58.358328+0000","last_peered":"2024-09-21T11:47:58.358328+0000","last_clean":"2024-09-21T11:47:58.358328+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:47:58.358328+0000","last_undegraded":"2024-09-21T11:47:58.358328+0000","last_fullsized":"2024-09-21T11:47:58.358328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:02.849786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:02.849786+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:22.149526+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":359,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.393114+0000","last_change":"2024-09-21T11:45:38.652432+0000","last_active":"2024-09-21T11:48:05.393114+0000","last_peered":"2024-09-21T11:48:05.393114+0000","last_clean":"2024-09-21T11:48:05.393114+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:48:05.393114+0000","last_undegraded":"2024-09-21T11:48:05.393114+0000","last_fullsized":"2024-09-21T11:48:05.393114+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:38.652380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:38.652380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:47.014191+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":402,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358496+0000","last_change":"2024-09-21T11:47:03.428846+0000","last_active":"2024-09-21T11:47:58.358496+0000","last_peered":"2024-09-21T11:47:58.358496+0000","last_clean":"2024-09-21T11:47:58.358496+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:47:58.358496+0000","last_undegraded":"2024-09-21T11:47:58.358496+0000","last_fullsized":"2024-09-21T11:47:58.358496+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":362,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.417820+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:48:05.417820+0000","last_peered":"2024-09-21T11:48:05.417820+0000","last_clean":"2024-09-21T11:48:05.417820+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:48:05.417820+0000","last_undegraded":"2024-09-21T11:48:05.417820+0000","last_fullsized":"2024-09-21T11:48:05.417820+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.604026+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":488,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357973+0000","last_change":"2024-09-21T11:47:03.415793+0000","last_active":"2024-09-21T11:47:58.357973+0000","last_peered":"2024-09-21T11:47:58.357973+0000","last_clean":"2024-09-21T11:47:58.357973+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:47:58.357973+0000","last_undegraded":"2024-09-21T11:47:58.357973+0000","last_fullsized":"2024-09-21T11:47:58.357973+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:10.174971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:10.174971+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":356,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.365481+0000","last_change":"2024-09-21T11:45:44.314664+0000","last_active":"2024-09-21T11:47:58.365481+0000","last_peered":"2024-09-21T11:47:58.365481+0000","last_clean":"2024-09-21T11:47:58.365481+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:47:58.365481+0000","last_undegraded":"2024-09-21T11:47:58.365481+0000","last_fullsized":"2024-09-21T11:47:58.365481+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:44.314461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.314461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":432,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.392444+0000","last_change":"2024-09-21T11:47:03.402357+0000","last_active":"2024-09-21T11:48:05.392444+0000","last_peered":"2024-09-21T11:48:05.392444+0000","last_clean":"2024-09-21T11:48:05.392444+0000","last_became_active":"2024-09-21T11:47:03.368296+0000","last_became_peered":"2024-09-21T11:47:03.368296+0000","last_unstale":"2024-09-21T11:48:05.392444+0000","last_undegraded":"2024-09-21T11:48:05.392444+0000","last_fullsized":"2024-09-21T11:48:05.392444+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:54.032795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.032795+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:23.550314+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":363,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.393794+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:48:05.393794+0000","last_peered":"2024-09-21T11:48:05.393794+0000","last_clean":"2024-09-21T11:48:05.393794+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:48:05.393794+0000","last_undegraded":"2024-09-21T11:48:05.393794+0000","last_fullsized":"2024-09-21T11:48:05.393794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:37.991785+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":56,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.414200+0000","last_change":"2024-09-21T11:47:03.379049+0000","last_active":"2024-09-21T11:48:05.414200+0000","last_peered":"2024-09-21T11:48:05.414200+0000","last_clean":"2024-09-21T11:48:05.414200+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:48:05.414200+0000","last_undegraded":"2024-09-21T11:48:05.414200+0000","last_fullsized":"2024-09-21T11:48:05.414200+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.269610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.269610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":341,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357269+0000","last_change":"2024-09-21T11:45:04.770055+0000","last_active":"2024-09-21T11:47:58.357269+0000","last_peered":"2024-09-21T11:47:58.357269+0000","last_clean":"2024-09-21T11:47:58.357269+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:47:58.357269+0000","last_undegraded":"2024-09-21T11:47:58.357269+0000","last_fullsized":"2024-09-21T11:47:58.357269+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:04.769963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:04.769963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":66,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.414429+0000","last_change":"2024-09-21T11:47:03.404473+0000","last_active":"2024-09-21T11:48:05.414429+0000","last_peered":"2024-09-21T11:48:05.414429+0000","last_clean":"2024-09-21T11:48:05.414429+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:48:05.414429+0000","last_undegraded":"2024-09-21T11:48:05.414429+0000","last_fullsized":"2024-09-21T11:48:05.414429+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:26.649189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:26.649189+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":370,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.420057+0000","last_change":"2024-09-21T11:48:04.553772+0000","last_active":"2024-09-21T11:48:05.420057+0000","last_peered":"2024-09-21T11:48:05.420057+0000","last_clean":"2024-09-21T11:48:05.420057+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:48:05.420057+0000","last_undegraded":"2024-09-21T11:48:05.420057+0000","last_fullsized":"2024-09-21T11:48:05.420057+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:04.553731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.553731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:21.248327+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":375,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357864+0000","last_change":"2024-09-21T11:47:03.377948+0000","last_active":"2024-09-21T11:47:58.357864+0000","last_peered":"2024-09-21T11:47:58.357864+0000","last_clean":"2024-09-21T11:47:58.357864+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:47:58.357864+0000","last_undegraded":"2024-09-21T11:47:58.357864+0000","last_fullsized":"2024-09-21T11:47:58.357864+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:53.848323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:53.848323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:08.048064+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":379,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.363866+0000","last_change":"2024-09-21T11:47:03.408821+0000","last_active":"2024-09-21T11:47:58.363866+0000","last_peered":"2024-09-21T11:47:58.363866+0000","last_clean":"2024-09-21T11:47:58.363866+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:47:58.363866+0000","last_undegraded":"2024-09-21T11:47:58.363866+0000","last_fullsized":"2024-09-21T11:47:58.363866+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:45:43.314803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:43.314803+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":376,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357393+0000","last_change":"2024-09-21T11:45:46.706221+0000","last_active":"2024-09-21T11:47:58.357393+0000","last_peered":"2024-09-21T11:47:58.357393+0000","last_clean":"2024-09-21T11:47:58.357393+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:47:58.357393+0000","last_undegraded":"2024-09-21T11:47:58.357393+0000","last_fullsized":"2024-09-21T11:47:58.357393+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:45:46.706130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.706130+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":366,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.358077+0000","last_change":"2024-09-21T11:47:03.381029+0000","last_active":"2024-09-21T11:47:58.358077+0000","last_peered":"2024-09-21T11:47:58.358077+0000","last_clean":"2024-09-21T11:47:58.358077+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:47:58.358077+0000","last_undegraded":"2024-09-21T11:47:58.358077+0000","last_fullsized":"2024-09-21T11:47:58.358077+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:36.952335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:41:36.952335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:08.212751+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":346,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.418442+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:48:05.418442+0000","last_peered":"2024-09-21T11:48:05.418442+0000","last_clean":"2024-09-21T11:48:05.418442+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:48:05.418442+0000","last_undegraded":"2024-09-21T11:48:05.418442+0000","last_fullsized":"2024-09-21T11:48:05.418442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:43.608235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":349,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357998+0000","last_change":"2024-09-21T11:45:45.730374+0000","last_active":"2024-09-21T11:47:58.357998+0000","last_peered":"2024-09-21T11:47:58.357998+0000","last_clean":"2024-09-21T11:47:58.357998+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:47:58.357998+0000","last_undegraded":"2024-09-21T11:47:58.357998+0000","last_fullsized":"2024-09-21T11:47:58.357998+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:45.730169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:45.730169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":410,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357003+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:47:58.357003+0000","last_peered":"2024-09-21T11:47:58.357003+0000","last_clean":"2024-09-21T11:47:58.357003+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:47:58.357003+0000","last_undegraded":"2024-09-21T11:47:58.357003+0000","last_fullsized":"2024-09-21T11:47:58.357003+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.365536+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:47:58.365536+0000","last_peered":"2024-09-21T11:47:58.365536+0000","last_clean":"2024-09-21T11:47:58.365536+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:47:58.365536+0000","last_undegraded":"2024-09-21T11:47:58.365536+0000","last_fullsized":"2024-09-21T11:47:58.365536+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":352,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.365492+0000","last_change":"2024-09-21T11:43:39.838067+0000","last_active":"2024-09-21T11:47:58.365492+0000","last_peered":"2024-09-21T11:47:58.365492+0000","last_clean":"2024-09-21T11:47:58.365492+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:47:58.365492+0000","last_undegraded":"2024-09-21T11:47:58.365492+0000","last_fullsized":"2024-09-21T11:47:58.365492+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:39.838002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:39.838002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:08.299743+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":350,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.417300+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:48:05.417300+0000","last_peered":"2024-09-21T11:48:05.417300+0000","last_clean":"2024-09-21T11:48:05.417300+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:48:05.417300+0000","last_undegraded":"2024-09-21T11:48:05.417300+0000","last_fullsized":"2024-09-21T11:48:05.417300+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:49.549791+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":56,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-21T11:48:05.414127+0000","last_change":"2024-09-21T11:47:03.374194+0000","last_active":"2024-09-21T11:48:05.414127+0000","last_peered":"2024-09-21T11:48:05.414127+0000","last_clean":"2024-09-21T11:48:05.414127+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:48:05.414127+0000","last_undegraded":"2024-09-21T11:48:05.414127+0000","last_fullsized":"2024-09-21T11:48:05.414127+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:27.297630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.297630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":375,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.357263+0000","last_change":"2024-09-21T11:47:03.377189+0000","last_active":"2024-09-21T11:47:58.357263+0000","last_peered":"2024-09-21T11:47:58.357263+0000","last_clean":"2024-09-21T11:47:58.357263+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:47:58.357263+0000","last_undegraded":"2024-09-21T11:47:58.357263+0000","last_fullsized":"2024-09-21T11:47:58.357263+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:11.587469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.587469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":429,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.368051+0000","last_change":"2024-09-21T11:47:03.392087+0000","last_active":"2024-09-21T11:47:58.368051+0000","last_peered":"2024-09-21T11:47:58.368051+0000","last_clean":"2024-09-21T11:47:58.368051+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:47:58.368051+0000","last_undegraded":"2024-09-21T11:47:58.368051+0000","last_fullsized":"2024-09-21T11:47:58.368051+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:16.675872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.675872+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:41.170726+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":360,"reported_epoch":224,"state":"active+clean","last_fresh":"2024-09-21T11:47:58.368674+0000","last_change":"2024-09-21T11:45:12.377954+0000","last_active":"2024-09-21T11:47:58.368674+0000","last_peered":"2024-09-21T11:47:58.368674+0000","last_clean":"2024-09-21T11:47:58.368674+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:47:58.368674+0000","last_undegraded":"2024-09-21T11:47:58.368674+0000","last_fullsized":"2024-09-21T11:47:58.368674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:12.377880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:12.377880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":3059679256,"num_objects":46688,"num_object_clones":0,"num_object_copies":140064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7299,"num_objects_unfound":0,"num_objects_dirty":46688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227496,"num_write_kb":8773825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18650,"num_bytes_recovered":1222180864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9409736704,"data_stored":9409724488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89890,"ondisk_log_size":89890,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4814,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":396,"ondisk_log_size":396,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":477,"num_read_kb":472,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234086,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1181408,"kb_used_data":871180,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":92563744,"statfs":{"total":95995035648,"available":94785273856,"internally_reserved":0,"allocated":892088320,"data_stored":891730114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":317651402},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463018,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1243264,"kb_used_data":969136,"kb_used_omap":1,"kb_used_meta":274110,"kb_avail":92501888,"statfs":{"total":95995035648,"available":94721933312,"internally_reserved":0,"allocated":992395264,"data_stored":992059058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":280689094},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724655,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2021976,"kb_used_data":1729536,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":91723176,"statfs":{"total":95995035648,"available":93924532224,"internally_reserved":0,"allocated":1771044864,"data_stored":1770691055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":299170246},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920886,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1132500,"kb_used_data":888900,"kb_used_omap":1,"kb_used_meta":243582,"kb_avail":92612652,"statfs":{"total":95995035648,"available":94835355648,"internally_reserved":0,"allocated":910233600,"data_stored":909876539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":249428421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117115,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1489220,"kb_used_data":1129012,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":92255932,"statfs":{"total":95995035648,"available":94470074368,"internally_reserved":0,"allocated":1156108288,"data_stored":1155766436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":368835015},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411457,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2130108,"kb_used_data":1724076,"kb_used_omap":1,"kb_used_meta":406014,"kb_avail":91615044,"statfs":{"total":95995035648,"available":93813805056,"internally_reserved":0,"allocated":1765453824,"data_stored":1765120459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":415758789},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1093836,"kb_used_data":837752,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":92651316,"statfs":{"total":95995035648,"available":94874947584,"internally_reserved":0,"allocated":857858048,"data_stored":857455367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":262207942},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738510,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1446936,"kb_used_data":1136712,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":92298216,"statfs":{"total":95995035648,"available":94513373184,"internally_reserved":0,"allocated":1163993088,"data_stored":1163632677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":317651397},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1154351104,"data_stored":1154351104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":855900160,"data_stored":855900160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1755058176,"data_stored":1755054104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1147797504,"data_stored":1147797504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":892665856,"data_stored":892665856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1742802944,"data_stored":1742798872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":975437824,"data_stored":975437824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":885723136,"data_stored":885719064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:48:17.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:48:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:17.393 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:17.479 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:17.500 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:17.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:17.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:17.600 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:17 smithi043 ceph-mon[29710]: pgmap v392: 148 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfilling, 145 active+clean; 2.3 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 797 op/s; 3234/114093 objects misplaced (2.835%); 19 MiB/s, 297 objects/s recovering 2024-09-21T11:48:17.600 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:17 smithi043 ceph-mon[29710]: osdmap e230: 8 total, 8 up, 8 in 2024-09-21T11:48:17.600 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:17 smithi043 ceph-mon[29710]: 6.14 scrub starts 2024-09-21T11:48:17.600 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:17 smithi043 ceph-mon[29710]: 6.14 scrub ok 2024-09-21T11:48:17.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:17 smithi043 ceph-mon[36454]: pgmap v392: 148 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfilling, 145 active+clean; 2.3 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 797 op/s; 3234/114093 objects misplaced (2.835%); 19 MiB/s, 297 objects/s recovering 2024-09-21T11:48:17.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:17 smithi043 ceph-mon[36454]: osdmap e230: 8 total, 8 up, 8 in 2024-09-21T11:48:17.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:17 smithi043 ceph-mon[36454]: 6.14 scrub starts 2024-09-21T11:48:17.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:17 smithi043 ceph-mon[36454]: 6.14 scrub ok 2024-09-21T11:48:17.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:17.627 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:17.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:17.628 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:17.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:17 smithi175 ceph-mon[32613]: pgmap v392: 148 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfilling, 145 active+clean; 2.3 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 797 op/s; 3234/114093 objects misplaced (2.835%); 19 MiB/s, 297 objects/s recovering 2024-09-21T11:48:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:17 smithi175 ceph-mon[32613]: osdmap e230: 8 total, 8 up, 8 in 2024-09-21T11:48:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:17 smithi175 ceph-mon[32613]: 6.14 scrub starts 2024-09-21T11:48:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:17 smithi175 ceph-mon[32613]: 6.14 scrub ok 2024-09-21T11:48:17.777 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:17.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:17.779 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:17.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:48:17.879 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:17.902 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:17.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:17.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:18.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:18.027 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:18.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:18.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:18.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:18.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:18.174 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:18.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:18.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:18.298 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:18.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:18.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:18.374 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:48:18.374 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:48:18.400 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:48:18.400 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:48:18.400 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:18.413 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:48:18.413 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:48:18.432 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:18.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:18.433 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:18.454 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:48:18.455 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:48:18.492 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:48:18.492 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:48:18.523 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:48:18.524 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:48:18.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:18.560 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:48:18.560 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:48:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:18.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:18.573 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:18.602 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:48:18.602 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:48:18.639 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:48:18.639 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:48:18.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:18.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:18.726 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:18.726 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[29710]: 3.10 scrub starts 2024-09-21T11:48:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[29710]: 3.10 scrub ok 2024-09-21T11:48:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[29710]: osdmap e231: 8 total, 8 up, 8 in 2024-09-21T11:48:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[29710]: 2.0 scrub starts 2024-09-21T11:48:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[29710]: 2.0 scrub ok 2024-09-21T11:48:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[29710]: 4.1c scrub starts 2024-09-21T11:48:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[29710]: 4.1c scrub ok 2024-09-21T11:48:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1577890143' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[36454]: 3.10 scrub starts 2024-09-21T11:48:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[36454]: 3.10 scrub ok 2024-09-21T11:48:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[36454]: osdmap e231: 8 total, 8 up, 8 in 2024-09-21T11:48:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[36454]: 2.0 scrub starts 2024-09-21T11:48:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[36454]: 2.0 scrub ok 2024-09-21T11:48:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[36454]: 4.1c scrub starts 2024-09-21T11:48:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[36454]: 4.1c scrub ok 2024-09-21T11:48:18.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:18 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1577890143' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:18 smithi175 ceph-mon[32613]: 3.10 scrub starts 2024-09-21T11:48:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:18 smithi175 ceph-mon[32613]: 3.10 scrub ok 2024-09-21T11:48:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:18 smithi175 ceph-mon[32613]: osdmap e231: 8 total, 8 up, 8 in 2024-09-21T11:48:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:18 smithi175 ceph-mon[32613]: 2.0 scrub starts 2024-09-21T11:48:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:18 smithi175 ceph-mon[32613]: 2.0 scrub ok 2024-09-21T11:48:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:18 smithi175 ceph-mon[32613]: 4.1c scrub starts 2024-09-21T11:48:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:18 smithi175 ceph-mon[32613]: 4.1c scrub ok 2024-09-21T11:48:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:18 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1577890143' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:18.828 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:18.869 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:18.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:18.870 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:18.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:19.006 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:19.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:19.007 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:19.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:19.147 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:19.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:19.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:19.249 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:19.286 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:19.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:19.287 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:19.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:19.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:19.424 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:19.424 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:19.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:19.554 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:19.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:19.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:19.655 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:19.689 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:19.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:19.690 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:19.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:48:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:19 smithi043 ceph-mon[29710]: pgmap v395: 148 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped, 1 peering, 145 active+clean; 2.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 790 op/s; 1142/106926 objects misplaced (1.068%); 40 MiB/s, 633 objects/s recovering 2024-09-21T11:48:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:19 smithi043 ceph-mon[29710]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-21T11:48:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:19 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:48:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:19 smithi043 ceph-mon[36454]: pgmap v395: 148 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped, 1 peering, 145 active+clean; 2.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 790 op/s; 1142/106926 objects misplaced (1.068%); 40 MiB/s, 633 objects/s recovering 2024-09-21T11:48:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:19 smithi043 ceph-mon[36454]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-21T11:48:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:19 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:48:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:19 smithi175 ceph-mon[32613]: pgmap v395: 148 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped, 1 peering, 145 active+clean; 2.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 790 op/s; 1142/106926 objects misplaced (1.068%); 40 MiB/s, 633 objects/s recovering 2024-09-21T11:48:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:19 smithi175 ceph-mon[32613]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-21T11:48:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:19 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:48:19.780 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:19.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:19.813 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:19.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:19.814 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:19.915 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:19.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:19.941 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:19.942 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:20.042 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:20.064 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:20.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:20.066 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:20.166 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:20.197 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:20.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:20.199 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:20.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:20.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:20.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:20.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:20.482 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:20.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:20.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:20.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:20.624 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:20.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:20.625 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:20.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:20.751 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:20.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:20.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:20.853 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:20.881 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:20.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:20.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:20.983 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:21.014 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:21.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:21.015 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:21.116 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:21.143 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:21.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:21.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:21.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:21.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:21.288 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:21.288 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:21.389 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:21.418 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:21.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:21.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:21.488 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:21.492 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:21.503 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:48:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:48:21.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:21.547 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:21.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:21.547 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:21.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:21.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:21.695 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:21.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:21 smithi043 ceph-mon[36454]: pgmap v396: 148 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped, 1 peering, 145 active+clean; 2.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 937 B/s rd, 0 B/s wr, 579 op/s; 1142/106926 objects misplaced (1.068%); 29 MiB/s, 464 objects/s recovering 2024-09-21T11:48:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:21 smithi043 ceph-mon[29710]: pgmap v396: 148 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped, 1 peering, 145 active+clean; 2.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 937 B/s rd, 0 B/s wr, 579 op/s; 1142/106926 objects misplaced (1.068%); 29 MiB/s, 464 objects/s recovering 2024-09-21T11:48:21.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:21 smithi175 ceph-mon[32613]: pgmap v396: 148 pgs: 1 active+remapped+backfilling, 1 active+clean+remapped, 1 peering, 145 active+clean; 2.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 937 B/s rd, 0 B/s wr, 579 op/s; 1142/106926 objects misplaced (1.068%); 29 MiB/s, 464 objects/s recovering 2024-09-21T11:48:21.797 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:21.824 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:21.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:21.825 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:21.865 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:48:21.871 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:48:21.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:21.963 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:21.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:21.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:22.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:22.108 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:22.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:22.109 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:22.210 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:22.235 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:22.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:22.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:22.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:22.371 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:22.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:22.372 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:22.473 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:22.506 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:22.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:22.507 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:22.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:22.644 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:22.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:22.645 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:22.746 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:22.777 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:22.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:22.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:22.879 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:22.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:22.906 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:22.907 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:22.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:48:22.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:22 smithi043 ceph-mon[29710]: 4.b scrub starts 2024-09-21T11:48:22.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:22 smithi043 ceph-mon[29710]: 4.b scrub ok 2024-09-21T11:48:22.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:22.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:22 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:48:22.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:22 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:48:22.976 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:22 smithi043 ceph-mon[36454]: 4.b scrub starts 2024-09-21T11:48:22.976 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:22 smithi043 ceph-mon[36454]: 4.b scrub ok 2024-09-21T11:48:22.976 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:22.976 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:22 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:48:22.976 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:22 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:48:23.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:23.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:22 smithi175 ceph-mon[32613]: 4.b scrub starts 2024-09-21T11:48:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:22 smithi175 ceph-mon[32613]: 4.b scrub ok 2024-09-21T11:48:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:22 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:48:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:22 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:48:23.035 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:23.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:23.036 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:23.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:23.189 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:23.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:23.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:23.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:23.325 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:23.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:23.326 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:23.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:23.440 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-21T11:48:19.919965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2323323443,"num_objects":35642,"num_object_clones":0,"num_object_copies":106926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1142,"num_objects_unfound":0,"num_objects_dirty":35642,"num_whiteouts":0,"num_read":1324,"num_read_kb":1181,"num_write":239290,"num_write_kb":8775498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23681,"num_bytes_recovered":1536891556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89681,"ondisk_log_size":89681,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":448,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9805160,"kb_used_data":7311940,"kb_used_omap":12,"kb_used_meta":2492787,"kb_avail":740156056,"statfs":{"total":767960285184,"available":757919801344,"internally_reserved":0,"allocated":7487426560,"data_stored":7484398232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2552614450},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-207028224,"num_objects":-3159,"num_object_clones":0,"num_object_copies":-9477,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-2817,"num_objects_unfound":0,"num_objects_dirty":-3159,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3159,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2536,"num_bytes_recovered":166199296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.461860"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":373,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.545391+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:48:16.545391+0000","last_peered":"2024-09-21T11:48:16.545391+0000","last_clean":"2024-09-21T11:48:16.545391+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:48:16.545391+0000","last_undegraded":"2024-09-21T11:48:16.545391+0000","last_fullsized":"2024-09-21T11:48:16.545391+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.863735+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":425,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464107+0000","last_change":"2024-09-21T11:47:03.433310+0000","last_active":"2024-09-21T11:48:15.464107+0000","last_peered":"2024-09-21T11:48:15.464107+0000","last_clean":"2024-09-21T11:48:15.464107+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:48:15.464107+0000","last_undegraded":"2024-09-21T11:48:15.464107+0000","last_fullsized":"2024-09-21T11:48:15.464107+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:44.553561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.553561+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.027789+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":64,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.467939+0000","last_change":"2024-09-21T11:47:03.372676+0000","last_active":"2024-09-21T11:48:15.467939+0000","last_peered":"2024-09-21T11:48:15.467939+0000","last_clean":"2024-09-21T11:48:15.467939+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:48:15.467939+0000","last_undegraded":"2024-09-21T11:48:15.467939+0000","last_fullsized":"2024-09-21T11:48:15.467939+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":381,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464322+0000","last_change":"2024-09-21T11:47:20.957722+0000","last_active":"2024-09-21T11:48:15.464322+0000","last_peered":"2024-09-21T11:48:15.464322+0000","last_clean":"2024-09-21T11:48:15.464322+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:48:15.464322+0000","last_undegraded":"2024-09-21T11:48:15.464322+0000","last_fullsized":"2024-09-21T11:48:15.464322+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:20.957679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.957679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:25.355154+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":361,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472561+0000","last_change":"2024-09-21T11:46:09.675727+0000","last_active":"2024-09-21T11:48:16.472561+0000","last_peered":"2024-09-21T11:48:16.472561+0000","last_clean":"2024-09-21T11:48:16.472561+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:48:16.472561+0000","last_undegraded":"2024-09-21T11:48:16.472561+0000","last_fullsized":"2024-09-21T11:48:16.472561+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:09.675148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:09.675148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":366,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472103+0000","last_change":"2024-09-21T11:46:55.804484+0000","last_active":"2024-09-21T11:48:16.472103+0000","last_peered":"2024-09-21T11:48:16.472103+0000","last_clean":"2024-09-21T11:48:16.472103+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:48:16.472103+0000","last_undegraded":"2024-09-21T11:48:16.472103+0000","last_fullsized":"2024-09-21T11:48:16.472103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.804419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.804419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":386,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472073+0000","last_change":"2024-09-21T11:45:07.788590+0000","last_active":"2024-09-21T11:48:16.472073+0000","last_peered":"2024-09-21T11:48:16.472073+0000","last_clean":"2024-09-21T11:48:16.472073+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:48:16.472073+0000","last_undegraded":"2024-09-21T11:48:16.472073+0000","last_fullsized":"2024-09-21T11:48:16.472073+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:07.788512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.788512+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":385,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464095+0000","last_change":"2024-09-21T11:47:03.371710+0000","last_active":"2024-09-21T11:48:15.464095+0000","last_peered":"2024-09-21T11:48:15.464095+0000","last_clean":"2024-09-21T11:48:15.464095+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:48:15.464095+0000","last_undegraded":"2024-09-21T11:48:15.464095+0000","last_fullsized":"2024-09-21T11:48:15.464095+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:16.876126+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":362,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524488+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:48:15.524488+0000","last_peered":"2024-09-21T11:48:15.524488+0000","last_clean":"2024-09-21T11:48:15.524488+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:48:15.524488+0000","last_undegraded":"2024-09-21T11:48:15.524488+0000","last_fullsized":"2024-09-21T11:48:15.524488+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:43.925152+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":391,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464602+0000","last_change":"2024-09-21T11:48:13.962997+0000","last_active":"2024-09-21T11:48:15.464602+0000","last_peered":"2024-09-21T11:48:15.464602+0000","last_clean":"2024-09-21T11:48:15.464602+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:48:15.464602+0000","last_undegraded":"2024-09-21T11:48:15.464602+0000","last_fullsized":"2024-09-21T11:48:15.464602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:13.962919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:13.962919+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:14.418938+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":370,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524831+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:48:15.524831+0000","last_peered":"2024-09-21T11:48:15.524831+0000","last_clean":"2024-09-21T11:48:15.524831+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:48:15.524831+0000","last_undegraded":"2024-09-21T11:48:15.524831+0000","last_fullsized":"2024-09-21T11:48:15.524831+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":374,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.544770+0000","last_change":"2024-09-21T11:47:03.385157+0000","last_active":"2024-09-21T11:48:16.544770+0000","last_peered":"2024-09-21T11:48:16.544770+0000","last_clean":"2024-09-21T11:48:16.544770+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:48:16.544770+0000","last_undegraded":"2024-09-21T11:48:16.544770+0000","last_fullsized":"2024-09-21T11:48:16.544770+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:06.385125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:06.385125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":392,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524226+0000","last_change":"2024-09-21T11:47:03.385245+0000","last_active":"2024-09-21T11:48:15.524226+0000","last_peered":"2024-09-21T11:48:15.524226+0000","last_clean":"2024-09-21T11:48:15.524226+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:48:15.524226+0000","last_undegraded":"2024-09-21T11:48:15.524226+0000","last_fullsized":"2024-09-21T11:48:15.524226+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":411,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464307+0000","last_change":"2024-09-21T11:48:13.005365+0000","last_active":"2024-09-21T11:48:15.464307+0000","last_peered":"2024-09-21T11:48:15.464307+0000","last_clean":"2024-09-21T11:48:15.464307+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:48:15.464307+0000","last_undegraded":"2024-09-21T11:48:15.464307+0000","last_fullsized":"2024-09-21T11:48:15.464307+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:48:13.005294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:13.005294+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:13.822018+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":376,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529485+0000","last_change":"2024-09-21T11:48:06.605705+0000","last_active":"2024-09-21T11:48:15.529485+0000","last_peered":"2024-09-21T11:48:15.529485+0000","last_clean":"2024-09-21T11:48:15.529485+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:48:15.529485+0000","last_undegraded":"2024-09-21T11:48:15.529485+0000","last_fullsized":"2024-09-21T11:48:15.529485+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:06.605645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.605645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.086689+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":380,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.526387+0000","last_change":"2024-09-21T11:47:03.380856+0000","last_active":"2024-09-21T11:48:15.526387+0000","last_peered":"2024-09-21T11:48:15.526387+0000","last_clean":"2024-09-21T11:48:15.526387+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:48:15.526387+0000","last_undegraded":"2024-09-21T11:48:15.526387+0000","last_fullsized":"2024-09-21T11:48:15.526387+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:13.288522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:13.288522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":376,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464765+0000","last_change":"2024-09-21T11:46:52.035524+0000","last_active":"2024-09-21T11:48:15.464765+0000","last_peered":"2024-09-21T11:48:15.464765+0000","last_clean":"2024-09-21T11:48:15.464765+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:48:15.464765+0000","last_undegraded":"2024-09-21T11:48:15.464765+0000","last_fullsized":"2024-09-21T11:48:15.464765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.035461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.035461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":364,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.545093+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:48:16.545093+0000","last_peered":"2024-09-21T11:48:16.545093+0000","last_clean":"2024-09-21T11:48:16.545093+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:48:16.545093+0000","last_undegraded":"2024-09-21T11:48:16.545093+0000","last_fullsized":"2024-09-21T11:48:16.545093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":469,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.470077+0000","last_change":"2024-09-21T11:47:03.424753+0000","last_active":"2024-09-21T11:48:15.470077+0000","last_peered":"2024-09-21T11:48:15.470077+0000","last_clean":"2024-09-21T11:48:15.470077+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:48:15.470077+0000","last_undegraded":"2024-09-21T11:48:15.470077+0000","last_fullsized":"2024-09-21T11:48:15.470077+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:40.530099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:40.530099+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":64,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.467530+0000","last_change":"2024-09-21T11:47:03.377667+0000","last_active":"2024-09-21T11:48:15.467530+0000","last_peered":"2024-09-21T11:48:15.467530+0000","last_clean":"2024-09-21T11:48:15.467530+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:48:15.467530+0000","last_undegraded":"2024-09-21T11:48:15.467530+0000","last_fullsized":"2024-09-21T11:48:15.467530+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.718478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.718478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":361,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.473228+0000","last_change":"2024-09-21T11:45:39.826902+0000","last_active":"2024-09-21T11:48:16.473228+0000","last_peered":"2024-09-21T11:48:16.473228+0000","last_clean":"2024-09-21T11:48:16.473228+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:48:16.473228+0000","last_undegraded":"2024-09-21T11:48:16.473228+0000","last_fullsized":"2024-09-21T11:48:16.473228+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:39.826701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:39.826701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":377,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464676+0000","last_change":"2024-09-21T11:47:03.371044+0000","last_active":"2024-09-21T11:48:15.464676+0000","last_peered":"2024-09-21T11:48:15.464676+0000","last_clean":"2024-09-21T11:48:15.464676+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:48:15.464676+0000","last_undegraded":"2024-09-21T11:48:15.464676+0000","last_fullsized":"2024-09-21T11:48:15.464676+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:45.093325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:45.093325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:15.837332+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":430,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.473587+0000","last_change":"2024-09-21T11:47:03.873307+0000","last_active":"2024-09-21T11:48:16.473587+0000","last_peered":"2024-09-21T11:48:16.473587+0000","last_clean":"2024-09-21T11:48:16.473587+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:48:16.473587+0000","last_undegraded":"2024-09-21T11:48:16.473587+0000","last_fullsized":"2024-09-21T11:48:16.473587+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:47:03.873239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.873239+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:22.833198+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":363,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464375+0000","last_change":"2024-09-21T11:45:37.670176+0000","last_active":"2024-09-21T11:48:15.464375+0000","last_peered":"2024-09-21T11:48:15.464375+0000","last_clean":"2024-09-21T11:48:15.464375+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:48:15.464375+0000","last_undegraded":"2024-09-21T11:48:15.464375+0000","last_fullsized":"2024-09-21T11:48:15.464375+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:37.670117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:37.670117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:50.969760+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":375,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529719+0000","last_change":"2024-09-21T11:46:47.611118+0000","last_active":"2024-09-21T11:48:15.529719+0000","last_peered":"2024-09-21T11:48:15.529719+0000","last_clean":"2024-09-21T11:48:15.529719+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:48:15.529719+0000","last_undegraded":"2024-09-21T11:48:15.529719+0000","last_fullsized":"2024-09-21T11:48:15.529719+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:46:47.611042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:47.611042+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":68,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.468119+0000","last_change":"2024-09-21T11:47:03.388975+0000","last_active":"2024-09-21T11:48:15.468119+0000","last_peered":"2024-09-21T11:48:15.468119+0000","last_clean":"2024-09-21T11:48:15.468119+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:48:15.468119+0000","last_undegraded":"2024-09-21T11:48:15.468119+0000","last_fullsized":"2024-09-21T11:48:15.468119+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:12.636105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.636105+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":392,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.545513+0000","last_change":"2024-09-21T11:45:48.385970+0000","last_active":"2024-09-21T11:48:16.545513+0000","last_peered":"2024-09-21T11:48:16.545513+0000","last_clean":"2024-09-21T11:48:16.545513+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:48:16.545513+0000","last_undegraded":"2024-09-21T11:48:16.545513+0000","last_fullsized":"2024-09-21T11:48:16.545513+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:45:48.385895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:48.385895+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":497,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.470221+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:48:15.470221+0000","last_peered":"2024-09-21T11:48:15.470221+0000","last_clean":"2024-09-21T11:48:15.470221+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:48:15.470221+0000","last_undegraded":"2024-09-21T11:48:15.470221+0000","last_fullsized":"2024-09-21T11:48:15.470221+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:38.235643+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"229'14960","reported_seq":20692,"reported_epoch":229,"state":"active+clean+remapped","last_fresh":"2024-09-21T11:48:15.876697+0000","last_change":"2024-09-21T11:48:15.876510+0000","last_active":"2024-09-21T11:48:15.876697+0000","last_peered":"2024-09-21T11:48:15.876697+0000","last_clean":"2024-09-21T11:48:15.876697+0000","last_became_active":"2024-09-21T11:47:04.533306+0000","last_became_peered":"2024-09-21T11:47:04.533306+0000","last_unstale":"2024-09-21T11:48:15.876697+0000","last_undegraded":"2024-09-21T11:48:15.876697+0000","last_fullsized":"2024-09-21T11:48:15.876697+0000","mapping_epoch":203,"log_start":"195'8400","ondisk_log_start":"195'8400","created":160,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":6560,"log_dups_size":0,"ondisk_log_size":6560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145883136,"num_objects":2226,"num_object_clones":0,"num_object_copies":6678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14960,"num_write_kb":549952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2785,"num_bytes_recovered":182517760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[2,5,6],"avail_no_missing":["2","1","5","6"],"object_location_counts":[{"shards":"2,5,6","objects":2226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":80,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.467895+0000","last_change":"2024-09-21T11:47:03.400465+0000","last_active":"2024-09-21T11:48:15.467895+0000","last_peered":"2024-09-21T11:48:15.467895+0000","last_clean":"2024-09-21T11:48:15.467895+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:48:15.467895+0000","last_undegraded":"2024-09-21T11:48:15.467895+0000","last_fullsized":"2024-09-21T11:48:15.467895+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:22.708839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.708839+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":395,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.473033+0000","last_change":"2024-09-21T11:47:03.391541+0000","last_active":"2024-09-21T11:48:16.473033+0000","last_peered":"2024-09-21T11:48:16.473033+0000","last_clean":"2024-09-21T11:48:16.473033+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:48:16.473033+0000","last_undegraded":"2024-09-21T11:48:16.473033+0000","last_fullsized":"2024-09-21T11:48:16.473033+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.836903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.836903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":471,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464806+0000","last_change":"2024-09-21T11:47:01.933697+0000","last_active":"2024-09-21T11:48:15.464806+0000","last_peered":"2024-09-21T11:48:15.464806+0000","last_clean":"2024-09-21T11:48:15.464806+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:48:15.464806+0000","last_undegraded":"2024-09-21T11:48:15.464806+0000","last_fullsized":"2024-09-21T11:48:15.464806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:01.933648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:01.933648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":372,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524220+0000","last_change":"2024-09-21T11:48:15.013107+0000","last_active":"2024-09-21T11:48:15.524220+0000","last_peered":"2024-09-21T11:48:15.524220+0000","last_clean":"2024-09-21T11:48:15.524220+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:48:15.524220+0000","last_undegraded":"2024-09-21T11:48:15.524220+0000","last_fullsized":"2024-09-21T11:48:15.524220+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:15.013058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:15.013058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.125286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"230'14922","reported_seq":15158,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.639681+0000","last_change":"2024-09-21T11:46:19.360961+0000","last_active":"2024-09-21T11:48:16.639681+0000","last_peered":"2024-09-21T11:48:16.639681+0000","last_clean":"2024-09-21T11:48:16.639681+0000","last_became_active":"2024-09-21T11:44:51.744224+0000","last_became_peered":"2024-09-21T11:44:51.744224+0000","last_unstale":"2024-09-21T11:48:16.639681+0000","last_undegraded":"2024-09-21T11:48:16.639681+0000","last_fullsized":"2024-09-21T11:48:16.639681+0000","mapping_epoch":160,"log_start":"196'9100","ondisk_log_start":"196'9100","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7561","last_scrub_stamp":"2024-09-21T11:46:19.360902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.360902+0000","objects_scrubbed":7459,"log_size":5822,"log_dups_size":0,"ondisk_log_size":5822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146538496,"num_objects":2236,"num_object_clones":0,"num_object_copies":6708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14922,"num_write_kb":549056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":410,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.471701+0000","last_change":"2024-09-21T11:46:58.789719+0000","last_active":"2024-09-21T11:48:16.471701+0000","last_peered":"2024-09-21T11:48:16.471701+0000","last_clean":"2024-09-21T11:48:16.471701+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:48:16.471701+0000","last_undegraded":"2024-09-21T11:48:16.471701+0000","last_fullsized":"2024-09-21T11:48:16.471701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:46:58.789625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.789625+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:19.133894+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":381,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.473630+0000","last_change":"2024-09-21T11:46:52.891978+0000","last_active":"2024-09-21T11:48:16.473630+0000","last_peered":"2024-09-21T11:48:16.473630+0000","last_clean":"2024-09-21T11:48:16.473630+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:48:16.473630+0000","last_undegraded":"2024-09-21T11:48:16.473630+0000","last_fullsized":"2024-09-21T11:48:16.473630+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.891793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.891793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:19.681093+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":363,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464170+0000","last_change":"2024-09-21T11:44:13.946705+0000","last_active":"2024-09-21T11:48:15.464170+0000","last_peered":"2024-09-21T11:48:15.464170+0000","last_clean":"2024-09-21T11:48:15.464170+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:48:15.464170+0000","last_undegraded":"2024-09-21T11:48:15.464170+0000","last_fullsized":"2024-09-21T11:48:15.464170+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:42.015785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":359,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464522+0000","last_change":"2024-09-21T11:46:24.717355+0000","last_active":"2024-09-21T11:48:15.464522+0000","last_peered":"2024-09-21T11:48:15.464522+0000","last_clean":"2024-09-21T11:48:15.464522+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:48:15.464522+0000","last_undegraded":"2024-09-21T11:48:15.464522+0000","last_fullsized":"2024-09-21T11:48:15.464522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.717285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.717285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:58.045440+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"229'14860","reported_seq":15048,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.725158+0000","last_change":"2024-09-21T11:48:05.746078+0000","last_active":"2024-09-21T11:48:15.725158+0000","last_peered":"2024-09-21T11:48:15.725158+0000","last_clean":"2024-09-21T11:48:15.725158+0000","last_became_active":"2024-09-21T11:44:51.744633+0000","last_became_peered":"2024-09-21T11:44:51.744633+0000","last_unstale":"2024-09-21T11:48:15.725158+0000","last_undegraded":"2024-09-21T11:48:15.725158+0000","last_fullsized":"2024-09-21T11:48:15.725158+0000","mapping_epoch":160,"log_start":"197'9400","ondisk_log_start":"197'9400","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'14372","last_scrub_stamp":"2024-09-21T11:48:05.746017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.746017+0000","objects_scrubbed":2702,"log_size":5460,"log_dups_size":0,"ondisk_log_size":5460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:10.815328+0000","scrub_duration":670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143851544,"num_objects":2196,"num_object_clones":0,"num_object_copies":6588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2196,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14860,"num_write_kb":545729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":404,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464318+0000","last_change":"2024-09-21T11:47:33.018713+0000","last_active":"2024-09-21T11:48:15.464318+0000","last_peered":"2024-09-21T11:48:15.464318+0000","last_clean":"2024-09-21T11:48:15.464318+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:48:15.464318+0000","last_undegraded":"2024-09-21T11:48:15.464318+0000","last_fullsized":"2024-09-21T11:48:15.464318+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:47:33.018639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:33.018639+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:41.906194+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":403,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.545782+0000","last_change":"2024-09-21T11:47:03.429013+0000","last_active":"2024-09-21T11:48:16.545782+0000","last_peered":"2024-09-21T11:48:16.545782+0000","last_clean":"2024-09-21T11:48:16.545782+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:48:16.545782+0000","last_undegraded":"2024-09-21T11:48:16.545782+0000","last_fullsized":"2024-09-21T11:48:16.545782+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:59.702813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.702813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":497,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.471589+0000","last_change":"2024-09-21T11:47:03.406087+0000","last_active":"2024-09-21T11:48:16.471589+0000","last_peered":"2024-09-21T11:48:16.471589+0000","last_clean":"2024-09-21T11:48:16.471589+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:48:16.471589+0000","last_undegraded":"2024-09-21T11:48:16.471589+0000","last_fullsized":"2024-09-21T11:48:16.471589+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:08.620519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.620519+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":361,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529659+0000","last_change":"2024-09-21T11:48:10.667288+0000","last_active":"2024-09-21T11:48:15.529659+0000","last_peered":"2024-09-21T11:48:15.529659+0000","last_clean":"2024-09-21T11:48:15.529659+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:48:15.529659+0000","last_undegraded":"2024-09-21T11:48:15.529659+0000","last_fullsized":"2024-09-21T11:48:15.529659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.667221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.667221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:11.606721+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"230'15167","reported_seq":15406,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.623829+0000","last_change":"2024-09-21T11:46:22.226583+0000","last_active":"2024-09-21T11:48:16.623829+0000","last_peered":"2024-09-21T11:48:16.623829+0000","last_clean":"2024-09-21T11:48:16.623829+0000","last_became_active":"2024-09-21T11:44:51.412662+0000","last_became_peered":"2024-09-21T11:44:51.412662+0000","last_unstale":"2024-09-21T11:48:16.623829+0000","last_undegraded":"2024-09-21T11:48:16.623829+0000","last_fullsized":"2024-09-21T11:48:16.623829+0000","mapping_epoch":160,"log_start":"197'9300","ondisk_log_start":"197'9300","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7855","last_scrub_stamp":"2024-09-21T11:46:22.226400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.226400+0000","objects_scrubbed":7719,"log_size":5867,"log_dups_size":0,"ondisk_log_size":5867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147783680,"num_objects":2255,"num_object_clones":0,"num_object_copies":6765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15167,"num_write_kb":557504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":76,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.466998+0000","last_change":"2024-09-21T11:47:03.398638+0000","last_active":"2024-09-21T11:48:15.466998+0000","last_peered":"2024-09-21T11:48:15.466998+0000","last_clean":"2024-09-21T11:48:15.466998+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:48:15.466998+0000","last_undegraded":"2024-09-21T11:48:15.466998+0000","last_fullsized":"2024-09-21T11:48:15.466998+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:12.227121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.227121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":386,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524609+0000","last_change":"2024-09-21T11:48:07.131578+0000","last_active":"2024-09-21T11:48:15.524609+0000","last_peered":"2024-09-21T11:48:15.524609+0000","last_clean":"2024-09-21T11:48:15.524609+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:48:15.524609+0000","last_undegraded":"2024-09-21T11:48:15.524609+0000","last_fullsized":"2024-09-21T11:48:15.524609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.131535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.131535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:08.103888+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":467,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.544769+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:48:16.544769+0000","last_peered":"2024-09-21T11:48:16.544769+0000","last_clean":"2024-09-21T11:48:16.544769+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:48:16.544769+0000","last_undegraded":"2024-09-21T11:48:16.544769+0000","last_fullsized":"2024-09-21T11:48:16.544769+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":361,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.544570+0000","last_change":"2024-09-21T11:46:28.010676+0000","last_active":"2024-09-21T11:48:16.544570+0000","last_peered":"2024-09-21T11:48:16.544570+0000","last_clean":"2024-09-21T11:48:16.544570+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:48:16.544570+0000","last_undegraded":"2024-09-21T11:48:16.544570+0000","last_fullsized":"2024-09-21T11:48:16.544570+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:28.010602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.010602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"230'14856","reported_seq":23988,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.716360+0000","last_change":"2024-09-21T11:47:54.309078+0000","last_active":"2024-09-21T11:48:16.716360+0000","last_peered":"2024-09-21T11:48:16.716360+0000","last_clean":"2024-09-21T11:48:16.716360+0000","last_became_active":"2024-09-21T11:47:54.270690+0000","last_became_peered":"2024-09-21T11:47:54.270690+0000","last_unstale":"2024-09-21T11:48:16.716360+0000","last_undegraded":"2024-09-21T11:48:16.716360+0000","last_fullsized":"2024-09-21T11:48:16.716360+0000","mapping_epoch":222,"log_start":"198'9800","ondisk_log_start":"198'9800","created":160,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'7360","last_scrub_stamp":"2024-09-21T11:46:17.038356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.038356+0000","objects_scrubbed":7242,"log_size":5056,"log_dups_size":0,"ondisk_log_size":5056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143654912,"num_objects":2192,"num_object_clones":0,"num_object_copies":6576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14856,"num_write_kb":545536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4441,"num_bytes_recovered":291045376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":404,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.544855+0000","last_change":"2024-09-21T11:47:03.750064+0000","last_active":"2024-09-21T11:48:16.544855+0000","last_peered":"2024-09-21T11:48:16.544855+0000","last_clean":"2024-09-21T11:48:16.544855+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:48:16.544855+0000","last_undegraded":"2024-09-21T11:48:16.544855+0000","last_fullsized":"2024-09-21T11:48:16.544855+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:47:03.749962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.749962+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":64,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.467615+0000","last_change":"2024-09-21T11:47:03.382675+0000","last_active":"2024-09-21T11:48:15.467615+0000","last_peered":"2024-09-21T11:48:15.467615+0000","last_clean":"2024-09-21T11:48:15.467615+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:48:15.467615+0000","last_undegraded":"2024-09-21T11:48:15.467615+0000","last_fullsized":"2024-09-21T11:48:15.467615+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:08.723593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.723593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":362,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469549+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:48:15.469549+0000","last_peered":"2024-09-21T11:48:15.469549+0000","last_clean":"2024-09-21T11:48:15.469549+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:48:15.469549+0000","last_undegraded":"2024-09-21T11:48:15.469549+0000","last_fullsized":"2024-09-21T11:48:15.469549+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.032409+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":64,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.467840+0000","last_change":"2024-09-21T11:47:03.379157+0000","last_active":"2024-09-21T11:48:15.467840+0000","last_peered":"2024-09-21T11:48:15.467840+0000","last_clean":"2024-09-21T11:48:15.467840+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:48:15.467840+0000","last_undegraded":"2024-09-21T11:48:15.467840+0000","last_fullsized":"2024-09-21T11:48:15.467840+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"229'15015","reported_seq":15166,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.750697+0000","last_change":"2024-09-21T11:44:51.745506+0000","last_active":"2024-09-21T11:48:15.750697+0000","last_peered":"2024-09-21T11:48:15.750697+0000","last_clean":"2024-09-21T11:48:15.750697+0000","last_became_active":"2024-09-21T11:44:51.745200+0000","last_became_peered":"2024-09-21T11:44:51.745200+0000","last_unstale":"2024-09-21T11:48:15.750697+0000","last_undegraded":"2024-09-21T11:48:15.750697+0000","last_fullsized":"2024-09-21T11:48:15.750697+0000","mapping_epoch":160,"log_start":"197'9600","ondisk_log_start":"197'9600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5415,"log_dups_size":0,"ondisk_log_size":5415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146866176,"num_objects":2241,"num_object_clones":0,"num_object_copies":6723,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15015,"num_write_kb":552192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"205'31","reported_seq":101,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.467561+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:48:15.467561+0000","last_peered":"2024-09-21T11:48:15.467561+0000","last_clean":"2024-09-21T11:48:15.467561+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:48:15.467561+0000","last_undegraded":"2024-09-21T11:48:15.467561+0000","last_fullsized":"2024-09-21T11:48:15.467561+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":369,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464848+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:48:15.464848+0000","last_peered":"2024-09-21T11:48:15.464848+0000","last_clean":"2024-09-21T11:48:15.464848+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:48:15.464848+0000","last_undegraded":"2024-09-21T11:48:15.464848+0000","last_fullsized":"2024-09-21T11:48:15.464848+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:39.997756+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":402,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524461+0000","last_change":"2024-09-21T11:47:20.000195+0000","last_active":"2024-09-21T11:48:15.524461+0000","last_peered":"2024-09-21T11:48:15.524461+0000","last_clean":"2024-09-21T11:48:15.524461+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:48:15.524461+0000","last_undegraded":"2024-09-21T11:48:15.524461+0000","last_fullsized":"2024-09-21T11:48:15.524461+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:47:20.000051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.000051+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:48.196772+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":378,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.545175+0000","last_change":"2024-09-21T11:47:03.385173+0000","last_active":"2024-09-21T11:48:16.545175+0000","last_peered":"2024-09-21T11:48:16.545175+0000","last_clean":"2024-09-21T11:48:16.545175+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:48:16.545175+0000","last_undegraded":"2024-09-21T11:48:16.545175+0000","last_fullsized":"2024-09-21T11:48:16.545175+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":353,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.471896+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:48:16.471896+0000","last_peered":"2024-09-21T11:48:16.471896+0000","last_clean":"2024-09-21T11:48:16.471896+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:48:16.471896+0000","last_undegraded":"2024-09-21T11:48:16.471896+0000","last_fullsized":"2024-09-21T11:48:16.471896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"230'14752","reported_seq":17105,"reported_epoch":230,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:48:16.729598+0000","last_change":"2024-09-21T11:48:13.435664+0000","last_active":"2024-09-21T11:48:16.729598+0000","last_peered":"2024-09-21T11:48:16.729598+0000","last_clean":"2024-09-21T11:47:02.342762+0000","last_became_active":"2024-09-21T11:47:04.531689+0000","last_became_peered":"2024-09-21T11:47:04.531689+0000","last_unstale":"2024-09-21T11:48:16.729598+0000","last_undegraded":"2024-09-21T11:48:16.729598+0000","last_fullsized":"2024-09-21T11:48:16.729598+0000","mapping_epoch":203,"log_start":"198'9700","ondisk_log_start":"198'9700","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5052,"log_dups_size":0,"ondisk_log_size":5052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145883136,"num_objects":2226,"num_object_clones":0,"num_object_copies":6678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1142,"num_objects_unfound":0,"num_objects_dirty":2226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14752,"num_write_kb":543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1095,"num_bytes_recovered":71761920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,2],"avail_no_missing":["3","2","4"],"object_location_counts":[{"shards":"2,3,4","objects":2226}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":76,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.468024+0000","last_change":"2024-09-21T11:47:03.399466+0000","last_active":"2024-09-21T11:48:15.468024+0000","last_peered":"2024-09-21T11:48:15.468024+0000","last_clean":"2024-09-21T11:48:15.468024+0000","last_became_active":"2024-09-21T11:47:03.377155+0000","last_became_peered":"2024-09-21T11:47:03.377155+0000","last_unstale":"2024-09-21T11:48:15.468024+0000","last_undegraded":"2024-09-21T11:48:15.468024+0000","last_fullsized":"2024-09-21T11:48:15.468024+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":68,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.466780+0000","last_change":"2024-09-21T11:47:03.391283+0000","last_active":"2024-09-21T11:48:15.466780+0000","last_peered":"2024-09-21T11:48:15.466780+0000","last_clean":"2024-09-21T11:48:15.466780+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:48:15.466780+0000","last_undegraded":"2024-09-21T11:48:15.466780+0000","last_fullsized":"2024-09-21T11:48:15.466780+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":388,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469359+0000","last_change":"2024-09-21T11:46:54.864288+0000","last_active":"2024-09-21T11:48:15.469359+0000","last_peered":"2024-09-21T11:48:15.469359+0000","last_clean":"2024-09-21T11:48:15.469359+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:48:15.469359+0000","last_undegraded":"2024-09-21T11:48:15.469359+0000","last_fullsized":"2024-09-21T11:48:15.469359+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:54.864074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.864074+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":371,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.076049+0000","last_change":"2024-09-21T11:48:16.076049+0000","last_active":"2024-09-21T11:48:16.076049+0000","last_peered":"2024-09-21T11:48:16.076049+0000","last_clean":"2024-09-21T11:48:16.076049+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:48:16.076049+0000","last_undegraded":"2024-09-21T11:48:16.076049+0000","last_fullsized":"2024-09-21T11:48:16.076049+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:16.076003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:16.076003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.071160+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":366,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529840+0000","last_change":"2024-09-21T11:48:07.613756+0000","last_active":"2024-09-21T11:48:15.529840+0000","last_peered":"2024-09-21T11:48:15.529840+0000","last_clean":"2024-09-21T11:48:15.529840+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:48:15.529840+0000","last_undegraded":"2024-09-21T11:48:15.529840+0000","last_fullsized":"2024-09-21T11:48:15.529840+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.613691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.613691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:32.231718+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"229'14704","reported_seq":24135,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.739345+0000","last_change":"2024-09-21T11:47:52.322057+0000","last_active":"2024-09-21T11:48:15.739345+0000","last_peered":"2024-09-21T11:48:15.739345+0000","last_clean":"2024-09-21T11:48:15.739345+0000","last_became_active":"2024-09-21T11:47:52.264063+0000","last_became_peered":"2024-09-21T11:47:52.264063+0000","last_unstale":"2024-09-21T11:48:15.739345+0000","last_undegraded":"2024-09-21T11:48:15.739345+0000","last_fullsized":"2024-09-21T11:48:15.739345+0000","mapping_epoch":220,"log_start":"198'10006","ondisk_log_start":"198'10006","created":160,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4698,"log_dups_size":0,"ondisk_log_size":4698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142737408,"num_objects":2178,"num_object_clones":0,"num_object_copies":6534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14704,"num_write_kb":540224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4633,"num_bytes_recovered":303628288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":428,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.546162+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:48:16.546162+0000","last_peered":"2024-09-21T11:48:16.546162+0000","last_clean":"2024-09-21T11:48:16.546162+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:48:16.546162+0000","last_undegraded":"2024-09-21T11:48:16.546162+0000","last_fullsized":"2024-09-21T11:48:16.546162+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":381,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.544645+0000","last_change":"2024-09-21T11:46:29.937866+0000","last_active":"2024-09-21T11:48:16.544645+0000","last_peered":"2024-09-21T11:48:16.544645+0000","last_clean":"2024-09-21T11:48:16.544645+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:48:16.544645+0000","last_undegraded":"2024-09-21T11:48:16.544645+0000","last_fullsized":"2024-09-21T11:48:16.544645+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.937799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.937799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":901,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529404+0000","last_change":"2024-09-21T11:47:03.389526+0000","last_active":"2024-09-21T11:48:15.529404+0000","last_peered":"2024-09-21T11:48:15.529404+0000","last_clean":"2024-09-21T11:48:15.529404+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:48:15.529404+0000","last_undegraded":"2024-09-21T11:48:15.529404+0000","last_fullsized":"2024-09-21T11:48:15.529404+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:46:29.761453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.761453+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":487,"num_read_kb":482,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":376,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524685+0000","last_change":"2024-09-21T11:47:22.025919+0000","last_active":"2024-09-21T11:48:15.524685+0000","last_peered":"2024-09-21T11:48:15.524685+0000","last_clean":"2024-09-21T11:48:15.524685+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:48:15.524685+0000","last_undegraded":"2024-09-21T11:48:15.524685+0000","last_fullsized":"2024-09-21T11:48:15.524685+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:22.025842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.025842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:29.017093+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":64,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.467421+0000","last_change":"2024-09-21T11:47:03.374507+0000","last_active":"2024-09-21T11:48:15.467421+0000","last_peered":"2024-09-21T11:48:15.467421+0000","last_clean":"2024-09-21T11:48:15.467421+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:48:15.467421+0000","last_undegraded":"2024-09-21T11:48:15.467421+0000","last_fullsized":"2024-09-21T11:48:15.467421+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"229'14852","reported_seq":15136,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.730538+0000","last_change":"2024-09-21T11:47:31.508923+0000","last_active":"2024-09-21T11:48:15.730538+0000","last_peered":"2024-09-21T11:48:15.730538+0000","last_clean":"2024-09-21T11:48:15.730538+0000","last_became_active":"2024-09-21T11:44:51.410701+0000","last_became_peered":"2024-09-21T11:44:51.410701+0000","last_unstale":"2024-09-21T11:48:15.730538+0000","last_undegraded":"2024-09-21T11:48:15.730538+0000","last_fullsized":"2024-09-21T11:48:15.730538+0000","mapping_epoch":160,"log_start":"196'8400","ondisk_log_start":"196'8400","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'12576","last_scrub_stamp":"2024-09-21T11:47:31.508822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:47:31.508822+0000","objects_scrubbed":4421,"log_size":6452,"log_dups_size":0,"ondisk_log_size":6452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.146813+0000","scrub_duration":1214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138805248,"num_objects":2118,"num_object_clones":0,"num_object_copies":6354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14852,"num_write_kb":543040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":488,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469987+0000","last_change":"2024-09-21T11:48:06.171363+0000","last_active":"2024-09-21T11:48:15.469987+0000","last_peered":"2024-09-21T11:48:15.469987+0000","last_clean":"2024-09-21T11:48:15.469987+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:48:15.469987+0000","last_undegraded":"2024-09-21T11:48:15.469987+0000","last_fullsized":"2024-09-21T11:48:15.469987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:48:06.171279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.171279+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:12.520316+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":381,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.471980+0000","last_change":"2024-09-21T11:46:57.833589+0000","last_active":"2024-09-21T11:48:16.471980+0000","last_peered":"2024-09-21T11:48:16.471980+0000","last_clean":"2024-09-21T11:48:16.471980+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:48:16.471980+0000","last_undegraded":"2024-09-21T11:48:16.471980+0000","last_fullsized":"2024-09-21T11:48:16.471980+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:57.833329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.833329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":433,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529474+0000","last_change":"2024-09-21T11:46:58.645189+0000","last_active":"2024-09-21T11:48:15.529474+0000","last_peered":"2024-09-21T11:48:15.529474+0000","last_clean":"2024-09-21T11:48:15.529474+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:48:15.529474+0000","last_undegraded":"2024-09-21T11:48:15.529474+0000","last_fullsized":"2024-09-21T11:48:15.529474+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:46:58.645122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.645122+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":370,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.530958+0000","last_change":"2024-09-21T11:47:25.395237+0000","last_active":"2024-09-21T11:48:15.530958+0000","last_peered":"2024-09-21T11:48:15.530958+0000","last_clean":"2024-09-21T11:48:15.530958+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:48:15.530958+0000","last_undegraded":"2024-09-21T11:48:15.530958+0000","last_fullsized":"2024-09-21T11:48:15.530958+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.395185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.395185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.204032+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":366,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524634+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:48:15.524634+0000","last_peered":"2024-09-21T11:48:15.524634+0000","last_clean":"2024-09-21T11:48:15.524634+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:48:15.524634+0000","last_undegraded":"2024-09-21T11:48:15.524634+0000","last_fullsized":"2024-09-21T11:48:15.524634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"229'14798","reported_seq":15062,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.856237+0000","last_change":"2024-09-21T11:48:09.830854+0000","last_active":"2024-09-21T11:48:15.856237+0000","last_peered":"2024-09-21T11:48:15.856237+0000","last_clean":"2024-09-21T11:48:15.856237+0000","last_became_active":"2024-09-21T11:44:51.410197+0000","last_became_peered":"2024-09-21T11:44:51.410197+0000","last_unstale":"2024-09-21T11:48:15.856237+0000","last_undegraded":"2024-09-21T11:48:15.856237+0000","last_fullsized":"2024-09-21T11:48:15.856237+0000","mapping_epoch":160,"log_start":"198'10000","ondisk_log_start":"198'10000","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"226'14544","last_scrub_stamp":"2024-09-21T11:48:09.830774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.830774+0000","objects_scrubbed":2470,"log_size":4798,"log_dups_size":0,"ondisk_log_size":4798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.225752+0000","scrub_duration":808,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144179200,"num_objects":2200,"num_object_clones":0,"num_object_copies":6600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14798,"num_write_kb":543936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":424,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464857+0000","last_change":"2024-09-21T11:48:05.072998+0000","last_active":"2024-09-21T11:48:15.464857+0000","last_peered":"2024-09-21T11:48:15.464857+0000","last_clean":"2024-09-21T11:48:15.464857+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:48:15.464857+0000","last_undegraded":"2024-09-21T11:48:15.464857+0000","last_fullsized":"2024-09-21T11:48:15.464857+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:05.072910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.072910+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:33.824160+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":387,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464410+0000","last_change":"2024-09-21T11:47:00.950268+0000","last_active":"2024-09-21T11:48:15.464410+0000","last_peered":"2024-09-21T11:48:15.464410+0000","last_clean":"2024-09-21T11:48:15.464410+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:48:15.464410+0000","last_undegraded":"2024-09-21T11:48:15.464410+0000","last_fullsized":"2024-09-21T11:48:15.464410+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:00.950208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:00.950208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":362,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524551+0000","last_change":"2024-09-21T11:45:11.312119+0000","last_active":"2024-09-21T11:48:15.524551+0000","last_peered":"2024-09-21T11:48:15.524551+0000","last_clean":"2024-09-21T11:48:15.524551+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:48:15.524551+0000","last_undegraded":"2024-09-21T11:48:15.524551+0000","last_fullsized":"2024-09-21T11:48:15.524551+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:11.312058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:11.312058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":360,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472083+0000","last_change":"2024-09-21T11:47:03.378923+0000","last_active":"2024-09-21T11:48:16.472083+0000","last_peered":"2024-09-21T11:48:16.472083+0000","last_clean":"2024-09-21T11:48:16.472083+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:48:16.472083+0000","last_undegraded":"2024-09-21T11:48:16.472083+0000","last_fullsized":"2024-09-21T11:48:16.472083+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:41:38.453579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:41:38.453579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"229'15158","reported_seq":21563,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.845448+0000","last_change":"2024-09-21T11:48:15.522333+0000","last_active":"2024-09-21T11:48:15.845448+0000","last_peered":"2024-09-21T11:48:15.845448+0000","last_clean":"2024-09-21T11:48:15.845448+0000","last_became_active":"2024-09-21T11:48:15.471002+0000","last_became_peered":"2024-09-21T11:48:15.471002+0000","last_unstale":"2024-09-21T11:48:15.845448+0000","last_undegraded":"2024-09-21T11:48:15.845448+0000","last_fullsized":"2024-09-21T11:48:15.845448+0000","mapping_epoch":228,"log_start":"198'10513","ondisk_log_start":"198'10513","created":160,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'10449","last_scrub_stamp":"2024-09-21T11:46:52.316998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.316998+0000","objects_scrubbed":7011,"log_size":4645,"log_dups_size":0,"ondisk_log_size":4645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144048128,"num_objects":2198,"num_object_clones":0,"num_object_copies":6594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15158,"num_write_kb":555392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3078,"num_bytes_recovered":201719808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":453,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.465100+0000","last_change":"2024-09-21T11:47:22.974837+0000","last_active":"2024-09-21T11:48:15.465100+0000","last_peered":"2024-09-21T11:48:15.465100+0000","last_clean":"2024-09-21T11:48:15.465100+0000","last_became_active":"2024-09-21T11:47:03.369099+0000","last_became_peered":"2024-09-21T11:47:03.369099+0000","last_unstale":"2024-09-21T11:48:15.465100+0000","last_undegraded":"2024-09-21T11:48:15.465100+0000","last_fullsized":"2024-09-21T11:48:15.465100+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:47:22.974761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.974761+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:31.827386+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":400,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469792+0000","last_change":"2024-09-21T11:47:03.365071+0000","last_active":"2024-09-21T11:48:15.469792+0000","last_peered":"2024-09-21T11:48:15.469792+0000","last_clean":"2024-09-21T11:48:15.469792+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:48:15.469792+0000","last_undegraded":"2024-09-21T11:48:15.469792+0000","last_fullsized":"2024-09-21T11:48:15.469792+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.692429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.692429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:54.181079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":64,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.467337+0000","last_change":"2024-09-21T11:47:03.375108+0000","last_active":"2024-09-21T11:48:15.467337+0000","last_peered":"2024-09-21T11:48:15.467337+0000","last_clean":"2024-09-21T11:48:15.467337+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:48:15.467337+0000","last_undegraded":"2024-09-21T11:48:15.467337+0000","last_fullsized":"2024-09-21T11:48:15.467337+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:50.008635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:50.008635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":375,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.470793+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:48:15.470793+0000","last_peered":"2024-09-21T11:48:15.470793+0000","last_clean":"2024-09-21T11:48:15.470793+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:48:15.470793+0000","last_undegraded":"2024-09-21T11:48:15.470793+0000","last_fullsized":"2024-09-21T11:48:15.470793+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"230'14811","reported_seq":23921,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.729907+0000","last_change":"2024-09-21T11:47:53.304799+0000","last_active":"2024-09-21T11:48:16.729907+0000","last_peered":"2024-09-21T11:48:16.729907+0000","last_clean":"2024-09-21T11:48:16.729907+0000","last_became_active":"2024-09-21T11:47:53.264856+0000","last_became_peered":"2024-09-21T11:47:53.264856+0000","last_unstale":"2024-09-21T11:48:16.729907+0000","last_undegraded":"2024-09-21T11:48:16.729907+0000","last_fullsized":"2024-09-21T11:48:16.729907+0000","mapping_epoch":221,"log_start":"198'9800","ondisk_log_start":"198'9800","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5011,"log_dups_size":0,"ondisk_log_size":5011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139264000,"num_objects":2125,"num_object_clones":0,"num_object_copies":6375,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14811,"num_write_kb":541952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4468,"num_bytes_recovered":292749312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":398,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464414+0000","last_change":"2024-09-21T11:46:11.730053+0000","last_active":"2024-09-21T11:48:15.464414+0000","last_peered":"2024-09-21T11:48:15.464414+0000","last_clean":"2024-09-21T11:48:15.464414+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:48:15.464414+0000","last_undegraded":"2024-09-21T11:48:15.464414+0000","last_fullsized":"2024-09-21T11:48:15.464414+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:11.729984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.729984+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.980319+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":365,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472969+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:48:16.472969+0000","last_peered":"2024-09-21T11:48:16.472969+0000","last_clean":"2024-09-21T11:48:16.472969+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:48:16.472969+0000","last_undegraded":"2024-09-21T11:48:16.472969+0000","last_fullsized":"2024-09-21T11:48:16.472969+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":479,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.470362+0000","last_change":"2024-09-21T11:46:52.834287+0000","last_active":"2024-09-21T11:48:15.470362+0000","last_peered":"2024-09-21T11:48:15.470362+0000","last_clean":"2024-09-21T11:48:15.470362+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:48:15.470362+0000","last_undegraded":"2024-09-21T11:48:15.470362+0000","last_fullsized":"2024-09-21T11:48:15.470362+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:52.834206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.834206+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":373,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464640+0000","last_change":"2024-09-21T11:48:07.992586+0000","last_active":"2024-09-21T11:48:15.464640+0000","last_peered":"2024-09-21T11:48:15.464640+0000","last_clean":"2024-09-21T11:48:15.464640+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:48:15.464640+0000","last_undegraded":"2024-09-21T11:48:15.464640+0000","last_fullsized":"2024-09-21T11:48:15.464640+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.992517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.992517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.884349+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"228'14902","reported_seq":21028,"reported_epoch":229,"state":"peering","last_fresh":"2024-09-21T11:48:15.468207+0000","last_change":"2024-09-21T11:48:15.468207+0000","last_active":"2024-09-21T11:48:15.468207+0000","last_peered":"2024-09-21T11:48:15.299235+0000","last_clean":"2024-09-21T11:47:02.336531+0000","last_became_active":"2024-09-21T11:47:04.492509+0000","last_became_peered":"2024-09-21T11:47:04.492509+0000","last_unstale":"2024-09-21T11:48:15.468207+0000","last_undegraded":"2024-09-21T11:48:15.468207+0000","last_fullsized":"2024-09-21T11:48:15.468207+0000","mapping_epoch":229,"log_start":"195'8300","ondisk_log_start":"195'8300","created":160,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7408","last_scrub_stamp":"2024-09-21T11:46:17.934135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.934135+0000","objects_scrubbed":7290,"log_size":6602,"log_dups_size":0,"ondisk_log_size":6602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"no scrub is scheduled","scrub_duration":3305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150339584,"num_objects":2294,"num_object_clones":0,"num_object_copies":6882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14902,"num_write_kb":550272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2943,"num_bytes_recovered":192872448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":410,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.544709+0000","last_change":"2024-09-21T11:47:03.414060+0000","last_active":"2024-09-21T11:48:16.544709+0000","last_peered":"2024-09-21T11:48:16.544709+0000","last_clean":"2024-09-21T11:48:16.544709+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:48:16.544709+0000","last_undegraded":"2024-09-21T11:48:16.544709+0000","last_fullsized":"2024-09-21T11:48:16.544709+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:42:15.417817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:15.417817+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":378,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524302+0000","last_change":"2024-09-21T11:48:08.172717+0000","last_active":"2024-09-21T11:48:15.524302+0000","last_peered":"2024-09-21T11:48:15.524302+0000","last_clean":"2024-09-21T11:48:15.524302+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:48:15.524302+0000","last_undegraded":"2024-09-21T11:48:15.524302+0000","last_fullsized":"2024-09-21T11:48:15.524302+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:08.172656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:08.172656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:18.830907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":363,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469186+0000","last_change":"2024-09-21T11:46:55.879988+0000","last_active":"2024-09-21T11:48:15.469186+0000","last_peered":"2024-09-21T11:48:15.469186+0000","last_clean":"2024-09-21T11:48:15.469186+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:48:15.469186+0000","last_undegraded":"2024-09-21T11:48:15.469186+0000","last_fullsized":"2024-09-21T11:48:15.469186+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.879761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.879761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":352,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.544586+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:48:16.544586+0000","last_peered":"2024-09-21T11:48:16.544586+0000","last_clean":"2024-09-21T11:48:16.544586+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:48:16.544586+0000","last_undegraded":"2024-09-21T11:48:16.544586+0000","last_fullsized":"2024-09-21T11:48:16.544586+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:23.420693+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"230'15048","reported_seq":15282,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.628692+0000","last_change":"2024-09-21T11:46:15.665868+0000","last_active":"2024-09-21T11:48:16.628692+0000","last_peered":"2024-09-21T11:48:16.628692+0000","last_clean":"2024-09-21T11:48:16.628692+0000","last_became_active":"2024-09-21T11:44:51.744226+0000","last_became_peered":"2024-09-21T11:44:51.744226+0000","last_unstale":"2024-09-21T11:48:16.628692+0000","last_undegraded":"2024-09-21T11:48:16.628692+0000","last_fullsized":"2024-09-21T11:48:16.628692+0000","mapping_epoch":160,"log_start":"197'9200","ondisk_log_start":"197'9200","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7351","last_scrub_stamp":"2024-09-21T11:46:15.665775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:15.665775+0000","objects_scrubbed":7260,"log_size":5848,"log_dups_size":0,"ondisk_log_size":5848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146407424,"num_objects":2234,"num_object_clones":0,"num_object_copies":6702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15048,"num_write_kb":553024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":459,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469695+0000","last_change":"2024-09-21T11:47:03.397050+0000","last_active":"2024-09-21T11:48:15.469695+0000","last_peered":"2024-09-21T11:48:15.469695+0000","last_clean":"2024-09-21T11:48:15.469695+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:48:15.469695+0000","last_undegraded":"2024-09-21T11:48:15.469695+0000","last_fullsized":"2024-09-21T11:48:15.469695+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:46:20.650403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.650403+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:51.167956+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":378,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.530121+0000","last_change":"2024-09-21T11:48:03.546909+0000","last_active":"2024-09-21T11:48:15.530121+0000","last_peered":"2024-09-21T11:48:15.530121+0000","last_clean":"2024-09-21T11:48:15.530121+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:48:15.530121+0000","last_undegraded":"2024-09-21T11:48:15.530121+0000","last_fullsized":"2024-09-21T11:48:15.530121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:03.546843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:03.546843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.594742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":379,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469793+0000","last_change":"2024-09-21T11:46:16.775013+0000","last_active":"2024-09-21T11:48:15.469793+0000","last_peered":"2024-09-21T11:48:15.469793+0000","last_clean":"2024-09-21T11:48:15.469793+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:48:15.469793+0000","last_undegraded":"2024-09-21T11:48:15.469793+0000","last_fullsized":"2024-09-21T11:48:15.469793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:16.774939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.774939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":366,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529398+0000","last_change":"2024-09-21T11:46:59.704981+0000","last_active":"2024-09-21T11:48:15.529398+0000","last_peered":"2024-09-21T11:48:15.529398+0000","last_clean":"2024-09-21T11:48:15.529398+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:48:15.529398+0000","last_undegraded":"2024-09-21T11:48:15.529398+0000","last_fullsized":"2024-09-21T11:48:15.529398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"229'14992","reported_seq":15232,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.785439+0000","last_change":"2024-09-21T11:46:27.656566+0000","last_active":"2024-09-21T11:48:15.785439+0000","last_peered":"2024-09-21T11:48:15.785439+0000","last_clean":"2024-09-21T11:48:15.785439+0000","last_became_active":"2024-09-21T11:44:51.745609+0000","last_became_peered":"2024-09-21T11:44:51.745609+0000","last_unstale":"2024-09-21T11:48:15.785439+0000","last_undegraded":"2024-09-21T11:48:15.785439+0000","last_fullsized":"2024-09-21T11:48:15.785439+0000","mapping_epoch":160,"log_start":"196'8600","ondisk_log_start":"196'8600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'8199","last_scrub_stamp":"2024-09-21T11:46:27.656360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.656360+0000","objects_scrubbed":8107,"log_size":6392,"log_dups_size":0,"ondisk_log_size":6392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:46.172763+0000","scrub_duration":2017,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147980288,"num_objects":2258,"num_object_clones":0,"num_object_copies":6774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14992,"num_write_kb":552000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":452,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464233+0000","last_change":"2024-09-21T11:48:04.111199+0000","last_active":"2024-09-21T11:48:15.464233+0000","last_peered":"2024-09-21T11:48:15.464233+0000","last_clean":"2024-09-21T11:48:15.464233+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:48:15.464233+0000","last_undegraded":"2024-09-21T11:48:15.464233+0000","last_fullsized":"2024-09-21T11:48:15.464233+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:48:04.111115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.111115+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:19.507697+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":408,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.530093+0000","last_change":"2024-09-21T11:47:26.411190+0000","last_active":"2024-09-21T11:48:15.530093+0000","last_peered":"2024-09-21T11:48:15.530093+0000","last_clean":"2024-09-21T11:48:15.530093+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:48:15.530093+0000","last_undegraded":"2024-09-21T11:48:15.530093+0000","last_fullsized":"2024-09-21T11:48:15.530093+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:26.411146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:26.411146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:48.131154+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":393,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469339+0000","last_change":"2024-09-21T11:47:03.385382+0000","last_active":"2024-09-21T11:48:15.469339+0000","last_peered":"2024-09-21T11:48:15.469339+0000","last_clean":"2024-09-21T11:48:15.469339+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:48:15.469339+0000","last_undegraded":"2024-09-21T11:48:15.469339+0000","last_fullsized":"2024-09-21T11:48:15.469339+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.766819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.766819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":375,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472655+0000","last_change":"2024-09-21T11:47:03.376001+0000","last_active":"2024-09-21T11:48:16.472655+0000","last_peered":"2024-09-21T11:48:16.472655+0000","last_clean":"2024-09-21T11:48:16.472655+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:48:16.472655+0000","last_undegraded":"2024-09-21T11:48:16.472655+0000","last_fullsized":"2024-09-21T11:48:16.472655+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.703712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.703712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"229'14944","reported_seq":15174,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.838797+0000","last_change":"2024-09-21T11:46:12.974904+0000","last_active":"2024-09-21T11:48:15.838797+0000","last_peered":"2024-09-21T11:48:15.838797+0000","last_clean":"2024-09-21T11:48:15.838797+0000","last_became_active":"2024-09-21T11:44:51.409710+0000","last_became_peered":"2024-09-21T11:44:51.409710+0000","last_unstale":"2024-09-21T11:48:15.838797+0000","last_undegraded":"2024-09-21T11:48:15.838797+0000","last_fullsized":"2024-09-21T11:48:15.838797+0000","mapping_epoch":160,"log_start":"197'9500","ondisk_log_start":"197'9500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7106","last_scrub_stamp":"2024-09-21T11:46:12.974687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.974687+0000","objects_scrubbed":7036,"log_size":5444,"log_dups_size":0,"ondisk_log_size":5444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1797,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148504576,"num_objects":2266,"num_object_clones":0,"num_object_copies":6798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14944,"num_write_kb":550720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":383,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.523907+0000","last_change":"2024-09-21T11:46:57.248649+0000","last_active":"2024-09-21T11:48:15.523907+0000","last_peered":"2024-09-21T11:48:15.523907+0000","last_clean":"2024-09-21T11:48:15.523907+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:48:15.523907+0000","last_undegraded":"2024-09-21T11:48:15.523907+0000","last_fullsized":"2024-09-21T11:48:15.523907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:57.248575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.248575+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:16.322877+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":398,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.530124+0000","last_change":"2024-09-21T11:48:09.683093+0000","last_active":"2024-09-21T11:48:15.530124+0000","last_peered":"2024-09-21T11:48:15.530124+0000","last_clean":"2024-09-21T11:48:15.530124+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:48:15.530124+0000","last_undegraded":"2024-09-21T11:48:15.530124+0000","last_fullsized":"2024-09-21T11:48:15.530124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:48:09.683008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.683008+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:38.676024+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":478,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529410+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:48:15.529410+0000","last_peered":"2024-09-21T11:48:15.529410+0000","last_clean":"2024-09-21T11:48:15.529410+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:48:15.529410+0000","last_undegraded":"2024-09-21T11:48:15.529410+0000","last_fullsized":"2024-09-21T11:48:15.529410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:24.037103+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":358,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524016+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:48:15.524016+0000","last_peered":"2024-09-21T11:48:15.524016+0000","last_clean":"2024-09-21T11:48:15.524016+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:48:15.524016+0000","last_undegraded":"2024-09-21T11:48:15.524016+0000","last_fullsized":"2024-09-21T11:48:15.524016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.006344+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":378,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469595+0000","last_change":"2024-09-21T11:45:46.470043+0000","last_active":"2024-09-21T11:48:15.469595+0000","last_peered":"2024-09-21T11:48:15.469595+0000","last_clean":"2024-09-21T11:48:15.469595+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:48:15.469595+0000","last_undegraded":"2024-09-21T11:48:15.469595+0000","last_fullsized":"2024-09-21T11:48:15.469595+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:46.469951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.469951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":408,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.473465+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:48:16.473465+0000","last_peered":"2024-09-21T11:48:16.473465+0000","last_clean":"2024-09-21T11:48:16.473465+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:48:16.473465+0000","last_undegraded":"2024-09-21T11:48:16.473465+0000","last_fullsized":"2024-09-21T11:48:16.473465+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":384,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524433+0000","last_change":"2024-09-21T11:47:25.905705+0000","last_active":"2024-09-21T11:48:15.524433+0000","last_peered":"2024-09-21T11:48:15.524433+0000","last_clean":"2024-09-21T11:48:15.524433+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:48:15.524433+0000","last_undegraded":"2024-09-21T11:48:15.524433+0000","last_fullsized":"2024-09-21T11:48:15.524433+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.905645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.905645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.034596+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":374,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529810+0000","last_change":"2024-09-21T11:47:28.380378+0000","last_active":"2024-09-21T11:48:15.529810+0000","last_peered":"2024-09-21T11:48:15.529810+0000","last_clean":"2024-09-21T11:48:15.529810+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:48:15.529810+0000","last_undegraded":"2024-09-21T11:48:15.529810+0000","last_fullsized":"2024-09-21T11:48:15.529810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:28.380245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:28.380245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:34.745499+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":401,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.545668+0000","last_change":"2024-09-21T11:46:28.973700+0000","last_active":"2024-09-21T11:48:16.545668+0000","last_peered":"2024-09-21T11:48:16.545668+0000","last_clean":"2024-09-21T11:48:16.545668+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:48:16.545668+0000","last_undegraded":"2024-09-21T11:48:16.545668+0000","last_fullsized":"2024-09-21T11:48:16.545668+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:28.973606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.973606+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":370,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529502+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:48:15.529502+0000","last_peered":"2024-09-21T11:48:15.529502+0000","last_clean":"2024-09-21T11:48:15.529502+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:48:15.529502+0000","last_undegraded":"2024-09-21T11:48:15.529502+0000","last_fullsized":"2024-09-21T11:48:15.529502+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:47.600096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":387,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.473393+0000","last_change":"2024-09-21T11:47:03.379216+0000","last_active":"2024-09-21T11:48:16.473393+0000","last_peered":"2024-09-21T11:48:16.473393+0000","last_clean":"2024-09-21T11:48:16.473393+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:48:16.473393+0000","last_undegraded":"2024-09-21T11:48:16.473393+0000","last_fullsized":"2024-09-21T11:48:16.473393+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:48.903980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:48.903980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":367,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469640+0000","last_change":"2024-09-21T11:44:45.577521+0000","last_active":"2024-09-21T11:48:15.469640+0000","last_peered":"2024-09-21T11:48:15.469640+0000","last_clean":"2024-09-21T11:48:15.469640+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:48:15.469640+0000","last_undegraded":"2024-09-21T11:48:15.469640+0000","last_fullsized":"2024-09-21T11:48:15.469640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.577451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.577451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":382,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524405+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:48:15.524405+0000","last_peered":"2024-09-21T11:48:15.524405+0000","last_clean":"2024-09-21T11:48:15.524405+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:48:15.524405+0000","last_undegraded":"2024-09-21T11:48:15.524405+0000","last_fullsized":"2024-09-21T11:48:15.524405+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":380,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.544934+0000","last_change":"2024-09-21T11:46:24.087039+0000","last_active":"2024-09-21T11:48:16.544934+0000","last_peered":"2024-09-21T11:48:16.544934+0000","last_clean":"2024-09-21T11:48:16.544934+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:48:16.544934+0000","last_undegraded":"2024-09-21T11:48:16.544934+0000","last_fullsized":"2024-09-21T11:48:16.544934+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.086918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.086918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":465,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472471+0000","last_change":"2024-09-21T11:47:02.849839+0000","last_active":"2024-09-21T11:48:16.472471+0000","last_peered":"2024-09-21T11:48:16.472471+0000","last_clean":"2024-09-21T11:48:16.472471+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:48:16.472471+0000","last_undegraded":"2024-09-21T11:48:16.472471+0000","last_fullsized":"2024-09-21T11:48:16.472471+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:02.849786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:02.849786+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:22.149526+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":367,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464972+0000","last_change":"2024-09-21T11:45:38.652432+0000","last_active":"2024-09-21T11:48:15.464972+0000","last_peered":"2024-09-21T11:48:15.464972+0000","last_clean":"2024-09-21T11:48:15.464972+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:48:15.464972+0000","last_undegraded":"2024-09-21T11:48:15.464972+0000","last_fullsized":"2024-09-21T11:48:15.464972+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:38.652380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:38.652380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:47.014191+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":414,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472680+0000","last_change":"2024-09-21T11:47:03.428846+0000","last_active":"2024-09-21T11:48:16.472680+0000","last_peered":"2024-09-21T11:48:16.472680+0000","last_clean":"2024-09-21T11:48:16.472680+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:48:16.472680+0000","last_undegraded":"2024-09-21T11:48:16.472680+0000","last_fullsized":"2024-09-21T11:48:16.472680+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":370,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529834+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:48:15.529834+0000","last_peered":"2024-09-21T11:48:15.529834+0000","last_clean":"2024-09-21T11:48:15.529834+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:48:15.529834+0000","last_undegraded":"2024-09-21T11:48:15.529834+0000","last_fullsized":"2024-09-21T11:48:15.529834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.604026+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":501,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524309+0000","last_change":"2024-09-21T11:47:03.415793+0000","last_active":"2024-09-21T11:48:15.524309+0000","last_peered":"2024-09-21T11:48:15.524309+0000","last_clean":"2024-09-21T11:48:15.524309+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:48:15.524309+0000","last_undegraded":"2024-09-21T11:48:15.524309+0000","last_fullsized":"2024-09-21T11:48:15.524309+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:10.174971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:10.174971+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":368,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.545346+0000","last_change":"2024-09-21T11:45:44.314664+0000","last_active":"2024-09-21T11:48:16.545346+0000","last_peered":"2024-09-21T11:48:16.545346+0000","last_clean":"2024-09-21T11:48:16.545346+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:48:16.545346+0000","last_undegraded":"2024-09-21T11:48:16.545346+0000","last_fullsized":"2024-09-21T11:48:16.545346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:44.314461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.314461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":440,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464813+0000","last_change":"2024-09-21T11:47:03.402357+0000","last_active":"2024-09-21T11:48:15.464813+0000","last_peered":"2024-09-21T11:48:15.464813+0000","last_clean":"2024-09-21T11:48:15.464813+0000","last_became_active":"2024-09-21T11:47:03.368296+0000","last_became_peered":"2024-09-21T11:47:03.368296+0000","last_unstale":"2024-09-21T11:48:15.464813+0000","last_undegraded":"2024-09-21T11:48:15.464813+0000","last_fullsized":"2024-09-21T11:48:15.464813+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:54.032795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.032795+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:23.550314+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":371,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.464701+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:48:15.464701+0000","last_peered":"2024-09-21T11:48:15.464701+0000","last_clean":"2024-09-21T11:48:15.464701+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:48:15.464701+0000","last_undegraded":"2024-09-21T11:48:15.464701+0000","last_fullsized":"2024-09-21T11:48:15.464701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:37.991785+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":64,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.466889+0000","last_change":"2024-09-21T11:47:03.379049+0000","last_active":"2024-09-21T11:48:15.466889+0000","last_peered":"2024-09-21T11:48:15.466889+0000","last_clean":"2024-09-21T11:48:15.466889+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:48:15.466889+0000","last_undegraded":"2024-09-21T11:48:15.466889+0000","last_fullsized":"2024-09-21T11:48:15.466889+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.269610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.269610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":353,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.471606+0000","last_change":"2024-09-21T11:45:04.770055+0000","last_active":"2024-09-21T11:48:16.471606+0000","last_peered":"2024-09-21T11:48:16.471606+0000","last_clean":"2024-09-21T11:48:16.471606+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:48:16.471606+0000","last_undegraded":"2024-09-21T11:48:16.471606+0000","last_fullsized":"2024-09-21T11:48:16.471606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:04.769963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:04.769963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":74,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.468160+0000","last_change":"2024-09-21T11:47:03.404473+0000","last_active":"2024-09-21T11:48:15.468160+0000","last_peered":"2024-09-21T11:48:15.468160+0000","last_clean":"2024-09-21T11:48:15.468160+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:48:15.468160+0000","last_undegraded":"2024-09-21T11:48:15.468160+0000","last_fullsized":"2024-09-21T11:48:15.468160+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:26.649189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:26.649189+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":378,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529989+0000","last_change":"2024-09-21T11:48:04.553772+0000","last_active":"2024-09-21T11:48:15.529989+0000","last_peered":"2024-09-21T11:48:15.529989+0000","last_clean":"2024-09-21T11:48:15.529989+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:48:15.529989+0000","last_undegraded":"2024-09-21T11:48:15.529989+0000","last_fullsized":"2024-09-21T11:48:15.529989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:04.553731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.553731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:21.248327+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":387,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472743+0000","last_change":"2024-09-21T11:47:03.377948+0000","last_active":"2024-09-21T11:48:16.472743+0000","last_peered":"2024-09-21T11:48:16.472743+0000","last_clean":"2024-09-21T11:48:16.472743+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:48:16.472743+0000","last_undegraded":"2024-09-21T11:48:16.472743+0000","last_fullsized":"2024-09-21T11:48:16.472743+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:53.848323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:53.848323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":391,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.544372+0000","last_change":"2024-09-21T11:47:03.408821+0000","last_active":"2024-09-21T11:48:16.544372+0000","last_peered":"2024-09-21T11:48:16.544372+0000","last_clean":"2024-09-21T11:48:16.544372+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:48:16.544372+0000","last_undegraded":"2024-09-21T11:48:16.544372+0000","last_fullsized":"2024-09-21T11:48:16.544372+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:45:43.314803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:43.314803+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":388,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.471804+0000","last_change":"2024-09-21T11:45:46.706221+0000","last_active":"2024-09-21T11:48:16.471804+0000","last_peered":"2024-09-21T11:48:16.471804+0000","last_clean":"2024-09-21T11:48:16.471804+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:48:16.471804+0000","last_undegraded":"2024-09-21T11:48:16.471804+0000","last_fullsized":"2024-09-21T11:48:16.471804+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:45:46.706130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.706130+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":384,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.524460+0000","last_change":"2024-09-21T11:48:10.126645+0000","last_active":"2024-09-21T11:48:15.524460+0000","last_peered":"2024-09-21T11:48:15.524460+0000","last_clean":"2024-09-21T11:48:15.524460+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:48:15.524460+0000","last_undegraded":"2024-09-21T11:48:15.524460+0000","last_fullsized":"2024-09-21T11:48:15.524460+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.126591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.126591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:10.829490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":354,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529219+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:48:15.529219+0000","last_peered":"2024-09-21T11:48:15.529219+0000","last_clean":"2024-09-21T11:48:15.529219+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:48:15.529219+0000","last_undegraded":"2024-09-21T11:48:15.529219+0000","last_fullsized":"2024-09-21T11:48:15.529219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:43.608235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":361,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472885+0000","last_change":"2024-09-21T11:45:45.730374+0000","last_active":"2024-09-21T11:48:16.472885+0000","last_peered":"2024-09-21T11:48:16.472885+0000","last_clean":"2024-09-21T11:48:16.472885+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:48:16.472885+0000","last_undegraded":"2024-09-21T11:48:16.472885+0000","last_fullsized":"2024-09-21T11:48:16.472885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:45.730169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:45.730169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":422,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.471597+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:48:16.471597+0000","last_peered":"2024-09-21T11:48:16.471597+0000","last_clean":"2024-09-21T11:48:16.471597+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:48:16.471597+0000","last_undegraded":"2024-09-21T11:48:16.471597+0000","last_fullsized":"2024-09-21T11:48:16.471597+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":372,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.544173+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:48:16.544173+0000","last_peered":"2024-09-21T11:48:16.544173+0000","last_clean":"2024-09-21T11:48:16.544173+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:48:16.544173+0000","last_undegraded":"2024-09-21T11:48:16.544173+0000","last_fullsized":"2024-09-21T11:48:16.544173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":370,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.469614+0000","last_change":"2024-09-21T11:48:09.104826+0000","last_active":"2024-09-21T11:48:15.469614+0000","last_peered":"2024-09-21T11:48:15.469614+0000","last_clean":"2024-09-21T11:48:15.469614+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:48:15.469614+0000","last_undegraded":"2024-09-21T11:48:15.469614+0000","last_fullsized":"2024-09-21T11:48:15.469614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:09.104767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.104767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.005085+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":358,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.529342+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:48:15.529342+0000","last_peered":"2024-09-21T11:48:15.529342+0000","last_clean":"2024-09-21T11:48:15.529342+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:48:15.529342+0000","last_undegraded":"2024-09-21T11:48:15.529342+0000","last_fullsized":"2024-09-21T11:48:15.529342+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:49.549791+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":64,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.467367+0000","last_change":"2024-09-21T11:47:03.374194+0000","last_active":"2024-09-21T11:48:15.467367+0000","last_peered":"2024-09-21T11:48:15.467367+0000","last_clean":"2024-09-21T11:48:15.467367+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:48:15.467367+0000","last_undegraded":"2024-09-21T11:48:15.467367+0000","last_fullsized":"2024-09-21T11:48:15.467367+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:27.297630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.297630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":387,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.472124+0000","last_change":"2024-09-21T11:47:03.377189+0000","last_active":"2024-09-21T11:48:16.472124+0000","last_peered":"2024-09-21T11:48:16.472124+0000","last_clean":"2024-09-21T11:48:16.472124+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:48:16.472124+0000","last_undegraded":"2024-09-21T11:48:16.472124+0000","last_fullsized":"2024-09-21T11:48:16.472124+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:11.587469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.587469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":439,"reported_epoch":229,"state":"active+clean","last_fresh":"2024-09-21T11:48:15.470220+0000","last_change":"2024-09-21T11:47:03.392087+0000","last_active":"2024-09-21T11:48:15.470220+0000","last_peered":"2024-09-21T11:48:15.470220+0000","last_clean":"2024-09-21T11:48:15.470220+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:48:15.470220+0000","last_undegraded":"2024-09-21T11:48:15.470220+0000","last_fullsized":"2024-09-21T11:48:15.470220+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:16.675872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.675872+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:41.170726+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":373,"reported_epoch":230,"state":"active+clean","last_fresh":"2024-09-21T11:48:16.545432+0000","last_change":"2024-09-21T11:45:12.377954+0000","last_active":"2024-09-21T11:48:16.545432+0000","last_peered":"2024-09-21T11:48:16.545432+0000","last_clean":"2024-09-21T11:48:16.545432+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:48:16.545432+0000","last_undegraded":"2024-09-21T11:48:16.545432+0000","last_fullsized":"2024-09-21T11:48:16.545432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:12.377880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:12.377880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":2322726936,"num_objects":35443,"num_object_clones":0,"num_object_copies":106329,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1142,"num_objects_unfound":0,"num_objects_dirty":35443,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":238741,"num_write_kb":8773825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23443,"num_bytes_recovered":1536294912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7308521472,"data_stored":7308509256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89122,"ondisk_log_size":89122,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4814,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":396,"ondisk_log_size":396,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":487,"num_read_kb":482,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234088,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1039380,"kb_used_data":719420,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":92705772,"statfs":{"total":95995035648,"available":94930710528,"internally_reserved":0,"allocated":736686080,"data_stored":736306757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463021,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1009844,"kb_used_data":717664,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":92735308,"statfs":{"total":95995035648,"available":94960955392,"internally_reserved":0,"allocated":734887936,"data_stored":734530642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":299170246},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724658,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1563716,"kb_used_data":1275696,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":92181436,"statfs":{"total":95995035648,"available":94393790464,"internally_reserved":0,"allocated":1306312704,"data_stored":1305937807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":294910406},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920888,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":980356,"kb_used_data":724276,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":92764796,"statfs":{"total":95995035648,"available":94991151104,"internally_reserved":0,"allocated":741658624,"data_stored":741280475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":262207941},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117118,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1212292,"kb_used_data":856240,"kb_used_omap":1,"kb_used_meta":356030,"kb_avail":92532860,"statfs":{"total":95995035648,"available":94753648640,"internally_reserved":0,"allocated":876789760,"data_stored":876421495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":364575175},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411460,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1699132,"kb_used_data":1284776,"kb_used_omap":1,"kb_used_meta":414334,"kb_avail":92046020,"statfs":{"total":95995035648,"available":94255124480,"internally_reserved":0,"allocated":1315610624,"data_stored":1315250846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":424278469},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1140160,"kb_used_data":883816,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":92604992,"statfs":{"total":95995035648,"available":94827511808,"internally_reserved":0,"allocated":905027584,"data_stored":904603786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":262207942},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738513,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1160280,"kb_used_data":850052,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":92584872,"statfs":{"total":95995035648,"available":94806908928,"internally_reserved":0,"allocated":870453248,"data_stored":870066424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":317651397},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":866058240,"data_stored":866058240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":908066816,"data_stored":908066816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1314852864,"data_stored":1314848792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":870055936,"data_stored":870055936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":735772672,"data_stored":735772672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1151275008,"data_stored":1151270936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":729808896,"data_stored":729808896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":732631040,"data_stored":732626968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:48:23.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph status --format=json 2024-09-21T11:48:23.480 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:23.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:23.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:23.529 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:48:23.582 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:23.613 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:23.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:23.614 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[36454]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:23.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[36454]: pgmap v397: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.9 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.17k op/s; 149/93810 objects misplaced (0.159%); 34 MiB/s, 536 objects/s recovering 2024-09-21T11:48:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[36454]: osdmap e232: 8 total, 8 up, 8 in 2024-09-21T11:48:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:48:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:48:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[36454]: 3.1a scrub starts 2024-09-21T11:48:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[36454]: 3.1a scrub ok 2024-09-21T11:48:23.646 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3125920679' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:48:23.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[29710]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:23.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[29710]: pgmap v397: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.9 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.17k op/s; 149/93810 objects misplaced (0.159%); 34 MiB/s, 536 objects/s recovering 2024-09-21T11:48:23.646 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[29710]: osdmap e232: 8 total, 8 up, 8 in 2024-09-21T11:48:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:48:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:48:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[29710]: 3.1a scrub starts 2024-09-21T11:48:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[29710]: 3.1a scrub ok 2024-09-21T11:48:23.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3125920679' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:48:23.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:23.759 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:23.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:23.760 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:23.810 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:48:23.811 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:48:23.811 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:48:23.811 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:48:23.811 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:48:23.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:23.884 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:23.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:23.885 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:23.986 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:24.007 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:24.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:24.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:23 smithi175 ceph-mon[32613]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:23 smithi175 ceph-mon[32613]: pgmap v397: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.9 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.17k op/s; 149/93810 objects misplaced (0.159%); 34 MiB/s, 536 objects/s recovering 2024-09-21T11:48:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:23 smithi175 ceph-mon[32613]: osdmap e232: 8 total, 8 up, 8 in 2024-09-21T11:48:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:23 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:48:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:23 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:48:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:23 smithi175 ceph-mon[32613]: 3.1a scrub starts 2024-09-21T11:48:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:23 smithi175 ceph-mon[32613]: 3.1a scrub ok 2024-09-21T11:48:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3125920679' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:48:24.109 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:24.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:24.130 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:24.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:24.231 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:24.256 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:24.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:24.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:24.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:24.382 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:24.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:24.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:24.484 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:24.505 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:24.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:24.506 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:24.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:48:24.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:24.691 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:24.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:24.692 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[29710]: 5.f scrub starts 2024-09-21T11:48:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[29710]: 5.f scrub ok 2024-09-21T11:48:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[29710]: 6.c scrub starts 2024-09-21T11:48:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[29710]: 6.c scrub ok 2024-09-21T11:48:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3125920679' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:48:24.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[29710]: osdmap e233: 8 total, 8 up, 8 in 2024-09-21T11:48:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[36454]: 5.f scrub starts 2024-09-21T11:48:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[36454]: 5.f scrub ok 2024-09-21T11:48:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[36454]: 6.c scrub starts 2024-09-21T11:48:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[36454]: 6.c scrub ok 2024-09-21T11:48:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3125920679' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:48:24.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:24 smithi043 ceph-mon[36454]: osdmap e233: 8 total, 8 up, 8 in 2024-09-21T11:48:24.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:24.820 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:24.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:24.821 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:24.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:24.950 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:24.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:24.951 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:24 smithi175 ceph-mon[32613]: 5.f scrub starts 2024-09-21T11:48:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:24 smithi175 ceph-mon[32613]: 5.f scrub ok 2024-09-21T11:48:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:24 smithi175 ceph-mon[32613]: 6.c scrub starts 2024-09-21T11:48:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:24 smithi175 ceph-mon[32613]: 6.c scrub ok 2024-09-21T11:48:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:24 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:24 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3125920679' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:48:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:24 smithi175 ceph-mon[32613]: osdmap e233: 8 total, 8 up, 8 in 2024-09-21T11:48:25.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:25.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:25.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:25.148 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:25.165 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:25.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:25.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:25.220 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:25.220 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:25.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:25.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:25.370 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:25.462 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:48:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:48:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:48:25.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:25.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:25.519 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:25.519 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:25.620 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:25.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:25.652 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:25.652 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:25.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:25 smithi043 ceph-mon[36454]: pgmap v400: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.9 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 794 B/s rd, 0 B/s wr, 679 op/s; 149/93810 objects misplaced (0.159%); 9.0 MiB/s, 143 objects/s recovering 2024-09-21T11:48:25.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:25 smithi043 ceph-mon[36454]: osdmap e234: 8 total, 8 up, 8 in 2024-09-21T11:48:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:25 smithi043 ceph-mon[29710]: pgmap v400: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.9 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 794 B/s rd, 0 B/s wr, 679 op/s; 149/93810 objects misplaced (0.159%); 9.0 MiB/s, 143 objects/s recovering 2024-09-21T11:48:25.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:25 smithi043 ceph-mon[29710]: osdmap e234: 8 total, 8 up, 8 in 2024-09-21T11:48:25.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:25.782 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:25.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:25.783 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:25.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:25.911 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:25.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:25.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:25 smithi175 ceph-mon[32613]: pgmap v400: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.9 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 794 B/s rd, 0 B/s wr, 679 op/s; 149/93810 objects misplaced (0.159%); 9.0 MiB/s, 143 objects/s recovering 2024-09-21T11:48:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:25 smithi175 ceph-mon[32613]: osdmap e234: 8 total, 8 up, 8 in 2024-09-21T11:48:26.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:26.048 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:48:26.048 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:48:26.051 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:26.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:26.052 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:26.060 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:26.095 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:48:26.096 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:48:26.137 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:48:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:48:26.153 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:26.176 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:48:26.176 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:48:26.181 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:26.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:26.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:26.217 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:48:26.217 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:48:26.251 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:48:26.251 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:48:26.282 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:26.290 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:48:26.291 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:48:26.307 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:26.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:26.308 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:26.333 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:48:26.333 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:48:26.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:26.444 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:26.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:26.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:26.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:26.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:26.573 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:26.573 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:26.633 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:48:26.633 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"4f2c5252-780d-11ef-baf6-efdc52797490","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":758,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":234,"num_osds":8,"num_up_osds":8,"osd_up_since":1726918789,"num_in_osds":8,"osd_in_since":1726919222,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":147},{"state_name":"active+remapped+backfilling","count":1}],"num_pgs":148,"num_pools":7,"num_objects":29828,"data_bytes":1942297139,"bytes_used":8888025088,"bytes_avail":759072260096,"bytes_total":767960285184,"misplaced_objects":149,"misplaced_total":89484,"misplaced_ratio":0.0016651021411649011,"recovering_objects_per_sec":154,"recovering_bytes_per_sec":10111354,"recovering_keys_per_sec":0,"num_objects_recovered":926,"num_bytes_recovered":60686336,"num_keys_recovered":0,"read_bytes_sec":1706,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":968},"fsmap":{"epoch":1,"btime":"2024-09-21T11:34:07:249039+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.43:8443/","prometheus":"http://172.21.15.43:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-21T11:41:31.702566+0000","services":{"rgw":{"daemons":{"summary":"","14445":{"start_epoch":12,"start_stamp":"2024-09-21T11:40:07.452481+0000","gid":14445,"addr":"172.21.15.43:0/2311158182","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev)","ceph_version_short":"19.3.0-5103-g5c1cdc3c","container_hostname":"smithi043","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi043","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024","kernel_version":"5.14.0-508.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32493484","num_handles":"1","os":"Linux","pid":"2","realm_id":"","realm_name":"","zone_id":"0b57ffae-5838-4433-ac11-de7e749d627a","zone_name":"default","zonegroup_id":"3832d8a7-33d0-4a2e-9811-afaccd40f4de","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T11:48:26.674 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:26.701 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:26.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:26.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:26.803 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:26.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:26.834 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:26.834 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:26.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:26.970 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:26.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:26.971 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:26.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:26 smithi043 ceph-mon[29710]: pgmap v402: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.8 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 970 op/s; 149/89484 objects misplaced (0.167%); 9.6 MiB/s, 154 objects/s recovering 2024-09-21T11:48:26.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:26 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/424794183' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:26.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:26 smithi043 ceph-mon[36454]: pgmap v402: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.8 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 970 op/s; 149/89484 objects misplaced (0.167%); 9.6 MiB/s, 154 objects/s recovering 2024-09-21T11:48:26.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:26 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/424794183' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:26 smithi175 ceph-mon[32613]: pgmap v402: 148 pgs: 1 active+remapped+backfilling, 147 active+clean; 1.8 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 970 op/s; 149/89484 objects misplaced (0.167%); 9.6 MiB/s, 154 objects/s recovering 2024-09-21T11:48:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:26 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/424794183' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:27.072 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:27.105 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:27.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:27.106 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:27.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:48:27.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:27.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:27.269 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:27.269 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:27.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:27.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:27.406 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:27.406 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:27.452 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-21T11:48:27.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:48:27.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:27.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:27.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:27.655 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:27.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:27.685 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:27 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1675675736' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T11:48:27.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:27 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1675675736' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T11:48:27.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:27.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:27.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:27.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:27.961 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:27.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:27.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:28.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:27 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1675675736' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T11:48:28.063 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:28.089 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:28.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:28.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:28.190 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:28.219 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:28.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:28.220 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:28.320 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:28.346 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:28.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:28.347 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:28.377 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:48:28.378 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:48:28.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:28.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:28.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:28.600 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:28.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:28.628 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:28.628 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:28.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:28.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:28.768 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:28.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:28.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:28.909 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:28.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:28 smithi043 ceph-mon[36454]: pgmap v403: 148 pgs: 148 active+clean; 1.6 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 779 op/s; 1.5 MiB/s, 23 objects/s recovering 2024-09-21T11:48:28.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:28 smithi043 ceph-mon[29710]: pgmap v403: 148 pgs: 148 active+clean; 1.6 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 779 op/s; 1.5 MiB/s, 23 objects/s recovering 2024-09-21T11:48:29.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:28 smithi175 ceph-mon[32613]: pgmap v403: 148 pgs: 148 active+clean; 1.6 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 779 op/s; 1.5 MiB/s, 23 objects/s recovering 2024-09-21T11:48:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:29.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:29.051 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:29.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:29.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:29.194 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:29.194 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:29.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:29.330 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:29.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:29.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:29.432 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:29.465 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:29.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:29.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:29.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:29.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:29.590 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:29.590 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:29.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:29.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:29.727 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:29.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:29.738 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:29.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:29.862 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:29.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:29.865 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:29.966 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:29.994 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:29.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:29.995 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:30.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:48:30.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:30.116 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:30.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:30.117 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:30.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:30.259 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:30.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:30.260 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:30.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:30.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:30.412 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:30.412 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:30.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:30.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:30.556 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:30.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:30.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:30.699 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:30.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:30.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:30.838 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:30.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:30.980 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:30.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:30.981 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:31.071 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:48:31.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:31.167 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:31.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:31.168 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:31.235 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:30 smithi175 ceph-mon[32613]: pgmap v404: 148 pgs: 148 active+clean; 1.6 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 691 B/s rd, 0 B/s wr, 631 op/s; 1.2 MiB/s, 19 objects/s recovering 2024-09-21T11:48:31.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:48:31.236 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:31.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:30 smithi043 ceph-mon[36454]: pgmap v404: 148 pgs: 148 active+clean; 1.6 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 691 B/s rd, 0 B/s wr, 631 op/s; 1.2 MiB/s, 19 objects/s recovering 2024-09-21T11:48:31.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:48:31.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:31.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:30 smithi043 ceph-mon[29710]: pgmap v404: 148 pgs: 148 active+clean; 1.6 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 691 B/s rd, 0 B/s wr, 631 op/s; 1.2 MiB/s, 19 objects/s recovering 2024-09-21T11:48:31.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:48:31.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:31.268 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:31.295 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:31.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:31.296 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:31.372 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:31.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:31.455 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:31.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:31.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:31.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:48:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:48:31.556 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:31.580 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:31.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:31.581 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:31.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:31.734 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:31.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:31.735 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:31.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:31.905 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:31.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:31.906 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:32.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:32.051 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:32.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:32.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:32.152 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:32.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.161 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]} : dispatch 2024-09-21T11:48:32.161 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.161 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.161 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]} : dispatch 2024-09-21T11:48:32.161 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.161 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:32.161 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.161 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]} : dispatch 2024-09-21T11:48:32.161 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.162 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.162 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]} : dispatch 2024-09-21T11:48:32.162 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.162 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:31 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:32.186 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:32.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:32.187 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]} : dispatch 2024-09-21T11:48:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]} : dispatch 2024-09-21T11:48:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]} : dispatch 2024-09-21T11:48:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:31 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:32.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:32.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:32.342 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:32.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:32.472 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:32.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:32.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:32.508 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:32.574 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:32.602 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:32.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:32.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:32.622 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:48:32.625 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:48:32.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:32.738 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:32.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:32.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:32.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:32.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:32.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:32.968 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:48:32.968 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:48:32.968 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:48:32.968 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:48:32.969 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:48:32.969 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:48:32.969 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:48:32.969 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:48:32.978 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:33.001 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:33.001 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:48:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:48:33.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:33.003 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:33.007 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:48:33.036 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:48:33.036 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:48:33.076 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:48:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:48:33.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:33.113 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:48:33.113 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:48:33.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:33.143 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:33.143 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:33.244 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:33.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[29710]: pgmap v405: 148 pgs: 148 active+clean; 1.3 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s; 1.1 MiB/s, 16 objects/s recovering 2024-09-21T11:48:33.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-21T11:48:33.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-21T11:48:33.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]': finished 2024-09-21T11:48:33.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[29710]: osdmap e235: 8 total, 8 up, 8 in 2024-09-21T11:48:33.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2567765167' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:33.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1032549317' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:33.258 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[36454]: pgmap v405: 148 pgs: 148 active+clean; 1.3 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s; 1.1 MiB/s, 16 objects/s recovering 2024-09-21T11:48:33.258 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-21T11:48:33.259 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-21T11:48:33.259 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]': finished 2024-09-21T11:48:33.259 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[36454]: osdmap e235: 8 total, 8 up, 8 in 2024-09-21T11:48:33.259 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2567765167' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:33.259 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:33 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1032549317' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:33.278 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:33.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:33.279 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:33.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:33.412 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:33.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:33.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:33.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:33 smithi175 ceph-mon[32613]: pgmap v405: 148 pgs: 148 active+clean; 1.3 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s; 1.1 MiB/s, 16 objects/s recovering 2024-09-21T11:48:33.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-21T11:48:33.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-21T11:48:33.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]': finished 2024-09-21T11:48:33.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:33 smithi175 ceph-mon[32613]: osdmap e235: 8 total, 8 up, 8 in 2024-09-21T11:48:33.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:33 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2567765167' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:33.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:33 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1032549317' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:33.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:33.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:33.548 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:33.548 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:33.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:33.685 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:33.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:33.686 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:33.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:33.824 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:33.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:33.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:33.926 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:33.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:33.962 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:33.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:34.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:34.092 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:34.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:34.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:34.163 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-21T11:48:32.003059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1362369075,"num_objects":20979,"num_object_clones":0,"num_object_copies":62937,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20979,"num_whiteouts":0,"num_read":1339,"num_read_kb":1196,"num_write":253953,"num_write_kb":8775498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24749,"num_bytes_recovered":1606884004,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85712,"ondisk_log_size":85712,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":447,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":7032136,"kb_used_data":4474632,"kb_used_omap":12,"kb_used_meta":2556659,"kb_avail":742929080,"statfs":{"total":767960285184,"available":760759377920,"internally_reserved":0,"allocated":4582023168,"data_stored":4578763993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2618019378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-674430976,"num_objects":-10291,"num_object_clones":0,"num_object_copies":-30873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-149,"num_objects_unfound":0,"num_objects_dirty":-10291,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10291,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":9306112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.081616"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":381,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.590888+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:48:24.590888+0000","last_peered":"2024-09-21T11:48:24.590888+0000","last_clean":"2024-09-21T11:48:24.590888+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:48:24.590888+0000","last_undegraded":"2024-09-21T11:48:24.590888+0000","last_fullsized":"2024-09-21T11:48:24.590888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.863735+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":435,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532855+0000","last_change":"2024-09-21T11:47:03.433310+0000","last_active":"2024-09-21T11:48:24.532855+0000","last_peered":"2024-09-21T11:48:24.532855+0000","last_clean":"2024-09-21T11:48:24.532855+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:48:24.532855+0000","last_undegraded":"2024-09-21T11:48:24.532855+0000","last_fullsized":"2024-09-21T11:48:24.532855+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:44.553561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.553561+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.027789+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":74,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533703+0000","last_change":"2024-09-21T11:47:03.372676+0000","last_active":"2024-09-21T11:48:24.533703+0000","last_peered":"2024-09-21T11:48:24.533703+0000","last_clean":"2024-09-21T11:48:24.533703+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:48:24.533703+0000","last_undegraded":"2024-09-21T11:48:24.533703+0000","last_fullsized":"2024-09-21T11:48:24.533703+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":391,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533122+0000","last_change":"2024-09-21T11:47:20.957722+0000","last_active":"2024-09-21T11:48:24.533122+0000","last_peered":"2024-09-21T11:48:24.533122+0000","last_clean":"2024-09-21T11:48:24.533122+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:48:24.533122+0000","last_undegraded":"2024-09-21T11:48:24.533122+0000","last_fullsized":"2024-09-21T11:48:24.533122+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:20.957679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.957679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:25.355154+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":369,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531630+0000","last_change":"2024-09-21T11:46:09.675727+0000","last_active":"2024-09-21T11:48:24.531630+0000","last_peered":"2024-09-21T11:48:24.531630+0000","last_clean":"2024-09-21T11:48:24.531630+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:48:24.531630+0000","last_undegraded":"2024-09-21T11:48:24.531630+0000","last_fullsized":"2024-09-21T11:48:24.531630+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:09.675148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:09.675148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":374,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532153+0000","last_change":"2024-09-21T11:46:55.804484+0000","last_active":"2024-09-21T11:48:24.532153+0000","last_peered":"2024-09-21T11:48:24.532153+0000","last_clean":"2024-09-21T11:48:24.532153+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:48:24.532153+0000","last_undegraded":"2024-09-21T11:48:24.532153+0000","last_fullsized":"2024-09-21T11:48:24.532153+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.804419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.804419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":394,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531732+0000","last_change":"2024-09-21T11:45:07.788590+0000","last_active":"2024-09-21T11:48:24.531732+0000","last_peered":"2024-09-21T11:48:24.531732+0000","last_clean":"2024-09-21T11:48:24.531732+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:48:24.531732+0000","last_undegraded":"2024-09-21T11:48:24.531732+0000","last_fullsized":"2024-09-21T11:48:24.531732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:07.788512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.788512+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":395,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532833+0000","last_change":"2024-09-21T11:47:03.371710+0000","last_active":"2024-09-21T11:48:24.532833+0000","last_peered":"2024-09-21T11:48:24.532833+0000","last_clean":"2024-09-21T11:48:24.532833+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:48:24.532833+0000","last_undegraded":"2024-09-21T11:48:24.532833+0000","last_fullsized":"2024-09-21T11:48:24.532833+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":372,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531094+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:48:24.531094+0000","last_peered":"2024-09-21T11:48:24.531094+0000","last_clean":"2024-09-21T11:48:24.531094+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:48:24.531094+0000","last_undegraded":"2024-09-21T11:48:24.531094+0000","last_fullsized":"2024-09-21T11:48:24.531094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:43.925152+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":401,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533320+0000","last_change":"2024-09-21T11:48:13.962997+0000","last_active":"2024-09-21T11:48:24.533320+0000","last_peered":"2024-09-21T11:48:24.533320+0000","last_clean":"2024-09-21T11:48:24.533320+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:48:24.533320+0000","last_undegraded":"2024-09-21T11:48:24.533320+0000","last_fullsized":"2024-09-21T11:48:24.533320+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:13.962919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:13.962919+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:14.418938+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":380,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530629+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:48:24.530629+0000","last_peered":"2024-09-21T11:48:24.530629+0000","last_clean":"2024-09-21T11:48:24.530629+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:48:24.530629+0000","last_undegraded":"2024-09-21T11:48:24.530629+0000","last_fullsized":"2024-09-21T11:48:24.530629+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":382,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.588607+0000","last_change":"2024-09-21T11:47:03.385157+0000","last_active":"2024-09-21T11:48:24.588607+0000","last_peered":"2024-09-21T11:48:24.588607+0000","last_clean":"2024-09-21T11:48:24.588607+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:48:24.588607+0000","last_undegraded":"2024-09-21T11:48:24.588607+0000","last_fullsized":"2024-09-21T11:48:24.588607+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:06.385125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:06.385125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":402,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531230+0000","last_change":"2024-09-21T11:47:03.385245+0000","last_active":"2024-09-21T11:48:24.531230+0000","last_peered":"2024-09-21T11:48:24.531230+0000","last_clean":"2024-09-21T11:48:24.531230+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:48:24.531230+0000","last_undegraded":"2024-09-21T11:48:24.531230+0000","last_fullsized":"2024-09-21T11:48:24.531230+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.017967+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":421,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533207+0000","last_change":"2024-09-21T11:48:13.005365+0000","last_active":"2024-09-21T11:48:24.533207+0000","last_peered":"2024-09-21T11:48:24.533207+0000","last_clean":"2024-09-21T11:48:24.533207+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:48:24.533207+0000","last_undegraded":"2024-09-21T11:48:24.533207+0000","last_fullsized":"2024-09-21T11:48:24.533207+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:48:13.005294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:13.005294+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:13.822018+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":386,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.597713+0000","last_change":"2024-09-21T11:48:06.605705+0000","last_active":"2024-09-21T11:48:24.597713+0000","last_peered":"2024-09-21T11:48:24.597713+0000","last_clean":"2024-09-21T11:48:24.597713+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:48:24.597713+0000","last_undegraded":"2024-09-21T11:48:24.597713+0000","last_fullsized":"2024-09-21T11:48:24.597713+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:06.605645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.605645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.086689+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":390,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530427+0000","last_change":"2024-09-21T11:47:03.380856+0000","last_active":"2024-09-21T11:48:24.530427+0000","last_peered":"2024-09-21T11:48:24.530427+0000","last_clean":"2024-09-21T11:48:24.530427+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:48:24.530427+0000","last_undegraded":"2024-09-21T11:48:24.530427+0000","last_fullsized":"2024-09-21T11:48:24.530427+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:13.288522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:13.288522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":386,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533614+0000","last_change":"2024-09-21T11:46:52.035524+0000","last_active":"2024-09-21T11:48:24.533614+0000","last_peered":"2024-09-21T11:48:24.533614+0000","last_clean":"2024-09-21T11:48:24.533614+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:48:24.533614+0000","last_undegraded":"2024-09-21T11:48:24.533614+0000","last_fullsized":"2024-09-21T11:48:24.533614+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.035461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.035461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":372,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.589867+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:48:24.589867+0000","last_peered":"2024-09-21T11:48:24.589867+0000","last_clean":"2024-09-21T11:48:24.589867+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:48:24.589867+0000","last_undegraded":"2024-09-21T11:48:24.589867+0000","last_fullsized":"2024-09-21T11:48:24.589867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":487,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531794+0000","last_change":"2024-09-21T11:48:18.120641+0000","last_active":"2024-09-21T11:48:24.531794+0000","last_peered":"2024-09-21T11:48:24.531794+0000","last_clean":"2024-09-21T11:48:24.531794+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:48:24.531794+0000","last_undegraded":"2024-09-21T11:48:24.531794+0000","last_fullsized":"2024-09-21T11:48:24.531794+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:48:18.120532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:18.120532+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:22.674543+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":74,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.534723+0000","last_change":"2024-09-21T11:47:03.377667+0000","last_active":"2024-09-21T11:48:24.534723+0000","last_peered":"2024-09-21T11:48:24.534723+0000","last_clean":"2024-09-21T11:48:24.534723+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:48:24.534723+0000","last_undegraded":"2024-09-21T11:48:24.534723+0000","last_fullsized":"2024-09-21T11:48:24.534723+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.718478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.718478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":370,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532364+0000","last_change":"2024-09-21T11:45:39.826902+0000","last_active":"2024-09-21T11:48:24.532364+0000","last_peered":"2024-09-21T11:48:24.532364+0000","last_clean":"2024-09-21T11:48:24.532364+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:48:24.532364+0000","last_undegraded":"2024-09-21T11:48:24.532364+0000","last_fullsized":"2024-09-21T11:48:24.532364+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:39.826701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:39.826701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":395,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533448+0000","last_change":"2024-09-21T11:48:22.941031+0000","last_active":"2024-09-21T11:48:24.533448+0000","last_peered":"2024-09-21T11:48:24.533448+0000","last_clean":"2024-09-21T11:48:24.533448+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:48:24.533448+0000","last_undegraded":"2024-09-21T11:48:24.533448+0000","last_fullsized":"2024-09-21T11:48:24.533448+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:22.940968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:22.940968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:48.005429+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":438,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532632+0000","last_change":"2024-09-21T11:47:03.873307+0000","last_active":"2024-09-21T11:48:24.532632+0000","last_peered":"2024-09-21T11:48:24.532632+0000","last_clean":"2024-09-21T11:48:24.532632+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:48:24.532632+0000","last_undegraded":"2024-09-21T11:48:24.532632+0000","last_fullsized":"2024-09-21T11:48:24.532632+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:47:03.873239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.873239+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":373,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533322+0000","last_change":"2024-09-21T11:45:37.670176+0000","last_active":"2024-09-21T11:48:24.533322+0000","last_peered":"2024-09-21T11:48:24.533322+0000","last_clean":"2024-09-21T11:48:24.533322+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:48:24.533322+0000","last_undegraded":"2024-09-21T11:48:24.533322+0000","last_fullsized":"2024-09-21T11:48:24.533322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:37.670117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:37.670117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:50.969760+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":385,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.598217+0000","last_change":"2024-09-21T11:46:47.611118+0000","last_active":"2024-09-21T11:48:24.598217+0000","last_peered":"2024-09-21T11:48:24.598217+0000","last_clean":"2024-09-21T11:48:24.598217+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:48:24.598217+0000","last_undegraded":"2024-09-21T11:48:24.598217+0000","last_fullsized":"2024-09-21T11:48:24.598217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:46:47.611042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:47.611042+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.533011+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":78,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533991+0000","last_change":"2024-09-21T11:47:03.388975+0000","last_active":"2024-09-21T11:48:24.533991+0000","last_peered":"2024-09-21T11:48:24.533991+0000","last_clean":"2024-09-21T11:48:24.533991+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:48:24.533991+0000","last_undegraded":"2024-09-21T11:48:24.533991+0000","last_fullsized":"2024-09-21T11:48:24.533991+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:12.636105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.636105+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":400,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.590333+0000","last_change":"2024-09-21T11:45:48.385970+0000","last_active":"2024-09-21T11:48:24.590333+0000","last_peered":"2024-09-21T11:48:24.590333+0000","last_clean":"2024-09-21T11:48:24.590333+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:48:24.590333+0000","last_undegraded":"2024-09-21T11:48:24.590333+0000","last_fullsized":"2024-09-21T11:48:24.590333+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:45:48.385895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:48.385895+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":511,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:25.469032+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:48:25.469032+0000","last_peered":"2024-09-21T11:48:25.469032+0000","last_clean":"2024-09-21T11:48:25.469032+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:48:25.469032+0000","last_undegraded":"2024-09-21T11:48:25.469032+0000","last_fullsized":"2024-09-21T11:48:25.469032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:38.235643+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"234'15867","reported_seq":21613,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:30.904196+0000","last_change":"2024-09-21T11:48:17.532789+0000","last_active":"2024-09-21T11:48:30.904196+0000","last_peered":"2024-09-21T11:48:30.904196+0000","last_clean":"2024-09-21T11:48:30.904196+0000","last_became_active":"2024-09-21T11:48:17.482971+0000","last_became_peered":"2024-09-21T11:48:17.482971+0000","last_unstale":"2024-09-21T11:48:30.904196+0000","last_undegraded":"2024-09-21T11:48:30.904196+0000","last_fullsized":"2024-09-21T11:48:30.904196+0000","mapping_epoch":230,"log_start":"205'11245","ondisk_log_start":"205'11245","created":160,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4622,"log_dups_size":0,"ondisk_log_size":4622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86441984,"num_objects":1319,"num_object_clones":0,"num_object_copies":3957,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15867,"num_write_kb":549952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2785,"num_bytes_recovered":182517760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":90,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.534618+0000","last_change":"2024-09-21T11:47:03.400465+0000","last_active":"2024-09-21T11:48:24.534618+0000","last_peered":"2024-09-21T11:48:24.534618+0000","last_clean":"2024-09-21T11:48:24.534618+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:48:24.534618+0000","last_undegraded":"2024-09-21T11:48:24.534618+0000","last_fullsized":"2024-09-21T11:48:24.534618+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:22.708839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.708839+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":403,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531308+0000","last_change":"2024-09-21T11:47:03.391541+0000","last_active":"2024-09-21T11:48:24.531308+0000","last_peered":"2024-09-21T11:48:24.531308+0000","last_clean":"2024-09-21T11:48:24.531308+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:48:24.531308+0000","last_undegraded":"2024-09-21T11:48:24.531308+0000","last_fullsized":"2024-09-21T11:48:24.531308+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.836903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.836903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":484,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:25.468766+0000","last_change":"2024-09-21T11:47:01.933697+0000","last_active":"2024-09-21T11:48:25.468766+0000","last_peered":"2024-09-21T11:48:25.468766+0000","last_clean":"2024-09-21T11:48:25.468766+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:48:25.468766+0000","last_undegraded":"2024-09-21T11:48:25.468766+0000","last_fullsized":"2024-09-21T11:48:25.468766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:01.933648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:01.933648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":382,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530765+0000","last_change":"2024-09-21T11:48:15.013107+0000","last_active":"2024-09-21T11:48:24.530765+0000","last_peered":"2024-09-21T11:48:24.530765+0000","last_clean":"2024-09-21T11:48:24.530765+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:48:24.530765+0000","last_undegraded":"2024-09-21T11:48:24.530765+0000","last_fullsized":"2024-09-21T11:48:24.530765+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:15.013058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:15.013058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.125286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"234'15900","reported_seq":16143,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:31.620712+0000","last_change":"2024-09-21T11:46:19.360961+0000","last_active":"2024-09-21T11:48:31.620712+0000","last_peered":"2024-09-21T11:48:31.620712+0000","last_clean":"2024-09-21T11:48:31.620712+0000","last_became_active":"2024-09-21T11:44:51.744224+0000","last_became_peered":"2024-09-21T11:44:51.744224+0000","last_unstale":"2024-09-21T11:48:31.620712+0000","last_undegraded":"2024-09-21T11:48:31.620712+0000","last_fullsized":"2024-09-21T11:48:31.620712+0000","mapping_epoch":160,"log_start":"198'10100","ondisk_log_start":"198'10100","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7561","last_scrub_stamp":"2024-09-21T11:46:19.360902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.360902+0000","objects_scrubbed":7459,"log_size":5800,"log_dups_size":0,"ondisk_log_size":5800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82444288,"num_objects":1258,"num_object_clones":0,"num_object_copies":3774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15900,"num_write_kb":549056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":418,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531022+0000","last_change":"2024-09-21T11:46:58.789719+0000","last_active":"2024-09-21T11:48:24.531022+0000","last_peered":"2024-09-21T11:48:24.531022+0000","last_clean":"2024-09-21T11:48:24.531022+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:48:24.531022+0000","last_undegraded":"2024-09-21T11:48:24.531022+0000","last_fullsized":"2024-09-21T11:48:24.531022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:46:58.789625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.789625+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":389,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533037+0000","last_change":"2024-09-21T11:46:52.891978+0000","last_active":"2024-09-21T11:48:24.533037+0000","last_peered":"2024-09-21T11:48:24.533037+0000","last_clean":"2024-09-21T11:48:24.533037+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:48:24.533037+0000","last_undegraded":"2024-09-21T11:48:24.533037+0000","last_fullsized":"2024-09-21T11:48:24.533037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.891793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.891793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":374,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532951+0000","last_change":"2024-09-21T11:44:13.946705+0000","last_active":"2024-09-21T11:48:24.532951+0000","last_peered":"2024-09-21T11:48:24.532951+0000","last_clean":"2024-09-21T11:48:24.532951+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:48:24.532951+0000","last_undegraded":"2024-09-21T11:48:24.532951+0000","last_fullsized":"2024-09-21T11:48:24.532951+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:42.015785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":370,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533327+0000","last_change":"2024-09-21T11:46:24.717355+0000","last_active":"2024-09-21T11:48:24.533327+0000","last_peered":"2024-09-21T11:48:24.533327+0000","last_clean":"2024-09-21T11:48:24.533327+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:48:24.533327+0000","last_undegraded":"2024-09-21T11:48:24.533327+0000","last_fullsized":"2024-09-21T11:48:24.533327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.717285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.717285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:58.045440+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"234'15749","reported_seq":15947,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:30.968395+0000","last_change":"2024-09-21T11:48:05.746078+0000","last_active":"2024-09-21T11:48:30.968395+0000","last_peered":"2024-09-21T11:48:30.968395+0000","last_clean":"2024-09-21T11:48:30.968395+0000","last_became_active":"2024-09-21T11:44:51.744633+0000","last_became_peered":"2024-09-21T11:44:51.744633+0000","last_unstale":"2024-09-21T11:48:30.968395+0000","last_undegraded":"2024-09-21T11:48:30.968395+0000","last_fullsized":"2024-09-21T11:48:30.968395+0000","mapping_epoch":160,"log_start":"198'10200","ondisk_log_start":"198'10200","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'14372","last_scrub_stamp":"2024-09-21T11:48:05.746017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.746017+0000","objects_scrubbed":2702,"log_size":5549,"log_dups_size":0,"ondisk_log_size":5549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:10.815328+0000","scrub_duration":670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85590040,"num_objects":1307,"num_object_clones":0,"num_object_copies":3921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1307,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15749,"num_write_kb":545729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":415,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533101+0000","last_change":"2024-09-21T11:47:33.018713+0000","last_active":"2024-09-21T11:48:24.533101+0000","last_peered":"2024-09-21T11:48:24.533101+0000","last_clean":"2024-09-21T11:48:24.533101+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:48:24.533101+0000","last_undegraded":"2024-09-21T11:48:24.533101+0000","last_fullsized":"2024-09-21T11:48:24.533101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:47:33.018639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:33.018639+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:41.906194+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":411,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.590597+0000","last_change":"2024-09-21T11:47:03.429013+0000","last_active":"2024-09-21T11:48:24.590597+0000","last_peered":"2024-09-21T11:48:24.590597+0000","last_clean":"2024-09-21T11:48:24.590597+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:48:24.590597+0000","last_undegraded":"2024-09-21T11:48:24.590597+0000","last_fullsized":"2024-09-21T11:48:24.590597+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:59.702813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.702813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":508,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:25.473866+0000","last_change":"2024-09-21T11:47:03.406087+0000","last_active":"2024-09-21T11:48:25.473866+0000","last_peered":"2024-09-21T11:48:25.473866+0000","last_clean":"2024-09-21T11:48:25.473866+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:48:25.473866+0000","last_undegraded":"2024-09-21T11:48:25.473866+0000","last_fullsized":"2024-09-21T11:48:25.473866+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:08.620519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.620519+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":372,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.597746+0000","last_change":"2024-09-21T11:48:10.667288+0000","last_active":"2024-09-21T11:48:24.597746+0000","last_peered":"2024-09-21T11:48:24.597746+0000","last_clean":"2024-09-21T11:48:24.597746+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:48:24.597746+0000","last_undegraded":"2024-09-21T11:48:24.597746+0000","last_fullsized":"2024-09-21T11:48:24.597746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.667221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.667221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:11.606721+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"234'16086","reported_seq":16333,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:31.653777+0000","last_change":"2024-09-21T11:46:22.226583+0000","last_active":"2024-09-21T11:48:31.653777+0000","last_peered":"2024-09-21T11:48:31.653777+0000","last_clean":"2024-09-21T11:48:31.653777+0000","last_became_active":"2024-09-21T11:44:51.412662+0000","last_became_peered":"2024-09-21T11:44:51.412662+0000","last_unstale":"2024-09-21T11:48:31.653777+0000","last_undegraded":"2024-09-21T11:48:31.653777+0000","last_fullsized":"2024-09-21T11:48:31.653777+0000","mapping_epoch":160,"log_start":"198'10300","ondisk_log_start":"198'10300","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7855","last_scrub_stamp":"2024-09-21T11:46:22.226400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.226400+0000","objects_scrubbed":7719,"log_size":5786,"log_dups_size":0,"ondisk_log_size":5786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87556096,"num_objects":1336,"num_object_clones":0,"num_object_copies":4008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16086,"num_write_kb":557504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":86,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533658+0000","last_change":"2024-09-21T11:47:03.398638+0000","last_active":"2024-09-21T11:48:24.533658+0000","last_peered":"2024-09-21T11:48:24.533658+0000","last_clean":"2024-09-21T11:48:24.533658+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:48:24.533658+0000","last_undegraded":"2024-09-21T11:48:24.533658+0000","last_fullsized":"2024-09-21T11:48:24.533658+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:12.227121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.227121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":397,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531157+0000","last_change":"2024-09-21T11:48:07.131578+0000","last_active":"2024-09-21T11:48:24.531157+0000","last_peered":"2024-09-21T11:48:24.531157+0000","last_clean":"2024-09-21T11:48:24.531157+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:48:24.531157+0000","last_undegraded":"2024-09-21T11:48:24.531157+0000","last_fullsized":"2024-09-21T11:48:24.531157+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.131535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.131535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:08.103888+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":479,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:26.672841+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:48:26.672841+0000","last_peered":"2024-09-21T11:48:26.672841+0000","last_clean":"2024-09-21T11:48:26.672841+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:48:26.672841+0000","last_undegraded":"2024-09-21T11:48:26.672841+0000","last_fullsized":"2024-09-21T11:48:26.672841+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.263748+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":369,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.588705+0000","last_change":"2024-09-21T11:46:28.010676+0000","last_active":"2024-09-21T11:48:24.588705+0000","last_peered":"2024-09-21T11:48:24.588705+0000","last_clean":"2024-09-21T11:48:24.588705+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:48:24.588705+0000","last_undegraded":"2024-09-21T11:48:24.588705+0000","last_fullsized":"2024-09-21T11:48:24.588705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:28.010602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.010602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"234'15811","reported_seq":24952,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:31.744378+0000","last_change":"2024-09-21T11:47:54.309078+0000","last_active":"2024-09-21T11:48:31.744378+0000","last_peered":"2024-09-21T11:48:31.744378+0000","last_clean":"2024-09-21T11:48:31.744378+0000","last_became_active":"2024-09-21T11:47:54.270690+0000","last_became_peered":"2024-09-21T11:47:54.270690+0000","last_unstale":"2024-09-21T11:48:31.744378+0000","last_undegraded":"2024-09-21T11:48:31.744378+0000","last_fullsized":"2024-09-21T11:48:31.744378+0000","mapping_epoch":222,"log_start":"200'10800","ondisk_log_start":"200'10800","created":160,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'7360","last_scrub_stamp":"2024-09-21T11:46:17.038356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.038356+0000","objects_scrubbed":7242,"log_size":5011,"log_dups_size":0,"ondisk_log_size":5011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81068032,"num_objects":1237,"num_object_clones":0,"num_object_copies":3711,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15811,"num_write_kb":545536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4441,"num_bytes_recovered":291045376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":413,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.589561+0000","last_change":"2024-09-21T11:47:03.750064+0000","last_active":"2024-09-21T11:48:24.589561+0000","last_peered":"2024-09-21T11:48:24.589561+0000","last_clean":"2024-09-21T11:48:24.589561+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:48:24.589561+0000","last_undegraded":"2024-09-21T11:48:24.589561+0000","last_fullsized":"2024-09-21T11:48:24.589561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:47:03.749962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.749962+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":74,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.535039+0000","last_change":"2024-09-21T11:47:03.382675+0000","last_active":"2024-09-21T11:48:24.535039+0000","last_peered":"2024-09-21T11:48:24.535039+0000","last_clean":"2024-09-21T11:48:24.535039+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:48:24.535039+0000","last_undegraded":"2024-09-21T11:48:24.535039+0000","last_fullsized":"2024-09-21T11:48:24.535039+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:08.723593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.723593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":373,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.584307+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:48:24.584307+0000","last_peered":"2024-09-21T11:48:24.584307+0000","last_clean":"2024-09-21T11:48:24.584307+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:48:24.584307+0000","last_undegraded":"2024-09-21T11:48:24.584307+0000","last_fullsized":"2024-09-21T11:48:24.584307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.032409+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":74,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533599+0000","last_change":"2024-09-21T11:47:03.379157+0000","last_active":"2024-09-21T11:48:24.533599+0000","last_peered":"2024-09-21T11:48:24.533599+0000","last_clean":"2024-09-21T11:48:24.533599+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:48:24.533599+0000","last_undegraded":"2024-09-21T11:48:24.533599+0000","last_fullsized":"2024-09-21T11:48:24.533599+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"234'15913","reported_seq":16074,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:30.968935+0000","last_change":"2024-09-21T11:44:51.745506+0000","last_active":"2024-09-21T11:48:30.968935+0000","last_peered":"2024-09-21T11:48:30.968935+0000","last_clean":"2024-09-21T11:48:30.968935+0000","last_became_active":"2024-09-21T11:44:51.745200+0000","last_became_peered":"2024-09-21T11:44:51.745200+0000","last_unstale":"2024-09-21T11:48:30.968935+0000","last_undegraded":"2024-09-21T11:48:30.968935+0000","last_fullsized":"2024-09-21T11:48:30.968935+0000","mapping_epoch":160,"log_start":"198'10400","ondisk_log_start":"198'10400","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5513,"log_dups_size":0,"ondisk_log_size":5513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88014848,"num_objects":1343,"num_object_clones":0,"num_object_copies":4029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15913,"num_write_kb":552192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"205'31","reported_seq":114,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:25.475206+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:48:25.475206+0000","last_peered":"2024-09-21T11:48:25.475206+0000","last_clean":"2024-09-21T11:48:25.475206+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:48:25.475206+0000","last_undegraded":"2024-09-21T11:48:25.475206+0000","last_fullsized":"2024-09-21T11:48:25.475206+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":380,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533465+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:48:24.533465+0000","last_peered":"2024-09-21T11:48:24.533465+0000","last_clean":"2024-09-21T11:48:24.533465+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:48:24.533465+0000","last_undegraded":"2024-09-21T11:48:24.533465+0000","last_fullsized":"2024-09-21T11:48:24.533465+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:39.997756+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":413,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530744+0000","last_change":"2024-09-21T11:47:20.000195+0000","last_active":"2024-09-21T11:48:24.530744+0000","last_peered":"2024-09-21T11:48:24.530744+0000","last_clean":"2024-09-21T11:48:24.530744+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:48:24.530744+0000","last_undegraded":"2024-09-21T11:48:24.530744+0000","last_fullsized":"2024-09-21T11:48:24.530744+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:47:20.000051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.000051+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:48.196772+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":387,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.590037+0000","last_change":"2024-09-21T11:47:03.385173+0000","last_active":"2024-09-21T11:48:24.590037+0000","last_peered":"2024-09-21T11:48:24.590037+0000","last_clean":"2024-09-21T11:48:24.590037+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:48:24.590037+0000","last_undegraded":"2024-09-21T11:48:24.590037+0000","last_fullsized":"2024-09-21T11:48:24.590037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":361,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531935+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:48:24.531935+0000","last_peered":"2024-09-21T11:48:24.531935+0000","last_clean":"2024-09-21T11:48:24.531935+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:48:24.531935+0000","last_undegraded":"2024-09-21T11:48:24.531935+0000","last_fullsized":"2024-09-21T11:48:24.531935+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"234'15712","reported_seq":20220,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:31.731486+0000","last_change":"2024-09-21T11:48:24.579575+0000","last_active":"2024-09-21T11:48:31.731486+0000","last_peered":"2024-09-21T11:48:31.731486+0000","last_clean":"2024-09-21T11:48:31.731486+0000","last_became_active":"2024-09-21T11:48:24.535114+0000","last_became_peered":"2024-09-21T11:48:24.535114+0000","last_unstale":"2024-09-21T11:48:31.731486+0000","last_undegraded":"2024-09-21T11:48:31.731486+0000","last_fullsized":"2024-09-21T11:48:31.731486+0000","mapping_epoch":233,"log_start":"200'10700","ondisk_log_start":"200'10700","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5012,"log_dups_size":0,"ondisk_log_size":5012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82968576,"num_objects":1266,"num_object_clones":0,"num_object_copies":3798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15712,"num_write_kb":543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":141754368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":86,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.534722+0000","last_change":"2024-09-21T11:47:03.399466+0000","last_active":"2024-09-21T11:48:24.534722+0000","last_peered":"2024-09-21T11:48:24.534722+0000","last_clean":"2024-09-21T11:48:24.534722+0000","last_became_active":"2024-09-21T11:47:03.377155+0000","last_became_peered":"2024-09-21T11:47:03.377155+0000","last_unstale":"2024-09-21T11:48:24.534722+0000","last_undegraded":"2024-09-21T11:48:24.534722+0000","last_fullsized":"2024-09-21T11:48:24.534722+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.573503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":78,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.534732+0000","last_change":"2024-09-21T11:47:03.391283+0000","last_active":"2024-09-21T11:48:24.534732+0000","last_peered":"2024-09-21T11:48:24.534732+0000","last_clean":"2024-09-21T11:48:24.534732+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:48:24.534732+0000","last_undegraded":"2024-09-21T11:48:24.534732+0000","last_fullsized":"2024-09-21T11:48:24.534732+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":399,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530832+0000","last_change":"2024-09-21T11:46:54.864288+0000","last_active":"2024-09-21T11:48:24.530832+0000","last_peered":"2024-09-21T11:48:24.530832+0000","last_clean":"2024-09-21T11:48:24.530832+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:48:24.530832+0000","last_undegraded":"2024-09-21T11:48:24.530832+0000","last_fullsized":"2024-09-21T11:48:24.530832+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:54.864074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.864074+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":381,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533600+0000","last_change":"2024-09-21T11:48:16.076049+0000","last_active":"2024-09-21T11:48:24.533600+0000","last_peered":"2024-09-21T11:48:24.533600+0000","last_clean":"2024-09-21T11:48:24.533600+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:48:24.533600+0000","last_undegraded":"2024-09-21T11:48:24.533600+0000","last_fullsized":"2024-09-21T11:48:24.533600+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:16.076003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:16.076003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.071160+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":377,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.599593+0000","last_change":"2024-09-21T11:48:07.613756+0000","last_active":"2024-09-21T11:48:24.599593+0000","last_peered":"2024-09-21T11:48:24.599593+0000","last_clean":"2024-09-21T11:48:24.599593+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:48:24.599593+0000","last_undegraded":"2024-09-21T11:48:24.599593+0000","last_fullsized":"2024-09-21T11:48:24.599593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.613691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.613691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:32.231718+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"234'15590","reported_seq":25031,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:30.906064+0000","last_change":"2024-09-21T11:47:52.322057+0000","last_active":"2024-09-21T11:48:30.906064+0000","last_peered":"2024-09-21T11:48:30.906064+0000","last_clean":"2024-09-21T11:48:30.906064+0000","last_became_active":"2024-09-21T11:47:52.264063+0000","last_became_peered":"2024-09-21T11:47:52.264063+0000","last_unstale":"2024-09-21T11:48:30.906064+0000","last_undegraded":"2024-09-21T11:48:30.906064+0000","last_fullsized":"2024-09-21T11:48:30.906064+0000","mapping_epoch":220,"log_start":"201'10906","ondisk_log_start":"201'10906","created":160,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4684,"log_dups_size":0,"ondisk_log_size":4684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84672512,"num_objects":1292,"num_object_clones":0,"num_object_copies":3876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15590,"num_write_kb":540224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4633,"num_bytes_recovered":303628288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":436,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.590499+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:48:24.590499+0000","last_peered":"2024-09-21T11:48:24.590499+0000","last_clean":"2024-09-21T11:48:24.590499+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:48:24.590499+0000","last_undegraded":"2024-09-21T11:48:24.590499+0000","last_fullsized":"2024-09-21T11:48:24.590499+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":389,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.589833+0000","last_change":"2024-09-21T11:46:29.937866+0000","last_active":"2024-09-21T11:48:24.589833+0000","last_peered":"2024-09-21T11:48:24.589833+0000","last_clean":"2024-09-21T11:48:24.589833+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:48:24.589833+0000","last_undegraded":"2024-09-21T11:48:24.589833+0000","last_fullsized":"2024-09-21T11:48:24.589833+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.937799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.937799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":934,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:29.476217+0000","last_change":"2024-09-21T11:48:17.596682+0000","last_active":"2024-09-21T11:48:29.476217+0000","last_peered":"2024-09-21T11:48:29.476217+0000","last_clean":"2024-09-21T11:48:29.476217+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:48:29.476217+0000","last_undegraded":"2024-09-21T11:48:29.476217+0000","last_fullsized":"2024-09-21T11:48:29.476217+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:48:17.596571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.596571+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.625607+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":502,"num_read_kb":497,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":386,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531227+0000","last_change":"2024-09-21T11:47:22.025919+0000","last_active":"2024-09-21T11:48:24.531227+0000","last_peered":"2024-09-21T11:48:24.531227+0000","last_clean":"2024-09-21T11:48:24.531227+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:48:24.531227+0000","last_undegraded":"2024-09-21T11:48:24.531227+0000","last_fullsized":"2024-09-21T11:48:24.531227+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:22.025842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.025842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:29.017093+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":74,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.534948+0000","last_change":"2024-09-21T11:47:03.374507+0000","last_active":"2024-09-21T11:48:24.534948+0000","last_peered":"2024-09-21T11:48:24.534948+0000","last_clean":"2024-09-21T11:48:24.534948+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:48:24.534948+0000","last_undegraded":"2024-09-21T11:48:24.534948+0000","last_fullsized":"2024-09-21T11:48:24.534948+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"234'15706","reported_seq":16000,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:30.951555+0000","last_change":"2024-09-21T11:47:31.508923+0000","last_active":"2024-09-21T11:48:30.951555+0000","last_peered":"2024-09-21T11:48:30.951555+0000","last_clean":"2024-09-21T11:48:30.951555+0000","last_became_active":"2024-09-21T11:44:51.410701+0000","last_became_peered":"2024-09-21T11:44:51.410701+0000","last_unstale":"2024-09-21T11:48:30.951555+0000","last_undegraded":"2024-09-21T11:48:30.951555+0000","last_fullsized":"2024-09-21T11:48:30.951555+0000","mapping_epoch":160,"log_start":"197'9300","ondisk_log_start":"197'9300","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'12576","last_scrub_stamp":"2024-09-21T11:47:31.508822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:47:31.508822+0000","objects_scrubbed":4421,"log_size":6406,"log_dups_size":0,"ondisk_log_size":6406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.146813+0000","scrub_duration":1214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82837504,"num_objects":1264,"num_object_clones":0,"num_object_copies":3792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15706,"num_write_kb":543040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":502,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:25.959065+0000","last_change":"2024-09-21T11:48:06.171363+0000","last_active":"2024-09-21T11:48:25.959065+0000","last_peered":"2024-09-21T11:48:25.959065+0000","last_clean":"2024-09-21T11:48:25.959065+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:48:25.959065+0000","last_undegraded":"2024-09-21T11:48:25.959065+0000","last_fullsized":"2024-09-21T11:48:25.959065+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:48:06.171279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.171279+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:12.520316+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":390,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531432+0000","last_change":"2024-09-21T11:46:57.833589+0000","last_active":"2024-09-21T11:48:24.531432+0000","last_peered":"2024-09-21T11:48:24.531432+0000","last_clean":"2024-09-21T11:48:24.531432+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:48:24.531432+0000","last_undegraded":"2024-09-21T11:48:24.531432+0000","last_fullsized":"2024-09-21T11:48:24.531432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:57.833329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.833329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":443,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.597951+0000","last_change":"2024-09-21T11:46:58.645189+0000","last_active":"2024-09-21T11:48:24.597951+0000","last_peered":"2024-09-21T11:48:24.597951+0000","last_clean":"2024-09-21T11:48:24.597951+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:48:24.597951+0000","last_undegraded":"2024-09-21T11:48:24.597951+0000","last_fullsized":"2024-09-21T11:48:24.597951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:46:58.645122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.645122+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.573944+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":381,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.598357+0000","last_change":"2024-09-21T11:47:25.395237+0000","last_active":"2024-09-21T11:48:24.598357+0000","last_peered":"2024-09-21T11:48:24.598357+0000","last_clean":"2024-09-21T11:48:24.598357+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:48:24.598357+0000","last_undegraded":"2024-09-21T11:48:24.598357+0000","last_fullsized":"2024-09-21T11:48:24.598357+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.395185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.395185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.204032+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":376,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531156+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:48:24.531156+0000","last_peered":"2024-09-21T11:48:24.531156+0000","last_clean":"2024-09-21T11:48:24.531156+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:48:24.531156+0000","last_undegraded":"2024-09-21T11:48:24.531156+0000","last_fullsized":"2024-09-21T11:48:24.531156+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"234'15695","reported_seq":15970,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:30.859670+0000","last_change":"2024-09-21T11:48:09.830854+0000","last_active":"2024-09-21T11:48:30.859670+0000","last_peered":"2024-09-21T11:48:30.859670+0000","last_clean":"2024-09-21T11:48:30.859670+0000","last_became_active":"2024-09-21T11:44:51.410197+0000","last_became_peered":"2024-09-21T11:44:51.410197+0000","last_unstale":"2024-09-21T11:48:30.859670+0000","last_undegraded":"2024-09-21T11:48:30.859670+0000","last_fullsized":"2024-09-21T11:48:30.859670+0000","mapping_epoch":160,"log_start":"201'10900","ondisk_log_start":"201'10900","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"226'14544","last_scrub_stamp":"2024-09-21T11:48:09.830774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.830774+0000","objects_scrubbed":2470,"log_size":4795,"log_dups_size":0,"ondisk_log_size":4795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.225752+0000","scrub_duration":808,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85393408,"num_objects":1303,"num_object_clones":0,"num_object_copies":3909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15695,"num_write_kb":543936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":434,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533682+0000","last_change":"2024-09-21T11:48:05.072998+0000","last_active":"2024-09-21T11:48:24.533682+0000","last_peered":"2024-09-21T11:48:24.533682+0000","last_clean":"2024-09-21T11:48:24.533682+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:48:24.533682+0000","last_undegraded":"2024-09-21T11:48:24.533682+0000","last_fullsized":"2024-09-21T11:48:24.533682+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:05.072910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.072910+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:33.824160+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":397,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533164+0000","last_change":"2024-09-21T11:47:00.950268+0000","last_active":"2024-09-21T11:48:24.533164+0000","last_peered":"2024-09-21T11:48:24.533164+0000","last_clean":"2024-09-21T11:48:24.533164+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:48:24.533164+0000","last_undegraded":"2024-09-21T11:48:24.533164+0000","last_fullsized":"2024-09-21T11:48:24.533164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:00.950208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:00.950208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":380,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531016+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:48:24.531016+0000","last_peered":"2024-09-21T11:48:24.531016+0000","last_clean":"2024-09-21T11:48:24.531016+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:48:24.531016+0000","last_undegraded":"2024-09-21T11:48:24.531016+0000","last_fullsized":"2024-09-21T11:48:24.531016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.795578+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":376,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531127+0000","last_change":"2024-09-21T11:48:23.285482+0000","last_active":"2024-09-21T11:48:24.531127+0000","last_peered":"2024-09-21T11:48:24.531127+0000","last_clean":"2024-09-21T11:48:24.531127+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:48:24.531127+0000","last_undegraded":"2024-09-21T11:48:24.531127+0000","last_fullsized":"2024-09-21T11:48:24.531127+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.285283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.285283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:52.473277+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"234'16070","reported_seq":22485,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:30.897510+0000","last_change":"2024-09-21T11:48:15.522333+0000","last_active":"2024-09-21T11:48:30.897510+0000","last_peered":"2024-09-21T11:48:30.897510+0000","last_clean":"2024-09-21T11:48:30.897510+0000","last_became_active":"2024-09-21T11:48:15.471002+0000","last_became_peered":"2024-09-21T11:48:15.471002+0000","last_unstale":"2024-09-21T11:48:30.897510+0000","last_undegraded":"2024-09-21T11:48:30.897510+0000","last_fullsized":"2024-09-21T11:48:30.897510+0000","mapping_epoch":228,"log_start":"205'11413","ondisk_log_start":"205'11413","created":160,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'10449","last_scrub_stamp":"2024-09-21T11:46:52.316998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.316998+0000","objects_scrubbed":7011,"log_size":4657,"log_dups_size":0,"ondisk_log_size":4657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84279296,"num_objects":1286,"num_object_clones":0,"num_object_copies":3858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16070,"num_write_kb":555392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3078,"num_bytes_recovered":201719808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":463,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533754+0000","last_change":"2024-09-21T11:47:22.974837+0000","last_active":"2024-09-21T11:48:24.533754+0000","last_peered":"2024-09-21T11:48:24.533754+0000","last_clean":"2024-09-21T11:48:24.533754+0000","last_became_active":"2024-09-21T11:47:03.369099+0000","last_became_peered":"2024-09-21T11:47:03.369099+0000","last_unstale":"2024-09-21T11:48:24.533754+0000","last_undegraded":"2024-09-21T11:48:24.533754+0000","last_fullsized":"2024-09-21T11:48:24.533754+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:47:22.974761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.974761+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:31.827386+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":410,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.584441+0000","last_change":"2024-09-21T11:47:03.365071+0000","last_active":"2024-09-21T11:48:24.584441+0000","last_peered":"2024-09-21T11:48:24.584441+0000","last_clean":"2024-09-21T11:48:24.584441+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:48:24.584441+0000","last_undegraded":"2024-09-21T11:48:24.584441+0000","last_fullsized":"2024-09-21T11:48:24.584441+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.692429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.692429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:54.181079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":74,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.534849+0000","last_change":"2024-09-21T11:47:03.375108+0000","last_active":"2024-09-21T11:48:24.534849+0000","last_peered":"2024-09-21T11:48:24.534849+0000","last_clean":"2024-09-21T11:48:24.534849+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:48:24.534849+0000","last_undegraded":"2024-09-21T11:48:24.534849+0000","last_fullsized":"2024-09-21T11:48:24.534849+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:50.008635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:50.008635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":385,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532936+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:48:24.532936+0000","last_peered":"2024-09-21T11:48:24.532936+0000","last_clean":"2024-09-21T11:48:24.532936+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:48:24.532936+0000","last_undegraded":"2024-09-21T11:48:24.532936+0000","last_fullsized":"2024-09-21T11:48:24.532936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"234'15715","reported_seq":24833,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:31.727175+0000","last_change":"2024-09-21T11:47:53.304799+0000","last_active":"2024-09-21T11:48:31.727175+0000","last_peered":"2024-09-21T11:48:31.727175+0000","last_clean":"2024-09-21T11:48:31.727175+0000","last_became_active":"2024-09-21T11:47:53.264856+0000","last_became_peered":"2024-09-21T11:47:53.264856+0000","last_unstale":"2024-09-21T11:48:31.727175+0000","last_undegraded":"2024-09-21T11:48:31.727175+0000","last_fullsized":"2024-09-21T11:48:31.727175+0000","mapping_epoch":221,"log_start":"200'10700","ondisk_log_start":"200'10700","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5015,"log_dups_size":0,"ondisk_log_size":5015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80019456,"num_objects":1221,"num_object_clones":0,"num_object_copies":3663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15715,"num_write_kb":541952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4468,"num_bytes_recovered":292749312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":408,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533356+0000","last_change":"2024-09-21T11:46:11.730053+0000","last_active":"2024-09-21T11:48:24.533356+0000","last_peered":"2024-09-21T11:48:24.533356+0000","last_clean":"2024-09-21T11:48:24.533356+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:48:24.533356+0000","last_undegraded":"2024-09-21T11:48:24.533356+0000","last_fullsized":"2024-09-21T11:48:24.533356+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:11.729984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.729984+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.980319+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":373,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532285+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:48:24.532285+0000","last_peered":"2024-09-21T11:48:24.532285+0000","last_clean":"2024-09-21T11:48:24.532285+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:48:24.532285+0000","last_undegraded":"2024-09-21T11:48:24.532285+0000","last_fullsized":"2024-09-21T11:48:24.532285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":492,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:25.471840+0000","last_change":"2024-09-21T11:46:52.834287+0000","last_active":"2024-09-21T11:48:25.471840+0000","last_peered":"2024-09-21T11:48:25.471840+0000","last_clean":"2024-09-21T11:48:25.471840+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:48:25.471840+0000","last_undegraded":"2024-09-21T11:48:25.471840+0000","last_fullsized":"2024-09-21T11:48:25.471840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:52.834206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.834206+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":383,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533472+0000","last_change":"2024-09-21T11:48:07.992586+0000","last_active":"2024-09-21T11:48:24.533472+0000","last_peered":"2024-09-21T11:48:24.533472+0000","last_clean":"2024-09-21T11:48:24.533472+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:48:24.533472+0000","last_undegraded":"2024-09-21T11:48:24.533472+0000","last_fullsized":"2024-09-21T11:48:24.533472+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.992517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.992517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.884349+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"234'15823","reported_seq":21967,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:30.909331+0000","last_change":"2024-09-21T11:48:16.538801+0000","last_active":"2024-09-21T11:48:30.909331+0000","last_peered":"2024-09-21T11:48:30.909331+0000","last_clean":"2024-09-21T11:48:30.909331+0000","last_became_active":"2024-09-21T11:48:16.476754+0000","last_became_peered":"2024-09-21T11:48:16.476754+0000","last_unstale":"2024-09-21T11:48:30.909331+0000","last_undegraded":"2024-09-21T11:48:30.909331+0000","last_fullsized":"2024-09-21T11:48:30.909331+0000","mapping_epoch":229,"log_start":"205'11187","ondisk_log_start":"205'11187","created":160,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7408","last_scrub_stamp":"2024-09-21T11:46:17.934135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.934135+0000","objects_scrubbed":7290,"log_size":4636,"log_dups_size":0,"ondisk_log_size":4636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89980928,"num_objects":1373,"num_object_clones":0,"num_object_copies":4119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15823,"num_write_kb":550272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2943,"num_bytes_recovered":192872448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":426,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.589456+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:48:24.589456+0000","last_peered":"2024-09-21T11:48:24.589456+0000","last_clean":"2024-09-21T11:48:24.589456+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:48:24.589456+0000","last_undegraded":"2024-09-21T11:48:24.589456+0000","last_fullsized":"2024-09-21T11:48:24.589456+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:41.378708+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":388,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531309+0000","last_change":"2024-09-21T11:48:08.172717+0000","last_active":"2024-09-21T11:48:24.531309+0000","last_peered":"2024-09-21T11:48:24.531309+0000","last_clean":"2024-09-21T11:48:24.531309+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:48:24.531309+0000","last_undegraded":"2024-09-21T11:48:24.531309+0000","last_fullsized":"2024-09-21T11:48:24.531309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:08.172656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:08.172656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:18.830907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":373,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530844+0000","last_change":"2024-09-21T11:46:55.879988+0000","last_active":"2024-09-21T11:48:24.530844+0000","last_peered":"2024-09-21T11:48:24.530844+0000","last_clean":"2024-09-21T11:48:24.530844+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:48:24.530844+0000","last_undegraded":"2024-09-21T11:48:24.530844+0000","last_fullsized":"2024-09-21T11:48:24.530844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.879761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.879761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":360,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.589732+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:48:24.589732+0000","last_peered":"2024-09-21T11:48:24.589732+0000","last_clean":"2024-09-21T11:48:24.589732+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:48:24.589732+0000","last_undegraded":"2024-09-21T11:48:24.589732+0000","last_fullsized":"2024-09-21T11:48:24.589732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"234'16005","reported_seq":16247,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:31.659453+0000","last_change":"2024-09-21T11:46:15.665868+0000","last_active":"2024-09-21T11:48:31.659453+0000","last_peered":"2024-09-21T11:48:31.659453+0000","last_clean":"2024-09-21T11:48:31.659453+0000","last_became_active":"2024-09-21T11:44:51.744226+0000","last_became_peered":"2024-09-21T11:44:51.744226+0000","last_unstale":"2024-09-21T11:48:31.659453+0000","last_undegraded":"2024-09-21T11:48:31.659453+0000","last_fullsized":"2024-09-21T11:48:31.659453+0000","mapping_epoch":160,"log_start":"198'10200","ondisk_log_start":"198'10200","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7351","last_scrub_stamp":"2024-09-21T11:46:15.665775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:15.665775+0000","objects_scrubbed":7260,"log_size":5805,"log_dups_size":0,"ondisk_log_size":5805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83689472,"num_objects":1277,"num_object_clones":0,"num_object_copies":3831,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16005,"num_write_kb":553024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":469,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.584357+0000","last_change":"2024-09-21T11:47:03.397050+0000","last_active":"2024-09-21T11:48:24.584357+0000","last_peered":"2024-09-21T11:48:24.584357+0000","last_clean":"2024-09-21T11:48:24.584357+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:48:24.584357+0000","last_undegraded":"2024-09-21T11:48:24.584357+0000","last_fullsized":"2024-09-21T11:48:24.584357+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:46:20.650403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.650403+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:51.167956+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":388,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.599682+0000","last_change":"2024-09-21T11:48:03.546909+0000","last_active":"2024-09-21T11:48:24.599682+0000","last_peered":"2024-09-21T11:48:24.599682+0000","last_clean":"2024-09-21T11:48:24.599682+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:48:24.599682+0000","last_undegraded":"2024-09-21T11:48:24.599682+0000","last_fullsized":"2024-09-21T11:48:24.599682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:03.546843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:03.546843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.594742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":389,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530961+0000","last_change":"2024-09-21T11:46:16.775013+0000","last_active":"2024-09-21T11:48:24.530961+0000","last_peered":"2024-09-21T11:48:24.530961+0000","last_clean":"2024-09-21T11:48:24.530961+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:48:24.530961+0000","last_undegraded":"2024-09-21T11:48:24.530961+0000","last_fullsized":"2024-09-21T11:48:24.530961+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:16.774939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.774939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":376,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.597858+0000","last_change":"2024-09-21T11:46:59.704981+0000","last_active":"2024-09-21T11:48:24.597858+0000","last_peered":"2024-09-21T11:48:24.597858+0000","last_clean":"2024-09-21T11:48:24.597858+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:48:24.597858+0000","last_undegraded":"2024-09-21T11:48:24.597858+0000","last_fullsized":"2024-09-21T11:48:24.597858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"234'15905","reported_seq":16156,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:30.958279+0000","last_change":"2024-09-21T11:46:27.656566+0000","last_active":"2024-09-21T11:48:30.958279+0000","last_peered":"2024-09-21T11:48:30.958279+0000","last_clean":"2024-09-21T11:48:30.958279+0000","last_became_active":"2024-09-21T11:44:51.745609+0000","last_became_peered":"2024-09-21T11:44:51.745609+0000","last_unstale":"2024-09-21T11:48:30.958279+0000","last_undegraded":"2024-09-21T11:48:30.958279+0000","last_fullsized":"2024-09-21T11:48:30.958279+0000","mapping_epoch":160,"log_start":"197'9500","ondisk_log_start":"197'9500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'8199","last_scrub_stamp":"2024-09-21T11:46:27.656360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.656360+0000","objects_scrubbed":8107,"log_size":6405,"log_dups_size":0,"ondisk_log_size":6405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:46.172763+0000","scrub_duration":2017,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88145920,"num_objects":1345,"num_object_clones":0,"num_object_copies":4035,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15905,"num_write_kb":552000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":462,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532993+0000","last_change":"2024-09-21T11:48:04.111199+0000","last_active":"2024-09-21T11:48:24.532993+0000","last_peered":"2024-09-21T11:48:24.532993+0000","last_clean":"2024-09-21T11:48:24.532993+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:48:24.532993+0000","last_undegraded":"2024-09-21T11:48:24.532993+0000","last_fullsized":"2024-09-21T11:48:24.532993+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:48:04.111115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.111115+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:19.507697+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":418,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.598781+0000","last_change":"2024-09-21T11:47:26.411190+0000","last_active":"2024-09-21T11:48:24.598781+0000","last_peered":"2024-09-21T11:48:24.598781+0000","last_clean":"2024-09-21T11:48:24.598781+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:48:24.598781+0000","last_undegraded":"2024-09-21T11:48:24.598781+0000","last_fullsized":"2024-09-21T11:48:24.598781+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:26.411146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:26.411146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:48.131154+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":403,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530546+0000","last_change":"2024-09-21T11:47:03.385382+0000","last_active":"2024-09-21T11:48:24.530546+0000","last_peered":"2024-09-21T11:48:24.530546+0000","last_clean":"2024-09-21T11:48:24.530546+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:48:24.530546+0000","last_undegraded":"2024-09-21T11:48:24.530546+0000","last_fullsized":"2024-09-21T11:48:24.530546+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.766819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.766819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":383,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531932+0000","last_change":"2024-09-21T11:47:03.376001+0000","last_active":"2024-09-21T11:48:24.531932+0000","last_peered":"2024-09-21T11:48:24.531932+0000","last_clean":"2024-09-21T11:48:24.531932+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:48:24.531932+0000","last_undegraded":"2024-09-21T11:48:24.531932+0000","last_fullsized":"2024-09-21T11:48:24.531932+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.703712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.703712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"234'15857","reported_seq":16097,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:30.977067+0000","last_change":"2024-09-21T11:46:12.974904+0000","last_active":"2024-09-21T11:48:30.977067+0000","last_peered":"2024-09-21T11:48:30.977067+0000","last_clean":"2024-09-21T11:48:30.977067+0000","last_became_active":"2024-09-21T11:44:51.409710+0000","last_became_peered":"2024-09-21T11:44:51.409710+0000","last_unstale":"2024-09-21T11:48:30.977067+0000","last_undegraded":"2024-09-21T11:48:30.977067+0000","last_fullsized":"2024-09-21T11:48:30.977067+0000","mapping_epoch":160,"log_start":"198'10400","ondisk_log_start":"198'10400","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7106","last_scrub_stamp":"2024-09-21T11:46:12.974687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.974687+0000","objects_scrubbed":7036,"log_size":5457,"log_dups_size":0,"ondisk_log_size":5457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1797,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88670208,"num_objects":1353,"num_object_clones":0,"num_object_copies":4059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15857,"num_write_kb":550720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":393,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530403+0000","last_change":"2024-09-21T11:46:57.248649+0000","last_active":"2024-09-21T11:48:24.530403+0000","last_peered":"2024-09-21T11:48:24.530403+0000","last_clean":"2024-09-21T11:48:24.530403+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:48:24.530403+0000","last_undegraded":"2024-09-21T11:48:24.530403+0000","last_fullsized":"2024-09-21T11:48:24.530403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:57.248575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.248575+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":408,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.598544+0000","last_change":"2024-09-21T11:48:09.683093+0000","last_active":"2024-09-21T11:48:24.598544+0000","last_peered":"2024-09-21T11:48:24.598544+0000","last_clean":"2024-09-21T11:48:24.598544+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:48:24.598544+0000","last_undegraded":"2024-09-21T11:48:24.598544+0000","last_fullsized":"2024-09-21T11:48:24.598544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:48:09.683008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.683008+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:38.676024+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":491,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:25.468909+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:48:25.468909+0000","last_peered":"2024-09-21T11:48:25.468909+0000","last_clean":"2024-09-21T11:48:25.468909+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:48:25.468909+0000","last_undegraded":"2024-09-21T11:48:25.468909+0000","last_fullsized":"2024-09-21T11:48:25.468909+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.600494+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":368,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530522+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:48:24.530522+0000","last_peered":"2024-09-21T11:48:24.530522+0000","last_clean":"2024-09-21T11:48:24.530522+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:48:24.530522+0000","last_undegraded":"2024-09-21T11:48:24.530522+0000","last_fullsized":"2024-09-21T11:48:24.530522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.006344+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":396,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.584194+0000","last_change":"2024-09-21T11:48:17.129730+0000","last_active":"2024-09-21T11:48:24.584194+0000","last_peered":"2024-09-21T11:48:24.584194+0000","last_clean":"2024-09-21T11:48:24.584194+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:48:24.584194+0000","last_undegraded":"2024-09-21T11:48:24.584194+0000","last_fullsized":"2024-09-21T11:48:24.584194+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.129675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.129675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.682622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":416,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532894+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:48:24.532894+0000","last_peered":"2024-09-21T11:48:24.532894+0000","last_clean":"2024-09-21T11:48:24.532894+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:48:24.532894+0000","last_undegraded":"2024-09-21T11:48:24.532894+0000","last_fullsized":"2024-09-21T11:48:24.532894+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":394,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531013+0000","last_change":"2024-09-21T11:47:25.905705+0000","last_active":"2024-09-21T11:48:24.531013+0000","last_peered":"2024-09-21T11:48:24.531013+0000","last_clean":"2024-09-21T11:48:24.531013+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:48:24.531013+0000","last_undegraded":"2024-09-21T11:48:24.531013+0000","last_fullsized":"2024-09-21T11:48:24.531013+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.905645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.905645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.034596+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":384,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.599522+0000","last_change":"2024-09-21T11:47:28.380378+0000","last_active":"2024-09-21T11:48:24.599522+0000","last_peered":"2024-09-21T11:48:24.599522+0000","last_clean":"2024-09-21T11:48:24.599522+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:48:24.599522+0000","last_undegraded":"2024-09-21T11:48:24.599522+0000","last_fullsized":"2024-09-21T11:48:24.599522+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:28.380245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:28.380245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:34.745499+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":409,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.590550+0000","last_change":"2024-09-21T11:46:28.973700+0000","last_active":"2024-09-21T11:48:24.590550+0000","last_peered":"2024-09-21T11:48:24.590550+0000","last_clean":"2024-09-21T11:48:24.590550+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:48:24.590550+0000","last_undegraded":"2024-09-21T11:48:24.590550+0000","last_fullsized":"2024-09-21T11:48:24.590550+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:28.973606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.973606+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":380,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.597951+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:48:24.597951+0000","last_peered":"2024-09-21T11:48:24.597951+0000","last_clean":"2024-09-21T11:48:24.597951+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:48:24.597951+0000","last_undegraded":"2024-09-21T11:48:24.597951+0000","last_fullsized":"2024-09-21T11:48:24.597951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:47.600096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":395,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532417+0000","last_change":"2024-09-21T11:47:03.379216+0000","last_active":"2024-09-21T11:48:24.532417+0000","last_peered":"2024-09-21T11:48:24.532417+0000","last_clean":"2024-09-21T11:48:24.532417+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:48:24.532417+0000","last_undegraded":"2024-09-21T11:48:24.532417+0000","last_fullsized":"2024-09-21T11:48:24.532417+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:48.903980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:48.903980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":385,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531401+0000","last_change":"2024-09-21T11:48:17.081179+0000","last_active":"2024-09-21T11:48:24.531401+0000","last_peered":"2024-09-21T11:48:24.531401+0000","last_clean":"2024-09-21T11:48:24.531401+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:48:24.531401+0000","last_undegraded":"2024-09-21T11:48:24.531401+0000","last_fullsized":"2024-09-21T11:48:24.531401+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.081131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.081131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.264468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":392,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531052+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:48:24.531052+0000","last_peered":"2024-09-21T11:48:24.531052+0000","last_clean":"2024-09-21T11:48:24.531052+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:48:24.531052+0000","last_undegraded":"2024-09-21T11:48:24.531052+0000","last_fullsized":"2024-09-21T11:48:24.531052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":388,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.589177+0000","last_change":"2024-09-21T11:46:24.087039+0000","last_active":"2024-09-21T11:48:24.589177+0000","last_peered":"2024-09-21T11:48:24.589177+0000","last_clean":"2024-09-21T11:48:24.589177+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:48:24.589177+0000","last_undegraded":"2024-09-21T11:48:24.589177+0000","last_fullsized":"2024-09-21T11:48:24.589177+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.086918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.086918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":476,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:25.474110+0000","last_change":"2024-09-21T11:47:02.849839+0000","last_active":"2024-09-21T11:48:25.474110+0000","last_peered":"2024-09-21T11:48:25.474110+0000","last_clean":"2024-09-21T11:48:25.474110+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:48:25.474110+0000","last_undegraded":"2024-09-21T11:48:25.474110+0000","last_fullsized":"2024-09-21T11:48:25.474110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:02.849786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:02.849786+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":377,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533627+0000","last_change":"2024-09-21T11:45:38.652432+0000","last_active":"2024-09-21T11:48:24.533627+0000","last_peered":"2024-09-21T11:48:24.533627+0000","last_clean":"2024-09-21T11:48:24.533627+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:48:24.533627+0000","last_undegraded":"2024-09-21T11:48:24.533627+0000","last_fullsized":"2024-09-21T11:48:24.533627+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:38.652380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:38.652380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:47.014191+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":422,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532591+0000","last_change":"2024-09-21T11:47:03.428846+0000","last_active":"2024-09-21T11:48:24.532591+0000","last_peered":"2024-09-21T11:48:24.532591+0000","last_clean":"2024-09-21T11:48:24.532591+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:48:24.532591+0000","last_undegraded":"2024-09-21T11:48:24.532591+0000","last_fullsized":"2024-09-21T11:48:24.532591+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":380,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.598138+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:48:24.598138+0000","last_peered":"2024-09-21T11:48:24.598138+0000","last_clean":"2024-09-21T11:48:24.598138+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:48:24.598138+0000","last_undegraded":"2024-09-21T11:48:24.598138+0000","last_fullsized":"2024-09-21T11:48:24.598138+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.604026+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":514,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:25.468842+0000","last_change":"2024-09-21T11:47:03.415793+0000","last_active":"2024-09-21T11:48:25.468842+0000","last_peered":"2024-09-21T11:48:25.468842+0000","last_clean":"2024-09-21T11:48:25.468842+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:48:25.468842+0000","last_undegraded":"2024-09-21T11:48:25.468842+0000","last_fullsized":"2024-09-21T11:48:25.468842+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:10.174971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:10.174971+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.999554+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":376,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.590018+0000","last_change":"2024-09-21T11:45:44.314664+0000","last_active":"2024-09-21T11:48:24.590018+0000","last_peered":"2024-09-21T11:48:24.590018+0000","last_clean":"2024-09-21T11:48:24.590018+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:48:24.590018+0000","last_undegraded":"2024-09-21T11:48:24.590018+0000","last_fullsized":"2024-09-21T11:48:24.590018+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:44.314461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.314461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":450,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533278+0000","last_change":"2024-09-21T11:47:03.402357+0000","last_active":"2024-09-21T11:48:24.533278+0000","last_peered":"2024-09-21T11:48:24.533278+0000","last_clean":"2024-09-21T11:48:24.533278+0000","last_became_active":"2024-09-21T11:47:03.368296+0000","last_became_peered":"2024-09-21T11:47:03.368296+0000","last_unstale":"2024-09-21T11:48:24.533278+0000","last_undegraded":"2024-09-21T11:48:24.533278+0000","last_fullsized":"2024-09-21T11:48:24.533278+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:54.032795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.032795+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":381,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533531+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:48:24.533531+0000","last_peered":"2024-09-21T11:48:24.533531+0000","last_clean":"2024-09-21T11:48:24.533531+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:48:24.533531+0000","last_undegraded":"2024-09-21T11:48:24.533531+0000","last_fullsized":"2024-09-21T11:48:24.533531+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:37.991785+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":74,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533630+0000","last_change":"2024-09-21T11:47:03.379049+0000","last_active":"2024-09-21T11:48:24.533630+0000","last_peered":"2024-09-21T11:48:24.533630+0000","last_clean":"2024-09-21T11:48:24.533630+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:48:24.533630+0000","last_undegraded":"2024-09-21T11:48:24.533630+0000","last_fullsized":"2024-09-21T11:48:24.533630+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.269610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.269610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":361,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531419+0000","last_change":"2024-09-21T11:45:04.770055+0000","last_active":"2024-09-21T11:48:24.531419+0000","last_peered":"2024-09-21T11:48:24.531419+0000","last_clean":"2024-09-21T11:48:24.531419+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:48:24.531419+0000","last_undegraded":"2024-09-21T11:48:24.531419+0000","last_fullsized":"2024-09-21T11:48:24.531419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:04.769963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:04.769963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":84,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.533803+0000","last_change":"2024-09-21T11:47:03.404473+0000","last_active":"2024-09-21T11:48:24.533803+0000","last_peered":"2024-09-21T11:48:24.533803+0000","last_clean":"2024-09-21T11:48:24.533803+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:48:24.533803+0000","last_undegraded":"2024-09-21T11:48:24.533803+0000","last_fullsized":"2024-09-21T11:48:24.533803+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:26.649189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:26.649189+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":388,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.599643+0000","last_change":"2024-09-21T11:48:04.553772+0000","last_active":"2024-09-21T11:48:24.599643+0000","last_peered":"2024-09-21T11:48:24.599643+0000","last_clean":"2024-09-21T11:48:24.599643+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:48:24.599643+0000","last_undegraded":"2024-09-21T11:48:24.599643+0000","last_fullsized":"2024-09-21T11:48:24.599643+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:04.553731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.553731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:21.248327+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":395,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532034+0000","last_change":"2024-09-21T11:47:03.377948+0000","last_active":"2024-09-21T11:48:24.532034+0000","last_peered":"2024-09-21T11:48:24.532034+0000","last_clean":"2024-09-21T11:48:24.532034+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:48:24.532034+0000","last_undegraded":"2024-09-21T11:48:24.532034+0000","last_fullsized":"2024-09-21T11:48:24.532034+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:53.848323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:53.848323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":399,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.587600+0000","last_change":"2024-09-21T11:47:03.408821+0000","last_active":"2024-09-21T11:48:24.587600+0000","last_peered":"2024-09-21T11:48:24.587600+0000","last_clean":"2024-09-21T11:48:24.587600+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:48:24.587600+0000","last_undegraded":"2024-09-21T11:48:24.587600+0000","last_fullsized":"2024-09-21T11:48:24.587600+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:45:43.314803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:43.314803+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":396,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531534+0000","last_change":"2024-09-21T11:45:46.706221+0000","last_active":"2024-09-21T11:48:24.531534+0000","last_peered":"2024-09-21T11:48:24.531534+0000","last_clean":"2024-09-21T11:48:24.531534+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:48:24.531534+0000","last_undegraded":"2024-09-21T11:48:24.531534+0000","last_fullsized":"2024-09-21T11:48:24.531534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:45:46.706130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.706130+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":394,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531069+0000","last_change":"2024-09-21T11:48:10.126645+0000","last_active":"2024-09-21T11:48:24.531069+0000","last_peered":"2024-09-21T11:48:24.531069+0000","last_clean":"2024-09-21T11:48:24.531069+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:48:24.531069+0000","last_undegraded":"2024-09-21T11:48:24.531069+0000","last_fullsized":"2024-09-21T11:48:24.531069+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.126591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.126591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:10.829490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":364,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.597494+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:48:24.597494+0000","last_peered":"2024-09-21T11:48:24.597494+0000","last_clean":"2024-09-21T11:48:24.597494+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:48:24.597494+0000","last_undegraded":"2024-09-21T11:48:24.597494+0000","last_fullsized":"2024-09-21T11:48:24.597494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:43.608235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":369,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.532180+0000","last_change":"2024-09-21T11:45:45.730374+0000","last_active":"2024-09-21T11:48:24.532180+0000","last_peered":"2024-09-21T11:48:24.532180+0000","last_clean":"2024-09-21T11:48:24.532180+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:48:24.532180+0000","last_undegraded":"2024-09-21T11:48:24.532180+0000","last_fullsized":"2024-09-21T11:48:24.532180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:45.730169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:45.730169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":430,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.530920+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:48:24.530920+0000","last_peered":"2024-09-21T11:48:24.530920+0000","last_clean":"2024-09-21T11:48:24.530920+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:48:24.530920+0000","last_undegraded":"2024-09-21T11:48:24.530920+0000","last_fullsized":"2024-09-21T11:48:24.530920+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":380,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.590994+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:48:24.590994+0000","last_peered":"2024-09-21T11:48:24.590994+0000","last_clean":"2024-09-21T11:48:24.590994+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:48:24.590994+0000","last_undegraded":"2024-09-21T11:48:24.590994+0000","last_fullsized":"2024-09-21T11:48:24.590994+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.212674+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":380,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.584205+0000","last_change":"2024-09-21T11:48:09.104826+0000","last_active":"2024-09-21T11:48:24.584205+0000","last_peered":"2024-09-21T11:48:24.584205+0000","last_clean":"2024-09-21T11:48:24.584205+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:48:24.584205+0000","last_undegraded":"2024-09-21T11:48:24.584205+0000","last_fullsized":"2024-09-21T11:48:24.584205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:09.104767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.104767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.005085+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":368,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.597628+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:48:24.597628+0000","last_peered":"2024-09-21T11:48:24.597628+0000","last_clean":"2024-09-21T11:48:24.597628+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:48:24.597628+0000","last_undegraded":"2024-09-21T11:48:24.597628+0000","last_fullsized":"2024-09-21T11:48:24.597628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:49.549791+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":74,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.534547+0000","last_change":"2024-09-21T11:47:03.374194+0000","last_active":"2024-09-21T11:48:24.534547+0000","last_peered":"2024-09-21T11:48:24.534547+0000","last_clean":"2024-09-21T11:48:24.534547+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:48:24.534547+0000","last_undegraded":"2024-09-21T11:48:24.534547+0000","last_fullsized":"2024-09-21T11:48:24.534547+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:27.297630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.297630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":395,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.531199+0000","last_change":"2024-09-21T11:47:03.377189+0000","last_active":"2024-09-21T11:48:24.531199+0000","last_peered":"2024-09-21T11:48:24.531199+0000","last_clean":"2024-09-21T11:48:24.531199+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:48:24.531199+0000","last_undegraded":"2024-09-21T11:48:24.531199+0000","last_fullsized":"2024-09-21T11:48:24.531199+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:11.587469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.587469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":450,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.585720+0000","last_change":"2024-09-21T11:47:03.392087+0000","last_active":"2024-09-21T11:48:24.585720+0000","last_peered":"2024-09-21T11:48:24.585720+0000","last_clean":"2024-09-21T11:48:24.585720+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:48:24.585720+0000","last_undegraded":"2024-09-21T11:48:24.585720+0000","last_fullsized":"2024-09-21T11:48:24.585720+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:16.675872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.675872+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:41.170726+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":381,"reported_epoch":234,"state":"active+clean","last_fresh":"2024-09-21T11:48:24.590234+0000","last_change":"2024-09-21T11:45:12.377954+0000","last_active":"2024-09-21T11:48:24.590234+0000","last_peered":"2024-09-21T11:48:24.590234+0000","last_clean":"2024-09-21T11:48:24.590234+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:48:24.590234+0000","last_undegraded":"2024-09-21T11:48:24.590234+0000","last_fullsized":"2024-09-21T11:48:24.590234+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:12.377880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:12.377880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":1361772568,"num_objects":20780,"num_object_clones":0,"num_object_copies":62340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253404,"num_write_kb":8773825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24511,"num_bytes_recovered":1606287360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4439748608,"data_stored":4439736392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85153,"ondisk_log_size":85153,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":4814,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":396,"ondisk_log_size":396,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":502,"num_read_kb":497,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234091,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":758024,"kb_used_data":433912,"kb_used_omap":1,"kb_used_meta":324094,"kb_avail":92987128,"statfs":{"total":95995035648,"available":95218819072,"internally_reserved":0,"allocated":444325888,"data_stored":443919706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":331872714},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463024,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":734648,"kb_used_data":442216,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":93010504,"statfs":{"total":95995035648,"available":95242756096,"internally_reserved":0,"allocated":452829184,"data_stored":452439706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":299170246},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724661,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":983304,"kb_used_data":703480,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":92761848,"statfs":{"total":95995035648,"available":94988132352,"internally_reserved":0,"allocated":720363520,"data_stored":719956439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":286390726},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920891,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":726348,"kb_used_data":452220,"kb_used_omap":1,"kb_used_meta":274110,"kb_avail":93018804,"statfs":{"total":95995035648,"available":95251255296,"internally_reserved":0,"allocated":463073280,"data_stored":462662947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280689093},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117121,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":860092,"kb_used_data":499884,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":92885060,"statfs":{"total":95995035648,"available":95114301440,"internally_reserved":0,"allocated":511881216,"data_stored":511486092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":368835015},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411463,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1263348,"kb_used_data":844836,"kb_used_omap":1,"kb_used_meta":418494,"kb_avail":92481804,"statfs":{"total":95995035648,"available":94701367296,"internally_reserved":0,"allocated":865112064,"data_stored":864725427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":428538309},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":895220,"kb_used_data":603044,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":92849932,"statfs":{"total":95995035648,"available":95078330368,"internally_reserved":0,"allocated":617517056,"data_stored":617066399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":299170246},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738516,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":811152,"kb_used_data":495040,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":92934000,"statfs":{"total":95995035648,"available":95164416000,"internally_reserved":0,"allocated":506920960,"data_stored":506507277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":323353029},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496173056,"data_stored":496173056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":600702976,"data_stored":600702976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":850464768,"data_stored":850460696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":496893952,"data_stored":496893952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":440926208,"data_stored":440926208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":683151360,"data_stored":683147288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":433979392,"data_stored":433979392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":437456896,"data_stored":437452824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:48:34.194 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:34.220 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:34.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:34.221 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:34.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:34.368 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:34.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:34.369 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:34.430 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:34 smithi043 ceph-mon[36454]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:34.430 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:34 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:34.430 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2567765167' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:48:34.430 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:34 smithi043 ceph-mon[36454]: osdmap e236: 8 total, 8 up, 8 in 2024-09-21T11:48:34.431 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:34 smithi043 ceph-mon[29710]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:34.431 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:34 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:34.431 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2567765167' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:48:34.431 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:34 smithi043 ceph-mon[29710]: osdmap e236: 8 total, 8 up, 8 in 2024-09-21T11:48:34.469 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:34.500 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:34.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:34.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:34.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:34 smithi175 ceph-mon[32613]: from='client.15543 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:34.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:34 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:34.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2567765167' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:48:34.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:34 smithi175 ceph-mon[32613]: osdmap e236: 8 total, 8 up, 8 in 2024-09-21T11:48:34.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:34.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:34.640 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:34.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:34.666 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:256997 8.f 8:fcc476ac:::benchmark_data_smithi043_97042_object119903:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236)", 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:34.416510+0000", 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011688181000000001, 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011720273999999999, 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 256997 2024-09-21T11:48:34.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.416510+0000", 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.416510+0000", 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.416511+0000", 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7000000000000003e-07 2024-09-21T11:48:34.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.416515+0000", 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0559999999999998e-06 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.416524+0000", 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.482e-06 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:34.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.416531+0000", 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4520000000000002e-06 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.416586+0000", 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4684e-05 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.416696+0000", 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110151 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.416803+0000", 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010691 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.423476+0000", 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0066729689999999999 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.424159+0000", 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00068294799999999995 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:34.671 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:256999 8.f 8:f8eaf684:::benchmark_data_smithi043_97042_object119905:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e236)", 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:34.418703+0000", 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0094953360000000001, 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0096143059999999995, 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 256999 2024-09-21T11:48:34.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.418703+0000", 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.418703+0000", 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.418704+0000", 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.398e-06 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.418709+0000", 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8930000000000004e-06 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.418727+0000", 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8097e-05 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.418736+0000", 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5709999999999998e-06 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.420035+0000", 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012992240000000001 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.420141+0000", 2024-09-21T11:48:34.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105504 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.420662+0000", 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052117699999999995 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.424192+0000", 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035298590000000002 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425267+0000", 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001075779 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:34.676 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:257002 8.7 e236/229 8:e9dabf68:::benchmark_data_smithi043_97042_object119908:head v 236'16088, mlcod=236'16086)", 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:34.425007+0000", 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.003190895, 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033816660000000002, 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425007+0000", 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425007+0000", 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425008+0000", 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7899999999999997e-07 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425014+0000", 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9490000000000001e-06 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425027+0000", 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2938999999999999e-05 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425048+0000", 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0645999999999999e-05 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425129+0000", 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1285999999999994e-05 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425342+0000", 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021342799999999999 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:257003 8.8 e236/160 8:1251598e:::benchmark_data_smithi043_97042_object119909:head v 236'16153, mlcod=236'16151)", 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:34.425809+0000", 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.002388767, 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026287519999999998, 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:48:34.680 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425809+0000", 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425809+0000", 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425810+0000", 2024-09-21T11:48:34.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2399999999999997e-07 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425816+0000", 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9320000000000001e-06 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425830+0000", 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3743999999999999e-05 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:48:34.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425838+0000", 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.746e-06 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425888+0000", 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0448999999999998e-05 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:34.425905+0000", 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7159e-05 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:48:34.683 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:48:34.684 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 4 2024-09-21T11:48:34.684 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:48:34.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:34.781 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:34.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:34.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:34.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:34.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:34.935 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:34.935 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:35.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:35.076 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:35.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:35.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:35.177 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:35.209 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:35.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:35.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:35.374 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:35.401 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:35.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:35.402 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:48:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:48:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:48:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:35 smithi043 ceph-mon[36454]: pgmap v408: 148 pgs: 148 active+clean; 1.3 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.11k op/s; 1.1 MiB/s, 17 objects/s recovering 2024-09-21T11:48:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:35 smithi043 ceph-mon[29710]: pgmap v408: 148 pgs: 148 active+clean; 1.3 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.11k op/s; 1.1 MiB/s, 17 objects/s recovering 2024-09-21T11:48:35.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:35.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:35 smithi175 ceph-mon[32613]: pgmap v408: 148 pgs: 148 active+clean; 1.3 GiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.11k op/s; 1.1 MiB/s, 17 objects/s recovering 2024-09-21T11:48:35.529 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:35.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:35.530 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:35.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:35.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:35.665 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:35.665 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:35.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:35.800 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:35.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:35.801 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:35.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:35.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:35.939 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:35.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:36.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:36.079 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:36.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:36.080 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:36.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:36.202 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:36.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:36.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:36.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:36.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:36.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:36.448 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:36.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:36.492 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:36.592 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:36.625 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:36.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:36.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:36.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:36.763 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:36.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:36.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:36.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:36.897 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:36.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:36.898 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:36.999 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:37.027 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:37.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:37.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:37.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:37.164 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:37.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:37.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:37.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:48:37.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:37.308 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:37.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:37.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:37.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:37.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:37 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:48:37.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:37 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:48:37.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:37 smithi043 ceph-mon[29710]: pgmap v409: 148 pgs: 148 active+clean; 1.2 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 746 op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:48:37.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:37 smithi043 ceph-mon[29710]: osdmap e237: 8 total, 8 up, 8 in 2024-09-21T11:48:37.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:37 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:48:37.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:37 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:48:37.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:37 smithi043 ceph-mon[36454]: pgmap v409: 148 pgs: 148 active+clean; 1.2 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 746 op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:48:37.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:37 smithi043 ceph-mon[36454]: osdmap e237: 8 total, 8 up, 8 in 2024-09-21T11:48:37.456 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:37.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:37.458 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:37 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:48:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:37 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:48:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:37 smithi175 ceph-mon[32613]: pgmap v409: 148 pgs: 148 active+clean; 1.2 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 746 op/s; 27 B/s, 1 objects/s recovering 2024-09-21T11:48:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:37 smithi175 ceph-mon[32613]: osdmap e237: 8 total, 8 up, 8 in 2024-09-21T11:48:37.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:37.585 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:37.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:37.586 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:37.687 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:37.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:37.716 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:37.716 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:37.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:37.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:37.850 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:37.850 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:37.951 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:37.976 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:37.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:37.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:38.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:38.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:38.110 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:38.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:38.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:38.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:38.257 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:38.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:38.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:38.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:48:38.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:38.408 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:38.408 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:38.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:38.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:38.557 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:38.558 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:38.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:38.717 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:38.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:38.725 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:38.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:48:38.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:38.828 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:38.884 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:38.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:38.885 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:38.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:39.041 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:39.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:39.042 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:39.143 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:39.168 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:39.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:39.169 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:39.269 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:39.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:39.307 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:39.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:39.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:39.445 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:39.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:39.446 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:39.547 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:39.583 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:39.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:39.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:39.610 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:39 smithi043 ceph-mon[36454]: pgmap v411: 148 pgs: 148 active+clean; 933 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 977 op/s; 36 B/s, 2 objects/s recovering 2024-09-21T11:48:39.610 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:39 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:48:39.610 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:39 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:48:39.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:39 smithi043 ceph-mon[29710]: pgmap v411: 148 pgs: 148 active+clean; 933 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 977 op/s; 36 B/s, 2 objects/s recovering 2024-09-21T11:48:39.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:39 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:48:39.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:39 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:48:39.684 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:39.714 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:39.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:39.715 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:39.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:39 smithi175 ceph-mon[32613]: pgmap v411: 148 pgs: 148 active+clean; 933 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 977 op/s; 36 B/s, 2 objects/s recovering 2024-09-21T11:48:39.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:39 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:48:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:39 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:48:39.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:39.852 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:39.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:39.853 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:39.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:40.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:40.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:40.035 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:40.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:40.151 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:48:40.152 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:40.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:40.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:40.157 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:48:40.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:40.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:48:40.282 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:40.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:40.283 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:40.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:40.443 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:40.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:40.444 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:40.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:40 smithi043 ceph-mon[36454]: osdmap e238: 8 total, 8 up, 8 in 2024-09-21T11:48:40.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:40 smithi043 ceph-mon[29710]: osdmap e238: 8 total, 8 up, 8 in 2024-09-21T11:48:40.544 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:40.589 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:40.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:40.590 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:40.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:40.730 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:40.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:40.731 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:40.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:40 smithi175 ceph-mon[32613]: osdmap e238: 8 total, 8 up, 8 in 2024-09-21T11:48:40.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:40.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:40.867 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:40.868 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:40.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:41.005 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:41.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:41.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:41.101 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-09-21T11:48:39.929319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":978131507,"num_objects":15116,"num_object_clones":0,"num_object_copies":45348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15116,"num_whiteouts":0,"num_read":1344,"num_read_kb":1201,"num_write":259816,"num_write_kb":8775498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24762,"num_bytes_recovered":1606884224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86374,"ondisk_log_size":86374,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":449,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":5874888,"kb_used_data":3334528,"kb_used_omap":12,"kb_used_meta":2540019,"kb_avail":744086328,"statfs":{"total":767960285184,"available":761944399872,"internally_reserved":0,"allocated":3414556672,"data_stored":3411678891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2600980018},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2,4,11,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-384237568,"num_objects":-5863,"num_object_clones":0,"num_object_copies":-17589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5863,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5863,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.921922"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":387,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.047070+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:48:36.047070+0000","last_peered":"2024-09-21T11:48:36.047070+0000","last_clean":"2024-09-21T11:48:36.047070+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:48:36.047070+0000","last_undegraded":"2024-09-21T11:48:36.047070+0000","last_fullsized":"2024-09-21T11:48:36.047070+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.863735+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":439,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.006739+0000","last_change":"2024-09-21T11:47:03.433310+0000","last_active":"2024-09-21T11:48:33.006739+0000","last_peered":"2024-09-21T11:48:33.006739+0000","last_clean":"2024-09-21T11:48:33.006739+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:48:33.006739+0000","last_undegraded":"2024-09-21T11:48:33.006739+0000","last_fullsized":"2024-09-21T11:48:33.006739+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:44.553561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.553561+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.027789+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":78,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.023775+0000","last_change":"2024-09-21T11:47:03.372676+0000","last_active":"2024-09-21T11:48:33.023775+0000","last_peered":"2024-09-21T11:48:33.023775+0000","last_clean":"2024-09-21T11:48:33.023775+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:48:33.023775+0000","last_undegraded":"2024-09-21T11:48:33.023775+0000","last_fullsized":"2024-09-21T11:48:33.023775+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.453795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":395,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.006984+0000","last_change":"2024-09-21T11:47:20.957722+0000","last_active":"2024-09-21T11:48:33.006984+0000","last_peered":"2024-09-21T11:48:33.006984+0000","last_clean":"2024-09-21T11:48:33.006984+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:48:33.006984+0000","last_undegraded":"2024-09-21T11:48:33.006984+0000","last_fullsized":"2024-09-21T11:48:33.006984+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:20.957679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.957679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":375,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.043584+0000","last_change":"2024-09-21T11:46:09.675727+0000","last_active":"2024-09-21T11:48:36.043584+0000","last_peered":"2024-09-21T11:48:36.043584+0000","last_clean":"2024-09-21T11:48:36.043584+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:48:36.043584+0000","last_undegraded":"2024-09-21T11:48:36.043584+0000","last_fullsized":"2024-09-21T11:48:36.043584+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:09.675148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:09.675148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":380,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.043568+0000","last_change":"2024-09-21T11:46:55.804484+0000","last_active":"2024-09-21T11:48:36.043568+0000","last_peered":"2024-09-21T11:48:36.043568+0000","last_clean":"2024-09-21T11:48:36.043568+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:48:36.043568+0000","last_undegraded":"2024-09-21T11:48:36.043568+0000","last_fullsized":"2024-09-21T11:48:36.043568+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.804419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.804419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":400,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.042036+0000","last_change":"2024-09-21T11:45:07.788590+0000","last_active":"2024-09-21T11:48:36.042036+0000","last_peered":"2024-09-21T11:48:36.042036+0000","last_clean":"2024-09-21T11:48:36.042036+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:48:36.042036+0000","last_undegraded":"2024-09-21T11:48:36.042036+0000","last_fullsized":"2024-09-21T11:48:36.042036+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:07.788512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.788512+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":399,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.006737+0000","last_change":"2024-09-21T11:47:03.371710+0000","last_active":"2024-09-21T11:48:33.006737+0000","last_peered":"2024-09-21T11:48:33.006737+0000","last_clean":"2024-09-21T11:48:33.006737+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:48:33.006737+0000","last_undegraded":"2024-09-21T11:48:33.006737+0000","last_fullsized":"2024-09-21T11:48:33.006737+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.957478+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":376,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.021177+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:48:33.021177+0000","last_peered":"2024-09-21T11:48:33.021177+0000","last_clean":"2024-09-21T11:48:33.021177+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:48:33.021177+0000","last_undegraded":"2024-09-21T11:48:33.021177+0000","last_fullsized":"2024-09-21T11:48:33.021177+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:43.925152+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":405,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007253+0000","last_change":"2024-09-21T11:48:13.962997+0000","last_active":"2024-09-21T11:48:33.007253+0000","last_peered":"2024-09-21T11:48:33.007253+0000","last_clean":"2024-09-21T11:48:33.007253+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:48:33.007253+0000","last_undegraded":"2024-09-21T11:48:33.007253+0000","last_fullsized":"2024-09-21T11:48:33.007253+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:13.962919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:13.962919+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:14.418938+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":384,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020249+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:48:33.020249+0000","last_peered":"2024-09-21T11:48:33.020249+0000","last_clean":"2024-09-21T11:48:33.020249+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:48:33.020249+0000","last_undegraded":"2024-09-21T11:48:33.020249+0000","last_fullsized":"2024-09-21T11:48:33.020249+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":388,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.044513+0000","last_change":"2024-09-21T11:47:03.385157+0000","last_active":"2024-09-21T11:48:36.044513+0000","last_peered":"2024-09-21T11:48:36.044513+0000","last_clean":"2024-09-21T11:48:36.044513+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:48:36.044513+0000","last_undegraded":"2024-09-21T11:48:36.044513+0000","last_fullsized":"2024-09-21T11:48:36.044513+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:06.385125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:06.385125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":406,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.021417+0000","last_change":"2024-09-21T11:47:03.385245+0000","last_active":"2024-09-21T11:48:33.021417+0000","last_peered":"2024-09-21T11:48:33.021417+0000","last_clean":"2024-09-21T11:48:33.021417+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:48:33.021417+0000","last_undegraded":"2024-09-21T11:48:33.021417+0000","last_fullsized":"2024-09-21T11:48:33.021417+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.017967+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":425,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007668+0000","last_change":"2024-09-21T11:48:13.005365+0000","last_active":"2024-09-21T11:48:33.007668+0000","last_peered":"2024-09-21T11:48:33.007668+0000","last_clean":"2024-09-21T11:48:33.007668+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:48:33.007668+0000","last_undegraded":"2024-09-21T11:48:33.007668+0000","last_fullsized":"2024-09-21T11:48:33.007668+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:48:13.005294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:13.005294+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:13.822018+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":390,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.027115+0000","last_change":"2024-09-21T11:48:06.605705+0000","last_active":"2024-09-21T11:48:33.027115+0000","last_peered":"2024-09-21T11:48:33.027115+0000","last_clean":"2024-09-21T11:48:33.027115+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:48:33.027115+0000","last_undegraded":"2024-09-21T11:48:33.027115+0000","last_fullsized":"2024-09-21T11:48:33.027115+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:06.605645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.605645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.086689+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":394,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020159+0000","last_change":"2024-09-21T11:47:03.380856+0000","last_active":"2024-09-21T11:48:33.020159+0000","last_peered":"2024-09-21T11:48:33.020159+0000","last_clean":"2024-09-21T11:48:33.020159+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:48:33.020159+0000","last_undegraded":"2024-09-21T11:48:33.020159+0000","last_fullsized":"2024-09-21T11:48:33.020159+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:13.288522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:13.288522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.041204+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":390,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007335+0000","last_change":"2024-09-21T11:46:52.035524+0000","last_active":"2024-09-21T11:48:33.007335+0000","last_peered":"2024-09-21T11:48:33.007335+0000","last_clean":"2024-09-21T11:48:33.007335+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:48:33.007335+0000","last_undegraded":"2024-09-21T11:48:33.007335+0000","last_fullsized":"2024-09-21T11:48:33.007335+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.035461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.035461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:30.990978+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":378,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.046767+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:48:36.046767+0000","last_peered":"2024-09-21T11:48:36.046767+0000","last_clean":"2024-09-21T11:48:36.046767+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:48:36.046767+0000","last_undegraded":"2024-09-21T11:48:36.046767+0000","last_fullsized":"2024-09-21T11:48:36.046767+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.281211+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":493,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.043669+0000","last_change":"2024-09-21T11:48:18.120641+0000","last_active":"2024-09-21T11:48:36.043669+0000","last_peered":"2024-09-21T11:48:36.043669+0000","last_clean":"2024-09-21T11:48:36.043669+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:48:36.043669+0000","last_undegraded":"2024-09-21T11:48:36.043669+0000","last_fullsized":"2024-09-21T11:48:36.043669+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:48:18.120532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:18.120532+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:22.674543+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":78,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.022942+0000","last_change":"2024-09-21T11:47:03.377667+0000","last_active":"2024-09-21T11:48:33.022942+0000","last_peered":"2024-09-21T11:48:33.022942+0000","last_clean":"2024-09-21T11:48:33.022942+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:48:33.022942+0000","last_undegraded":"2024-09-21T11:48:33.022942+0000","last_fullsized":"2024-09-21T11:48:33.022942+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.718478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.718478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":376,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.044412+0000","last_change":"2024-09-21T11:45:39.826902+0000","last_active":"2024-09-21T11:48:36.044412+0000","last_peered":"2024-09-21T11:48:36.044412+0000","last_clean":"2024-09-21T11:48:36.044412+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:48:36.044412+0000","last_undegraded":"2024-09-21T11:48:36.044412+0000","last_fullsized":"2024-09-21T11:48:36.044412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:39.826701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:39.826701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":399,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007409+0000","last_change":"2024-09-21T11:48:22.941031+0000","last_active":"2024-09-21T11:48:33.007409+0000","last_peered":"2024-09-21T11:48:33.007409+0000","last_clean":"2024-09-21T11:48:33.007409+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:48:33.007409+0000","last_undegraded":"2024-09-21T11:48:33.007409+0000","last_fullsized":"2024-09-21T11:48:33.007409+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:22.940968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:22.940968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:48.005429+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":444,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.042875+0000","last_change":"2024-09-21T11:47:03.873307+0000","last_active":"2024-09-21T11:48:36.042875+0000","last_peered":"2024-09-21T11:48:36.042875+0000","last_clean":"2024-09-21T11:48:36.042875+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:48:36.042875+0000","last_undegraded":"2024-09-21T11:48:36.042875+0000","last_fullsized":"2024-09-21T11:48:36.042875+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:47:03.873239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.873239+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":377,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007148+0000","last_change":"2024-09-21T11:45:37.670176+0000","last_active":"2024-09-21T11:48:33.007148+0000","last_peered":"2024-09-21T11:48:33.007148+0000","last_clean":"2024-09-21T11:48:33.007148+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:48:33.007148+0000","last_undegraded":"2024-09-21T11:48:33.007148+0000","last_fullsized":"2024-09-21T11:48:33.007148+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:37.670117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:37.670117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:50.969760+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":389,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.027252+0000","last_change":"2024-09-21T11:46:47.611118+0000","last_active":"2024-09-21T11:48:33.027252+0000","last_peered":"2024-09-21T11:48:33.027252+0000","last_clean":"2024-09-21T11:48:33.027252+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:48:33.027252+0000","last_undegraded":"2024-09-21T11:48:33.027252+0000","last_fullsized":"2024-09-21T11:48:33.027252+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:46:47.611042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:47.611042+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.533011+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":82,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.023558+0000","last_change":"2024-09-21T11:47:03.388975+0000","last_active":"2024-09-21T11:48:33.023558+0000","last_peered":"2024-09-21T11:48:33.023558+0000","last_clean":"2024-09-21T11:48:33.023558+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:48:33.023558+0000","last_undegraded":"2024-09-21T11:48:33.023558+0000","last_fullsized":"2024-09-21T11:48:33.023558+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:12.636105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.636105+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":406,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.048708+0000","last_change":"2024-09-21T11:45:48.385970+0000","last_active":"2024-09-21T11:48:36.048708+0000","last_peered":"2024-09-21T11:48:36.048708+0000","last_clean":"2024-09-21T11:48:36.048708+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:48:36.048708+0000","last_undegraded":"2024-09-21T11:48:36.048708+0000","last_fullsized":"2024-09-21T11:48:36.048708+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:45:48.385895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:48.385895+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":516,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.005671+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:48:33.005671+0000","last_peered":"2024-09-21T11:48:33.005671+0000","last_clean":"2024-09-21T11:48:33.005671+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:48:33.005671+0000","last_undegraded":"2024-09-21T11:48:33.005671+0000","last_fullsized":"2024-09-21T11:48:33.005671+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:38.235643+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"236'16218","reported_seq":21968,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:35.919849+0000","last_change":"2024-09-21T11:48:17.532789+0000","last_active":"2024-09-21T11:48:35.919849+0000","last_peered":"2024-09-21T11:48:35.919849+0000","last_clean":"2024-09-21T11:48:35.919849+0000","last_became_active":"2024-09-21T11:48:17.482971+0000","last_became_peered":"2024-09-21T11:48:17.482971+0000","last_unstale":"2024-09-21T11:48:35.919849+0000","last_undegraded":"2024-09-21T11:48:35.919849+0000","last_fullsized":"2024-09-21T11:48:35.919849+0000","mapping_epoch":230,"log_start":"205'11445","ondisk_log_start":"205'11445","created":160,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4773,"log_dups_size":0,"ondisk_log_size":4773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63438848,"num_objects":968,"num_object_clones":0,"num_object_copies":2904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16218,"num_write_kb":549952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2785,"num_bytes_recovered":182517760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":94,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.023241+0000","last_change":"2024-09-21T11:47:03.400465+0000","last_active":"2024-09-21T11:48:33.023241+0000","last_peered":"2024-09-21T11:48:33.023241+0000","last_clean":"2024-09-21T11:48:33.023241+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:48:33.023241+0000","last_undegraded":"2024-09-21T11:48:33.023241+0000","last_fullsized":"2024-09-21T11:48:33.023241+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:22.708839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.708839+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":409,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.045478+0000","last_change":"2024-09-21T11:47:03.391541+0000","last_active":"2024-09-21T11:48:36.045478+0000","last_peered":"2024-09-21T11:48:36.045478+0000","last_clean":"2024-09-21T11:48:36.045478+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:48:36.045478+0000","last_undegraded":"2024-09-21T11:48:36.045478+0000","last_fullsized":"2024-09-21T11:48:36.045478+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.836903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.836903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":489,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007736+0000","last_change":"2024-09-21T11:47:01.933697+0000","last_active":"2024-09-21T11:48:33.007736+0000","last_peered":"2024-09-21T11:48:33.007736+0000","last_clean":"2024-09-21T11:48:33.007736+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:48:33.007736+0000","last_undegraded":"2024-09-21T11:48:33.007736+0000","last_fullsized":"2024-09-21T11:48:33.007736+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:01.933648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:01.933648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":386,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020947+0000","last_change":"2024-09-21T11:48:15.013107+0000","last_active":"2024-09-21T11:48:33.020947+0000","last_peered":"2024-09-21T11:48:33.020947+0000","last_clean":"2024-09-21T11:48:33.020947+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:48:33.020947+0000","last_undegraded":"2024-09-21T11:48:33.020947+0000","last_fullsized":"2024-09-21T11:48:33.020947+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:15.013058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:15.013058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.125286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"237'16255","reported_seq":16505,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.675276+0000","last_change":"2024-09-21T11:46:19.360961+0000","last_active":"2024-09-21T11:48:36.675276+0000","last_peered":"2024-09-21T11:48:36.675276+0000","last_clean":"2024-09-21T11:48:36.675276+0000","last_became_active":"2024-09-21T11:44:51.744224+0000","last_became_peered":"2024-09-21T11:44:51.744224+0000","last_unstale":"2024-09-21T11:48:36.675276+0000","last_undegraded":"2024-09-21T11:48:36.675276+0000","last_fullsized":"2024-09-21T11:48:36.675276+0000","mapping_epoch":160,"log_start":"198'10400","ondisk_log_start":"198'10400","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7561","last_scrub_stamp":"2024-09-21T11:46:19.360902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.360902+0000","objects_scrubbed":7459,"log_size":5855,"log_dups_size":0,"ondisk_log_size":5855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59179008,"num_objects":903,"num_object_clones":0,"num_object_copies":2709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16255,"num_write_kb":549056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":424,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.045287+0000","last_change":"2024-09-21T11:46:58.789719+0000","last_active":"2024-09-21T11:48:36.045287+0000","last_peered":"2024-09-21T11:48:36.045287+0000","last_clean":"2024-09-21T11:48:36.045287+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:48:36.045287+0000","last_undegraded":"2024-09-21T11:48:36.045287+0000","last_fullsized":"2024-09-21T11:48:36.045287+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:46:58.789625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.789625+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":395,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.044840+0000","last_change":"2024-09-21T11:46:52.891978+0000","last_active":"2024-09-21T11:48:36.044840+0000","last_peered":"2024-09-21T11:48:36.044840+0000","last_clean":"2024-09-21T11:48:36.044840+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:48:36.044840+0000","last_undegraded":"2024-09-21T11:48:36.044840+0000","last_fullsized":"2024-09-21T11:48:36.044840+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.891793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.891793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":378,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.006756+0000","last_change":"2024-09-21T11:44:13.946705+0000","last_active":"2024-09-21T11:48:33.006756+0000","last_peered":"2024-09-21T11:48:33.006756+0000","last_clean":"2024-09-21T11:48:33.006756+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:48:33.006756+0000","last_undegraded":"2024-09-21T11:48:33.006756+0000","last_fullsized":"2024-09-21T11:48:33.006756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:42.015785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":374,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007250+0000","last_change":"2024-09-21T11:46:24.717355+0000","last_active":"2024-09-21T11:48:33.007250+0000","last_peered":"2024-09-21T11:48:33.007250+0000","last_clean":"2024-09-21T11:48:33.007250+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:48:33.007250+0000","last_undegraded":"2024-09-21T11:48:33.007250+0000","last_fullsized":"2024-09-21T11:48:33.007250+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.717285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.717285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:58.045440+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"236'16144","reported_seq":16346,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:35.927118+0000","last_change":"2024-09-21T11:48:05.746078+0000","last_active":"2024-09-21T11:48:35.927118+0000","last_peered":"2024-09-21T11:48:35.927118+0000","last_clean":"2024-09-21T11:48:35.927118+0000","last_became_active":"2024-09-21T11:44:51.744633+0000","last_became_peered":"2024-09-21T11:44:51.744633+0000","last_unstale":"2024-09-21T11:48:35.927118+0000","last_undegraded":"2024-09-21T11:48:35.927118+0000","last_fullsized":"2024-09-21T11:48:35.927118+0000","mapping_epoch":160,"log_start":"200'10741","ondisk_log_start":"200'10741","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'14372","last_scrub_stamp":"2024-09-21T11:48:05.746017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.746017+0000","objects_scrubbed":2702,"log_size":5403,"log_dups_size":0,"ondisk_log_size":5403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:10.815328+0000","scrub_duration":670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59703320,"num_objects":912,"num_object_clones":0,"num_object_copies":2736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":912,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16144,"num_write_kb":545729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":419,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.006946+0000","last_change":"2024-09-21T11:47:33.018713+0000","last_active":"2024-09-21T11:48:33.006946+0000","last_peered":"2024-09-21T11:48:33.006946+0000","last_clean":"2024-09-21T11:48:33.006946+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:48:33.006946+0000","last_undegraded":"2024-09-21T11:48:33.006946+0000","last_fullsized":"2024-09-21T11:48:33.006946+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:47:33.018639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:33.018639+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:41.906194+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":418,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.048864+0000","last_change":"2024-09-21T11:47:03.429013+0000","last_active":"2024-09-21T11:48:36.048864+0000","last_peered":"2024-09-21T11:48:36.048864+0000","last_clean":"2024-09-21T11:48:36.048864+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:48:36.048864+0000","last_undegraded":"2024-09-21T11:48:36.048864+0000","last_fullsized":"2024-09-21T11:48:36.048864+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:59.702813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.702813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":515,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.045108+0000","last_change":"2024-09-21T11:47:03.406087+0000","last_active":"2024-09-21T11:48:36.045108+0000","last_peered":"2024-09-21T11:48:36.045108+0000","last_clean":"2024-09-21T11:48:36.045108+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:48:36.045108+0000","last_undegraded":"2024-09-21T11:48:36.045108+0000","last_fullsized":"2024-09-21T11:48:36.045108+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:08.620519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.620519+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":376,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.026554+0000","last_change":"2024-09-21T11:48:10.667288+0000","last_active":"2024-09-21T11:48:33.026554+0000","last_peered":"2024-09-21T11:48:33.026554+0000","last_clean":"2024-09-21T11:48:33.026554+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:48:33.026554+0000","last_undegraded":"2024-09-21T11:48:33.026554+0000","last_fullsized":"2024-09-21T11:48:33.026554+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.667221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.667221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:11.606721+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"237'16472","reported_seq":16724,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.668246+0000","last_change":"2024-09-21T11:46:22.226583+0000","last_active":"2024-09-21T11:48:36.668246+0000","last_peered":"2024-09-21T11:48:36.668246+0000","last_clean":"2024-09-21T11:48:36.668246+0000","last_became_active":"2024-09-21T11:44:51.412662+0000","last_became_peered":"2024-09-21T11:44:51.412662+0000","last_unstale":"2024-09-21T11:48:36.668246+0000","last_undegraded":"2024-09-21T11:48:36.668246+0000","last_fullsized":"2024-09-21T11:48:36.668246+0000","mapping_epoch":160,"log_start":"199'10700","ondisk_log_start":"199'10700","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7855","last_scrub_stamp":"2024-09-21T11:46:22.226400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.226400+0000","objects_scrubbed":7719,"log_size":5772,"log_dups_size":0,"ondisk_log_size":5772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62259200,"num_objects":950,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16472,"num_write_kb":557504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":90,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.027449+0000","last_change":"2024-09-21T11:47:03.398638+0000","last_active":"2024-09-21T11:48:33.027449+0000","last_peered":"2024-09-21T11:48:33.027449+0000","last_clean":"2024-09-21T11:48:33.027449+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:48:33.027449+0000","last_undegraded":"2024-09-21T11:48:33.027449+0000","last_fullsized":"2024-09-21T11:48:33.027449+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:12.227121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.227121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":401,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.021221+0000","last_change":"2024-09-21T11:48:07.131578+0000","last_active":"2024-09-21T11:48:33.021221+0000","last_peered":"2024-09-21T11:48:33.021221+0000","last_clean":"2024-09-21T11:48:33.021221+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:48:33.021221+0000","last_undegraded":"2024-09-21T11:48:33.021221+0000","last_fullsized":"2024-09-21T11:48:33.021221+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.131535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.131535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:08.103888+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":486,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.046021+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:48:36.046021+0000","last_peered":"2024-09-21T11:48:36.046021+0000","last_clean":"2024-09-21T11:48:36.046021+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:48:36.046021+0000","last_undegraded":"2024-09-21T11:48:36.046021+0000","last_fullsized":"2024-09-21T11:48:36.046021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.263748+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":375,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.044662+0000","last_change":"2024-09-21T11:46:28.010676+0000","last_active":"2024-09-21T11:48:36.044662+0000","last_peered":"2024-09-21T11:48:36.044662+0000","last_clean":"2024-09-21T11:48:36.044662+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:48:36.044662+0000","last_undegraded":"2024-09-21T11:48:36.044662+0000","last_fullsized":"2024-09-21T11:48:36.044662+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:28.010602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.010602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"237'16155","reported_seq":25302,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.737074+0000","last_change":"2024-09-21T11:47:54.309078+0000","last_active":"2024-09-21T11:48:36.737074+0000","last_peered":"2024-09-21T11:48:36.737074+0000","last_clean":"2024-09-21T11:48:36.737074+0000","last_became_active":"2024-09-21T11:47:54.270690+0000","last_became_peered":"2024-09-21T11:47:54.270690+0000","last_unstale":"2024-09-21T11:48:36.737074+0000","last_undegraded":"2024-09-21T11:48:36.737074+0000","last_fullsized":"2024-09-21T11:48:36.737074+0000","mapping_epoch":222,"log_start":"205'11100","ondisk_log_start":"205'11100","created":160,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'7360","last_scrub_stamp":"2024-09-21T11:46:17.038356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.038356+0000","objects_scrubbed":7242,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58523648,"num_objects":893,"num_object_clones":0,"num_object_copies":2679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16155,"num_write_kb":545536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4441,"num_bytes_recovered":291045376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":419,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.046518+0000","last_change":"2024-09-21T11:47:03.750064+0000","last_active":"2024-09-21T11:48:36.046518+0000","last_peered":"2024-09-21T11:48:36.046518+0000","last_clean":"2024-09-21T11:48:36.046518+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:48:36.046518+0000","last_undegraded":"2024-09-21T11:48:36.046518+0000","last_fullsized":"2024-09-21T11:48:36.046518+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:47:03.749962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.749962+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":78,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.023169+0000","last_change":"2024-09-21T11:47:03.382675+0000","last_active":"2024-09-21T11:48:33.023169+0000","last_peered":"2024-09-21T11:48:33.023169+0000","last_clean":"2024-09-21T11:48:33.023169+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:48:33.023169+0000","last_undegraded":"2024-09-21T11:48:33.023169+0000","last_fullsized":"2024-09-21T11:48:33.023169+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:08.723593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.723593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":377,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.005254+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:48:33.005254+0000","last_peered":"2024-09-21T11:48:33.005254+0000","last_clean":"2024-09-21T11:48:33.005254+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:48:33.005254+0000","last_undegraded":"2024-09-21T11:48:33.005254+0000","last_fullsized":"2024-09-21T11:48:33.005254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.032409+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":78,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.023115+0000","last_change":"2024-09-21T11:47:03.379157+0000","last_active":"2024-09-21T11:48:33.023115+0000","last_peered":"2024-09-21T11:48:33.023115+0000","last_clean":"2024-09-21T11:48:33.023115+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:48:33.023115+0000","last_undegraded":"2024-09-21T11:48:33.023115+0000","last_fullsized":"2024-09-21T11:48:33.023115+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.475150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"236'16273","reported_seq":16436,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:35.980765+0000","last_change":"2024-09-21T11:44:51.745506+0000","last_active":"2024-09-21T11:48:35.980765+0000","last_peered":"2024-09-21T11:48:35.980765+0000","last_clean":"2024-09-21T11:48:35.980765+0000","last_became_active":"2024-09-21T11:44:51.745200+0000","last_became_peered":"2024-09-21T11:44:51.745200+0000","last_unstale":"2024-09-21T11:48:35.980765+0000","last_undegraded":"2024-09-21T11:48:35.980765+0000","last_fullsized":"2024-09-21T11:48:35.980765+0000","mapping_epoch":160,"log_start":"200'10900","ondisk_log_start":"200'10900","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5373,"log_dups_size":0,"ondisk_log_size":5373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64421888,"num_objects":983,"num_object_clones":0,"num_object_copies":2949,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16273,"num_write_kb":552192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"205'31","reported_seq":119,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.023277+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:48:33.023277+0000","last_peered":"2024-09-21T11:48:33.023277+0000","last_clean":"2024-09-21T11:48:33.023277+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:48:33.023277+0000","last_undegraded":"2024-09-21T11:48:33.023277+0000","last_fullsized":"2024-09-21T11:48:33.023277+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":384,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007788+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:48:33.007788+0000","last_peered":"2024-09-21T11:48:33.007788+0000","last_clean":"2024-09-21T11:48:33.007788+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:48:33.007788+0000","last_undegraded":"2024-09-21T11:48:33.007788+0000","last_fullsized":"2024-09-21T11:48:33.007788+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:39.997756+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":417,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020723+0000","last_change":"2024-09-21T11:47:20.000195+0000","last_active":"2024-09-21T11:48:33.020723+0000","last_peered":"2024-09-21T11:48:33.020723+0000","last_clean":"2024-09-21T11:48:33.020723+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:48:33.020723+0000","last_undegraded":"2024-09-21T11:48:33.020723+0000","last_fullsized":"2024-09-21T11:48:33.020723+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:47:20.000051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.000051+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:48.196772+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":393,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.046324+0000","last_change":"2024-09-21T11:47:03.385173+0000","last_active":"2024-09-21T11:48:36.046324+0000","last_peered":"2024-09-21T11:48:36.046324+0000","last_clean":"2024-09-21T11:48:36.046324+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:48:36.046324+0000","last_undegraded":"2024-09-21T11:48:36.046324+0000","last_fullsized":"2024-09-21T11:48:36.046324+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:32.321908+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":367,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.043144+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:48:36.043144+0000","last_peered":"2024-09-21T11:48:36.043144+0000","last_clean":"2024-09-21T11:48:36.043144+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:48:36.043144+0000","last_undegraded":"2024-09-21T11:48:36.043144+0000","last_fullsized":"2024-09-21T11:48:36.043144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.188178+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"237'16091","reported_seq":20605,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.745823+0000","last_change":"2024-09-21T11:48:24.579575+0000","last_active":"2024-09-21T11:48:36.745823+0000","last_peered":"2024-09-21T11:48:36.745823+0000","last_clean":"2024-09-21T11:48:36.745823+0000","last_became_active":"2024-09-21T11:48:24.535114+0000","last_became_peered":"2024-09-21T11:48:24.535114+0000","last_unstale":"2024-09-21T11:48:36.745823+0000","last_undegraded":"2024-09-21T11:48:36.745823+0000","last_fullsized":"2024-09-21T11:48:36.745823+0000","mapping_epoch":233,"log_start":"205'11000","ondisk_log_start":"205'11000","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58130432,"num_objects":887,"num_object_clones":0,"num_object_copies":2661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16091,"num_write_kb":543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":141754368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":26,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.046503+0000","last_change":"2024-09-21T11:48:33.075081+0000","last_active":"2024-09-21T11:48:36.046503+0000","last_peered":"2024-09-21T11:48:36.046503+0000","last_clean":"2024-09-21T11:48:36.046503+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:48:36.046503+0000","last_undegraded":"2024-09-21T11:48:36.046503+0000","last_fullsized":"2024-09-21T11:48:36.046503+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.876172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":82,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.027475+0000","last_change":"2024-09-21T11:47:03.391283+0000","last_active":"2024-09-21T11:48:33.027475+0000","last_peered":"2024-09-21T11:48:33.027475+0000","last_clean":"2024-09-21T11:48:33.027475+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:48:33.027475+0000","last_undegraded":"2024-09-21T11:48:33.027475+0000","last_fullsized":"2024-09-21T11:48:33.027475+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.552518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":405,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.043414+0000","last_change":"2024-09-21T11:46:54.864288+0000","last_active":"2024-09-21T11:48:36.043414+0000","last_peered":"2024-09-21T11:48:36.043414+0000","last_clean":"2024-09-21T11:48:36.043414+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:48:36.043414+0000","last_undegraded":"2024-09-21T11:48:36.043414+0000","last_fullsized":"2024-09-21T11:48:36.043414+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:54.864074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.864074+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":387,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.048951+0000","last_change":"2024-09-21T11:48:16.076049+0000","last_active":"2024-09-21T11:48:36.048951+0000","last_peered":"2024-09-21T11:48:36.048951+0000","last_clean":"2024-09-21T11:48:36.048951+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:48:36.048951+0000","last_undegraded":"2024-09-21T11:48:36.048951+0000","last_fullsized":"2024-09-21T11:48:36.048951+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:16.076003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:16.076003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.071160+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":381,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.028791+0000","last_change":"2024-09-21T11:48:07.613756+0000","last_active":"2024-09-21T11:48:33.028791+0000","last_peered":"2024-09-21T11:48:33.028791+0000","last_clean":"2024-09-21T11:48:33.028791+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:48:33.028791+0000","last_undegraded":"2024-09-21T11:48:33.028791+0000","last_fullsized":"2024-09-21T11:48:33.028791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.613691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.613691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:32.231718+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"236'15929","reported_seq":25374,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:35.896013+0000","last_change":"2024-09-21T11:47:52.322057+0000","last_active":"2024-09-21T11:48:35.896013+0000","last_peered":"2024-09-21T11:48:35.896013+0000","last_clean":"2024-09-21T11:48:35.896013+0000","last_became_active":"2024-09-21T11:47:52.264063+0000","last_became_peered":"2024-09-21T11:47:52.264063+0000","last_unstale":"2024-09-21T11:48:35.896013+0000","last_undegraded":"2024-09-21T11:48:35.896013+0000","last_fullsized":"2024-09-21T11:48:35.896013+0000","mapping_epoch":220,"log_start":"205'11106","ondisk_log_start":"205'11106","created":160,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4823,"log_dups_size":0,"ondisk_log_size":4823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62455808,"num_objects":953,"num_object_clones":0,"num_object_copies":2859,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15929,"num_write_kb":540224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4633,"num_bytes_recovered":303628288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":442,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.048733+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:48:36.048733+0000","last_peered":"2024-09-21T11:48:36.048733+0000","last_clean":"2024-09-21T11:48:36.048733+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:48:36.048733+0000","last_undegraded":"2024-09-21T11:48:36.048733+0000","last_fullsized":"2024-09-21T11:48:36.048733+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.401660+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":395,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.045357+0000","last_change":"2024-09-21T11:46:29.937866+0000","last_active":"2024-09-21T11:48:36.045357+0000","last_peered":"2024-09-21T11:48:36.045357+0000","last_clean":"2024-09-21T11:48:36.045357+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:48:36.045357+0000","last_undegraded":"2024-09-21T11:48:36.045357+0000","last_fullsized":"2024-09-21T11:48:36.045357+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.937799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.937799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":943,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:34.482285+0000","last_change":"2024-09-21T11:48:17.596682+0000","last_active":"2024-09-21T11:48:34.482285+0000","last_peered":"2024-09-21T11:48:34.482285+0000","last_clean":"2024-09-21T11:48:34.482285+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:48:34.482285+0000","last_undegraded":"2024-09-21T11:48:34.482285+0000","last_fullsized":"2024-09-21T11:48:34.482285+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:48:17.596571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.596571+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.625607+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":507,"num_read_kb":502,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":390,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020909+0000","last_change":"2024-09-21T11:47:22.025919+0000","last_active":"2024-09-21T11:48:33.020909+0000","last_peered":"2024-09-21T11:48:33.020909+0000","last_clean":"2024-09-21T11:48:33.020909+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:48:33.020909+0000","last_undegraded":"2024-09-21T11:48:33.020909+0000","last_fullsized":"2024-09-21T11:48:33.020909+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:22.025842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.025842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":78,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.022301+0000","last_change":"2024-09-21T11:47:03.374507+0000","last_active":"2024-09-21T11:48:33.022301+0000","last_peered":"2024-09-21T11:48:33.022301+0000","last_clean":"2024-09-21T11:48:33.022301+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:48:33.022301+0000","last_undegraded":"2024-09-21T11:48:33.022301+0000","last_fullsized":"2024-09-21T11:48:33.022301+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"236'16080","reported_seq":16378,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:35.942172+0000","last_change":"2024-09-21T11:47:31.508923+0000","last_active":"2024-09-21T11:48:35.942172+0000","last_peered":"2024-09-21T11:48:35.942172+0000","last_clean":"2024-09-21T11:48:35.942172+0000","last_became_active":"2024-09-21T11:44:51.410701+0000","last_became_peered":"2024-09-21T11:44:51.410701+0000","last_unstale":"2024-09-21T11:48:35.942172+0000","last_undegraded":"2024-09-21T11:48:35.942172+0000","last_fullsized":"2024-09-21T11:48:35.942172+0000","mapping_epoch":160,"log_start":"197'9600","ondisk_log_start":"197'9600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'12576","last_scrub_stamp":"2024-09-21T11:47:31.508822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:47:31.508822+0000","objects_scrubbed":4421,"log_size":6480,"log_dups_size":0,"ondisk_log_size":6480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.146813+0000","scrub_duration":1214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58327040,"num_objects":890,"num_object_clones":0,"num_object_copies":2670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16080,"num_write_kb":543040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":507,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.005727+0000","last_change":"2024-09-21T11:48:06.171363+0000","last_active":"2024-09-21T11:48:33.005727+0000","last_peered":"2024-09-21T11:48:33.005727+0000","last_clean":"2024-09-21T11:48:33.005727+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:48:33.005727+0000","last_undegraded":"2024-09-21T11:48:33.005727+0000","last_fullsized":"2024-09-21T11:48:33.005727+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:48:06.171279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.171279+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:12.520316+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":396,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.042420+0000","last_change":"2024-09-21T11:46:57.833589+0000","last_active":"2024-09-21T11:48:36.042420+0000","last_peered":"2024-09-21T11:48:36.042420+0000","last_clean":"2024-09-21T11:48:36.042420+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:48:36.042420+0000","last_undegraded":"2024-09-21T11:48:36.042420+0000","last_fullsized":"2024-09-21T11:48:36.042420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:57.833329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.833329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":447,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.026790+0000","last_change":"2024-09-21T11:46:58.645189+0000","last_active":"2024-09-21T11:48:33.026790+0000","last_peered":"2024-09-21T11:48:33.026790+0000","last_clean":"2024-09-21T11:48:33.026790+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:48:33.026790+0000","last_undegraded":"2024-09-21T11:48:33.026790+0000","last_fullsized":"2024-09-21T11:48:33.026790+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:46:58.645122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.645122+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.573944+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":385,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.030037+0000","last_change":"2024-09-21T11:47:25.395237+0000","last_active":"2024-09-21T11:48:33.030037+0000","last_peered":"2024-09-21T11:48:33.030037+0000","last_clean":"2024-09-21T11:48:33.030037+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:48:33.030037+0000","last_undegraded":"2024-09-21T11:48:33.030037+0000","last_fullsized":"2024-09-21T11:48:33.030037+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.395185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.395185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.204032+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":380,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020820+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:48:33.020820+0000","last_peered":"2024-09-21T11:48:33.020820+0000","last_clean":"2024-09-21T11:48:33.020820+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:48:33.020820+0000","last_undegraded":"2024-09-21T11:48:33.020820+0000","last_fullsized":"2024-09-21T11:48:33.020820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:30.020245+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"236'16044","reported_seq":16323,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:35.875145+0000","last_change":"2024-09-21T11:48:09.830854+0000","last_active":"2024-09-21T11:48:35.875145+0000","last_peered":"2024-09-21T11:48:35.875145+0000","last_clean":"2024-09-21T11:48:35.875145+0000","last_became_active":"2024-09-21T11:44:51.410197+0000","last_became_peered":"2024-09-21T11:44:51.410197+0000","last_unstale":"2024-09-21T11:48:35.875145+0000","last_undegraded":"2024-09-21T11:48:35.875145+0000","last_fullsized":"2024-09-21T11:48:35.875145+0000","mapping_epoch":160,"log_start":"205'11200","ondisk_log_start":"205'11200","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"226'14544","last_scrub_stamp":"2024-09-21T11:48:09.830774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.830774+0000","objects_scrubbed":2470,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.225752+0000","scrub_duration":808,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62521344,"num_objects":954,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16044,"num_write_kb":543936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":438,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007500+0000","last_change":"2024-09-21T11:48:05.072998+0000","last_active":"2024-09-21T11:48:33.007500+0000","last_peered":"2024-09-21T11:48:33.007500+0000","last_clean":"2024-09-21T11:48:33.007500+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:48:33.007500+0000","last_undegraded":"2024-09-21T11:48:33.007500+0000","last_fullsized":"2024-09-21T11:48:33.007500+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:05.072910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.072910+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:33.824160+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":401,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007018+0000","last_change":"2024-09-21T11:47:00.950268+0000","last_active":"2024-09-21T11:48:33.007018+0000","last_peered":"2024-09-21T11:48:33.007018+0000","last_clean":"2024-09-21T11:48:33.007018+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:48:33.007018+0000","last_undegraded":"2024-09-21T11:48:33.007018+0000","last_fullsized":"2024-09-21T11:48:33.007018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:00.950208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:00.950208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":384,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020799+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:48:33.020799+0000","last_peered":"2024-09-21T11:48:33.020799+0000","last_clean":"2024-09-21T11:48:33.020799+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:48:33.020799+0000","last_undegraded":"2024-09-21T11:48:33.020799+0000","last_fullsized":"2024-09-21T11:48:33.020799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.795578+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":382,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.041627+0000","last_change":"2024-09-21T11:48:23.285482+0000","last_active":"2024-09-21T11:48:36.041627+0000","last_peered":"2024-09-21T11:48:36.041627+0000","last_clean":"2024-09-21T11:48:36.041627+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:48:36.041627+0000","last_undegraded":"2024-09-21T11:48:36.041627+0000","last_fullsized":"2024-09-21T11:48:36.041627+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.285283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.285283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:52.473277+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"236'16433","reported_seq":22851,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:35.909410+0000","last_change":"2024-09-21T11:48:15.522333+0000","last_active":"2024-09-21T11:48:35.909410+0000","last_peered":"2024-09-21T11:48:35.909410+0000","last_clean":"2024-09-21T11:48:35.909410+0000","last_became_active":"2024-09-21T11:48:15.471002+0000","last_became_peered":"2024-09-21T11:48:15.471002+0000","last_unstale":"2024-09-21T11:48:35.909410+0000","last_undegraded":"2024-09-21T11:48:35.909410+0000","last_fullsized":"2024-09-21T11:48:35.909410+0000","mapping_epoch":228,"log_start":"205'11613","ondisk_log_start":"205'11613","created":160,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'10449","last_scrub_stamp":"2024-09-21T11:46:52.316998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.316998+0000","objects_scrubbed":7011,"log_size":4820,"log_dups_size":0,"ondisk_log_size":4820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60489728,"num_objects":923,"num_object_clones":0,"num_object_copies":2769,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16433,"num_write_kb":555392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3078,"num_bytes_recovered":201719808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":492,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.035175+0000","last_change":"2024-09-21T11:48:33.035096+0000","last_active":"2024-09-21T11:48:33.035175+0000","last_peered":"2024-09-21T11:48:33.035175+0000","last_clean":"2024-09-21T11:48:33.035175+0000","last_became_active":"2024-09-21T11:48:33.022321+0000","last_became_peered":"2024-09-21T11:48:33.022321+0000","last_unstale":"2024-09-21T11:48:33.035175+0000","last_undegraded":"2024-09-21T11:48:33.035175+0000","last_fullsized":"2024-09-21T11:48:33.035175+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:47:22.974761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.974761+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:40.979174+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":414,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.005419+0000","last_change":"2024-09-21T11:47:03.365071+0000","last_active":"2024-09-21T11:48:33.005419+0000","last_peered":"2024-09-21T11:48:33.005419+0000","last_clean":"2024-09-21T11:48:33.005419+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:48:33.005419+0000","last_undegraded":"2024-09-21T11:48:33.005419+0000","last_fullsized":"2024-09-21T11:48:33.005419+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.692429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.692429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:54.181079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":78,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.022197+0000","last_change":"2024-09-21T11:47:03.375108+0000","last_active":"2024-09-21T11:48:33.022197+0000","last_peered":"2024-09-21T11:48:33.022197+0000","last_clean":"2024-09-21T11:48:33.022197+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:48:33.022197+0000","last_undegraded":"2024-09-21T11:48:33.022197+0000","last_fullsized":"2024-09-21T11:48:33.022197+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:50.008635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:50.008635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":391,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.046706+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:48:36.046706+0000","last_peered":"2024-09-21T11:48:36.046706+0000","last_clean":"2024-09-21T11:48:36.046706+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:48:36.046706+0000","last_undegraded":"2024-09-21T11:48:36.046706+0000","last_fullsized":"2024-09-21T11:48:36.046706+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"237'16103","reported_seq":25226,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.752231+0000","last_change":"2024-09-21T11:47:53.304799+0000","last_active":"2024-09-21T11:48:36.752231+0000","last_peered":"2024-09-21T11:48:36.752231+0000","last_clean":"2024-09-21T11:48:36.752231+0000","last_became_active":"2024-09-21T11:47:53.264856+0000","last_became_peered":"2024-09-21T11:47:53.264856+0000","last_unstale":"2024-09-21T11:48:36.752231+0000","last_undegraded":"2024-09-21T11:48:36.752231+0000","last_fullsized":"2024-09-21T11:48:36.752231+0000","mapping_epoch":221,"log_start":"205'11100","ondisk_log_start":"205'11100","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.133366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54591488,"num_objects":833,"num_object_clones":0,"num_object_copies":2499,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16103,"num_write_kb":541952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4468,"num_bytes_recovered":292749312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":412,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007250+0000","last_change":"2024-09-21T11:46:11.730053+0000","last_active":"2024-09-21T11:48:33.007250+0000","last_peered":"2024-09-21T11:48:33.007250+0000","last_clean":"2024-09-21T11:48:33.007250+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:48:33.007250+0000","last_undegraded":"2024-09-21T11:48:33.007250+0000","last_fullsized":"2024-09-21T11:48:33.007250+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:11.729984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.729984+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.980319+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":379,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.044334+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:48:36.044334+0000","last_peered":"2024-09-21T11:48:36.044334+0000","last_clean":"2024-09-21T11:48:36.044334+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:48:36.044334+0000","last_undegraded":"2024-09-21T11:48:36.044334+0000","last_fullsized":"2024-09-21T11:48:36.044334+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:27.332154+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":499,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.043598+0000","last_change":"2024-09-21T11:46:52.834287+0000","last_active":"2024-09-21T11:48:36.043598+0000","last_peered":"2024-09-21T11:48:36.043598+0000","last_clean":"2024-09-21T11:48:36.043598+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:48:36.043598+0000","last_undegraded":"2024-09-21T11:48:36.043598+0000","last_fullsized":"2024-09-21T11:48:36.043598+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:52.834206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.834206+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":387,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007218+0000","last_change":"2024-09-21T11:48:07.992586+0000","last_active":"2024-09-21T11:48:33.007218+0000","last_peered":"2024-09-21T11:48:33.007218+0000","last_clean":"2024-09-21T11:48:33.007218+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:48:33.007218+0000","last_undegraded":"2024-09-21T11:48:33.007218+0000","last_fullsized":"2024-09-21T11:48:33.007218+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.992517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.992517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.884349+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"236'16200","reported_seq":22348,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:35.849291+0000","last_change":"2024-09-21T11:48:16.538801+0000","last_active":"2024-09-21T11:48:35.849291+0000","last_peered":"2024-09-21T11:48:35.849291+0000","last_clean":"2024-09-21T11:48:35.849291+0000","last_became_active":"2024-09-21T11:48:16.476754+0000","last_became_peered":"2024-09-21T11:48:16.476754+0000","last_unstale":"2024-09-21T11:48:35.849291+0000","last_undegraded":"2024-09-21T11:48:35.849291+0000","last_fullsized":"2024-09-21T11:48:35.849291+0000","mapping_epoch":229,"log_start":"205'11387","ondisk_log_start":"205'11387","created":160,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7408","last_scrub_stamp":"2024-09-21T11:46:17.934135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.934135+0000","objects_scrubbed":7290,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65273856,"num_objects":996,"num_object_clones":0,"num_object_copies":2988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16200,"num_write_kb":550272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2943,"num_bytes_recovered":192872448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":432,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.046421+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:48:36.046421+0000","last_peered":"2024-09-21T11:48:36.046421+0000","last_clean":"2024-09-21T11:48:36.046421+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:48:36.046421+0000","last_undegraded":"2024-09-21T11:48:36.046421+0000","last_fullsized":"2024-09-21T11:48:36.046421+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:41.378708+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":392,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.021472+0000","last_change":"2024-09-21T11:48:08.172717+0000","last_active":"2024-09-21T11:48:33.021472+0000","last_peered":"2024-09-21T11:48:33.021472+0000","last_clean":"2024-09-21T11:48:33.021472+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:48:33.021472+0000","last_undegraded":"2024-09-21T11:48:33.021472+0000","last_fullsized":"2024-09-21T11:48:33.021472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:08.172656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:08.172656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:18.830907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":379,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.042800+0000","last_change":"2024-09-21T11:46:55.879988+0000","last_active":"2024-09-21T11:48:36.042800+0000","last_peered":"2024-09-21T11:48:36.042800+0000","last_clean":"2024-09-21T11:48:36.042800+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:48:36.042800+0000","last_undegraded":"2024-09-21T11:48:36.042800+0000","last_fullsized":"2024-09-21T11:48:36.042800+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.879761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.879761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":366,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.045267+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:48:36.045267+0000","last_peered":"2024-09-21T11:48:36.045267+0000","last_clean":"2024-09-21T11:48:36.045267+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:48:36.045267+0000","last_undegraded":"2024-09-21T11:48:36.045267+0000","last_fullsized":"2024-09-21T11:48:36.045267+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"237'16341","reported_seq":16589,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.650108+0000","last_change":"2024-09-21T11:46:15.665868+0000","last_active":"2024-09-21T11:48:36.650108+0000","last_peered":"2024-09-21T11:48:36.650108+0000","last_clean":"2024-09-21T11:48:36.650108+0000","last_became_active":"2024-09-21T11:44:51.744226+0000","last_became_peered":"2024-09-21T11:44:51.744226+0000","last_unstale":"2024-09-21T11:48:36.650108+0000","last_undegraded":"2024-09-21T11:48:36.650108+0000","last_fullsized":"2024-09-21T11:48:36.650108+0000","mapping_epoch":160,"log_start":"198'10500","ondisk_log_start":"198'10500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7351","last_scrub_stamp":"2024-09-21T11:46:15.665775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:15.665775+0000","objects_scrubbed":7260,"log_size":5841,"log_dups_size":0,"ondisk_log_size":5841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61669376,"num_objects":941,"num_object_clones":0,"num_object_copies":2823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16341,"num_write_kb":553024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":473,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.005208+0000","last_change":"2024-09-21T11:47:03.397050+0000","last_active":"2024-09-21T11:48:33.005208+0000","last_peered":"2024-09-21T11:48:33.005208+0000","last_clean":"2024-09-21T11:48:33.005208+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:48:33.005208+0000","last_undegraded":"2024-09-21T11:48:33.005208+0000","last_fullsized":"2024-09-21T11:48:33.005208+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:46:20.650403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.650403+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:51.167956+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":392,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.029150+0000","last_change":"2024-09-21T11:48:03.546909+0000","last_active":"2024-09-21T11:48:33.029150+0000","last_peered":"2024-09-21T11:48:33.029150+0000","last_clean":"2024-09-21T11:48:33.029150+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:48:33.029150+0000","last_undegraded":"2024-09-21T11:48:33.029150+0000","last_fullsized":"2024-09-21T11:48:33.029150+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:03.546843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:03.546843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.594742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":395,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.042979+0000","last_change":"2024-09-21T11:46:16.775013+0000","last_active":"2024-09-21T11:48:36.042979+0000","last_peered":"2024-09-21T11:48:36.042979+0000","last_clean":"2024-09-21T11:48:36.042979+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:48:36.042979+0000","last_undegraded":"2024-09-21T11:48:36.042979+0000","last_fullsized":"2024-09-21T11:48:36.042979+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:16.774939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.774939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.931030+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":380,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.026694+0000","last_change":"2024-09-21T11:46:59.704981+0000","last_active":"2024-09-21T11:48:33.026694+0000","last_peered":"2024-09-21T11:48:33.026694+0000","last_clean":"2024-09-21T11:48:33.026694+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:48:33.026694+0000","last_undegraded":"2024-09-21T11:48:33.026694+0000","last_fullsized":"2024-09-21T11:48:33.026694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.588396+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"236'16270","reported_seq":16525,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:35.949839+0000","last_change":"2024-09-21T11:46:27.656566+0000","last_active":"2024-09-21T11:48:35.949839+0000","last_peered":"2024-09-21T11:48:35.949839+0000","last_clean":"2024-09-21T11:48:35.949839+0000","last_became_active":"2024-09-21T11:44:51.745609+0000","last_became_peered":"2024-09-21T11:44:51.745609+0000","last_unstale":"2024-09-21T11:48:35.949839+0000","last_undegraded":"2024-09-21T11:48:35.949839+0000","last_fullsized":"2024-09-21T11:48:35.949839+0000","mapping_epoch":160,"log_start":"197'9800","ondisk_log_start":"197'9800","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'8199","last_scrub_stamp":"2024-09-21T11:46:27.656360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.656360+0000","objects_scrubbed":8107,"log_size":6470,"log_dups_size":0,"ondisk_log_size":6470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:46.172763+0000","scrub_duration":2017,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64225280,"num_objects":980,"num_object_clones":0,"num_object_copies":2940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16270,"num_write_kb":552000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":466,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.006842+0000","last_change":"2024-09-21T11:48:04.111199+0000","last_active":"2024-09-21T11:48:33.006842+0000","last_peered":"2024-09-21T11:48:33.006842+0000","last_clean":"2024-09-21T11:48:33.006842+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:48:33.006842+0000","last_undegraded":"2024-09-21T11:48:33.006842+0000","last_fullsized":"2024-09-21T11:48:33.006842+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:48:04.111115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.111115+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:19.507697+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":422,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.027939+0000","last_change":"2024-09-21T11:47:26.411190+0000","last_active":"2024-09-21T11:48:33.027939+0000","last_peered":"2024-09-21T11:48:33.027939+0000","last_clean":"2024-09-21T11:48:33.027939+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:48:33.027939+0000","last_undegraded":"2024-09-21T11:48:33.027939+0000","last_fullsized":"2024-09-21T11:48:33.027939+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:26.411146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:26.411146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:48.131154+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":409,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.042521+0000","last_change":"2024-09-21T11:47:03.385382+0000","last_active":"2024-09-21T11:48:36.042521+0000","last_peered":"2024-09-21T11:48:36.042521+0000","last_clean":"2024-09-21T11:48:36.042521+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:48:36.042521+0000","last_undegraded":"2024-09-21T11:48:36.042521+0000","last_fullsized":"2024-09-21T11:48:36.042521+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.766819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.766819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":389,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.043683+0000","last_change":"2024-09-21T11:47:03.376001+0000","last_active":"2024-09-21T11:48:36.043683+0000","last_peered":"2024-09-21T11:48:36.043683+0000","last_clean":"2024-09-21T11:48:36.043683+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:48:36.043683+0000","last_undegraded":"2024-09-21T11:48:36.043683+0000","last_fullsized":"2024-09-21T11:48:36.043683+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.703712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.703712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"236'16259","reported_seq":16503,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:35.978889+0000","last_change":"2024-09-21T11:46:12.974904+0000","last_active":"2024-09-21T11:48:35.978889+0000","last_peered":"2024-09-21T11:48:35.978889+0000","last_clean":"2024-09-21T11:48:35.978889+0000","last_became_active":"2024-09-21T11:44:51.409710+0000","last_became_peered":"2024-09-21T11:44:51.409710+0000","last_unstale":"2024-09-21T11:48:35.978889+0000","last_undegraded":"2024-09-21T11:48:35.978889+0000","last_fullsized":"2024-09-21T11:48:35.978889+0000","mapping_epoch":160,"log_start":"200'10860","ondisk_log_start":"200'10860","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7106","last_scrub_stamp":"2024-09-21T11:46:12.974687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.974687+0000","objects_scrubbed":7036,"log_size":5399,"log_dups_size":0,"ondisk_log_size":5399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.015746+0000","scrub_duration":1797,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62324736,"num_objects":951,"num_object_clones":0,"num_object_copies":2853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16259,"num_write_kb":550720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":397,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020506+0000","last_change":"2024-09-21T11:46:57.248649+0000","last_active":"2024-09-21T11:48:33.020506+0000","last_peered":"2024-09-21T11:48:33.020506+0000","last_clean":"2024-09-21T11:48:33.020506+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:48:33.020506+0000","last_undegraded":"2024-09-21T11:48:33.020506+0000","last_fullsized":"2024-09-21T11:48:33.020506+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:57.248575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.248575+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":412,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.027530+0000","last_change":"2024-09-21T11:48:09.683093+0000","last_active":"2024-09-21T11:48:33.027530+0000","last_peered":"2024-09-21T11:48:33.027530+0000","last_clean":"2024-09-21T11:48:33.027530+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:48:33.027530+0000","last_undegraded":"2024-09-21T11:48:33.027530+0000","last_fullsized":"2024-09-21T11:48:33.027530+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:48:09.683008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.683008+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:38.676024+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":496,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.027015+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:48:33.027015+0000","last_peered":"2024-09-21T11:48:33.027015+0000","last_clean":"2024-09-21T11:48:33.027015+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:48:33.027015+0000","last_undegraded":"2024-09-21T11:48:33.027015+0000","last_fullsized":"2024-09-21T11:48:33.027015+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.600494+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":372,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020360+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:48:33.020360+0000","last_peered":"2024-09-21T11:48:33.020360+0000","last_clean":"2024-09-21T11:48:33.020360+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:48:33.020360+0000","last_undegraded":"2024-09-21T11:48:33.020360+0000","last_fullsized":"2024-09-21T11:48:33.020360+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.006344+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":400,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.005133+0000","last_change":"2024-09-21T11:48:17.129730+0000","last_active":"2024-09-21T11:48:33.005133+0000","last_peered":"2024-09-21T11:48:33.005133+0000","last_clean":"2024-09-21T11:48:33.005133+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:48:33.005133+0000","last_undegraded":"2024-09-21T11:48:33.005133+0000","last_fullsized":"2024-09-21T11:48:33.005133+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.129675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.129675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.682622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":422,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.044642+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:48:36.044642+0000","last_peered":"2024-09-21T11:48:36.044642+0000","last_clean":"2024-09-21T11:48:36.044642+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:48:36.044642+0000","last_undegraded":"2024-09-21T11:48:36.044642+0000","last_fullsized":"2024-09-21T11:48:36.044642+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":398,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020622+0000","last_change":"2024-09-21T11:47:25.905705+0000","last_active":"2024-09-21T11:48:33.020622+0000","last_peered":"2024-09-21T11:48:33.020622+0000","last_clean":"2024-09-21T11:48:33.020622+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:48:33.020622+0000","last_undegraded":"2024-09-21T11:48:33.020622+0000","last_fullsized":"2024-09-21T11:48:33.020622+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.905645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.905645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.034596+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":388,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.028709+0000","last_change":"2024-09-21T11:47:28.380378+0000","last_active":"2024-09-21T11:48:33.028709+0000","last_peered":"2024-09-21T11:48:33.028709+0000","last_clean":"2024-09-21T11:48:33.028709+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:48:33.028709+0000","last_undegraded":"2024-09-21T11:48:33.028709+0000","last_fullsized":"2024-09-21T11:48:33.028709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:28.380245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:28.380245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:34.745499+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":415,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.048798+0000","last_change":"2024-09-21T11:46:28.973700+0000","last_active":"2024-09-21T11:48:36.048798+0000","last_peered":"2024-09-21T11:48:36.048798+0000","last_clean":"2024-09-21T11:48:36.048798+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:48:36.048798+0000","last_undegraded":"2024-09-21T11:48:36.048798+0000","last_fullsized":"2024-09-21T11:48:36.048798+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:28.973606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.973606+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":384,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.027145+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:48:33.027145+0000","last_peered":"2024-09-21T11:48:33.027145+0000","last_clean":"2024-09-21T11:48:33.027145+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:48:33.027145+0000","last_undegraded":"2024-09-21T11:48:33.027145+0000","last_fullsized":"2024-09-21T11:48:33.027145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:47.600096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":401,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.044501+0000","last_change":"2024-09-21T11:47:03.379216+0000","last_active":"2024-09-21T11:48:36.044501+0000","last_peered":"2024-09-21T11:48:36.044501+0000","last_clean":"2024-09-21T11:48:36.044501+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:48:36.044501+0000","last_undegraded":"2024-09-21T11:48:36.044501+0000","last_fullsized":"2024-09-21T11:48:36.044501+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:48.903980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:48.903980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":391,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.042855+0000","last_change":"2024-09-21T11:48:17.081179+0000","last_active":"2024-09-21T11:48:36.042855+0000","last_peered":"2024-09-21T11:48:36.042855+0000","last_clean":"2024-09-21T11:48:36.042855+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:48:36.042855+0000","last_undegraded":"2024-09-21T11:48:36.042855+0000","last_fullsized":"2024-09-21T11:48:36.042855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.081131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.081131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.264468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":396,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.021140+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:48:33.021140+0000","last_peered":"2024-09-21T11:48:33.021140+0000","last_clean":"2024-09-21T11:48:33.021140+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:48:33.021140+0000","last_undegraded":"2024-09-21T11:48:33.021140+0000","last_fullsized":"2024-09-21T11:48:33.021140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":394,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.046642+0000","last_change":"2024-09-21T11:46:24.087039+0000","last_active":"2024-09-21T11:48:36.046642+0000","last_peered":"2024-09-21T11:48:36.046642+0000","last_clean":"2024-09-21T11:48:36.046642+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:48:36.046642+0000","last_undegraded":"2024-09-21T11:48:36.046642+0000","last_fullsized":"2024-09-21T11:48:36.046642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.086918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.086918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":483,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.042734+0000","last_change":"2024-09-21T11:47:02.849839+0000","last_active":"2024-09-21T11:48:36.042734+0000","last_peered":"2024-09-21T11:48:36.042734+0000","last_clean":"2024-09-21T11:48:36.042734+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:48:36.042734+0000","last_undegraded":"2024-09-21T11:48:36.042734+0000","last_fullsized":"2024-09-21T11:48:36.042734+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:02.849786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:02.849786+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":381,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007296+0000","last_change":"2024-09-21T11:45:38.652432+0000","last_active":"2024-09-21T11:48:33.007296+0000","last_peered":"2024-09-21T11:48:33.007296+0000","last_clean":"2024-09-21T11:48:33.007296+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:48:33.007296+0000","last_undegraded":"2024-09-21T11:48:33.007296+0000","last_fullsized":"2024-09-21T11:48:33.007296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:38.652380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:38.652380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:47.014191+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":428,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.042934+0000","last_change":"2024-09-21T11:47:03.428846+0000","last_active":"2024-09-21T11:48:36.042934+0000","last_peered":"2024-09-21T11:48:36.042934+0000","last_clean":"2024-09-21T11:48:36.042934+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:48:36.042934+0000","last_undegraded":"2024-09-21T11:48:36.042934+0000","last_fullsized":"2024-09-21T11:48:36.042934+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:29.284078+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":384,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.026831+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:48:33.026831+0000","last_peered":"2024-09-21T11:48:33.026831+0000","last_clean":"2024-09-21T11:48:33.026831+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:48:33.026831+0000","last_undegraded":"2024-09-21T11:48:33.026831+0000","last_fullsized":"2024-09-21T11:48:33.026831+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.604026+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":519,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.021089+0000","last_change":"2024-09-21T11:47:03.415793+0000","last_active":"2024-09-21T11:48:33.021089+0000","last_peered":"2024-09-21T11:48:33.021089+0000","last_clean":"2024-09-21T11:48:33.021089+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:48:33.021089+0000","last_undegraded":"2024-09-21T11:48:33.021089+0000","last_fullsized":"2024-09-21T11:48:33.021089+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:10.174971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:10.174971+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.999554+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":382,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.048453+0000","last_change":"2024-09-21T11:45:44.314664+0000","last_active":"2024-09-21T11:48:36.048453+0000","last_peered":"2024-09-21T11:48:36.048453+0000","last_clean":"2024-09-21T11:48:36.048453+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:48:36.048453+0000","last_undegraded":"2024-09-21T11:48:36.048453+0000","last_fullsized":"2024-09-21T11:48:36.048453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:44.314461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.314461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":473,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.033639+0000","last_change":"2024-09-21T11:48:33.033557+0000","last_active":"2024-09-21T11:48:33.033639+0000","last_peered":"2024-09-21T11:48:33.033639+0000","last_clean":"2024-09-21T11:48:33.033639+0000","last_became_active":"2024-09-21T11:48:33.022647+0000","last_became_peered":"2024-09-21T11:48:33.022647+0000","last_unstale":"2024-09-21T11:48:33.033639+0000","last_undegraded":"2024-09-21T11:48:33.033639+0000","last_fullsized":"2024-09-21T11:48:33.033639+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:54.032795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.032795+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":385,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.007500+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:48:33.007500+0000","last_peered":"2024-09-21T11:48:33.007500+0000","last_clean":"2024-09-21T11:48:33.007500+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:48:33.007500+0000","last_undegraded":"2024-09-21T11:48:33.007500+0000","last_fullsized":"2024-09-21T11:48:33.007500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:37.991785+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":78,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.027366+0000","last_change":"2024-09-21T11:47:03.379049+0000","last_active":"2024-09-21T11:48:33.027366+0000","last_peered":"2024-09-21T11:48:33.027366+0000","last_clean":"2024-09-21T11:48:33.027366+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:48:33.027366+0000","last_undegraded":"2024-09-21T11:48:33.027366+0000","last_fullsized":"2024-09-21T11:48:33.027366+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.269610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.269610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":367,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.041691+0000","last_change":"2024-09-21T11:45:04.770055+0000","last_active":"2024-09-21T11:48:36.041691+0000","last_peered":"2024-09-21T11:48:36.041691+0000","last_clean":"2024-09-21T11:48:36.041691+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:48:36.041691+0000","last_undegraded":"2024-09-21T11:48:36.041691+0000","last_fullsized":"2024-09-21T11:48:36.041691+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:04.769963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:04.769963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":88,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.027580+0000","last_change":"2024-09-21T11:47:03.404473+0000","last_active":"2024-09-21T11:48:33.027580+0000","last_peered":"2024-09-21T11:48:33.027580+0000","last_clean":"2024-09-21T11:48:33.027580+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:48:33.027580+0000","last_undegraded":"2024-09-21T11:48:33.027580+0000","last_fullsized":"2024-09-21T11:48:33.027580+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:26.649189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:26.649189+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":392,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.029024+0000","last_change":"2024-09-21T11:48:04.553772+0000","last_active":"2024-09-21T11:48:33.029024+0000","last_peered":"2024-09-21T11:48:33.029024+0000","last_clean":"2024-09-21T11:48:33.029024+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:48:33.029024+0000","last_undegraded":"2024-09-21T11:48:33.029024+0000","last_fullsized":"2024-09-21T11:48:33.029024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:04.553731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.553731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:21.248327+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":401,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.044093+0000","last_change":"2024-09-21T11:47:03.377948+0000","last_active":"2024-09-21T11:48:36.044093+0000","last_peered":"2024-09-21T11:48:36.044093+0000","last_clean":"2024-09-21T11:48:36.044093+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:48:36.044093+0000","last_undegraded":"2024-09-21T11:48:36.044093+0000","last_fullsized":"2024-09-21T11:48:36.044093+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:53.848323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:53.848323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":405,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.044350+0000","last_change":"2024-09-21T11:47:03.408821+0000","last_active":"2024-09-21T11:48:36.044350+0000","last_peered":"2024-09-21T11:48:36.044350+0000","last_clean":"2024-09-21T11:48:36.044350+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:48:36.044350+0000","last_undegraded":"2024-09-21T11:48:36.044350+0000","last_fullsized":"2024-09-21T11:48:36.044350+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:45:43.314803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:43.314803+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":402,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.041819+0000","last_change":"2024-09-21T11:45:46.706221+0000","last_active":"2024-09-21T11:48:36.041819+0000","last_peered":"2024-09-21T11:48:36.041819+0000","last_clean":"2024-09-21T11:48:36.041819+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:48:36.041819+0000","last_undegraded":"2024-09-21T11:48:36.041819+0000","last_fullsized":"2024-09-21T11:48:36.041819+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:45:46.706130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.706130+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":398,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.020706+0000","last_change":"2024-09-21T11:48:10.126645+0000","last_active":"2024-09-21T11:48:33.020706+0000","last_peered":"2024-09-21T11:48:33.020706+0000","last_clean":"2024-09-21T11:48:33.020706+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:48:33.020706+0000","last_undegraded":"2024-09-21T11:48:33.020706+0000","last_fullsized":"2024-09-21T11:48:33.020706+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.126591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.126591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:10.829490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":368,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.026266+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:48:33.026266+0000","last_peered":"2024-09-21T11:48:33.026266+0000","last_clean":"2024-09-21T11:48:33.026266+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:48:33.026266+0000","last_undegraded":"2024-09-21T11:48:33.026266+0000","last_fullsized":"2024-09-21T11:48:33.026266+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:43.608235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":375,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.044238+0000","last_change":"2024-09-21T11:45:45.730374+0000","last_active":"2024-09-21T11:48:36.044238+0000","last_peered":"2024-09-21T11:48:36.044238+0000","last_clean":"2024-09-21T11:48:36.044238+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:48:36.044238+0000","last_undegraded":"2024-09-21T11:48:36.044238+0000","last_fullsized":"2024-09-21T11:48:36.044238+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:45.730169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:45.730169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":436,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.045200+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:48:36.045200+0000","last_peered":"2024-09-21T11:48:36.045200+0000","last_clean":"2024-09-21T11:48:36.045200+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:48:36.045200+0000","last_undegraded":"2024-09-21T11:48:36.045200+0000","last_fullsized":"2024-09-21T11:48:36.045200+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":386,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.047145+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:48:36.047145+0000","last_peered":"2024-09-21T11:48:36.047145+0000","last_clean":"2024-09-21T11:48:36.047145+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:48:36.047145+0000","last_undegraded":"2024-09-21T11:48:36.047145+0000","last_fullsized":"2024-09-21T11:48:36.047145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.212674+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":384,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.005647+0000","last_change":"2024-09-21T11:48:09.104826+0000","last_active":"2024-09-21T11:48:33.005647+0000","last_peered":"2024-09-21T11:48:33.005647+0000","last_clean":"2024-09-21T11:48:33.005647+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:48:33.005647+0000","last_undegraded":"2024-09-21T11:48:33.005647+0000","last_fullsized":"2024-09-21T11:48:33.005647+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:09.104767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.104767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.005085+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":372,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.026908+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:48:33.026908+0000","last_peered":"2024-09-21T11:48:33.026908+0000","last_clean":"2024-09-21T11:48:33.026908+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:48:33.026908+0000","last_undegraded":"2024-09-21T11:48:33.026908+0000","last_fullsized":"2024-09-21T11:48:33.026908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:49.549791+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":78,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.022348+0000","last_change":"2024-09-21T11:47:03.374194+0000","last_active":"2024-09-21T11:48:33.022348+0000","last_peered":"2024-09-21T11:48:33.022348+0000","last_clean":"2024-09-21T11:48:33.022348+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:48:33.022348+0000","last_undegraded":"2024-09-21T11:48:33.022348+0000","last_fullsized":"2024-09-21T11:48:33.022348+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:27.297630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.297630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":401,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.045410+0000","last_change":"2024-09-21T11:47:03.377189+0000","last_active":"2024-09-21T11:48:36.045410+0000","last_peered":"2024-09-21T11:48:36.045410+0000","last_clean":"2024-09-21T11:48:36.045410+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:48:36.045410+0000","last_undegraded":"2024-09-21T11:48:36.045410+0000","last_fullsized":"2024-09-21T11:48:36.045410+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:11.587469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.587469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":454,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-21T11:48:33.005658+0000","last_change":"2024-09-21T11:47:03.392087+0000","last_active":"2024-09-21T11:48:33.005658+0000","last_peered":"2024-09-21T11:48:33.005658+0000","last_clean":"2024-09-21T11:48:33.005658+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:48:33.005658+0000","last_undegraded":"2024-09-21T11:48:33.005658+0000","last_fullsized":"2024-09-21T11:48:33.005658+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:16.675872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.675872+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:41.170726+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":387,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-21T11:48:36.048651+0000","last_change":"2024-09-21T11:45:12.377954+0000","last_active":"2024-09-21T11:48:36.048651+0000","last_peered":"2024-09-21T11:48:36.048651+0000","last_clean":"2024-09-21T11:48:36.048651+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:48:36.048651+0000","last_undegraded":"2024-09-21T11:48:36.048651+0000","last_fullsized":"2024-09-21T11:48:36.048651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:12.377880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:12.377880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":977535000,"num_objects":14917,"num_object_clones":0,"num_object_copies":44751,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259267,"num_write_kb":8773825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24511,"num_bytes_recovered":1606287360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3261607936,"data_stored":3261595720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85815,"ondisk_log_size":85815,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":396,"ondisk_log_size":396,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":507,"num_read_kb":502,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234092,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":645724,"kb_used_data":317452,"kb_used_omap":1,"kb_used_meta":328254,"kb_avail":93099428,"statfs":{"total":95995035648,"available":95333814272,"internally_reserved":0,"allocated":325070848,"data_stored":324713663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":336132554},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463025,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":611788,"kb_used_data":327932,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":93133364,"statfs":{"total":95995035648,"available":95368564736,"internally_reserved":0,"allocated":335802368,"data_stored":335461887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":290650566},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724662,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":812192,"kb_used_data":520016,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":92932960,"statfs":{"total":95995035648,"available":95163351040,"internally_reserved":0,"allocated":532496384,"data_stored":532134314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":299170246},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920892,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":600676,"kb_used_data":334736,"kb_used_omap":1,"kb_used_meta":265790,"kb_avail":93144476,"statfs":{"total":95995035648,"available":95379943424,"internally_reserved":0,"allocated":342769664,"data_stored":342408328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":272169413},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117122,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":713312,"kb_used_data":361356,"kb_used_omap":1,"kb_used_meta":351870,"kb_avail":93031840,"statfs":{"total":95995035648,"available":95264604160,"internally_reserved":0,"allocated":370028544,"data_stored":369677126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":360315335},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411464,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1084952,"kb_used_data":674760,"kb_used_omap":1,"kb_used_meta":410174,"kb_avail":92660200,"statfs":{"total":95995035648,"available":94884044800,"internally_reserved":0,"allocated":690954240,"data_stored":690600656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":420018629},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":726836,"kb_used_data":442980,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":93018316,"statfs":{"total":95995035648,"available":95250755584,"internally_reserved":0,"allocated":453611520,"data_stored":453229790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":290650566},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738517,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":679408,"kb_used_data":355296,"kb_used_omap":1,"kb_used_meta":324094,"kb_avail":93065744,"statfs":{"total":95995035648,"available":95299321856,"internally_reserved":0,"allocated":363823104,"data_stored":363453127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":331872709},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":356057088,"data_stored":356057088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":434503680,"data_stored":434503680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":676401152,"data_stored":676397080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":352256000,"data_stored":352256000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":318111744,"data_stored":318111744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":490278912,"data_stored":490274840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":315621376,"data_stored":315621376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":318377984,"data_stored":318373912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:48:41.107 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-21T11:48:41.107 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:48:41.107 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_1 2024-09-21T11:48:41.107 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_1 by 10 min 8 2024-09-21T11:48:41.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:48:41.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:41.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:41.162 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:41.262 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:41.290 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:41.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:41.291 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:41.345 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:48:41.391 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:41.416 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:41.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:41.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:41 smithi175 ceph-mon[32613]: pgmap v413: 148 pgs: 148 active+clean; 933 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 739 B/s rd, 0 B/s wr, 847 op/s; 31 B/s, 1 objects/s recovering 2024-09-21T11:48:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:41 smithi175 ceph-mon[32613]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:41 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/72719555' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:48:41.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:48:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:48:41.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:41.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:41.560 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:41.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:41.592 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:41 smithi043 ceph-mon[36454]: pgmap v413: 148 pgs: 148 active+clean; 933 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 739 B/s rd, 0 B/s wr, 847 op/s; 31 B/s, 1 objects/s recovering 2024-09-21T11:48:41.592 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:41 smithi043 ceph-mon[36454]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:41.592 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:41 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/72719555' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:48:41.592 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:41 smithi043 ceph-mon[29710]: pgmap v413: 148 pgs: 148 active+clean; 933 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 739 B/s rd, 0 B/s wr, 847 op/s; 31 B/s, 1 objects/s recovering 2024-09-21T11:48:41.592 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:41 smithi043 ceph-mon[29710]: from='client.15555 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:41.593 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:41 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/72719555' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:48:41.661 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:41.690 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:41.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:41.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:41.767 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:48:41.768 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:48:41.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:41.817 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:41.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:41.818 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:41.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:41.957 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:41.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:41.958 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:42.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:42.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:42.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:42.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:42.247 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:42.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:42.248 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:42.349 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:42.382 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:42.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:42.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:42.425 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:42.425 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:42.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/72719555' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:48:42.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[29710]: osdmap e239: 8 total, 8 up, 8 in 2024-09-21T11:48:42.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T11:48:42.426 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T11:48:42.426 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:42.426 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:42.426 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/72719555' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:48:42.426 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[36454]: osdmap e239: 8 total, 8 up, 8 in 2024-09-21T11:48:42.426 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T11:48:42.426 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:42 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T11:48:42.483 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:42.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:48:42.512 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:42.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:42.513 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:42.614 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:42.641 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:42.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:42.642 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:42.743 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:42 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/72719555' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:48:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:42 smithi175 ceph-mon[32613]: osdmap e239: 8 total, 8 up, 8 in 2024-09-21T11:48:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:42 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T11:48:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:42 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T11:48:42.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:42.773 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:42.774 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:42.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:42.908 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:42.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:42.909 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:43.010 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:43.032 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:43.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:43.033 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:43.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:43.179 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:43.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:43.180 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:43.209 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:43.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:43.350 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:43.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:43.351 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:43.451 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:43.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:43.481 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:43.481 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:43.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:43 smithi043 ceph-mon[36454]: pgmap v415: 148 pgs: 148 active+clean; 580 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.86k op/s 2024-09-21T11:48:43.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:43 smithi043 ceph-mon[36454]: 5.7 scrub starts 2024-09-21T11:48:43.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:43 smithi043 ceph-mon[36454]: 5.7 scrub ok 2024-09-21T11:48:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:43 smithi043 ceph-mon[29710]: pgmap v415: 148 pgs: 148 active+clean; 580 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.86k op/s 2024-09-21T11:48:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:43 smithi043 ceph-mon[29710]: 5.7 scrub starts 2024-09-21T11:48:43.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:43 smithi043 ceph-mon[29710]: 5.7 scrub ok 2024-09-21T11:48:43.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:43.620 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:43.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:43.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:43.657 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:43.660 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:43.723 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:43.748 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:43.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:43.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:43.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:43 smithi175 ceph-mon[32613]: pgmap v415: 148 pgs: 148 active+clean; 580 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.86k op/s 2024-09-21T11:48:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:43 smithi175 ceph-mon[32613]: 5.7 scrub starts 2024-09-21T11:48:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:43 smithi175 ceph-mon[32613]: 5.7 scrub ok 2024-09-21T11:48:43.850 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:43.879 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:43.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:43.880 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:43.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:44.028 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:44.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:44.029 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:44.130 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:44.168 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:44.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:44.169 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:44.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:44.297 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:44.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:44.298 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:44.398 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:44.423 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:44.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:44.424 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:44.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:44.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:44.569 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:44.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:44.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:44.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:44 smithi043 ceph-mon[36454]: 8.a scrub starts 2024-09-21T11:48:44.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:44 smithi043 ceph-mon[36454]: 8.a scrub ok 2024-09-21T11:48:44.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:44 smithi043 ceph-mon[36454]: 5.d scrub starts 2024-09-21T11:48:44.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:44 smithi043 ceph-mon[36454]: 5.d scrub ok 2024-09-21T11:48:44.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2234350220' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:44.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:44 smithi043 ceph-mon[29710]: 8.a scrub starts 2024-09-21T11:48:44.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:44 smithi043 ceph-mon[29710]: 8.a scrub ok 2024-09-21T11:48:44.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:44 smithi043 ceph-mon[29710]: 5.d scrub starts 2024-09-21T11:48:44.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:44 smithi043 ceph-mon[29710]: 5.d scrub ok 2024-09-21T11:48:44.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2234350220' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:44.723 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:44.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:44.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:44 smithi175 ceph-mon[32613]: 8.a scrub starts 2024-09-21T11:48:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:44 smithi175 ceph-mon[32613]: 8.a scrub ok 2024-09-21T11:48:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:44 smithi175 ceph-mon[32613]: 5.d scrub starts 2024-09-21T11:48:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:44 smithi175 ceph-mon[32613]: 5.d scrub ok 2024-09-21T11:48:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2234350220' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:44.802 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:48:44.802 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:48:44.802 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:48:44.802 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:48:44.803 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:48:44.803 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:48:44.803 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:48:44.803 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:48:44.803 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:48:44.803 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:48:44.804 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:48:44.804 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:48:44.804 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:48:44.804 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:48:44.804 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:48:44.804 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:48:44.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:44.838 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:48:44.842 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:48:44.863 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:44.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:44.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:44.965 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:44.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:44.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:44.994 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:45.095 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:45.135 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:45.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:45.136 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:45.237 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:45.246 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:48:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:48:45.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:48:45.266 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:45.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:45.267 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:45.368 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:45.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:45.419 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:45.419 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:45.520 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:45.536 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-mon[36454]: pgmap v416: 148 pgs: 148 active+clean; 580 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 648 B/s rd, 0 B/s wr, 992 op/s 2024-09-21T11:48:45.536 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-mon[36454]: 6.18 scrub starts 2024-09-21T11:48:45.536 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-mon[36454]: 6.18 scrub ok 2024-09-21T11:48:45.536 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-mon[36454]: 8.3 scrub starts 2024-09-21T11:48:45.536 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-mon[36454]: 8.3 scrub ok 2024-09-21T11:48:45.536 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-mon[29710]: pgmap v416: 148 pgs: 148 active+clean; 580 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 648 B/s rd, 0 B/s wr, 992 op/s 2024-09-21T11:48:45.536 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-mon[29710]: 6.18 scrub starts 2024-09-21T11:48:45.536 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-mon[29710]: 6.18 scrub ok 2024-09-21T11:48:45.536 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-mon[29710]: 8.3 scrub starts 2024-09-21T11:48:45.537 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:45 smithi043 ceph-mon[29710]: 8.3 scrub ok 2024-09-21T11:48:45.548 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:45.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:45.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:45.650 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:45.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:45.681 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:45.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:45 smithi175 ceph-mon[32613]: pgmap v416: 148 pgs: 148 active+clean; 580 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 648 B/s rd, 0 B/s wr, 992 op/s 2024-09-21T11:48:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:45 smithi175 ceph-mon[32613]: 6.18 scrub starts 2024-09-21T11:48:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:45 smithi175 ceph-mon[32613]: 6.18 scrub ok 2024-09-21T11:48:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:45 smithi175 ceph-mon[32613]: 8.3 scrub starts 2024-09-21T11:48:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:45 smithi175 ceph-mon[32613]: 8.3 scrub ok 2024-09-21T11:48:45.782 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:45.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:45.822 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:45.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:45.874 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:48:45.874 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:48:45.924 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:45.955 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:45.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:45.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:46.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:46.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:46.092 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:46.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:46.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:46.223 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:46.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:46.224 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:46.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:46.366 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:46.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:46.367 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:46.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:46.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[29710]: 5.10 scrub starts 2024-09-21T11:48:46.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[29710]: 5.10 scrub ok 2024-09-21T11:48:46.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[29710]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:46.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[29710]: osdmap e240: 8 total, 8 up, 8 in 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[36454]: 5.10 scrub starts 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[36454]: 5.10 scrub ok 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[36454]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:48:46.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:48:46.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:48:46.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[36454]: osdmap e240: 8 total, 8 up, 8 in 2024-09-21T11:48:46.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:46.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:46.510 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:46.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:46.646 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:46.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:46.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:46.748 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:46 smithi175 ceph-mon[32613]: 5.10 scrub starts 2024-09-21T11:48:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:46 smithi175 ceph-mon[32613]: 5.10 scrub ok 2024-09-21T11:48:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:46 smithi175 ceph-mon[32613]: from='client.15570 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:46 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:48:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:46 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:48:46.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:46 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:48:46.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:46 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:48:46.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:46 smithi175 ceph-mon[32613]: osdmap e240: 8 total, 8 up, 8 in 2024-09-21T11:48:46.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:48:46.777 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:46.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:46.779 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:46.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:46.921 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:46.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:46.922 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:47.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:47.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:47.064 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:47.064 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:47.127 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:48:47.165 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:47.197 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:47.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:47.198 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:47.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:47.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:47.337 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:47.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:47.438 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:47.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:47.478 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:47.478 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:47.515 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:47 smithi175 ceph-mon[32613]: pgmap v418: 148 pgs: 148 active+clean; 559 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 901 op/s 2024-09-21T11:48:47.529 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:48:47.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:47.626 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:47.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:47.627 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:47.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:47 smithi043 ceph-mon[29710]: pgmap v418: 148 pgs: 148 active+clean; 559 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 901 op/s 2024-09-21T11:48:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:47 smithi043 ceph-mon[36454]: pgmap v418: 148 pgs: 148 active+clean; 559 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 901 op/s 2024-09-21T11:48:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:47.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:47.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:47.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:47.901 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:47.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:47.902 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:47.925 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:47.951 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-21T11:48:43.930895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":608049715,"num_objects":9469,"num_object_clones":0,"num_object_copies":28407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9469,"num_whiteouts":0,"num_read":1349,"num_read_kb":1206,"num_write":265463,"num_write_kb":8775498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24762,"num_bytes_recovered":1606884224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86621,"ondisk_log_size":86621,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":447,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":4803536,"kb_used_data":2206260,"kb_used_omap":12,"kb_used_meta":2596915,"kb_avail":745157680,"statfs":{"total":767960285184,"available":763041464320,"internally_reserved":0,"allocated":2259210240,"data_stored":2256253044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2659241522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,12,9,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-513474560,"num_objects":-7835,"num_object_clones":0,"num_object_copies":-23505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7835,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7835,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.897434"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":391,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.366040+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:48:41.366040+0000","last_peered":"2024-09-21T11:48:41.366040+0000","last_clean":"2024-09-21T11:48:41.366040+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:48:41.366040+0000","last_undegraded":"2024-09-21T11:48:41.366040+0000","last_fullsized":"2024-09-21T11:48:41.366040+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.863735+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":443,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.287852+0000","last_change":"2024-09-21T11:47:03.433310+0000","last_active":"2024-09-21T11:48:39.287852+0000","last_peered":"2024-09-21T11:48:39.287852+0000","last_clean":"2024-09-21T11:48:39.287852+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:48:39.287852+0000","last_undegraded":"2024-09-21T11:48:39.287852+0000","last_fullsized":"2024-09-21T11:48:39.287852+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:44.553561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.553561+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.027789+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":82,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.296487+0000","last_change":"2024-09-21T11:47:03.372676+0000","last_active":"2024-09-21T11:48:39.296487+0000","last_peered":"2024-09-21T11:48:39.296487+0000","last_clean":"2024-09-21T11:48:39.296487+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:48:39.296487+0000","last_undegraded":"2024-09-21T11:48:39.296487+0000","last_fullsized":"2024-09-21T11:48:39.296487+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.453795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":399,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288383+0000","last_change":"2024-09-21T11:47:20.957722+0000","last_active":"2024-09-21T11:48:39.288383+0000","last_peered":"2024-09-21T11:48:39.288383+0000","last_clean":"2024-09-21T11:48:39.288383+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:48:39.288383+0000","last_undegraded":"2024-09-21T11:48:39.288383+0000","last_fullsized":"2024-09-21T11:48:39.288383+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:20.957679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.957679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":379,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.354464+0000","last_change":"2024-09-21T11:46:09.675727+0000","last_active":"2024-09-21T11:48:41.354464+0000","last_peered":"2024-09-21T11:48:41.354464+0000","last_clean":"2024-09-21T11:48:41.354464+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:48:41.354464+0000","last_undegraded":"2024-09-21T11:48:41.354464+0000","last_fullsized":"2024-09-21T11:48:41.354464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:09.675148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:09.675148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":384,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.364810+0000","last_change":"2024-09-21T11:46:55.804484+0000","last_active":"2024-09-21T11:48:41.364810+0000","last_peered":"2024-09-21T11:48:41.364810+0000","last_clean":"2024-09-21T11:48:41.364810+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:48:41.364810+0000","last_undegraded":"2024-09-21T11:48:41.364810+0000","last_fullsized":"2024-09-21T11:48:41.364810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.804419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.804419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":404,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.355077+0000","last_change":"2024-09-21T11:45:07.788590+0000","last_active":"2024-09-21T11:48:41.355077+0000","last_peered":"2024-09-21T11:48:41.355077+0000","last_clean":"2024-09-21T11:48:41.355077+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:48:41.355077+0000","last_undegraded":"2024-09-21T11:48:41.355077+0000","last_fullsized":"2024-09-21T11:48:41.355077+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:07.788512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.788512+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":403,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.287768+0000","last_change":"2024-09-21T11:47:03.371710+0000","last_active":"2024-09-21T11:48:39.287768+0000","last_peered":"2024-09-21T11:48:39.287768+0000","last_clean":"2024-09-21T11:48:39.287768+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:48:39.287768+0000","last_undegraded":"2024-09-21T11:48:39.287768+0000","last_fullsized":"2024-09-21T11:48:39.287768+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.957478+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":380,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.302327+0000","last_change":"2024-09-21T11:42:25.062239+0000","last_active":"2024-09-21T11:48:39.302327+0000","last_peered":"2024-09-21T11:48:39.302327+0000","last_clean":"2024-09-21T11:48:39.302327+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:48:39.302327+0000","last_undegraded":"2024-09-21T11:48:39.302327+0000","last_fullsized":"2024-09-21T11:48:39.302327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:25.062176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:25.062176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:43.925152+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":409,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288652+0000","last_change":"2024-09-21T11:48:13.962997+0000","last_active":"2024-09-21T11:48:39.288652+0000","last_peered":"2024-09-21T11:48:39.288652+0000","last_clean":"2024-09-21T11:48:39.288652+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:48:39.288652+0000","last_undegraded":"2024-09-21T11:48:39.288652+0000","last_fullsized":"2024-09-21T11:48:39.288652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:13.962919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:13.962919+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:14.418938+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":388,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.302387+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:48:39.302387+0000","last_peered":"2024-09-21T11:48:39.302387+0000","last_clean":"2024-09-21T11:48:39.302387+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:48:39.302387+0000","last_undegraded":"2024-09-21T11:48:39.302387+0000","last_fullsized":"2024-09-21T11:48:39.302387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.033067+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":392,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365156+0000","last_change":"2024-09-21T11:47:03.385157+0000","last_active":"2024-09-21T11:48:41.365156+0000","last_peered":"2024-09-21T11:48:41.365156+0000","last_clean":"2024-09-21T11:48:41.365156+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:48:41.365156+0000","last_undegraded":"2024-09-21T11:48:41.365156+0000","last_fullsized":"2024-09-21T11:48:41.365156+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:06.385125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:06.385125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":410,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301320+0000","last_change":"2024-09-21T11:47:03.385245+0000","last_active":"2024-09-21T11:48:39.301320+0000","last_peered":"2024-09-21T11:48:39.301320+0000","last_clean":"2024-09-21T11:48:39.301320+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:48:39.301320+0000","last_undegraded":"2024-09-21T11:48:39.301320+0000","last_fullsized":"2024-09-21T11:48:39.301320+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.017967+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":429,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288384+0000","last_change":"2024-09-21T11:48:13.005365+0000","last_active":"2024-09-21T11:48:39.288384+0000","last_peered":"2024-09-21T11:48:39.288384+0000","last_clean":"2024-09-21T11:48:39.288384+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:48:39.288384+0000","last_undegraded":"2024-09-21T11:48:39.288384+0000","last_fullsized":"2024-09-21T11:48:39.288384+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:48:13.005294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:13.005294+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:13.822018+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":394,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.305996+0000","last_change":"2024-09-21T11:48:06.605705+0000","last_active":"2024-09-21T11:48:39.305996+0000","last_peered":"2024-09-21T11:48:39.305996+0000","last_clean":"2024-09-21T11:48:39.305996+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:48:39.305996+0000","last_undegraded":"2024-09-21T11:48:39.305996+0000","last_fullsized":"2024-09-21T11:48:39.305996+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:06.605645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.605645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.086689+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":398,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301003+0000","last_change":"2024-09-21T11:47:03.380856+0000","last_active":"2024-09-21T11:48:39.301003+0000","last_peered":"2024-09-21T11:48:39.301003+0000","last_clean":"2024-09-21T11:48:39.301003+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:48:39.301003+0000","last_undegraded":"2024-09-21T11:48:39.301003+0000","last_fullsized":"2024-09-21T11:48:39.301003+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:13.288522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:13.288522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.041204+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":394,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.289329+0000","last_change":"2024-09-21T11:46:52.035524+0000","last_active":"2024-09-21T11:48:39.289329+0000","last_peered":"2024-09-21T11:48:39.289329+0000","last_clean":"2024-09-21T11:48:39.289329+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:48:39.289329+0000","last_undegraded":"2024-09-21T11:48:39.289329+0000","last_fullsized":"2024-09-21T11:48:39.289329+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.035461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.035461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:30.990978+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":382,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365807+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:48:41.365807+0000","last_peered":"2024-09-21T11:48:41.365807+0000","last_clean":"2024-09-21T11:48:41.365807+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:48:41.365807+0000","last_undegraded":"2024-09-21T11:48:41.365807+0000","last_fullsized":"2024-09-21T11:48:41.365807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.281211+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":497,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.348070+0000","last_change":"2024-09-21T11:48:18.120641+0000","last_active":"2024-09-21T11:48:41.348070+0000","last_peered":"2024-09-21T11:48:41.348070+0000","last_clean":"2024-09-21T11:48:41.348070+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:48:41.348070+0000","last_undegraded":"2024-09-21T11:48:41.348070+0000","last_fullsized":"2024-09-21T11:48:41.348070+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:48:18.120532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:18.120532+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:22.674543+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":82,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.296371+0000","last_change":"2024-09-21T11:47:03.377667+0000","last_active":"2024-09-21T11:48:39.296371+0000","last_peered":"2024-09-21T11:48:39.296371+0000","last_clean":"2024-09-21T11:48:39.296371+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:48:39.296371+0000","last_undegraded":"2024-09-21T11:48:39.296371+0000","last_fullsized":"2024-09-21T11:48:39.296371+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.718478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.718478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":380,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365201+0000","last_change":"2024-09-21T11:45:39.826902+0000","last_active":"2024-09-21T11:48:41.365201+0000","last_peered":"2024-09-21T11:48:41.365201+0000","last_clean":"2024-09-21T11:48:41.365201+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:48:41.365201+0000","last_undegraded":"2024-09-21T11:48:41.365201+0000","last_fullsized":"2024-09-21T11:48:41.365201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:39.826701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:39.826701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":403,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.289689+0000","last_change":"2024-09-21T11:48:22.941031+0000","last_active":"2024-09-21T11:48:39.289689+0000","last_peered":"2024-09-21T11:48:39.289689+0000","last_clean":"2024-09-21T11:48:39.289689+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:48:39.289689+0000","last_undegraded":"2024-09-21T11:48:39.289689+0000","last_fullsized":"2024-09-21T11:48:39.289689+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:22.940968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:22.940968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:48.005429+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":448,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.364398+0000","last_change":"2024-09-21T11:47:03.873307+0000","last_active":"2024-09-21T11:48:41.364398+0000","last_peered":"2024-09-21T11:48:41.364398+0000","last_clean":"2024-09-21T11:48:41.364398+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:48:41.364398+0000","last_undegraded":"2024-09-21T11:48:41.364398+0000","last_fullsized":"2024-09-21T11:48:41.364398+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:47:03.873239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.873239+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":381,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288203+0000","last_change":"2024-09-21T11:45:37.670176+0000","last_active":"2024-09-21T11:48:39.288203+0000","last_peered":"2024-09-21T11:48:39.288203+0000","last_clean":"2024-09-21T11:48:39.288203+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:48:39.288203+0000","last_undegraded":"2024-09-21T11:48:39.288203+0000","last_fullsized":"2024-09-21T11:48:39.288203+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:37.670117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:37.670117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:50.969760+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":393,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.305940+0000","last_change":"2024-09-21T11:46:47.611118+0000","last_active":"2024-09-21T11:48:39.305940+0000","last_peered":"2024-09-21T11:48:39.305940+0000","last_clean":"2024-09-21T11:48:39.305940+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:48:39.305940+0000","last_undegraded":"2024-09-21T11:48:39.305940+0000","last_fullsized":"2024-09-21T11:48:39.305940+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:46:47.611042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:47.611042+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.533011+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":86,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.297581+0000","last_change":"2024-09-21T11:47:03.388975+0000","last_active":"2024-09-21T11:48:39.297581+0000","last_peered":"2024-09-21T11:48:39.297581+0000","last_clean":"2024-09-21T11:48:39.297581+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:48:39.297581+0000","last_undegraded":"2024-09-21T11:48:39.297581+0000","last_fullsized":"2024-09-21T11:48:39.297581+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:12.636105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.636105+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":410,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.367285+0000","last_change":"2024-09-21T11:45:48.385970+0000","last_active":"2024-09-21T11:48:41.367285+0000","last_peered":"2024-09-21T11:48:41.367285+0000","last_clean":"2024-09-21T11:48:41.367285+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:48:41.367285+0000","last_undegraded":"2024-09-21T11:48:41.367285+0000","last_fullsized":"2024-09-21T11:48:41.367285+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:45:48.385895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:48.385895+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":521,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.290796+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:48:39.290796+0000","last_peered":"2024-09-21T11:48:39.290796+0000","last_clean":"2024-09-21T11:48:39.290796+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:48:39.290796+0000","last_undegraded":"2024-09-21T11:48:39.290796+0000","last_fullsized":"2024-09-21T11:48:39.290796+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.120507+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"238'16562","reported_seq":22316,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:40.917049+0000","last_change":"2024-09-21T11:48:17.532789+0000","last_active":"2024-09-21T11:48:40.917049+0000","last_peered":"2024-09-21T11:48:40.917049+0000","last_clean":"2024-09-21T11:48:40.917049+0000","last_became_active":"2024-09-21T11:48:17.482971+0000","last_became_peered":"2024-09-21T11:48:17.482971+0000","last_unstale":"2024-09-21T11:48:40.917049+0000","last_undegraded":"2024-09-21T11:48:40.917049+0000","last_fullsized":"2024-09-21T11:48:40.917049+0000","mapping_epoch":230,"log_start":"208'11645","ondisk_log_start":"208'11645","created":160,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":624,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16562,"num_write_kb":549952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2785,"num_bytes_recovered":182517760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":98,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.296113+0000","last_change":"2024-09-21T11:47:03.400465+0000","last_active":"2024-09-21T11:48:39.296113+0000","last_peered":"2024-09-21T11:48:39.296113+0000","last_clean":"2024-09-21T11:48:39.296113+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:48:39.296113+0000","last_undegraded":"2024-09-21T11:48:39.296113+0000","last_fullsized":"2024-09-21T11:48:39.296113+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:22.708839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.708839+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":414,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.361162+0000","last_change":"2024-09-21T11:47:03.391541+0000","last_active":"2024-09-21T11:48:41.361162+0000","last_peered":"2024-09-21T11:48:41.361162+0000","last_clean":"2024-09-21T11:48:41.361162+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:48:41.361162+0000","last_undegraded":"2024-09-21T11:48:41.361162+0000","last_fullsized":"2024-09-21T11:48:41.361162+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.836903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.836903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":494,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.289960+0000","last_change":"2024-09-21T11:47:01.933697+0000","last_active":"2024-09-21T11:48:39.289960+0000","last_peered":"2024-09-21T11:48:39.289960+0000","last_clean":"2024-09-21T11:48:39.289960+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:48:39.289960+0000","last_undegraded":"2024-09-21T11:48:39.289960+0000","last_fullsized":"2024-09-21T11:48:39.289960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:01.933648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:01.933648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":390,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301961+0000","last_change":"2024-09-21T11:48:15.013107+0000","last_active":"2024-09-21T11:48:39.301961+0000","last_peered":"2024-09-21T11:48:39.301961+0000","last_clean":"2024-09-21T11:48:39.301961+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:48:39.301961+0000","last_undegraded":"2024-09-21T11:48:39.301961+0000","last_fullsized":"2024-09-21T11:48:39.301961+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:15.013058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:15.013058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.125286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"239'16627","reported_seq":16881,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.673884+0000","last_change":"2024-09-21T11:46:19.360961+0000","last_active":"2024-09-21T11:48:41.673884+0000","last_peered":"2024-09-21T11:48:41.673884+0000","last_clean":"2024-09-21T11:48:41.673884+0000","last_became_active":"2024-09-21T11:44:51.744224+0000","last_became_peered":"2024-09-21T11:44:51.744224+0000","last_unstale":"2024-09-21T11:48:41.673884+0000","last_undegraded":"2024-09-21T11:48:41.673884+0000","last_fullsized":"2024-09-21T11:48:41.673884+0000","mapping_epoch":160,"log_start":"200'10800","ondisk_log_start":"200'10800","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7561","last_scrub_stamp":"2024-09-21T11:46:19.360902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.360902+0000","objects_scrubbed":7459,"log_size":5827,"log_dups_size":0,"ondisk_log_size":5827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34799616,"num_objects":531,"num_object_clones":0,"num_object_copies":1593,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16627,"num_write_kb":549056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":428,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.360907+0000","last_change":"2024-09-21T11:46:58.789719+0000","last_active":"2024-09-21T11:48:41.360907+0000","last_peered":"2024-09-21T11:48:41.360907+0000","last_clean":"2024-09-21T11:48:41.360907+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:48:41.360907+0000","last_undegraded":"2024-09-21T11:48:41.360907+0000","last_fullsized":"2024-09-21T11:48:41.360907+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:46:58.789625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.789625+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":400,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.370655+0000","last_change":"2024-09-21T11:46:52.891978+0000","last_active":"2024-09-21T11:48:41.370655+0000","last_peered":"2024-09-21T11:48:41.370655+0000","last_clean":"2024-09-21T11:48:41.370655+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:48:41.370655+0000","last_undegraded":"2024-09-21T11:48:41.370655+0000","last_fullsized":"2024-09-21T11:48:41.370655+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.891793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.891793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":382,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288128+0000","last_change":"2024-09-21T11:44:13.946705+0000","last_active":"2024-09-21T11:48:39.288128+0000","last_peered":"2024-09-21T11:48:39.288128+0000","last_clean":"2024-09-21T11:48:39.288128+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:48:39.288128+0000","last_undegraded":"2024-09-21T11:48:39.288128+0000","last_fullsized":"2024-09-21T11:48:39.288128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:42.015785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":378,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288569+0000","last_change":"2024-09-21T11:46:24.717355+0000","last_active":"2024-09-21T11:48:39.288569+0000","last_peered":"2024-09-21T11:48:39.288569+0000","last_clean":"2024-09-21T11:48:39.288569+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:48:39.288569+0000","last_undegraded":"2024-09-21T11:48:39.288569+0000","last_fullsized":"2024-09-21T11:48:39.288569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.717285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.717285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:58.045440+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"238'16501","reported_seq":16707,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:40.979958+0000","last_change":"2024-09-21T11:48:05.746078+0000","last_active":"2024-09-21T11:48:40.979958+0000","last_peered":"2024-09-21T11:48:40.979958+0000","last_clean":"2024-09-21T11:48:40.979958+0000","last_became_active":"2024-09-21T11:44:51.744633+0000","last_became_peered":"2024-09-21T11:44:51.744633+0000","last_unstale":"2024-09-21T11:48:40.979958+0000","last_undegraded":"2024-09-21T11:48:40.979958+0000","last_fullsized":"2024-09-21T11:48:40.979958+0000","mapping_epoch":160,"log_start":"205'11141","ondisk_log_start":"205'11141","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'14372","last_scrub_stamp":"2024-09-21T11:48:05.746017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.746017+0000","objects_scrubbed":2702,"log_size":5360,"log_dups_size":0,"ondisk_log_size":5360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:10.815328+0000","scrub_duration":670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36306968,"num_objects":555,"num_object_clones":0,"num_object_copies":1665,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":555,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16501,"num_write_kb":545729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":423,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288357+0000","last_change":"2024-09-21T11:47:33.018713+0000","last_active":"2024-09-21T11:48:39.288357+0000","last_peered":"2024-09-21T11:48:39.288357+0000","last_clean":"2024-09-21T11:48:39.288357+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:48:39.288357+0000","last_undegraded":"2024-09-21T11:48:39.288357+0000","last_fullsized":"2024-09-21T11:48:39.288357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:47:33.018639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:33.018639+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:41.906194+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":422,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.367551+0000","last_change":"2024-09-21T11:47:03.429013+0000","last_active":"2024-09-21T11:48:41.367551+0000","last_peered":"2024-09-21T11:48:41.367551+0000","last_clean":"2024-09-21T11:48:41.367551+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:48:41.367551+0000","last_undegraded":"2024-09-21T11:48:41.367551+0000","last_fullsized":"2024-09-21T11:48:41.367551+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:59.702813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.702813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":520,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.360719+0000","last_change":"2024-09-21T11:47:03.406087+0000","last_active":"2024-09-21T11:48:41.360719+0000","last_peered":"2024-09-21T11:48:41.360719+0000","last_clean":"2024-09-21T11:48:41.360719+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:48:41.360719+0000","last_undegraded":"2024-09-21T11:48:41.360719+0000","last_fullsized":"2024-09-21T11:48:41.360719+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:08.620519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.620519+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":380,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.306222+0000","last_change":"2024-09-21T11:48:10.667288+0000","last_active":"2024-09-21T11:48:39.306222+0000","last_peered":"2024-09-21T11:48:39.306222+0000","last_clean":"2024-09-21T11:48:39.306222+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:48:39.306222+0000","last_undegraded":"2024-09-21T11:48:39.306222+0000","last_fullsized":"2024-09-21T11:48:39.306222+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.667221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.667221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:11.606721+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"239'16848","reported_seq":17106,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.674076+0000","last_change":"2024-09-21T11:46:22.226583+0000","last_active":"2024-09-21T11:48:41.674076+0000","last_peered":"2024-09-21T11:48:41.674076+0000","last_clean":"2024-09-21T11:48:41.674076+0000","last_became_active":"2024-09-21T11:44:51.412662+0000","last_became_peered":"2024-09-21T11:44:51.412662+0000","last_unstale":"2024-09-21T11:48:41.674076+0000","last_undegraded":"2024-09-21T11:48:41.674076+0000","last_fullsized":"2024-09-21T11:48:41.674076+0000","mapping_epoch":160,"log_start":"200'11000","ondisk_log_start":"200'11000","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7855","last_scrub_stamp":"2024-09-21T11:46:22.226400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.226400+0000","objects_scrubbed":7719,"log_size":5848,"log_dups_size":0,"ondisk_log_size":5848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37617664,"num_objects":574,"num_object_clones":0,"num_object_copies":1722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16848,"num_write_kb":557504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":94,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.295309+0000","last_change":"2024-09-21T11:47:03.398638+0000","last_active":"2024-09-21T11:48:39.295309+0000","last_peered":"2024-09-21T11:48:39.295309+0000","last_clean":"2024-09-21T11:48:39.295309+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:48:39.295309+0000","last_undegraded":"2024-09-21T11:48:39.295309+0000","last_fullsized":"2024-09-21T11:48:39.295309+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:12.227121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.227121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":405,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.302403+0000","last_change":"2024-09-21T11:48:07.131578+0000","last_active":"2024-09-21T11:48:39.302403+0000","last_peered":"2024-09-21T11:48:39.302403+0000","last_clean":"2024-09-21T11:48:39.302403+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:48:39.302403+0000","last_undegraded":"2024-09-21T11:48:39.302403+0000","last_fullsized":"2024-09-21T11:48:39.302403+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.131535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.131535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:08.103888+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":491,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365294+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:48:41.365294+0000","last_peered":"2024-09-21T11:48:41.365294+0000","last_clean":"2024-09-21T11:48:41.365294+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:48:41.365294+0000","last_undegraded":"2024-09-21T11:48:41.365294+0000","last_fullsized":"2024-09-21T11:48:41.365294+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.263748+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":379,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365286+0000","last_change":"2024-09-21T11:46:28.010676+0000","last_active":"2024-09-21T11:48:41.365286+0000","last_peered":"2024-09-21T11:48:41.365286+0000","last_clean":"2024-09-21T11:48:41.365286+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:48:41.365286+0000","last_undegraded":"2024-09-21T11:48:41.365286+0000","last_fullsized":"2024-09-21T11:48:41.365286+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:28.010602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.010602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"239'16498","reported_seq":25649,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.752267+0000","last_change":"2024-09-21T11:47:54.309078+0000","last_active":"2024-09-21T11:48:41.752267+0000","last_peered":"2024-09-21T11:48:41.752267+0000","last_clean":"2024-09-21T11:48:41.752267+0000","last_became_active":"2024-09-21T11:47:54.270690+0000","last_became_peered":"2024-09-21T11:47:54.270690+0000","last_unstale":"2024-09-21T11:48:41.752267+0000","last_undegraded":"2024-09-21T11:48:41.752267+0000","last_fullsized":"2024-09-21T11:48:41.752267+0000","mapping_epoch":222,"log_start":"207'11400","ondisk_log_start":"207'11400","created":160,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'7360","last_scrub_stamp":"2024-09-21T11:46:17.038356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.038356+0000","objects_scrubbed":7242,"log_size":5098,"log_dups_size":0,"ondisk_log_size":5098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36044800,"num_objects":550,"num_object_clones":0,"num_object_copies":1650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16498,"num_write_kb":545536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4441,"num_bytes_recovered":291045376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":423,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365505+0000","last_change":"2024-09-21T11:47:03.750064+0000","last_active":"2024-09-21T11:48:41.365505+0000","last_peered":"2024-09-21T11:48:41.365505+0000","last_clean":"2024-09-21T11:48:41.365505+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:48:41.365505+0000","last_undegraded":"2024-09-21T11:48:41.365505+0000","last_fullsized":"2024-09-21T11:48:41.365505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:47:03.749962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.749962+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":82,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.295962+0000","last_change":"2024-09-21T11:47:03.382675+0000","last_active":"2024-09-21T11:48:39.295962+0000","last_peered":"2024-09-21T11:48:39.295962+0000","last_clean":"2024-09-21T11:48:39.295962+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:48:39.295962+0000","last_undegraded":"2024-09-21T11:48:39.295962+0000","last_fullsized":"2024-09-21T11:48:39.295962+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:08.723593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.723593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":381,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.290303+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:48:39.290303+0000","last_peered":"2024-09-21T11:48:39.290303+0000","last_clean":"2024-09-21T11:48:39.290303+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:48:39.290303+0000","last_undegraded":"2024-09-21T11:48:39.290303+0000","last_fullsized":"2024-09-21T11:48:39.290303+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.032409+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":82,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.296409+0000","last_change":"2024-09-21T11:47:03.379157+0000","last_active":"2024-09-21T11:48:39.296409+0000","last_peered":"2024-09-21T11:48:39.296409+0000","last_clean":"2024-09-21T11:48:39.296409+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:48:39.296409+0000","last_undegraded":"2024-09-21T11:48:39.296409+0000","last_fullsized":"2024-09-21T11:48:39.296409+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.475150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"238'16651","reported_seq":16820,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:40.985011+0000","last_change":"2024-09-21T11:44:51.745506+0000","last_active":"2024-09-21T11:48:40.985011+0000","last_peered":"2024-09-21T11:48:40.985011+0000","last_clean":"2024-09-21T11:48:40.985011+0000","last_became_active":"2024-09-21T11:44:51.745200+0000","last_became_peered":"2024-09-21T11:44:51.745200+0000","last_unstale":"2024-09-21T11:48:40.985011+0000","last_undegraded":"2024-09-21T11:48:40.985011+0000","last_fullsized":"2024-09-21T11:48:40.985011+0000","mapping_epoch":160,"log_start":"204'11200","ondisk_log_start":"204'11200","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5451,"log_dups_size":0,"ondisk_log_size":5451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39649280,"num_objects":605,"num_object_clones":0,"num_object_copies":1815,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16651,"num_write_kb":552192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"205'31","reported_seq":124,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.295909+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:48:39.295909+0000","last_peered":"2024-09-21T11:48:39.295909+0000","last_clean":"2024-09-21T11:48:39.295909+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:48:39.295909+0000","last_undegraded":"2024-09-21T11:48:39.295909+0000","last_fullsized":"2024-09-21T11:48:39.295909+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.477411+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":388,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.289570+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:48:39.289570+0000","last_peered":"2024-09-21T11:48:39.289570+0000","last_clean":"2024-09-21T11:48:39.289570+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:48:39.289570+0000","last_undegraded":"2024-09-21T11:48:39.289570+0000","last_fullsized":"2024-09-21T11:48:39.289570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:39.997756+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":421,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301835+0000","last_change":"2024-09-21T11:47:20.000195+0000","last_active":"2024-09-21T11:48:39.301835+0000","last_peered":"2024-09-21T11:48:39.301835+0000","last_clean":"2024-09-21T11:48:39.301835+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:48:39.301835+0000","last_undegraded":"2024-09-21T11:48:39.301835+0000","last_fullsized":"2024-09-21T11:48:39.301835+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:47:20.000051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.000051+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:48.196772+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":397,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365667+0000","last_change":"2024-09-21T11:47:03.385173+0000","last_active":"2024-09-21T11:48:41.365667+0000","last_peered":"2024-09-21T11:48:41.365667+0000","last_clean":"2024-09-21T11:48:41.365667+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:48:41.365667+0000","last_undegraded":"2024-09-21T11:48:41.365667+0000","last_fullsized":"2024-09-21T11:48:41.365667+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:32.321908+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":371,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.354724+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:48:41.354724+0000","last_peered":"2024-09-21T11:48:41.354724+0000","last_clean":"2024-09-21T11:48:41.354724+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:48:41.354724+0000","last_undegraded":"2024-09-21T11:48:41.354724+0000","last_fullsized":"2024-09-21T11:48:41.354724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.188178+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"239'16448","reported_seq":20966,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.776899+0000","last_change":"2024-09-21T11:48:24.579575+0000","last_active":"2024-09-21T11:48:41.776899+0000","last_peered":"2024-09-21T11:48:41.776899+0000","last_clean":"2024-09-21T11:48:41.776899+0000","last_became_active":"2024-09-21T11:48:24.535114+0000","last_became_peered":"2024-09-21T11:48:24.535114+0000","last_unstale":"2024-09-21T11:48:41.776899+0000","last_undegraded":"2024-09-21T11:48:41.776899+0000","last_fullsized":"2024-09-21T11:48:41.776899+0000","mapping_epoch":233,"log_start":"208'11400","ondisk_log_start":"208'11400","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5048,"log_dups_size":0,"ondisk_log_size":5048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34734080,"num_objects":530,"num_object_clones":0,"num_object_copies":1590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16448,"num_write_kb":543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":141754368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":30,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.349836+0000","last_change":"2024-09-21T11:48:33.075081+0000","last_active":"2024-09-21T11:48:41.349836+0000","last_peered":"2024-09-21T11:48:41.349836+0000","last_clean":"2024-09-21T11:48:41.349836+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:48:41.349836+0000","last_undegraded":"2024-09-21T11:48:41.349836+0000","last_fullsized":"2024-09-21T11:48:41.349836+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.876172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":86,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.296231+0000","last_change":"2024-09-21T11:47:03.391283+0000","last_active":"2024-09-21T11:48:39.296231+0000","last_peered":"2024-09-21T11:48:39.296231+0000","last_clean":"2024-09-21T11:48:39.296231+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:48:39.296231+0000","last_undegraded":"2024-09-21T11:48:39.296231+0000","last_fullsized":"2024-09-21T11:48:39.296231+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.552518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":409,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.348330+0000","last_change":"2024-09-21T11:46:54.864288+0000","last_active":"2024-09-21T11:48:41.348330+0000","last_peered":"2024-09-21T11:48:41.348330+0000","last_clean":"2024-09-21T11:48:41.348330+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:48:41.348330+0000","last_undegraded":"2024-09-21T11:48:41.348330+0000","last_fullsized":"2024-09-21T11:48:41.348330+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:54.864074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.864074+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":391,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.351749+0000","last_change":"2024-09-21T11:48:16.076049+0000","last_active":"2024-09-21T11:48:41.351749+0000","last_peered":"2024-09-21T11:48:41.351749+0000","last_clean":"2024-09-21T11:48:41.351749+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:48:41.351749+0000","last_undegraded":"2024-09-21T11:48:41.351749+0000","last_fullsized":"2024-09-21T11:48:41.351749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:16.076003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:16.076003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.071160+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":385,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.309769+0000","last_change":"2024-09-21T11:48:07.613756+0000","last_active":"2024-09-21T11:48:39.309769+0000","last_peered":"2024-09-21T11:48:39.309769+0000","last_clean":"2024-09-21T11:48:39.309769+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:48:39.309769+0000","last_undegraded":"2024-09-21T11:48:39.309769+0000","last_fullsized":"2024-09-21T11:48:39.309769+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.613691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.613691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:32.231718+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"238'16289","reported_seq":25738,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:40.885235+0000","last_change":"2024-09-21T11:47:52.322057+0000","last_active":"2024-09-21T11:48:40.885235+0000","last_peered":"2024-09-21T11:48:40.885235+0000","last_clean":"2024-09-21T11:48:40.885235+0000","last_became_active":"2024-09-21T11:47:52.264063+0000","last_became_peered":"2024-09-21T11:47:52.264063+0000","last_unstale":"2024-09-21T11:48:40.885235+0000","last_undegraded":"2024-09-21T11:48:40.885235+0000","last_fullsized":"2024-09-21T11:48:40.885235+0000","mapping_epoch":220,"log_start":"208'11406","ondisk_log_start":"208'11406","created":160,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":4883,"log_dups_size":0,"ondisk_log_size":4883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38862848,"num_objects":593,"num_object_clones":0,"num_object_copies":1779,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16289,"num_write_kb":540224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4633,"num_bytes_recovered":303628288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":447,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.367403+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:48:41.367403+0000","last_peered":"2024-09-21T11:48:41.367403+0000","last_clean":"2024-09-21T11:48:41.367403+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:48:41.367403+0000","last_undegraded":"2024-09-21T11:48:41.367403+0000","last_fullsized":"2024-09-21T11:48:41.367403+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.401660+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":399,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365876+0000","last_change":"2024-09-21T11:46:29.937866+0000","last_active":"2024-09-21T11:48:41.365876+0000","last_peered":"2024-09-21T11:48:41.365876+0000","last_clean":"2024-09-21T11:48:41.365876+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:48:41.365876+0000","last_undegraded":"2024-09-21T11:48:41.365876+0000","last_fullsized":"2024-09-21T11:48:41.365876+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.937799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.937799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":952,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.490804+0000","last_change":"2024-09-21T11:48:17.596682+0000","last_active":"2024-09-21T11:48:39.490804+0000","last_peered":"2024-09-21T11:48:39.490804+0000","last_clean":"2024-09-21T11:48:39.490804+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:48:39.490804+0000","last_undegraded":"2024-09-21T11:48:39.490804+0000","last_fullsized":"2024-09-21T11:48:39.490804+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:48:17.596571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.596571+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.625607+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":507,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":394,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301816+0000","last_change":"2024-09-21T11:47:22.025919+0000","last_active":"2024-09-21T11:48:39.301816+0000","last_peered":"2024-09-21T11:48:39.301816+0000","last_clean":"2024-09-21T11:48:39.301816+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:48:39.301816+0000","last_undegraded":"2024-09-21T11:48:39.301816+0000","last_fullsized":"2024-09-21T11:48:39.301816+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:22.025842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.025842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":82,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.295774+0000","last_change":"2024-09-21T11:47:03.374507+0000","last_active":"2024-09-21T11:48:39.295774+0000","last_peered":"2024-09-21T11:48:39.295774+0000","last_clean":"2024-09-21T11:48:39.295774+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:48:39.295774+0000","last_undegraded":"2024-09-21T11:48:39.295774+0000","last_fullsized":"2024-09-21T11:48:39.295774+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"238'16406","reported_seq":16708,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:40.962144+0000","last_change":"2024-09-21T11:47:31.508923+0000","last_active":"2024-09-21T11:48:40.962144+0000","last_peered":"2024-09-21T11:48:40.962144+0000","last_clean":"2024-09-21T11:48:40.962144+0000","last_became_active":"2024-09-21T11:44:51.410701+0000","last_became_peered":"2024-09-21T11:44:51.410701+0000","last_unstale":"2024-09-21T11:48:40.962144+0000","last_undegraded":"2024-09-21T11:48:40.962144+0000","last_fullsized":"2024-09-21T11:48:40.962144+0000","mapping_epoch":160,"log_start":"198'10000","ondisk_log_start":"198'10000","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'12576","last_scrub_stamp":"2024-09-21T11:47:31.508822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:47:31.508822+0000","objects_scrubbed":4421,"log_size":6406,"log_dups_size":0,"ondisk_log_size":6406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.146813+0000","scrub_duration":1214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36962304,"num_objects":564,"num_object_clones":0,"num_object_copies":1692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16406,"num_write_kb":543040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":512,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.290644+0000","last_change":"2024-09-21T11:48:06.171363+0000","last_active":"2024-09-21T11:48:39.290644+0000","last_peered":"2024-09-21T11:48:39.290644+0000","last_clean":"2024-09-21T11:48:39.290644+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:48:39.290644+0000","last_undegraded":"2024-09-21T11:48:39.290644+0000","last_fullsized":"2024-09-21T11:48:39.290644+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:48:06.171279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.171279+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:12.520316+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":400,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.354536+0000","last_change":"2024-09-21T11:46:57.833589+0000","last_active":"2024-09-21T11:48:41.354536+0000","last_peered":"2024-09-21T11:48:41.354536+0000","last_clean":"2024-09-21T11:48:41.354536+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:48:41.354536+0000","last_undegraded":"2024-09-21T11:48:41.354536+0000","last_fullsized":"2024-09-21T11:48:41.354536+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:57.833329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.833329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":451,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.305976+0000","last_change":"2024-09-21T11:46:58.645189+0000","last_active":"2024-09-21T11:48:39.305976+0000","last_peered":"2024-09-21T11:48:39.305976+0000","last_clean":"2024-09-21T11:48:39.305976+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:48:39.305976+0000","last_undegraded":"2024-09-21T11:48:39.305976+0000","last_fullsized":"2024-09-21T11:48:39.305976+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:46:58.645122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.645122+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.573944+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":389,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.307111+0000","last_change":"2024-09-21T11:47:25.395237+0000","last_active":"2024-09-21T11:48:39.307111+0000","last_peered":"2024-09-21T11:48:39.307111+0000","last_clean":"2024-09-21T11:48:39.307111+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:48:39.307111+0000","last_undegraded":"2024-09-21T11:48:39.307111+0000","last_fullsized":"2024-09-21T11:48:39.307111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.395185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.395185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.204032+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":384,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301671+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:48:39.301671+0000","last_peered":"2024-09-21T11:48:39.301671+0000","last_clean":"2024-09-21T11:48:39.301671+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:48:39.301671+0000","last_undegraded":"2024-09-21T11:48:39.301671+0000","last_fullsized":"2024-09-21T11:48:39.301671+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:30.020245+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"238'16383","reported_seq":16666,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:40.872191+0000","last_change":"2024-09-21T11:48:09.830854+0000","last_active":"2024-09-21T11:48:40.872191+0000","last_peered":"2024-09-21T11:48:40.872191+0000","last_clean":"2024-09-21T11:48:40.872191+0000","last_became_active":"2024-09-21T11:44:51.410197+0000","last_became_peered":"2024-09-21T11:44:51.410197+0000","last_unstale":"2024-09-21T11:48:40.872191+0000","last_undegraded":"2024-09-21T11:48:40.872191+0000","last_fullsized":"2024-09-21T11:48:40.872191+0000","mapping_epoch":160,"log_start":"208'11600","ondisk_log_start":"208'11600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"226'14544","last_scrub_stamp":"2024-09-21T11:48:09.830774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.830774+0000","objects_scrubbed":2470,"log_size":4783,"log_dups_size":0,"ondisk_log_size":4783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.225752+0000","scrub_duration":808,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40304640,"num_objects":615,"num_object_clones":0,"num_object_copies":1845,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16383,"num_write_kb":543936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":442,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.289408+0000","last_change":"2024-09-21T11:48:05.072998+0000","last_active":"2024-09-21T11:48:39.289408+0000","last_peered":"2024-09-21T11:48:39.289408+0000","last_clean":"2024-09-21T11:48:39.289408+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:48:39.289408+0000","last_undegraded":"2024-09-21T11:48:39.289408+0000","last_fullsized":"2024-09-21T11:48:39.289408+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:05.072910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.072910+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:33.824160+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":406,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288512+0000","last_change":"2024-09-21T11:47:00.950268+0000","last_active":"2024-09-21T11:48:39.288512+0000","last_peered":"2024-09-21T11:48:39.288512+0000","last_clean":"2024-09-21T11:48:39.288512+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:48:39.288512+0000","last_undegraded":"2024-09-21T11:48:39.288512+0000","last_fullsized":"2024-09-21T11:48:39.288512+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:00.950208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:00.950208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":388,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301578+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:48:39.301578+0000","last_peered":"2024-09-21T11:48:39.301578+0000","last_clean":"2024-09-21T11:48:39.301578+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:48:39.301578+0000","last_undegraded":"2024-09-21T11:48:39.301578+0000","last_fullsized":"2024-09-21T11:48:39.301578+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.795578+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":386,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.355956+0000","last_change":"2024-09-21T11:48:23.285482+0000","last_active":"2024-09-21T11:48:41.355956+0000","last_peered":"2024-09-21T11:48:41.355956+0000","last_clean":"2024-09-21T11:48:41.355956+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:48:41.355956+0000","last_undegraded":"2024-09-21T11:48:41.355956+0000","last_fullsized":"2024-09-21T11:48:41.355956+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.285283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.285283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:52.473277+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"238'16770","reported_seq":23193,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:40.954077+0000","last_change":"2024-09-21T11:48:15.522333+0000","last_active":"2024-09-21T11:48:40.954077+0000","last_peered":"2024-09-21T11:48:40.954077+0000","last_clean":"2024-09-21T11:48:40.954077+0000","last_became_active":"2024-09-21T11:48:15.471002+0000","last_became_peered":"2024-09-21T11:48:15.471002+0000","last_unstale":"2024-09-21T11:48:40.954077+0000","last_undegraded":"2024-09-21T11:48:40.954077+0000","last_fullsized":"2024-09-21T11:48:40.954077+0000","mapping_epoch":228,"log_start":"208'11913","ondisk_log_start":"208'11913","created":160,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'10449","last_scrub_stamp":"2024-09-21T11:46:52.316998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.316998+0000","objects_scrubbed":7011,"log_size":4857,"log_dups_size":0,"ondisk_log_size":4857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38404096,"num_objects":586,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16770,"num_write_kb":555392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3078,"num_bytes_recovered":201719808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":496,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.289069+0000","last_change":"2024-09-21T11:48:33.035096+0000","last_active":"2024-09-21T11:48:39.289069+0000","last_peered":"2024-09-21T11:48:39.289069+0000","last_clean":"2024-09-21T11:48:39.289069+0000","last_became_active":"2024-09-21T11:48:33.022321+0000","last_became_peered":"2024-09-21T11:48:33.022321+0000","last_unstale":"2024-09-21T11:48:39.289069+0000","last_undegraded":"2024-09-21T11:48:39.289069+0000","last_fullsized":"2024-09-21T11:48:39.289069+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:47:22.974761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.974761+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:40.979174+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":419,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.290554+0000","last_change":"2024-09-21T11:47:03.365071+0000","last_active":"2024-09-21T11:48:39.290554+0000","last_peered":"2024-09-21T11:48:39.290554+0000","last_clean":"2024-09-21T11:48:39.290554+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:48:39.290554+0000","last_undegraded":"2024-09-21T11:48:39.290554+0000","last_fullsized":"2024-09-21T11:48:39.290554+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.692429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.692429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:54.181079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":82,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.295686+0000","last_change":"2024-09-21T11:47:03.375108+0000","last_active":"2024-09-21T11:48:39.295686+0000","last_peered":"2024-09-21T11:48:39.295686+0000","last_clean":"2024-09-21T11:48:39.295686+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:48:39.295686+0000","last_undegraded":"2024-09-21T11:48:39.295686+0000","last_fullsized":"2024-09-21T11:48:39.295686+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:50.008635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:50.008635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":395,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.350264+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:48:41.350264+0000","last_peered":"2024-09-21T11:48:41.350264+0000","last_clean":"2024-09-21T11:48:41.350264+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:48:41.350264+0000","last_undegraded":"2024-09-21T11:48:41.350264+0000","last_fullsized":"2024-09-21T11:48:41.350264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:37.893541+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"239'16440","reported_seq":25568,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.736878+0000","last_change":"2024-09-21T11:47:53.304799+0000","last_active":"2024-09-21T11:48:41.736878+0000","last_peered":"2024-09-21T11:48:41.736878+0000","last_clean":"2024-09-21T11:48:41.736878+0000","last_became_active":"2024-09-21T11:47:53.264856+0000","last_became_peered":"2024-09-21T11:47:53.264856+0000","last_unstale":"2024-09-21T11:48:41.736878+0000","last_undegraded":"2024-09-21T11:48:41.736878+0000","last_fullsized":"2024-09-21T11:48:41.736878+0000","mapping_epoch":221,"log_start":"207'11400","ondisk_log_start":"207'11400","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5040,"log_dups_size":0,"ondisk_log_size":5040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.133366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32505856,"num_objects":496,"num_object_clones":0,"num_object_copies":1488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16440,"num_write_kb":541952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4468,"num_bytes_recovered":292749312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":416,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288224+0000","last_change":"2024-09-21T11:46:11.730053+0000","last_active":"2024-09-21T11:48:39.288224+0000","last_peered":"2024-09-21T11:48:39.288224+0000","last_clean":"2024-09-21T11:48:39.288224+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:48:39.288224+0000","last_undegraded":"2024-09-21T11:48:39.288224+0000","last_fullsized":"2024-09-21T11:48:39.288224+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:11.729984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.729984+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.980319+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":383,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.355000+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:48:41.355000+0000","last_peered":"2024-09-21T11:48:41.355000+0000","last_clean":"2024-09-21T11:48:41.355000+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:48:41.355000+0000","last_undegraded":"2024-09-21T11:48:41.355000+0000","last_fullsized":"2024-09-21T11:48:41.355000+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:27.332154+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":504,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.347648+0000","last_change":"2024-09-21T11:46:52.834287+0000","last_active":"2024-09-21T11:48:41.347648+0000","last_peered":"2024-09-21T11:48:41.347648+0000","last_clean":"2024-09-21T11:48:41.347648+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:48:41.347648+0000","last_undegraded":"2024-09-21T11:48:41.347648+0000","last_fullsized":"2024-09-21T11:48:41.347648+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:52.834206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.834206+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":391,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.289251+0000","last_change":"2024-09-21T11:48:07.992586+0000","last_active":"2024-09-21T11:48:39.289251+0000","last_peered":"2024-09-21T11:48:39.289251+0000","last_clean":"2024-09-21T11:48:39.289251+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:48:39.289251+0000","last_undegraded":"2024-09-21T11:48:39.289251+0000","last_fullsized":"2024-09-21T11:48:39.289251+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.992517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.992517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.884349+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"238'16572","reported_seq":22724,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:40.917000+0000","last_change":"2024-09-21T11:48:16.538801+0000","last_active":"2024-09-21T11:48:40.917000+0000","last_peered":"2024-09-21T11:48:40.917000+0000","last_clean":"2024-09-21T11:48:40.917000+0000","last_became_active":"2024-09-21T11:48:16.476754+0000","last_became_peered":"2024-09-21T11:48:16.476754+0000","last_unstale":"2024-09-21T11:48:40.917000+0000","last_undegraded":"2024-09-21T11:48:40.917000+0000","last_fullsized":"2024-09-21T11:48:40.917000+0000","mapping_epoch":229,"log_start":"208'11687","ondisk_log_start":"208'11687","created":160,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7408","last_scrub_stamp":"2024-09-21T11:46:17.934135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.934135+0000","objects_scrubbed":7290,"log_size":4885,"log_dups_size":0,"ondisk_log_size":4885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":624,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16572,"num_write_kb":550272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2943,"num_bytes_recovered":192872448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":436,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365400+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:48:41.365400+0000","last_peered":"2024-09-21T11:48:41.365400+0000","last_clean":"2024-09-21T11:48:41.365400+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:48:41.365400+0000","last_undegraded":"2024-09-21T11:48:41.365400+0000","last_fullsized":"2024-09-21T11:48:41.365400+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:41.378708+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":396,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301399+0000","last_change":"2024-09-21T11:48:08.172717+0000","last_active":"2024-09-21T11:48:39.301399+0000","last_peered":"2024-09-21T11:48:39.301399+0000","last_clean":"2024-09-21T11:48:39.301399+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:48:39.301399+0000","last_undegraded":"2024-09-21T11:48:39.301399+0000","last_fullsized":"2024-09-21T11:48:39.301399+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:08.172656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:08.172656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:18.830907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":383,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.347390+0000","last_change":"2024-09-21T11:46:55.879988+0000","last_active":"2024-09-21T11:48:41.347390+0000","last_peered":"2024-09-21T11:48:41.347390+0000","last_clean":"2024-09-21T11:48:41.347390+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:48:41.347390+0000","last_undegraded":"2024-09-21T11:48:41.347390+0000","last_fullsized":"2024-09-21T11:48:41.347390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.879761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.879761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":370,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365651+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:48:41.365651+0000","last_peered":"2024-09-21T11:48:41.365651+0000","last_clean":"2024-09-21T11:48:41.365651+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:48:41.365651+0000","last_undegraded":"2024-09-21T11:48:41.365651+0000","last_fullsized":"2024-09-21T11:48:41.365651+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"239'16702","reported_seq":16954,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.669225+0000","last_change":"2024-09-21T11:46:15.665868+0000","last_active":"2024-09-21T11:48:41.669225+0000","last_peered":"2024-09-21T11:48:41.669225+0000","last_clean":"2024-09-21T11:48:41.669225+0000","last_became_active":"2024-09-21T11:44:51.744226+0000","last_became_peered":"2024-09-21T11:44:51.744226+0000","last_unstale":"2024-09-21T11:48:41.669225+0000","last_undegraded":"2024-09-21T11:48:41.669225+0000","last_fullsized":"2024-09-21T11:48:41.669225+0000","mapping_epoch":160,"log_start":"200'10900","ondisk_log_start":"200'10900","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7351","last_scrub_stamp":"2024-09-21T11:46:15.665775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:15.665775+0000","objects_scrubbed":7260,"log_size":5802,"log_dups_size":0,"ondisk_log_size":5802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":580,"num_object_clones":0,"num_object_copies":1740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16702,"num_write_kb":553024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":477,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.290333+0000","last_change":"2024-09-21T11:47:03.397050+0000","last_active":"2024-09-21T11:48:39.290333+0000","last_peered":"2024-09-21T11:48:39.290333+0000","last_clean":"2024-09-21T11:48:39.290333+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:48:39.290333+0000","last_undegraded":"2024-09-21T11:48:39.290333+0000","last_fullsized":"2024-09-21T11:48:39.290333+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:46:20.650403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.650403+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:51.167956+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":396,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.309978+0000","last_change":"2024-09-21T11:48:03.546909+0000","last_active":"2024-09-21T11:48:39.309978+0000","last_peered":"2024-09-21T11:48:39.309978+0000","last_clean":"2024-09-21T11:48:39.309978+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:48:39.309978+0000","last_undegraded":"2024-09-21T11:48:39.309978+0000","last_fullsized":"2024-09-21T11:48:39.309978+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:03.546843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:03.546843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.594742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":399,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.346756+0000","last_change":"2024-09-21T11:46:16.775013+0000","last_active":"2024-09-21T11:48:41.346756+0000","last_peered":"2024-09-21T11:48:41.346756+0000","last_clean":"2024-09-21T11:48:41.346756+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:48:41.346756+0000","last_undegraded":"2024-09-21T11:48:41.346756+0000","last_fullsized":"2024-09-21T11:48:41.346756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:16.774939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.774939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.931030+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":384,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.305884+0000","last_change":"2024-09-21T11:46:59.704981+0000","last_active":"2024-09-21T11:48:39.305884+0000","last_peered":"2024-09-21T11:48:39.305884+0000","last_clean":"2024-09-21T11:48:39.305884+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:48:39.305884+0000","last_undegraded":"2024-09-21T11:48:39.305884+0000","last_fullsized":"2024-09-21T11:48:39.305884+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.588396+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"238'16616","reported_seq":16875,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:40.891010+0000","last_change":"2024-09-21T11:46:27.656566+0000","last_active":"2024-09-21T11:48:40.891010+0000","last_peered":"2024-09-21T11:48:40.891010+0000","last_clean":"2024-09-21T11:48:40.891010+0000","last_became_active":"2024-09-21T11:44:51.745609+0000","last_became_peered":"2024-09-21T11:44:51.745609+0000","last_unstale":"2024-09-21T11:48:40.891010+0000","last_undegraded":"2024-09-21T11:48:40.891010+0000","last_fullsized":"2024-09-21T11:48:40.891010+0000","mapping_epoch":160,"log_start":"198'10200","ondisk_log_start":"198'10200","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'8199","last_scrub_stamp":"2024-09-21T11:46:27.656360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.656360+0000","objects_scrubbed":8107,"log_size":6416,"log_dups_size":0,"ondisk_log_size":6416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:46.172763+0000","scrub_duration":2017,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41549824,"num_objects":634,"num_object_clones":0,"num_object_copies":1902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16616,"num_write_kb":552000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":471,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288565+0000","last_change":"2024-09-21T11:48:04.111199+0000","last_active":"2024-09-21T11:48:39.288565+0000","last_peered":"2024-09-21T11:48:39.288565+0000","last_clean":"2024-09-21T11:48:39.288565+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:48:39.288565+0000","last_undegraded":"2024-09-21T11:48:39.288565+0000","last_fullsized":"2024-09-21T11:48:39.288565+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:48:04.111115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.111115+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:19.507697+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":426,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.308103+0000","last_change":"2024-09-21T11:47:26.411190+0000","last_active":"2024-09-21T11:48:39.308103+0000","last_peered":"2024-09-21T11:48:39.308103+0000","last_clean":"2024-09-21T11:48:39.308103+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:48:39.308103+0000","last_undegraded":"2024-09-21T11:48:39.308103+0000","last_fullsized":"2024-09-21T11:48:39.308103+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:26.411146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:26.411146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:48.131154+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":413,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.346392+0000","last_change":"2024-09-21T11:47:03.385382+0000","last_active":"2024-09-21T11:48:41.346392+0000","last_peered":"2024-09-21T11:48:41.346392+0000","last_clean":"2024-09-21T11:48:41.346392+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:48:41.346392+0000","last_undegraded":"2024-09-21T11:48:41.346392+0000","last_fullsized":"2024-09-21T11:48:41.346392+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.766819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.766819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:38.890518+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":393,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.354568+0000","last_change":"2024-09-21T11:47:03.376001+0000","last_active":"2024-09-21T11:48:41.354568+0000","last_peered":"2024-09-21T11:48:41.354568+0000","last_clean":"2024-09-21T11:48:41.354568+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:48:41.354568+0000","last_undegraded":"2024-09-21T11:48:41.354568+0000","last_fullsized":"2024-09-21T11:48:41.354568+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.703712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.703712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"238'16601","reported_seq":16849,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:40.987243+0000","last_change":"2024-09-21T11:46:12.974904+0000","last_active":"2024-09-21T11:48:40.987243+0000","last_peered":"2024-09-21T11:48:40.987243+0000","last_clean":"2024-09-21T11:48:40.987243+0000","last_became_active":"2024-09-21T11:44:51.409710+0000","last_became_peered":"2024-09-21T11:44:51.409710+0000","last_unstale":"2024-09-21T11:48:40.987243+0000","last_undegraded":"2024-09-21T11:48:40.987243+0000","last_fullsized":"2024-09-21T11:48:40.987243+0000","mapping_epoch":160,"log_start":"204'11160","ondisk_log_start":"204'11160","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7106","last_scrub_stamp":"2024-09-21T11:46:12.974687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.974687+0000","objects_scrubbed":7036,"log_size":5441,"log_dups_size":0,"ondisk_log_size":5441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.015746+0000","scrub_duration":1797,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39911424,"num_objects":609,"num_object_clones":0,"num_object_copies":1827,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16601,"num_write_kb":550720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":401,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.300523+0000","last_change":"2024-09-21T11:46:57.248649+0000","last_active":"2024-09-21T11:48:39.300523+0000","last_peered":"2024-09-21T11:48:39.300523+0000","last_clean":"2024-09-21T11:48:39.300523+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:48:39.300523+0000","last_undegraded":"2024-09-21T11:48:39.300523+0000","last_fullsized":"2024-09-21T11:48:39.300523+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:57.248575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.248575+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":416,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.307370+0000","last_change":"2024-09-21T11:48:09.683093+0000","last_active":"2024-09-21T11:48:39.307370+0000","last_peered":"2024-09-21T11:48:39.307370+0000","last_clean":"2024-09-21T11:48:39.307370+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:48:39.307370+0000","last_undegraded":"2024-09-21T11:48:39.307370+0000","last_fullsized":"2024-09-21T11:48:39.307370+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:48:09.683008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.683008+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:38.676024+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":501,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.306715+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:48:39.306715+0000","last_peered":"2024-09-21T11:48:39.306715+0000","last_clean":"2024-09-21T11:48:39.306715+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:48:39.306715+0000","last_undegraded":"2024-09-21T11:48:39.306715+0000","last_fullsized":"2024-09-21T11:48:39.306715+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.600494+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":376,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301273+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:48:39.301273+0000","last_peered":"2024-09-21T11:48:39.301273+0000","last_clean":"2024-09-21T11:48:39.301273+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:48:39.301273+0000","last_undegraded":"2024-09-21T11:48:39.301273+0000","last_fullsized":"2024-09-21T11:48:39.301273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.006344+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":404,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.289987+0000","last_change":"2024-09-21T11:48:17.129730+0000","last_active":"2024-09-21T11:48:39.289987+0000","last_peered":"2024-09-21T11:48:39.289987+0000","last_clean":"2024-09-21T11:48:39.289987+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:48:39.289987+0000","last_undegraded":"2024-09-21T11:48:39.289987+0000","last_fullsized":"2024-09-21T11:48:39.289987+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.129675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.129675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.682622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":426,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.370356+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:48:41.370356+0000","last_peered":"2024-09-21T11:48:41.370356+0000","last_clean":"2024-09-21T11:48:41.370356+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:48:41.370356+0000","last_undegraded":"2024-09-21T11:48:41.370356+0000","last_fullsized":"2024-09-21T11:48:41.370356+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":402,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301470+0000","last_change":"2024-09-21T11:47:25.905705+0000","last_active":"2024-09-21T11:48:39.301470+0000","last_peered":"2024-09-21T11:48:39.301470+0000","last_clean":"2024-09-21T11:48:39.301470+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:48:39.301470+0000","last_undegraded":"2024-09-21T11:48:39.301470+0000","last_fullsized":"2024-09-21T11:48:39.301470+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.905645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.905645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.034596+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":392,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.309685+0000","last_change":"2024-09-21T11:47:28.380378+0000","last_active":"2024-09-21T11:48:39.309685+0000","last_peered":"2024-09-21T11:48:39.309685+0000","last_clean":"2024-09-21T11:48:39.309685+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:48:39.309685+0000","last_undegraded":"2024-09-21T11:48:39.309685+0000","last_fullsized":"2024-09-21T11:48:39.309685+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:28.380245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:28.380245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.638314+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":419,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.367455+0000","last_change":"2024-09-21T11:46:28.973700+0000","last_active":"2024-09-21T11:48:41.367455+0000","last_peered":"2024-09-21T11:48:41.367455+0000","last_clean":"2024-09-21T11:48:41.367455+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:48:41.367455+0000","last_undegraded":"2024-09-21T11:48:41.367455+0000","last_fullsized":"2024-09-21T11:48:41.367455+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:28.973606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.973606+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":388,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.306863+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:48:39.306863+0000","last_peered":"2024-09-21T11:48:39.306863+0000","last_clean":"2024-09-21T11:48:39.306863+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:48:39.306863+0000","last_undegraded":"2024-09-21T11:48:39.306863+0000","last_fullsized":"2024-09-21T11:48:39.306863+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:47.600096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":405,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365299+0000","last_change":"2024-09-21T11:47:03.379216+0000","last_active":"2024-09-21T11:48:41.365299+0000","last_peered":"2024-09-21T11:48:41.365299+0000","last_clean":"2024-09-21T11:48:41.365299+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:48:41.365299+0000","last_undegraded":"2024-09-21T11:48:41.365299+0000","last_fullsized":"2024-09-21T11:48:41.365299+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:48.903980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:48.903980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":395,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.347556+0000","last_change":"2024-09-21T11:48:17.081179+0000","last_active":"2024-09-21T11:48:41.347556+0000","last_peered":"2024-09-21T11:48:41.347556+0000","last_clean":"2024-09-21T11:48:41.347556+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:48:41.347556+0000","last_undegraded":"2024-09-21T11:48:41.347556+0000","last_fullsized":"2024-09-21T11:48:41.347556+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.081131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.081131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.264468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":400,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.302243+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:48:39.302243+0000","last_peered":"2024-09-21T11:48:39.302243+0000","last_clean":"2024-09-21T11:48:39.302243+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:48:39.302243+0000","last_undegraded":"2024-09-21T11:48:39.302243+0000","last_fullsized":"2024-09-21T11:48:39.302243+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.995538+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":398,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365589+0000","last_change":"2024-09-21T11:46:24.087039+0000","last_active":"2024-09-21T11:48:41.365589+0000","last_peered":"2024-09-21T11:48:41.365589+0000","last_clean":"2024-09-21T11:48:41.365589+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:48:41.365589+0000","last_undegraded":"2024-09-21T11:48:41.365589+0000","last_fullsized":"2024-09-21T11:48:41.365589+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.086918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.086918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":488,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.356175+0000","last_change":"2024-09-21T11:47:02.849839+0000","last_active":"2024-09-21T11:48:41.356175+0000","last_peered":"2024-09-21T11:48:41.356175+0000","last_clean":"2024-09-21T11:48:41.356175+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:48:41.356175+0000","last_undegraded":"2024-09-21T11:48:41.356175+0000","last_fullsized":"2024-09-21T11:48:41.356175+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:02.849786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:02.849786+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":385,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288915+0000","last_change":"2024-09-21T11:45:38.652432+0000","last_active":"2024-09-21T11:48:39.288915+0000","last_peered":"2024-09-21T11:48:39.288915+0000","last_clean":"2024-09-21T11:48:39.288915+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:48:39.288915+0000","last_undegraded":"2024-09-21T11:48:39.288915+0000","last_fullsized":"2024-09-21T11:48:39.288915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:38.652380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:38.652380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:47.014191+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":432,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.356344+0000","last_change":"2024-09-21T11:47:03.428846+0000","last_active":"2024-09-21T11:48:41.356344+0000","last_peered":"2024-09-21T11:48:41.356344+0000","last_clean":"2024-09-21T11:48:41.356344+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:48:41.356344+0000","last_undegraded":"2024-09-21T11:48:41.356344+0000","last_fullsized":"2024-09-21T11:48:41.356344+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:29.284078+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":388,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.306480+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:48:39.306480+0000","last_peered":"2024-09-21T11:48:39.306480+0000","last_clean":"2024-09-21T11:48:39.306480+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:48:39.306480+0000","last_undegraded":"2024-09-21T11:48:39.306480+0000","last_fullsized":"2024-09-21T11:48:39.306480+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:44.604026+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":524,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.302098+0000","last_change":"2024-09-21T11:47:03.415793+0000","last_active":"2024-09-21T11:48:39.302098+0000","last_peered":"2024-09-21T11:48:39.302098+0000","last_clean":"2024-09-21T11:48:39.302098+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:48:39.302098+0000","last_undegraded":"2024-09-21T11:48:39.302098+0000","last_fullsized":"2024-09-21T11:48:39.302098+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:10.174971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:10.174971+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.999554+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":386,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.366961+0000","last_change":"2024-09-21T11:45:44.314664+0000","last_active":"2024-09-21T11:48:41.366961+0000","last_peered":"2024-09-21T11:48:41.366961+0000","last_clean":"2024-09-21T11:48:41.366961+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:48:41.366961+0000","last_undegraded":"2024-09-21T11:48:41.366961+0000","last_fullsized":"2024-09-21T11:48:41.366961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:44.314461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.314461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":477,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288477+0000","last_change":"2024-09-21T11:48:33.033557+0000","last_active":"2024-09-21T11:48:39.288477+0000","last_peered":"2024-09-21T11:48:39.288477+0000","last_clean":"2024-09-21T11:48:39.288477+0000","last_became_active":"2024-09-21T11:48:33.022647+0000","last_became_peered":"2024-09-21T11:48:33.022647+0000","last_unstale":"2024-09-21T11:48:39.288477+0000","last_undegraded":"2024-09-21T11:48:39.288477+0000","last_fullsized":"2024-09-21T11:48:39.288477+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:54.032795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.032795+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.014222+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":389,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.288374+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:48:39.288374+0000","last_peered":"2024-09-21T11:48:39.288374+0000","last_clean":"2024-09-21T11:48:39.288374+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:48:39.288374+0000","last_undegraded":"2024-09-21T11:48:39.288374+0000","last_fullsized":"2024-09-21T11:48:39.288374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":82,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.295212+0000","last_change":"2024-09-21T11:47:03.379049+0000","last_active":"2024-09-21T11:48:39.295212+0000","last_peered":"2024-09-21T11:48:39.295212+0000","last_clean":"2024-09-21T11:48:39.295212+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:48:39.295212+0000","last_undegraded":"2024-09-21T11:48:39.295212+0000","last_fullsized":"2024-09-21T11:48:39.295212+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.269610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.269610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":371,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.354382+0000","last_change":"2024-09-21T11:45:04.770055+0000","last_active":"2024-09-21T11:48:41.354382+0000","last_peered":"2024-09-21T11:48:41.354382+0000","last_clean":"2024-09-21T11:48:41.354382+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:48:41.354382+0000","last_undegraded":"2024-09-21T11:48:41.354382+0000","last_fullsized":"2024-09-21T11:48:41.354382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:04.769963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:04.769963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":92,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.295474+0000","last_change":"2024-09-21T11:47:03.404473+0000","last_active":"2024-09-21T11:48:39.295474+0000","last_peered":"2024-09-21T11:48:39.295474+0000","last_clean":"2024-09-21T11:48:39.295474+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:48:39.295474+0000","last_undegraded":"2024-09-21T11:48:39.295474+0000","last_fullsized":"2024-09-21T11:48:39.295474+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:26.649189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:26.649189+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":396,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.309897+0000","last_change":"2024-09-21T11:48:04.553772+0000","last_active":"2024-09-21T11:48:39.309897+0000","last_peered":"2024-09-21T11:48:39.309897+0000","last_clean":"2024-09-21T11:48:39.309897+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:48:39.309897+0000","last_undegraded":"2024-09-21T11:48:39.309897+0000","last_fullsized":"2024-09-21T11:48:39.309897+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:04.553731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.553731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:21.248327+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":405,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.354662+0000","last_change":"2024-09-21T11:47:03.377948+0000","last_active":"2024-09-21T11:48:41.354662+0000","last_peered":"2024-09-21T11:48:41.354662+0000","last_clean":"2024-09-21T11:48:41.354662+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:48:41.354662+0000","last_undegraded":"2024-09-21T11:48:41.354662+0000","last_fullsized":"2024-09-21T11:48:41.354662+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:53.848323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:53.848323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":409,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.365008+0000","last_change":"2024-09-21T11:47:03.408821+0000","last_active":"2024-09-21T11:48:41.365008+0000","last_peered":"2024-09-21T11:48:41.365008+0000","last_clean":"2024-09-21T11:48:41.365008+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:48:41.365008+0000","last_undegraded":"2024-09-21T11:48:41.365008+0000","last_fullsized":"2024-09-21T11:48:41.365008+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:45:43.314803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:43.314803+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":406,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.354534+0000","last_change":"2024-09-21T11:45:46.706221+0000","last_active":"2024-09-21T11:48:41.354534+0000","last_peered":"2024-09-21T11:48:41.354534+0000","last_clean":"2024-09-21T11:48:41.354534+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:48:41.354534+0000","last_undegraded":"2024-09-21T11:48:41.354534+0000","last_fullsized":"2024-09-21T11:48:41.354534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:45:46.706130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.706130+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":402,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.301545+0000","last_change":"2024-09-21T11:48:10.126645+0000","last_active":"2024-09-21T11:48:39.301545+0000","last_peered":"2024-09-21T11:48:39.301545+0000","last_clean":"2024-09-21T11:48:39.301545+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:48:39.301545+0000","last_undegraded":"2024-09-21T11:48:39.301545+0000","last_fullsized":"2024-09-21T11:48:39.301545+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.126591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.126591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:10.829490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":372,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.305539+0000","last_change":"2024-09-21T11:43:25.614957+0000","last_active":"2024-09-21T11:48:39.305539+0000","last_peered":"2024-09-21T11:48:39.305539+0000","last_clean":"2024-09-21T11:48:39.305539+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:48:39.305539+0000","last_undegraded":"2024-09-21T11:48:39.305539+0000","last_fullsized":"2024-09-21T11:48:39.305539+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:25.614898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:25.614898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:43.608235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":379,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.354902+0000","last_change":"2024-09-21T11:45:45.730374+0000","last_active":"2024-09-21T11:48:41.354902+0000","last_peered":"2024-09-21T11:48:41.354902+0000","last_clean":"2024-09-21T11:48:41.354902+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:48:41.354902+0000","last_undegraded":"2024-09-21T11:48:41.354902+0000","last_fullsized":"2024-09-21T11:48:41.354902+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:45.730169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:45.730169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":440,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.360814+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:48:41.360814+0000","last_peered":"2024-09-21T11:48:41.360814+0000","last_clean":"2024-09-21T11:48:41.360814+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:48:41.360814+0000","last_undegraded":"2024-09-21T11:48:41.360814+0000","last_fullsized":"2024-09-21T11:48:41.360814+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":390,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.366132+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:48:41.366132+0000","last_peered":"2024-09-21T11:48:41.366132+0000","last_clean":"2024-09-21T11:48:41.366132+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:48:41.366132+0000","last_undegraded":"2024-09-21T11:48:41.366132+0000","last_fullsized":"2024-09-21T11:48:41.366132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.212674+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":388,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.290253+0000","last_change":"2024-09-21T11:48:09.104826+0000","last_active":"2024-09-21T11:48:39.290253+0000","last_peered":"2024-09-21T11:48:39.290253+0000","last_clean":"2024-09-21T11:48:39.290253+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:48:39.290253+0000","last_undegraded":"2024-09-21T11:48:39.290253+0000","last_fullsized":"2024-09-21T11:48:39.290253+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:09.104767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.104767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.005085+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":376,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.305891+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:48:39.305891+0000","last_peered":"2024-09-21T11:48:39.305891+0000","last_clean":"2024-09-21T11:48:39.305891+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:48:39.305891+0000","last_undegraded":"2024-09-21T11:48:39.305891+0000","last_fullsized":"2024-09-21T11:48:39.305891+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:49.549791+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":82,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.296228+0000","last_change":"2024-09-21T11:47:03.374194+0000","last_active":"2024-09-21T11:48:39.296228+0000","last_peered":"2024-09-21T11:48:39.296228+0000","last_clean":"2024-09-21T11:48:39.296228+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:48:39.296228+0000","last_undegraded":"2024-09-21T11:48:39.296228+0000","last_fullsized":"2024-09-21T11:48:39.296228+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:27.297630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.297630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":405,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.361046+0000","last_change":"2024-09-21T11:47:03.377189+0000","last_active":"2024-09-21T11:48:41.361046+0000","last_peered":"2024-09-21T11:48:41.361046+0000","last_clean":"2024-09-21T11:48:41.361046+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:48:41.361046+0000","last_undegraded":"2024-09-21T11:48:41.361046+0000","last_fullsized":"2024-09-21T11:48:41.361046+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:11.587469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.587469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":458,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-21T11:48:39.290792+0000","last_change":"2024-09-21T11:47:03.392087+0000","last_active":"2024-09-21T11:48:39.290792+0000","last_peered":"2024-09-21T11:48:39.290792+0000","last_clean":"2024-09-21T11:48:39.290792+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:48:39.290792+0000","last_undegraded":"2024-09-21T11:48:39.290792+0000","last_fullsized":"2024-09-21T11:48:39.290792+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:16.675872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.675872+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:41.170726+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":391,"reported_epoch":239,"state":"active+clean","last_fresh":"2024-09-21T11:48:41.367195+0000","last_change":"2024-09-21T11:45:12.377954+0000","last_active":"2024-09-21T11:48:41.367195+0000","last_peered":"2024-09-21T11:48:41.367195+0000","last_clean":"2024-09-21T11:48:41.367195+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:48:41.367195+0000","last_undegraded":"2024-09-21T11:48:41.367195+0000","last_fullsized":"2024-09-21T11:48:41.367195+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:12.377880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:12.377880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":607453208,"num_objects":9270,"num_object_clones":0,"num_object_copies":27810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":264914,"num_write_kb":8773825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24511,"num_bytes_recovered":1606287360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2116628480,"data_stored":2116616264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86062,"ondisk_log_size":86062,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":396,"ondisk_log_size":396,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":512,"num_read_kb":507,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234093,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":522424,"kb_used_data":202468,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":93222728,"statfs":{"total":95995035648,"available":95460073472,"internally_reserved":0,"allocated":207327232,"data_stored":206959465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463026,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":505000,"kb_used_data":216980,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":93240152,"statfs":{"total":95995035648,"available":95477915648,"internally_reserved":0,"allocated":222187520,"data_stored":221836457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":294910406},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724663,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":636092,"kb_used_data":334184,"kb_used_omap":1,"kb_used_meta":301886,"kb_avail":93109060,"statfs":{"total":95995035648,"available":95343677440,"internally_reserved":0,"allocated":342204416,"data_stored":341831764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":309131718},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920893,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":488252,"kb_used_data":218216,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":93256900,"statfs":{"total":95995035648,"available":95495065600,"internally_reserved":0,"allocated":223453184,"data_stored":223081266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":276429253},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117123,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":604012,"kb_used_data":225752,"kb_used_omap":1,"kb_used_meta":378238,"kb_avail":93141140,"statfs":{"total":95995035648,"available":95376527360,"internally_reserved":0,"allocated":231170048,"data_stored":230813339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":387316167},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411465,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":927860,"kb_used_data":509344,"kb_used_omap":1,"kb_used_meta":418494,"kb_avail":92817292,"statfs":{"total":95995035648,"available":95044907008,"internally_reserved":0,"allocated":521568256,"data_stored":521204090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":428538309},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":572556,"kb_used_data":280316,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":93172596,"statfs":{"total":95995035648,"available":95408738304,"internally_reserved":0,"allocated":287043584,"data_stored":286651272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":299170246},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738518,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":547340,"kb_used_data":219000,"kb_used_omap":1,"kb_used_meta":328254,"kb_avail":93197812,"statfs":{"total":95995035648,"available":95434559488,"internally_reserved":0,"allocated":224256000,"data_stored":223875391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":336132549},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":216399872,"data_stored":216399872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":272433152,"data_stored":272433152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":508170240,"data_stored":508166168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":209453056,"data_stored":209453056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":202178560,"data_stored":202178560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":306188288,"data_stored":306184216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":201326592,"data_stored":201326592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":200478720,"data_stored":200474648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:48:47.956 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-21T11:48:48.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:48.045 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:48.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:48.046 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:48.143 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:48:48.143 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:48:48.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:48.179 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:48.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:48.180 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:48.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:48:48.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:48.314 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:48.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:48.316 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:48.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:48.465 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:48.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:48.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:48.567 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:48.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:48.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:48.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:48.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:48 smithi043 ceph-mon[29710]: 3.1d scrub starts 2024-09-21T11:48:48.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:48 smithi043 ceph-mon[29710]: 3.1d scrub ok 2024-09-21T11:48:48.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:48 smithi043 ceph-mon[29710]: 5.1 scrub starts 2024-09-21T11:48:48.631 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:48 smithi043 ceph-mon[29710]: 5.1 scrub ok 2024-09-21T11:48:48.631 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:48 smithi043 ceph-mon[36454]: 3.1d scrub starts 2024-09-21T11:48:48.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:48 smithi043 ceph-mon[36454]: 3.1d scrub ok 2024-09-21T11:48:48.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:48 smithi043 ceph-mon[36454]: 5.1 scrub starts 2024-09-21T11:48:48.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:48 smithi043 ceph-mon[36454]: 5.1 scrub ok 2024-09-21T11:48:48.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:48.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:48 smithi175 ceph-mon[32613]: 3.1d scrub starts 2024-09-21T11:48:48.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:48 smithi175 ceph-mon[32613]: 3.1d scrub ok 2024-09-21T11:48:48.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:48 smithi175 ceph-mon[32613]: 5.1 scrub starts 2024-09-21T11:48:48.698 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:48 smithi175 ceph-mon[32613]: 5.1 scrub ok 2024-09-21T11:48:48.729 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:48.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:48.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:48.780 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:48.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:48.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:48.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:48.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:49.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:49.016 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:49.016 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:49.117 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:49.148 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:49.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:49.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:49.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:49.305 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:49.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:48:49.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:49.307 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:49.385 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:48:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:48:49.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:49.416 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:48:49.416 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:48:49.437 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:48:49.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:49.458 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:49.458 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:49.461 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:48:49.462 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:48:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[36454]: pgmap v419: 148 pgs: 148 active+clean; 242 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-21T11:48:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[36454]: 8.5 scrub starts 2024-09-21T11:48:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[36454]: 8.5 scrub ok 2024-09-21T11:48:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3164413780' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[36454]: 2.1 scrub starts 2024-09-21T11:48:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[36454]: 2.1 scrub ok 2024-09-21T11:48:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2349239674' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[29710]: pgmap v419: 148 pgs: 148 active+clean; 242 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-21T11:48:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[29710]: 8.5 scrub starts 2024-09-21T11:48:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[29710]: 8.5 scrub ok 2024-09-21T11:48:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3164413780' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[29710]: 2.1 scrub starts 2024-09-21T11:48:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[29710]: 2.1 scrub ok 2024-09-21T11:48:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2349239674' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:49.502 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:48:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:48:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:49 smithi175 ceph-mon[32613]: pgmap v419: 148 pgs: 148 active+clean; 242 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-21T11:48:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:49 smithi175 ceph-mon[32613]: 8.5 scrub starts 2024-09-21T11:48:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:49 smithi175 ceph-mon[32613]: 8.5 scrub ok 2024-09-21T11:48:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3164413780' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:48:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:49 smithi175 ceph-mon[32613]: 2.1 scrub starts 2024-09-21T11:48:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:49 smithi175 ceph-mon[32613]: 2.1 scrub ok 2024-09-21T11:48:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2349239674' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:49.540 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:48:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:48:49.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:49.573 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:48:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:48:49.596 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:49.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:49.597 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:49.609 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:48:49.610 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:48:49.631 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Removed 137091 objects 2024-09-21T11:48:49.631 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Clean up completed and total clean up time :137.004 2024-09-21T11:48:49.641 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:48:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:48:49.697 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:49.700 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:48:49.720 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:49.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:49.721 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:49.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:49.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:49.845 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:49.946 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:49.968 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:49.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:49.969 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:50.069 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:50.091 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:50.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:50.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:50.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:50.215 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:50.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:50.216 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:50.317 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:50.339 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:50.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:50.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:50.440 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:50.462 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:50.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:50.462 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:48:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:48:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3164413780' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:48:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[36454]: osdmap e241: 8 total, 8 up, 8 in 2024-09-21T11:48:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[36454]: 5.1e scrub starts 2024-09-21T11:48:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[36454]: 5.1e scrub ok 2024-09-21T11:48:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[36454]: 5.a scrub starts 2024-09-21T11:48:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[36454]: 5.a scrub ok 2024-09-21T11:48:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:48:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:48:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3164413780' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:48:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[29710]: osdmap e241: 8 total, 8 up, 8 in 2024-09-21T11:48:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[29710]: 5.1e scrub starts 2024-09-21T11:48:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[29710]: 5.1e scrub ok 2024-09-21T11:48:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[29710]: 5.a scrub starts 2024-09-21T11:48:50.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:50 smithi043 ceph-mon[29710]: 5.a scrub ok 2024-09-21T11:48:50.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:50.584 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:50.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:50.585 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:50.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:50.707 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:50.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:50.708 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:50 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:48:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:50 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:48:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3164413780' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:48:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:50 smithi175 ceph-mon[32613]: osdmap e241: 8 total, 8 up, 8 in 2024-09-21T11:48:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:50 smithi175 ceph-mon[32613]: 5.1e scrub starts 2024-09-21T11:48:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:50 smithi175 ceph-mon[32613]: 5.1e scrub ok 2024-09-21T11:48:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:50 smithi175 ceph-mon[32613]: 5.a scrub starts 2024-09-21T11:48:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:50 smithi175 ceph-mon[32613]: 5.a scrub ok 2024-09-21T11:48:50.809 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:50.830 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:50.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:50.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:50.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:50.942 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:48:50.943 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:48:50.943 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:48:50.943 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:48:50.943 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:48:50.953 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:50.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:50.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:51.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:51.080 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:51.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:51.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:51.182 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:51.206 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:51.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:51.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:51.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:51.332 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:51.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:51.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:51.434 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:51.458 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:51.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:51.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:51.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:48:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:48:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:51 smithi175 ceph-mon[32613]: pgmap v421: 148 pgs: 148 active+clean; 242 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 676 op/s 2024-09-21T11:48:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:51 smithi175 ceph-mon[32613]: 6.10 scrub starts 2024-09-21T11:48:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:51 smithi175 ceph-mon[32613]: 6.10 scrub ok 2024-09-21T11:48:51.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:51 smithi175 ceph-mon[32613]: 6.17 scrub starts 2024-09-21T11:48:51.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:51 smithi175 ceph-mon[32613]: 6.17 scrub ok 2024-09-21T11:48:51.560 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:51.585 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:51.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:51.586 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:51.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:51.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:51.714 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:51 smithi043 ceph-mon[36454]: pgmap v421: 148 pgs: 148 active+clean; 242 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 676 op/s 2024-09-21T11:48:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:51 smithi043 ceph-mon[36454]: 6.10 scrub starts 2024-09-21T11:48:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:51 smithi043 ceph-mon[36454]: 6.10 scrub ok 2024-09-21T11:48:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:51 smithi043 ceph-mon[36454]: 6.17 scrub starts 2024-09-21T11:48:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:51 smithi043 ceph-mon[36454]: 6.17 scrub ok 2024-09-21T11:48:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:51 smithi043 ceph-mon[29710]: pgmap v421: 148 pgs: 148 active+clean; 242 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 676 op/s 2024-09-21T11:48:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:51 smithi043 ceph-mon[29710]: 6.10 scrub starts 2024-09-21T11:48:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:51 smithi043 ceph-mon[29710]: 6.10 scrub ok 2024-09-21T11:48:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:51 smithi043 ceph-mon[29710]: 6.17 scrub starts 2024-09-21T11:48:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:51 smithi043 ceph-mon[29710]: 6.17 scrub ok 2024-09-21T11:48:51.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:51.841 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:51.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:51.842 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:51.943 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:51.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:51.966 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:51.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:52.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:52.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:52.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:52.194 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:52.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:48:52.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:52.215 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:52.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:52.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:52.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:48:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:52.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:52.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:52.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:52.465 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:52.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:52.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:52.475 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:52.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: 8.9 scrub starts 2024-09-21T11:48:52.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: 8.9 scrub ok 2024-09-21T11:48:52.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:52.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: 6.16 scrub starts 2024-09-21T11:48:52.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: 6.16 scrub ok 2024-09-21T11:48:52.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:48:52.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: 5.1c scrub starts 2024-09-21T11:48:52.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: 5.1c scrub ok 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: 8.9 scrub starts 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: 8.9 scrub ok 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: 6.16 scrub starts 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: 6.16 scrub ok 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: 5.1c scrub starts 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: 5.1c scrub ok 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:48:52.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:48:52.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:52 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:48:52.567 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:52.570 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:48:52.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:52.589 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:52.590 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:52.690 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:52.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: 8.9 scrub starts 2024-09-21T11:48:52.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: 8.9 scrub ok 2024-09-21T11:48:52.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:48:52.693 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: 6.16 scrub starts 2024-09-21T11:48:52.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: 6.16 scrub ok 2024-09-21T11:48:52.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:48:52.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: 5.1c scrub starts 2024-09-21T11:48:52.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: 5.1c scrub ok 2024-09-21T11:48:52.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:48:52.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:48:52.694 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:52 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:48:52.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:52.712 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:52.713 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:52.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:52.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:52.837 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:52.838 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:52.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:52.947 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:48:52.957 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:48:52.957 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:48:52.957 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_1 2024-09-21T11:48:52.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:48:52.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:52.963 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:53.036 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:48:53.036 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:48:53.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:48:53.036 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:48:53.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.036 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126212 7.1 7:86c8b838:::benchmark_data_smithi043_70646_object8804:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.194683+0000", 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 353.83817272800002, 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.097383589000000007, 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126212 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.037 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194683+0000", 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194683+0000", 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194685+0000", 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9920000000000002e-06 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194691+0000", 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4809999999999997e-06 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194702+0000", 2024-09-21T11:48:53.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1559999999999999e-05 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194711+0000", 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5250000000000005e-06 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194786+0000", 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4846999999999997e-05 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194894+0000", 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010852400000000001 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.195001+0000", 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106406 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197371+0000", 2024-09-21T11:48:53.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002370367 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197897+0000", 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052575600000000005 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291967+0000", 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.094070592999999994 2024-09-21T11:48:53.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.292026+0000", 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8671000000000001e-05 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.292067+0000", 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0867e-05 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.043 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:185199 7.1 7:848115c2:::benchmark_data_smithi043_70646_object67791:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:57.851664+0000", 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.18119155800002, 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092637500999999997, 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.044 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 185199 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851664+0000", 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851664+0000", 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851666+0000", 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2249999999999999e-06 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851671+0000", 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8919999999999999e-06 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851686+0000", 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5675000000000002e-05 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851694+0000", 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9489999999999996e-06 2024-09-21T11:48:53.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851811+0000", 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116335 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851954+0000", 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143941 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.852069+0000", 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011415499999999999 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.854468+0000", 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023996629999999998 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.854814+0000", 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00034581899999999998 2024-09-21T11:48:53.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.944168+0000", 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.089353558999999999 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.944259+0000", 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0932000000000003e-05 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.944302+0000", 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3356000000000003e-05 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221225 7.1 7:81c1cf62:::benchmark_data_smithi043_70646_object103817:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.185689+0000", 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 265.84716702100002, 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092272482000000003, 2024-09-21T11:48:53.051 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221225 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185689+0000", 2024-09-21T11:48:53.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185689+0000", 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185690+0000", 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4079999999999999e-06 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.053 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185696+0000", 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3789999999999997e-06 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185708+0000", 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2383e-05 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185714+0000", 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5709999999999997e-06 2024-09-21T11:48:53.054 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185775+0000", 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1477999999999997e-05 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185891+0000", 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011618800000000001 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.055 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186010+0000", 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118808 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.188391+0000", 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023805369999999998 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.189115+0000", 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072425500000000004 2024-09-21T11:48:53.056 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.277871+0000", 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.088755954999999997 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.277921+0000", 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9827999999999998e-05 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.277961+0000", 2024-09-21T11:48:53.057 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0692000000000003e-05 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43899 8.2 8:43c05007:::benchmark_data_smithi043_97042_object43898:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.986744+0000", 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 212.04611222599999, 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.132095406, 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.058 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43899 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986744+0000", 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.059 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986744+0000", 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986745+0000", 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.632e-06 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986800+0000", 2024-09-21T11:48:53.060 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4793999999999997e-05 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986816+0000", 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5444e-05 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986824+0000", 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5799999999999992e-06 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.061 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986889+0000", 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4658000000000003e-05 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987030+0000", 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014126500000000001 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987181+0000", 2024-09-21T11:48:53.062 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015071799999999999 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989750+0000", 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025688159999999998 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.991302+0000", 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015523080000000001 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.063 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118726+0000", 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12742430799999999 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118787+0000", 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1141000000000003e-05 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118839+0000", 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1742e-05 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.064 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43905 8.2 8:49886794:::benchmark_data_smithi043_97042_object43904:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.987297+0000", 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 212.04555859499999, 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13219620100000001, 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.065 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43905 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987297+0000", 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987297+0000", 2024-09-21T11:48:53.066 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.067 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.067 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.067 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.067 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987298+0000", 2024-09-21T11:48:53.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:53.069 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9999999999999996e-07 2024-09-21T11:48:53.069 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.069 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.069 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987377+0000", 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8947000000000002e-05 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987387+0000", 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4960000000000002e-06 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.070 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987394+0000", 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3159999999999999e-06 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.988450+0000", 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001055802 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.988542+0000", 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1652999999999998e-05 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.071 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.988627+0000", 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5554000000000001e-05 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989839+0000", 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001212288 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.991597+0000", 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001757303 2024-09-21T11:48:53.072 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.119404+0000", 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12780698200000001 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.119460+0000", 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.622e-05 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.119494+0000", 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3739999999999999e-05 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46235 8.2 8:4e6f27b7:::benchmark_data_smithi043_97042_object46234:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.589574+0000", 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 210.44328237600001, 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.126942478, 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.074 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46235 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589574+0000", 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589574+0000", 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589585+0000", 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0998999999999999e-05 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589616+0000", 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1123e-05 2024-09-21T11:48:53.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589627+0000", 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1486e-05 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589634+0000", 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0740000000000004e-06 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589713+0000", 2024-09-21T11:48:53.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8753999999999998e-05 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589860+0000", 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014660000000000001 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590003+0000", 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143516 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596149+0000", 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0061461579999999997 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596193+0000", 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3176000000000001e-05 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716435+0000", 2024-09-21T11:48:53.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.120242746 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716472+0000", 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6868e-05 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716516+0000", 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3977999999999998e-05 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.080 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46237 8.2 8:4ddff5f6:::benchmark_data_smithi043_97042_object46236:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.590497+0000", 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 210.44235896999999, 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.126123824, 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.081 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46237 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590497+0000", 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590497+0000", 2024-09-21T11:48:53.082 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590498+0000", 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0729999999999999e-06 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590524+0000", 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6284000000000001e-05 2024-09-21T11:48:53.083 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590535+0000", 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0913999999999999e-05 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590545+0000", 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0054999999999999e-05 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.084 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.591661+0000", 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001115324 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.591765+0000", 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104269 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.591884+0000", 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118842 2024-09-21T11:48:53.085 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596154+0000", 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0042705729999999997 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596233+0000", 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8449999999999996e-05 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.086 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716553+0000", 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12031977000000001 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716582+0000", 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9628000000000002e-05 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716621+0000", 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8642e-05 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.087 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46241 8.2 8:41f5ae22:::benchmark_data_smithi043_97042_object46240:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.590600+0000", 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 210.44225651900001, 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.126117386, 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.088 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46241 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590600+0000", 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590600+0000", 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.089 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590600+0000", 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1500000000000004e-07 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590739+0000", 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013872799999999999 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.090 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590746+0000", 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5830000000000001e-06 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590752+0000", 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5099999999999998e-06 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.593857+0000", 2024-09-21T11:48:53.091 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003104674 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.593998+0000", 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141125 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.594116+0000", 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118461 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.092 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596202+0000", 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020862720000000001 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598936+0000", 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002733666 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716656+0000", 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.117719485 2024-09-21T11:48:53.093 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716685+0000", 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9683e-05 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716717+0000", 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1584000000000001e-05 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.094 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46255 8.2 8:4a6e54a1:::benchmark_data_smithi043_97042_object46254:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.598672+0000", 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 210.43418379600001, 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.118134724, 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46255 2024-09-21T11:48:53.095 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598672+0000", 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598672+0000", 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.096 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598674+0000", 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6640000000000001e-06 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598702+0000", 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7993000000000001e-05 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598714+0000", 2024-09-21T11:48:53.097 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2078e-05 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598721+0000", 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9099999999999999e-06 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598969+0000", 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000248006 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.098 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.599102+0000", 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133526 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.599254+0000", 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000151302 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.601110+0000", 2024-09-21T11:48:53.099 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001856749 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.601489+0000", 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037851399999999999 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716744+0000", 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.115254947 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.100 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716771+0000", 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7053e-05 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716807+0000", 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5982000000000002e-05 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.101 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47783 8.2 8:4b213eb9:::benchmark_data_smithi043_97042_object47782:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.709555+0000", 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 209.323301155, 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.135854843, 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47783 2024-09-21T11:48:53.102 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709555+0000", 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709555+0000", 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.103 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709556+0000", 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4160000000000001e-06 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709667+0000", 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110912 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709676+0000", 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6349999999999995e-06 2024-09-21T11:48:53.104 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709682+0000", 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8649999999999998e-06 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.710257+0000", 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057514299999999997 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.105 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.710363+0000", 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010655200000000001 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.710488+0000", 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124943 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712767+0000", 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022785959999999999 2024-09-21T11:48:53.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712779+0000", 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2417e-05 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845284+0000", 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.132504967 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845364+0000", 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9629000000000006e-05 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845410+0000", 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5768e-05 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:51215 8.2 8:470c6f96:::benchmark_data_smithi043_97042_object51214:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:48:53.108 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:25.913685+0000", 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 207.11917107400001, 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.098319711000000004, 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 51215 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913685+0000", 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913685+0000", 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913686+0000", 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1829999999999999e-06 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913793+0000", 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106716 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913802+0000", 2024-09-21T11:48:53.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5389999999999996e-06 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913809+0000", 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7139999999999996e-06 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.919718+0000", 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0059089989999999998 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.919851+0000", 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000132526 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.920234+0000", 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000383397 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.925843+0000", 2024-09-21T11:48:53.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056085889999999998 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.925917+0000", 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4525999999999998e-05 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.011831+0000", 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.085914040999999997 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.011952+0000", 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000120442 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.012005+0000", 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3038999999999998e-05 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.115 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:51227 8.2 8:4082683c:::benchmark_data_smithi043_97042_object51226:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:25.921381+0000", 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 207.11147494900001, 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.094778826999999996, 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 51227 2024-09-21T11:48:53.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921381+0000", 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921381+0000", 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.117 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921382+0000", 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9000000000000005e-07 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921406+0000", 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3465e-05 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921473+0000", 2024-09-21T11:48:53.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7885999999999993e-05 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921482+0000", 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3610000000000001e-06 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.924030+0000", 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025484309999999999 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.924134+0000", 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104109 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.924253+0000", 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118822 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.925849+0000", 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015963360000000001 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.927356+0000", 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015062750000000001 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.016021+0000", 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.088665620000000001 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.016114+0000", 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2369999999999998e-05 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.016160+0000", 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6162000000000003e-05 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.122 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66128 8.2 8:4c9d601c:::benchmark_data_smithi043_97042_object66127:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.918161+0000", 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 197.114694722, 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13079232700000001, 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66128 2024-09-21T11:48:53.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918161+0000", 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918161+0000", 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918162+0000", 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0610000000000001e-06 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918211+0000", 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9125000000000001e-05 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918221+0000", 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.183e-06 2024-09-21T11:48:53.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918227+0000", 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5470000000000002e-06 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919404+0000", 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011771520000000001 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919510+0000", 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106103 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919632+0000", 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012106400000000001 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.921625+0000", 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019929919999999999 2024-09-21T11:48:53.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.923993+0000", 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023686279999999998 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048774+0000", 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.124780935 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048871+0000", 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6514000000000005e-05 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048954+0000", 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3022999999999994e-05 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.129 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:68279 8.2 8:4095089f:::benchmark_data_smithi043_97042_object68278:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:37.675190+0000", 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 195.35766569500001, 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.098842746999999995, 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 68279 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675190+0000", 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675190+0000", 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675192+0000", 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4130000000000001e-06 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675440+0000", 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024801000000000001 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675452+0000", 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1754e-05 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675459+0000", 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4120000000000004e-06 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675521+0000", 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2074000000000002e-05 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675614+0000", 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3327999999999998e-05 2024-09-21T11:48:53.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675723+0000", 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108779 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.677638+0000", 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001914397 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680355+0000", 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027178079999999999 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773903+0000", 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093547443999999993 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773991+0000", 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8202999999999996e-05 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.774033+0000", 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2125000000000001e-05 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:68271 8.0 e178/160 8:076b09a0:::benchmark_data_smithi043_97042_object68270:head v 178'4262, mlcod=178'4258)", 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:37.680051+0000", 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 195.352804969, 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093011187999999995, 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.136 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680051+0000", 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680051+0000", 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680052+0000", 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4199999999999995e-07 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680119+0000", 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7389000000000001e-05 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680126+0000", 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1069999999999999e-06 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680131+0000", 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5310000000000003e-06 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680178+0000", 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7292000000000001e-05 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680191+0000", 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2585999999999999e-05 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.772967+0000", 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092776485000000006 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773062+0000", 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5056000000000006e-05 2024-09-21T11:48:53.140 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:68278 8.0 e178/160 8:0020ddcf:::benchmark_data_smithi043_97042_object68277:head v 178'4263, mlcod=178'4260)", 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:37.681594+0000", 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 195.351262252, 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.095449811999999995, 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.141 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681594+0000", 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681594+0000", 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681594+0000", 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1999999999999999e-07 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681624+0000", 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9839e-05 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681634+0000", 2024-09-21T11:48:53.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8099999999999992e-06 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681641+0000", 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5610000000000004e-06 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.770591+0000", 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.088950076000000003 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.770613+0000", 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2410999999999999e-05 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.776994+0000", 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0063813350000000001 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.777044+0000", 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9159999999999997e-05 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.145 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:68285 8.0 e178/160 8:0330324f:::benchmark_data_smithi043_97042_object68284:head v 178'4264, mlcod=178'4260)", 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:37.685442+0000", 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 195.34741372900001, 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093833114999999995, 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.685442+0000", 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.685442+0000", 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.685443+0000", 2024-09-21T11:48:53.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1500000000000005e-07 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.685495+0000", 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2326e-05 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.768117+0000", 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.082621662999999998 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.768127+0000", 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0193999999999999e-05 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773140+0000", 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005013399 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773161+0000", 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0126999999999999e-05 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.779234+0000", 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0060736840000000002 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.779275+0000", 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1207e-05 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:123646 8.2 8:4e3911e1:::benchmark_data_smithi043_97042_object123645:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:21.826953+0000", 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 151.20590348299999, 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11253259, 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.151 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123646 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.826953+0000", 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.826953+0000", 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.826954+0000", 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.398e-06 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.826983+0000", 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8931000000000001e-05 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.826993+0000", 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0366e-05 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827000+0000", 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9639999999999997e-06 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827273+0000", 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000272916 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827397+0000", 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012345800000000001 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827532+0000", 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013517600000000001 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.830113+0000", 2024-09-21T11:48:53.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025809880000000002 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.836834+0000", 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.006721744 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939366+0000", 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.102531452 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939426+0000", 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0059999999999998e-05 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939485+0000", 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9137e-05 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.157 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:123647 8.2 8:40a5b330:::benchmark_data_smithi043_97042_object123646:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:21.827047+0000", 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 151.205808752, 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11257774, 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123647 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827047+0000", 2024-09-21T11:48:53.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827047+0000", 2024-09-21T11:48:53.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827048+0000", 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2399999999999998e-07 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827087+0000", 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9209999999999999e-05 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827095+0000", 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3950000000000003e-06 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827100+0000", 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3759999999999999e-06 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.830139+0000", 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030388669999999998 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.830263+0000", 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124721 2024-09-21T11:48:53.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.830393+0000", 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129195 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833041+0000", 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002648451 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.839629+0000", 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0065879090000000003 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939529+0000", 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.099900177000000007 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939588+0000", 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8683e-05 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939625+0000", 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7132000000000003e-05 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:123655 8.2 8:4f314857:::benchmark_data_smithi043_97042_object123654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:21.833269+0000", 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 151.19958734400001, 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.109746765, 2024-09-21T11:48:53.165 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123655 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:48:53.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833269+0000", 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833269+0000", 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833270+0000", 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1880000000000001e-06 2024-09-21T11:48:53.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833325+0000", 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5247000000000001e-05 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833335+0000", 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5410000000000006e-06 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:48:53.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833340+0000", 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.293e-06 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833393+0000", 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3057999999999999e-05 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833521+0000", 2024-09-21T11:48:53.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012788099999999999 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833672+0000", 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015077299999999999 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.837990+0000", 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043184779999999997 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.839933+0000", 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019426389999999999 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.942944+0000", 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.103011493 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:48:53.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.942987+0000", 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2731999999999997e-05 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.943015+0000", 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8442e-05 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:48:53.172 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:48:53.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:53.173 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:53.174 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:53.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:53.297 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:53.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:53.298 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:53.399 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:53.418 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:53.419 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:53.423 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:53.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:53.424 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:53.448 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:48:53.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:53.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:53.547 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:53.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:53.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:53.672 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:53.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:53.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:53.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:53 smithi043 ceph-mon[36454]: pgmap v422: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-21T11:48:53.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:53 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4232373988' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:48:53.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:53 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:48:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:53 smithi043 ceph-mon[29710]: pgmap v422: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-21T11:48:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:53 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4232373988' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:48:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:53 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:48:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:53 smithi175 ceph-mon[32613]: pgmap v422: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-21T11:48:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:53 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4232373988' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:48:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:53 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:48:53.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:53.796 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:53.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:53.797 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:53.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:53.919 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:53.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:53.920 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:54.021 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:54.043 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:54.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:54.044 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:54.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:54.168 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:54.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:54.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:54.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:48:54.269 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:54.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:54.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:54.393 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:54.414 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:54.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:54.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:48:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:48:54.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:54 smithi043 ceph-mon[36454]: 4.a scrub starts 2024-09-21T11:48:54.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:54 smithi043 ceph-mon[36454]: 4.a scrub ok 2024-09-21T11:48:54.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:54 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:54.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:54 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:48:54.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:54 smithi043 ceph-mon[36454]: osdmap e242: 8 total, 8 up, 8 in 2024-09-21T11:48:54.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:54 smithi043 ceph-mon[29710]: 4.a scrub starts 2024-09-21T11:48:54.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:54 smithi043 ceph-mon[29710]: 4.a scrub ok 2024-09-21T11:48:54.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:54 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:54.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:54 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:48:54.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:54 smithi043 ceph-mon[29710]: osdmap e242: 8 total, 8 up, 8 in 2024-09-21T11:48:54.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:54 smithi175 ceph-mon[32613]: 4.a scrub starts 2024-09-21T11:48:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:54 smithi175 ceph-mon[32613]: 4.a scrub ok 2024-09-21T11:48:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:54 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:48:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:54 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:48:54.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:54 smithi175 ceph-mon[32613]: osdmap e242: 8 total, 8 up, 8 in 2024-09-21T11:48:54.514 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:48:54.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:54.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:54.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:54.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:54.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:54.667 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:54.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:54.790 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:54.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:54.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:54.891 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:54.914 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:54.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:54.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:55.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:55.038 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:55.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:55.039 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:55.046 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:48:55.046 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:48:55.046 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:48:55.046 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:48:55.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:55.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:55.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:55.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:55.288 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:55.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:55.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:55.293 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-21T11:48:53.934724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":1359,"num_read_kb":1216,"num_write":274733,"num_write_kb":8775498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24762,"num_bytes_recovered":1606884224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86691,"ondisk_log_size":86691,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":447,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2809300,"kb_used_data":228860,"kb_used_omap":12,"kb_used_meta":2580275,"kb_avail":747151916,"statfs":{"total":767960285184,"available":765083561984,"internally_reserved":0,"allocated":234352640,"data_stored":231285217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2642202162},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-585695256,"num_objects":-8938,"num_object_clones":0,"num_object_copies":-26814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8938,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":8938,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003179"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":395,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456398+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:48:49.456398+0000","last_peered":"2024-09-21T11:48:49.456398+0000","last_clean":"2024-09-21T11:48:49.456398+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:48:49.456398+0000","last_undegraded":"2024-09-21T11:48:49.456398+0000","last_fullsized":"2024-09-21T11:48:49.456398+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.863735+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":449,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.435367+0000","last_change":"2024-09-21T11:47:03.433310+0000","last_active":"2024-09-21T11:48:49.435367+0000","last_peered":"2024-09-21T11:48:49.435367+0000","last_clean":"2024-09-21T11:48:49.435367+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:48:49.435367+0000","last_undegraded":"2024-09-21T11:48:49.435367+0000","last_fullsized":"2024-09-21T11:48:49.435367+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:44.553561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.553561+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.027789+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":88,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.446701+0000","last_change":"2024-09-21T11:47:03.372676+0000","last_active":"2024-09-21T11:48:49.446701+0000","last_peered":"2024-09-21T11:48:49.446701+0000","last_clean":"2024-09-21T11:48:49.446701+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:48:49.446701+0000","last_undegraded":"2024-09-21T11:48:49.446701+0000","last_fullsized":"2024-09-21T11:48:49.446701+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.453795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":405,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436101+0000","last_change":"2024-09-21T11:47:20.957722+0000","last_active":"2024-09-21T11:48:49.436101+0000","last_peered":"2024-09-21T11:48:49.436101+0000","last_clean":"2024-09-21T11:48:49.436101+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:48:49.436101+0000","last_undegraded":"2024-09-21T11:48:49.436101+0000","last_fullsized":"2024-09-21T11:48:49.436101+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:20.957679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.957679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":383,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.454976+0000","last_change":"2024-09-21T11:46:09.675727+0000","last_active":"2024-09-21T11:48:49.454976+0000","last_peered":"2024-09-21T11:48:49.454976+0000","last_clean":"2024-09-21T11:48:49.454976+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:48:49.454976+0000","last_undegraded":"2024-09-21T11:48:49.454976+0000","last_fullsized":"2024-09-21T11:48:49.454976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:09.675148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:09.675148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":388,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.452292+0000","last_change":"2024-09-21T11:46:55.804484+0000","last_active":"2024-09-21T11:48:49.452292+0000","last_peered":"2024-09-21T11:48:49.452292+0000","last_clean":"2024-09-21T11:48:49.452292+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:48:49.452292+0000","last_undegraded":"2024-09-21T11:48:49.452292+0000","last_fullsized":"2024-09-21T11:48:49.452292+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.804419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.804419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":408,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.452605+0000","last_change":"2024-09-21T11:45:07.788590+0000","last_active":"2024-09-21T11:48:49.452605+0000","last_peered":"2024-09-21T11:48:49.452605+0000","last_clean":"2024-09-21T11:48:49.452605+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:48:49.452605+0000","last_undegraded":"2024-09-21T11:48:49.452605+0000","last_fullsized":"2024-09-21T11:48:49.452605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:07.788512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.788512+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":409,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.435316+0000","last_change":"2024-09-21T11:47:03.371710+0000","last_active":"2024-09-21T11:48:49.435316+0000","last_peered":"2024-09-21T11:48:49.435316+0000","last_clean":"2024-09-21T11:48:49.435316+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:48:49.435316+0000","last_undegraded":"2024-09-21T11:48:49.435316+0000","last_fullsized":"2024-09-21T11:48:49.435316+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.957478+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":394,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.441309+0000","last_change":"2024-09-21T11:48:47.077320+0000","last_active":"2024-09-21T11:48:49.441309+0000","last_peered":"2024-09-21T11:48:49.441309+0000","last_clean":"2024-09-21T11:48:49.441309+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:48:49.441309+0000","last_undegraded":"2024-09-21T11:48:49.441309+0000","last_fullsized":"2024-09-21T11:48:49.441309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.077259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.077259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:00.730980+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":415,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436435+0000","last_change":"2024-09-21T11:48:13.962997+0000","last_active":"2024-09-21T11:48:49.436435+0000","last_peered":"2024-09-21T11:48:49.436435+0000","last_clean":"2024-09-21T11:48:49.436435+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:48:49.436435+0000","last_undegraded":"2024-09-21T11:48:49.436435+0000","last_fullsized":"2024-09-21T11:48:49.436435+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:13.962919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:13.962919+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:14.418938+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":394,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.440688+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:48:49.440688+0000","last_peered":"2024-09-21T11:48:49.440688+0000","last_clean":"2024-09-21T11:48:49.440688+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:48:49.440688+0000","last_undegraded":"2024-09-21T11:48:49.440688+0000","last_fullsized":"2024-09-21T11:48:49.440688+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.033067+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":404,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456563+0000","last_change":"2024-09-21T11:48:44.523906+0000","last_active":"2024-09-21T11:48:49.456563+0000","last_peered":"2024-09-21T11:48:49.456563+0000","last_clean":"2024-09-21T11:48:49.456563+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:48:49.456563+0000","last_undegraded":"2024-09-21T11:48:49.456563+0000","last_fullsized":"2024-09-21T11:48:49.456563+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:44.523826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.523826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:11.435468+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":416,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.440802+0000","last_change":"2024-09-21T11:47:03.385245+0000","last_active":"2024-09-21T11:48:49.440802+0000","last_peered":"2024-09-21T11:48:49.440802+0000","last_clean":"2024-09-21T11:48:49.440802+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:48:49.440802+0000","last_undegraded":"2024-09-21T11:48:49.440802+0000","last_fullsized":"2024-09-21T11:48:49.440802+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.017967+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":435,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.435789+0000","last_change":"2024-09-21T11:48:13.005365+0000","last_active":"2024-09-21T11:48:49.435789+0000","last_peered":"2024-09-21T11:48:49.435789+0000","last_clean":"2024-09-21T11:48:49.435789+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:48:49.435789+0000","last_undegraded":"2024-09-21T11:48:49.435789+0000","last_fullsized":"2024-09-21T11:48:49.435789+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:48:13.005294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:13.005294+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:13.822018+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":400,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.447791+0000","last_change":"2024-09-21T11:48:06.605705+0000","last_active":"2024-09-21T11:48:49.447791+0000","last_peered":"2024-09-21T11:48:49.447791+0000","last_clean":"2024-09-21T11:48:49.447791+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:48:49.447791+0000","last_undegraded":"2024-09-21T11:48:49.447791+0000","last_fullsized":"2024-09-21T11:48:49.447791+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:06.605645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.605645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.086689+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":404,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.440332+0000","last_change":"2024-09-21T11:47:03.380856+0000","last_active":"2024-09-21T11:48:49.440332+0000","last_peered":"2024-09-21T11:48:49.440332+0000","last_clean":"2024-09-21T11:48:49.440332+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:48:49.440332+0000","last_undegraded":"2024-09-21T11:48:49.440332+0000","last_fullsized":"2024-09-21T11:48:49.440332+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:13.288522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:13.288522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.041204+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":400,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436417+0000","last_change":"2024-09-21T11:46:52.035524+0000","last_active":"2024-09-21T11:48:49.436417+0000","last_peered":"2024-09-21T11:48:49.436417+0000","last_clean":"2024-09-21T11:48:49.436417+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:48:49.436417+0000","last_undegraded":"2024-09-21T11:48:49.436417+0000","last_fullsized":"2024-09-21T11:48:49.436417+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.035461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.035461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:30.990978+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":386,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456486+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:48:49.456486+0000","last_peered":"2024-09-21T11:48:49.456486+0000","last_clean":"2024-09-21T11:48:49.456486+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:48:49.456486+0000","last_undegraded":"2024-09-21T11:48:49.456486+0000","last_fullsized":"2024-09-21T11:48:49.456486+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.281211+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":501,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.447680+0000","last_change":"2024-09-21T11:48:18.120641+0000","last_active":"2024-09-21T11:48:49.447680+0000","last_peered":"2024-09-21T11:48:49.447680+0000","last_clean":"2024-09-21T11:48:49.447680+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:48:49.447680+0000","last_undegraded":"2024-09-21T11:48:49.447680+0000","last_fullsized":"2024-09-21T11:48:49.447680+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:48:18.120532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:18.120532+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:22.674543+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":88,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.453660+0000","last_change":"2024-09-21T11:47:03.377667+0000","last_active":"2024-09-21T11:48:49.453660+0000","last_peered":"2024-09-21T11:48:49.453660+0000","last_clean":"2024-09-21T11:48:49.453660+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:48:49.453660+0000","last_undegraded":"2024-09-21T11:48:49.453660+0000","last_fullsized":"2024-09-21T11:48:49.453660+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.718478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.718478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.452839+0000","last_change":"2024-09-21T11:45:39.826902+0000","last_active":"2024-09-21T11:48:49.452839+0000","last_peered":"2024-09-21T11:48:49.452839+0000","last_clean":"2024-09-21T11:48:49.452839+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:48:49.452839+0000","last_undegraded":"2024-09-21T11:48:49.452839+0000","last_fullsized":"2024-09-21T11:48:49.452839+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:39.826701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:39.826701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":409,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436562+0000","last_change":"2024-09-21T11:48:22.941031+0000","last_active":"2024-09-21T11:48:49.436562+0000","last_peered":"2024-09-21T11:48:49.436562+0000","last_clean":"2024-09-21T11:48:49.436562+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:48:49.436562+0000","last_undegraded":"2024-09-21T11:48:49.436562+0000","last_fullsized":"2024-09-21T11:48:49.436562+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:22.940968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:22.940968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:48.005429+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":452,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.454768+0000","last_change":"2024-09-21T11:47:03.873307+0000","last_active":"2024-09-21T11:48:49.454768+0000","last_peered":"2024-09-21T11:48:49.454768+0000","last_clean":"2024-09-21T11:48:49.454768+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:48:49.454768+0000","last_undegraded":"2024-09-21T11:48:49.454768+0000","last_fullsized":"2024-09-21T11:48:49.454768+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:47:03.873239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.873239+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":387,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436150+0000","last_change":"2024-09-21T11:45:37.670176+0000","last_active":"2024-09-21T11:48:49.436150+0000","last_peered":"2024-09-21T11:48:49.436150+0000","last_clean":"2024-09-21T11:48:49.436150+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:48:49.436150+0000","last_undegraded":"2024-09-21T11:48:49.436150+0000","last_fullsized":"2024-09-21T11:48:49.436150+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:37.670117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:37.670117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:50.969760+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":399,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.448162+0000","last_change":"2024-09-21T11:46:47.611118+0000","last_active":"2024-09-21T11:48:49.448162+0000","last_peered":"2024-09-21T11:48:49.448162+0000","last_clean":"2024-09-21T11:48:49.448162+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:48:49.448162+0000","last_undegraded":"2024-09-21T11:48:49.448162+0000","last_fullsized":"2024-09-21T11:48:49.448162+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:46:47.611042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:47.611042+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.533011+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":92,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.446975+0000","last_change":"2024-09-21T11:47:03.388975+0000","last_active":"2024-09-21T11:48:49.446975+0000","last_peered":"2024-09-21T11:48:49.446975+0000","last_clean":"2024-09-21T11:48:49.446975+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:48:49.446975+0000","last_undegraded":"2024-09-21T11:48:49.446975+0000","last_fullsized":"2024-09-21T11:48:49.446975+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:12.636105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.636105+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":414,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.462679+0000","last_change":"2024-09-21T11:45:48.385970+0000","last_active":"2024-09-21T11:48:49.462679+0000","last_peered":"2024-09-21T11:48:49.462679+0000","last_clean":"2024-09-21T11:48:49.462679+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:48:49.462679+0000","last_undegraded":"2024-09-21T11:48:49.462679+0000","last_fullsized":"2024-09-21T11:48:49.462679+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:45:48.385895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:48.385895+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":529,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.443718+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:48:49.443718+0000","last_peered":"2024-09-21T11:48:49.443718+0000","last_clean":"2024-09-21T11:48:49.443718+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:48:49.443718+0000","last_undegraded":"2024-09-21T11:48:49.443718+0000","last_fullsized":"2024-09-21T11:48:49.443718+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.120507+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"241'17186","reported_seq":22958,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.604996+0000","last_change":"2024-09-21T11:48:44.931648+0000","last_active":"2024-09-21T11:48:49.604996+0000","last_peered":"2024-09-21T11:48:49.604996+0000","last_clean":"2024-09-21T11:48:49.604996+0000","last_became_active":"2024-09-21T11:48:17.482971+0000","last_became_peered":"2024-09-21T11:48:17.482971+0000","last_unstale":"2024-09-21T11:48:49.604996+0000","last_undegraded":"2024-09-21T11:48:49.604996+0000","last_fullsized":"2024-09-21T11:48:49.604996+0000","mapping_epoch":230,"log_start":"213'12345","ondisk_log_start":"213'12345","created":160,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'16847","last_scrub_stamp":"2024-09-21T11:48:44.929632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.929632+0000","objects_scrubbed":347,"log_size":4841,"log_dups_size":0,"ondisk_log_size":4841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:00.630166+0000","scrub_duration":281,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17186,"num_write_kb":549952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2785,"num_bytes_recovered":182517760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":104,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.450175+0000","last_change":"2024-09-21T11:47:03.400465+0000","last_active":"2024-09-21T11:48:49.450175+0000","last_peered":"2024-09-21T11:48:49.450175+0000","last_clean":"2024-09-21T11:48:49.450175+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:48:49.450175+0000","last_undegraded":"2024-09-21T11:48:49.450175+0000","last_fullsized":"2024-09-21T11:48:49.450175+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:22.708839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.708839+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":418,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.454276+0000","last_change":"2024-09-21T11:47:03.391541+0000","last_active":"2024-09-21T11:48:49.454276+0000","last_peered":"2024-09-21T11:48:49.454276+0000","last_clean":"2024-09-21T11:48:49.454276+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:48:49.454276+0000","last_undegraded":"2024-09-21T11:48:49.454276+0000","last_fullsized":"2024-09-21T11:48:49.454276+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.836903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.836903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":502,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436514+0000","last_change":"2024-09-21T11:47:01.933697+0000","last_active":"2024-09-21T11:48:49.436514+0000","last_peered":"2024-09-21T11:48:49.436514+0000","last_clean":"2024-09-21T11:48:49.436514+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:48:49.436514+0000","last_undegraded":"2024-09-21T11:48:49.436514+0000","last_fullsized":"2024-09-21T11:48:49.436514+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:01.933648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:01.933648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":396,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.440978+0000","last_change":"2024-09-21T11:48:15.013107+0000","last_active":"2024-09-21T11:48:49.440978+0000","last_peered":"2024-09-21T11:48:49.440978+0000","last_clean":"2024-09-21T11:48:49.440978+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:48:49.440978+0000","last_undegraded":"2024-09-21T11:48:49.440978+0000","last_fullsized":"2024-09-21T11:48:49.440978+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:15.013058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:15.013058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.125286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"241'17158","reported_seq":17416,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.577017+0000","last_change":"2024-09-21T11:46:19.360961+0000","last_active":"2024-09-21T11:48:49.577017+0000","last_peered":"2024-09-21T11:48:49.577017+0000","last_clean":"2024-09-21T11:48:49.577017+0000","last_became_active":"2024-09-21T11:44:51.744224+0000","last_became_peered":"2024-09-21T11:44:51.744224+0000","last_unstale":"2024-09-21T11:48:49.577017+0000","last_undegraded":"2024-09-21T11:48:49.577017+0000","last_fullsized":"2024-09-21T11:48:49.577017+0000","mapping_epoch":160,"log_start":"205'11300","ondisk_log_start":"205'11300","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7561","last_scrub_stamp":"2024-09-21T11:46:19.360902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.360902+0000","objects_scrubbed":7459,"log_size":5858,"log_dups_size":0,"ondisk_log_size":5858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2269,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17158,"num_write_kb":549056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":432,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.453977+0000","last_change":"2024-09-21T11:46:58.789719+0000","last_active":"2024-09-21T11:48:49.453977+0000","last_peered":"2024-09-21T11:48:49.453977+0000","last_clean":"2024-09-21T11:48:49.453977+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:48:49.453977+0000","last_undegraded":"2024-09-21T11:48:49.453977+0000","last_fullsized":"2024-09-21T11:48:49.453977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:46:58.789625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.789625+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":404,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.453535+0000","last_change":"2024-09-21T11:46:52.891978+0000","last_active":"2024-09-21T11:48:49.453535+0000","last_peered":"2024-09-21T11:48:49.453535+0000","last_clean":"2024-09-21T11:48:49.453535+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:48:49.453535+0000","last_undegraded":"2024-09-21T11:48:49.453535+0000","last_fullsized":"2024-09-21T11:48:49.453535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.891793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.891793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":396,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.435534+0000","last_change":"2024-09-21T11:48:47.933145+0000","last_active":"2024-09-21T11:48:49.435534+0000","last_peered":"2024-09-21T11:48:49.435534+0000","last_clean":"2024-09-21T11:48:49.435534+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:48:49.435534+0000","last_undegraded":"2024-09-21T11:48:49.435534+0000","last_fullsized":"2024-09-21T11:48:49.435534+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.933099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.933099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:15.039482+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436287+0000","last_change":"2024-09-21T11:46:24.717355+0000","last_active":"2024-09-21T11:48:49.436287+0000","last_peered":"2024-09-21T11:48:49.436287+0000","last_clean":"2024-09-21T11:48:49.436287+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:48:49.436287+0000","last_undegraded":"2024-09-21T11:48:49.436287+0000","last_fullsized":"2024-09-21T11:48:49.436287+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.717285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.717285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:58.045440+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"241'17056","reported_seq":17268,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.618692+0000","last_change":"2024-09-21T11:48:05.746078+0000","last_active":"2024-09-21T11:48:49.618692+0000","last_peered":"2024-09-21T11:48:49.618692+0000","last_clean":"2024-09-21T11:48:49.618692+0000","last_became_active":"2024-09-21T11:44:51.744633+0000","last_became_peered":"2024-09-21T11:44:51.744633+0000","last_unstale":"2024-09-21T11:48:49.618692+0000","last_undegraded":"2024-09-21T11:48:49.618692+0000","last_fullsized":"2024-09-21T11:48:49.618692+0000","mapping_epoch":160,"log_start":"208'11641","ondisk_log_start":"208'11641","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"225'14372","last_scrub_stamp":"2024-09-21T11:48:05.746017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.746017+0000","objects_scrubbed":2702,"log_size":5415,"log_dups_size":0,"ondisk_log_size":5415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:10.815328+0000","scrub_duration":670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17056,"num_write_kb":545729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":429,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.435621+0000","last_change":"2024-09-21T11:47:33.018713+0000","last_active":"2024-09-21T11:48:49.435621+0000","last_peered":"2024-09-21T11:48:49.435621+0000","last_clean":"2024-09-21T11:48:49.435621+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:48:49.435621+0000","last_undegraded":"2024-09-21T11:48:49.435621+0000","last_fullsized":"2024-09-21T11:48:49.435621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:47:33.018639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:33.018639+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":426,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.462932+0000","last_change":"2024-09-21T11:47:03.429013+0000","last_active":"2024-09-21T11:48:49.462932+0000","last_peered":"2024-09-21T11:48:49.462932+0000","last_clean":"2024-09-21T11:48:49.462932+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:48:49.462932+0000","last_undegraded":"2024-09-21T11:48:49.462932+0000","last_fullsized":"2024-09-21T11:48:49.462932+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:59.702813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.702813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":526,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.453844+0000","last_change":"2024-09-21T11:47:03.406087+0000","last_active":"2024-09-21T11:48:49.453844+0000","last_peered":"2024-09-21T11:48:49.453844+0000","last_clean":"2024-09-21T11:48:49.453844+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:48:49.453844+0000","last_undegraded":"2024-09-21T11:48:49.453844+0000","last_fullsized":"2024-09-21T11:48:49.453844+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:08.620519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.620519+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":386,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.447321+0000","last_change":"2024-09-21T11:48:10.667288+0000","last_active":"2024-09-21T11:48:49.447321+0000","last_peered":"2024-09-21T11:48:49.447321+0000","last_clean":"2024-09-21T11:48:49.447321+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:48:49.447321+0000","last_undegraded":"2024-09-21T11:48:49.447321+0000","last_fullsized":"2024-09-21T11:48:49.447321+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.667221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.667221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:11.606721+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"241'17422","reported_seq":17684,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.606623+0000","last_change":"2024-09-21T11:46:22.226583+0000","last_active":"2024-09-21T11:48:49.606623+0000","last_peered":"2024-09-21T11:48:49.606623+0000","last_clean":"2024-09-21T11:48:49.606623+0000","last_became_active":"2024-09-21T11:44:51.412662+0000","last_became_peered":"2024-09-21T11:44:51.412662+0000","last_unstale":"2024-09-21T11:48:49.606623+0000","last_undegraded":"2024-09-21T11:48:49.606623+0000","last_fullsized":"2024-09-21T11:48:49.606623+0000","mapping_epoch":160,"log_start":"207'11600","ondisk_log_start":"207'11600","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7855","last_scrub_stamp":"2024-09-21T11:46:22.226400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.226400+0000","objects_scrubbed":7719,"log_size":5822,"log_dups_size":0,"ondisk_log_size":5822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3159,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17422,"num_write_kb":557504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":100,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.442829+0000","last_change":"2024-09-21T11:47:03.398638+0000","last_active":"2024-09-21T11:48:49.442829+0000","last_peered":"2024-09-21T11:48:49.442829+0000","last_clean":"2024-09-21T11:48:49.442829+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:48:49.442829+0000","last_undegraded":"2024-09-21T11:48:49.442829+0000","last_fullsized":"2024-09-21T11:48:49.442829+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:12.227121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.227121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":411,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.442321+0000","last_change":"2024-09-21T11:48:07.131578+0000","last_active":"2024-09-21T11:48:49.442321+0000","last_peered":"2024-09-21T11:48:49.442321+0000","last_clean":"2024-09-21T11:48:49.442321+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:48:49.442321+0000","last_undegraded":"2024-09-21T11:48:49.442321+0000","last_fullsized":"2024-09-21T11:48:49.442321+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.131535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.131535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:08.103888+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":497,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456046+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:48:49.456046+0000","last_peered":"2024-09-21T11:48:49.456046+0000","last_clean":"2024-09-21T11:48:49.456046+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:48:49.456046+0000","last_undegraded":"2024-09-21T11:48:49.456046+0000","last_fullsized":"2024-09-21T11:48:49.456046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.263748+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":383,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.457107+0000","last_change":"2024-09-21T11:46:28.010676+0000","last_active":"2024-09-21T11:48:49.457107+0000","last_peered":"2024-09-21T11:48:49.457107+0000","last_clean":"2024-09-21T11:48:49.457107+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:48:49.457107+0000","last_undegraded":"2024-09-21T11:48:49.457107+0000","last_fullsized":"2024-09-21T11:48:49.457107+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:28.010602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.010602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"241'17048","reported_seq":26203,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.630246+0000","last_change":"2024-09-21T11:47:54.309078+0000","last_active":"2024-09-21T11:48:49.630246+0000","last_peered":"2024-09-21T11:48:49.630246+0000","last_clean":"2024-09-21T11:48:49.630246+0000","last_became_active":"2024-09-21T11:47:54.270690+0000","last_became_peered":"2024-09-21T11:47:54.270690+0000","last_unstale":"2024-09-21T11:48:49.630246+0000","last_undegraded":"2024-09-21T11:48:49.630246+0000","last_fullsized":"2024-09-21T11:48:49.630246+0000","mapping_epoch":222,"log_start":"211'12000","ondisk_log_start":"211'12000","created":160,"last_epoch_clean":223,"parent":"0.0","parent_split_bits":0,"last_scrub":"193'7360","last_scrub_stamp":"2024-09-21T11:46:17.038356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.038356+0000","objects_scrubbed":7242,"log_size":5048,"log_dups_size":0,"ondisk_log_size":5048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3381,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17048,"num_write_kb":545536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4441,"num_bytes_recovered":291045376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":427,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456200+0000","last_change":"2024-09-21T11:47:03.750064+0000","last_active":"2024-09-21T11:48:49.456200+0000","last_peered":"2024-09-21T11:48:49.456200+0000","last_clean":"2024-09-21T11:48:49.456200+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:48:49.456200+0000","last_undegraded":"2024-09-21T11:48:49.456200+0000","last_fullsized":"2024-09-21T11:48:49.456200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:47:03.749962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.749962+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":88,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.454267+0000","last_change":"2024-09-21T11:47:03.382675+0000","last_active":"2024-09-21T11:48:49.454267+0000","last_peered":"2024-09-21T11:48:49.454267+0000","last_clean":"2024-09-21T11:48:49.454267+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:48:49.454267+0000","last_undegraded":"2024-09-21T11:48:49.454267+0000","last_fullsized":"2024-09-21T11:48:49.454267+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:08.723593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.723593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":387,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.442494+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:48:49.442494+0000","last_peered":"2024-09-21T11:48:49.442494+0000","last_clean":"2024-09-21T11:48:49.442494+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:48:49.442494+0000","last_undegraded":"2024-09-21T11:48:49.442494+0000","last_fullsized":"2024-09-21T11:48:49.442494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.032409+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":88,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.446607+0000","last_change":"2024-09-21T11:47:03.379157+0000","last_active":"2024-09-21T11:48:49.446607+0000","last_peered":"2024-09-21T11:48:49.446607+0000","last_clean":"2024-09-21T11:48:49.446607+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:48:49.446607+0000","last_undegraded":"2024-09-21T11:48:49.446607+0000","last_fullsized":"2024-09-21T11:48:49.446607+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.475150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"241'17256","reported_seq":17431,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.605143+0000","last_change":"2024-09-21T11:44:51.745506+0000","last_active":"2024-09-21T11:48:49.605143+0000","last_peered":"2024-09-21T11:48:49.605143+0000","last_clean":"2024-09-21T11:48:49.605143+0000","last_became_active":"2024-09-21T11:44:51.745200+0000","last_became_peered":"2024-09-21T11:44:51.745200+0000","last_unstale":"2024-09-21T11:48:49.605143+0000","last_undegraded":"2024-09-21T11:48:49.605143+0000","last_fullsized":"2024-09-21T11:48:49.605143+0000","mapping_epoch":160,"log_start":"208'11800","ondisk_log_start":"208'11800","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17256,"num_write_kb":552192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"205'31","reported_seq":132,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.454205+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:48:49.454205+0000","last_peered":"2024-09-21T11:48:49.454205+0000","last_clean":"2024-09-21T11:48:49.454205+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:48:49.454205+0000","last_undegraded":"2024-09-21T11:48:49.454205+0000","last_fullsized":"2024-09-21T11:48:49.454205+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.477411+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":394,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436168+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:48:49.436168+0000","last_peered":"2024-09-21T11:48:49.436168+0000","last_clean":"2024-09-21T11:48:49.436168+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:48:49.436168+0000","last_undegraded":"2024-09-21T11:48:49.436168+0000","last_fullsized":"2024-09-21T11:48:49.436168+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.947561+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":427,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.442158+0000","last_change":"2024-09-21T11:47:20.000195+0000","last_active":"2024-09-21T11:48:49.442158+0000","last_peered":"2024-09-21T11:48:49.442158+0000","last_clean":"2024-09-21T11:48:49.442158+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:48:49.442158+0000","last_undegraded":"2024-09-21T11:48:49.442158+0000","last_fullsized":"2024-09-21T11:48:49.442158+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:47:20.000051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.000051+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":401,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456070+0000","last_change":"2024-09-21T11:47:03.385173+0000","last_active":"2024-09-21T11:48:49.456070+0000","last_peered":"2024-09-21T11:48:49.456070+0000","last_clean":"2024-09-21T11:48:49.456070+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:48:49.456070+0000","last_undegraded":"2024-09-21T11:48:49.456070+0000","last_fullsized":"2024-09-21T11:48:49.456070+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:32.321908+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":376,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.452096+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:48:49.452096+0000","last_peered":"2024-09-21T11:48:49.452096+0000","last_clean":"2024-09-21T11:48:49.452096+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:48:49.452096+0000","last_undegraded":"2024-09-21T11:48:49.452096+0000","last_fullsized":"2024-09-21T11:48:49.452096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.188178+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"241'16978","reported_seq":21500,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.626472+0000","last_change":"2024-09-21T11:48:24.579575+0000","last_active":"2024-09-21T11:48:49.626472+0000","last_peered":"2024-09-21T11:48:49.626472+0000","last_clean":"2024-09-21T11:48:49.626472+0000","last_became_active":"2024-09-21T11:48:24.535114+0000","last_became_peered":"2024-09-21T11:48:24.535114+0000","last_unstale":"2024-09-21T11:48:49.626472+0000","last_undegraded":"2024-09-21T11:48:49.626472+0000","last_fullsized":"2024-09-21T11:48:49.626472+0000","mapping_epoch":233,"log_start":"212'11900","ondisk_log_start":"212'11900","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5078,"log_dups_size":0,"ondisk_log_size":5078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16978,"num_write_kb":543296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":141754368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":34,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.449884+0000","last_change":"2024-09-21T11:48:33.075081+0000","last_active":"2024-09-21T11:48:49.449884+0000","last_peered":"2024-09-21T11:48:49.449884+0000","last_clean":"2024-09-21T11:48:49.449884+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:48:49.449884+0000","last_undegraded":"2024-09-21T11:48:49.449884+0000","last_fullsized":"2024-09-21T11:48:49.449884+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.876172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":92,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.448705+0000","last_change":"2024-09-21T11:47:03.391283+0000","last_active":"2024-09-21T11:48:49.448705+0000","last_peered":"2024-09-21T11:48:49.448705+0000","last_clean":"2024-09-21T11:48:49.448705+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:48:49.448705+0000","last_undegraded":"2024-09-21T11:48:49.448705+0000","last_fullsized":"2024-09-21T11:48:49.448705+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.552518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":421,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.445784+0000","last_change":"2024-09-21T11:48:48.994249+0000","last_active":"2024-09-21T11:48:49.445784+0000","last_peered":"2024-09-21T11:48:49.445784+0000","last_clean":"2024-09-21T11:48:49.445784+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:48:49.445784+0000","last_undegraded":"2024-09-21T11:48:49.445784+0000","last_fullsized":"2024-09-21T11:48:49.445784+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:48:48.994172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:48:48.994172+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:11.491020+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":396,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.445537+0000","last_change":"2024-09-21T11:48:16.076049+0000","last_active":"2024-09-21T11:48:49.445537+0000","last_peered":"2024-09-21T11:48:49.445537+0000","last_clean":"2024-09-21T11:48:49.445537+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:48:49.445537+0000","last_undegraded":"2024-09-21T11:48:49.445537+0000","last_fullsized":"2024-09-21T11:48:49.445537+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:16.076003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:16.076003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.071160+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":391,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.449743+0000","last_change":"2024-09-21T11:48:07.613756+0000","last_active":"2024-09-21T11:48:49.449743+0000","last_peered":"2024-09-21T11:48:49.449743+0000","last_clean":"2024-09-21T11:48:49.449743+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:48:49.449743+0000","last_undegraded":"2024-09-21T11:48:49.449743+0000","last_fullsized":"2024-09-21T11:48:49.449743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.613691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.613691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:32.231718+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"241'16882","reported_seq":26350,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.611105+0000","last_change":"2024-09-21T11:48:43.896691+0000","last_active":"2024-09-21T11:48:49.611105+0000","last_peered":"2024-09-21T11:48:49.611105+0000","last_clean":"2024-09-21T11:48:49.611105+0000","last_became_active":"2024-09-21T11:47:52.264063+0000","last_became_peered":"2024-09-21T11:47:52.264063+0000","last_unstale":"2024-09-21T11:48:49.611105+0000","last_undegraded":"2024-09-21T11:48:49.611105+0000","last_fullsized":"2024-09-21T11:48:49.611105+0000","mapping_epoch":220,"log_start":"212'12006","ondisk_log_start":"212'12006","created":160,"last_epoch_clean":221,"parent":"0.0","parent_split_bits":0,"last_scrub":"239'16501","last_scrub_stamp":"2024-09-21T11:48:43.896523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:43.896523+0000","objects_scrubbed":389,"log_size":4876,"log_dups_size":0,"ondisk_log_size":4876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:49.832412+0000","scrub_duration":249,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16882,"num_write_kb":540224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4633,"num_bytes_recovered":303628288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":451,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.462766+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:48:49.462766+0000","last_peered":"2024-09-21T11:48:49.462766+0000","last_clean":"2024-09-21T11:48:49.462766+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:48:49.462766+0000","last_undegraded":"2024-09-21T11:48:49.462766+0000","last_fullsized":"2024-09-21T11:48:49.462766+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.401660+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":403,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.458024+0000","last_change":"2024-09-21T11:46:29.937866+0000","last_active":"2024-09-21T11:48:49.458024+0000","last_peered":"2024-09-21T11:48:49.458024+0000","last_clean":"2024-09-21T11:48:49.458024+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:48:49.458024+0000","last_undegraded":"2024-09-21T11:48:49.458024+0000","last_fullsized":"2024-09-21T11:48:49.458024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.937799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.937799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":968,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.507061+0000","last_change":"2024-09-21T11:48:17.596682+0000","last_active":"2024-09-21T11:48:49.507061+0000","last_peered":"2024-09-21T11:48:49.507061+0000","last_clean":"2024-09-21T11:48:49.507061+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:48:49.507061+0000","last_undegraded":"2024-09-21T11:48:49.507061+0000","last_fullsized":"2024-09-21T11:48:49.507061+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:48:17.596571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.596571+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.625607+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":522,"num_read_kb":517,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":409,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.441073+0000","last_change":"2024-09-21T11:48:42.042870+0000","last_active":"2024-09-21T11:48:49.441073+0000","last_peered":"2024-09-21T11:48:49.441073+0000","last_clean":"2024-09-21T11:48:49.441073+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:48:49.441073+0000","last_undegraded":"2024-09-21T11:48:49.441073+0000","last_fullsized":"2024-09-21T11:48:49.441073+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:42.042814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:42.042814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:09.988920+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":88,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.454074+0000","last_change":"2024-09-21T11:47:03.374507+0000","last_active":"2024-09-21T11:48:49.454074+0000","last_peered":"2024-09-21T11:48:49.454074+0000","last_clean":"2024-09-21T11:48:49.454074+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:48:49.454074+0000","last_undegraded":"2024-09-21T11:48:49.454074+0000","last_fullsized":"2024-09-21T11:48:49.454074+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.599936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"241'16970","reported_seq":17278,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.602258+0000","last_change":"2024-09-21T11:47:31.508923+0000","last_active":"2024-09-21T11:48:49.602258+0000","last_peered":"2024-09-21T11:48:49.602258+0000","last_clean":"2024-09-21T11:48:49.602258+0000","last_became_active":"2024-09-21T11:44:51.410701+0000","last_became_peered":"2024-09-21T11:44:51.410701+0000","last_unstale":"2024-09-21T11:48:49.602258+0000","last_undegraded":"2024-09-21T11:48:49.602258+0000","last_fullsized":"2024-09-21T11:48:49.602258+0000","mapping_epoch":160,"log_start":"199'10500","ondisk_log_start":"199'10500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"215'12576","last_scrub_stamp":"2024-09-21T11:47:31.508822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:47:31.508822+0000","objects_scrubbed":4421,"log_size":6470,"log_dups_size":0,"ondisk_log_size":6470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1214,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16970,"num_write_kb":543040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":520,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.443414+0000","last_change":"2024-09-21T11:48:06.171363+0000","last_active":"2024-09-21T11:48:49.443414+0000","last_peered":"2024-09-21T11:48:49.443414+0000","last_clean":"2024-09-21T11:48:49.443414+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:48:49.443414+0000","last_undegraded":"2024-09-21T11:48:49.443414+0000","last_fullsized":"2024-09-21T11:48:49.443414+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:48:06.171279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.171279+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:12.520316+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":404,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.451745+0000","last_change":"2024-09-21T11:46:57.833589+0000","last_active":"2024-09-21T11:48:49.451745+0000","last_peered":"2024-09-21T11:48:49.451745+0000","last_clean":"2024-09-21T11:48:49.451745+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:48:49.451745+0000","last_undegraded":"2024-09-21T11:48:49.451745+0000","last_fullsized":"2024-09-21T11:48:49.451745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:57.833329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.833329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":457,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.447688+0000","last_change":"2024-09-21T11:46:58.645189+0000","last_active":"2024-09-21T11:48:49.447688+0000","last_peered":"2024-09-21T11:48:49.447688+0000","last_clean":"2024-09-21T11:48:49.447688+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:48:49.447688+0000","last_undegraded":"2024-09-21T11:48:49.447688+0000","last_fullsized":"2024-09-21T11:48:49.447688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:46:58.645122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.645122+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.573944+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":395,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.448529+0000","last_change":"2024-09-21T11:47:25.395237+0000","last_active":"2024-09-21T11:48:49.448529+0000","last_peered":"2024-09-21T11:48:49.448529+0000","last_clean":"2024-09-21T11:48:49.448529+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:48:49.448529+0000","last_undegraded":"2024-09-21T11:48:49.448529+0000","last_fullsized":"2024-09-21T11:48:49.448529+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.395185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.395185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":391,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.440957+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:48:49.440957+0000","last_peered":"2024-09-21T11:48:49.440957+0000","last_clean":"2024-09-21T11:48:49.440957+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:48:49.440957+0000","last_undegraded":"2024-09-21T11:48:49.440957+0000","last_fullsized":"2024-09-21T11:48:49.440957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:30.020245+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"241'16998","reported_seq":17287,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.576920+0000","last_change":"2024-09-21T11:48:09.830854+0000","last_active":"2024-09-21T11:48:49.576920+0000","last_peered":"2024-09-21T11:48:49.576920+0000","last_clean":"2024-09-21T11:48:49.576920+0000","last_became_active":"2024-09-21T11:44:51.410197+0000","last_became_peered":"2024-09-21T11:44:51.410197+0000","last_unstale":"2024-09-21T11:48:49.576920+0000","last_undegraded":"2024-09-21T11:48:49.576920+0000","last_fullsized":"2024-09-21T11:48:49.576920+0000","mapping_epoch":160,"log_start":"212'12200","ondisk_log_start":"212'12200","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"226'14544","last_scrub_stamp":"2024-09-21T11:48:09.830774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.830774+0000","objects_scrubbed":2470,"log_size":4798,"log_dups_size":0,"ondisk_log_size":4798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.225752+0000","scrub_duration":808,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16998,"num_write_kb":543936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":448,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436501+0000","last_change":"2024-09-21T11:48:05.072998+0000","last_active":"2024-09-21T11:48:49.436501+0000","last_peered":"2024-09-21T11:48:49.436501+0000","last_clean":"2024-09-21T11:48:49.436501+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:48:49.436501+0000","last_undegraded":"2024-09-21T11:48:49.436501+0000","last_fullsized":"2024-09-21T11:48:49.436501+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:05.072910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.072910+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:33.824160+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":420,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.435703+0000","last_change":"2024-09-21T11:48:41.922086+0000","last_active":"2024-09-21T11:48:49.435703+0000","last_peered":"2024-09-21T11:48:49.435703+0000","last_clean":"2024-09-21T11:48:49.435703+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:48:49.435703+0000","last_undegraded":"2024-09-21T11:48:49.435703+0000","last_fullsized":"2024-09-21T11:48:49.435703+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:41.922012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:41.922012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:45.884616+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":394,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.440781+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:48:49.440781+0000","last_peered":"2024-09-21T11:48:49.440781+0000","last_clean":"2024-09-21T11:48:49.440781+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:48:49.440781+0000","last_undegraded":"2024-09-21T11:48:49.440781+0000","last_fullsized":"2024-09-21T11:48:49.440781+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.795578+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":390,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456139+0000","last_change":"2024-09-21T11:48:23.285482+0000","last_active":"2024-09-21T11:48:49.456139+0000","last_peered":"2024-09-21T11:48:49.456139+0000","last_clean":"2024-09-21T11:48:49.456139+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:48:49.456139+0000","last_undegraded":"2024-09-21T11:48:49.456139+0000","last_fullsized":"2024-09-21T11:48:49.456139+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.285283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.285283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:52.473277+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"241'17356","reported_seq":23785,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.617633+0000","last_change":"2024-09-21T11:48:15.522333+0000","last_active":"2024-09-21T11:48:49.617633+0000","last_peered":"2024-09-21T11:48:49.617633+0000","last_clean":"2024-09-21T11:48:49.617633+0000","last_became_active":"2024-09-21T11:48:15.471002+0000","last_became_peered":"2024-09-21T11:48:15.471002+0000","last_unstale":"2024-09-21T11:48:49.617633+0000","last_undegraded":"2024-09-21T11:48:49.617633+0000","last_fullsized":"2024-09-21T11:48:49.617633+0000","mapping_epoch":228,"log_start":"212'12513","ondisk_log_start":"212'12513","created":160,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"198'10449","last_scrub_stamp":"2024-09-21T11:46:52.316998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.316998+0000","objects_scrubbed":7011,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2655,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17356,"num_write_kb":555392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3078,"num_bytes_recovered":201719808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":502,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.435973+0000","last_change":"2024-09-21T11:48:33.035096+0000","last_active":"2024-09-21T11:48:49.435973+0000","last_peered":"2024-09-21T11:48:49.435973+0000","last_clean":"2024-09-21T11:48:49.435973+0000","last_became_active":"2024-09-21T11:48:33.022321+0000","last_became_peered":"2024-09-21T11:48:33.022321+0000","last_unstale":"2024-09-21T11:48:49.435973+0000","last_undegraded":"2024-09-21T11:48:49.435973+0000","last_fullsized":"2024-09-21T11:48:49.435973+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:47:22.974761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:22.974761+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":425,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.442867+0000","last_change":"2024-09-21T11:47:03.365071+0000","last_active":"2024-09-21T11:48:49.442867+0000","last_peered":"2024-09-21T11:48:49.442867+0000","last_clean":"2024-09-21T11:48:49.442867+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:48:49.442867+0000","last_undegraded":"2024-09-21T11:48:49.442867+0000","last_fullsized":"2024-09-21T11:48:49.442867+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.692429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.692429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:54.181079+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":88,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.453985+0000","last_change":"2024-09-21T11:47:03.375108+0000","last_active":"2024-09-21T11:48:49.453985+0000","last_peered":"2024-09-21T11:48:49.453985+0000","last_clean":"2024-09-21T11:48:49.453985+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:48:49.453985+0000","last_undegraded":"2024-09-21T11:48:49.453985+0000","last_fullsized":"2024-09-21T11:48:49.453985+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:50.008635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:50.008635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":399,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.450750+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:48:49.450750+0000","last_peered":"2024-09-21T11:48:49.450750+0000","last_clean":"2024-09-21T11:48:49.450750+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:48:49.450750+0000","last_undegraded":"2024-09-21T11:48:49.450750+0000","last_fullsized":"2024-09-21T11:48:49.450750+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:37.893541+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"241'16936","reported_seq":26069,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.626276+0000","last_change":"2024-09-21T11:47:53.304799+0000","last_active":"2024-09-21T11:48:49.626276+0000","last_peered":"2024-09-21T11:48:49.626276+0000","last_clean":"2024-09-21T11:48:49.626276+0000","last_became_active":"2024-09-21T11:47:53.264856+0000","last_became_peered":"2024-09-21T11:47:53.264856+0000","last_unstale":"2024-09-21T11:48:49.626276+0000","last_undegraded":"2024-09-21T11:48:49.626276+0000","last_fullsized":"2024-09-21T11:48:49.626276+0000","mapping_epoch":221,"log_start":"210'11900","ondisk_log_start":"210'11900","created":160,"last_epoch_clean":222,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:44:50.393209+0000","objects_scrubbed":0,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.133366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16936,"num_write_kb":541952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4468,"num_bytes_recovered":292749312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":422,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436247+0000","last_change":"2024-09-21T11:46:11.730053+0000","last_active":"2024-09-21T11:48:49.436247+0000","last_peered":"2024-09-21T11:48:49.436247+0000","last_clean":"2024-09-21T11:48:49.436247+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:48:49.436247+0000","last_undegraded":"2024-09-21T11:48:49.436247+0000","last_fullsized":"2024-09-21T11:48:49.436247+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:11.729984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.729984+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.980319+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":387,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.455351+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:48:49.455351+0000","last_peered":"2024-09-21T11:48:49.455351+0000","last_clean":"2024-09-21T11:48:49.455351+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:48:49.455351+0000","last_undegraded":"2024-09-21T11:48:49.455351+0000","last_fullsized":"2024-09-21T11:48:49.455351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:27.332154+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":518,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.446223+0000","last_change":"2024-09-21T11:48:43.940489+0000","last_active":"2024-09-21T11:48:49.446223+0000","last_peered":"2024-09-21T11:48:49.446223+0000","last_clean":"2024-09-21T11:48:49.446223+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:48:49.446223+0000","last_undegraded":"2024-09-21T11:48:49.446223+0000","last_fullsized":"2024-09-21T11:48:49.446223+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:48:43.940235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:43.940235+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:58.064586+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":397,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436389+0000","last_change":"2024-09-21T11:48:07.992586+0000","last_active":"2024-09-21T11:48:49.436389+0000","last_peered":"2024-09-21T11:48:49.436389+0000","last_clean":"2024-09-21T11:48:49.436389+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:48:49.436389+0000","last_undegraded":"2024-09-21T11:48:49.436389+0000","last_fullsized":"2024-09-21T11:48:49.436389+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.992517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.992517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.884349+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"241'17196","reported_seq":23354,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.629950+0000","last_change":"2024-09-21T11:48:16.538801+0000","last_active":"2024-09-21T11:48:49.629950+0000","last_peered":"2024-09-21T11:48:49.629950+0000","last_clean":"2024-09-21T11:48:49.629950+0000","last_became_active":"2024-09-21T11:48:16.476754+0000","last_became_peered":"2024-09-21T11:48:16.476754+0000","last_unstale":"2024-09-21T11:48:49.629950+0000","last_undegraded":"2024-09-21T11:48:49.629950+0000","last_fullsized":"2024-09-21T11:48:49.629950+0000","mapping_epoch":229,"log_start":"212'12287","ondisk_log_start":"212'12287","created":160,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'7408","last_scrub_stamp":"2024-09-21T11:46:17.934135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.934135+0000","objects_scrubbed":7290,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"queued for scrub","scrub_duration":3305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17196,"num_write_kb":550272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2943,"num_bytes_recovered":192872448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":440,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456102+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:48:49.456102+0000","last_peered":"2024-09-21T11:48:49.456102+0000","last_clean":"2024-09-21T11:48:49.456102+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:48:49.456102+0000","last_undegraded":"2024-09-21T11:48:49.456102+0000","last_fullsized":"2024-09-21T11:48:49.456102+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:41.378708+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":402,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.440891+0000","last_change":"2024-09-21T11:48:08.172717+0000","last_active":"2024-09-21T11:48:49.440891+0000","last_peered":"2024-09-21T11:48:49.440891+0000","last_clean":"2024-09-21T11:48:49.440891+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:48:49.440891+0000","last_undegraded":"2024-09-21T11:48:49.440891+0000","last_fullsized":"2024-09-21T11:48:49.440891+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:08.172656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:08.172656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:18.830907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":396,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.993307+0000","last_change":"2024-09-21T11:48:49.993307+0000","last_active":"2024-09-21T11:48:49.993307+0000","last_peered":"2024-09-21T11:48:49.993307+0000","last_clean":"2024-09-21T11:48:49.993307+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:48:49.993307+0000","last_undegraded":"2024-09-21T11:48:49.993307+0000","last_fullsized":"2024-09-21T11:48:49.993307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.993230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.993230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:12.892451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":375,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.457980+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:48:49.457980+0000","last_peered":"2024-09-21T11:48:49.457980+0000","last_clean":"2024-09-21T11:48:49.457980+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:48:49.457980+0000","last_undegraded":"2024-09-21T11:48:49.457980+0000","last_fullsized":"2024-09-21T11:48:49.457980+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"241'17282","reported_seq":17538,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.582268+0000","last_change":"2024-09-21T11:46:15.665868+0000","last_active":"2024-09-21T11:48:49.582268+0000","last_peered":"2024-09-21T11:48:49.582268+0000","last_clean":"2024-09-21T11:48:49.582268+0000","last_became_active":"2024-09-21T11:44:51.744226+0000","last_became_peered":"2024-09-21T11:44:51.744226+0000","last_unstale":"2024-09-21T11:48:49.582268+0000","last_undegraded":"2024-09-21T11:48:49.582268+0000","last_fullsized":"2024-09-21T11:48:49.582268+0000","mapping_epoch":160,"log_start":"205'11500","ondisk_log_start":"205'11500","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7351","last_scrub_stamp":"2024-09-21T11:46:15.665775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:15.665775+0000","objects_scrubbed":7260,"log_size":5782,"log_dups_size":0,"ondisk_log_size":5782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17282,"num_write_kb":553024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":484,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.442645+0000","last_change":"2024-09-21T11:47:03.397050+0000","last_active":"2024-09-21T11:48:49.442645+0000","last_peered":"2024-09-21T11:48:49.442645+0000","last_clean":"2024-09-21T11:48:49.442645+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:48:49.442645+0000","last_undegraded":"2024-09-21T11:48:49.442645+0000","last_fullsized":"2024-09-21T11:48:49.442645+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:46:20.650403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.650403+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:51.167956+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":402,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.450090+0000","last_change":"2024-09-21T11:48:03.546909+0000","last_active":"2024-09-21T11:48:49.450090+0000","last_peered":"2024-09-21T11:48:49.450090+0000","last_clean":"2024-09-21T11:48:49.450090+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:48:49.450090+0000","last_undegraded":"2024-09-21T11:48:49.450090+0000","last_fullsized":"2024-09-21T11:48:49.450090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:03.546843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:03.546843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.594742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":403,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.445766+0000","last_change":"2024-09-21T11:46:16.775013+0000","last_active":"2024-09-21T11:48:49.445766+0000","last_peered":"2024-09-21T11:48:49.445766+0000","last_clean":"2024-09-21T11:48:49.445766+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:48:49.445766+0000","last_undegraded":"2024-09-21T11:48:49.445766+0000","last_fullsized":"2024-09-21T11:48:49.445766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:16.774939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.774939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.931030+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":391,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.447608+0000","last_change":"2024-09-21T11:46:59.704981+0000","last_active":"2024-09-21T11:48:49.447608+0000","last_peered":"2024-09-21T11:48:49.447608+0000","last_clean":"2024-09-21T11:48:49.447608+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:48:49.447608+0000","last_undegraded":"2024-09-21T11:48:49.447608+0000","last_fullsized":"2024-09-21T11:48:49.447608+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.588396+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"241'17250","reported_seq":17527,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.622764+0000","last_change":"2024-09-21T11:48:48.383848+0000","last_active":"2024-09-21T11:48:49.622764+0000","last_peered":"2024-09-21T11:48:49.622764+0000","last_clean":"2024-09-21T11:48:49.622764+0000","last_became_active":"2024-09-21T11:44:51.745609+0000","last_became_peered":"2024-09-21T11:44:51.745609+0000","last_unstale":"2024-09-21T11:48:49.622764+0000","last_undegraded":"2024-09-21T11:48:49.622764+0000","last_fullsized":"2024-09-21T11:48:49.622764+0000","mapping_epoch":160,"log_start":"200'10800","ondisk_log_start":"200'10800","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"240'17162","last_scrub_stamp":"2024-09-21T11:48:48.383750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:48:48.383750+0000","objects_scrubbed":96,"log_size":6450,"log_dups_size":0,"ondisk_log_size":6450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:13.279212+0000","scrub_duration":292,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17250,"num_write_kb":552000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":477,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.435526+0000","last_change":"2024-09-21T11:48:04.111199+0000","last_active":"2024-09-21T11:48:49.435526+0000","last_peered":"2024-09-21T11:48:49.435526+0000","last_clean":"2024-09-21T11:48:49.435526+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:48:49.435526+0000","last_undegraded":"2024-09-21T11:48:49.435526+0000","last_fullsized":"2024-09-21T11:48:49.435526+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:48:04.111115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.111115+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:19.507697+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":432,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.449121+0000","last_change":"2024-09-21T11:47:26.411190+0000","last_active":"2024-09-21T11:48:49.449121+0000","last_peered":"2024-09-21T11:48:49.449121+0000","last_clean":"2024-09-21T11:48:49.449121+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:48:49.449121+0000","last_undegraded":"2024-09-21T11:48:49.449121+0000","last_fullsized":"2024-09-21T11:48:49.449121+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:26.411146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:47:26.411146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":418,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.445005+0000","last_change":"2024-09-21T11:47:03.385382+0000","last_active":"2024-09-21T11:48:49.445005+0000","last_peered":"2024-09-21T11:48:49.445005+0000","last_clean":"2024-09-21T11:48:49.445005+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:48:49.445005+0000","last_undegraded":"2024-09-21T11:48:49.445005+0000","last_fullsized":"2024-09-21T11:48:49.445005+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.766819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.766819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:38.890518+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":397,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.455055+0000","last_change":"2024-09-21T11:47:03.376001+0000","last_active":"2024-09-21T11:48:49.455055+0000","last_peered":"2024-09-21T11:48:49.455055+0000","last_clean":"2024-09-21T11:48:49.455055+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:48:49.455055+0000","last_undegraded":"2024-09-21T11:48:49.455055+0000","last_fullsized":"2024-09-21T11:48:49.455055+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.703712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.703712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"241'17210","reported_seq":17464,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.616905+0000","last_change":"2024-09-21T11:46:12.974904+0000","last_active":"2024-09-21T11:48:49.616905+0000","last_peered":"2024-09-21T11:48:49.616905+0000","last_clean":"2024-09-21T11:48:49.616905+0000","last_became_active":"2024-09-21T11:44:51.409710+0000","last_became_peered":"2024-09-21T11:44:51.409710+0000","last_unstale":"2024-09-21T11:48:49.616905+0000","last_undegraded":"2024-09-21T11:48:49.616905+0000","last_fullsized":"2024-09-21T11:48:49.616905+0000","mapping_epoch":160,"log_start":"208'11760","ondisk_log_start":"208'11760","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"192'7106","last_scrub_stamp":"2024-09-21T11:46:12.974687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:44:50.393209+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.974687+0000","objects_scrubbed":7036,"log_size":5450,"log_dups_size":0,"ondisk_log_size":5450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.015746+0000","scrub_duration":1797,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17210,"num_write_kb":550720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":408,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.441354+0000","last_change":"2024-09-21T11:46:57.248649+0000","last_active":"2024-09-21T11:48:49.441354+0000","last_peered":"2024-09-21T11:48:49.441354+0000","last_clean":"2024-09-21T11:48:49.441354+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:48:49.441354+0000","last_undegraded":"2024-09-21T11:48:49.441354+0000","last_fullsized":"2024-09-21T11:48:49.441354+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:57.248575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.248575+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.044421+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":422,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.448769+0000","last_change":"2024-09-21T11:48:09.683093+0000","last_active":"2024-09-21T11:48:49.448769+0000","last_peered":"2024-09-21T11:48:49.448769+0000","last_clean":"2024-09-21T11:48:49.448769+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:48:49.448769+0000","last_undegraded":"2024-09-21T11:48:49.448769+0000","last_fullsized":"2024-09-21T11:48:49.448769+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:48:09.683008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.683008+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:38.676024+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":510,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.447491+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:48:49.447491+0000","last_peered":"2024-09-21T11:48:49.447491+0000","last_clean":"2024-09-21T11:48:49.447491+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:48:49.447491+0000","last_undegraded":"2024-09-21T11:48:49.447491+0000","last_fullsized":"2024-09-21T11:48:49.447491+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.600494+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":383,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.441209+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:48:49.441209+0000","last_peered":"2024-09-21T11:48:49.441209+0000","last_clean":"2024-09-21T11:48:49.441209+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:48:49.441209+0000","last_undegraded":"2024-09-21T11:48:49.441209+0000","last_fullsized":"2024-09-21T11:48:49.441209+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:53.006344+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":410,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.442154+0000","last_change":"2024-09-21T11:48:17.129730+0000","last_active":"2024-09-21T11:48:49.442154+0000","last_peered":"2024-09-21T11:48:49.442154+0000","last_clean":"2024-09-21T11:48:49.442154+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:48:49.442154+0000","last_undegraded":"2024-09-21T11:48:49.442154+0000","last_fullsized":"2024-09-21T11:48:49.442154+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.129675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.129675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.682622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":430,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.453251+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:48:49.453251+0000","last_peered":"2024-09-21T11:48:49.453251+0000","last_clean":"2024-09-21T11:48:49.453251+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:48:49.453251+0000","last_undegraded":"2024-09-21T11:48:49.453251+0000","last_fullsized":"2024-09-21T11:48:49.453251+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":408,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.440503+0000","last_change":"2024-09-21T11:47:25.905705+0000","last_active":"2024-09-21T11:48:49.440503+0000","last_peered":"2024-09-21T11:48:49.440503+0000","last_clean":"2024-09-21T11:48:49.440503+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:48:49.440503+0000","last_undegraded":"2024-09-21T11:48:49.440503+0000","last_fullsized":"2024-09-21T11:48:49.440503+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:25.905645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:25.905645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:55.034596+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":398,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.449647+0000","last_change":"2024-09-21T11:47:28.380378+0000","last_active":"2024-09-21T11:48:49.449647+0000","last_peered":"2024-09-21T11:48:49.449647+0000","last_clean":"2024-09-21T11:48:49.449647+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:48:49.449647+0000","last_undegraded":"2024-09-21T11:48:49.449647+0000","last_fullsized":"2024-09-21T11:48:49.449647+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:28.380245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:28.380245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.638314+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":423,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.462850+0000","last_change":"2024-09-21T11:46:28.973700+0000","last_active":"2024-09-21T11:48:49.462850+0000","last_peered":"2024-09-21T11:48:49.462850+0000","last_clean":"2024-09-21T11:48:49.462850+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:48:49.462850+0000","last_undegraded":"2024-09-21T11:48:49.462850+0000","last_fullsized":"2024-09-21T11:48:49.462850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:28.973606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.973606+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":394,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.447627+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:48:49.447627+0000","last_peered":"2024-09-21T11:48:49.447627+0000","last_clean":"2024-09-21T11:48:49.447627+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:48:49.447627+0000","last_undegraded":"2024-09-21T11:48:49.447627+0000","last_fullsized":"2024-09-21T11:48:49.447627+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":409,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.452918+0000","last_change":"2024-09-21T11:47:03.379216+0000","last_active":"2024-09-21T11:48:49.452918+0000","last_peered":"2024-09-21T11:48:49.452918+0000","last_clean":"2024-09-21T11:48:49.452918+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:48:49.452918+0000","last_undegraded":"2024-09-21T11:48:49.452918+0000","last_fullsized":"2024-09-21T11:48:49.452918+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:48.903980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:48.903980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":399,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.445710+0000","last_change":"2024-09-21T11:48:17.081179+0000","last_active":"2024-09-21T11:48:49.445710+0000","last_peered":"2024-09-21T11:48:49.445710+0000","last_clean":"2024-09-21T11:48:49.445710+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:48:49.445710+0000","last_undegraded":"2024-09-21T11:48:49.445710+0000","last_fullsized":"2024-09-21T11:48:49.445710+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.081131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.081131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.264468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":406,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.441214+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:48:49.441214+0000","last_peered":"2024-09-21T11:48:49.441214+0000","last_clean":"2024-09-21T11:48:49.441214+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:48:49.441214+0000","last_undegraded":"2024-09-21T11:48:49.441214+0000","last_fullsized":"2024-09-21T11:48:49.441214+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.995538+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":402,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.455935+0000","last_change":"2024-09-21T11:46:24.087039+0000","last_active":"2024-09-21T11:48:49.455935+0000","last_peered":"2024-09-21T11:48:49.455935+0000","last_clean":"2024-09-21T11:48:49.455935+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:48:49.455935+0000","last_undegraded":"2024-09-21T11:48:49.455935+0000","last_fullsized":"2024-09-21T11:48:49.455935+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.086918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.086918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":494,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456404+0000","last_change":"2024-09-21T11:47:02.849839+0000","last_active":"2024-09-21T11:48:49.456404+0000","last_peered":"2024-09-21T11:48:49.456404+0000","last_clean":"2024-09-21T11:48:49.456404+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:48:49.456404+0000","last_undegraded":"2024-09-21T11:48:49.456404+0000","last_fullsized":"2024-09-21T11:48:49.456404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:02.849786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:02.849786+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":391,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436288+0000","last_change":"2024-09-21T11:45:38.652432+0000","last_active":"2024-09-21T11:48:49.436288+0000","last_peered":"2024-09-21T11:48:49.436288+0000","last_clean":"2024-09-21T11:48:49.436288+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:48:49.436288+0000","last_undegraded":"2024-09-21T11:48:49.436288+0000","last_fullsized":"2024-09-21T11:48:49.436288+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:38.652380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:38.652380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":436,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456551+0000","last_change":"2024-09-21T11:47:03.428846+0000","last_active":"2024-09-21T11:48:49.456551+0000","last_peered":"2024-09-21T11:48:49.456551+0000","last_clean":"2024-09-21T11:48:49.456551+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:48:49.456551+0000","last_undegraded":"2024-09-21T11:48:49.456551+0000","last_fullsized":"2024-09-21T11:48:49.456551+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:29.284078+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":394,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.448284+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:48:49.448284+0000","last_peered":"2024-09-21T11:48:49.448284+0000","last_clean":"2024-09-21T11:48:49.448284+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:48:49.448284+0000","last_undegraded":"2024-09-21T11:48:49.448284+0000","last_fullsized":"2024-09-21T11:48:49.448284+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":532,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.441111+0000","last_change":"2024-09-21T11:47:03.415793+0000","last_active":"2024-09-21T11:48:49.441111+0000","last_peered":"2024-09-21T11:48:49.441111+0000","last_clean":"2024-09-21T11:48:49.441111+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:48:49.441111+0000","last_undegraded":"2024-09-21T11:48:49.441111+0000","last_fullsized":"2024-09-21T11:48:49.441111+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:10.174971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:10.174971+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.999554+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":398,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:51.483725+0000","last_change":"2024-09-21T11:48:51.483725+0000","last_active":"2024-09-21T11:48:51.483725+0000","last_peered":"2024-09-21T11:48:51.483725+0000","last_clean":"2024-09-21T11:48:51.483725+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:48:51.483725+0000","last_undegraded":"2024-09-21T11:48:51.483725+0000","last_fullsized":"2024-09-21T11:48:51.483725+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:51.483662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:51.483662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:01.284415+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":483,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.435882+0000","last_change":"2024-09-21T11:48:33.033557+0000","last_active":"2024-09-21T11:48:49.435882+0000","last_peered":"2024-09-21T11:48:49.435882+0000","last_clean":"2024-09-21T11:48:49.435882+0000","last_became_active":"2024-09-21T11:48:33.022647+0000","last_became_peered":"2024-09-21T11:48:33.022647+0000","last_unstale":"2024-09-21T11:48:49.435882+0000","last_undegraded":"2024-09-21T11:48:49.435882+0000","last_fullsized":"2024-09-21T11:48:49.435882+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:54.032795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.032795+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.014222+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":395,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.436405+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:48:49.436405+0000","last_peered":"2024-09-21T11:48:49.436405+0000","last_clean":"2024-09-21T11:48:49.436405+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:48:49.436405+0000","last_undegraded":"2024-09-21T11:48:49.436405+0000","last_fullsized":"2024-09-21T11:48:49.436405+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.948354+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":88,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.442749+0000","last_change":"2024-09-21T11:47:03.379049+0000","last_active":"2024-09-21T11:48:49.442749+0000","last_peered":"2024-09-21T11:48:49.442749+0000","last_clean":"2024-09-21T11:48:49.442749+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:48:49.442749+0000","last_undegraded":"2024-09-21T11:48:49.442749+0000","last_fullsized":"2024-09-21T11:48:49.442749+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.269610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.269610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":375,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.452208+0000","last_change":"2024-09-21T11:45:04.770055+0000","last_active":"2024-09-21T11:48:49.452208+0000","last_peered":"2024-09-21T11:48:49.452208+0000","last_clean":"2024-09-21T11:48:49.452208+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:48:49.452208+0000","last_undegraded":"2024-09-21T11:48:49.452208+0000","last_fullsized":"2024-09-21T11:48:49.452208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:04.769963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:04.769963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":98,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.442945+0000","last_change":"2024-09-21T11:47:03.404473+0000","last_active":"2024-09-21T11:48:49.442945+0000","last_peered":"2024-09-21T11:48:49.442945+0000","last_clean":"2024-09-21T11:48:49.442945+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:48:49.442945+0000","last_undegraded":"2024-09-21T11:48:49.442945+0000","last_fullsized":"2024-09-21T11:48:49.442945+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:26.649189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:26.649189+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":402,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.449951+0000","last_change":"2024-09-21T11:48:04.553772+0000","last_active":"2024-09-21T11:48:49.449951+0000","last_peered":"2024-09-21T11:48:49.449951+0000","last_clean":"2024-09-21T11:48:49.449951+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:48:49.449951+0000","last_undegraded":"2024-09-21T11:48:49.449951+0000","last_fullsized":"2024-09-21T11:48:49.449951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:04.553731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.553731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:21.248327+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":409,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.455130+0000","last_change":"2024-09-21T11:47:03.377948+0000","last_active":"2024-09-21T11:48:49.455130+0000","last_peered":"2024-09-21T11:48:49.455130+0000","last_clean":"2024-09-21T11:48:49.455130+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:48:49.455130+0000","last_undegraded":"2024-09-21T11:48:49.455130+0000","last_fullsized":"2024-09-21T11:48:49.455130+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:53.848323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:53.848323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":421,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:50.487842+0000","last_change":"2024-09-21T11:48:50.487842+0000","last_active":"2024-09-21T11:48:50.487842+0000","last_peered":"2024-09-21T11:48:50.487842+0000","last_clean":"2024-09-21T11:48:50.487842+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:48:50.487842+0000","last_undegraded":"2024-09-21T11:48:50.487842+0000","last_fullsized":"2024-09-21T11:48:50.487842+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:48:50.487786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.487786+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:13.704639+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":410,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.452364+0000","last_change":"2024-09-21T11:45:46.706221+0000","last_active":"2024-09-21T11:48:49.452364+0000","last_peered":"2024-09-21T11:48:49.452364+0000","last_clean":"2024-09-21T11:48:49.452364+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:48:49.452364+0000","last_undegraded":"2024-09-21T11:48:49.452364+0000","last_fullsized":"2024-09-21T11:48:49.452364+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:45:46.706130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:46.706130+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":408,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.440586+0000","last_change":"2024-09-21T11:48:10.126645+0000","last_active":"2024-09-21T11:48:49.440586+0000","last_peered":"2024-09-21T11:48:49.440586+0000","last_clean":"2024-09-21T11:48:49.440586+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:48:49.440586+0000","last_undegraded":"2024-09-21T11:48:49.440586+0000","last_fullsized":"2024-09-21T11:48:49.440586+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.126591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.126591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:10.829490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":386,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.447139+0000","last_change":"2024-09-21T11:48:44.639778+0000","last_active":"2024-09-21T11:48:49.447139+0000","last_peered":"2024-09-21T11:48:49.447139+0000","last_clean":"2024-09-21T11:48:49.447139+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:48:49.447139+0000","last_undegraded":"2024-09-21T11:48:49.447139+0000","last_fullsized":"2024-09-21T11:48:49.447139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:44.639638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.639638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:04.448990+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":383,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.455253+0000","last_change":"2024-09-21T11:45:45.730374+0000","last_active":"2024-09-21T11:48:49.455253+0000","last_peered":"2024-09-21T11:48:49.455253+0000","last_clean":"2024-09-21T11:48:49.455253+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:48:49.455253+0000","last_undegraded":"2024-09-21T11:48:49.455253+0000","last_fullsized":"2024-09-21T11:48:49.455253+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:45.730169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:45.730169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":444,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.453892+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:48:49.453892+0000","last_peered":"2024-09-21T11:48:49.453892+0000","last_clean":"2024-09-21T11:48:49.453892+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:48:49.453892+0000","last_undegraded":"2024-09-21T11:48:49.453892+0000","last_fullsized":"2024-09-21T11:48:49.453892+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":394,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.456476+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:48:49.456476+0000","last_peered":"2024-09-21T11:48:49.456476+0000","last_clean":"2024-09-21T11:48:49.456476+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:48:49.456476+0000","last_undegraded":"2024-09-21T11:48:49.456476+0000","last_fullsized":"2024-09-21T11:48:49.456476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.212674+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":394,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.442687+0000","last_change":"2024-09-21T11:48:09.104826+0000","last_active":"2024-09-21T11:48:49.442687+0000","last_peered":"2024-09-21T11:48:49.442687+0000","last_clean":"2024-09-21T11:48:49.442687+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:48:49.442687+0000","last_undegraded":"2024-09-21T11:48:49.442687+0000","last_fullsized":"2024-09-21T11:48:49.442687+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:09.104767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.104767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.005085+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":382,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.447717+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:48:49.447717+0000","last_peered":"2024-09-21T11:48:49.447717+0000","last_clean":"2024-09-21T11:48:49.447717+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:48:49.447717+0000","last_undegraded":"2024-09-21T11:48:49.447717+0000","last_fullsized":"2024-09-21T11:48:49.447717+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:48:49.549791+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":88,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.453509+0000","last_change":"2024-09-21T11:47:03.374194+0000","last_active":"2024-09-21T11:48:49.453509+0000","last_peered":"2024-09-21T11:48:49.453509+0000","last_clean":"2024-09-21T11:48:49.453509+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:48:49.453509+0000","last_undegraded":"2024-09-21T11:48:49.453509+0000","last_fullsized":"2024-09-21T11:48:49.453509+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:27.297630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:27.297630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":409,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.454183+0000","last_change":"2024-09-21T11:47:03.377189+0000","last_active":"2024-09-21T11:48:49.454183+0000","last_peered":"2024-09-21T11:48:49.454183+0000","last_clean":"2024-09-21T11:48:49.454183+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:48:49.454183+0000","last_undegraded":"2024-09-21T11:48:49.454183+0000","last_fullsized":"2024-09-21T11:48:49.454183+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:11.587469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.587469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":472,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.443763+0000","last_change":"2024-09-21T11:48:45.147257+0000","last_active":"2024-09-21T11:48:49.443763+0000","last_peered":"2024-09-21T11:48:49.443763+0000","last_clean":"2024-09-21T11:48:49.443763+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:48:49.443763+0000","last_undegraded":"2024-09-21T11:48:49.443763+0000","last_fullsized":"2024-09-21T11:48:49.443763+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:45.147190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:45.147190+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:06.653209+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":403,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-21T11:48:49.523077+0000","last_change":"2024-09-21T11:48:49.523077+0000","last_active":"2024-09-21T11:48:49.523077+0000","last_peered":"2024-09-21T11:48:49.523077+0000","last_clean":"2024-09-21T11:48:49.523077+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:48:49.523077+0000","last_undegraded":"2024-09-21T11:48:49.523077+0000","last_fullsized":"2024-09-21T11:48:49.523077+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.522878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.522878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:04.701345+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":274184,"num_write_kb":8773825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24511,"num_bytes_recovered":1606287360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":226230272,"data_stored":226230272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86132,"ondisk_log_size":86132,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":396,"ondisk_log_size":396,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":522,"num_read_kb":517,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234095,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":317144,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":93428008,"statfs":{"total":95995035648,"available":95670280192,"internally_reserved":0,"allocated":1380352,"data_stored":1000684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":323353034},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463028,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":285196,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":93459956,"statfs":{"total":95995035648,"available":95702994944,"internally_reserved":0,"allocated":1368064,"data_stored":1001028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":290650566},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724665,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306840,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":93438312,"statfs":{"total":95995035648,"available":95680831488,"internally_reserved":0,"allocated":794624,"data_stored":410071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":313391558},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920895,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":270752,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":93474400,"statfs":{"total":95995035648,"available":95717785600,"internally_reserved":0,"allocated":798720,"data_stored":410829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":276429253},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117125,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":374864,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":374078,"kb_avail":93370288,"statfs":{"total":95995035648,"available":95611174912,"internally_reserved":0,"allocated":782336,"data_stored":409654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":383056327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411467,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":636052,"kb_used_data":221696,"kb_used_omap":1,"kb_used_meta":414334,"kb_avail":93109100,"statfs":{"total":95995035648,"available":95343718400,"internally_reserved":0,"allocated":227016704,"data_stored":226640637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":424278469},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":292976,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":93452176,"statfs":{"total":95995035648,"available":95695028224,"internally_reserved":0,"allocated":819200,"data_stored":410915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":299170246},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738520,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":325476,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":324094,"kb_avail":93419676,"statfs":{"total":95995035648,"available":95661748224,"internally_reserved":0,"allocated":1392640,"data_stored":1001399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":331872709},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":226230272,"data_stored":226230272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:48:55.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool set unique_pool_1 pgp_num 16 2024-09-21T11:48:55.389 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:55.411 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:55.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:55.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:55.489 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:48:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:48:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:48:55.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:55 smithi043 ceph-mon[29710]: pgmap v424: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-21T11:48:55.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:55 smithi043 ceph-mon[29710]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:55.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:55 smithi043 ceph-mon[36454]: pgmap v424: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-21T11:48:55.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:55 smithi043 ceph-mon[36454]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:55.496 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:48:55.496 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:48:55.512 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:55.515 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:48:55.515 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:48:55.531 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:48:55.532 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:48:55.533 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:55.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:55.534 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:55.550 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:48:55.550 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:48:55.573 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:48:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:48:55.589 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:48:55.589 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:48:55.606 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:48:55.606 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:48:55.623 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:48:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:48:55.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:55.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:55.659 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:55.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:55.701 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:48:55.723 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:55 smithi175 ceph-mon[32613]: pgmap v424: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-21T11:48:55.723 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:55 smithi175 ceph-mon[32613]: from='client.25288 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:48:55.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:55.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:55.784 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:55.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:55.907 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:55.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:55.908 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:55.947 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:48:56.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:56.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:56.034 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:56.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:56.157 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:56.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:56.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:56.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:56.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:56.282 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:56.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:56.384 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:56.400 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:48:56.400 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:48:56.406 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:56.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:56.407 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:56.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:56.530 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:56.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:56.531 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:56.632 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:56.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:56.658 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:56.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:56.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:56 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3733365863' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:56.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:56 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3733365863' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:56.741 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:56 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3733365863' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:48:56.754 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:56.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:56.785 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:56.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:56.786 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:56.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:56.910 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:56.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:56.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:57.012 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:57.036 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:57.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:57.037 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:57.040 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:48:57.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:57.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:57.160 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:57.261 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:57.284 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:57.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:57.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:57.326 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:48:57.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:57.408 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:57.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:57.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:57.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:57.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:57.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:57.637 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:57.658 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:57.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:57.659 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:57.727 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:57 smithi175 ceph-mon[32613]: pgmap v425: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 483 op/s 2024-09-21T11:48:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:57 smithi043 ceph-mon[29710]: pgmap v425: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 483 op/s 2024-09-21T11:48:57.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:57 smithi043 ceph-mon[36454]: pgmap v425: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 483 op/s 2024-09-21T11:48:57.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:57.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:57.786 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:57.787 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:57.826 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:48:57.826 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:48:57.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:57.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:57.910 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:57.910 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:57.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:48:58.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:48:58.063 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:58.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:58.064 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:48:58.165 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:58.187 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:58.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:58.188 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:48:58.289 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:58.310 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:58.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:58.311 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:58.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:58.433 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:58.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:58.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:58.467 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 8 pgp_num to 16 2024-09-21T11:48:58.479 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:48:58.534 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:48:58.555 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:58.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:58.556 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:48:58.657 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:58.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:58 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1561861360' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:48:58.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:58 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:48:58.676 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:58 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1561861360' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:48:58.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:58 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:48:58.679 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:58.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:58.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:58.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:48:58.729 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:48:58.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:58 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1561861360' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:48:58.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:58 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:48:58.781 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:58.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:58.803 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:58.803 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:58.904 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:58.925 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:58.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:58.926 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:59.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:48:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:59.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:59.050 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:48:59.151 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:48:59.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:59.174 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:59.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:48:59.234 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:48:59.235 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:48:59.275 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:59.296 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:59.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:59.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:48:59.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:48:59.421 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:59.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:59.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:48:59.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:48:59.547 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:48:59.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:59.550 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:48:59.651 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:59.674 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:59.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:59.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:59.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:59 smithi043 ceph-mon[29710]: pgmap v426: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 455 op/s 2024-09-21T11:48:59.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:59 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]': finished 2024-09-21T11:48:59.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:48:59 smithi043 ceph-mon[29710]: osdmap e243: 8 total, 8 up, 8 in 2024-09-21T11:48:59.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:59 smithi043 ceph-mon[36454]: pgmap v426: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 455 op/s 2024-09-21T11:48:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:59 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]': finished 2024-09-21T11:48:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:48:59 smithi043 ceph-mon[36454]: osdmap e243: 8 total, 8 up, 8 in 2024-09-21T11:48:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:59 smithi175 ceph-mon[32613]: pgmap v426: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 455 op/s 2024-09-21T11:48:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:59 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num", "val": "16"}]': finished 2024-09-21T11:48:59.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:48:59 smithi175 ceph-mon[32613]: osdmap e243: 8 total, 8 up, 8 in 2024-09-21T11:48:59.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:48:59.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:59.798 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:59.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:48:59.899 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:48:59.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:48:59.923 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:48:59.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:00.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:00.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:00.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:00.118 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:49:00.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:00.172 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-21T11:49:00.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-21T11:49:00.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:00.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:00.175 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:00.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:00.297 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:00.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:00.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:00.364 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:49:00.398 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:00.419 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:00.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:00.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:00.469 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:49:00.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:00.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:00.543 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:00.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:00.632 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:49:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:49:00.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:00.652 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:49:00.652 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:49:00.662 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3773942513' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:00.662 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3773942513' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:00.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:00.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:00.670 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:49:00.671 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:49:00.686 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:49:00.686 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:49:00.704 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:49:00.704 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:49:00.721 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:49:00.721 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:49:00.738 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:49:00.738 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:49:00.755 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:49:00.755 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:49:00.759 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3773942513' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:00.771 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:00.792 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:00.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:00.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:00.858 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:49:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:49:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:49:00.859 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:49:00.859 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:49:00.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:00.915 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:00.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:00.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:01.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:01.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:01.040 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:01.040 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:01.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:01.163 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:01.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:01.164 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:01.264 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:01.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:01.287 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:01.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:01.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:01.411 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:01.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:01.412 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:01.507 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:49:01.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:01.529 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:49:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:49:01.529 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:01 smithi175 ceph-mon[32613]: pgmap v428: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:49:01.529 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:01 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:01.529 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3773942513' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:01.529 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:01 smithi175 ceph-mon[32613]: osdmap e244: 8 total, 8 up, 8 in 2024-09-21T11:49:01.529 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2977630225' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:01.530 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:01.549 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:01.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:01.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:01.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:01.671 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:01.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:01.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[36454]: pgmap v428: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:49:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3773942513' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[36454]: osdmap e244: 8 total, 8 up, 8 in 2024-09-21T11:49:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2977630225' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:01.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[29710]: pgmap v428: 148 pgs: 148 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-21T11:49:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3773942513' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[29710]: osdmap e244: 8 total, 8 up, 8 in 2024-09-21T11:49:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2977630225' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:01.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:01.751 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:49:01.777 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:01.799 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:01.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:01.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:01.900 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:01.922 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:01.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:01.923 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:02.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:02.046 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:02.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:02.047 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:02.148 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:02.169 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:02.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:02.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:02.241 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:49:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:49:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:49:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:49:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.241 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:99019 7.d 7:bdee4b66:::benchmark_data_smithi043_70646_object99018:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:26.324885+0000", 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 395.90523456199998, 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.109393896, 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99019 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324885+0000", 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324885+0000", 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324887+0000", 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8419999999999999e-06 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324939+0000", 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1867000000000003e-05 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324952+0000", 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2622999999999999e-05 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324960+0000", 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.244 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.325018+0000", 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7787000000000001e-05 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.325122+0000", 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104081 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1", 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.325256+0000", 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013424699999999999 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.327618+0000", 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023618419999999998 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.330240+0000", 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026218690000000002 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.434164+0000", 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.103923735 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.434226+0000", 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2242000000000005e-05 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.434279+0000", 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000002e-05 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:99032 7.d 7:b4661550:::benchmark_data_smithi043_70646_object99031:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:26.337848+0000", 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 395.89227176600002, 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10748263299999999, 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99032 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337848+0000", 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337848+0000", 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337849+0000", 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.331e-06 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337893+0000", 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3432e-05 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337904+0000", 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1518999999999999e-05 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337912+0000", 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0369999999999995e-06 2024-09-21T11:49:02.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337974+0000", 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1711999999999995e-05 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338086+0000", 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011175000000000001 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1", 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338186+0000", 2024-09-21T11:49:02.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9905e-05 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.340545+0000", 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023587730000000002 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.341117+0000", 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057252099999999999 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.445226+0000", 2024-09-21T11:49:02.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10410860399999999 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.445284+0000", 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8069999999999998e-05 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.445331+0000", 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6978999999999999e-05 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.253 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:99272 7.7 7:e3c50c45:::benchmark_data_smithi043_70646_object99271:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:26.706556+0000", 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 395.52356395599998, 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096941552, 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99272 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706556+0000", 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706556+0000", 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706557+0000", 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5290000000000001e-06 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706601+0000", 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3544000000000002e-05 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706613+0000", 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1625e-05 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706619+0000", 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5919999999999997e-06 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706666+0000", 2024-09-21T11:49:02.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6526999999999997e-05 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706752+0000", 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6734e-05 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706847+0000", 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4136999999999997e-05 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.709350+0000", 2024-09-21T11:49:02.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002503244 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.710741+0000", 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013909250000000001 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.803398+0000", 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092657623999999994 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.803445+0000", 2024-09-21T11:49:02.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6652e-05 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.803497+0000", 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2419000000000001e-05 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123033 7.d 7:bdea6e43:::benchmark_data_smithi043_70646_object5625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.860948+0000", 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 366.369171521, 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10815586100000001, 2024-09-21T11:49:02.259 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123033 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860948+0000", 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860948+0000", 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860950+0000", 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5680000000000001e-06 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860955+0000", 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3929999999999999e-06 2024-09-21T11:49:02.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860972+0000", 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6901e-05 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860980+0000", 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0779999999999996e-06 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.861037+0000", 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6368e-05 2024-09-21T11:49:02.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.861319+0000", 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028206500000000002 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1,3", 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.862396+0000", 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010775660000000001 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.867300+0000", 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0049041199999999997 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.868070+0000", 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00077001800000000003 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.872379+0000", 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043081150000000004 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.968899+0000", 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096520717000000006 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.968976+0000", 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7199000000000004e-05 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.969104+0000", 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012775300000000001 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123045 7.d 7:be75f9a0:::benchmark_data_smithi043_70646_object5637:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.874453+0000", 2024-09-21T11:49:02.265 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 366.35566666599999, 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.100889214, 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123045 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874453+0000", 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874453+0000", 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874454+0000", 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.17e-06 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874460+0000", 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2460000000000003e-06 2024-09-21T11:49:02.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874468+0000", 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3049999999999993e-06 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874475+0000", 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0740000000000004e-06 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874536+0000", 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0645e-05 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874666+0000", 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000130264 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1,3", 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874801+0000", 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013530200000000001 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879404+0000", 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004603261 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879516+0000", 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011174400000000001 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882653+0000", 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003136808 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975138+0000", 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092484760999999999 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975214+0000", 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5873999999999999e-05 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975342+0000", 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012876000000000001 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123052 7.d 7:b4211c57:::benchmark_data_smithi043_70646_object5644:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.879526+0000", 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 366.35059379, 2024-09-21T11:49:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096040904999999996, 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123052 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879526+0000", 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879526+0000", 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879528+0000", 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.058e-06 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879534+0000", 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9290000000000003e-06 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879547+0000", 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2625e-05 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879555+0000", 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3289999999999994e-06 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879604+0000", 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8906999999999998e-05 2024-09-21T11:49:02.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879739+0000", 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013482199999999999 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1,3", 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879886+0000", 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014718199999999999 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882961+0000", 2024-09-21T11:49:02.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003075468 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.885347+0000", 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002385543 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.885927+0000", 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00058006400000000004 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975391+0000", 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.089464284000000005 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975459+0000", 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7566999999999999e-05 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975567+0000", 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108127 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126201 7.d 7:bd4d612f:::benchmark_data_smithi043_70646_object8793:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.184391+0000", 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 363.04572931799999, 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.107348185, 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126201 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.278 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184391+0000", 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184391+0000", 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184392+0000", 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5090000000000001e-06 2024-09-21T11:49:02.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184397+0000", 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3839999999999996e-06 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184408+0000", 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.031e-05 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184415+0000", 2024-09-21T11:49:02.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3039999999999999e-06 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.185017+0000", 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00060169300000000004 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.185146+0000", 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012897000000000001 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1,3", 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.185294+0000", 2024-09-21T11:49:02.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014784299999999999 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.187915+0000", 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026212169999999999 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194804+0000", 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0068889140000000003 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194847+0000", 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3689000000000001e-05 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291506+0000", 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096658620000000001 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291615+0000", 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109423 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291739+0000", 2024-09-21T11:49:02.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012330899999999999 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126205 7.d 7:bf875b8c:::benchmark_data_smithi043_70646_object8797:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.189901+0000", 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 363.040218659, 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.102518362, 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126205 2024-09-21T11:49:02.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189901+0000", 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189901+0000", 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189902+0000", 2024-09-21T11:49:02.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2589999999999999e-06 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189907+0000", 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.729e-06 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189915+0000", 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8860000000000004e-06 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189921+0000", 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1330000000000004e-06 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.190242+0000", 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032039900000000002 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.190365+0000", 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012315199999999999 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1,3", 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.190603+0000", 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000237777 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194889+0000", 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0042860900000000002 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194975+0000", 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6694999999999996e-05 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.198044+0000", 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030690719999999999 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291775+0000", 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093730331 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291866+0000", 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0890999999999994e-05 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.292420+0000", 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00055394799999999996 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126213 7.7 7:efdaa55a:::benchmark_data_smithi043_70646_object8805:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.194786+0000", 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 363.035333654, 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.099879320999999993, 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126213 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194786+0000", 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194786+0000", 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194788+0000", 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4559999999999999e-06 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194793+0000", 2024-09-21T11:49:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.31e-06 2024-09-21T11:49:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194804+0000", 2024-09-21T11:49:02.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0886e-05 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194811+0000", 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2300000000000002e-06 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194876+0000", 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4975999999999996e-05 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194991+0000", 2024-09-21T11:49:02.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114567 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.195150+0000", 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015947499999999999 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197421+0000", 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022710370000000001 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.288199+0000", 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090777468 2024-09-21T11:49:02.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.294580+0000", 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0063818929999999996 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.294623+0000", 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2929000000000002e-05 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.294666+0000", 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2094000000000001e-05 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.296 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220977 7.c 7:39262562:::benchmark_data_smithi043_70646_object103569:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.889732+0000", 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 275.34038763699999, 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.128598458, 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220977 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.297 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889732+0000", 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889732+0000", 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889733+0000", 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9400000000000004e-07 2024-09-21T11:49:02.298 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889737+0000", 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.335e-06 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889747+0000", 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8079999999999999e-06 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889754+0000", 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8399999999999997e-06 2024-09-21T11:49:02.299 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.891865+0000", 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021112990000000001 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.891983+0000", 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117707 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892088+0000", 2024-09-21T11:49:02.300 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105102 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894742+0000", 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026534750000000002 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.897215+0000", 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024734470000000001 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018256+0000", 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.121041197 2024-09-21T11:49:02.301 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018295+0000", 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8430000000000003e-05 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018331+0000", 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5923999999999997e-05 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.302 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220983 7.7 7:e4a77ba0:::benchmark_data_smithi043_70646_object103575:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.892652+0000", 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 275.33746820300001, 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.125681194, 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220983 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.303 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892652+0000", 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892652+0000", 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892653+0000", 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.178e-06 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.304 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892656+0000", 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6650000000000001e-06 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892670+0000", 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3560000000000001e-05 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892677+0000", 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1579999999999999e-06 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.305 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892734+0000", 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6365e-05 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892833+0000", 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9402999999999997e-05 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892935+0000", 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102348 2024-09-21T11:49:02.306 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.895993+0000", 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003057471 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896614+0000", 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00062117100000000001 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018250+0000", 2024-09-21T11:49:02.307 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12163622 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018298+0000", 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7837000000000003e-05 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018333+0000", 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4817999999999998e-05 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.308 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220986 7.c 7:34da84d9:::benchmark_data_smithi043_70646_object103578:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.894893+0000", 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 275.33522644800001, 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.123588221, 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220986 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.309 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894893+0000", 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894893+0000", 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894895+0000", 2024-09-21T11:49:02.310 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.175e-06 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894899+0000", 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0199999999999996e-06 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894905+0000", 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4540000000000004e-06 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.311 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894910+0000", 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4750000000000004e-06 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894962+0000", 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2067000000000001e-05 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.895060+0000", 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8333999999999997e-05 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:49:02.312 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.895158+0000", 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7813999999999996e-05 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.897402+0000", 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022445289999999999 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.898115+0000", 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00071272200000000005 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.313 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018382+0000", 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.120266574 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018436+0000", 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3965999999999998e-05 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018482+0000", 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6091000000000003e-05 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.314 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220987 7.7 7:eba202fb:::benchmark_data_smithi043_70646_object103579:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.896951+0000", 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 275.33316881399998, 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12351648699999999, 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220987 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.315 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896951+0000", 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896951+0000", 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896952+0000", 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9700000000000005e-07 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.316 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896956+0000", 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3869999999999997e-06 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896967+0000", 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0363000000000001e-05 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896980+0000", 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3407999999999999e-05 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.897040+0000", 2024-09-21T11:49:02.317 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9947000000000001e-05 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.897148+0000", 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010828900000000001 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.897255+0000", 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010709000000000001 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.318 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.899364+0000", 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021090129999999999 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.901062+0000", 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016971289999999999 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.020394+0000", 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11933194699999999 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.319 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.020435+0000", 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1430000000000001e-05 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.020468+0000", 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2586999999999998e-05 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221226 7.d 7:b01dbbaf:::benchmark_data_smithi043_70646_object103818:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:49:02.320 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.186275+0000", 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 275.04384527500002, 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10689198599999999, 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221226 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186275+0000", 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.321 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186275+0000", 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186277+0000", 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.018e-06 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186282+0000", 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4890000000000003e-06 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.322 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186295+0000", 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2962999999999999e-05 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186303+0000", 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4619999999999999e-06 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186363+0000", 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0896000000000001e-05 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.323 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186460+0000", 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6302000000000001e-05 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186558+0000", 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8073999999999996e-05 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.189168+0000", 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002609765 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.276314+0000", 2024-09-21T11:49:02.324 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.087146766000000001 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293019+0000", 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016704455 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293108+0000", 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9046000000000001e-05 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293167+0000", 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8749999999999998e-05 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.325 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46252 8.9 8:969b922f:::benchmark_data_smithi043_97042_object46251:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.597214+0000", 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 219.632905381, 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11913404700000001, 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46252 2024-09-21T11:49:02.326 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597214+0000", 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597214+0000", 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597222+0000", 2024-09-21T11:49:02.327 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8909999999999995e-06 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597262+0000", 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9393e-05 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597277+0000", 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5421e-05 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597284+0000", 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1620000000000002e-06 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.328 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597351+0000", 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6948999999999999e-05 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597501+0000", 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014983499999999999 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597630+0000", 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012867 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604093+0000", 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0064636370000000004 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604140+0000", 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6717e-05 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716241+0000", 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.112100726 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.330 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716302+0000", 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0639e-05 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716349+0000", 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7006999999999999e-05 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46253 8.9 8:9d2b1260:::benchmark_data_smithi043_97042_object46252:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.597498+0000", 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 219.632621435, 2024-09-21T11:49:02.331 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.140088831, 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46253 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597498+0000", 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.332 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597498+0000", 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597499+0000", 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3399999999999998e-07 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597572+0000", 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3135999999999999e-05 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597580+0000", 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0830000000000004e-06 2024-09-21T11:49:02.333 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597588+0000", 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9969999999999998e-06 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.599249+0000", 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016605820000000001 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.599363+0000", 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011366600000000001 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.334 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.599532+0000", 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016955299999999999 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604099+0000", 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045664690000000001 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604188+0000", 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8968000000000006e-05 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.335 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737476+0000", 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.133288615 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737535+0000", 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8573000000000003e-05 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737587+0000", 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2454999999999998e-05 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.336 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46254 8.9 8:9dcdf461:::benchmark_data_smithi043_97042_object46253:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.598445+0000", 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 219.631674989, 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13929741600000001, 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46254 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598445+0000", 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598445+0000", 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598446+0000", 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9999999999999995e-07 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598516+0000", 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9726000000000003e-05 2024-09-21T11:49:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598526+0000", 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0349999999999999e-05 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598533+0000", 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7129999999999999e-06 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.601614+0000", 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030816649999999999 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.339 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.601758+0000", 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014389200000000001 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.601919+0000", 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016073800000000001 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604200+0000", 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022812470000000001 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.340 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604222+0000", 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1481e-05 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737643+0000", 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.133421174 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737704+0000", 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0919999999999999e-05 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.341 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737742+0000", 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8510000000000002e-05 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46259 8.9 8:9a5eb1e7:::benchmark_data_smithi043_97042_object46258:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.600174+0000", 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 219.629946209, 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13768232, 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.342 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46259 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600174+0000", 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600174+0000", 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.343 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600175+0000", 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2899999999999999e-06 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600235+0000", 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0517999999999999e-05 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600260+0000", 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4201000000000001e-05 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.344 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600267+0000", 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2640000000000002e-06 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.603150+0000", 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002883409 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.603264+0000", 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113286 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.345 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.603398+0000", 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013435899999999999 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604295+0000", 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00089736199999999996 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604666+0000", 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370379 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.346 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737780+0000", 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13311397799999999 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737824+0000", 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4549999999999999e-05 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737856+0000", 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1724000000000003e-05 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.347 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47790 8.9 8:98dbfe67:::benchmark_data_smithi043_97042_object47789:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.712965+0000", 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 218.51715467, 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13282617399999999, 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47790 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.348 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712965+0000", 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712965+0000", 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712967+0000", 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.59e-06 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.349 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713048+0000", 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1173000000000004e-05 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713073+0000", 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5333999999999998e-05 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713088+0000", 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4436e-05 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.350 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713161+0000", 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3152999999999996e-05 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713271+0000", 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109849 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713392+0000", 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000121477 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.351 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.715175+0000", 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017832729999999999 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.717167+0000", 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001991044 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845693+0000", 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12852650299999999 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.352 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845750+0000", 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6895999999999999e-05 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845791+0000", 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1446000000000003e-05 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66144 8.9 8:9c2c8472:::benchmark_data_smithi043_97042_object66143:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.930304+0000", 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 206.29981577800001, 2024-09-21T11:49:02.353 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12634800199999999, 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66144 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930304+0000", 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.354 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930304+0000", 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930305+0000", 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.342e-06 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930332+0000", 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6105000000000002e-05 2024-09-21T11:49:02.355 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930341+0000", 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7739999999999999e-06 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930347+0000", 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6149999999999996e-06 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930389+0000", 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1933e-05 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.356 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930471+0000", 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2466999999999995e-05 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930559+0000", 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8022999999999994e-05 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.932240+0000", 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016807599999999999 2024-09-21T11:49:02.357 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.933402+0000", 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011622270000000001 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.056538+0000", 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12313563700000001 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.056610+0000", 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1798000000000006e-05 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:49:02.358 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:49:02.359 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.056652+0000", 2024-09-21T11:49:02.359 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2320999999999997e-05 2024-09-21T11:49:02.359 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.359 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.359 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.359 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:49:02.359 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:49:02.359 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:49:02.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:02.382 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:02.382 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:02.482 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:02.503 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:02.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:02.504 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:02.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:02.626 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:02.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:02.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:02.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:02.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:02.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:02.749 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:02.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:02.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:02.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:02.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:02.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:02.874 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:02.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:02.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:02.998 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:02.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:02.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:03.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:49:03.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:03.122 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:03.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:03.123 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:03.224 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:03.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:03.245 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:03.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:03.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:03.371 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:03.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:03.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:03.380 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:49:03.473 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:03.479 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'unique_pool_1' removed 2024-09-21T11:49:03.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:03 smithi175 ceph-mon[32613]: pgmap v430: 148 pgs: 148 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:49:03.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:03 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4286894896' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:49:03.485 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:03 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:49:03.501 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:03.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:03.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:03.603 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:03.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:49:03.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:03.631 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:03.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:03.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:03 smithi043 ceph-mon[36454]: pgmap v430: 148 pgs: 148 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:49:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:03 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4286894896' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:49:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:03 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:49:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:03 smithi043 ceph-mon[29710]: pgmap v430: 148 pgs: 148 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:49:03.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:03 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4286894896' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:49:03.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:03 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:49:03.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:03.755 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:03.755 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:03.856 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:03.878 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:03.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:03.879 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:03.893 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:49:03.893 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:49:03.893 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:49:03.893 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:49:03.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:04.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:04.004 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:04.019 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:49:04.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:04.125 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:04.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:04.126 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:04.128 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-21T11:49:04.128 INFO:tasks.radosbench:Beginning radosbench... 2024-09-21T11:49:04.129 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-21T11:49:04.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool create unique_pool_2 16 2024-09-21T11:49:04.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:04.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:04.256 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:04.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:04.274 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:04.357 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:04.378 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:04.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:04.379 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:04.480 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:04.502 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:04.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:04.503 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:04.604 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:04.626 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:04.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:04.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:04.636 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:49:04.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:04 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:49:04.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:04 smithi043 ceph-mon[36454]: osdmap e245: 8 total, 8 up, 8 in 2024-09-21T11:49:04.663 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:04 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:49:04.663 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:04 smithi043 ceph-mon[29710]: osdmap e245: 8 total, 8 up, 8 in 2024-09-21T11:49:04.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:04.750 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:04.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:04.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:04.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:04 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:49:04.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:04 smithi175 ceph-mon[32613]: osdmap e245: 8 total, 8 up, 8 in 2024-09-21T11:49:04.850 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:04.853 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:04.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:04.877 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:04.877 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:04.927 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:49:04.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:05.002 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:05.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:05.003 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:05.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:05.125 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:05.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:05.126 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:05.173 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:49:05.173 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:49:05.227 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:05.249 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:05.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:05.250 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:05.281 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:49:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:49:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:49:05.351 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:05.372 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:05.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:05.373 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:05.382 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:49:05.383 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:49:05.392 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:49:05.393 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:49:05.393 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:49:05.393 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:49:05.393 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:49:05.399 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:49:05.400 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:49:05.415 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:49:05.416 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:49:05.432 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:49:05.432 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:49:05.455 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:49:05.455 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:49:05.473 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:49:05.473 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:49:05.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:05.480 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:05.494 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:49:05.495 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:49:05.495 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:49:05.502 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:05.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:05.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:05.520 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:49:05.520 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:49:05.534 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:05 smithi043 ceph-mon[29710]: pgmap v432: 132 pgs: 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:49:05.534 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:05 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3056844195' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:49:05.534 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:05 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:49:05.534 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:05 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/694002837' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:05.534 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:05 smithi043 ceph-mon[36454]: pgmap v432: 132 pgs: 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:49:05.534 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:05 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3056844195' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:49:05.535 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:05 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:49:05.535 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:05 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/694002837' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:05.604 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:05.625 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:05.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:05.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:05.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:05 smithi175 ceph-mon[32613]: pgmap v432: 132 pgs: 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:49:05.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:05 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3056844195' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:49:05.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:05 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:49:05.713 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:05 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/694002837' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:05.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:05.749 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:05.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:05.750 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:05.850 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:05.871 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:05.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:05.872 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:05.973 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:06.003 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:06.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:06.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:06.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:49:06.105 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:06.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:06.129 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:06.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:06.251 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:06.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:06.252 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:06.300 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:49:06.353 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:06.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:06.375 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:06.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:06.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:06.499 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:06.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:06.500 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:06.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:06.623 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:06.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:06.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:06.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:06 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:06 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:49:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:06 smithi043 ceph-mon[36454]: osdmap e246: 8 total, 8 up, 8 in 2024-09-21T11:49:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:06 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:06 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:49:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:06 smithi043 ceph-mon[29710]: osdmap e246: 8 total, 8 up, 8 in 2024-09-21T11:49:06.745 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:06.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:06.746 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:06.749 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:06 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:06.749 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:06 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:49:06.749 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:06 smithi175 ceph-mon[32613]: osdmap e246: 8 total, 8 up, 8 in 2024-09-21T11:49:06.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:06.849 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:49:06.849 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:49:06.870 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:06.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:06.871 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:06.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:06.994 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:06.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:06.995 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:07.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:07.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:07.119 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:07.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:07.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:07.242 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:07.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:07.242 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:07.343 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:07.365 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:07.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:07.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:07.468 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:07.490 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:07.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:07.491 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'unique_pool_2' created 2024-09-21T11:49:07.592 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:07.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:49:07.613 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:07.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:07.614 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:07.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:07.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:07.738 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:07.739 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:07.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[29710]: pgmap v434: 132 pgs: 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T11:49:07.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[29710]: 5.4 scrub starts 2024-09-21T11:49:07.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[29710]: 5.4 scrub ok 2024-09-21T11:49:07.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1038902448' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-21T11:49:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[29710]: 4.11 scrub starts 2024-09-21T11:49:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[29710]: 4.11 scrub ok 2024-09-21T11:49:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[36454]: pgmap v434: 132 pgs: 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T11:49:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[36454]: 5.4 scrub starts 2024-09-21T11:49:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[36454]: 5.4 scrub ok 2024-09-21T11:49:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1038902448' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-21T11:49:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[36454]: 4.11 scrub starts 2024-09-21T11:49:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:07 smithi043 ceph-mon[36454]: 4.11 scrub ok 2024-09-21T11:49:07.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:07 smithi175 ceph-mon[32613]: pgmap v434: 132 pgs: 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T11:49:07.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:07 smithi175 ceph-mon[32613]: 5.4 scrub starts 2024-09-21T11:49:07.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:07 smithi175 ceph-mon[32613]: 5.4 scrub ok 2024-09-21T11:49:07.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1038902448' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-21T11:49:07.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:07 smithi175 ceph-mon[32613]: 4.11 scrub starts 2024-09-21T11:49:07.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:07 smithi175 ceph-mon[32613]: 4.11 scrub ok 2024-09-21T11:49:07.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:07.862 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:07.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:07.863 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:07.964 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:07.986 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:07.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:07.987 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:08.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:08.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:08.114 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:08.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-21T11:49:08.193 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:08.215 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:08.237 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:08.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:08.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:08.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:08.362 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:08.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:08.363 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:08.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:08.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:08.486 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:08.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:08.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:08.611 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:08.670 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:49:08.670 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:49:08.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:08.733 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:08.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:08.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[29710]: 6.0 scrub starts 2024-09-21T11:49:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[29710]: 6.0 scrub ok 2024-09-21T11:49:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1038902448' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-21T11:49:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[29710]: osdmap e247: 8 total, 8 up, 8 in 2024-09-21T11:49:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[29710]: 6.1a scrub starts 2024-09-21T11:49:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[29710]: 6.1a scrub ok 2024-09-21T11:49:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[29710]: 6.13 scrub starts 2024-09-21T11:49:08.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[29710]: 6.13 scrub ok 2024-09-21T11:49:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[36454]: 6.0 scrub starts 2024-09-21T11:49:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[36454]: 6.0 scrub ok 2024-09-21T11:49:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1038902448' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-21T11:49:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[36454]: osdmap e247: 8 total, 8 up, 8 in 2024-09-21T11:49:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[36454]: 6.1a scrub starts 2024-09-21T11:49:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[36454]: 6.1a scrub ok 2024-09-21T11:49:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[36454]: 6.13 scrub starts 2024-09-21T11:49:08.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:08 smithi043 ceph-mon[36454]: 6.13 scrub ok 2024-09-21T11:49:08.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:08 smithi175 ceph-mon[32613]: 6.0 scrub starts 2024-09-21T11:49:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:08 smithi175 ceph-mon[32613]: 6.0 scrub ok 2024-09-21T11:49:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:08 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1038902448' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-21T11:49:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:08 smithi175 ceph-mon[32613]: osdmap e247: 8 total, 8 up, 8 in 2024-09-21T11:49:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:08 smithi175 ceph-mon[32613]: 6.1a scrub starts 2024-09-21T11:49:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:08 smithi175 ceph-mon[32613]: 6.1a scrub ok 2024-09-21T11:49:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:08 smithi175 ceph-mon[32613]: 6.13 scrub starts 2024-09-21T11:49:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:08 smithi175 ceph-mon[32613]: 6.13 scrub ok 2024-09-21T11:49:08.835 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:08.857 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:08.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:08.858 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:08.917 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:08.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:49:08.959 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:08.980 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:08.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:08.981 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:09.082 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:09.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:09.105 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:09.105 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:09.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:09.228 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:09.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:09.229 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:09.330 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:09.352 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:09.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:09.353 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:09.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:49:09.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:49:09.454 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:09.476 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:09.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:09.477 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:09.578 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:09.600 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:09.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:09.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:09.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: pgmap v436: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:49:09.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:49:09.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: osdmap e248: 8 total, 8 up, 8 in 2024-09-21T11:49:09.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: 3.e scrub starts 2024-09-21T11:49:09.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: 3.e scrub ok 2024-09-21T11:49:09.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: 6.1d scrub starts 2024-09-21T11:49:09.632 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: 6.1d scrub ok 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: 4.d scrub starts 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: 4.d scrub ok 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: 6.1b scrub starts 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[36454]: 6.1b scrub ok 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: pgmap v436: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: osdmap e248: 8 total, 8 up, 8 in 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: 3.e scrub starts 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: 3.e scrub ok 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: 6.1d scrub starts 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: 6.1d scrub ok 2024-09-21T11:49:09.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: 4.d scrub starts 2024-09-21T11:49:09.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: 4.d scrub ok 2024-09-21T11:49:09.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: 6.1b scrub starts 2024-09-21T11:49:09.634 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:09 smithi043 ceph-mon[29710]: 6.1b scrub ok 2024-09-21T11:49:09.702 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:09.724 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:09.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:09.725 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:09.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: pgmap v436: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:49:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:49:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: osdmap e248: 8 total, 8 up, 8 in 2024-09-21T11:49:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: 3.e scrub starts 2024-09-21T11:49:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: 3.e scrub ok 2024-09-21T11:49:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: 6.1d scrub starts 2024-09-21T11:49:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: 6.1d scrub ok 2024-09-21T11:49:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: 4.d scrub starts 2024-09-21T11:49:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: 4.d scrub ok 2024-09-21T11:49:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: 6.1b scrub starts 2024-09-21T11:49:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:09 smithi175 ceph-mon[32613]: 6.1b scrub ok 2024-09-21T11:49:09.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:09.850 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:09.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:09.851 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:09.949 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:09.951 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:09.974 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:09.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:09.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:09.981 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:10.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:10.098 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:10.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:10.099 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:10.200 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:10.222 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:10.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:10.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:10.324 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:10.346 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:10.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:10.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:10.448 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:10.460 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:49:10.460 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:49:10.471 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:10.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:10.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:10.529 INFO:teuthology.orchestra.run.smithi043.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-21T11:49:10.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:10.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:10.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:10.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:10.697 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:10.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:10.721 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:10.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:10.822 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:10.844 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:10.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:10.845 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:10.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:10.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:10.970 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T11:49:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T11:49:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[36454]: osdmap e249: 8 total, 8 up, 8 in 2024-09-21T11:49:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[36454]: 4.1 scrub starts 2024-09-21T11:49:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[36454]: 4.1 scrub ok 2024-09-21T11:49:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3277350717' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:49:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:49:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T11:49:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T11:49:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[29710]: osdmap e249: 8 total, 8 up, 8 in 2024-09-21T11:49:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[29710]: 4.1 scrub starts 2024-09-21T11:49:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[29710]: 4.1 scrub ok 2024-09-21T11:49:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3277350717' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:49:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:10 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:49:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:10 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T11:49:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:10 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T11:49:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:10 smithi175 ceph-mon[32613]: osdmap e249: 8 total, 8 up, 8 in 2024-09-21T11:49:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:10 smithi175 ceph-mon[32613]: 4.1 scrub starts 2024-09-21T11:49:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:10 smithi175 ceph-mon[32613]: 4.1 scrub ok 2024-09-21T11:49:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3277350717' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:49:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:10 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:49:11.071 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:11.093 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:11.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:11.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:11.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:11.217 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:11.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:11.218 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:11.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:11.345 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:11.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:11.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:11.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:11.470 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:11.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:11.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:11.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:11.594 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:11.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:11.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:11.642 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:11 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T11:49:11.642 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:11 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T11:49:11.642 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:11 smithi043 ceph-mon[36454]: pgmap v439: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:49:11.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:11 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:49:11.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:11 smithi043 ceph-mon[36454]: osdmap e250: 8 total, 8 up, 8 in 2024-09-21T11:49:11.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:11 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T11:49:11.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:11 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T11:49:11.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:11 smithi043 ceph-mon[29710]: pgmap v439: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:49:11.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:11 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:49:11.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:11 smithi043 ceph-mon[29710]: osdmap e250: 8 total, 8 up, 8 in 2024-09-21T11:49:11.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:49:11.696 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:11.722 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:11.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:11.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:11 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T11:49:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:11 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T11:49:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:11 smithi175 ceph-mon[32613]: pgmap v439: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:49:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:11 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:49:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:11 smithi175 ceph-mon[32613]: osdmap e250: 8 total, 8 up, 8 in 2024-09-21T11:49:11.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:49:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:49:11.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:11.850 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:11.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:11.851 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:11.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:11.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:11.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:12.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:12.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:12.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:12.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:12.223 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:12.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:12.224 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:12.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:12.347 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:12.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:12.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:12.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:12.471 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:12.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:12.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:12.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:12.596 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:12.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:12.598 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:12.698 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:12.724 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:12.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:12.725 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:12.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:12 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:12.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:12 smithi043 ceph-mon[29710]: 3.18 scrub starts 2024-09-21T11:49:12.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:12 smithi043 ceph-mon[29710]: 3.18 scrub ok 2024-09-21T11:49:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:12 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:12 smithi043 ceph-mon[36454]: 3.18 scrub starts 2024-09-21T11:49:12.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:12 smithi043 ceph-mon[36454]: 3.18 scrub ok 2024-09-21T11:49:12.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:12.848 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:12.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:12.849 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:12.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:12.974 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:12.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:12.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:13.010 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:13.010 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:49:13.011 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:49:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:12 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:12 smithi175 ceph-mon[32613]: 3.18 scrub starts 2024-09-21T11:49:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:12 smithi175 ceph-mon[32613]: 3.18 scrub ok 2024-09-21T11:49:13.037 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:49:13.038 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:49:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:49:13.065 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:49:13.075 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:13.080 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:49:13.081 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:49:13.098 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:49:13.098 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:49:13.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:13.103 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:13.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:13.115 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:49:13.116 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:49:13.133 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:49:13.133 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:49:13.150 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:49:13.150 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:49:13.193 DEBUG:teuthology.orchestra.run.smithi043:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-21T11:49:13.193 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-21T11:49:13.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:49:13.196 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-21T11:49:13.196 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-21T11:49:13.203 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:13.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:13.228 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:13.228 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:13.252 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:hints = 1 2024-09-21T11:49:13.252 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-21T11:49:13.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Object prefix: benchmark_data_smithi043_126002 2024-09-21T11:49:13.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:49:13.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 0 0 0 0 0 0 - 0 2024-09-21T11:49:13.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:13.360 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:13.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:13.361 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:13.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:13.491 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:13.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:13.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:49:13.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:13.623 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:13.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:13.624 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:13.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:13.768 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:13.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:13.769 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:13.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:13.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:13.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:13.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:13.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[36454]: pgmap v441: 148 pgs: 148 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:49:13.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[36454]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:49:13.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[36454]: 6.3 scrub starts 2024-09-21T11:49:13.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[36454]: 6.3 scrub ok 2024-09-21T11:49:13.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2284322356' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:13.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1968969033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[29710]: pgmap v441: 148 pgs: 148 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:49:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[29710]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:49:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[29710]: 6.3 scrub starts 2024-09-21T11:49:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[29710]: 6.3 scrub ok 2024-09-21T11:49:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2284322356' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1968969033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:13 smithi175 ceph-mon[32613]: pgmap v441: 148 pgs: 148 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:49:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:13 smithi175 ceph-mon[32613]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:49:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:13 smithi175 ceph-mon[32613]: 6.3 scrub starts 2024-09-21T11:49:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:13 smithi175 ceph-mon[32613]: 6.3 scrub ok 2024-09-21T11:49:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2284322356' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1968969033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:14.012 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:14.043 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:14.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:14.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:14.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:14.174 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:14.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:14.175 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:14.252 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 1 16 3176 3160 197.489 197.5 0.00388197 0.00504009 2024-09-21T11:49:14.276 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:14.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:14.309 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:14.310 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:14.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:14.445 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:14.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:14.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:14.547 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:14.575 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:14.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:14.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:14.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:14.710 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:14.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:14.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:14.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:14.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[29710]: 3.16 scrub starts 2024-09-21T11:49:14.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[29710]: 3.16 scrub ok 2024-09-21T11:49:14.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2284322356' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[29710]: osdmap e251: 8 total, 8 up, 8 in 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[29710]: 4.1b scrub starts 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[29710]: 4.1b scrub ok 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[29710]: 5.0 scrub starts 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[29710]: 5.0 scrub ok 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[36454]: 3.16 scrub starts 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[36454]: 3.16 scrub ok 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:49:14.824 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2284322356' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:14.825 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[36454]: osdmap e251: 8 total, 8 up, 8 in 2024-09-21T11:49:14.825 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[36454]: 4.1b scrub starts 2024-09-21T11:49:14.825 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[36454]: 4.1b scrub ok 2024-09-21T11:49:14.825 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[36454]: 5.0 scrub starts 2024-09-21T11:49:14.825 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:14 smithi043 ceph-mon[36454]: 5.0 scrub ok 2024-09-21T11:49:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:14.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:14.866 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:14.921 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:49:14.921 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:49:14.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:49:14.921 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:49:14.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.921 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:220988 7.f 7:fa5453fd:::benchmark_data_smithi043_70646_object103580:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:49:14.921 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:44:26.898041+0000", 2024-09-21T11:49:14.921 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 288.01944945499997, 2024-09-21T11:49:14.921 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.125302578, 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 220988 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.922 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898041+0000", 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898041+0000", 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898042+0000", 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0729999999999999e-06 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.923 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898046+0000", 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.506e-06 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898053+0000", 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0380000000000002e-06 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898059+0000", 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4459999999999998e-06 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898124+0000", 2024-09-21T11:49:14.924 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5030999999999995e-05 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898228+0000", 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103584 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.898341+0000", 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113522 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.900967+0000", 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002625432 2024-09-21T11:49:14.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:26.901062+0000", 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5518000000000003e-05 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.023195+0000", 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.122133055 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.926 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.023295+0000", 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9841999999999997e-05 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:44:27.023344+0000", 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8531e-05 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.927 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43882 8.c 8:35a2a8fe:::benchmark_data_smithi043_97042_object43881:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 233.94352762099999, 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14113139399999999, 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43882 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.928 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973964+0000", 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.28e-06 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.929 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973985+0000", 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1178e-05 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973994+0000", 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0119999999999994e-06 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974001+0000", 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4570000000000002e-06 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974047+0000", 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6393000000000001e-05 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974162+0000", 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001149 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974245+0000", 2024-09-21T11:49:14.931 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2937999999999998e-05 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.975846+0000", 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001600944 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976044+0000", 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019846299999999999 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114984+0000", 2024-09-21T11:49:14.932 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13893909600000001 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115046+0000", 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2924999999999998e-05 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115094+0000", 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7808e-05 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.934 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43886 8.e 8:7c60ce26:::benchmark_data_smithi043_97042_object43885:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:49:14.934 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:49:14.934 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 233.940615092, 2024-09-21T11:49:14.934 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13826519300000001, 2024-09-21T11:49:14.934 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.934 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:14.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:14.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:14.935 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:14.935 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43886 2024-09-21T11:49:14.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.935 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.935 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.935 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:49:14.935 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976877+0000", 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976898+0000", 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1528000000000001e-05 2024-09-21T11:49:14.936 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976910+0000", 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1565e-05 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976918+0000", 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7090000000000002e-06 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976966+0000", 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8168999999999999e-05 2024-09-21T11:49:14.937 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.977045+0000", 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9594999999999999e-05 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.977134+0000", 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8233999999999996e-05 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:14.938 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.978395+0000", 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012616459999999999 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.978816+0000", 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000420522 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981299+0000", 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024827949999999999 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981332+0000", 2024-09-21T11:49:14.939 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3321999999999997e-05 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115141+0000", 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133808599 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43890 8.6 8:68dae88a:::benchmark_data_smithi043_97042_object43889:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:49:14.940 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 233.93748705600001, 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13554245000000001, 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43890 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:49:14.941 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980005+0000", 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2470000000000001e-06 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.942 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980022+0000", 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7380999999999998e-05 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980030+0000", 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999995e-06 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980035+0000", 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7090000000000001e-06 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980083+0000", 2024-09-21T11:49:14.943 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7709e-05 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980191+0000", 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010809799999999999 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980327+0000", 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013611600000000001 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115211+0000", 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13488383300000001 2024-09-21T11:49:14.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115245+0000", 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4292000000000003e-05 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115423+0000", 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017810499999999999 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115476+0000", 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2747000000000002e-05 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115546+0000", 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0159000000000004e-05 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:43886 8.e e172/160 ondisk, result = 0)", 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 233.93626709099999, 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133923551, 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981224+0000", 2024-09-21T11:49:14.947 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5899999999999996e-07 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981227+0000", 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8609999999999998e-06 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981234+0000", 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4039999999999998e-06 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981239+0000", 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3939999999999998e-06 2024-09-21T11:49:14.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981291+0000", 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2352000000000003e-05 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981296+0000", 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6079999999999998e-06 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115147+0000", 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133851373 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.949 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:43882 8.c e172/160 ondisk, result = 0)", 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.982128+0000", 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 233.93536280000001, 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13297520400000001, 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.982128+0000", 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.982128+0000", 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.982128+0000", 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2900000000000003e-07 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.982131+0000", 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1439999999999999e-06 2024-09-21T11:49:14.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114868+0000", 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13273664199999999 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114884+0000", 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5569e-05 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.952 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114967+0000", 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3372000000000001e-05 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114977+0000", 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6779999999999995e-06 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115103+0000", 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012616999999999999 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.953 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:43890 8.6 e172/160 ondisk, result = 0)", 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.983212+0000", 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 233.93427875200001, 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13234341699999999, 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983212+0000", 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983212+0000", 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983213+0000", 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8000000000000005e-07 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983217+0000", 2024-09-21T11:49:14.955 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5879999999999999e-06 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115347+0000", 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.132130321 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115357+0000", 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0016e-05 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.956 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115409+0000", 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1205000000000002e-05 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115417+0000", 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5730000000000008e-06 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115555+0000", 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013793400000000001 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:46235 8.2 e173/160 8:4e6f27b7:::benchmark_data_smithi043_97042_object46234:head v 173'2880, mlcod=173'2878)", 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:22.590295+0000", 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 232.32719589000001, 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12607759099999999, 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590295+0000", 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590295+0000", 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590295+0000", 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7199999999999998e-07 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590327+0000", 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1486000000000003e-05 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590336+0000", 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9830000000000002e-06 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590342+0000", 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5169999999999996e-06 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590412+0000", 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9840999999999998e-05 2024-09-21T11:49:14.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711199+0000", 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.120787109 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.716291+0000", 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050917089999999998 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.961 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.716372+0000", 2024-09-21T11:49:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1273999999999995e-05 2024-09-21T11:49:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:46239 8.1 8:8e86b97b:::benchmark_data_smithi043_97042_object46238:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:14.962 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 232.326735386, 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.144299073, 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46239 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:49:14.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590756+0000", 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590781+0000", 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5000000000000001e-05 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711205+0000", 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.120423856 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711223+0000", 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.751e-05 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711326+0000", 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103274 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711483+0000", 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015652 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711616+0000", 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000133518 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.715084+0000", 2024-09-21T11:49:14.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034678579999999999 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.715128+0000", 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3659000000000003e-05 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.734947+0000", 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019819593999999999 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.735006+0000", 2024-09-21T11:49:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8415000000000002e-05 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.735054+0000", 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8513000000000001e-05 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47773 8.6 8:6c5bd339:::benchmark_data_smithi043_97042_object47772:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:14.968 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 231.21345466299999, 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.141128856, 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47773 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:49:14.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704037+0000", 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-06 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704057+0000", 2024-09-21T11:49:14.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0608999999999999e-05 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704068+0000", 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0611000000000001e-05 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704077+0000", 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1710000000000008e-06 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704131+0000", 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3525e-05 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704244+0000", 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113169 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704377+0000", 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013258899999999999 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:14.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.706707+0000", 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023301260000000001 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.706973+0000", 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026666 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845071+0000", 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13809718700000001 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845116+0000", 2024-09-21T11:49:14.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5581999999999998e-05 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845165+0000", 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8603999999999997e-05 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47778 8.e 8:7e5bf474:::benchmark_data_smithi043_97042_object47777:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 231.21000971999999, 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13783190200000001, 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.974 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47778 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707482+0000", 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2750000000000001e-06 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707504+0000", 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1684000000000001e-05 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.976 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707515+0000", 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0844e-05 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707522+0000", 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6090000000000003e-06 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707577+0000", 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4703000000000001e-05 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707687+0000", 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110474 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707808+0000", 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012113299999999999 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845066+0000", 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.137257564 2024-09-21T11:49:14.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845181+0000", 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011519 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845218+0000", 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6356000000000002e-05 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845277+0000", 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9357000000000001e-05 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845313+0000", 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5713000000000002e-05 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.980 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47781 8.6 8:69458834:::benchmark_data_smithi043_97042_object47780:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 231.20919922300001, 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13727709099999999, 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47781 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708293+0000", 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2440000000000001e-06 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708313+0000", 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0846e-05 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708324+0000", 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0563e-05 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708331+0000", 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0820000000000001e-06 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.983 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708421+0000", 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0422999999999998e-05 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708530+0000", 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108286 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708651+0000", 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120821 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:14.984 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845324+0000", 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.136673447 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845358+0000", 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3760999999999997e-05 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845478+0000", 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120512 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845531+0000", 2024-09-21T11:49:14.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2714999999999999e-05 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845568+0000", 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7391000000000002e-05 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47784 8.6 8:65eeb18a:::benchmark_data_smithi043_97042_object47783:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 231.207314844, 2024-09-21T11:49:14.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13626728099999999, 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47784 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:49:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710177+0000", 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.229e-06 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710199+0000", 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469000000000001e-05 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710210+0000", 2024-09-21T11:49:14.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0387000000000001e-05 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710217+0000", 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8709999999999999e-06 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710448+0000", 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023145800000000001 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710539+0000", 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0539999999999997e-05 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710618+0000", 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9515e-05 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845369+0000", 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134750694 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845618+0000", 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024913600000000001 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846364+0000", 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074579699999999997 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846408+0000", 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4425000000000002e-05 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846443+0000", 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4759999999999999e-05 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47773 8.6 e173/160 ondisk, result = 0)", 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 231.20725120500001, 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134942961, 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:14.993 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:14.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710240+0000", 2024-09-21T11:49:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5600000000000001e-07 2024-09-21T11:49:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710242+0000", 2024-09-21T11:49:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2239999999999998e-06 2024-09-21T11:49:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710248+0000", 2024-09-21T11:49:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5949999999999998e-06 2024-09-21T11:49:14.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710252+0000", 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3950000000000003e-06 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845062+0000", 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13480961599999999 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845067+0000", 2024-09-21T11:49:14.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1019999999999996e-06 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845182+0000", 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115573 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47778 8.e e173/160 ondisk, result = 0)", 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:49:14.998 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 231.207203419, 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134785869, 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:14.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710290+0000", 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5339999999999998e-06 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710296+0000", 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0970000000000002e-06 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710301+0000", 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7809999999999996e-06 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845053+0000", 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13475179800000001 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845061+0000", 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845073+0000", 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2425999999999999e-05 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.002 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47781 8.6 e173/160 ondisk, result = 0)", 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 231.20653297800001, 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13441215100000001, 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710962+0000", 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.692e-06 2024-09-21T11:49:15.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842829+0000", 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13186700100000001 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842844+0000", 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5281000000000002e-05 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845347+0000", 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002502659 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845353+0000", 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6510000000000004e-06 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845370+0000", 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6214000000000001e-05 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.006 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47792 8.e 8:71619300:::benchmark_data_smithi043_97042_object47791:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 231.20430544800001, 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14195813500000001, 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47792 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713186+0000", 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713211+0000", 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4343999999999999e-05 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.843341+0000", 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13012984499999999 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.843353+0000", 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2554e-05 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845654+0000", 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023008730000000002 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845787+0000", 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013336700000000001 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845921+0000", 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000134031 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.853059+0000", 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071370510000000002 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.853133+0000", 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4765000000000007e-05 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855050+0000", 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019165880000000001 2024-09-21T11:49:15.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855099+0000", 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8680000000000001e-05 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855143+0000", 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4626000000000003e-05 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:66142 8.1 8:87e3d573:::benchmark_data_smithi043_97042_object66141:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:49:15.012 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 218.99126505800001, 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13618391299999999, 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 66142 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:49:15.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926226+0000", 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3500000000000005e-07 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:15.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926267+0000", 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0932e-05 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926277+0000", 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8609999999999993e-06 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926283+0000", 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4010000000000002e-06 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:15.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926387+0000", 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103999 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926630+0000", 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000242938 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926754+0000", 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123433 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.930232+0000", 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034784540000000002 2024-09-21T11:49:15.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:15.017 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:14 smithi175 ceph-mon[32613]: 3.16 scrub starts 2024-09-21T11:49:15.017 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:14 smithi175 ceph-mon[32613]: 3.16 scrub ok 2024-09-21T11:49:15.017 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:14 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:49:15.017 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:14 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:49:15.018 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2284322356' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:15.018 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:14 smithi175 ceph-mon[32613]: osdmap e251: 8 total, 8 up, 8 in 2024-09-21T11:49:15.018 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:14 smithi175 ceph-mon[32613]: 4.1b scrub starts 2024-09-21T11:49:15.018 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:14 smithi175 ceph-mon[32613]: 4.1b scrub ok 2024-09-21T11:49:15.018 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:14 smithi175 ceph-mon[32613]: 5.0 scrub starts 2024-09-21T11:49:15.018 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:14 smithi175 ceph-mon[32613]: 5.0 scrub ok 2024-09-21T11:49:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.930829+0000", 2024-09-21T11:49:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059640099999999998 2024-09-21T11:49:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.059677+0000", 2024-09-21T11:49:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12884799699999999 2024-09-21T11:49:15.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.059734+0000", 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7191999999999997e-05 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.062409+0000", 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026754700000000001 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.019 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:214600 8.e 8:73033169:::benchmark_data_smithi043_97042_object77506:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:45.010179+0000", 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.907311258999997, 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.126581217, 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 214600 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:15.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010179+0000", 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010179+0000", 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010180+0000", 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.19e-06 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010185+0000", 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3379999999999998e-06 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010193+0000", 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9540000000000004e-06 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010199+0000", 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3060000000000003e-06 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.019393+0000", 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0091935850000000006 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.019525+0000", 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000132762 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.019639+0000", 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011328900000000001 2024-09-21T11:49:15.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.029430+0000", 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0097918099999999998 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.029461+0000", 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.044e-05 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.136664+0000", 2024-09-21T11:49:15.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.10720318299999999 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.136716+0000", 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2232999999999999e-05 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.136760+0000", 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4127e-05 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:214603 8.e 8:77636cf3:::benchmark_data_smithi043_97042_object77509:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:45.010326+0000", 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 89.907164194000003, 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.126728642, 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 214603 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:15.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010326+0000", 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010326+0000", 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010327+0000", 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7300000000000005e-07 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010331+0000", 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4769999999999999e-06 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010348+0000", 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7924e-05 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.010355+0000", 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1480000000000003e-06 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.023017+0000", 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01266195 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.023133+0000", 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116144 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.026474+0000", 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003340843 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:15.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.031178+0000", 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047048250000000002 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.033571+0000", 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023923550000000001 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.136797+0000", 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.103226691 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:15.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.137012+0000", 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000214825 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:45.137055+0000", 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2687e-05 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:15.031 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:49:15.049 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:15.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:15.050 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:15.150 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:15.190 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:15.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:15.192 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:15.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 2 15 6323 6308 197.11 196.75 0.00374289 0.00506246 2024-09-21T11:49:15.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:15.333 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:15.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:15.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:15.434 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:15.465 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:15.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:15.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:15.487 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:49:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:49:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:49:15.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:15.605 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:15.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:15.606 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:15.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:15.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:15 smithi043 ceph-mon[36454]: pgmap v443: 148 pgs: 148 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-09-21T11:49:15.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:15 smithi043 ceph-mon[36454]: 4.1a scrub starts 2024-09-21T11:49:15.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:15 smithi043 ceph-mon[36454]: 4.1a scrub ok 2024-09-21T11:49:15.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:15 smithi043 ceph-mon[29710]: pgmap v443: 148 pgs: 148 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-09-21T11:49:15.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:15 smithi043 ceph-mon[29710]: 4.1a scrub starts 2024-09-21T11:49:15.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:15 smithi043 ceph-mon[29710]: 4.1a scrub ok 2024-09-21T11:49:15.748 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:15.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:15.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:15.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:15.889 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:15.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:15.890 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:15.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:16.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:49:16.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:15 smithi175 ceph-mon[32613]: pgmap v443: 148 pgs: 148 active+clean; 583 KiB data, 2.6 GiB used, 713 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-09-21T11:49:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:15 smithi175 ceph-mon[32613]: 4.1a scrub starts 2024-09-21T11:49:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:15 smithi175 ceph-mon[32613]: 4.1a scrub ok 2024-09-21T11:49:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:16.019 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:16.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:16.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:16.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:16.170 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:16.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:16.171 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:16.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 3 16 9319 9303 193.797 187.188 0.00503739 0.00514829 2024-09-21T11:49:16.272 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:16.303 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:16.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:16.304 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:16.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:16.465 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:16.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:16.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:16.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:16.608 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:16.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:16.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:16.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:49:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:49:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[36454]: osdmap e252: 8 total, 8 up, 8 in 2024-09-21T11:49:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[36454]: 4.f scrub starts 2024-09-21T11:49:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[36454]: 4.f scrub ok 2024-09-21T11:49:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[36454]: pgmap v445: 148 pgs: 148 active+clean; 96 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 16 MiB/s wr, 254 op/s 2024-09-21T11:49:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:49:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:49:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:49:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[29710]: osdmap e252: 8 total, 8 up, 8 in 2024-09-21T11:49:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[29710]: 4.f scrub starts 2024-09-21T11:49:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[29710]: 4.f scrub ok 2024-09-21T11:49:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[29710]: pgmap v445: 148 pgs: 148 active+clean; 96 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 16 MiB/s wr, 254 op/s 2024-09-21T11:49:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:49:16.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:16 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:16.743 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:16.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:16.744 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:16.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:16.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:16.886 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:16.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:16.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:49:16.986 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:17.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:16 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:49:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:16 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:49:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:16 smithi175 ceph-mon[32613]: osdmap e252: 8 total, 8 up, 8 in 2024-09-21T11:49:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:16 smithi175 ceph-mon[32613]: 4.f scrub starts 2024-09-21T11:49:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:16 smithi175 ceph-mon[32613]: 4.f scrub ok 2024-09-21T11:49:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:16 smithi175 ceph-mon[32613]: pgmap v445: 148 pgs: 148 active+clean; 96 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 16 MiB/s wr, 254 op/s 2024-09-21T11:49:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:16 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:49:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:16 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:17.018 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:17.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:17.019 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:17.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:17.175 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:17.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:17.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:17.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 4 15 12087 12072 188.609 173.062 0.00909755 0.00529307 2024-09-21T11:49:17.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:17.304 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:17.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:17.305 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:17.363 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:17.406 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:17.434 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:17.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:17.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:17.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:17.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:49:17.580 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:17.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:17.581 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:17.682 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:17.708 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:17.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:17.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:17.725 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:49:17.729 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:49:17.799 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:17 smithi043 ceph-mon[29710]: 3.1 scrub starts 2024-09-21T11:49:17.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:17 smithi043 ceph-mon[29710]: 3.1 scrub ok 2024-09-21T11:49:17.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:17 smithi043 ceph-mon[29710]: 5.1d scrub starts 2024-09-21T11:49:17.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:17 smithi043 ceph-mon[29710]: 5.1d scrub ok 2024-09-21T11:49:17.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:17 smithi043 ceph-mon[36454]: 3.1 scrub starts 2024-09-21T11:49:17.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:17 smithi043 ceph-mon[36454]: 3.1 scrub ok 2024-09-21T11:49:17.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:17 smithi043 ceph-mon[36454]: 5.1d scrub starts 2024-09-21T11:49:17.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:17 smithi043 ceph-mon[36454]: 5.1d scrub ok 2024-09-21T11:49:17.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:17.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:17.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:17.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:18.006 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:18.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:18.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:18.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:17 smithi175 ceph-mon[32613]: 3.1 scrub starts 2024-09-21T11:49:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:17 smithi175 ceph-mon[32613]: 3.1 scrub ok 2024-09-21T11:49:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:17 smithi175 ceph-mon[32613]: 5.1d scrub starts 2024-09-21T11:49:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:17 smithi175 ceph-mon[32613]: 5.1d scrub ok 2024-09-21T11:49:18.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:18.150 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:49:18.150 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:49:18.151 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:49:18.151 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:49:18.151 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:49:18.151 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:49:18.151 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:49:18.151 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:49:18.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:18.152 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:18.152 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:18.157 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:49:18.157 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:49:18.195 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:49:18.195 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:49:18.234 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:49:18.235 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:49:18.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:18.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 5 16 14412 14396 179.935 145.25 0.00533183 0.00554964 2024-09-21T11:49:18.271 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:49:18.271 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:49:18.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:18.296 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:18.296 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:18.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:18.433 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:18.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:18.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:18.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:18.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:18.576 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:18.677 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:18.705 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:18.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:18.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:18.806 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:18.838 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:18.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:18.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:18.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:18.972 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:18.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:18.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:18 smithi043 ceph-mon[29710]: from='client.25571 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:49:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:18 smithi043 ceph-mon[29710]: pgmap v446: 148 pgs: 148 active+clean; 590 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.3 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-21T11:49:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:18 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2537364991' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:18 smithi043 ceph-mon[36454]: from='client.25571 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:49:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:18 smithi043 ceph-mon[36454]: pgmap v446: 148 pgs: 148 active+clean; 590 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.3 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-21T11:49:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:18 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2537364991' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:19.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:18 smithi175 ceph-mon[32613]: from='client.25571 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:49:19.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:18 smithi175 ceph-mon[32613]: pgmap v446: 148 pgs: 148 active+clean; 590 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.3 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-21T11:49:19.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:18 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2537364991' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:19.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:19.111 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:19.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:19.112 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:19.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:19.246 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:19.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:19.247 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:19.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 6 16 16981 16965 176.705 160.562 0.00304149 0.00565116 2024-09-21T11:49:19.338 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-21T11:49:15.940556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":100211227,"num_objects":1719,"num_object_clones":0,"num_object_copies":5157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1719,"num_whiteouts":0,"num_read":1383,"num_read_kb":1240,"num_write":2069,"num_write_kb":98953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":596864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2079,"ondisk_log_size":2079,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":442,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":3066524,"kb_used_data":443056,"kb_used_omap":12,"kb_used_meta":2623283,"kb_avail":746894692,"statfs":{"total":767960285184,"available":764820164608,"internally_reserved":0,"allocated":453689344,"data_stored":450735666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2686242354},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":99614720,"num_objects":1520,"num_object_clones":0,"num_object_copies":4560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1520,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":1520,"num_write_kb":97280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001755"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":413,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658224+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:49:10.658224+0000","last_peered":"2024-09-21T11:49:10.658224+0000","last_clean":"2024-09-21T11:49:10.658224+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:49:10.658224+0000","last_undegraded":"2024-09-21T11:49:10.658224+0000","last_fullsized":"2024-09-21T11:49:10.658224+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.863735+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":471,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.557956+0000","last_change":"2024-09-21T11:47:03.433310+0000","last_active":"2024-09-21T11:49:15.557956+0000","last_peered":"2024-09-21T11:49:15.557956+0000","last_clean":"2024-09-21T11:49:15.557956+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:49:15.557956+0000","last_undegraded":"2024-09-21T11:49:15.557956+0000","last_fullsized":"2024-09-21T11:49:15.557956+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:45:44.553561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:44.553561+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:59.048693+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":104,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848397+0000","last_change":"2024-09-21T11:47:03.372676+0000","last_active":"2024-09-21T11:49:09.848397+0000","last_peered":"2024-09-21T11:49:09.848397+0000","last_clean":"2024-09-21T11:49:09.848397+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:49:09.848397+0000","last_undegraded":"2024-09-21T11:49:09.848397+0000","last_fullsized":"2024-09-21T11:49:09.848397+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.453795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":435,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558656+0000","last_change":"2024-09-21T11:49:08.053201+0000","last_active":"2024-09-21T11:49:15.558656+0000","last_peered":"2024-09-21T11:49:15.558656+0000","last_clean":"2024-09-21T11:49:15.558656+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:49:15.558656+0000","last_undegraded":"2024-09-21T11:49:15.558656+0000","last_fullsized":"2024-09-21T11:49:15.558656+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.733987+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":409,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525303+0000","last_change":"2024-09-21T11:49:09.143672+0000","last_active":"2024-09-21T11:49:10.525303+0000","last_peered":"2024-09-21T11:49:10.525303+0000","last_clean":"2024-09-21T11:49:10.525303+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:49:10.525303+0000","last_undegraded":"2024-09-21T11:49:10.525303+0000","last_fullsized":"2024-09-21T11:49:10.525303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:09.143598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:09.143598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:16.417642+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":406,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525033+0000","last_change":"2024-09-21T11:46:55.804484+0000","last_active":"2024-09-21T11:49:10.525033+0000","last_peered":"2024-09-21T11:49:10.525033+0000","last_clean":"2024-09-21T11:49:10.525033+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:49:10.525033+0000","last_undegraded":"2024-09-21T11:49:10.525033+0000","last_fullsized":"2024-09-21T11:49:10.525033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.804419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.804419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":426,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525037+0000","last_change":"2024-09-21T11:45:07.788590+0000","last_active":"2024-09-21T11:49:10.525037+0000","last_peered":"2024-09-21T11:49:10.525037+0000","last_clean":"2024-09-21T11:49:10.525037+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:49:10.525037+0000","last_undegraded":"2024-09-21T11:49:10.525037+0000","last_fullsized":"2024-09-21T11:49:10.525037+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:45:07.788512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:07.788512+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:57.169493+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":431,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.557947+0000","last_change":"2024-09-21T11:47:03.371710+0000","last_active":"2024-09-21T11:49:15.557947+0000","last_peered":"2024-09-21T11:49:15.557947+0000","last_clean":"2024-09-21T11:49:15.557947+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:49:15.557947+0000","last_undegraded":"2024-09-21T11:49:15.557947+0000","last_fullsized":"2024-09-21T11:49:15.557947+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.957478+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":412,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957589+0000","last_change":"2024-09-21T11:48:47.077320+0000","last_active":"2024-09-21T11:49:10.957589+0000","last_peered":"2024-09-21T11:49:10.957589+0000","last_clean":"2024-09-21T11:49:10.957589+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:49:10.957589+0000","last_undegraded":"2024-09-21T11:49:10.957589+0000","last_fullsized":"2024-09-21T11:49:10.957589+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.077259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.077259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:00.730980+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":445,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558806+0000","last_change":"2024-09-21T11:49:15.028297+0000","last_active":"2024-09-21T11:49:15.558806+0000","last_peered":"2024-09-21T11:49:15.558806+0000","last_clean":"2024-09-21T11:49:15.558806+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:49:15.558806+0000","last_undegraded":"2024-09-21T11:49:15.558806+0000","last_fullsized":"2024-09-21T11:49:15.558806+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:49:15.028208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:15.028208+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.470742+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":412,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957069+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:49:10.957069+0000","last_peered":"2024-09-21T11:49:10.957069+0000","last_clean":"2024-09-21T11:49:10.957069+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:49:10.957069+0000","last_undegraded":"2024-09-21T11:49:10.957069+0000","last_fullsized":"2024-09-21T11:49:10.957069+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.033067+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":422,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.657449+0000","last_change":"2024-09-21T11:48:44.523906+0000","last_active":"2024-09-21T11:49:10.657449+0000","last_peered":"2024-09-21T11:49:10.657449+0000","last_clean":"2024-09-21T11:49:10.657449+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:49:10.657449+0000","last_undegraded":"2024-09-21T11:49:10.657449+0000","last_fullsized":"2024-09-21T11:49:10.657449+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:44.523826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.523826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:11.435468+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":434,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957185+0000","last_change":"2024-09-21T11:47:03.385245+0000","last_active":"2024-09-21T11:49:10.957185+0000","last_peered":"2024-09-21T11:49:10.957185+0000","last_clean":"2024-09-21T11:49:10.957185+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:49:10.957185+0000","last_undegraded":"2024-09-21T11:49:10.957185+0000","last_fullsized":"2024-09-21T11:49:10.957185+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.017967+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":465,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558110+0000","last_change":"2024-09-21T11:49:14.010678+0000","last_active":"2024-09-21T11:49:15.558110+0000","last_peered":"2024-09-21T11:49:15.558110+0000","last_clean":"2024-09-21T11:49:15.558110+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:49:15.558110+0000","last_undegraded":"2024-09-21T11:49:15.558110+0000","last_fullsized":"2024-09-21T11:49:15.558110+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.823398+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":422,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565331+0000","last_change":"2024-09-21T11:48:06.605705+0000","last_active":"2024-09-21T11:49:15.565331+0000","last_peered":"2024-09-21T11:49:15.565331+0000","last_clean":"2024-09-21T11:49:15.565331+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:49:15.565331+0000","last_undegraded":"2024-09-21T11:49:15.565331+0000","last_fullsized":"2024-09-21T11:49:15.565331+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:06.605645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.605645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":422,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.956889+0000","last_change":"2024-09-21T11:47:03.380856+0000","last_active":"2024-09-21T11:49:10.956889+0000","last_peered":"2024-09-21T11:49:10.956889+0000","last_clean":"2024-09-21T11:49:10.956889+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:49:10.956889+0000","last_undegraded":"2024-09-21T11:49:10.956889+0000","last_fullsized":"2024-09-21T11:49:10.956889+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:13.288522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:13.288522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.041204+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":422,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.559233+0000","last_change":"2024-09-21T11:46:52.035524+0000","last_active":"2024-09-21T11:49:15.559233+0000","last_peered":"2024-09-21T11:49:15.559233+0000","last_clean":"2024-09-21T11:49:15.559233+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:49:15.559233+0000","last_undegraded":"2024-09-21T11:49:15.559233+0000","last_fullsized":"2024-09-21T11:49:15.559233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.035461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.035461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:30.990978+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":404,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658224+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:49:10.658224+0000","last_peered":"2024-09-21T11:49:10.658224+0000","last_clean":"2024-09-21T11:49:10.658224+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:49:10.658224+0000","last_undegraded":"2024-09-21T11:49:10.658224+0000","last_fullsized":"2024-09-21T11:49:10.658224+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:31.281211+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":519,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525031+0000","last_change":"2024-09-21T11:48:18.120641+0000","last_active":"2024-09-21T11:49:10.525031+0000","last_peered":"2024-09-21T11:49:10.525031+0000","last_clean":"2024-09-21T11:49:10.525031+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:49:10.525031+0000","last_undegraded":"2024-09-21T11:49:10.525031+0000","last_fullsized":"2024-09-21T11:49:10.525031+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:48:18.120532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:18.120532+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:22.674543+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":104,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.849193+0000","last_change":"2024-09-21T11:47:03.377667+0000","last_active":"2024-09-21T11:49:09.849193+0000","last_peered":"2024-09-21T11:49:09.849193+0000","last_clean":"2024-09-21T11:49:09.849193+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:49:09.849193+0000","last_undegraded":"2024-09-21T11:49:09.849193+0000","last_fullsized":"2024-09-21T11:49:09.849193+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.718478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.718478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":402,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525732+0000","last_change":"2024-09-21T11:45:39.826902+0000","last_active":"2024-09-21T11:49:10.525732+0000","last_peered":"2024-09-21T11:49:10.525732+0000","last_clean":"2024-09-21T11:49:10.525732+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:49:10.525732+0000","last_undegraded":"2024-09-21T11:49:10.525732+0000","last_fullsized":"2024-09-21T11:49:10.525732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:39.826701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:39.826701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:05.160837+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":431,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558891+0000","last_change":"2024-09-21T11:48:22.941031+0000","last_active":"2024-09-21T11:49:15.558891+0000","last_peered":"2024-09-21T11:49:15.558891+0000","last_clean":"2024-09-21T11:49:15.558891+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:49:15.558891+0000","last_undegraded":"2024-09-21T11:49:15.558891+0000","last_fullsized":"2024-09-21T11:49:15.558891+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:22.940968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:22.940968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:48.005429+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":470,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524574+0000","last_change":"2024-09-21T11:47:03.873307+0000","last_active":"2024-09-21T11:49:10.524574+0000","last_peered":"2024-09-21T11:49:10.524574+0000","last_clean":"2024-09-21T11:49:10.524574+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:49:10.524574+0000","last_undegraded":"2024-09-21T11:49:10.524574+0000","last_fullsized":"2024-09-21T11:49:10.524574+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:47:03.873239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.873239+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":417,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558612+0000","last_change":"2024-09-21T11:48:51.931885+0000","last_active":"2024-09-21T11:49:15.558612+0000","last_peered":"2024-09-21T11:49:15.558612+0000","last_clean":"2024-09-21T11:49:15.558612+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:49:15.558612+0000","last_undegraded":"2024-09-21T11:49:15.558612+0000","last_fullsized":"2024-09-21T11:49:15.558612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:51.931832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:51.931832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:11.306219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":421,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565447+0000","last_change":"2024-09-21T11:46:47.611118+0000","last_active":"2024-09-21T11:49:15.565447+0000","last_peered":"2024-09-21T11:49:15.565447+0000","last_clean":"2024-09-21T11:49:15.565447+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:49:15.565447+0000","last_undegraded":"2024-09-21T11:49:15.565447+0000","last_fullsized":"2024-09-21T11:49:15.565447+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:46:47.611042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:47.611042+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.533011+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":108,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848303+0000","last_change":"2024-09-21T11:47:03.388975+0000","last_active":"2024-09-21T11:49:09.848303+0000","last_peered":"2024-09-21T11:49:09.848303+0000","last_clean":"2024-09-21T11:49:09.848303+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:49:09.848303+0000","last_undegraded":"2024-09-21T11:49:09.848303+0000","last_fullsized":"2024-09-21T11:49:09.848303+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:12.636105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.636105+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:57.739008+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":432,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658298+0000","last_change":"2024-09-21T11:45:48.385970+0000","last_active":"2024-09-21T11:49:10.658298+0000","last_peered":"2024-09-21T11:49:10.658298+0000","last_clean":"2024-09-21T11:49:10.658298+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:49:10.658298+0000","last_undegraded":"2024-09-21T11:49:10.658298+0000","last_fullsized":"2024-09-21T11:49:10.658298+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:45:48.385895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:48.385895+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:57.547299+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":551,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.792307+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:49:10.792307+0000","last_peered":"2024-09-21T11:49:10.792307+0000","last_clean":"2024-09-21T11:49:10.792307+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:49:10.792307+0000","last_undegraded":"2024-09-21T11:49:10.792307+0000","last_fullsized":"2024-09-21T11:49:10.792307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.120507+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524029+0000","last_change":"2024-09-21T11:49:08.748763+0000","last_active":"2024-09-21T11:49:10.524029+0000","last_peered":"2024-09-21T11:49:10.524029+0000","last_clean":"2024-09-21T11:49:10.524029+0000","last_became_active":"2024-09-21T11:49:08.748359+0000","last_became_peered":"2024-09-21T11:49:08.748359+0000","last_unstale":"2024-09-21T11:49:10.524029+0000","last_undegraded":"2024-09-21T11:49:10.524029+0000","last_fullsized":"2024-09-21T11:49:10.524029+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.761336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":120,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.849445+0000","last_change":"2024-09-21T11:47:03.400465+0000","last_active":"2024-09-21T11:49:09.849445+0000","last_peered":"2024-09-21T11:49:09.849445+0000","last_clean":"2024-09-21T11:49:09.849445+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:49:09.849445+0000","last_undegraded":"2024-09-21T11:49:09.849445+0000","last_fullsized":"2024-09-21T11:49:09.849445+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:22.708839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.708839+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":436,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524272+0000","last_change":"2024-09-21T11:47:03.391541+0000","last_active":"2024-09-21T11:49:10.524272+0000","last_peered":"2024-09-21T11:49:10.524272+0000","last_clean":"2024-09-21T11:49:10.524272+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:49:10.524272+0000","last_undegraded":"2024-09-21T11:49:10.524272+0000","last_fullsized":"2024-09-21T11:49:10.524272+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.836903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.836903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":530,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.559326+0000","last_change":"2024-09-21T11:47:01.933697+0000","last_active":"2024-09-21T11:49:15.559326+0000","last_peered":"2024-09-21T11:49:15.559326+0000","last_clean":"2024-09-21T11:49:15.559326+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:49:15.559326+0000","last_undegraded":"2024-09-21T11:49:15.559326+0000","last_fullsized":"2024-09-21T11:49:15.559326+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:01.933648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:01.933648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:03.078139+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":415,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957343+0000","last_change":"2024-09-21T11:48:15.013107+0000","last_active":"2024-09-21T11:49:10.957343+0000","last_peered":"2024-09-21T11:49:10.957343+0000","last_clean":"2024-09-21T11:49:10.957343+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:49:10.957343+0000","last_undegraded":"2024-09-21T11:49:10.957343+0000","last_fullsized":"2024-09-21T11:49:10.957343+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:15.013058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:15.013058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:15.125286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"252'502","reported_seq":523,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.853904+0000","last_change":"2024-09-21T11:49:08.520005+0000","last_active":"2024-09-21T11:49:15.853904+0000","last_peered":"2024-09-21T11:49:15.853904+0000","last_clean":"2024-09-21T11:49:15.853904+0000","last_became_active":"2024-09-21T11:49:08.519622+0000","last_became_peered":"2024-09-21T11:49:08.519622+0000","last_unstale":"2024-09-21T11:49:15.853904+0000","last_undegraded":"2024-09-21T11:49:15.853904+0000","last_fullsized":"2024-09-21T11:49:15.853904+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:20.297469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":502,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":502,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":450,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.523898+0000","last_change":"2024-09-21T11:46:58.789719+0000","last_active":"2024-09-21T11:49:10.523898+0000","last_peered":"2024-09-21T11:49:10.523898+0000","last_clean":"2024-09-21T11:49:10.523898+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:49:10.523898+0000","last_undegraded":"2024-09-21T11:49:10.523898+0000","last_fullsized":"2024-09-21T11:49:10.523898+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:46:58.789625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.789625+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":422,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525957+0000","last_change":"2024-09-21T11:46:52.891978+0000","last_active":"2024-09-21T11:49:10.525957+0000","last_peered":"2024-09-21T11:49:10.525957+0000","last_clean":"2024-09-21T11:49:10.525957+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:49:10.525957+0000","last_undegraded":"2024-09-21T11:49:10.525957+0000","last_fullsized":"2024-09-21T11:49:10.525957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.891793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.891793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":418,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558037+0000","last_change":"2024-09-21T11:48:47.933145+0000","last_active":"2024-09-21T11:49:15.558037+0000","last_peered":"2024-09-21T11:49:15.558037+0000","last_clean":"2024-09-21T11:49:15.558037+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:49:15.558037+0000","last_undegraded":"2024-09-21T11:49:15.558037+0000","last_fullsized":"2024-09-21T11:49:15.558037+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.933099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.933099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:15.039482+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":406,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558797+0000","last_change":"2024-09-21T11:46:24.717355+0000","last_active":"2024-09-21T11:49:15.558797+0000","last_peered":"2024-09-21T11:49:15.558797+0000","last_clean":"2024-09-21T11:49:15.558797+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:49:15.558797+0000","last_undegraded":"2024-09-21T11:49:15.558797+0000","last_fullsized":"2024-09-21T11:49:15.558797+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.717285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.717285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:02.052582+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525580+0000","last_change":"2024-09-21T11:49:08.609939+0000","last_active":"2024-09-21T11:49:10.525580+0000","last_peered":"2024-09-21T11:49:10.525580+0000","last_clean":"2024-09-21T11:49:10.525580+0000","last_became_active":"2024-09-21T11:49:08.609618+0000","last_became_peered":"2024-09-21T11:49:08.609618+0000","last_unstale":"2024-09-21T11:49:10.525580+0000","last_undegraded":"2024-09-21T11:49:10.525580+0000","last_fullsized":"2024-09-21T11:49:10.525580+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.213949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":459,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558312+0000","last_change":"2024-09-21T11:49:10.053574+0000","last_active":"2024-09-21T11:49:15.558312+0000","last_peered":"2024-09-21T11:49:15.558312+0000","last_clean":"2024-09-21T11:49:15.558312+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:49:15.558312+0000","last_undegraded":"2024-09-21T11:49:15.558312+0000","last_fullsized":"2024-09-21T11:49:15.558312+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:49:10.053515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:10.053515+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:34.201371+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":444,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658424+0000","last_change":"2024-09-21T11:47:03.429013+0000","last_active":"2024-09-21T11:49:10.658424+0000","last_peered":"2024-09-21T11:49:10.658424+0000","last_clean":"2024-09-21T11:49:10.658424+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:49:10.658424+0000","last_undegraded":"2024-09-21T11:49:10.658424+0000","last_fullsized":"2024-09-21T11:49:10.658424+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:46:59.702813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.702813+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":548,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.523711+0000","last_change":"2024-09-21T11:47:03.406087+0000","last_active":"2024-09-21T11:49:10.523711+0000","last_peered":"2024-09-21T11:49:10.523711+0000","last_clean":"2024-09-21T11:49:10.523711+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:49:10.523711+0000","last_undegraded":"2024-09-21T11:49:10.523711+0000","last_fullsized":"2024-09-21T11:49:10.523711+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:08.620519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.620519+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":416,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565982+0000","last_change":"2024-09-21T11:49:12.548705+0000","last_active":"2024-09-21T11:49:15.565982+0000","last_peered":"2024-09-21T11:49:15.565982+0000","last_clean":"2024-09-21T11:49:15.565982+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:49:15.565982+0000","last_undegraded":"2024-09-21T11:49:15.565982+0000","last_fullsized":"2024-09-21T11:49:15.565982+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:12.548650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:12.548650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:22.749399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957553+0000","last_change":"2024-09-21T11:49:08.518112+0000","last_active":"2024-09-21T11:49:10.957553+0000","last_peered":"2024-09-21T11:49:10.957553+0000","last_clean":"2024-09-21T11:49:10.957553+0000","last_became_active":"2024-09-21T11:49:08.517344+0000","last_became_peered":"2024-09-21T11:49:08.517344+0000","last_unstale":"2024-09-21T11:49:10.957553+0000","last_undegraded":"2024-09-21T11:49:10.957553+0000","last_fullsized":"2024-09-21T11:49:10.957553+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:18.854040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":117,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848624+0000","last_change":"2024-09-21T11:47:03.398638+0000","last_active":"2024-09-21T11:49:09.848624+0000","last_peered":"2024-09-21T11:49:09.848624+0000","last_clean":"2024-09-21T11:49:09.848624+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:49:09.848624+0000","last_undegraded":"2024-09-21T11:49:09.848624+0000","last_fullsized":"2024-09-21T11:49:09.848624+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:12.227121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.227121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:00.651403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":437,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957630+0000","last_change":"2024-09-21T11:49:09.924450+0000","last_active":"2024-09-21T11:49:10.957630+0000","last_peered":"2024-09-21T11:49:10.957630+0000","last_clean":"2024-09-21T11:49:10.957630+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:49:10.957630+0000","last_undegraded":"2024-09-21T11:49:10.957630+0000","last_fullsized":"2024-09-21T11:49:10.957630+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:09.924400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:09.924400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:29.238919+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":519,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.657871+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:49:10.657871+0000","last_peered":"2024-09-21T11:49:10.657871+0000","last_clean":"2024-09-21T11:49:10.657871+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:49:10.657871+0000","last_undegraded":"2024-09-21T11:49:10.657871+0000","last_fullsized":"2024-09-21T11:49:10.657871+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.263748+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":409,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.657549+0000","last_change":"2024-09-21T11:49:07.464269+0000","last_active":"2024-09-21T11:49:10.657549+0000","last_peered":"2024-09-21T11:49:10.657549+0000","last_clean":"2024-09-21T11:49:10.657549+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:49:10.657549+0000","last_undegraded":"2024-09-21T11:49:10.657549+0000","last_fullsized":"2024-09-21T11:49:10.657549+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.464204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.464204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:12.076873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"252'506","reported_seq":527,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.865593+0000","last_change":"2024-09-21T11:49:08.609660+0000","last_active":"2024-09-21T11:49:15.865593+0000","last_peered":"2024-09-21T11:49:15.865593+0000","last_clean":"2024-09-21T11:49:15.865593+0000","last_became_active":"2024-09-21T11:49:08.609226+0000","last_became_peered":"2024-09-21T11:49:08.609226+0000","last_unstale":"2024-09-21T11:49:15.865593+0000","last_undegraded":"2024-09-21T11:49:15.865593+0000","last_fullsized":"2024-09-21T11:49:15.865593+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:28.968012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33161216,"num_objects":506,"num_object_clones":0,"num_object_copies":1518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":506,"num_write_kb":32384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":445,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658072+0000","last_change":"2024-09-21T11:47:03.750064+0000","last_active":"2024-09-21T11:49:10.658072+0000","last_peered":"2024-09-21T11:49:10.658072+0000","last_clean":"2024-09-21T11:49:10.658072+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:49:10.658072+0000","last_undegraded":"2024-09-21T11:49:10.658072+0000","last_fullsized":"2024-09-21T11:49:10.658072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:47:03.749962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.749962+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":105,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848829+0000","last_change":"2024-09-21T11:47:03.382675+0000","last_active":"2024-09-21T11:49:09.848829+0000","last_peered":"2024-09-21T11:49:09.848829+0000","last_clean":"2024-09-21T11:49:09.848829+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:49:09.848829+0000","last_undegraded":"2024-09-21T11:49:09.848829+0000","last_fullsized":"2024-09-21T11:49:09.848829+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:08.723593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:08.723593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:55.676239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":405,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.791895+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:49:10.791895+0000","last_peered":"2024-09-21T11:49:10.791895+0000","last_clean":"2024-09-21T11:49:10.791895+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:49:10.791895+0000","last_undegraded":"2024-09-21T11:49:10.791895+0000","last_fullsized":"2024-09-21T11:49:10.791895+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.032409+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":105,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848166+0000","last_change":"2024-09-21T11:47:03.379157+0000","last_active":"2024-09-21T11:49:09.848166+0000","last_peered":"2024-09-21T11:49:09.848166+0000","last_clean":"2024-09-21T11:49:09.848166+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:49:09.848166+0000","last_undegraded":"2024-09-21T11:49:09.848166+0000","last_fullsized":"2024-09-21T11:49:09.848166+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.475150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.657602+0000","last_change":"2024-09-21T11:49:08.516652+0000","last_active":"2024-09-21T11:49:10.657602+0000","last_peered":"2024-09-21T11:49:10.657602+0000","last_clean":"2024-09-21T11:49:10.657602+0000","last_became_active":"2024-09-21T11:49:08.516355+0000","last_became_peered":"2024-09-21T11:49:08.516355+0000","last_unstale":"2024-09-21T11:49:10.657602+0000","last_undegraded":"2024-09-21T11:49:10.657602+0000","last_fullsized":"2024-09-21T11:49:10.657602+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:20.328463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"205'31","reported_seq":153,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.476793+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:49:10.476793+0000","last_peered":"2024-09-21T11:49:10.476793+0000","last_clean":"2024-09-21T11:49:10.476793+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:49:10.476793+0000","last_undegraded":"2024-09-21T11:49:10.476793+0000","last_fullsized":"2024-09-21T11:49:10.476793+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.477411+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":416,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558311+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:49:15.558311+0000","last_peered":"2024-09-21T11:49:15.558311+0000","last_clean":"2024-09-21T11:49:15.558311+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:49:15.558311+0000","last_undegraded":"2024-09-21T11:49:15.558311+0000","last_fullsized":"2024-09-21T11:49:15.558311+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.947561+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":445,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957292+0000","last_change":"2024-09-21T11:47:20.000195+0000","last_active":"2024-09-21T11:49:10.957292+0000","last_peered":"2024-09-21T11:49:10.957292+0000","last_clean":"2024-09-21T11:49:10.957292+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:49:10.957292+0000","last_undegraded":"2024-09-21T11:49:10.957292+0000","last_fullsized":"2024-09-21T11:49:10.957292+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:47:20.000051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.000051+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:05.955756+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":419,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658140+0000","last_change":"2024-09-21T11:47:03.385173+0000","last_active":"2024-09-21T11:49:10.658140+0000","last_peered":"2024-09-21T11:49:10.658140+0000","last_clean":"2024-09-21T11:49:10.658140+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:49:10.658140+0000","last_undegraded":"2024-09-21T11:49:10.658140+0000","last_fullsized":"2024-09-21T11:49:10.658140+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:32.321908+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":394,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524841+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:49:10.524841+0000","last_peered":"2024-09-21T11:49:10.524841+0000","last_clean":"2024-09-21T11:49:10.524841+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:49:10.524841+0000","last_undegraded":"2024-09-21T11:49:10.524841+0000","last_fullsized":"2024-09-21T11:49:10.524841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.188178+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957057+0000","last_change":"2024-09-21T11:49:08.518054+0000","last_active":"2024-09-21T11:49:10.957057+0000","last_peered":"2024-09-21T11:49:10.957057+0000","last_clean":"2024-09-21T11:49:10.957057+0000","last_became_active":"2024-09-21T11:49:08.517758+0000","last_became_peered":"2024-09-21T11:49:08.517758+0000","last_unstale":"2024-09-21T11:49:10.957057+0000","last_undegraded":"2024-09-21T11:49:10.957057+0000","last_fullsized":"2024-09-21T11:49:10.957057+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:18.682352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":52,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.523703+0000","last_change":"2024-09-21T11:48:33.075081+0000","last_active":"2024-09-21T11:49:10.523703+0000","last_peered":"2024-09-21T11:49:10.523703+0000","last_clean":"2024-09-21T11:49:10.523703+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:49:10.523703+0000","last_undegraded":"2024-09-21T11:49:10.523703+0000","last_fullsized":"2024-09-21T11:49:10.523703+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.876172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":109,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848541+0000","last_change":"2024-09-21T11:47:03.391283+0000","last_active":"2024-09-21T11:49:09.848541+0000","last_peered":"2024-09-21T11:49:09.848541+0000","last_clean":"2024-09-21T11:49:09.848541+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:49:09.848541+0000","last_undegraded":"2024-09-21T11:49:09.848541+0000","last_fullsized":"2024-09-21T11:49:09.848541+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.552518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":439,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.523929+0000","last_change":"2024-09-21T11:48:48.994249+0000","last_active":"2024-09-21T11:49:10.523929+0000","last_peered":"2024-09-21T11:49:10.523929+0000","last_clean":"2024-09-21T11:49:10.523929+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:49:10.523929+0000","last_undegraded":"2024-09-21T11:49:10.523929+0000","last_fullsized":"2024-09-21T11:49:10.523929+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:48:48.994172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:48:48.994172+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:11.491020+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":414,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524340+0000","last_change":"2024-09-21T11:48:16.076049+0000","last_active":"2024-09-21T11:49:10.524340+0000","last_peered":"2024-09-21T11:49:10.524340+0000","last_clean":"2024-09-21T11:49:10.524340+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:49:10.524340+0000","last_undegraded":"2024-09-21T11:49:10.524340+0000","last_fullsized":"2024-09-21T11:49:10.524340+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:16.076003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:16.076003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.071160+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":413,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.566067+0000","last_change":"2024-09-21T11:48:07.613756+0000","last_active":"2024-09-21T11:49:15.566067+0000","last_peered":"2024-09-21T11:49:15.566067+0000","last_clean":"2024-09-21T11:49:15.566067+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:49:15.566067+0000","last_undegraded":"2024-09-21T11:49:15.566067+0000","last_fullsized":"2024-09-21T11:49:15.566067+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.613691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.613691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:32.231718+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658057+0000","last_change":"2024-09-21T11:49:08.515501+0000","last_active":"2024-09-21T11:49:10.658057+0000","last_peered":"2024-09-21T11:49:10.658057+0000","last_clean":"2024-09-21T11:49:10.658057+0000","last_became_active":"2024-09-21T11:49:08.515204+0000","last_became_peered":"2024-09-21T11:49:08.515204+0000","last_unstale":"2024-09-21T11:49:10.658057+0000","last_undegraded":"2024-09-21T11:49:10.658057+0000","last_fullsized":"2024-09-21T11:49:10.658057+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:17.221155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":469,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658371+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:49:10.658371+0000","last_peered":"2024-09-21T11:49:10.658371+0000","last_clean":"2024-09-21T11:49:10.658371+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:49:10.658371+0000","last_undegraded":"2024-09-21T11:49:10.658371+0000","last_fullsized":"2024-09-21T11:49:10.658371+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.401660+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":421,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.657590+0000","last_change":"2024-09-21T11:46:29.937866+0000","last_active":"2024-09-21T11:49:10.657590+0000","last_peered":"2024-09-21T11:49:10.657590+0000","last_clean":"2024-09-21T11:49:10.657590+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:49:10.657590+0000","last_undegraded":"2024-09-21T11:49:10.657590+0000","last_fullsized":"2024-09-21T11:49:10.657590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.937799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.937799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1015,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565186+0000","last_change":"2024-09-21T11:48:17.596682+0000","last_active":"2024-09-21T11:49:15.565186+0000","last_peered":"2024-09-21T11:49:15.565186+0000","last_clean":"2024-09-21T11:49:15.565186+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:49:15.565186+0000","last_undegraded":"2024-09-21T11:49:15.565186+0000","last_fullsized":"2024-09-21T11:49:15.565186+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:48:17.596571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.596571+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.625607+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":547,"num_read_kb":542,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":427,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957728+0000","last_change":"2024-09-21T11:48:42.042870+0000","last_active":"2024-09-21T11:49:10.957728+0000","last_peered":"2024-09-21T11:49:10.957728+0000","last_clean":"2024-09-21T11:49:10.957728+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:49:10.957728+0000","last_undegraded":"2024-09-21T11:49:10.957728+0000","last_fullsized":"2024-09-21T11:49:10.957728+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:42.042814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:42.042814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:09.988920+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":105,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848457+0000","last_change":"2024-09-21T11:47:03.374507+0000","last_active":"2024-09-21T11:49:09.848457+0000","last_peered":"2024-09-21T11:49:09.848457+0000","last_clean":"2024-09-21T11:49:09.848457+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:49:09.848457+0000","last_undegraded":"2024-09-21T11:49:09.848457+0000","last_fullsized":"2024-09-21T11:49:09.848457+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:10.289871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:10.289871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.599936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524095+0000","last_change":"2024-09-21T11:49:08.516921+0000","last_active":"2024-09-21T11:49:10.524095+0000","last_peered":"2024-09-21T11:49:10.524095+0000","last_clean":"2024-09-21T11:49:10.524095+0000","last_became_active":"2024-09-21T11:49:08.516627+0000","last_became_peered":"2024-09-21T11:49:08.516627+0000","last_unstale":"2024-09-21T11:49:10.524095+0000","last_undegraded":"2024-09-21T11:49:10.524095+0000","last_fullsized":"2024-09-21T11:49:10.524095+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:08.150202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":542,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.792265+0000","last_change":"2024-09-21T11:48:06.171363+0000","last_active":"2024-09-21T11:49:10.792265+0000","last_peered":"2024-09-21T11:49:10.792265+0000","last_clean":"2024-09-21T11:49:10.792265+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:49:10.792265+0000","last_undegraded":"2024-09-21T11:49:10.792265+0000","last_fullsized":"2024-09-21T11:49:10.792265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:48:06.171279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:06.171279+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:12.520316+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":422,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524367+0000","last_change":"2024-09-21T11:46:57.833589+0000","last_active":"2024-09-21T11:49:10.524367+0000","last_peered":"2024-09-21T11:49:10.524367+0000","last_clean":"2024-09-21T11:49:10.524367+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:49:10.524367+0000","last_undegraded":"2024-09-21T11:49:10.524367+0000","last_fullsized":"2024-09-21T11:49:10.524367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:57.833329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.833329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":479,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565279+0000","last_change":"2024-09-21T11:46:58.645189+0000","last_active":"2024-09-21T11:49:15.565279+0000","last_peered":"2024-09-21T11:49:15.565279+0000","last_clean":"2024-09-21T11:49:15.565279+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:49:15.565279+0000","last_undegraded":"2024-09-21T11:49:15.565279+0000","last_fullsized":"2024-09-21T11:49:15.565279+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:46:58.645122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.645122+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.573944+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":425,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565692+0000","last_change":"2024-09-21T11:49:06.639805+0000","last_active":"2024-09-21T11:49:15.565692+0000","last_peered":"2024-09-21T11:49:15.565692+0000","last_clean":"2024-09-21T11:49:15.565692+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:49:15.565692+0000","last_undegraded":"2024-09-21T11:49:15.565692+0000","last_fullsized":"2024-09-21T11:49:15.565692+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:06.639732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:06.639732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:08.731626+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":409,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957630+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:49:10.957630+0000","last_peered":"2024-09-21T11:49:10.957630+0000","last_clean":"2024-09-21T11:49:10.957630+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:49:10.957630+0000","last_undegraded":"2024-09-21T11:49:10.957630+0000","last_fullsized":"2024-09-21T11:49:10.957630+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:30.020245+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"252'512","reported_seq":533,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.856725+0000","last_change":"2024-09-21T11:49:08.519327+0000","last_active":"2024-09-21T11:49:15.856725+0000","last_peered":"2024-09-21T11:49:15.856725+0000","last_clean":"2024-09-21T11:49:15.856725+0000","last_became_active":"2024-09-21T11:49:08.519011+0000","last_became_peered":"2024-09-21T11:49:08.519011+0000","last_unstale":"2024-09-21T11:49:15.856725+0000","last_undegraded":"2024-09-21T11:49:15.856725+0000","last_fullsized":"2024-09-21T11:49:15.856725+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.190157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":512,"num_object_clones":0,"num_object_copies":1536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":471,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.559400+0000","last_change":"2024-09-21T11:48:05.072998+0000","last_active":"2024-09-21T11:49:15.559400+0000","last_peered":"2024-09-21T11:49:15.559400+0000","last_clean":"2024-09-21T11:49:15.559400+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:49:15.559400+0000","last_undegraded":"2024-09-21T11:49:15.559400+0000","last_fullsized":"2024-09-21T11:49:15.559400+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:05.072910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.072910+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:33.824160+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":442,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558469+0000","last_change":"2024-09-21T11:48:41.922086+0000","last_active":"2024-09-21T11:49:15.558469+0000","last_peered":"2024-09-21T11:49:15.558469+0000","last_clean":"2024-09-21T11:49:15.558469+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:49:15.558469+0000","last_undegraded":"2024-09-21T11:49:15.558469+0000","last_fullsized":"2024-09-21T11:49:15.558469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:41.922012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:41.922012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:45.884616+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":413,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957502+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:49:10.957502+0000","last_peered":"2024-09-21T11:49:10.957502+0000","last_clean":"2024-09-21T11:49:10.957502+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:49:10.957502+0000","last_undegraded":"2024-09-21T11:49:10.957502+0000","last_fullsized":"2024-09-21T11:49:10.957502+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:23.795578+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":409,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525537+0000","last_change":"2024-09-21T11:48:23.285482+0000","last_active":"2024-09-21T11:49:10.525537+0000","last_peered":"2024-09-21T11:49:10.525537+0000","last_clean":"2024-09-21T11:49:10.525537+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:49:10.525537+0000","last_undegraded":"2024-09-21T11:49:10.525537+0000","last_fullsized":"2024-09-21T11:49:10.525537+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.285283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.285283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:52.473277+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.847928+0000","last_change":"2024-09-21T11:49:08.748461+0000","last_active":"2024-09-21T11:49:09.847928+0000","last_peered":"2024-09-21T11:49:09.847928+0000","last_clean":"2024-09-21T11:49:09.847928+0000","last_became_active":"2024-09-21T11:49:08.748155+0000","last_became_peered":"2024-09-21T11:49:08.748155+0000","last_unstale":"2024-09-21T11:49:09.847928+0000","last_undegraded":"2024-09-21T11:49:09.847928+0000","last_fullsized":"2024-09-21T11:49:09.847928+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:18.114940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":533,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.559240+0000","last_change":"2024-09-21T11:49:09.037775+0000","last_active":"2024-09-21T11:49:15.559240+0000","last_peered":"2024-09-21T11:49:15.559240+0000","last_clean":"2024-09-21T11:49:15.559240+0000","last_became_active":"2024-09-21T11:48:33.022321+0000","last_became_peered":"2024-09-21T11:48:33.022321+0000","last_unstale":"2024-09-21T11:49:15.559240+0000","last_undegraded":"2024-09-21T11:49:15.559240+0000","last_fullsized":"2024-09-21T11:49:15.559240+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:49:09.037713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:09.037713+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:23.218918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":443,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.792199+0000","last_change":"2024-09-21T11:47:03.365071+0000","last_active":"2024-09-21T11:49:10.792199+0000","last_peered":"2024-09-21T11:49:10.792199+0000","last_clean":"2024-09-21T11:49:10.792199+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:49:10.792199+0000","last_undegraded":"2024-09-21T11:49:10.792199+0000","last_fullsized":"2024-09-21T11:49:10.792199+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.692429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.692429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:55.151252+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":104,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848319+0000","last_change":"2024-09-21T11:47:03.375108+0000","last_active":"2024-09-21T11:49:09.848319+0000","last_peered":"2024-09-21T11:49:09.848319+0000","last_clean":"2024-09-21T11:49:09.848319+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:49:09.848319+0000","last_undegraded":"2024-09-21T11:49:09.848319+0000","last_fullsized":"2024-09-21T11:49:09.848319+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:50.008635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:50.008635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":418,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.523849+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:49:10.523849+0000","last_peered":"2024-09-21T11:49:10.523849+0000","last_clean":"2024-09-21T11:49:10.523849+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:49:10.523849+0000","last_undegraded":"2024-09-21T11:49:10.523849+0000","last_fullsized":"2024-09-21T11:49:10.523849+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:37.893541+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524721+0000","last_change":"2024-09-21T11:49:08.517565+0000","last_active":"2024-09-21T11:49:10.524721+0000","last_peered":"2024-09-21T11:49:10.524721+0000","last_clean":"2024-09-21T11:49:10.524721+0000","last_became_active":"2024-09-21T11:49:08.517011+0000","last_became_peered":"2024-09-21T11:49:08.517011+0000","last_unstale":"2024-09-21T11:49:10.524721+0000","last_undegraded":"2024-09-21T11:49:10.524721+0000","last_fullsized":"2024-09-21T11:49:10.524721+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.617618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":445,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558699+0000","last_change":"2024-09-21T11:46:11.730053+0000","last_active":"2024-09-21T11:49:15.558699+0000","last_peered":"2024-09-21T11:49:15.558699+0000","last_clean":"2024-09-21T11:49:15.558699+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:49:15.558699+0000","last_undegraded":"2024-09-21T11:49:15.558699+0000","last_fullsized":"2024-09-21T11:49:15.558699+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:11.729984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.729984+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:00.074374+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":406,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525555+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:49:10.525555+0000","last_peered":"2024-09-21T11:49:10.525555+0000","last_clean":"2024-09-21T11:49:10.525555+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:49:10.525555+0000","last_undegraded":"2024-09-21T11:49:10.525555+0000","last_fullsized":"2024-09-21T11:49:10.525555+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:27.332154+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":541,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524981+0000","last_change":"2024-09-21T11:48:43.940489+0000","last_active":"2024-09-21T11:49:10.524981+0000","last_peered":"2024-09-21T11:49:10.524981+0000","last_clean":"2024-09-21T11:49:10.524981+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:49:10.524981+0000","last_undegraded":"2024-09-21T11:49:10.524981+0000","last_fullsized":"2024-09-21T11:49:10.524981+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:48:43.940235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:43.940235+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:58.064586+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":420,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.559132+0000","last_change":"2024-09-21T11:48:07.992586+0000","last_active":"2024-09-21T11:49:15.559132+0000","last_peered":"2024-09-21T11:49:15.559132+0000","last_clean":"2024-09-21T11:49:15.559132+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:49:15.559132+0000","last_undegraded":"2024-09-21T11:49:15.559132+0000","last_fullsized":"2024-09-21T11:49:15.559132+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.992517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.992517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.884349+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957394+0000","last_change":"2024-09-21T11:49:08.748457+0000","last_active":"2024-09-21T11:49:10.957394+0000","last_peered":"2024-09-21T11:49:10.957394+0000","last_clean":"2024-09-21T11:49:10.957394+0000","last_became_active":"2024-09-21T11:49:08.748171+0000","last_became_peered":"2024-09-21T11:49:08.748171+0000","last_unstale":"2024-09-21T11:49:10.957394+0000","last_undegraded":"2024-09-21T11:49:10.957394+0000","last_fullsized":"2024-09-21T11:49:10.957394+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:19.920819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":459,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.657964+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:49:10.657964+0000","last_peered":"2024-09-21T11:49:10.657964+0000","last_clean":"2024-09-21T11:49:10.657964+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:49:10.657964+0000","last_undegraded":"2024-09-21T11:49:10.657964+0000","last_fullsized":"2024-09-21T11:49:10.657964+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:41.378708+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":421,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957294+0000","last_change":"2024-09-21T11:48:08.172717+0000","last_active":"2024-09-21T11:49:10.957294+0000","last_peered":"2024-09-21T11:49:10.957294+0000","last_clean":"2024-09-21T11:49:10.957294+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:49:10.957294+0000","last_undegraded":"2024-09-21T11:49:10.957294+0000","last_fullsized":"2024-09-21T11:49:10.957294+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:08.172656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:08.172656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:18.830907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":414,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524571+0000","last_change":"2024-09-21T11:48:49.993307+0000","last_active":"2024-09-21T11:49:10.524571+0000","last_peered":"2024-09-21T11:49:10.524571+0000","last_clean":"2024-09-21T11:49:10.524571+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:49:10.524571+0000","last_undegraded":"2024-09-21T11:49:10.524571+0000","last_fullsized":"2024-09-21T11:49:10.524571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.993230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.993230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:12.892451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":393,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.657321+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:49:10.657321+0000","last_peered":"2024-09-21T11:49:10.657321+0000","last_clean":"2024-09-21T11:49:10.657321+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:49:10.657321+0000","last_undegraded":"2024-09-21T11:49:10.657321+0000","last_fullsized":"2024-09-21T11:49:10.657321+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957696+0000","last_change":"2024-09-21T11:49:08.517663+0000","last_active":"2024-09-21T11:49:10.957696+0000","last_peered":"2024-09-21T11:49:10.957696+0000","last_clean":"2024-09-21T11:49:10.957696+0000","last_became_active":"2024-09-21T11:49:08.516975+0000","last_became_peered":"2024-09-21T11:49:08.516975+0000","last_unstale":"2024-09-21T11:49:10.957696+0000","last_undegraded":"2024-09-21T11:49:10.957696+0000","last_fullsized":"2024-09-21T11:49:10.957696+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:24.243398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":510,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.792146+0000","last_change":"2024-09-21T11:48:53.117184+0000","last_active":"2024-09-21T11:49:10.792146+0000","last_peered":"2024-09-21T11:49:10.792146+0000","last_clean":"2024-09-21T11:49:10.792146+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:49:10.792146+0000","last_undegraded":"2024-09-21T11:49:10.792146+0000","last_fullsized":"2024-09-21T11:49:10.792146+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:48:53.117090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:53.117090+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:07.030880+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":425,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.566518+0000","last_change":"2024-09-21T11:48:03.546909+0000","last_active":"2024-09-21T11:49:15.566518+0000","last_peered":"2024-09-21T11:49:15.566518+0000","last_clean":"2024-09-21T11:49:15.566518+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:49:15.566518+0000","last_undegraded":"2024-09-21T11:49:15.566518+0000","last_fullsized":"2024-09-21T11:49:15.566518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:03.546843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:03.546843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.594742+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":422,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524685+0000","last_change":"2024-09-21T11:46:16.775013+0000","last_active":"2024-09-21T11:49:10.524685+0000","last_peered":"2024-09-21T11:49:10.524685+0000","last_clean":"2024-09-21T11:49:10.524685+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:49:10.524685+0000","last_undegraded":"2024-09-21T11:49:10.524685+0000","last_fullsized":"2024-09-21T11:49:10.524685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:16.774939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.774939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.931030+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":413,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565255+0000","last_change":"2024-09-21T11:46:59.704981+0000","last_active":"2024-09-21T11:49:15.565255+0000","last_peered":"2024-09-21T11:49:15.565255+0000","last_clean":"2024-09-21T11:49:15.565255+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:49:15.565255+0000","last_undegraded":"2024-09-21T11:49:15.565255+0000","last_fullsized":"2024-09-21T11:49:15.565255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:26.588396+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848642+0000","last_change":"2024-09-21T11:49:08.610149+0000","last_active":"2024-09-21T11:49:09.848642+0000","last_peered":"2024-09-21T11:49:09.848642+0000","last_clean":"2024-09-21T11:49:09.848642+0000","last_became_active":"2024-09-21T11:49:08.609829+0000","last_became_peered":"2024-09-21T11:49:08.609829+0000","last_unstale":"2024-09-21T11:49:09.848642+0000","last_undegraded":"2024-09-21T11:49:09.848642+0000","last_fullsized":"2024-09-21T11:49:09.848642+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.929195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":499,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558166+0000","last_change":"2024-09-21T11:48:04.111199+0000","last_active":"2024-09-21T11:49:15.558166+0000","last_peered":"2024-09-21T11:49:15.558166+0000","last_clean":"2024-09-21T11:49:15.558166+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:49:15.558166+0000","last_undegraded":"2024-09-21T11:49:15.558166+0000","last_fullsized":"2024-09-21T11:49:15.558166+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:48:04.111115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.111115+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:19.507697+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":463,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.566328+0000","last_change":"2024-09-21T11:49:08.611218+0000","last_active":"2024-09-21T11:49:15.566328+0000","last_peered":"2024-09-21T11:49:15.566328+0000","last_clean":"2024-09-21T11:49:15.566328+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:49:15.566328+0000","last_undegraded":"2024-09-21T11:49:15.566328+0000","last_fullsized":"2024-09-21T11:49:15.566328+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.611159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.611159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.323039+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":436,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.523519+0000","last_change":"2024-09-21T11:47:03.385382+0000","last_active":"2024-09-21T11:49:10.523519+0000","last_peered":"2024-09-21T11:49:10.523519+0000","last_clean":"2024-09-21T11:49:10.523519+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:49:10.523519+0000","last_undegraded":"2024-09-21T11:49:10.523519+0000","last_fullsized":"2024-09-21T11:49:10.523519+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.766819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.766819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:38.890518+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":416,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525334+0000","last_change":"2024-09-21T11:47:03.376001+0000","last_active":"2024-09-21T11:49:10.525334+0000","last_peered":"2024-09-21T11:49:10.525334+0000","last_clean":"2024-09-21T11:49:10.525334+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:49:10.525334+0000","last_undegraded":"2024-09-21T11:49:10.525334+0000","last_fullsized":"2024-09-21T11:49:10.525334+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:29.703712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:29.703712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":17,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658077+0000","last_change":"2024-09-21T11:49:08.517117+0000","last_active":"2024-09-21T11:49:10.658077+0000","last_peered":"2024-09-21T11:49:10.658077+0000","last_clean":"2024-09-21T11:49:10.658077+0000","last_became_active":"2024-09-21T11:49:08.516494+0000","last_became_peered":"2024-09-21T11:49:08.516494+0000","last_unstale":"2024-09-21T11:49:10.658077+0000","last_undegraded":"2024-09-21T11:49:10.658077+0000","last_fullsized":"2024-09-21T11:49:10.658077+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.332843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":426,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.956833+0000","last_change":"2024-09-21T11:46:57.248649+0000","last_active":"2024-09-21T11:49:10.956833+0000","last_peered":"2024-09-21T11:49:10.956833+0000","last_clean":"2024-09-21T11:49:10.956833+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:49:10.956833+0000","last_undegraded":"2024-09-21T11:49:10.956833+0000","last_fullsized":"2024-09-21T11:49:10.956833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:46:57.248575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.248575+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.044421+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":445,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565690+0000","last_change":"2024-09-21T11:48:09.683093+0000","last_active":"2024-09-21T11:49:15.565690+0000","last_peered":"2024-09-21T11:49:15.565690+0000","last_clean":"2024-09-21T11:49:15.565690+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:49:15.565690+0000","last_undegraded":"2024-09-21T11:49:15.565690+0000","last_fullsized":"2024-09-21T11:49:15.565690+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:48:09.683008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.683008+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:38.676024+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":537,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565717+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:49:15.565717+0000","last_peered":"2024-09-21T11:49:15.565717+0000","last_clean":"2024-09-21T11:49:15.565717+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:49:15.565717+0000","last_undegraded":"2024-09-21T11:49:15.565717+0000","last_fullsized":"2024-09-21T11:49:15.565717+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:28.600494+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":401,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957003+0000","last_change":"2024-09-21T11:44:13.315306+0000","last_active":"2024-09-21T11:49:10.957003+0000","last_peered":"2024-09-21T11:49:10.957003+0000","last_clean":"2024-09-21T11:49:10.957003+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:49:10.957003+0000","last_undegraded":"2024-09-21T11:49:10.957003+0000","last_fullsized":"2024-09-21T11:49:10.957003+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:13.315228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:13.315228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:56.016562+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":428,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.792005+0000","last_change":"2024-09-21T11:48:17.129730+0000","last_active":"2024-09-21T11:49:10.792005+0000","last_peered":"2024-09-21T11:49:10.792005+0000","last_clean":"2024-09-21T11:49:10.792005+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:49:10.792005+0000","last_undegraded":"2024-09-21T11:49:10.792005+0000","last_fullsized":"2024-09-21T11:49:10.792005+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.129675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.129675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.682622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":448,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525881+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:49:10.525881+0000","last_peered":"2024-09-21T11:49:10.525881+0000","last_clean":"2024-09-21T11:49:10.525881+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:49:10.525881+0000","last_undegraded":"2024-09-21T11:49:10.525881+0000","last_fullsized":"2024-09-21T11:49:10.525881+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":434,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957462+0000","last_change":"2024-09-21T11:49:08.902428+0000","last_active":"2024-09-21T11:49:10.957462+0000","last_peered":"2024-09-21T11:49:10.957462+0000","last_clean":"2024-09-21T11:49:10.957462+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:49:10.957462+0000","last_undegraded":"2024-09-21T11:49:10.957462+0000","last_fullsized":"2024-09-21T11:49:10.957462+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.902382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.902382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.218309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":420,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565966+0000","last_change":"2024-09-21T11:47:28.380378+0000","last_active":"2024-09-21T11:49:15.565966+0000","last_peered":"2024-09-21T11:49:15.565966+0000","last_clean":"2024-09-21T11:49:15.565966+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:49:15.565966+0000","last_undegraded":"2024-09-21T11:49:15.565966+0000","last_fullsized":"2024-09-21T11:49:15.565966+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:47:28.380245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:47:28.380245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.638314+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":441,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658376+0000","last_change":"2024-09-21T11:46:28.973700+0000","last_active":"2024-09-21T11:49:10.658376+0000","last_peered":"2024-09-21T11:49:10.658376+0000","last_clean":"2024-09-21T11:49:10.658376+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:49:10.658376+0000","last_undegraded":"2024-09-21T11:49:10.658376+0000","last_fullsized":"2024-09-21T11:49:10.658376+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:28.973606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.973606+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:06.470329+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":416,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565887+0000","last_change":"2024-09-21T11:44:09.521096+0000","last_active":"2024-09-21T11:49:15.565887+0000","last_peered":"2024-09-21T11:49:15.565887+0000","last_clean":"2024-09-21T11:49:15.565887+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:49:15.565887+0000","last_undegraded":"2024-09-21T11:49:15.565887+0000","last_fullsized":"2024-09-21T11:49:15.565887+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:09.521037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:09.521037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:58.666571+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":427,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525755+0000","last_change":"2024-09-21T11:47:03.379216+0000","last_active":"2024-09-21T11:49:10.525755+0000","last_peered":"2024-09-21T11:49:10.525755+0000","last_clean":"2024-09-21T11:49:10.525755+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:49:10.525755+0000","last_undegraded":"2024-09-21T11:49:10.525755+0000","last_fullsized":"2024-09-21T11:49:10.525755+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:48.903980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:48.903980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":417,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524774+0000","last_change":"2024-09-21T11:48:17.081179+0000","last_active":"2024-09-21T11:49:10.524774+0000","last_peered":"2024-09-21T11:49:10.524774+0000","last_clean":"2024-09-21T11:49:10.524774+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:49:10.524774+0000","last_undegraded":"2024-09-21T11:49:10.524774+0000","last_fullsized":"2024-09-21T11:49:10.524774+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.081131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.081131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:40.264468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":424,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957530+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:49:10.957530+0000","last_peered":"2024-09-21T11:49:10.957530+0000","last_clean":"2024-09-21T11:49:10.957530+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:49:10.957530+0000","last_undegraded":"2024-09-21T11:49:10.957530+0000","last_fullsized":"2024-09-21T11:49:10.957530+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:36.995538+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":420,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.657667+0000","last_change":"2024-09-21T11:46:24.087039+0000","last_active":"2024-09-21T11:49:10.657667+0000","last_peered":"2024-09-21T11:49:10.657667+0000","last_clean":"2024-09-21T11:49:10.657667+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:49:10.657667+0000","last_undegraded":"2024-09-21T11:49:10.657667+0000","last_fullsized":"2024-09-21T11:49:10.657667+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.086918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.086918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":516,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525976+0000","last_change":"2024-09-21T11:47:02.849839+0000","last_active":"2024-09-21T11:49:10.525976+0000","last_peered":"2024-09-21T11:49:10.525976+0000","last_clean":"2024-09-21T11:49:10.525976+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:49:10.525976+0000","last_undegraded":"2024-09-21T11:49:10.525976+0000","last_fullsized":"2024-09-21T11:49:10.525976+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:02.849786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:02.849786+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":421,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.559060+0000","last_change":"2024-09-21T11:48:50.927661+0000","last_active":"2024-09-21T11:49:15.559060+0000","last_peered":"2024-09-21T11:49:15.559060+0000","last_clean":"2024-09-21T11:49:15.559060+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:49:15.559060+0000","last_undegraded":"2024-09-21T11:49:15.559060+0000","last_fullsized":"2024-09-21T11:49:15.559060+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:50.927613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.927613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:15.925828+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":454,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525821+0000","last_change":"2024-09-21T11:47:03.428846+0000","last_active":"2024-09-21T11:49:10.525821+0000","last_peered":"2024-09-21T11:49:10.525821+0000","last_clean":"2024-09-21T11:49:10.525821+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:49:10.525821+0000","last_undegraded":"2024-09-21T11:49:10.525821+0000","last_fullsized":"2024-09-21T11:49:10.525821+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:29.284078+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":416,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.567319+0000","last_change":"2024-09-21T11:44:08.517158+0000","last_active":"2024-09-21T11:49:15.567319+0000","last_peered":"2024-09-21T11:49:15.567319+0000","last_clean":"2024-09-21T11:49:15.567319+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:49:15.567319+0000","last_undegraded":"2024-09-21T11:49:15.567319+0000","last_fullsized":"2024-09-21T11:49:15.567319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:08.517093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:44:08.517093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:56.618953+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":554,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957378+0000","last_change":"2024-09-21T11:47:03.415793+0000","last_active":"2024-09-21T11:49:10.957378+0000","last_peered":"2024-09-21T11:49:10.957378+0000","last_clean":"2024-09-21T11:49:10.957378+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:49:10.957378+0000","last_undegraded":"2024-09-21T11:49:10.957378+0000","last_fullsized":"2024-09-21T11:49:10.957378+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:10.174971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:10.174971+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:25.999554+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":416,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.657998+0000","last_change":"2024-09-21T11:48:51.483725+0000","last_active":"2024-09-21T11:49:10.657998+0000","last_peered":"2024-09-21T11:49:10.657998+0000","last_clean":"2024-09-21T11:49:10.657998+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:49:10.657998+0000","last_undegraded":"2024-09-21T11:49:10.657998+0000","last_fullsized":"2024-09-21T11:49:10.657998+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:51.483662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:51.483662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:01.284415+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":505,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558182+0000","last_change":"2024-09-21T11:48:33.033557+0000","last_active":"2024-09-21T11:49:15.558182+0000","last_peered":"2024-09-21T11:49:15.558182+0000","last_clean":"2024-09-21T11:49:15.558182+0000","last_became_active":"2024-09-21T11:48:33.022647+0000","last_became_peered":"2024-09-21T11:48:33.022647+0000","last_unstale":"2024-09-21T11:49:15.558182+0000","last_undegraded":"2024-09-21T11:49:15.558182+0000","last_fullsized":"2024-09-21T11:49:15.558182+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:54.032795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.032795+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:35.014222+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":417,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.558975+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:49:15.558975+0000","last_peered":"2024-09-21T11:49:15.558975+0000","last_clean":"2024-09-21T11:49:15.558975+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:49:15.558975+0000","last_undegraded":"2024-09-21T11:49:15.558975+0000","last_fullsized":"2024-09-21T11:49:15.558975+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:39.948354+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":104,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848493+0000","last_change":"2024-09-21T11:47:03.379049+0000","last_active":"2024-09-21T11:49:09.848493+0000","last_peered":"2024-09-21T11:49:09.848493+0000","last_clean":"2024-09-21T11:49:09.848493+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:49:09.848493+0000","last_undegraded":"2024-09-21T11:49:09.848493+0000","last_fullsized":"2024-09-21T11:49:09.848493+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:45.269610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:45.269610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:54.655560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":393,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524422+0000","last_change":"2024-09-21T11:45:04.770055+0000","last_active":"2024-09-21T11:49:10.524422+0000","last_peered":"2024-09-21T11:49:10.524422+0000","last_clean":"2024-09-21T11:49:10.524422+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:49:10.524422+0000","last_undegraded":"2024-09-21T11:49:10.524422+0000","last_fullsized":"2024-09-21T11:49:10.524422+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:04.769963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:45:04.769963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:56.185755+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":114,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.848674+0000","last_change":"2024-09-21T11:47:03.404473+0000","last_active":"2024-09-21T11:49:09.848674+0000","last_peered":"2024-09-21T11:49:09.848674+0000","last_clean":"2024-09-21T11:49:09.848674+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:49:09.848674+0000","last_undegraded":"2024-09-21T11:49:09.848674+0000","last_fullsized":"2024-09-21T11:49:09.848674+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:26.649189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:26.649189+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":424,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.566374+0000","last_change":"2024-09-21T11:48:04.553772+0000","last_active":"2024-09-21T11:49:15.566374+0000","last_peered":"2024-09-21T11:49:15.566374+0000","last_clean":"2024-09-21T11:49:15.566374+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:49:15.566374+0000","last_undegraded":"2024-09-21T11:49:15.566374+0000","last_fullsized":"2024-09-21T11:49:15.566374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:04.553731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.553731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:21.248327+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":427,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525397+0000","last_change":"2024-09-21T11:47:03.377948+0000","last_active":"2024-09-21T11:49:10.525397+0000","last_peered":"2024-09-21T11:49:10.525397+0000","last_clean":"2024-09-21T11:49:10.525397+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:49:10.525397+0000","last_undegraded":"2024-09-21T11:49:10.525397+0000","last_fullsized":"2024-09-21T11:49:10.525397+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:53.848323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:53.848323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":439,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.657376+0000","last_change":"2024-09-21T11:48:50.487842+0000","last_active":"2024-09-21T11:49:10.657376+0000","last_peered":"2024-09-21T11:49:10.657376+0000","last_clean":"2024-09-21T11:49:10.657376+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:49:10.657376+0000","last_undegraded":"2024-09-21T11:49:10.657376+0000","last_fullsized":"2024-09-21T11:49:10.657376+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:48:50.487786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.487786+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:13.704639+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":436,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524780+0000","last_change":"2024-09-21T11:49:07.148305+0000","last_active":"2024-09-21T11:49:10.524780+0000","last_peered":"2024-09-21T11:49:10.524780+0000","last_clean":"2024-09-21T11:49:10.524780+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:49:10.524780+0000","last_undegraded":"2024-09-21T11:49:10.524780+0000","last_fullsized":"2024-09-21T11:49:10.524780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:49:07.148241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.148241+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:07.992983+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":426,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.957465+0000","last_change":"2024-09-21T11:48:10.126645+0000","last_active":"2024-09-21T11:49:10.957465+0000","last_peered":"2024-09-21T11:49:10.957465+0000","last_clean":"2024-09-21T11:49:10.957465+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:49:10.957465+0000","last_undegraded":"2024-09-21T11:49:10.957465+0000","last_fullsized":"2024-09-21T11:49:10.957465+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.126591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.126591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":408,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.564908+0000","last_change":"2024-09-21T11:48:44.639778+0000","last_active":"2024-09-21T11:49:15.564908+0000","last_peered":"2024-09-21T11:49:15.564908+0000","last_clean":"2024-09-21T11:49:15.564908+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:49:15.564908+0000","last_undegraded":"2024-09-21T11:49:15.564908+0000","last_fullsized":"2024-09-21T11:49:15.564908+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:44.639638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.639638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:04.448990+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":409,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.525500+0000","last_change":"2024-09-21T11:49:08.171420+0000","last_active":"2024-09-21T11:49:10.525500+0000","last_peered":"2024-09-21T11:49:10.525500+0000","last_clean":"2024-09-21T11:49:10.525500+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:49:10.525500+0000","last_undegraded":"2024-09-21T11:49:10.525500+0000","last_fullsized":"2024-09-21T11:49:10.525500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.171357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.171357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:24.012559+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":462,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.523808+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:49:10.523808+0000","last_peered":"2024-09-21T11:49:10.523808+0000","last_clean":"2024-09-21T11:49:10.523808+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:49:10.523808+0000","last_undegraded":"2024-09-21T11:49:10.523808+0000","last_fullsized":"2024-09-21T11:49:10.523808+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":412,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658247+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:49:10.658247+0000","last_peered":"2024-09-21T11:49:10.658247+0000","last_clean":"2024-09-21T11:49:10.658247+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:49:10.658247+0000","last_undegraded":"2024-09-21T11:49:10.658247+0000","last_fullsized":"2024-09-21T11:49:10.658247+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:24.212674+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":412,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.791869+0000","last_change":"2024-09-21T11:48:09.104826+0000","last_active":"2024-09-21T11:49:10.791869+0000","last_peered":"2024-09-21T11:49:10.791869+0000","last_clean":"2024-09-21T11:49:10.791869+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:49:10.791869+0000","last_undegraded":"2024-09-21T11:49:10.791869+0000","last_fullsized":"2024-09-21T11:49:10.791869+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:09.104767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.104767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:34.005085+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":404,"reported_epoch":252,"state":"active+clean","last_fresh":"2024-09-21T11:49:15.565187+0000","last_change":"2024-09-21T11:44:12.549176+0000","last_active":"2024-09-21T11:49:15.565187+0000","last_peered":"2024-09-21T11:49:15.565187+0000","last_clean":"2024-09-21T11:49:15.565187+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:49:15.565187+0000","last_undegraded":"2024-09-21T11:49:15.565187+0000","last_fullsized":"2024-09-21T11:49:15.565187+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:12.549091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:44:12.549091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:49:59.646538+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":112,"reported_epoch":249,"state":"active+clean","last_fresh":"2024-09-21T11:49:09.849049+0000","last_change":"2024-09-21T11:49:08.636912+0000","last_active":"2024-09-21T11:49:09.849049+0000","last_peered":"2024-09-21T11:49:09.849049+0000","last_clean":"2024-09-21T11:49:09.849049+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:49:09.849049+0000","last_undegraded":"2024-09-21T11:49:09.849049+0000","last_fullsized":"2024-09-21T11:49:09.849049+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.636860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.636860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:22.892875+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":427,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.524193+0000","last_change":"2024-09-21T11:47:03.377189+0000","last_active":"2024-09-21T11:49:10.524193+0000","last_peered":"2024-09-21T11:49:10.524193+0000","last_clean":"2024-09-21T11:49:10.524193+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:49:10.524193+0000","last_undegraded":"2024-09-21T11:49:10.524193+0000","last_fullsized":"2024-09-21T11:49:10.524193+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:11.587469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:11.587469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":490,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.792338+0000","last_change":"2024-09-21T11:48:45.147257+0000","last_active":"2024-09-21T11:49:10.792338+0000","last_peered":"2024-09-21T11:49:10.792338+0000","last_clean":"2024-09-21T11:49:10.792338+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:49:10.792338+0000","last_undegraded":"2024-09-21T11:49:10.792338+0000","last_fullsized":"2024-09-21T11:49:10.792338+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:45.147190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:45.147190+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:06.653209+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":421,"reported_epoch":250,"state":"active+clean","last_fresh":"2024-09-21T11:49:10.658297+0000","last_change":"2024-09-21T11:48:49.523077+0000","last_active":"2024-09-21T11:49:10.658297+0000","last_peered":"2024-09-21T11:49:10.658297+0000","last_clean":"2024-09-21T11:49:10.658297+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:49:10.658297+0000","last_undegraded":"2024-09-21T11:49:10.658297+0000","last_fullsized":"2024-09-21T11:49:10.658297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.522878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.522878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:04.701345+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":99614720,"num_objects":1520,"num_object_clones":0,"num_object_copies":4560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1520,"num_write_kb":97280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405733376,"data_stored":405733376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1520,"ondisk_log_size":1520,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":396,"ondisk_log_size":396,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":547,"num_read_kb":542,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234100,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":588992,"kb_used_data":256560,"kb_used_omap":1,"kb_used_meta":332414,"kb_avail":93156160,"statfs":{"total":95995035648,"available":95391907840,"internally_reserved":0,"allocated":262717440,"data_stored":262346109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":340392394},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463032,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":285152,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":93460000,"statfs":{"total":95995035648,"available":95703040000,"internally_reserved":0,"allocated":1323008,"data_stored":974651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":290650566},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724669,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":306804,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":93438348,"statfs":{"total":95995035648,"available":95680868352,"internally_reserved":0,"allocated":749568,"data_stored":383694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":313391558},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920900,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":343368,"kb_used_data":69240,"kb_used_omap":1,"kb_used_meta":274110,"kb_avail":93401784,"statfs":{"total":95995035648,"available":95643426816,"internally_reserved":0,"allocated":70901760,"data_stored":70522206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280689093},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117129,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":374836,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":374078,"kb_avail":93370316,"statfs":{"total":95995035648,"available":95611203584,"internally_reserved":0,"allocated":749568,"data_stored":390564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":383056327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411471,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":544824,"kb_used_data":112416,"kb_used_omap":1,"kb_used_meta":432382,"kb_avail":93200328,"statfs":{"total":95995035648,"available":95437135872,"internally_reserved":0,"allocated":115113984,"data_stored":114751595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":442759621},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":292940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":93452212,"statfs":{"total":95995035648,"available":95695065088,"internally_reserved":0,"allocated":774144,"data_stored":384538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":299170246},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738524,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":329608,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":328254,"kb_avail":93415544,"statfs":{"total":95995035648,"available":95657517056,"internally_reserved":0,"allocated":1359872,"data_stored":982309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":336132549},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":103350272,"data_stored":103350272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":302383104,"data_stored":302383104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:49:19.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool set .mgr pgp_num 1 2024-09-21T11:49:19.347 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:19.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:19.436 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:19.436 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:19.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:19.564 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:19.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:19.565 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:19.580 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:19 smithi043 ceph-mon[29710]: 4.5 scrub starts 2024-09-21T11:49:19.580 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:19 smithi043 ceph-mon[29710]: 4.5 scrub ok 2024-09-21T11:49:19.580 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:19 smithi043 ceph-mon[36454]: 4.5 scrub starts 2024-09-21T11:49:19.580 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:19 smithi043 ceph-mon[36454]: 4.5 scrub ok 2024-09-21T11:49:19.666 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:19.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:19.711 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:19.812 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:19.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:19.848 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:19.849 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:19.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:19 smithi175 ceph-mon[32613]: 4.5 scrub starts 2024-09-21T11:49:19.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:19 smithi175 ceph-mon[32613]: 4.5 scrub ok 2024-09-21T11:49:19.950 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:19.988 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:19.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:19.989 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:20.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:20.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:20.134 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:20.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:20.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 7 16 19538 19522 174.289 159.812 0.00538264 0.00573138 2024-09-21T11:49:20.269 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:20.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:20.270 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:20.371 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:20.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:20.406 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:20.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:20.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:20.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:20.545 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:20.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:20.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:20 smithi175 ceph-mon[32613]: pgmap v447: 148 pgs: 148 active+clean; 590 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-21T11:49:20.675 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:20.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:20.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:20.777 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:20.809 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:20.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:20.810 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:20.845 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:20 smithi043 ceph-mon[36454]: pgmap v447: 148 pgs: 148 active+clean; 590 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-21T11:49:20.846 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:20 smithi043 ceph-mon[29710]: pgmap v447: 148 pgs: 148 active+clean; 590 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-21T11:49:20.865 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:20.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:20.919 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:49:20.919 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:49:20.929 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:20.963 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:20.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:20.964 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:21.065 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:21.098 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:21.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:21.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:21.200 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:21.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:21.236 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:21.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:21.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 8 16 21709 21693 169.463 135.688 0.00639726 0.00589452 2024-09-21T11:49:21.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:21.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:21.376 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:21.477 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:21.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:21.526 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:21.526 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:21.589 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:49:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:49:21.627 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:21.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:21.663 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:21.664 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:21.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:21.799 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:21.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:21.800 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:21.851 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:21 smithi175 ceph-mon[32613]: 6.b scrub starts 2024-09-21T11:49:21.851 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:21 smithi175 ceph-mon[32613]: 6.b scrub ok 2024-09-21T11:49:21.851 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:21 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:21.901 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:21.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:21 smithi043 ceph-mon[36454]: 6.b scrub starts 2024-09-21T11:49:21.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:21 smithi043 ceph-mon[36454]: 6.b scrub ok 2024-09-21T11:49:21.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:21 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:21.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:21 smithi043 ceph-mon[29710]: 6.b scrub starts 2024-09-21T11:49:21.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:21 smithi043 ceph-mon[29710]: 6.b scrub ok 2024-09-21T11:49:21.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:21 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:21.933 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:21.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:21.934 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:21.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:49:22.035 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:22.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:49:22.066 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:22.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:22.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:22.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:22.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:22.230 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:22.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:22.254 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 9 16 23970 23954 166.333 141.312 0.0125918 0.00600424 2024-09-21T11:49:22.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:22.386 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:22.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:22.387 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:22.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:22.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:22.534 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:22.535 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:22.635 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:22.663 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:22.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:22.664 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:22.765 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:22.792 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:22.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:22.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:22.794 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:22.821 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:22.895 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:22.926 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:22.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:22.927 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:22.936 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:22 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T11:49:22.936 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:22 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T11:49:22.936 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:22 smithi043 ceph-mon[36454]: pgmap v448: 148 pgs: 148 active+clean; 1.3 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 162 MiB/s wr, 2.60k op/s 2024-09-21T11:49:22.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:22 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T11:49:22.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:22 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T11:49:22.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:22 smithi043 ceph-mon[29710]: pgmap v448: 148 pgs: 148 active+clean; 1.3 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 162 MiB/s wr, 2.60k op/s 2024-09-21T11:49:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:22 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T11:49:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:22 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T11:49:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:22 smithi175 ceph-mon[32613]: pgmap v448: 148 pgs: 148 active+clean; 1.3 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 162 MiB/s wr, 2.60k op/s 2024-09-21T11:49:23.027 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:23.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:23.060 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:23.060 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:23.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:23.193 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:23.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:23.194 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:23.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 10 16 25782 25766 161.024 113.25 0.00350301 0.00620445 2024-09-21T11:49:23.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:23.330 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:23.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:23.331 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:23.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:23.465 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:23.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:23.466 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:23.551 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:49:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:49:23.567 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:23.592 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:49:23.592 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:49:23.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:23.598 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:23.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:49:23.618 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 1 pgp_num to 1 2024-09-21T11:49:23.639 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:49:23.639 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:49:23.664 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:49:23.665 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:49:23.695 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:49:23.696 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:49:23.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:23.728 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:49:23.728 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:49:23.740 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:23.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:23.741 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:23.762 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:49:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:49:23.804 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:49:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:49:23.805 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[29710]: 3.12 scrub starts 2024-09-21T11:49:23.805 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[29710]: 3.12 scrub ok 2024-09-21T11:49:23.805 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2857126934' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:49:23.805 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:49:23.805 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1094913847' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[29710]: 5.1a scrub starts 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[29710]: 5.1a scrub ok 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3959367713' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[36454]: 3.12 scrub starts 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[36454]: 3.12 scrub ok 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2857126934' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1094913847' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[36454]: 5.1a scrub starts 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[36454]: 5.1a scrub ok 2024-09-21T11:49:23.806 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3959367713' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:23.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:23.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:23.888 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:23.889 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:23.935 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:49:23.935 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:49:23.935 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:49:23.936 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:49:23.936 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:49:23.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:23 smithi175 ceph-mon[32613]: 3.12 scrub starts 2024-09-21T11:49:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:23 smithi175 ceph-mon[32613]: 3.12 scrub ok 2024-09-21T11:49:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2857126934' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:49:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:23 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:49:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1094913847' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:49:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:23 smithi175 ceph-mon[32613]: 5.1a scrub starts 2024-09-21T11:49:24.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:23 smithi175 ceph-mon[32613]: 5.1a scrub ok 2024-09-21T11:49:24.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3959367713' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:24.029 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:24.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:24.030 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:24.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:24.163 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:24.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:24.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:24.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 11 16 27882 27866 158.316 131.25 0.00328602 0.00631036 2024-09-21T11:49:24.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:24.299 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:24.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:24.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:24.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:24.437 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:24.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:24.438 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:24.539 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:24.569 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:24.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:24.570 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:24.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:24.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:24.711 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:24.711 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:24.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:24.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:24.855 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:24.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:24.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:24.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T11:49:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T11:49:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[29710]: 3.6 scrub starts 2024-09-21T11:49:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[29710]: 3.6 scrub ok 2024-09-21T11:49:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:49:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1094913847' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[29710]: osdmap e253: 8 total, 8 up, 8 in 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[29710]: pgmap v450: 148 pgs: 148 active+clean; 1.3 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 163 MiB/s wr, 2.60k op/s 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[36454]: 3.6 scrub starts 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[36454]: 3.6 scrub ok 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1094913847' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[36454]: osdmap e253: 8 total, 8 up, 8 in 2024-09-21T11:49:24.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:24 smithi043 ceph-mon[36454]: pgmap v450: 148 pgs: 148 active+clean; 1.3 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 163 MiB/s wr, 2.60k op/s 2024-09-21T11:49:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:24 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T11:49:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:24 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T11:49:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:24 smithi175 ceph-mon[32613]: 3.6 scrub starts 2024-09-21T11:49:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:24 smithi175 ceph-mon[32613]: 3.6 scrub ok 2024-09-21T11:49:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:24 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:24 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:49:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:24 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1094913847' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:49:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:24 smithi175 ceph-mon[32613]: osdmap e253: 8 total, 8 up, 8 in 2024-09-21T11:49:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:24 smithi175 ceph-mon[32613]: pgmap v450: 148 pgs: 148 active+clean; 1.3 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 163 MiB/s wr, 2.60k op/s 2024-09-21T11:49:25.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:25.122 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:25.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:25.123 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:25.224 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:25.245 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:25.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:25.246 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:25.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 12 16 30017 30001 156.242 133.438 0.00268849 0.00637635 2024-09-21T11:49:25.347 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:25.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:25.380 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:25.381 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:25.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:49:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:49:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:49:25.519 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:25.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:25.521 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:25.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:25.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:25.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:25.745 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:25.776 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:25.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:25.777 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:25.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:25.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:25.900 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:25.901 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:26.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:26.040 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:26.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:26.041 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:26.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:26.176 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:26.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:26.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:26.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 13 16 31539 31523 151.54 95.125 0.00978311 0.00659278 2024-09-21T11:49:26.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:26.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:26.307 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:26.307 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:26.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:26.445 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:26.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:26.446 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:26.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:26.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:26.570 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:26.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:26.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:26.708 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:26.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:26.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:26.810 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:26.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:26.845 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:26.845 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:26.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:26.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:26.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:27.075 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:27.109 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:27.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:27.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:27.210 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:27.244 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:27.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:27.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:27.253 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 14 16 33312 33296 148.63 110.812 0.00218829 0.00671708 2024-09-21T11:49:27.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:27 smithi175 ceph-mon[32613]: pgmap v451: 148 pgs: 148 active+clean; 1.4 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 139 MiB/s wr, 2.22k op/s 2024-09-21T11:49:27.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:27.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:27.369 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:27.369 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:27.470 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:27.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:27 smithi043 ceph-mon[29710]: pgmap v451: 148 pgs: 148 active+clean; 1.4 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 139 MiB/s wr, 2.22k op/s 2024-09-21T11:49:27.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:27 smithi043 ceph-mon[36454]: pgmap v451: 148 pgs: 148 active+clean; 1.4 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1023 B/s rd, 139 MiB/s wr, 2.22k op/s 2024-09-21T11:49:27.503 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:27.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:27.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:27.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:27.642 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:27.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:27.643 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:27.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:27.780 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:27.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:27.781 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:27.881 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:27.911 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:27.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:27.912 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:28.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:28.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:28.036 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:28.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:28.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:28.177 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:28.177 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:28.254 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 15 15 34815 34800 144.987 94 0.0110377 0.00689094 2024-09-21T11:49:28.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:28.304 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:28.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:28.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:28.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:28.428 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:28.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:28.429 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:28.530 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:28.565 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:28.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:28.566 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:28.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:28.702 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:28.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:28.703 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:28.804 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:28.839 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:28.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:28.840 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:28.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:28.979 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:28.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:28.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:29.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:29.103 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:29.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:29.104 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:29.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:29.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:29.248 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:29.254 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 16 16 36345 36329 141.897 95.5625 0.00444445 0.00704161 2024-09-21T11:49:29.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:29 smithi175 ceph-mon[32613]: pgmap v452: 148 pgs: 148 active+clean; 1.9 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 139 MiB/s wr, 2.22k op/s 2024-09-21T11:49:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:29 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:49:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:29 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:49:29.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:29.372 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:29.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:29.373 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:29.473 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:29.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:29 smithi043 ceph-mon[36454]: pgmap v452: 148 pgs: 148 active+clean; 1.9 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 139 MiB/s wr, 2.22k op/s 2024-09-21T11:49:29.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:29 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:49:29.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:29 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:49:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:29 smithi043 ceph-mon[29710]: pgmap v452: 148 pgs: 148 active+clean; 1.9 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 139 MiB/s wr, 2.22k op/s 2024-09-21T11:49:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:29 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:49:29.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:29 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:49:29.506 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:29.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:29.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:29.608 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:29.641 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:29.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:29.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:29.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:29.783 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:29.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:29.784 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:29.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:29.918 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:29.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:29.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:30.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:30.058 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:30.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:30.059 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:30.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:30.200 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:30.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:30.201 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:30.254 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 17 16 37929 37913 139.374 99 0.00418719 0.00717042 2024-09-21T11:49:30.302 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:30.333 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:30.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:30.334 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:30.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:30.476 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:30.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:30.478 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:30.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:30 smithi043 ceph-mon[29710]: osdmap e254: 8 total, 8 up, 8 in 2024-09-21T11:49:30.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:30 smithi043 ceph-mon[36454]: osdmap e254: 8 total, 8 up, 8 in 2024-09-21T11:49:30.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:30 smithi175 ceph-mon[32613]: osdmap e254: 8 total, 8 up, 8 in 2024-09-21T11:49:30.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:30.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:30.619 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:30.620 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:30.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:30.751 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:30.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:30.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:30.853 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:30.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:30.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:30.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:31.023 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:31.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:31.024 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:31.124 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:31.152 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:31.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:31.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:31.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:49:31.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:49:31.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[36454]: pgmap v454: 148 pgs: 148 active+clean; 1.9 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-21T11:49:31.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[36454]: osdmap e255: 8 total, 8 up, 8 in 2024-09-21T11:49:31.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:49:31.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:49:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:49:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[29710]: pgmap v454: 148 pgs: 148 active+clean; 1.9 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-21T11:49:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[29710]: osdmap e255: 8 total, 8 up, 8 in 2024-09-21T11:49:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:49:31.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:31.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:31.255 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 18 16 39524 39508 137.169 99.6875 0.00239976 0.00726711 2024-09-21T11:49:31.275 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:31.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:31.276 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:31.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:31.416 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:31.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:31.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:31.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:49:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:49:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:31 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:49:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:31 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:49:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:31 smithi175 ceph-mon[32613]: pgmap v454: 148 pgs: 148 active+clean; 1.9 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-21T11:49:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:31 smithi175 ceph-mon[32613]: osdmap e255: 8 total, 8 up, 8 in 2024-09-21T11:49:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:49:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:31.518 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:31.543 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:31.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:31.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:31.645 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:31.679 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:31.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:31.680 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:31.781 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:31.803 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:31.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:31.804 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:31.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:31.940 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:31.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:31.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:32.042 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:32.078 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:32.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:32.081 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:32.182 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:32.213 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:32.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:32.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:32.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:32.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:32.254 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 19 16 41151 41135 135.301 101.688 0.00509168 0.00738515 2024-09-21T11:49:32.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:32.316 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:32.347 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:32.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:32.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:32.449 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:32.479 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:32.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:32.480 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:32.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:32.620 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:32.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:32.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:32.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:32.763 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:32.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:32.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:32.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:32.905 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:32.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:32.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:33.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:33.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:49:33.044 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:33.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:33.047 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:33.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:33.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:33.193 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:33.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:33.254 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:49:33.253653+0000 min lat: 0.00137184 max lat: 0.17025 avg lat: 0.00743125 2024-09-21T11:49:33.254 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:49:33.255 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 20 15 43052 43037 134.478 118.875 0.00849462 0.00743125 2024-09-21T11:49:33.294 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:33.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:33.327 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:33.327 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:33.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:33.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:33.469 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:33.470 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:33.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:33.588 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:33 smithi043 ceph-mon[29710]: pgmap v456: 148 pgs: 148 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 139 MiB/s wr, 2.23k op/s 2024-09-21T11:49:33.589 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:33 smithi043 ceph-mon[36454]: pgmap v456: 148 pgs: 148 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 139 MiB/s wr, 2.23k op/s 2024-09-21T11:49:33.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:33.605 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:33.605 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:33.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:33.746 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:33.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:33.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:33.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:33 smithi175 ceph-mon[32613]: pgmap v456: 148 pgs: 148 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 139 MiB/s wr, 2.23k op/s 2024-09-21T11:49:33.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:33.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:33.889 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:33.889 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:33.989 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:34.023 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:34.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:34.024 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:34.033 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:34.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:34.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:49:34.163 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:34.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:34.164 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:34.254 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 21 16 44838 44822 133.386 111.562 0.00507796 0.00749173 2024-09-21T11:49:34.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:34.301 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:34.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:34.302 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:34.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:34.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:34.441 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:34.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:34.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:34.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:34.584 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:34.584 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:34.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:34.737 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:49:34.737 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:49:34.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:49:34.737 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:49:34.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.737 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:33299 9.6 e253/247 9:6978965f:::benchmark_data_smithi043_126002_object33298:head v 253'2077, mlcod=253'2075)", 2024-09-21T11:49:34.737 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:27.238305+0000", 2024-09-21T11:49:34.737 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.491513469, 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134820261, 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238305+0000", 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238305+0000", 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238305+0000", 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.99e-07 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238362+0000", 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6215000000000003e-05 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238371+0000", 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2380000000000003e-06 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238378+0000", 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6209999999999998e-06 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238426+0000", 2024-09-21T11:49:34.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8285999999999998e-05 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238438+0000", 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2172e-05 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.370011+0000", 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.131573405 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.373125+0000", 2024-09-21T11:49:34.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031136250000000001 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:33296 9.a e253/247 9:519cf60a:::benchmark_data_smithi043_126002_object33295:head v 253'2097, mlcod=253'2093)", 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:27.238405+0000", 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.4914130959999996, 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.135676716, 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238405+0000", 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238405+0000", 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238406+0000", 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2899999999999999e-07 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238456+0000", 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0195000000000003e-05 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238468+0000", 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1822e-05 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238472+0000", 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9349999999999996e-06 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238517+0000", 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5417000000000002e-05 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.238536+0000", 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9267e-05 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.372567+0000", 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13403082799999999 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.374082+0000", 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001514823 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:33307 9.0 9:0e6e4077:::benchmark_data_smithi043_126002_object33306:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:27.239867+0000", 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.4899516799999999, 2024-09-21T11:49:34.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13994072099999999, 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 33307 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.239867+0000", 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.239867+0000", 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.239868+0000", 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.468e-06 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.239890+0000", 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1602000000000001e-05 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.369995+0000", 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13010550900000001 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.370179+0000", 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018426200000000001 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.371126+0000", 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00094660600000000001 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.371248+0000", 2024-09-21T11:49:34.749 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121848 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.371374+0000", 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012662799999999999 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.376234+0000", 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0048593589999999997 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.750 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.751 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.377964+0000", 2024-09-21T11:49:34.751 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017303069999999999 2024-09-21T11:49:34.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.751 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.751 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.379693+0000", 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017291030000000001 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.379763+0000", 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0072000000000003e-05 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.752 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.379807+0000", 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3956999999999999e-05 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:33305 9.a e253/247 9:52aa345c:::benchmark_data_smithi043_126002_object33304:head v 253'2098, mlcod=253'2095)", 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:27.240043+0000", 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 7.4897751660000003, 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13500746, 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.753 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.240043+0000", 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.240043+0000", 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.754 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.240044+0000", 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8800000000000002e-07 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.240118+0000", 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4247999999999998e-05 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.369737+0000", 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.129618767 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.755 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.369748+0000", 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1189000000000001e-05 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.369921+0000", 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017292100000000001 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.369942+0000", 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1608e-05 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.756 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.375019+0000", 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050761410000000002 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.375051+0000", 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1998000000000001e-05 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.757 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34370 9.4 9:29acfc26:::benchmark_data_smithi043_126002_object34369:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:27.905582+0000", 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.8242359869999998, 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.15283993800000001, 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34370 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.758 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905582+0000", 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905582+0000", 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905583+0000", 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8200000000000008e-07 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.759 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905615+0000", 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1678999999999999e-05 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905625+0000", 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0497e-05 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905633+0000", 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2089999999999999e-06 2024-09-21T11:49:34.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905739+0000", 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106629 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905872+0000", 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013283699999999999 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.761 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.906005+0000", 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013278899999999999 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.908168+0000", 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021633070000000002 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.908980+0000", 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00081154399999999998 2024-09-21T11:49:34.762 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.058161+0000", 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14918126200000001 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.058379+0000", 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021786200000000001 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.058422+0000", 2024-09-21T11:49:34.763 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3340999999999997e-05 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34371 9.4 9:2f4a83b6:::benchmark_data_smithi043_126002_object34370:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:27.905653+0000", 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.8241656009999998, 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.15289630900000001, 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.764 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34371 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905653+0000", 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905653+0000", 2024-09-21T11:49:34.765 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905653+0000", 2024-09-21T11:49:34.766 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8599999999999998e-07 2024-09-21T11:49:34.766 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.766 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.766 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905678+0000", 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4338e-05 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905686+0000", 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.456e-06 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.905692+0000", 2024-09-21T11:49:34.767 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1940000000000003e-06 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.907121+0000", 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014285750000000001 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.907212+0000", 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0945000000000004e-05 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.768 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.907297+0000", 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4921999999999998e-05 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.908202+0000", 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00090547299999999995 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.909880+0000", 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001677869 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.769 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.058458+0000", 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14857741399999999 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.058516+0000", 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8619999999999998e-05 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.058549+0000", 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2817000000000001e-05 2024-09-21T11:49:34.770 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34383 9.4 9:2662154a:::benchmark_data_smithi043_126002_object34382:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:27.911480+0000", 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.8183378210000001, 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14719148800000001, 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.771 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.772 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.772 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34383 2024-09-21T11:49:34.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.774 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.774 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.774 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.911480+0000", 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.911480+0000", 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.775 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.911481+0000", 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1900000000000001e-07 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.911498+0000", 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6483000000000001e-05 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.911506+0000", 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8180000000000004e-06 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.776 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.911510+0000", 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7999999999999998e-06 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.911547+0000", 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6366999999999999e-05 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.911619+0000", 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2551000000000004e-05 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:49:34.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.911696+0000", 2024-09-21T11:49:34.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6287999999999999e-05 2024-09-21T11:49:34.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.052326+0000", 2024-09-21T11:49:34.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14063055599999999 2024-09-21T11:49:34.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.052470+0000", 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000143509 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.058580+0000", 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061104339999999997 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.058623+0000", 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2774000000000001e-05 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.058672+0000", 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8989000000000001e-05 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:34383 9.4 e253/247 ondisk, result = 0)", 2024-09-21T11:49:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:27.913234+0000", 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.816583799, 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.139100961, 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.913234+0000", 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.913234+0000", 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.913236+0000", 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4670000000000001e-06 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:27.913244+0000", 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4200000000000007e-06 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.050971+0000", 2024-09-21T11:49:34.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.137726238 2024-09-21T11:49:34.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.783 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:34.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:34.784 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:34.785 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:34.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.050985+0000", 2024-09-21T11:49:34.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4450999999999999e-05 2024-09-21T11:49:34.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.052312+0000", 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013266280000000001 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.052320+0000", 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2490000000000001e-06 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.052335+0000", 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5508000000000001e-05 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.786 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34879 9.4 9:2d14d963:::benchmark_data_smithi043_126002_object34878:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.278865+0000", 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4509527699999998, 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.16660401999999999, 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.787 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34879 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278865+0000", 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278865+0000", 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278866+0000", 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6899999999999996e-07 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278888+0000", 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1725e-05 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278897+0000", 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-21T11:49:34.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278904+0000", 2024-09-21T11:49:34.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-21T11:49:34.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.790 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.790 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.280772+0000", 2024-09-21T11:49:34.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018679180000000001 2024-09-21T11:49:34.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.280895+0000", 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000122831 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281041+0000", 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014673799999999999 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.283617+0000", 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025755309999999998 2024-09-21T11:49:34.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.286918+0000", 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003300786 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.445111+0000", 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.15819361500000001 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.445414+0000", 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030314399999999998 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.445469+0000", 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4987000000000001e-05 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34899 9.0 9:0981b4a4:::benchmark_data_smithi043_126002_object34898:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.289658+0000", 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4401604619999997, 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.144176096, 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34899 2024-09-21T11:49:34.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289658+0000", 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289658+0000", 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289659+0000", 2024-09-21T11:49:34.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1799999999999999e-06 2024-09-21T11:49:34.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289716+0000", 2024-09-21T11:49:34.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7116e-05 2024-09-21T11:49:34.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289725+0000", 2024-09-21T11:49:34.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6580000000000008e-06 2024-09-21T11:49:34.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289730+0000", 2024-09-21T11:49:34.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1089999999999997e-06 2024-09-21T11:49:34.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289769+0000", 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9190000000000003e-05 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289854+0000", 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5035000000000002e-05 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289939+0000", 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4884999999999998e-05 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433679+0000", 2024-09-21T11:49:34.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.143740267 2024-09-21T11:49:34.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433729+0000", 2024-09-21T11:49:34.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9465999999999997e-05 2024-09-21T11:49:34.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433761+0000", 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1878999999999998e-05 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433793+0000", 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1980999999999998e-05 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433834+0000", 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1329999999999999e-05 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34900 9.0 9:0951d3db:::benchmark_data_smithi043_126002_object34899:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.289745+0000", 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4400735620000003, 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14583637499999999, 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.801 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34900 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289745+0000", 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289745+0000", 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289745+0000", 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289759+0000", 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4107000000000001e-05 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289763+0000", 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2169999999999996e-06 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289767+0000", 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6650000000000001e-06 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432234+0000", 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.142466542 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432402+0000", 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001684 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432535+0000", 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000133172 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.435067+0000", 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002531578 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.435101+0000", 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4585000000000002e-05 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.435489+0000", 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00038781200000000002 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.435541+0000", 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2231000000000002e-05 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.435581+0000", 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9677e-05 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34901 9.0 9:052379b2:::benchmark_data_smithi043_126002_object34900:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.289773+0000", 2024-09-21T11:49:34.807 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4400456029999997, 2024-09-21T11:49:34.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14593888399999999, 2024-09-21T11:49:34.808 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34901 2024-09-21T11:49:34.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.808 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289773+0000", 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289773+0000", 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289773+0000", 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4900000000000001e-07 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289786+0000", 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2714000000000001e-05 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289789+0000", 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5659999999999998e-06 2024-09-21T11:49:34.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289792+0000", 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8509999999999999e-06 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433871+0000", 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14407895400000001 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433998+0000", 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012711499999999999 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.434100+0000", 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102285 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.435220+0000", 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011196159999999999 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.435625+0000", 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000404836 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.435661+0000", 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6199999999999999e-05 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.435691+0000", 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9641e-05 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.435712+0000", 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0757000000000001e-05 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:34897 9.d e253/247 9:bfc2b959:::benchmark_data_smithi043_126002_object34896:head v 253'2164, mlcod=253'2160)", 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.289973+0000", 2024-09-21T11:49:34.814 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.439845386, 2024-09-21T11:49:34.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14274896100000001, 2024-09-21T11:49:34.815 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.815 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289973+0000", 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289973+0000", 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.289973+0000", 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4000000000000001e-07 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290054+0000", 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0339000000000004e-05 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290063+0000", 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6840000000000002e-06 2024-09-21T11:49:34.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290068+0000", 2024-09-21T11:49:34.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6069999999999998e-06 2024-09-21T11:49:34.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290119+0000", 2024-09-21T11:49:34.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1116999999999999e-05 2024-09-21T11:49:34.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290135+0000", 2024-09-21T11:49:34.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6008999999999999e-05 2024-09-21T11:49:34.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432684+0000", 2024-09-21T11:49:34.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.142549179 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432722+0000", 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7385999999999997e-05 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34903 9.4 9:2355ddf3:::benchmark_data_smithi043_126002_object34902:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.290287+0000", 2024-09-21T11:49:34.820 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4395307480000001, 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.157487343, 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34903 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290287+0000", 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290287+0000", 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290288+0000", 2024-09-21T11:49:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6400000000000001e-07 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290305+0000", 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7146000000000002e-05 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290313+0000", 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8490000000000006e-06 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290318+0000", 2024-09-21T11:49:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2769999999999999e-06 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.290368+0000", 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0300999999999998e-05 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432338+0000", 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14197010400000001 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432508+0000", 2024-09-21T11:49:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016997400000000001 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433944+0000", 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014360460000000001 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.434442+0000", 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000498347 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.447529+0000", 2024-09-21T11:49:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013086710999999999 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.447727+0000", 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019742699999999999 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.447775+0000", 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8297000000000002e-05 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:34899 9.0 e253/247 ondisk, result = 0)", 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.291619+0000", 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4381989529999997, 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.142117671, 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.291619+0000", 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.291619+0000", 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.291620+0000", 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7400000000000003e-07 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.291624+0000", 2024-09-21T11:49:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6430000000000001e-06 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432136+0000", 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14051217599999999 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432160+0000", 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4122999999999999e-05 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433716+0000", 2024-09-21T11:49:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015558729999999999 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433723+0000", 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9770000000000003e-06 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433737+0000", 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4205e-05 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:34899 9.0 e253/247 ondisk, result = 0)", 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.291822+0000", 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 6.4379957909999996, 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14201997799999999, 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.291822+0000", 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.291822+0000", 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.291823+0000", 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8500000000000001e-07 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.291826+0000", 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3409999999999999e-06 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432701+0000", 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14087432 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.432711+0000", 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0618000000000001e-05 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433753+0000", 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010419299999999999 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433758+0000", 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2520000000000001e-06 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.433842+0000", 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4832000000000004e-05 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:39505 9.4 9:239c099a:::benchmark_data_smithi043_126002_object39504:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e255)", 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:31.203341+0000", 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5264773470000002, 2024-09-21T11:49:34.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14100553699999999, 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 39505 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203341+0000", 2024-09-21T11:49:34.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203341+0000", 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203343+0000", 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8080000000000001e-06 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203365+0000", 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2368e-05 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203377+0000", 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1609e-05 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203384+0000", 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5320000000000001e-06 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203466+0000", 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1929999999999997e-05 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203637+0000", 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000171039 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.204021+0000", 2024-09-21T11:49:34.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00038427200000000001 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.207532+0000", 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035105380000000001 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.207544+0000", 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1747e-05 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.344222+0000", 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13667874399999999 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.344299+0000", 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6050999999999994e-05 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.344346+0000", 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7899000000000003e-05 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.841 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:39506 9.4 9:295351e8:::benchmark_data_smithi043_126002_object39505:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e255)", 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:31.203408+0000", 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5264102579999999, 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14818445299999999, 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 39506 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203408+0000", 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203408+0000", 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203408+0000", 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0800000000000005e-07 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203442+0000", 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3609000000000003e-05 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203449+0000", 2024-09-21T11:49:34.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.323e-06 2024-09-21T11:49:34.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.203457+0000", 2024-09-21T11:49:34.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9929999999999995e-06 2024-09-21T11:49:34.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.205763+0000", 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023057770000000002 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.205877+0000", 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011370300000000001 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.205999+0000", 2024-09-21T11:49:34.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000122197 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.207550+0000", 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00155116 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.208214+0000", 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00066351599999999995 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.351497+0000", 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.143283407 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.351551+0000", 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4082000000000002e-05 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.351592+0000", 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1177999999999998e-05 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.848 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:39521 9.4 9:2b1312e8:::benchmark_data_smithi043_126002_object39520:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e255)", 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:31.209134+0000", 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.520684063, 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14537425700000001, 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.849 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 39521 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.209134+0000", 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.209134+0000", 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.209135+0000", 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.133e-06 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.209153+0000", 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7878999999999999e-05 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.209161+0000", 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0460000000000006e-06 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.209166+0000", 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7319999999999998e-06 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.209203+0000", 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7153999999999997e-05 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.209291+0000", 2024-09-21T11:49:34.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8094000000000001e-05 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.209371+0000", 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9352999999999997e-05 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.341787+0000", 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13241618199999999 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.342732+0000", 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00094478499999999996 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.354403+0000", 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011671222 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.354457+0000", 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4224999999999997e-05 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.354508+0000", 2024-09-21T11:49:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1452000000000001e-05 2024-09-21T11:49:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:39522 9.4 9:2e651e65:::benchmark_data_smithi043_126002_object39521:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e255)", 2024-09-21T11:49:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:31.210534+0000", 2024-09-21T11:49:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.5192840749999998, 2024-09-21T11:49:34.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14409459499999999, 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 39522 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.210534+0000", 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.210534+0000", 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.210535+0000", 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5900000000000005e-07 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.210554+0000", 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8958e-05 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.210561+0000", 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.878e-06 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.210566+0000", 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7539999999999997e-06 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.210615+0000", 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9521000000000002e-05 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.210705+0000", 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0016e-05 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.210814+0000", 2024-09-21T11:49:34.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010846200000000001 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.341915+0000", 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.131100838 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.342740+0000", 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082564699999999999 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.354551+0000", 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01181109 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.354596+0000", 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5167000000000003e-05 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:34.861 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:34.862 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:31.354629+0000", 2024-09-21T11:49:34.862 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2305000000000003e-05 2024-09-21T11:49:34.862 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.862 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.862 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.862 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:34.862 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:34.862 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:49:34.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:34.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:34.927 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:35.028 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:35.059 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:35.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:35.061 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:35.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:35.221 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:35.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:35.222 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:35.254 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 22 16 46388 46372 131.726 96.875 0.0312508 0.00758282 2024-09-21T11:49:35.323 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:35.356 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:35.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:35.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:35.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:49:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:49:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:49:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:35 smithi043 ceph-mon[36454]: pgmap v457: 148 pgs: 148 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 124 MiB/s wr, 1.99k op/s 2024-09-21T11:49:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:35 smithi043 ceph-mon[29710]: pgmap v457: 148 pgs: 148 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 124 MiB/s wr, 1.99k op/s 2024-09-21T11:49:35.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:35.492 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:35.493 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:35.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:49:35.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:35.624 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:35.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:35.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:35.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:35.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:35.752 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:35.752 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:35.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:35 smithi175 ceph-mon[32613]: pgmap v457: 148 pgs: 148 active+clean; 2.4 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 124 MiB/s wr, 1.99k op/s 2024-09-21T11:49:35.853 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:35.884 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:35.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:35.885 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:35.985 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:36.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:36.016 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:36.017 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:36.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:36.158 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:36.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:36.159 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:36.254 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 23 15 48015 48000 130.423 101.75 0.00679751 0.00766193 2024-09-21T11:49:36.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:36.299 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:36.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:36.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:36.313 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:49:36.401 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:36.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:36.434 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:36.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:36.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:49:36.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:36.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:36 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2759486108' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:36.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:36 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2759486108' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:36.592 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:36.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:36.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:36.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:36.775 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:36 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2759486108' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:36.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:36.827 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:36.827 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:36.927 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:36.953 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:36.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:36.954 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:37.054 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:37.087 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:37.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:37.088 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:37.189 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:37.220 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:37.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:37.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:37.255 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 24 16 49460 49444 128.748 90.25 0.00551252 0.0077623 2024-09-21T11:49:37.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:37.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:37.365 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:37.466 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:37.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:37.501 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:37.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:37.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:37.642 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:37.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:37.643 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:37 smithi043 ceph-mon[36454]: pgmap v458: 148 pgs: 148 active+clean; 2.5 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:49:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:37 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:37 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2759486108' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:37 smithi043 ceph-mon[36454]: osdmap e256: 8 total, 8 up, 8 in 2024-09-21T11:49:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:37 smithi043 ceph-mon[29710]: pgmap v458: 148 pgs: 148 active+clean; 2.5 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:49:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:37 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:37 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2759486108' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:37 smithi043 ceph-mon[29710]: osdmap e256: 8 total, 8 up, 8 in 2024-09-21T11:49:37.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:37.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:37 smithi175 ceph-mon[32613]: pgmap v458: 148 pgs: 148 active+clean; 2.5 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:49:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:37 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:37 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2759486108' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:37 smithi175 ceph-mon[32613]: osdmap e256: 8 total, 8 up, 8 in 2024-09-21T11:49:37.776 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:37.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:37.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:37.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:37.906 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:37.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:37.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:38.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:38.037 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:38.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:38.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:38.044 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:38.046 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:38.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:38.182 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:38.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:38.183 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:38.255 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 25 15 51143 51128 127.808 105.25 0.00568063 0.00781955 2024-09-21T11:49:38.284 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:38.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:38.316 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:38.317 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:38.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:38.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:38.442 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:38.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:38.543 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:38.571 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:38.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:38.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:38.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:38.713 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:38.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:38.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:38.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:38.855 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:38.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:38.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:52059 9.2 e256/247 9:48c88353:::benchmark_data_smithi043_126002_object52058:head v 256'3320, mlcod=256'3316)", 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:38.882243+0000", 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.003551308, 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036149709999999998, 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:38.888 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.882243+0000", 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.882243+0000", 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.882244+0000", 2024-09-21T11:49:38.889 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0999999999999997e-07 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.882297+0000", 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3526999999999998e-05 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.882309+0000", 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2179e-05 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.882317+0000", 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1130000000000003e-06 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.882622+0000", 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000305383 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.882670+0000", 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8480999999999999e-05 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.891 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:52065 9.2 e256/247 9:44f33679:::benchmark_data_smithi043_126002_object52064:head v 256'3321, mlcod=256'3316)", 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:38.884697+0000", 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.001097663, 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001302026, 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.884697+0000", 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.884697+0000", 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.884697+0000", 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1500000000000003e-07 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.884723+0000", 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5296e-05 2024-09-21T11:49:38.893 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.884738+0000", 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5469e-05 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.884746+0000", 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4340000000000003e-06 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:38.894 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.885145+0000", 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00039897000000000001 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:38.885191+0000", 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6589999999999999e-05 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 2 2024-09-21T11:49:38.895 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:49:38.953 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:49:38.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 0 1 2024-09-21T11:49:38.958 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:39.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:39.043 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:39.043 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:39.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:39.182 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:49:39.182 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:49:39.203 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:39.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:39.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:39.222 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:49:39.223 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:49:39.256 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 26 16 52584 52568 126.348 90 0.0110181 0.00790779 2024-09-21T11:49:39.266 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:49:39.266 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:49:39.304 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:49:39.304 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:49:39.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:39.336 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:49:39.336 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:49:39.370 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:39.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:39.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:39.376 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:49:39.376 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:49:39.414 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:49:39.414 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:49:39.450 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:49:39.450 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:49:39.472 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:39.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[29710]: pgmap v460: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 128 MiB/s wr, 2.05k op/s 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/611440771' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[36454]: pgmap v460: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 128 MiB/s wr, 2.05k op/s 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:49:39.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/611440771' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:39.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:39.504 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:39.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:39.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:39.648 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:39.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:39.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:39.750 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:39.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:39 smithi175 ceph-mon[32613]: pgmap v460: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 128 MiB/s wr, 2.05k op/s 2024-09-21T11:49:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:39 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:49:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:39 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:49:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:39 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:49:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:39 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:49:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/611440771' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:39.783 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:39.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:39.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:39.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:39.915 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:39.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:39.916 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:39.918 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:49:40.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:40.060 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:40.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:40.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:40.063 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:49:40.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:40.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:40.221 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:40.258 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 27 16 54097 54081 125.161 94.5625 0.00511079 0.0079813 2024-09-21T11:49:40.323 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:40.353 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:40.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:40.354 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:40.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:40.500 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:40.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:40.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:40.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:40.635 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:40.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:40.636 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:40.737 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:40 smithi043 ceph-mon[36454]: osdmap e257: 8 total, 8 up, 8 in 2024-09-21T11:49:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:40 smithi043 ceph-mon[29710]: osdmap e257: 8 total, 8 up, 8 in 2024-09-21T11:49:40.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:40 smithi175 ceph-mon[32613]: osdmap e257: 8 total, 8 up, 8 in 2024-09-21T11:49:40.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:40.812 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:40.813 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:40.827 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:40.865 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:40.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:40.948 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:40.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:40.949 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:41.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:41.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:41.093 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:41.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:41.232 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:41.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:41.233 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:41.258 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 28 16 55603 55587 124.053 94.125 0.00900654 0.00805352 2024-09-21T11:49:41.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:41.369 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:41.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:41.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:41.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:41.495 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:41.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:41.496 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:41.557 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:41 smithi043 ceph-mon[36454]: pgmap v462: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-21T11:49:41.557 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:41 smithi043 ceph-mon[29710]: pgmap v462: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-21T11:49:41.597 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:41.632 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:41.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:41.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:41.687 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:41.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:41.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:41 smithi175 ceph-mon[32613]: pgmap v462: 148 pgs: 148 active+clean; 2.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-21T11:49:41.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:49:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:49:41.775 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:41.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:41.776 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:41.877 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:41.907 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:41.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:41.908 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:42.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:42.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:42.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:42.148 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:42.177 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:42.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:42.178 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:42.263 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 29 16 56993 56977 122.752 86.875 0.00911358 0.00814018 2024-09-21T11:49:42.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:42.327 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:42.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:42.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:42.382 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T11:49:42.429 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:42.456 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:42.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:42.457 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:42.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:42.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:42.587 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:42.588 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:42.594 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:49:42.594 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:49:42.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:42.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/74890988' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:49:42.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:42 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:49:42.611 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:42.611 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/74890988' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:49:42.611 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:42 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:49:42.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:42.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:42.730 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:42.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/74890988' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:49:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:42 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:49:42.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:49:42.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:42.887 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:42.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:42.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:42.989 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:43.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:43.031 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:43.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:43.132 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:43.162 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:43.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:43.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:43.263 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 30 16 58246 58230 121.271 78.3125 0.00813197 0.00824113 2024-09-21T11:49:43.264 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:43.296 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:43.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:43.297 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:43.388 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:49:43.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:43.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 2 1 2024-09-21T11:49:43.448 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:43.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:43.449 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:43.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:43.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:43.598 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:43.599 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:43.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:43.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:43 smithi043 ceph-mon[36454]: pgmap v463: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 MiB/s wr, 2.01k op/s 2024-09-21T11:49:43.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:43 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:49:43.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:43 smithi043 ceph-mon[36454]: osdmap e258: 8 total, 8 up, 8 in 2024-09-21T11:49:43.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:43 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1015153127' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:49:43.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:43 smithi043 ceph-mon[29710]: pgmap v463: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 MiB/s wr, 2.01k op/s 2024-09-21T11:49:43.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:43 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:49:43.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:43 smithi043 ceph-mon[29710]: osdmap e258: 8 total, 8 up, 8 in 2024-09-21T11:49:43.705 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:43 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1015153127' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:49:43.740 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:43.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:43.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:43.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:43 smithi175 ceph-mon[32613]: pgmap v463: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 125 MiB/s wr, 2.01k op/s 2024-09-21T11:49:43.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:43 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:49:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:43 smithi175 ceph-mon[32613]: osdmap e258: 8 total, 8 up, 8 in 2024-09-21T11:49:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:43 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1015153127' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:49:43.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:43.885 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:43.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:43.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:43.987 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:44.014 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:44.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:44.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:44.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:44.150 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:44.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:44.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:44.251 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:44.263 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 31 16 59540 59524 119.968 80.875 0.0122859 0.00832992 2024-09-21T11:49:44.281 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:44.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:44.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:44.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:44.425 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:44.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:44.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:44.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:44.568 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:44.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:44.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:44.670 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:44.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:44 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:44.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1015153127' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:49:44.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:44 smithi043 ceph-mon[29710]: osdmap e259: 8 total, 8 up, 8 in 2024-09-21T11:49:44.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:44 smithi043 ceph-mon[29710]: 6.15 scrub starts 2024-09-21T11:49:44.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:44 smithi043 ceph-mon[29710]: 6.15 scrub ok 2024-09-21T11:49:44.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:44 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:44.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1015153127' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:49:44.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:44 smithi043 ceph-mon[36454]: osdmap e259: 8 total, 8 up, 8 in 2024-09-21T11:49:44.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:44 smithi043 ceph-mon[36454]: 6.15 scrub starts 2024-09-21T11:49:44.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:44 smithi043 ceph-mon[36454]: 6.15 scrub ok 2024-09-21T11:49:44.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:44.701 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:44.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:44 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:49:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1015153127' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:49:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:44 smithi175 ceph-mon[32613]: osdmap e259: 8 total, 8 up, 8 in 2024-09-21T11:49:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:44 smithi175 ceph-mon[32613]: 6.15 scrub starts 2024-09-21T11:49:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:44 smithi175 ceph-mon[32613]: 6.15 scrub ok 2024-09-21T11:49:44.802 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:44.840 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:44.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:44.841 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:44.942 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:44.974 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:44.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:44.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:45.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:45.117 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:45.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:45.119 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:45.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:49:45.193 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:45.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:45.264 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 32 16 61074 61058 119.211 95.875 0.0124548 0.00838366 2024-09-21T11:49:45.276 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:45.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:45.277 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:45.378 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:45.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:49:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:49:45.411 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:45.462 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:45.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:45.463 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:45.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:45.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:45.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:45.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:45.662 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[29710]: pgmap v466: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-21T11:49:45.662 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:49:45.662 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:49:45.663 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[29710]: osdmap e260: 8 total, 8 up, 8 in 2024-09-21T11:49:45.663 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[29710]: 5.8 scrub starts 2024-09-21T11:49:45.663 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[29710]: 5.8 scrub ok 2024-09-21T11:49:45.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[36454]: pgmap v466: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-21T11:49:45.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:49:45.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:49:45.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[36454]: osdmap e260: 8 total, 8 up, 8 in 2024-09-21T11:49:45.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[36454]: 5.8 scrub starts 2024-09-21T11:49:45.663 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:45 smithi043 ceph-mon[36454]: 5.8 scrub ok 2024-09-21T11:49:45.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:45.735 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:45.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:45.736 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:45 smithi175 ceph-mon[32613]: pgmap v466: 148 pgs: 148 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-21T11:49:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:45 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:49:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:45 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:49:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:45 smithi175 ceph-mon[32613]: osdmap e260: 8 total, 8 up, 8 in 2024-09-21T11:49:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:45 smithi175 ceph-mon[32613]: 5.8 scrub starts 2024-09-21T11:49:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:45 smithi175 ceph-mon[32613]: 5.8 scrub ok 2024-09-21T11:49:45.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:45.861 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:45.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:45.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:45.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:45.995 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:45.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:45.996 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:46.097 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:46.127 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:46.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:46.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:49:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:49:46.193 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:49:46.194 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:49:46.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:49:46.228 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:49:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:49:46.257 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:46.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:46.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:46.264 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 33 16 62339 62323 117.994 79.0625 0.014781 0.00846867 2024-09-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:49:46.273 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:49:46.320 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:49:46.321 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:49:46.358 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:49:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:49:46.387 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:46.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:46.388 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:49:46.412 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:49:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:46.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:46.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:46 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/864134991' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:46.455 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:46.455 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:46 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/864134991' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:46.455 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:49:46.456 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:49:46.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:46.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:46.546 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:46.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:46.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:46.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:46.679 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:46.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:46.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:49:46.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:46 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/864134991' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:46.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:46.827 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:46.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:46.828 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:46.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:46.962 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:46.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:46.963 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:47.064 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:47.090 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:47.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:47.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:47.192 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:47.233 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:47.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:47.234 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:47.264 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 34 16 63628 63612 116.893 80.5625 0.00967747 0.00854921 2024-09-21T11:49:47.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:47.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:47.374 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:47.374 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:47.454 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T11:49:47.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:47.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:47.524 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:47.524 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:47.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:47.666 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:47.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:47.667 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[36454]: 4.8 scrub starts 2024-09-21T11:49:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[36454]: 4.8 scrub ok 2024-09-21T11:49:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[36454]: pgmap v468: 148 pgs: 148 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-09-21T11:49:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3094287861' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:49:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:49:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[36454]: 5.6 scrub starts 2024-09-21T11:49:47.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[36454]: 5.6 scrub ok 2024-09-21T11:49:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[29710]: 4.8 scrub starts 2024-09-21T11:49:47.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[29710]: 4.8 scrub ok 2024-09-21T11:49:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[29710]: pgmap v468: 148 pgs: 148 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-09-21T11:49:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3094287861' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:49:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:49:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[29710]: 5.6 scrub starts 2024-09-21T11:49:47.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:47 smithi043 ceph-mon[29710]: 5.6 scrub ok 2024-09-21T11:49:47.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:47 smithi175 ceph-mon[32613]: 4.8 scrub starts 2024-09-21T11:49:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:47 smithi175 ceph-mon[32613]: 4.8 scrub ok 2024-09-21T11:49:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:47 smithi175 ceph-mon[32613]: pgmap v468: 148 pgs: 148 active+clean; 3.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 92 MiB/s wr, 1.48k op/s 2024-09-21T11:49:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3094287861' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:49:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:47 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:49:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:47 smithi175 ceph-mon[32613]: 5.6 scrub starts 2024-09-21T11:49:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:47 smithi175 ceph-mon[32613]: 5.6 scrub ok 2024-09-21T11:49:47.767 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:47.801 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:47.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:47.802 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:47.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:47.934 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:47.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:47.935 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:48.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:48.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:48.077 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:48.177 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:48.215 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:48.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:48.215 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:48.264 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 35 16 65097 65081 116.176 91.8125 0.00845876 0.00860277 2024-09-21T11:49:48.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:48.358 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:48.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:48.359 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:48.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:48.482 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:48.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:48.483 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:48.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:48.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[36454]: 3.15 scrub starts 2024-09-21T11:49:48.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[36454]: 3.15 scrub ok 2024-09-21T11:49:48.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[36454]: 3.8 scrub starts 2024-09-21T11:49:48.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[36454]: 3.8 scrub ok 2024-09-21T11:49:48.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:49:48.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[36454]: osdmap e261: 8 total, 8 up, 8 in 2024-09-21T11:49:48.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[36454]: 6.14 scrub starts 2024-09-21T11:49:48.600 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[36454]: 6.14 scrub ok 2024-09-21T11:49:48.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[29710]: 3.15 scrub starts 2024-09-21T11:49:48.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[29710]: 3.15 scrub ok 2024-09-21T11:49:48.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[29710]: 3.8 scrub starts 2024-09-21T11:49:48.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[29710]: 3.8 scrub ok 2024-09-21T11:49:48.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:49:48.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[29710]: osdmap e261: 8 total, 8 up, 8 in 2024-09-21T11:49:48.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[29710]: 6.14 scrub starts 2024-09-21T11:49:48.601 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:48 smithi043 ceph-mon[29710]: 6.14 scrub ok 2024-09-21T11:49:48.614 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:48.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:48.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:48.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:48.752 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:48.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:48.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:48.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:48 smithi175 ceph-mon[32613]: 3.15 scrub starts 2024-09-21T11:49:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:48 smithi175 ceph-mon[32613]: 3.15 scrub ok 2024-09-21T11:49:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:48 smithi175 ceph-mon[32613]: 3.8 scrub starts 2024-09-21T11:49:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:48 smithi175 ceph-mon[32613]: 3.8 scrub ok 2024-09-21T11:49:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:48 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:49:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:48 smithi175 ceph-mon[32613]: osdmap e261: 8 total, 8 up, 8 in 2024-09-21T11:49:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:48 smithi175 ceph-mon[32613]: 6.14 scrub starts 2024-09-21T11:49:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:48 smithi175 ceph-mon[32613]: 6.14 scrub ok 2024-09-21T11:49:48.776 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:99015 7.e 7:74e80bab:::benchmark_data_smithi043_70646_object99014:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.320644+0000", 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 442.45225856500002, 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12664123799999999, 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.777 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 99015 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320644+0000", 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320644+0000", 2024-09-21T11:49:48.778 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320646+0000", 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8899999999999999e-06 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320673+0000", 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7270000000000001e-05 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.779 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320685+0000", 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2326e-05 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320693+0000", 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6559999999999992e-06 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320812+0000", 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000118983 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.780 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.320928+0000", 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115841 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.321051+0000", 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123437 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.781 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.324857+0000", 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038061000000000002 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334002+0000", 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0091444979999999992 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447169+0000", 2024-09-21T11:49:48.782 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.113167226 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447221+0000", 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2080000000000003e-05 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447285+0000", 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3930999999999995e-05 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:99021 7.e 7:7a4ac083:::benchmark_data_smithi043_70646_object99020:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.325207+0000", 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 442.44769495499997, 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13657670399999999, 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 99021 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.784 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325207+0000", 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325207+0000", 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325208+0000", 2024-09-21T11:49:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8800000000000002e-07 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325229+0000", 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1180999999999999e-05 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325239+0000", 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8400000000000007e-06 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.786 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.325247+0000", 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3790000000000006e-06 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330846+0000", 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055982740000000003 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.330952+0000", 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106563 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:49:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.331082+0000", 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013003799999999999 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334345+0000", 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032629339999999999 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447059+0000", 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11271336 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461671+0000", 2024-09-21T11:49:48.788 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014612823 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461721+0000", 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9688000000000003e-05 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461784+0000", 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2836000000000006e-05 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.14805.0:99029 7.e 7:77218e16:::benchmark_data_smithi043_70646_object99028:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:49:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:42:26.334806+0000", 2024-09-21T11:49:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 442.438095739, 2024-09-21T11:49:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12710760600000001, 2024-09-21T11:49:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.790 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.14805", 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 99029 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334806+0000", 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334806+0000", 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334808+0000", 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.64e-06 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334835+0000", 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7240999999999999e-05 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334848+0000", 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2415000000000001e-05 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.334856+0000", 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0209999999999999e-06 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335270+0000", 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000413905 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335377+0000", 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107583 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.335502+0000", 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000124595 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447122+0000", 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.111620521 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.447133+0000", 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0701e-05 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461829+0000", 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014696074999999999 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461873+0000", 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3374999999999997e-05 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:42:26.461914+0000", 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1533999999999999e-05 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:43897 8.7 e172/160 8:e1365265:::benchmark_data_smithi043_97042_object43896:head v 172'2747, mlcod=172'2745)", 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.983714+0000", 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 267.789187889, 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13119756599999999, 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983714+0000", 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983714+0000", 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983715+0000", 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9399999999999995e-07 2024-09-21T11:49:48.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983751+0000", 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6603000000000002e-05 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983760+0000", 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9570000000000008e-06 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983793+0000", 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2298000000000001e-05 2024-09-21T11:49:48.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983841+0000", 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8581999999999997e-05 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.983859+0000", 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7870999999999999e-05 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.985313+0000", 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014542960000000001 2024-09-21T11:49:48.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114912+0000", 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.129598465 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:43896 8.3 e172/160 8:c3bd4fd6:::benchmark_data_smithi043_97042_object43895:head v 172'2779, mlcod=172'2776)", 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.986209+0000", 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 267.78669285199999, 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.141335514, 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.800 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.986209+0000", 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.986209+0000", 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.986210+0000", 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1900000000000002e-07 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.986276+0000", 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6451000000000005e-05 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115147+0000", 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.128870817 2024-09-21T11:49:48.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115157+0000", 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5210000000000008e-06 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115327+0000", 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017030699999999999 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115348+0000", 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1239000000000001e-05 2024-09-21T11:49:48.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.127490+0000", 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012141457 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.127545+0000", 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5102999999999997e-05 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47776 8.a 8:5cdb45ac:::benchmark_data_smithi043_97042_object47775:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:49:48.804 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.707198+0000", 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 265.06570398299999, 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.138168134, 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47776 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707198+0000", 2024-09-21T11:49:48.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707198+0000", 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707199+0000", 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4600000000000004e-07 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707221+0000", 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1926e-05 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707229+0000", 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3790000000000006e-06 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707235+0000", 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6579999999999999e-06 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.711247+0000", 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040119769999999999 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.711340+0000", 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3184999999999996e-05 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.711418+0000", 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8454999999999993e-05 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712678+0000", 2024-09-21T11:49:48.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001259879 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.712907+0000", 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022910199999999999 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845250+0000", 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.132342829 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845313+0000", 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2891999999999993e-05 2024-09-21T11:49:48.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845366+0000", 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3205999999999998e-05 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:47786 8.d e173/160 8:b7d985cc:::benchmark_data_smithi043_97042_object47785:head v 173'2954, mlcod=173'2951)", 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.713191+0000", 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 265.05971111600002, 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12957838899999999, 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.810 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713191+0000", 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713191+0000", 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713192+0000", 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7700000000000004e-07 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713252+0000", 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0386000000000001e-05 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713268+0000", 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5628e-05 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713272+0000", 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3120000000000004e-06 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713306+0000", 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4202999999999998e-05 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713318+0000", 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1362999999999999e-05 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.714394+0000", 2024-09-21T11:49:48.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010759980000000001 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842769+0000", 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.128375922 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47776 8.a e173/160 ondisk, result = 0)", 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.715233+0000", 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 265.05766938900001, 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.130140586, 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715233+0000", 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715233+0000", 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715233+0000", 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5000000000000002e-07 2024-09-21T11:49:48.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.715240+0000", 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4869999999999999e-06 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845097+0000", 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.129857427 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845108+0000", 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0572e-05 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845235+0000", 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000126809 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845244+0000", 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0399999999999998e-06 2024-09-21T11:49:48.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845373+0000", 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012960099999999999 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:66131 8.a 8:57d7f4ec:::benchmark_data_smithi043_97042_object66130:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:35.920447+0000", 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 252.852454983, 2024-09-21T11:49:48.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.136759779, 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 66131 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.920447+0000", 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.920447+0000", 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.920449+0000", 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.299e-06 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.920472+0000", 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.34e-05 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.920482+0000", 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8330000000000006e-06 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.920489+0000", 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8310000000000003e-06 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.920711+0000", 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022221400000000001 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.920841+0000", 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012975899999999999 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.920982+0000", 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000141221 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.924087+0000", 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031048769999999998 2024-09-21T11:49:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926795+0000", 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027087090000000001 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.057110+0000", 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.130314501 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.057164+0000", 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3885999999999999e-05 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.057207+0000", 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3248999999999999e-05 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:201654 8.3 8:ce8cefd5:::benchmark_data_smithi043_97042_object64560:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:30.450139+0000", 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 138.32276363299999, 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.122410989, 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.824 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 201654 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.450139+0000", 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.450139+0000", 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.450140+0000", 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.545e-06 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.450146+0000", 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5439999999999998e-06 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.450157+0000", 2024-09-21T11:49:48.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1419e-05 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.450164+0000", 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0999999999999998e-06 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.560121+0000", 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.109957283 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.560264+0000", 2024-09-21T11:49:48.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014243 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,5,6", 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.560408+0000", 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014392300000000001 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.564842+0000", 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044338069999999997 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.567177+0000", 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00233527 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.567214+0000", 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6702999999999997e-05 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.572366+0000", 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051523289999999998 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.572435+0000", 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9218e-05 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:30.572550+0000", 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114418 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:213527 8.b e217/203 8:dd265d4d:::benchmark_data_smithi043_97042_object76433:head v 217'13148, mlcod=217'13145)", 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:43.584335+0000", 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.18856760600001, 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.136107323, 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584335+0000", 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584335+0000", 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584336+0000", 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.051e-06 2024-09-21T11:49:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584341+0000", 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5459999999999999e-06 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584349+0000", 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9300000000000003e-06 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584355+0000", 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5990000000000001e-06 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.587567+0000", 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00321273 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.587588+0000", 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0400000000000001e-05 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.720412+0000", 2024-09-21T11:49:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.132824519 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.720442+0000", 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9547999999999999e-05 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:213534 8.3 8:c9d96f39:::benchmark_data_smithi043_97042_object76440:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-21T11:49:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:43.584904+0000", 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.187998067, 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.144719604, 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 213534 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584904+0000", 2024-09-21T11:49:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584904+0000", 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584906+0000", 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3999999999999999e-06 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584911+0000", 2024-09-21T11:49:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.694e-06 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584922+0000", 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0472000000000001e-05 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.584930+0000", 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8469999999999996e-06 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.587262+0000", 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002332616 2024-09-21T11:49:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.587375+0000", 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011295599999999999 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,5,6", 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.587493+0000", 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011738799999999999 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717181+0000", 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.129688425 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717222+0000", 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1541000000000001e-05 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717273+0000", 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0964000000000002e-05 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.729470+0000", 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012197019 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.729516+0000", 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5994e-05 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.729624+0000", 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010728799999999999 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:213536 8.3 8:c4045e94:::benchmark_data_smithi043_97042_object76442:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:43.586790+0000", 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.186112128, 2024-09-21T11:49:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14306465400000001, 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 213536 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586790+0000", 2024-09-21T11:49:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586790+0000", 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586792+0000", 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.525e-06 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586797+0000", 2024-09-21T11:49:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6080000000000003e-06 2024-09-21T11:49:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586810+0000", 2024-09-21T11:49:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2568000000000001e-05 2024-09-21T11:49:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586818+0000", 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7829999999999999e-06 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592399+0000", 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055810390000000003 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592545+0000", 2024-09-21T11:49:48.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014598899999999999 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,5,6", 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592675+0000", 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013015600000000001 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717314+0000", 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12463894 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717353+0000", 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8976000000000001e-05 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.719860+0000", 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025070370000000002 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.729633+0000", 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0097733230000000004 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.729729+0000", 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6408000000000003e-05 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.729855+0000", 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012530199999999999 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:213530 8.b e217/203 8:d1e11e05:::benchmark_data_smithi043_97042_object76436:head v 217'13149, mlcod=217'13145)", 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:43.586896+0000", 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.18600599, 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133570722, 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586896+0000", 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586896+0000", 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586896+0000", 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6199999999999999e-07 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586900+0000", 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1109999999999999e-06 2024-09-21T11:49:48.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586904+0000", 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6e-06 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.586908+0000", 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9389999999999999e-06 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.590171+0000", 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032631930000000002 2024-09-21T11:49:48.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.590186+0000", 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5072000000000001e-05 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.720446+0000", 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.130259809 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.720467+0000", 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0735999999999999e-05 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.851 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.15150.0:213541 8.d e217/160 8:b8825fa7:::benchmark_data_smithi043_97042_object76447:head v 217'13313, mlcod=217'13311)", 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:43.591964+0000", 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.18093815899999, 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.123221679, 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.591964+0000", 2024-09-21T11:49:48.852 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.591964+0000", 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.591965+0000", 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5499999999999998e-07 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.591970+0000", 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1560000000000003e-06 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.591977+0000", 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6639999999999998e-06 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.591983+0000", 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2660000000000001e-06 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592033+0000", 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0853000000000003e-05 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592054+0000", 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0074000000000001e-05 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.715150+0000", 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12309630000000001 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.715186+0000", 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5911000000000002e-05 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:213534 8.3 e217/203 ondisk, result = 0)", 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:43.592367+0000", 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.180535256, 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124821716, 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592367+0000", 2024-09-21T11:49:48.856 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592367+0000", 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592368+0000", 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3900000000000005e-07 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592371+0000", 2024-09-21T11:49:48.857 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8319999999999999e-06 2024-09-21T11:49:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592381+0000", 2024-09-21T11:49:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6460000000000005e-06 2024-09-21T11:49:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.858 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592387+0000", 2024-09-21T11:49:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1369999999999999e-06 2024-09-21T11:49:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717165+0000", 2024-09-21T11:49:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124778103 2024-09-21T11:49:48.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717174+0000", 2024-09-21T11:49:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8599999999999999e-06 2024-09-21T11:49:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717189+0000", 2024-09-21T11:49:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4599e-05 2024-09-21T11:49:48.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:213543 8.3 8:c2799e16:::benchmark_data_smithi043_97042_object76449:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-21T11:49:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:43.592405+0000", 2024-09-21T11:49:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.180496926, 2024-09-21T11:49:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14229615300000001, 2024-09-21T11:49:48.861 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 213543 2024-09-21T11:49:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.862 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592405+0000", 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592405+0000", 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592407+0000", 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.221e-06 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592411+0000", 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0080000000000004e-06 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592419+0000", 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9190000000000005e-06 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592426+0000", 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2430000000000002e-06 2024-09-21T11:49:48.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.720053+0000", 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.127627723 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.720190+0000", 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013679299999999999 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,5,6", 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.720334+0000", 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000143306 2024-09-21T11:49:48.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.724455+0000", 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041211939999999999 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.724516+0000", 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0829000000000003e-05 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.730108+0000", 2024-09-21T11:49:48.866 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0055928999999999996 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.734532+0000", 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044234629999999999 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.734591+0000", 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8909999999999997e-05 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.734701+0000", 2024-09-21T11:49:48.867 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011064400000000001 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:213534 8.3 e217/203 ondisk, result = 0)", 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:43.592704+0000", 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 125.18019851, 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124525767, 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.868 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592704+0000", 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592704+0000", 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592704+0000", 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7999999999999995e-07 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.869 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592712+0000", 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3710000000000002e-06 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592720+0000", 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.79e-06 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.592726+0000", 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2899999999999999e-06 2024-09-21T11:49:48.870 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717211+0000", 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.124484728 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717217+0000", 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6379999999999998e-06 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:43.717229+0000", 2024-09-21T11:49:48.871 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2269999999999999e-05 2024-09-21T11:49:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.872 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:216152 8.a 8:5bb2904f:::benchmark_data_smithi043_97042_object79058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-21T11:49:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:46.973159+0000", 2024-09-21T11:49:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 121.799743218, 2024-09-21T11:49:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.123890466, 2024-09-21T11:49:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:48.873 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 216152 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.973159+0000", 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.973159+0000", 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.874 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.973160+0000", 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3319999999999999e-06 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.973165+0000", 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6999999999999999e-06 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.973176+0000", 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1270999999999999e-05 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.875 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.973183+0000", 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8029999999999999e-06 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.973228+0000", 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4975999999999998e-05 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.973355+0000", 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012691699999999999 2024-09-21T11:49:48.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,3,5", 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.973476+0000", 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121167 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.982134+0000", 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0086576250000000004 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.982146+0000", 2024-09-21T11:49:48.877 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2587e-05 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.982183+0000", 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.667e-05 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:47.096874+0000", 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.11469073 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:47.096932+0000", 2024-09-21T11:49:48.878 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8625000000000002e-05 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:47.097049+0000", 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117063 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:216162 8.7 8:e1e3e293:::benchmark_data_smithi043_97042_object79068:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:47:46.979979+0000", 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 121.79292320899999, 2024-09-21T11:49:48.879 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.126665533, 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 216162 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.979979+0000", 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.979979+0000", 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.979980+0000", 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.238e-06 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.979984+0000", 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9700000000000001e-06 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.881 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.979994+0000", 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5419999999999994e-06 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.980000+0000", 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.533e-06 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.980514+0000", 2024-09-21T11:49:48.882 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051381799999999998 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.980632+0000", 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011753500000000001 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,3,6", 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:46.980753+0000", 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012144799999999999 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.883 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:47.092654+0000", 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.111901158 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:47.092739+0000", 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4431000000000005e-05 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:47.092776+0000", 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6832000000000002e-05 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:49:48.884 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:47.106248+0000", 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013472728999999999 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:47.106312+0000", 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4073999999999997e-05 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:47:47.106645+0000", 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00033222500000000001 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.885 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:49:48.886 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:49:48.886 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:49:48.927 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:48.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:48.928 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:49.029 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:49.066 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:49.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:49.067 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:49.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:49.210 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:49.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:49.211 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:49.264 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 36 16 66478 66462 115.347 86.3125 0.00863613 0.00866455 2024-09-21T11:49:49.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:49.349 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:49.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:49.351 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:49.451 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:49.479 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:49.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:49.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:49.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:49 smithi043 ceph-mon[36454]: pgmap v470: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 917 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:49:49.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:49 smithi043 ceph-mon[36454]: 5.13 scrub starts 2024-09-21T11:49:49.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:49 smithi043 ceph-mon[36454]: 5.13 scrub ok 2024-09-21T11:49:49.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:49 smithi043 ceph-mon[29710]: pgmap v470: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 917 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:49:49.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:49 smithi043 ceph-mon[29710]: 5.13 scrub starts 2024-09-21T11:49:49.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:49 smithi043 ceph-mon[29710]: 5.13 scrub ok 2024-09-21T11:49:49.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 3 1 2024-09-21T11:49:49.581 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:49.608 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:49.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:49.609 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:49.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:49 smithi175 ceph-mon[32613]: pgmap v470: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 917 B/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:49:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:49 smithi175 ceph-mon[32613]: 5.13 scrub starts 2024-09-21T11:49:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:49 smithi175 ceph-mon[32613]: 5.13 scrub ok 2024-09-21T11:49:49.762 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:49.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:49.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:49.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:49.915 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:49:49.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:49.927 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:49.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:50.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:50.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:50.074 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:50.175 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:50.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:50.212 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:50.212 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:50.265 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 37 16 67793 67777 114.45 82.1875 0.0153915 0.0087325 2024-09-21T11:49:50.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:50.351 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:50.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:50.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:50.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:50.490 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:50.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:50.491 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:50.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:50.628 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:50.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:50.629 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:50.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:50.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:50.756 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:50.756 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:50 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T11:49:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:50 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T11:49:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:50 smithi175 ceph-mon[32613]: 3.d scrub starts 2024-09-21T11:49:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:50 smithi175 ceph-mon[32613]: 3.d scrub ok 2024-09-21T11:49:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:50 smithi175 ceph-mon[32613]: 3.1a scrub starts 2024-09-21T11:49:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:50 smithi175 ceph-mon[32613]: 3.1a scrub ok 2024-09-21T11:49:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:50 smithi175 ceph-mon[32613]: 5.17 scrub starts 2024-09-21T11:49:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:50 smithi175 ceph-mon[32613]: 5.17 scrub ok 2024-09-21T11:49:50.856 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:50.892 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:50.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:50.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T11:49:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T11:49:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[29710]: 3.d scrub starts 2024-09-21T11:49:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[29710]: 3.d scrub ok 2024-09-21T11:49:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[29710]: 3.1a scrub starts 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[29710]: 3.1a scrub ok 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[29710]: 5.17 scrub starts 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[29710]: 5.17 scrub ok 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[36454]: 3.d scrub starts 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[36454]: 3.d scrub ok 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[36454]: 3.1a scrub starts 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[36454]: 3.1a scrub ok 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[36454]: 5.17 scrub starts 2024-09-21T11:49:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:50 smithi043 ceph-mon[36454]: 5.17 scrub ok 2024-09-21T11:49:50.994 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:51.030 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:51.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:51.031 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:51.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:51.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:51.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:51.265 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 38 16 69114 69098 113.611 82.5625 0.00877989 0.00879664 2024-09-21T11:49:51.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:51.313 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:51.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:51.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:51.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:51.456 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:51.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:51.457 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:51.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:51.590 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:51.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:51.591 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:51.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:51.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:51.729 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:51.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:51 smithi175 ceph-mon[32613]: pgmap v471: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 778 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-21T11:49:51.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:51 smithi175 ceph-mon[32613]: 2.0 scrub starts 2024-09-21T11:49:51.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:51 smithi175 ceph-mon[32613]: 2.0 scrub ok 2024-09-21T11:49:51.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:49:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:49:51.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:51.868 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:51.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:51.869 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:51.872 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:51 smithi043 ceph-mon[36454]: pgmap v471: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 778 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-21T11:49:51.872 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:51 smithi043 ceph-mon[36454]: 2.0 scrub starts 2024-09-21T11:49:51.872 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:51 smithi043 ceph-mon[36454]: 2.0 scrub ok 2024-09-21T11:49:51.873 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:51 smithi043 ceph-mon[29710]: pgmap v471: 148 pgs: 148 active+clean; 3.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 778 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-21T11:49:51.873 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:51 smithi043 ceph-mon[29710]: 2.0 scrub starts 2024-09-21T11:49:51.873 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:51 smithi043 ceph-mon[29710]: 2.0 scrub ok 2024-09-21T11:49:51.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:52.008 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:52.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:52.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:49:52.124 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:52.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:52.266 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 39 16 70375 70359 112.719 78.8125 0.0109658 0.00886678 2024-09-21T11:49:52.273 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:52.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:52.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:52.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:52.393 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:52.446 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:52.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:52.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:52.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:52.591 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:52.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:52.592 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:52.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:52.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:52.727 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:52.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:52.827 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:52.859 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:52.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:52.860 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:52.864 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:52.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:52 smithi043 ceph-mon[36454]: 4.0 scrub starts 2024-09-21T11:49:52.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:52 smithi043 ceph-mon[36454]: 4.0 scrub ok 2024-09-21T11:49:52.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:52 smithi043 ceph-mon[36454]: osdmap e262: 8 total, 8 up, 8 in 2024-09-21T11:49:52.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:52.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:52 smithi043 ceph-mon[29710]: 4.0 scrub starts 2024-09-21T11:49:52.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:52 smithi043 ceph-mon[29710]: 4.0 scrub ok 2024-09-21T11:49:52.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:52 smithi043 ceph-mon[29710]: osdmap e262: 8 total, 8 up, 8 in 2024-09-21T11:49:52.960 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:53.007 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:53.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:53.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:53.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:49:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:52 smithi175 ceph-mon[32613]: 4.0 scrub starts 2024-09-21T11:49:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:52 smithi175 ceph-mon[32613]: 4.0 scrub ok 2024-09-21T11:49:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:52 smithi175 ceph-mon[32613]: osdmap e262: 8 total, 8 up, 8 in 2024-09-21T11:49:53.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:53.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:53.165 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:53.165 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:53.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:53.266 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:49:53.264978+0000 min lat: 0.00137184 max lat: 0.17025 avg lat: 0.00895175 2024-09-21T11:49:53.266 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:49:53.266 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 40 16 71489 71473 111.64 69.625 0.0237717 0.00895175 2024-09-21T11:49:53.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:53.295 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:53.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:53.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:53.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:53.441 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:53.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:53.526 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:49:53.543 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:53.564 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:53.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:53.565 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:53.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:53.702 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:53.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:53.703 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:53.803 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:53.843 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:53.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:53.844 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:53.944 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:53.972 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:53.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:53.972 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[36454]: pgmap v472: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-21T11:49:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2933317923' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:49:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[29710]: pgmap v472: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-21T11:49:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2933317923' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:53 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:49:54.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:53 smithi175 ceph-mon[32613]: pgmap v472: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.69k op/s 2024-09-21T11:49:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:53 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2933317923' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:49:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:53 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:49:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:53 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:49:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:53 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:49:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:53 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:49:54.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:54.112 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:54.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:54.113 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:54.214 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:54.249 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:54.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:54.250 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:54.266 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 41 16 72630 72614 110.656 71.3125 0.00365393 0.00902943 2024-09-21T11:49:54.350 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:54.381 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:54.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:54.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:54.483 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:54.509 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:54.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:54.510 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:54.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:54.641 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:54.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:54.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:54.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:54.782 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:54.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:54.783 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:54.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:54.921 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:54.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:54.923 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:49:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:49:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2933317923' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[36454]: osdmap e263: 8 total, 8 up, 8 in 2024-09-21T11:49:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[36454]: 6.5 scrub starts 2024-09-21T11:49:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[36454]: 6.5 scrub ok 2024-09-21T11:49:54.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:49:54.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:49:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2933317923' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[29710]: osdmap e263: 8 total, 8 up, 8 in 2024-09-21T11:49:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[29710]: 6.5 scrub starts 2024-09-21T11:49:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:54 smithi043 ceph-mon[29710]: 6.5 scrub ok 2024-09-21T11:49:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:54 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:49:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:54 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:49:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:54 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2933317923' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:49:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:54 smithi175 ceph-mon[32613]: osdmap e263: 8 total, 8 up, 8 in 2024-09-21T11:49:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:54 smithi175 ceph-mon[32613]: 6.5 scrub starts 2024-09-21T11:49:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:54 smithi175 ceph-mon[32613]: 6.5 scrub ok 2024-09-21T11:49:55.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:55.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:55.062 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:55.163 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:55.194 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:55.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:55.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:55.266 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 42 16 73854 73838 109.843 76.5 0.011206 0.00909878 2024-09-21T11:49:55.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:55.340 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:55.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:55.341 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:55.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:55.475 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:55.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:55.476 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:49:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:49:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:49:55.548 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T11:49:55.576 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:49:55.605 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:55.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:55.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:49:55.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:55.771 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:55.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:55.772 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:55.802 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[36454]: pgmap v475: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 786 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-21T11:49:55.802 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/88431087' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:49:55.802 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[36454]: 6.4 scrub starts 2024-09-21T11:49:55.802 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[36454]: 6.4 scrub ok 2024-09-21T11:49:55.802 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[36454]: 3.7 scrub starts 2024-09-21T11:49:55.802 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[36454]: 3.7 scrub ok 2024-09-21T11:49:55.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[29710]: pgmap v475: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 786 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-21T11:49:55.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/88431087' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:49:55.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[29710]: 6.4 scrub starts 2024-09-21T11:49:55.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[29710]: 6.4 scrub ok 2024-09-21T11:49:55.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[29710]: 3.7 scrub starts 2024-09-21T11:49:55.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:55 smithi043 ceph-mon[29710]: 3.7 scrub ok 2024-09-21T11:49:55.873 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:55.908 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:55.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:55.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:56.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:55 smithi175 ceph-mon[32613]: pgmap v475: 148 pgs: 148 active+clean; 4.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 786 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-21T11:49:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:55 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/88431087' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:49:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:55 smithi175 ceph-mon[32613]: 6.4 scrub starts 2024-09-21T11:49:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:55 smithi175 ceph-mon[32613]: 6.4 scrub ok 2024-09-21T11:49:56.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:55 smithi175 ceph-mon[32613]: 3.7 scrub starts 2024-09-21T11:49:56.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:55 smithi175 ceph-mon[32613]: 3.7 scrub ok 2024-09-21T11:49:56.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:56.056 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:56.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:56.157 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:56.186 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:56.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:56.187 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:56.267 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 43 16 75124 75108 109.134 79.375 0.0101988 0.0091578 2024-09-21T11:49:56.278 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:49:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:49:56.288 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:56.315 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:49:56.315 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:49:56.318 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:56.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:56.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:56.350 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:49:56.350 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:49:56.388 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:49:56.388 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:49:56.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:56.423 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:49:56.423 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:49:56.463 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:49:56.463 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:49:56.465 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:56.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:56.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:56.498 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:49:56.498 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:49:56.533 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:49:56.533 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:49:56.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:56.609 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:56.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:56.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:56.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:56.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:56 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/88431087' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:49:56.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:56 smithi043 ceph-mon[36454]: osdmap e264: 8 total, 8 up, 8 in 2024-09-21T11:49:56.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:56 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/847936180' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:56 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/88431087' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:49:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:56 smithi043 ceph-mon[29710]: osdmap e264: 8 total, 8 up, 8 in 2024-09-21T11:49:56.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:56 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/847936180' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:56.746 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:56.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:56.747 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:56.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:56.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:56.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:56.984 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:57.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:56 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/88431087' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:49:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:56 smithi175 ceph-mon[32613]: osdmap e264: 8 total, 8 up, 8 in 2024-09-21T11:49:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:56 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/847936180' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:49:57.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 4 1 2024-09-21T11:49:57.028 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:57.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:57.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:57.130 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:57.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:57.173 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:57.173 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:57.233 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:49:57.233 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:49:57.266 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 44 16 76376 76360 108.431 78.25 0.00727263 0.0092167 2024-09-21T11:49:57.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:57.302 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:57.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:57.302 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:57.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:57.450 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:57.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:57.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:57.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:57.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:57.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:57.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:57.732 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:57.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:57.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:57.833 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:57.864 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:57.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:57.865 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:57.941 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:57 smithi043 ceph-mon[36454]: pgmap v477: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-21T11:49:57.941 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:57 smithi043 ceph-mon[36454]: 6.a scrub starts 2024-09-21T11:49:57.941 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:57 smithi043 ceph-mon[36454]: 6.a scrub ok 2024-09-21T11:49:57.941 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:57 smithi043 ceph-mon[36454]: osdmap e265: 8 total, 8 up, 8 in 2024-09-21T11:49:57.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:57 smithi043 ceph-mon[29710]: pgmap v477: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-21T11:49:57.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:57 smithi043 ceph-mon[29710]: 6.a scrub starts 2024-09-21T11:49:57.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:57 smithi043 ceph-mon[29710]: 6.a scrub ok 2024-09-21T11:49:57.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:57 smithi043 ceph-mon[29710]: osdmap e265: 8 total, 8 up, 8 in 2024-09-21T11:49:57.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:58.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:58.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:58.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:57 smithi175 ceph-mon[32613]: pgmap v477: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-21T11:49:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:57 smithi175 ceph-mon[32613]: 6.a scrub starts 2024-09-21T11:49:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:57 smithi175 ceph-mon[32613]: 6.a scrub ok 2024-09-21T11:49:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:57 smithi175 ceph-mon[32613]: osdmap e265: 8 total, 8 up, 8 in 2024-09-21T11:49:58.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:58.141 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:58.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:58.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:58.243 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:58.266 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 45 16 77692 77676 107.85 82.25 0.0176457 0.00926711 2024-09-21T11:49:58.274 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:58.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:58.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:58.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:58.413 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:58.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:58.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:58.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:49:58.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:58.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:49:58.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:49:58.689 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:58.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:58.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:49:58.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:49:58.826 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:58.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:58.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:49:58.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:58.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:58.967 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:58.967 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:58 smithi043 ceph-mon[36454]: 5.12 scrub starts 2024-09-21T11:49:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:58 smithi043 ceph-mon[36454]: 5.12 scrub ok 2024-09-21T11:49:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:58 smithi043 ceph-mon[36454]: 3.13 scrub starts 2024-09-21T11:49:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:58 smithi043 ceph-mon[36454]: 3.13 scrub ok 2024-09-21T11:49:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:58 smithi043 ceph-mon[29710]: 5.12 scrub starts 2024-09-21T11:49:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:58 smithi043 ceph-mon[29710]: 5.12 scrub ok 2024-09-21T11:49:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:58 smithi043 ceph-mon[29710]: 3.13 scrub starts 2024-09-21T11:49:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:58 smithi043 ceph-mon[29710]: 3.13 scrub ok 2024-09-21T11:49:59.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:58 smithi175 ceph-mon[32613]: 5.12 scrub starts 2024-09-21T11:49:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:58 smithi175 ceph-mon[32613]: 5.12 scrub ok 2024-09-21T11:49:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:58 smithi175 ceph-mon[32613]: 3.13 scrub starts 2024-09-21T11:49:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:58 smithi175 ceph-mon[32613]: 3.13 scrub ok 2024-09-21T11:49:59.068 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:59.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:59.101 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:59.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:59.202 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:49:59.226 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:59.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:59.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:49:59.266 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 46 16 78912 78896 107.163 76.25 0.0143149 0.00932731 2024-09-21T11:49:59.330 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:49:59.363 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:49:59.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:59.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:49:59.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:59.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:59.506 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:59.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:59.647 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:59.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:59.648 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:49:59.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:49:59.749 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:49:59.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:59.793 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:59.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:49:59.872 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:59 smithi043 ceph-mon[36454]: pgmap v479: 148 pgs: 148 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 906 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-21T11:49:59.872 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:59 smithi043 ceph-mon[36454]: 4.1e scrub starts 2024-09-21T11:49:59.872 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:49:59 smithi043 ceph-mon[36454]: 4.1e scrub ok 2024-09-21T11:49:59.872 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:59 smithi043 ceph-mon[29710]: pgmap v479: 148 pgs: 148 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 906 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-21T11:49:59.872 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:59 smithi043 ceph-mon[29710]: 4.1e scrub starts 2024-09-21T11:49:59.872 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:49:59 smithi043 ceph-mon[29710]: 4.1e scrub ok 2024-09-21T11:49:59.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:49:59.936 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:49:59.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:49:59.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:59 smithi175 ceph-mon[32613]: pgmap v479: 148 pgs: 148 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 906 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-21T11:50:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:59 smithi175 ceph-mon[32613]: 4.1e scrub starts 2024-09-21T11:50:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:49:59 smithi175 ceph-mon[32613]: 4.1e scrub ok 2024-09-21T11:50:00.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:00.067 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:00.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:00.069 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:00.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:00.219 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:00.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:00.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:00.267 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 47 16 80190 80174 106.582 79.875 0.00793062 0.0093773 2024-09-21T11:50:00.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:00.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:00.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:00.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:00.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:00.495 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:00.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:00.596 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:00.624 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:00.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:00.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:00.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[29710]: 4.18 scrub starts 2024-09-21T11:50:00.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[29710]: 4.18 scrub ok 2024-09-21T11:50:00.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:50:00.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:50:00.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[29710]: 3.4 scrub starts 2024-09-21T11:50:00.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[29710]: 3.4 scrub ok 2024-09-21T11:50:00.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[29710]: pgmap v480: 148 pgs: 148 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 795 B/s rd, 59 MiB/s wr, 938 op/s 2024-09-21T11:50:00.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[29710]: overall HEALTH_OK 2024-09-21T11:50:00.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[36454]: 4.18 scrub starts 2024-09-21T11:50:00.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[36454]: 4.18 scrub ok 2024-09-21T11:50:00.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:50:00.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:50:00.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[36454]: 3.4 scrub starts 2024-09-21T11:50:00.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[36454]: 3.4 scrub ok 2024-09-21T11:50:00.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[36454]: pgmap v480: 148 pgs: 148 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 795 B/s rd, 59 MiB/s wr, 938 op/s 2024-09-21T11:50:00.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:00 smithi043 ceph-mon[36454]: overall HEALTH_OK 2024-09-21T11:50:00.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:00.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:00.763 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:00.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:00.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:00.903 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:00.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:00.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:00.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:00 smithi175 ceph-mon[32613]: 4.18 scrub starts 2024-09-21T11:50:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:00 smithi175 ceph-mon[32613]: 4.18 scrub ok 2024-09-21T11:50:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:00 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:50:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:00 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:50:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:00 smithi175 ceph-mon[32613]: 3.4 scrub starts 2024-09-21T11:50:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:00 smithi175 ceph-mon[32613]: 3.4 scrub ok 2024-09-21T11:50:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:00 smithi175 ceph-mon[32613]: pgmap v480: 148 pgs: 148 active+clean; 4.6 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 795 B/s rd, 59 MiB/s wr, 938 op/s 2024-09-21T11:50:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:00 smithi175 ceph-mon[32613]: overall HEALTH_OK 2024-09-21T11:50:01.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:01.047 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:01.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:01.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:01.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:01.185 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:01.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:01.186 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:01.279 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 48 16 81447 81431 105.971 78.5625 0.00573909 0.00942854 2024-09-21T11:50:01.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:01.321 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:01.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:01.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:01.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:01.464 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:01.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:01.466 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:01.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:01.597 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:50:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:50:01.615 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:01.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:01.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:01.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:01.748 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:01.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:01.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:01.849 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:01.879 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:01.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:01.880 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:01.903 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:50:01.903 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:50:01.904 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:01 smithi043 ceph-mon[36454]: osdmap e266: 8 total, 8 up, 8 in 2024-09-21T11:50:01.904 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:01 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:01.904 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:01.904 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:01 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:01.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:01 smithi043 ceph-mon[29710]: osdmap e266: 8 total, 8 up, 8 in 2024-09-21T11:50:01.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:01 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:01.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:01.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:01 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:01.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:02.007 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:02.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:02.008 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:02.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:01 smithi175 ceph-mon[32613]: osdmap e266: 8 total, 8 up, 8 in 2024-09-21T11:50:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:01 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:01 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:02.109 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:02.142 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:02.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:02.143 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:02.244 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:02.270 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:02.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:02.271 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:02.279 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 49 16 82563 82547 105.232 69.75 0.00349549 0.00949366 2024-09-21T11:50:02.303 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T11:50:02.372 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:02.407 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:02.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:02.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:02.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:50:02.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:02.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:02.570 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:02.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:02.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:02.716 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:02.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:02.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:02.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:50:02.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:02.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:02.847 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:02.847 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:02.948 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:02.979 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:02.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:02.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[36454]: pgmap v482: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.6 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-21T11:50:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3718893661' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:50:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3718893661' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:50:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[36454]: osdmap e267: 8 total, 8 up, 8 in 2024-09-21T11:50:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[36454]: 6.11 scrub starts 2024-09-21T11:50:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[36454]: 6.11 scrub ok 2024-09-21T11:50:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[36454]: 3.11 scrub starts 2024-09-21T11:50:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[36454]: 3.11 scrub ok 2024-09-21T11:50:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[29710]: pgmap v482: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.6 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-21T11:50:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3718893661' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:50:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3718893661' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:50:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[29710]: osdmap e267: 8 total, 8 up, 8 in 2024-09-21T11:50:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[29710]: 6.11 scrub starts 2024-09-21T11:50:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[29710]: 6.11 scrub ok 2024-09-21T11:50:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[29710]: 3.11 scrub starts 2024-09-21T11:50:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:02 smithi043 ceph-mon[29710]: 3.11 scrub ok 2024-09-21T11:50:03.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:02 smithi175 ceph-mon[32613]: pgmap v482: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.6 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-21T11:50:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3718893661' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:50:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3718893661' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:50:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:02 smithi175 ceph-mon[32613]: osdmap e267: 8 total, 8 up, 8 in 2024-09-21T11:50:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:02 smithi175 ceph-mon[32613]: 6.11 scrub starts 2024-09-21T11:50:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:02 smithi175 ceph-mon[32613]: 6.11 scrub ok 2024-09-21T11:50:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:02 smithi175 ceph-mon[32613]: 3.11 scrub starts 2024-09-21T11:50:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:02 smithi175 ceph-mon[32613]: 3.11 scrub ok 2024-09-21T11:50:03.081 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:03.115 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:03.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:03.116 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:03.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:03.256 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:03.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:03.257 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:03.279 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 50 16 83643 83627 104.478 67.5 0.011487 0.00956646 2024-09-21T11:50:03.359 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:03.390 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:03.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:03.391 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:03.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:03.523 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:03.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:03.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:03.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:50:03.624 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:03.652 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:03.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:03.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:03.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:03.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:03.815 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:03.901 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:03.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:03.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:03.970 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[36454]: 6.16 scrub starts 2024-09-21T11:50:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[36454]: 6.16 scrub ok 2024-09-21T11:50:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[36454]: 3.19 scrub starts 2024-09-21T11:50:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[36454]: 3.19 scrub ok 2024-09-21T11:50:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[36454]: 5.d scrub starts 2024-09-21T11:50:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[36454]: 5.d scrub ok 2024-09-21T11:50:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[36454]: 4.19 scrub starts 2024-09-21T11:50:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[36454]: 4.19 scrub ok 2024-09-21T11:50:03.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[29710]: 6.16 scrub starts 2024-09-21T11:50:03.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[29710]: 6.16 scrub ok 2024-09-21T11:50:03.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[29710]: 3.19 scrub starts 2024-09-21T11:50:03.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[29710]: 3.19 scrub ok 2024-09-21T11:50:03.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[29710]: 5.d scrub starts 2024-09-21T11:50:03.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[29710]: 5.d scrub ok 2024-09-21T11:50:03.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[29710]: 4.19 scrub starts 2024-09-21T11:50:03.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:03 smithi043 ceph-mon[29710]: 4.19 scrub ok 2024-09-21T11:50:04.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:03 smithi175 ceph-mon[32613]: 6.16 scrub starts 2024-09-21T11:50:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:03 smithi175 ceph-mon[32613]: 6.16 scrub ok 2024-09-21T11:50:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:03 smithi175 ceph-mon[32613]: 3.19 scrub starts 2024-09-21T11:50:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:03 smithi175 ceph-mon[32613]: 3.19 scrub ok 2024-09-21T11:50:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:03 smithi175 ceph-mon[32613]: 5.d scrub starts 2024-09-21T11:50:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:03 smithi175 ceph-mon[32613]: 5.d scrub ok 2024-09-21T11:50:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:03 smithi175 ceph-mon[32613]: 4.19 scrub starts 2024-09-21T11:50:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:03 smithi175 ceph-mon[32613]: 4.19 scrub ok 2024-09-21T11:50:04.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:04.103 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:04.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:04.104 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:04.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:04.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:04.245 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:04.279 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 51 15 84870 84855 103.934 76.75 0.0080251 0.00961697 2024-09-21T11:50:04.346 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:04.381 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:04.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:04.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:04.448 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:04.483 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:04.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 5 1 2024-09-21T11:50:04.510 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:04.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:04.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:04.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:04.673 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:04.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:04.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:04.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:04 smithi043 ceph-mon[36454]: pgmap v484: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 769 B/s rd, 59 MiB/s wr, 944 op/s 2024-09-21T11:50:04.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:04 smithi043 ceph-mon[29710]: pgmap v484: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 769 B/s rd, 59 MiB/s wr, 944 op/s 2024-09-21T11:50:04.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:04.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:04.802 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:04.802 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:04.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:04.936 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:04.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:04.937 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:05.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:04 smithi175 ceph-mon[32613]: pgmap v484: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 769 B/s rd, 59 MiB/s wr, 944 op/s 2024-09-21T11:50:05.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:05.063 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:05.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:05.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:05.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:05.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:05.203 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:05.204 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:05.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 52 16 86015 85999 103.31 71.5 0.0157031 0.00967291 2024-09-21T11:50:05.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:05.345 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:05.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:05.346 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:05.350 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:50:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:50:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:50:05.352 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:50:05.352 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:50:05.390 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:50:05.390 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:50:05.424 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:50:05.425 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:50:05.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:05.471 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:50:05.471 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:50:05.485 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:05.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:05.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:05.506 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:50:05.506 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:50:05.547 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:50:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:50:05.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:05.588 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:50:05.588 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:50:05.635 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:50:05.636 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:50:05.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:05.638 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:05.638 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:05.663 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:05.739 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:05.770 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:05.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:05.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:05.872 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:05.907 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:05.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:05.908 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:05.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:05 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1953334129' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:05.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:05 smithi043 ceph-mon[36454]: 6.12 scrub starts 2024-09-21T11:50:05.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:05 smithi043 ceph-mon[36454]: 6.12 scrub ok 2024-09-21T11:50:05.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:05 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1953334129' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:05.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:05 smithi043 ceph-mon[29710]: 6.12 scrub starts 2024-09-21T11:50:05.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:05 smithi043 ceph-mon[29710]: 6.12 scrub ok 2024-09-21T11:50:06.009 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:06.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:05 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1953334129' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:06.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:05 smithi175 ceph-mon[32613]: 6.12 scrub starts 2024-09-21T11:50:06.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:05 smithi175 ceph-mon[32613]: 6.12 scrub ok 2024-09-21T11:50:06.043 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:06.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:06.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:06.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:06.178 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:06.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:06.179 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:06.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 53 16 87195 87179 102.753 73.75 0.010056 0.00972741 2024-09-21T11:50:06.280 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:06.315 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:06.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:06.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:06.417 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:06.453 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:06.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:06.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:06.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:06.589 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:06.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:06.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:06.656 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:50:06.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:06.719 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:06.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:06.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:06.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:06.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:06.864 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:06.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[36454]: 5.1e scrub starts 2024-09-21T11:50:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[36454]: 5.1e scrub ok 2024-09-21T11:50:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[36454]: pgmap v485: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 57 MiB/s wr, 917 op/s 2024-09-21T11:50:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3688175594' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[36454]: 6.1f scrub starts 2024-09-21T11:50:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[36454]: 3.f scrub starts 2024-09-21T11:50:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[36454]: 3.f scrub ok 2024-09-21T11:50:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[36454]: 6.1f scrub ok 2024-09-21T11:50:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[29710]: 5.1e scrub starts 2024-09-21T11:50:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[29710]: 5.1e scrub ok 2024-09-21T11:50:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[29710]: pgmap v485: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 57 MiB/s wr, 917 op/s 2024-09-21T11:50:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3688175594' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[29710]: 6.1f scrub starts 2024-09-21T11:50:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[29710]: 3.f scrub starts 2024-09-21T11:50:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[29710]: 3.f scrub ok 2024-09-21T11:50:06.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:06 smithi043 ceph-mon[29710]: 6.1f scrub ok 2024-09-21T11:50:07.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:07.002 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:07.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:07.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:06 smithi175 ceph-mon[32613]: 5.1e scrub starts 2024-09-21T11:50:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:06 smithi175 ceph-mon[32613]: 5.1e scrub ok 2024-09-21T11:50:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:06 smithi175 ceph-mon[32613]: pgmap v485: 148 pgs: 148 active+clean; 5.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 57 MiB/s wr, 917 op/s 2024-09-21T11:50:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:06 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3688175594' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:06 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:06 smithi175 ceph-mon[32613]: 6.1f scrub starts 2024-09-21T11:50:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:06 smithi175 ceph-mon[32613]: 3.f scrub starts 2024-09-21T11:50:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:06 smithi175 ceph-mon[32613]: 3.f scrub ok 2024-09-21T11:50:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:06 smithi175 ceph-mon[32613]: 6.1f scrub ok 2024-09-21T11:50:07.102 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:07.138 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:07.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:07.139 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:07.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:07.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:07.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:07.279 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 54 16 88439 88423 102.29 77.75 0.00780081 0.00977086 2024-09-21T11:50:07.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:07.422 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:07.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:07.423 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:07.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:07.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:07.555 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:07.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:07.656 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:07.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:07.687 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:07.687 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:07.788 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:07.823 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:07.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:07.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:07.925 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:07.960 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:07.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:07.961 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[36454]: 4.c scrub starts 2024-09-21T11:50:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[36454]: 4.c scrub ok 2024-09-21T11:50:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:50:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[36454]: osdmap e268: 8 total, 8 up, 8 in 2024-09-21T11:50:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[36454]: osdmap e269: 8 total, 8 up, 8 in 2024-09-21T11:50:07.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[29710]: 4.c scrub starts 2024-09-21T11:50:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[29710]: 4.c scrub ok 2024-09-21T11:50:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:50:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[29710]: osdmap e268: 8 total, 8 up, 8 in 2024-09-21T11:50:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:07 smithi043 ceph-mon[29710]: osdmap e269: 8 total, 8 up, 8 in 2024-09-21T11:50:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:07 smithi175 ceph-mon[32613]: 4.c scrub starts 2024-09-21T11:50:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:07 smithi175 ceph-mon[32613]: 4.c scrub ok 2024-09-21T11:50:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:07 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:07 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:50:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:07 smithi175 ceph-mon[32613]: osdmap e268: 8 total, 8 up, 8 in 2024-09-21T11:50:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:07 smithi175 ceph-mon[32613]: osdmap e269: 8 total, 8 up, 8 in 2024-09-21T11:50:08.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:08.094 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:08.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:08.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:08.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:08.234 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:08.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:08.235 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:08.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 55 16 89687 89671 101.848 78 0.00901269 0.00981359 2024-09-21T11:50:08.335 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:08.343 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:08.359 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:08.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:08.360 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:08.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:08.492 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:08.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:08.493 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:08.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:08.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:08.645 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:08.746 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:08.779 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:08.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:08.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:08.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:08.926 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:08.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:08.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:08.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:08 smithi043 ceph-mon[29710]: pgmap v488: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 60 MiB/s wr, 960 op/s 2024-09-21T11:50:08.932 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:08 smithi043 ceph-mon[36454]: pgmap v488: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 60 MiB/s wr, 960 op/s 2024-09-21T11:50:09.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:08 smithi175 ceph-mon[32613]: pgmap v488: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 60 MiB/s wr, 960 op/s 2024-09-21T11:50:09.028 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:09.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:09.065 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:09.065 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:09.166 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:09.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:09.195 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:09.196 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:09.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 56 16 90852 90836 101.33 72.8125 0.0271738 0.00986349 2024-09-21T11:50:09.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:09.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:09.338 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:09.419 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:50:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:50:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:50:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:50:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245657 8.b 8:db11300a:::benchmark_data_smithi043_97042_object108563:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.419 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.69896561, 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167927969999999, 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245657 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565747+0000", 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5340000000000001e-06 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565752+0000", 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8459999999999999e-06 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565762+0000", 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6029999999999994e-06 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565769+0000", 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3019999999999997e-06 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565833+0000", 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4139000000000004e-05 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565842+0000", 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2550000000000003e-06 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576490+0000", 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.010647965 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576618+0000", 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000128233 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576763+0000", 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000144845 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582393+0000", 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056301229999999999 2024-09-21T11:50:09.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582458+0000", 2024-09-21T11:50:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4715000000000006e-05 2024-09-21T11:50:09.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582472+0000", 2024-09-21T11:50:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3985000000000001e-05 2024-09-21T11:50:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582509+0000", 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7063000000000001e-05 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582538+0000", 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9189000000000001e-05 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245660 8.b 8:d0b775d8:::benchmark_data_smithi043_97042_object108566:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.427 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.696040573, 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0141818570000001, 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245660 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:50:09.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0200000000000003e-07 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568673+0000", 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2029999999999999e-06 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568679+0000", 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7590000000000003e-06 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568684+0000", 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1800000000000004e-06 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568765+0000", 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0898999999999999e-05 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568774+0000", 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2e-06 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.579784+0000", 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0110101520000001 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.579907+0000", 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000122404 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.580042+0000", 2024-09-21T11:50:09.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000135205 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582646+0000", 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026039240000000001 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582746+0000", 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100447 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582764+0000", 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7265000000000002e-05 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582813+0000", 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9487999999999998e-05 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582852+0000", 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9329000000000002e-05 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.434 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245661 8.b 8:d3f46cc5:::benchmark_data_smithi043_97042_object108567:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:50:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.696011759, 2024-09-21T11:50:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026528527, 2024-09-21T11:50:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.435 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245661 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568700+0000", 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9200000000000001e-07 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568702+0000", 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4049999999999998e-06 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568708+0000", 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0900000000000001e-06 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568715+0000", 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3319999999999997e-06 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568799+0000", 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4618999999999997e-05 2024-09-21T11:50:09.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568806+0000", 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3239999999999999e-06 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582891+0000", 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0140855200000001 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.583014+0000", 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012294699999999999 2024-09-21T11:50:09.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.583136+0000", 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121651 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588881+0000", 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057450959999999999 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588920+0000", 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9140999999999997e-05 2024-09-21T11:50:09.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595113+0000", 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061930309999999999 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595177+0000", 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4243000000000001e-05 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595228+0000", 2024-09-21T11:50:09.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0636000000000001e-05 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245694 8.b 8:d4a2706b:::benchmark_data_smithi043_97042_object108600:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.666598454, 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99723669299999995, 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.442 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245694 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598114+0000", 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.203e-06 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598118+0000", 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3760000000000001e-06 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598129+0000", 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0373999999999999e-05 2024-09-21T11:50:09.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598137+0000", 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598208+0000", 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1235000000000005e-05 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598217+0000", 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0480000000000004e-06 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585700+0000", 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98748349499999999 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585809+0000", 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010897500000000001 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585910+0000", 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100326 2024-09-21T11:50:09.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588958+0000", 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030481290000000001 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595096+0000", 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.006137829 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595241+0000", 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014534899999999999 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595298+0000", 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7256999999999997e-05 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595349+0000", 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1267999999999998e-05 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.448 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245711 8.b 8:d32e232b:::benchmark_data_smithi043_97042_object108617:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.650356578, 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98116202900000005, 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245711 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614357+0000", 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9910000000000001e-06 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614362+0000", 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8119999999999996e-06 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614381+0000", 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8799000000000001e-05 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614390+0000", 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6289999999999999e-06 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614452+0000", 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2620999999999996e-05 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614462+0000", 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9869999999999994e-06 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588986+0000", 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.974523374 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.589275+0000", 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028897299999999999 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.589384+0000", 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109497 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595361+0000", 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059765479999999999 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595400+0000", 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8847999999999998e-05 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595444+0000", 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4270999999999997e-05 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595473+0000", 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9339000000000001e-05 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595517+0000", 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3340000000000002e-05 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245713 8.b 8:d46d9f23:::benchmark_data_smithi043_97042_object108619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.455 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.649553423, 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98051237099999999, 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245713 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615159+0000", 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.339e-06 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615164+0000", 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6659999999999999e-06 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615180+0000", 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5835e-05 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615188+0000", 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7600000000000008e-06 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615244+0000", 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5172e-05 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615252+0000", 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6969999999999999e-06 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592015+0000", 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97676251999999997 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592146+0000", 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000131094 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592299+0000", 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015273399999999999 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595406+0000", 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031073920000000001 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595425+0000", 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9119000000000001e-05 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595582+0000", 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001573 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595632+0000", 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9282e-05 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595670+0000", 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8461000000000003e-05 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245721 8.b 8:de2b861f:::benchmark_data_smithi043_97042_object108627:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.641647694, 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98811751400000003, 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245721 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623065+0000", 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.553e-06 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623072+0000", 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4729999999999997e-06 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623087+0000", 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5197999999999999e-05 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623095+0000", 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2369999999999992e-06 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623159+0000", 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3953000000000002e-05 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623168+0000", 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0350000000000007e-06 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595704+0000", 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97253655100000003 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595813+0000", 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108168 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595916+0000", 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103482 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604767+0000", 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088508360000000008 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604858+0000", 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0768999999999998e-05 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611090+0000", 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0062325690000000003 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611136+0000", 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5197000000000001e-05 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611181+0000", 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5493e-05 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245797 8.b 8:d43bc7ef:::benchmark_data_smithi043_97042_object108703:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.563403678, 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.910005379, 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245797 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.469 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701309+0000", 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1999999999999999e-06 2024-09-21T11:50:09.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701315+0000", 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701326+0000", 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0956e-05 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701334+0000", 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7570000000000004e-06 2024-09-21T11:50:09.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701426+0000", 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2200999999999994e-05 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701436+0000", 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0175e-05 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.598806+0000", 2024-09-21T11:50:09.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.89737045500000001 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.598902+0000", 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5308000000000003e-05 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.599142+0000", 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024078399999999999 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604823+0000", 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056808700000000002 2024-09-21T11:50:09.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.610984+0000", 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061603750000000001 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611191+0000", 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000207612 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611273+0000", 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1924999999999999e-05 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611313+0000", 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9651999999999999e-05 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245872 8.b 8:d0b8f221:::benchmark_data_smithi043_97042_object108778:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.47605486, 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.82278842799999996, 2024-09-21T11:50:09.475 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245872 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788658+0000", 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.708e-06 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788664+0000", 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5140000000000001e-06 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788679+0000", 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5764e-05 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788691+0000", 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.173e-05 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788774+0000", 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3029999999999996e-05 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788783+0000", 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9840000000000007e-06 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.601802+0000", 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81301899799999999 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.601917+0000", 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011464 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.602046+0000", 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000129228 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604894+0000", 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028480250000000001 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611032+0000", 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061384710000000004 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611319+0000", 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028687299999999999 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611405+0000", 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5921999999999995e-05 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611445+0000", 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9541e-05 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245873 8.b 8:dccc5dce:::benchmark_data_smithi043_97042_object108779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.475998007, 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.82286793400000002, 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245873 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.482 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788714+0000", 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5300000000000004e-07 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788717+0000", 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2849999999999999e-06 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788723+0000", 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3300000000000004e-06 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788729+0000", 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3979999999999998e-06 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788805+0000", 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6068000000000004e-05 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788810+0000", 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4720000000000003e-06 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604923+0000", 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81611315500000003 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.605021+0000", 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7503999999999993e-05 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.605122+0000", 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010059400000000001 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.610912+0000", 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057902530000000004 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611078+0000", 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016624000000000001 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611455+0000", 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037679500000000003 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611538+0000", 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3288999999999995e-05 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611581+0000", 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2997999999999997e-05 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245890 8.b 8:ddfc19d9:::benchmark_data_smithi043_97042_object108796:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.805073+0000", 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.45963827200001, 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81340417600000003, 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245890 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805073+0000", 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805073+0000", 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805074+0000", 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3799999999999999e-06 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805079+0000", 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805090+0000", 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0535e-05 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805096+0000", 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6429999999999998e-06 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805183+0000", 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6765e-05 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805195+0000", 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1684e-05 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.607855+0000", 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.80266011599999998 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.607979+0000", 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012364099999999999 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.608105+0000", 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012666500000000001 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611632+0000", 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035269120000000001 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611643+0000", 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0995000000000001e-05 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618369+0000", 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0067257940000000002 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618427+0000", 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8192999999999997e-05 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618477+0000", 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9858000000000003e-05 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.494 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245900 8.b 8:da6e4cef:::benchmark_data_smithi043_97042_object108806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.814804+0000", 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.44990711299999, 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.809993615, 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245900 2024-09-21T11:50:09.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814804+0000", 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814804+0000", 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814806+0000", 2024-09-21T11:50:09.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.767e-06 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814811+0000", 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.814e-06 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814825+0000", 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3949e-05 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814832+0000", 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6000000000000001e-06 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814903+0000", 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0462000000000004e-05 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814913+0000", 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0161e-05 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611670+0000", 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79675690499999996 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611808+0000", 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013816800000000001 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611894+0000", 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6110000000000001e-05 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618523+0000", 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0066291659999999997 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624649+0000", 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061260459999999996 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624696+0000", 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7271999999999997e-05 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624750+0000", 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3782000000000001e-05 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624798+0000", 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7413000000000002e-05 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245905 8.b 8:dde0f4b3:::benchmark_data_smithi043_97042_object108811:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.819241+0000", 2024-09-21T11:50:09.501 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.44547012699999, 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.80567893899999998, 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245905 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819241+0000", 2024-09-21T11:50:09.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819241+0000", 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819243+0000", 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.277e-06 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819249+0000", 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4659999999999998e-06 2024-09-21T11:50:09.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819276+0000", 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7506000000000001e-05 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819285+0000", 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7239999999999998e-06 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819348+0000", 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2675999999999994e-05 2024-09-21T11:50:09.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819356+0000", 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2470000000000008e-06 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.615392+0000", 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.796035728 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.615494+0000", 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102256 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.615604+0000", 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011042799999999999 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624636+0000", 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090319690000000008 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624655+0000", 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8525999999999998e-05 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624840+0000", 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018551700000000001 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624885+0000", 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624920+0000", 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4875000000000001e-05 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.507 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245931 8.b 8:d43f52c0:::benchmark_data_smithi043_97042_object108837:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.852205+0000", 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.412506427, 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77959361400000005, 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245931 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852205+0000", 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852205+0000", 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852206+0000", 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.09e-07 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852209+0000", 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.76e-06 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852220+0000", 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0198e-05 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852225+0000", 2024-09-21T11:50:09.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8220000000000003e-06 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852292+0000", 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6230999999999997e-05 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852301+0000", 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1339999999999993e-06 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618556+0000", 2024-09-21T11:50:09.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76625512600000001 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618656+0000", 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010033400000000001 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618785+0000", 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012859199999999999 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624659+0000", 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058743650000000003 2024-09-21T11:50:09.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631628+0000", 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0069688379999999998 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631704+0000", 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6015999999999999e-05 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631756+0000", 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1678999999999998e-05 2024-09-21T11:50:09.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631798+0000", 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.261e-05 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245933 8.b 8:d86bb2d6:::benchmark_data_smithi043_97042_object108839:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.855701+0000", 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.409010663, 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79554628400000005, 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.514 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245933 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855701+0000", 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855701+0000", 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855702+0000", 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1200000000000001e-06 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855706+0000", 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8879999999999999e-06 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855716+0000", 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0241000000000001e-05 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855721+0000", 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.366e-06 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855764+0000", 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2432999999999998e-05 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855770+0000", 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3470000000000004e-06 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.621640+0000", 2024-09-21T11:50:09.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76586964800000001 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.621756+0000", 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116568 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.621882+0000", 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125795 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631670+0000", 2024-09-21T11:50:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0097883689999999999 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631892+0000", 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000221797 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651081+0000", 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019188877 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651195+0000", 2024-09-21T11:50:09.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113743 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651247+0000", 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2092000000000002e-05 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245945 8.b 8:d81ed0d8:::benchmark_data_smithi043_97042_object108851:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.892389+0000", 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 105.372322192, 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.75901874300000005, 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.520 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245945 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892389+0000", 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892389+0000", 2024-09-21T11:50:09.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892390+0000", 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1629999999999999e-06 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892394+0000", 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892404+0000", 2024-09-21T11:50:09.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0132000000000001e-05 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892411+0000", 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9040000000000003e-06 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892458+0000", 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6913000000000003e-05 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892468+0000", 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0227999999999999e-05 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624952+0000", 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.73248412399999996 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.625090+0000", 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000138064 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.625205+0000", 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011459999999999999 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631842+0000", 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0066366690000000004 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631897+0000", 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5130000000000002e-05 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651292+0000", 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019394734 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651372+0000", 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0452999999999997e-05 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651408+0000", 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5737e-05 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:34879 9.4 e253/247 9:2d14d963:::benchmark_data_smithi043_126002_object34878:head v 253'2159, mlcod=253'2156)", 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.281595+0000", 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 40.983116533999997, 2024-09-21T11:50:09.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.16338703199999999, 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281595+0000", 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281595+0000", 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281595+0000", 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4099999999999998e-07 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281616+0000", 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.073e-05 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281626+0000", 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7259999999999997e-06 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281632+0000", 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0650000000000004e-06 2024-09-21T11:50:09.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.282792+0000", 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001160473 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.282821+0000", 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9113999999999999e-05 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.444917+0000", 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.162096037 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.444982+0000", 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4245999999999994e-05 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34884 9.2 9:45649dcd:::benchmark_data_smithi043_126002_object34883:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.281707+0000", 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 40.983004197, 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.15995288299999999, 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34884 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281707+0000", 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281707+0000", 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281708+0000", 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3600000000000002e-07 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281726+0000", 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8668000000000002e-05 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281735+0000", 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8899999999999996e-06 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281741+0000", 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9970000000000004e-06 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.283970+0000", 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022283200000000002 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284082+0000", 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011244 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284210+0000", 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000128057 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.286716+0000", 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025060909999999998 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.287038+0000", 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032201700000000002 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.287104+0000", 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5921999999999997e-05 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.441608+0000", 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.15450330500000001 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.441660+0000", 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2339999999999997e-05 2024-09-21T11:50:09.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34891 9.c 9:3bf97aa6:::benchmark_data_smithi043_126002_object34890:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.284551+0000", 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 40.980160067, 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.160549359, 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:09.536 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34891 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284551+0000", 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284551+0000", 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284552+0000", 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2929999999999999e-06 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284576+0000", 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3563000000000001e-05 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284586+0000", 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9350000000000006e-06 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284593+0000", 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3100000000000003e-06 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284689+0000", 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5951000000000003e-05 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284802+0000", 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011327700000000001 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,7", 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.284918+0000", 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115492 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.443593+0000", 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.15867461299999999 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.443668+0000", 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5323999999999999e-05 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.444992+0000", 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0013244109999999999 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.445049+0000", 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7108999999999998e-05 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.445100+0000", 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1081000000000001e-05 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:34892 9.7 e253/247 9:e59c7ff4:::benchmark_data_smithi043_126002_object34891:head v 253'2234, mlcod=253'2231)", 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.287941+0000", 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 40.976770307000002, 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.15996538399999999, 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.287941+0000", 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.287941+0000", 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.287942+0000", 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7400000000000002e-07 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.288041+0000", 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9691999999999994e-05 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.441419+0000", 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.15337746999999999 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.441427+0000", 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1100000000000003e-06 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.441518+0000", 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1349999999999998e-05 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:09.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.441536+0000", 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7552999999999999e-05 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.447872+0000", 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063365510000000002 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.447906+0000", 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3884000000000003e-05 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.545 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:09.546 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:09.546 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:50:09.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:09.574 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:09.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:09.575 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:09.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:50:09.676 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:09.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:09.702 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:09.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:09.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T11:50:09.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:09.885 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:09.885 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:09.977 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2284399809' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:50:09.977 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:09 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:50:09.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2284399809' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:50:09.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:09 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:50:09.986 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:10.020 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:10.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:10.021 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:10.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:10.158 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:10.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:10.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:10.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2284399809' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:50:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:09 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:50:10.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 57 16 91979 91963 100.788 70.4375 0.00438198 0.00991699 2024-09-21T11:50:10.291 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:10.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:10.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:10.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:10.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:50:10.432 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:10.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:10.433 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:10.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:10.576 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:10.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:10.577 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:10.678 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:10.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:50:10.704 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:10.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:10.704 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:10.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:10.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:10.853 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:10.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:10.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:11.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:11.010 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:11.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:10 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:50:11.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:10 smithi175 ceph-mon[32613]: osdmap e270: 8 total, 8 up, 8 in 2024-09-21T11:50:11.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:10 smithi175 ceph-mon[32613]: pgmap v490: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 960 op/s 2024-09-21T11:50:11.096 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:11.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:11.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:11.147 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:11.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:10 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:50:11.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:10 smithi043 ceph-mon[29710]: osdmap e270: 8 total, 8 up, 8 in 2024-09-21T11:50:11.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:10 smithi043 ceph-mon[29710]: pgmap v490: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 960 op/s 2024-09-21T11:50:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:10 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:50:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:10 smithi043 ceph-mon[36454]: osdmap e270: 8 total, 8 up, 8 in 2024-09-21T11:50:11.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:10 smithi043 ceph-mon[36454]: pgmap v490: 148 pgs: 148 active+clean; 5.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 960 op/s 2024-09-21T11:50:11.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:11.279 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:11.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 58 16 93179 93163 100.343 75 0.0103822 0.00996056 2024-09-21T11:50:11.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:11.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:11.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:11.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 6 1 2024-09-21T11:50:11.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:11.452 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:11.452 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:11.481 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:11.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:50:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:50:11.538 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:11.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:11.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:11.603 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:11.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:11.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:11.753 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:11.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:11.754 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:11.821 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:50:11.821 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:50:11.821 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:50:11.821 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:50:11.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:11.883 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:11.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:11.884 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:11.916 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:11 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:50:11.916 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:11 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:50:11.916 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:11 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:11.916 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:11 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:50:11.916 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:11 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:50:11.916 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:11 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:11.985 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:12.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:12.018 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:12.018 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:12.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:12.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:12.159 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:12.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:12.208 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:11 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:50:12.209 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:11 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:50:12.209 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:11 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:12.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:12.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 59 16 94230 94214 99.7561 65.6875 0.00538741 0.0100185 2024-09-21T11:50:12.302 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:12.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:12.303 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:12.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:12.459 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:12.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:12.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:12.463 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:50:12.662 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:50:12.662 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:50:12.662 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:50:12.663 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:50:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:50:12.663 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:50:12.663 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:50:12.664 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:50:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:50:12.664 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:50:12.664 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:50:12.665 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:12.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:50:12.700 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:50:12.701 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:50:12.735 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:12.737 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:12.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:12.738 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:12.745 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:50:12.745 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:50:12.839 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:12.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:12.884 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:12.885 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:12.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:13.031 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:13.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:13.032 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:13.043 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:12 smithi043 ceph-mon[36454]: osdmap e271: 8 total, 8 up, 8 in 2024-09-21T11:50:13.043 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:12 smithi043 ceph-mon[36454]: pgmap v492: 148 pgs: 148 active+clean; 5.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 82 KiB/s rd, 100 MiB/s wr, 1.74k op/s 2024-09-21T11:50:13.043 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:12 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4261628616' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:13.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:12 smithi043 ceph-mon[29710]: osdmap e271: 8 total, 8 up, 8 in 2024-09-21T11:50:13.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:12 smithi043 ceph-mon[29710]: pgmap v492: 148 pgs: 148 active+clean; 5.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 82 KiB/s rd, 100 MiB/s wr, 1.74k op/s 2024-09-21T11:50:13.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:12 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4261628616' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:13.120 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:13.134 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:13.192 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:13.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:13.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:13.257 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:12 smithi175 ceph-mon[32613]: osdmap e271: 8 total, 8 up, 8 in 2024-09-21T11:50:13.258 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:12 smithi175 ceph-mon[32613]: pgmap v492: 148 pgs: 148 active+clean; 5.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 82 KiB/s rd, 100 MiB/s wr, 1.74k op/s 2024-09-21T11:50:13.258 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:12 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4261628616' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:13.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:50:13.279570+0000 min lat: 0.00137184 max lat: 0.17025 avg lat: 0.0100914 2024-09-21T11:50:13.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:50:13.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 60 16 95144 95128 99.0458 57.125 0.0135199 0.0100914 2024-09-21T11:50:13.294 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:13.343 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:13.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:13.344 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:13.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:13.486 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:13.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:13.488 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:13.588 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:13.623 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:13.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:13.624 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:13.725 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:13.760 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:13.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:13.761 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:13.779 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:50:13.779 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:50:13.862 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:13.896 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:50:13.904 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:13.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:13.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:14.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:14.050 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:14.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:14.051 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:14.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:14.201 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:14.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:14.202 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2307890161' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:13 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2307890161' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:13 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2307890161' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:13 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:14.280 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 61 16 96227 96211 98.5318 67.6875 0.00809574 0.0101437 2024-09-21T11:50:14.304 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:14.333 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:14.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:14.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:14.436 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:14.470 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:14.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:14.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:14.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:14.582 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:50:14.615 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:14.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:14.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:14.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:14.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:14.754 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:14.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:14.855 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:14.897 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:14.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:14.898 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:14.915 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T11:50:14.989 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:14.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:15.038 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:15.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:15.040 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:15.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:15.168 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:14 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:14 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:50:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:14 smithi175 ceph-mon[32613]: osdmap e272: 8 total, 8 up, 8 in 2024-09-21T11:50:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:14 smithi175 ceph-mon[32613]: pgmap v494: 148 pgs: 148 active+clean; 5.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 72 KiB/s rd, 62 MiB/s wr, 1.12k op/s 2024-09-21T11:50:15.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/261761323' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:50:15.175 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:15.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:15.176 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:15.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:14 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:15.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:14 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:50:15.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:14 smithi043 ceph-mon[36454]: osdmap e272: 8 total, 8 up, 8 in 2024-09-21T11:50:15.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:14 smithi043 ceph-mon[36454]: pgmap v494: 148 pgs: 148 active+clean; 5.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 72 KiB/s rd, 62 MiB/s wr, 1.12k op/s 2024-09-21T11:50:15.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/261761323' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:50:15.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:14 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:15.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:14 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:50:15.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:14 smithi043 ceph-mon[29710]: osdmap e272: 8 total, 8 up, 8 in 2024-09-21T11:50:15.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:14 smithi043 ceph-mon[29710]: pgmap v494: 148 pgs: 148 active+clean; 5.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 72 KiB/s rd, 62 MiB/s wr, 1.12k op/s 2024-09-21T11:50:15.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/261761323' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:50:15.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:15.281 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 62 16 97378 97362 98.1029 71.9375 0.00719618 0.0101883 2024-09-21T11:50:15.314 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:15.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:15.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:15.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:15.447 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:15.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:15.448 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:15.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:50:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:50:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:50:15.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:15.586 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:15.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:15.587 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:15.627 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:50:15.627 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:50:15.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:50:15.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:15.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:15.734 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:15.834 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:15.865 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:15.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:15.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:15.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:16.004 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:16.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:16.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:16.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 7 1 2024-09-21T11:50:16.105 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:16.134 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:16.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:16.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:16.186 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:16.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:16.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/261761323' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:50:16.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:15 smithi043 ceph-mon[36454]: osdmap e273: 8 total, 8 up, 8 in 2024-09-21T11:50:16.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/261761323' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:50:16.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:15 smithi043 ceph-mon[29710]: osdmap e273: 8 total, 8 up, 8 in 2024-09-21T11:50:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/261761323' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:50:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:15 smithi175 ceph-mon[32613]: osdmap e273: 8 total, 8 up, 8 in 2024-09-21T11:50:16.262 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:16.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:16.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:16.281 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 63 16 98548 98532 97.7056 73.125 0.0133777 0.0102297 2024-09-21T11:50:16.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:16.396 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:16.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:16.397 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:16.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:16.552 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:16.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:16.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:16.653 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:16.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:50:16.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:16.706 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:16.707 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:16.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:16.849 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:16.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:16.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:16.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:50:16.934 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:50:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:50:16.951 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:16.976 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:50:16.976 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:50:16.992 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:16.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:16.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:16.996 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:17.014 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:50:17.014 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:50:17.049 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:50:17.049 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:50:17.077 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:17.087 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:50:17.088 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:50:17.093 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:17.132 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:17.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:17.133 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:17.134 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:50:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:50:17.170 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:50:17.171 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:50:17.209 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:50:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:50:17.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:17.281 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 64 16 99609 99593 97.2149 66.3125 0.0208937 0.010281 2024-09-21T11:50:17.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:17.287 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:17.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:17.288 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:17 smithi043 ceph-mon[36454]: pgmap v496: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 73 KiB/s rd, 73 MiB/s wr, 1.29k op/s 2024-09-21T11:50:17.288 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:17 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:17.288 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:17 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:17.288 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:17 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/713772729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:17.288 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:17 smithi043 ceph-mon[29710]: pgmap v496: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 73 KiB/s rd, 73 MiB/s wr, 1.29k op/s 2024-09-21T11:50:17.288 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:17 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:17.288 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:17 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:17.288 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/713772729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:17.389 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:17.400 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:17 smithi175 ceph-mon[32613]: pgmap v496: 148 pgs: 148 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 73 KiB/s rd, 73 MiB/s wr, 1.29k op/s 2024-09-21T11:50:17.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:17 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:17.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:17 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:17.401 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:17 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/713772729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:17.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:17.440 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:17.440 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:17.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:17.575 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:17.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:17.575 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:17.676 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:17.678 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:50:17.678 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:50:17.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.678 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:100060 9.f 9:f3175b66:::benchmark_data_smithi043_126002_object100059:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e273)", 2024-09-21T11:50:17.678 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:17.667642+0000", 2024-09-21T11:50:17.678 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0056011849999999998, 2024-09-21T11:50:17.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056331480000000001, 2024-09-21T11:50:17.678 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:17.678 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:17.678 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100060 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667642+0000", 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667642+0000", 2024-09-21T11:50:17.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667643+0000", 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3e-07 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667721+0000", 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8758999999999996e-05 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667731+0000", 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0003999999999999e-05 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667737+0000", 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8680000000000004e-06 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667792+0000", 2024-09-21T11:50:17.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4466999999999999e-05 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667910+0000", 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011830800000000001 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.668065+0000", 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015488899999999999 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.682 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:100062 9.7 9:e0443d0c:::benchmark_data_smithi043_126002_object100061:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e273)", 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:17.669255+0000", 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0039885449999999996, 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0041042229999999997, 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100062 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:17.683 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.669255+0000", 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.669255+0000", 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.669256+0000", 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.249e-06 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.669310+0000", 2024-09-21T11:50:17.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4070000000000002e-05 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.669320+0000", 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0293e-05 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.669327+0000", 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9600000000000003e-06 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:17.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671420+0000", 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020928600000000002 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671545+0000", 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124888 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671700+0000", 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015467700000000001 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.686 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25565.0:100061 9.b e273/247 9:dfbb155c:::benchmark_data_smithi043_126002_object100060:head v 273'6314, mlcod=273'6312)", 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:17.669970+0000", 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0032736179999999998, 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0034607449999999999, 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.669970+0000", 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.669970+0000", 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.669970+0000", 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.92e-07 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.670021+0000", 2024-09-21T11:50:17.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0501000000000003e-05 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.670030+0000", 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5589999999999994e-06 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.670036+0000", 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8619999999999999e-06 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.670098+0000", 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1396999999999995e-05 2024-09-21T11:50:17.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.670115+0000", 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7567999999999998e-05 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:100063 9.7 9.bc64ac57 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e273)", 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:17.671219+0000", 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0020237720000000001, 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022649189999999998, 2024-09-21T11:50:17.690 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "queued for pg", 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100063 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671219+0000", 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671219+0000", 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671221+0000", 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.212e-06 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671271+0000", 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0399000000000003e-05 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671279+0000", 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8150000000000006e-06 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671285+0000", 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3199999999999996e-06 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.693 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:100067 9.a 9:57e906d0:::benchmark_data_smithi043_126002_object100066:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e273)", 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:17.672887+0000", 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.00035666199999999999, 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00065743900000000001, 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100067 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.672887+0000", 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.672887+0000", 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.672888+0000", 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0380000000000001e-06 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.672963+0000", 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5631999999999997e-05 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.672987+0000", 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3989999999999999e-05 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.672999+0000", 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1609e-05 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.696 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.673073+0000", 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3947000000000002e-05 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.673181+0000", 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010848099999999999 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,7", 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.673330+0000", 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000148403 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:17.697 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.698 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25565.0:100060 9.f e273/247)", 2024-09-21T11:50:17.698 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:17.673024+0000", 2024-09-21T11:50:17.698 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.00021876600000000001, 2024-09-21T11:50:17.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057310900000000003, 2024-09-21T11:50:17.698 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:17.698 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:17.699 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "queued for pg", 2024-09-21T11:50:17.699 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:17.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:17.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.673024+0000", 2024-09-21T11:50:17.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:17.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.673024+0000", 2024-09-21T11:50:17.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.673025+0000", 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8400000000000005e-07 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.673028+0000", 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1559999999999999e-06 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:17.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.673035+0000", 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7780000000000002e-06 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.673040+0000", 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.95e-06 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:100055 9.a 9:57fb4b59:::benchmark_data_smithi043_126002_object100054:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e273)", 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:17.664655+0000", 2024-09-21T11:50:17.702 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0085881800000000008, 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0089893270000000001, 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100055 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.664655+0000", 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.664655+0000", 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.664656+0000", 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.347e-06 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:17.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.664712+0000", 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5485999999999997e-05 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.664725+0000", 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2714000000000001e-05 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.664732+0000", 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3320000000000003e-06 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:17.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.664781+0000", 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8859000000000001e-05 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.664891+0000", 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110246 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,7", 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.665028+0000", 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013745000000000001 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671365+0000", 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0063363409999999997 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671397+0000", 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1770999999999998e-05 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:100057 9.a 9:51ff3e6a:::benchmark_data_smithi043_126002_object100056:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e273)", 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:17.665843+0000", 2024-09-21T11:50:17.707 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0074000389999999997, 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0078779999999999996, 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100057 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.665843+0000", 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.665843+0000", 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.665844+0000", 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0079999999999999e-06 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.665915+0000", 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0654000000000005e-05 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.665924+0000", 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5640000000000002e-06 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.665938+0000", 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3274e-05 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.668226+0000", 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022884300000000002 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.668392+0000", 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016636400000000001 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,7", 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.668560+0000", 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000167076 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.671800+0000", 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0032409700000000001 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.672326+0000", 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052554000000000001 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:100059 9.6 9:6e3bab06:::benchmark_data_smithi043_126002_object100058:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e273)", 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:17.667392+0000", 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0058511020000000004, 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0064025159999999996, 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:50:17.712 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 100059 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667392+0000", 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667392+0000", 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:17.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667393+0000", 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2600000000000001e-07 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667457+0000", 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4008000000000001e-05 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:17.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667466+0000", 2024-09-21T11:50:17.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0569999999999999e-06 2024-09-21T11:50:17.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.715 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:17.715 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667472+0000", 2024-09-21T11:50:17.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1909999999999997e-06 2024-09-21T11:50:17.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:17.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.667509+0000", 2024-09-21T11:50:17.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6581000000000001e-05 2024-09-21T11:50:17.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.668254+0000", 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00074550599999999995 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.668417+0000", 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016304700000000001 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:17.672083+0000", 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036656739999999998 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.717 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:17.718 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:50:17.718 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 9 2024-09-21T11:50:17.718 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:50:17.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:17.720 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:17.720 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:17.820 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:17.848 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:17.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:17.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:17.950 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:17.981 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:17.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:17.981 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:18.082 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:18.118 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:18.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:18.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:18.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:18.259 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:18.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:18.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:18.281 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 65 16 100780 100764 96.8453 73.1875 0.0159721 0.0103209 2024-09-21T11:50:18.337 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T11:50:18.361 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:18.392 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:18.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:18.393 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:18.494 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:18.530 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:18.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:18.531 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:18.632 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:18.661 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:18.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:18.661 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:18.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:18 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2964074263' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:50:18.673 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:18 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:50:18.729 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:50:18.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:18 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2964074263' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:50:18.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:18 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:50:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:18 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2964074263' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:50:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:18 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:50:18.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:18.796 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:18.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:18.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:18.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:18.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:18.938 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:18.938 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:19.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:19.093 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:19.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:19.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:19.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:19.213 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:19.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:19.234 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:19.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 1 1 2024-09-21T11:50:19.281 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 66 16 102006 101990 96.539 76.625 0.00504041 0.010354 2024-09-21T11:50:19.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:19.378 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:19.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:19.420 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:19.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:19.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:19 smithi043 ceph-mon[29710]: pgmap v497: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 15 KiB/s rd, 70 MiB/s wr, 1.15k op/s 2024-09-21T11:50:19.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:19 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:50:19.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:19 smithi043 ceph-mon[29710]: osdmap e274: 8 total, 8 up, 8 in 2024-09-21T11:50:19.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:19 smithi043 ceph-mon[36454]: pgmap v497: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 15 KiB/s rd, 70 MiB/s wr, 1.15k op/s 2024-09-21T11:50:19.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:19 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:50:19.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:19 smithi043 ceph-mon[36454]: osdmap e274: 8 total, 8 up, 8 in 2024-09-21T11:50:19.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:19.558 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:19.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:19.559 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:19.637 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:19 smithi175 ceph-mon[32613]: pgmap v497: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 15 KiB/s rd, 70 MiB/s wr, 1.15k op/s 2024-09-21T11:50:19.638 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:19 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:50:19.638 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:19 smithi175 ceph-mon[32613]: osdmap e274: 8 total, 8 up, 8 in 2024-09-21T11:50:19.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:19.705 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:19.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:19.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:19.789 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:50:19.789 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:50:19.789 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:50:19.789 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:50:19.790 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:50:19.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:19.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:19.836 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:19.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:19.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:19.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:20.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:20.113 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:20.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:20.135 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:20.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:20.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:20.281 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:20.281 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 67 16 103066 103050 96.0867 66.25 0.0109115 0.0104016 2024-09-21T11:50:20.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:20.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:20.383 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:20.417 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:20.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:20.418 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:20.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:50:20.518 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:20.550 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:20.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:20.551 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:20.597 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:50:20.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:20.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:20.724 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:20.724 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:20.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:20.861 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:20.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:20.862 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:20.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:20.996 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:20.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:20.997 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:21.007 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:21.098 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:21.127 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:21.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:21.128 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:21.228 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:21.270 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:21.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:21.271 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:21.282 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 68 16 104182 104166 95.6995 69.75 0.0112769 0.0104447 2024-09-21T11:50:21.354 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:50:21.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:21.401 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T11:50:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:21.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:21.420 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:21.439 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:21 smithi175 ceph-mon[32613]: pgmap v499: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 842 B/s rd, 55 MiB/s wr, 885 op/s 2024-09-21T11:50:21.439 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:21 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1649366755' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:21.439 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:21 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:21.439 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:21 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2857124672' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:50:21.439 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:50:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:50:21.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:21.560 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:21.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:21.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:21.600 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:50:21.600 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:50:21.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:50:21.600 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:50:21.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.600 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43896 8.3 8:c3bd4fd6:::benchmark_data_smithi043_97042_object43895:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:50:21.600 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.982177+0000", 2024-09-21T11:50:21.600 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 300.61498929999999, 2024-09-21T11:50:21.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14573064599999999, 2024-09-21T11:50:21.600 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43896 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982177+0000", 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982177+0000", 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982178+0000", 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5600000000000005e-07 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982209+0000", 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0809000000000003e-05 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982218+0000", 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0089999999999996e-06 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.982224+0000", 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5350000000000001e-06 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.984992+0000", 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027677230000000001 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.985115+0000", 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012332000000000001 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.985242+0000", 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012655500000000001 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989565+0000", 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043232679999999999 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992180+0000", 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002614487 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127774+0000", 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13559412700000001 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127839+0000", 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4784999999999997e-05 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127908+0000", 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9271999999999997e-05 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43898 8.3 8:cb8873a2:::benchmark_data_smithi043_97042_object43897:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.986596+0000", 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 300.61057043300002, 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.1414368, 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.606 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43898 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986596+0000", 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986596+0000", 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986598+0000", 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.844e-06 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986632+0000", 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.413e-05 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986644+0000", 2024-09-21T11:50:21.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1443e-05 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986653+0000", 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5550000000000008e-06 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987082+0000", 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042925400000000001 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987555+0000", 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047269100000000002 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987683+0000", 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000128329 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992217+0000", 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045337889999999999 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992269+0000", 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1941999999999998e-05 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127950+0000", 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13568037199999999 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.127999+0000", 2024-09-21T11:50:21.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9444999999999998e-05 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.128033+0000", 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4005999999999999e-05 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43901 8.3 8:c3170c6a:::benchmark_data_smithi043_97042_object43900:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.986960+0000", 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 300.610206167, 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.144105175, 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.612 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.613 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:50:21.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:21.613 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43901 2024-09-21T11:50:21.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.613 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986960+0000", 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986960+0000", 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986961+0000", 2024-09-21T11:50:21.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1800000000000004e-07 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987010+0000", 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8708999999999997e-05 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987021+0000", 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0548e-05 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987028+0000", 2024-09-21T11:50:21.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9949999999999999e-06 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989603+0000", 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025754509999999999 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989731+0000", 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012751 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989866+0000", 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000135105 2024-09-21T11:50:21.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992250+0000", 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023844780000000002 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.992368+0000", 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117924 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.130962+0000", 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13859406799999999 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.131025+0000", 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2992999999999998e-05 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.131066+0000", 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0475999999999997e-05 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.618 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66127 8.4 8:2a1dd9a8:::benchmark_data_smithi043_97042_object66126:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.916244+0000", 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 285.68092255599998, 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13852240499999999, 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66127 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916244+0000", 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916244+0000", 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916246+0000", 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.542e-06 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916321+0000", 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5754999999999996e-05 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916332+0000", 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0617e-05 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.916339+0000", 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9040000000000003e-06 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.917528+0000", 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001189486 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.917639+0000", 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110455 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.917760+0000", 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012136 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920338+0000", 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002577916 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048623+0000", 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12828446499999999 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.054693+0000", 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0060701349999999999 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.054727+0000", 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4706000000000003e-05 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.054766+0000", 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9063999999999997e-05 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66130 8.4 8:216cc0c6:::benchmark_data_smithi043_97042_object66129:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.919278+0000", 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 285.67788899700003, 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.138675363, 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66130 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919278+0000", 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919278+0000", 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919279+0000", 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8700000000000004e-07 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919324+0000", 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5812000000000001e-05 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919334+0000", 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9499999999999996e-06 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919341+0000", 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3969999999999999e-06 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920363+0000", 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010227229999999999 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.045883+0000", 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12551943400000001 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.046054+0000", 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017078800000000001 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.051620+0000", 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005566718 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.054778+0000", 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00315745 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.057852+0000", 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030744449999999999 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.057909+0000", 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6851000000000002e-05 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.057953+0000", 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3908e-05 2024-09-21T11:50:21.630 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66136 8.4 8:2cd25696:::benchmark_data_smithi043_97042_object66135:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.920996+0000", 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 285.67617091699998, 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.137070152, 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:50:21.631 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66136 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920996+0000", 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920996+0000", 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.920997+0000", 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6500000000000008e-07 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.921088+0000", 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1389000000000002e-05 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.045913+0000", 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.124825011 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.045924+0000", 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1165000000000001e-05 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048778+0000", 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002853707 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048896+0000", 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011762800000000001 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.049029+0000", 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013383399999999999 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.051658+0000", 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002628306 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.054782+0000", 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0031247150000000001 2024-09-21T11:50:21.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.057989+0000", 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0032060830000000002 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.058034+0000", 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5878999999999999e-05 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.058066+0000", 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1470000000000002e-05 2024-09-21T11:50:21.636 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:213539 8.8 8:1142105d:::benchmark_data_smithi043_97042_object76445:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:47:43.586798+0000", 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 158.0103684, 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14603690899999999, 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:50:21.637 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 213539 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.586798+0000", 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.586798+0000", 2024-09-21T11:50:21.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.586800+0000", 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8789999999999999e-06 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.586806+0000", 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9039999999999997e-06 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.586817+0000", 2024-09-21T11:50:21.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1134e-05 2024-09-21T11:50:21.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.586824+0000", 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9670000000000004e-06 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.586876+0000", 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1731999999999999e-05 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.586985+0000", 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010896900000000001 2024-09-21T11:50:21.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.587085+0000", 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9720000000000001e-05 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.592369+0000", 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0052841770000000001 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.592457+0000", 2024-09-21T11:50:21.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8739999999999994e-05 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.732714+0000", 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14025663099999999 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.732790+0000", 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5662999999999997e-05 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:47:43.732835+0000", 2024-09-21T11:50:21.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5392999999999997e-05 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:33296 9.a 9:519cf60a:::benchmark_data_smithi043_126002_object33295:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.236391+0000", 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 54.360775932999999, 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13819759600000001, 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.644 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 33296 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.236391+0000", 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.236391+0000", 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.236392+0000", 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4100000000000001e-06 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.236429+0000", 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7160000000000003e-05 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.236440+0000", 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0562e-05 2024-09-21T11:50:21.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.236448+0000", 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1459999999999996e-06 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.236533+0000", 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4970999999999997e-05 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.236653+0000", 2024-09-21T11:50:21.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011986500000000001 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,7", 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.236790+0000", 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000137224 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238504+0000", 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017137400000000001 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.241370+0000", 2024-09-21T11:50:21.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028664889999999998 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.374487+0000", 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13311706600000001 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.374540+0000", 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3013000000000002e-05 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.374588+0000", 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.795e-05 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:33303 9.f 9:fe46cbe6:::benchmark_data_smithi043_126002_object33302:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.238656+0000", 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 54.358510271999997, 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13809898700000001, 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.650 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 33303 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238656+0000", 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238656+0000", 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238658+0000", 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.308e-06 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238738+0000", 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9921999999999998e-05 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238748+0000", 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0197000000000001e-05 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238755+0000", 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7469999999999999e-06 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238801+0000", 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6251000000000002e-05 2024-09-21T11:50:21.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238911+0000", 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110361 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.239040+0000", 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012851599999999999 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.240868+0000", 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018286999999999999 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.374479+0000", 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.133610645 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.376659+0000", 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002180249 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.376714+0000", 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5201999999999998e-05 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.376755+0000", 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0889000000000001e-05 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34365 9.f 9:f4fad2b0:::benchmark_data_smithi043_126002_object34364:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.656 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.902623+0000", 2024-09-21T11:50:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 53.694543488999997, 2024-09-21T11:50:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14905105799999999, 2024-09-21T11:50:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.657 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34365 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.902623+0000", 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.902623+0000", 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.902625+0000", 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.443e-06 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.902652+0000", 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7222000000000001e-05 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.902660+0000", 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3659999999999992e-06 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.902666+0000", 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8749999999999997e-06 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.903050+0000", 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00038444800000000002 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.903410+0000", 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035997900000000002 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.903543+0000", 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013248900000000001 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.905222+0000", 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016787869999999999 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.908770+0000", 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035484119999999999 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.051605+0000", 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14283506500000001 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.051642+0000", 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6841999999999997e-05 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.051674+0000", 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2129999999999999e-05 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34372 9.7 9:e4c6b68c:::benchmark_data_smithi043_126002_object34371:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.906042+0000", 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 53.691124422000001, 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15524845100000001, 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.663 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34372 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906042+0000", 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906042+0000", 2024-09-21T11:50:21.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906044+0000", 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7400000000000001e-06 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906079+0000", 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4907999999999998e-05 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906095+0000", 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6444999999999999e-05 2024-09-21T11:50:21.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906103+0000", 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4229999999999999e-06 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906401+0000", 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029829999999999999 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906530+0000", 2024-09-21T11:50:21.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012938000000000001 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906664+0000", 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013341699999999999 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.910227+0000", 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035628650000000001 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911684+0000", 2024-09-21T11:50:21.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014569489999999999 2024-09-21T11:50:21.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061207+0000", 2024-09-21T11:50:21.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14952306700000001 2024-09-21T11:50:21.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061260+0000", 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3653999999999998e-05 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061291+0000", 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0303000000000001e-05 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34374 9.7 9:e9ec9b10:::benchmark_data_smithi043_126002_object34373:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.669 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.906121+0000", 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 53.691046040000003, 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15530081300000001, 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34374 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906121+0000", 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906121+0000", 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906121+0000", 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7100000000000002e-07 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906143+0000", 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1503999999999999e-05 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906147+0000", 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5800000000000002e-06 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.906151+0000", 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6150000000000001e-06 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.908433+0000", 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00228261 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.908557+0000", 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012380400000000001 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.908689+0000", 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000131743 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911731+0000", 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030423970000000001 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.913837+0000", 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021052150000000001 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061331+0000", 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14749439 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.674 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061379+0000", 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8355e-05 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061421+0000", 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2129000000000003e-05 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34378 9.7 9:e9b27f0f:::benchmark_data_smithi043_126002_object34377:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.675 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.908281+0000", 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 53.688885435000003, 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15326944300000001, 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34378 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.676 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.908281+0000", 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.908281+0000", 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.677 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.908283+0000", 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.542e-06 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.908316+0000", 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3101e-05 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.908328+0000", 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1789e-05 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.678 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.908335+0000", 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7340000000000008e-06 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.910256+0000", 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001921023 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.910393+0000", 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013684999999999999 2024-09-21T11:50:21.679 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.910527+0000", 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133948 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.913875+0000", 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003347802 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.680 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.913944+0000", 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8907999999999998e-05 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061470+0000", 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14752568499999999 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061515+0000", 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5695000000000002e-05 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.681 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.682 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.682 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061551+0000", 2024-09-21T11:50:21.682 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:21.683 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5366e-05 2024-09-21T11:50:21.683 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.683 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.683 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.683 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34381 9.7 9:e82301a8:::benchmark_data_smithi043_126002_object34380:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.910253+0000", 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 53.686913545000003, 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15140720799999999, 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34381 2024-09-21T11:50:21.684 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.910253+0000", 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.910253+0000", 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.685 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.910254+0000", 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3969999999999999e-06 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.910341+0000", 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6462e-05 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.910354+0000", 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2591e-05 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.686 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.910361+0000", 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8800000000000008e-06 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911766+0000", 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001404867 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911852+0000", 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5463000000000005e-05 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:21.687 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911994+0000", 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142705 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.913920+0000", 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001925822 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.913948+0000", 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8212999999999999e-05 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061590+0000", 2024-09-21T11:50:21.688 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14764129600000001 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061631+0000", 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1425999999999999e-05 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.061660+0000", 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9085999999999999e-05 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.689 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34382 9.7 9:e3f8dece:::benchmark_data_smithi043_126002_object34381:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.911096+0000", 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 53.686070878000002, 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15595714999999999, 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34382 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.690 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911096+0000", 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911096+0000", 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911097+0000", 2024-09-21T11:50:21.691 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9199999999999999e-07 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911165+0000", 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8218000000000003e-05 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911175+0000", 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8579999999999995e-06 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.911180+0000", 2024-09-21T11:50:21.692 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2669999999999997e-06 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.912902+0000", 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00172167 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.913037+0000", 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013495 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.693 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.913172+0000", 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000135723 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.913955+0000", 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00078258199999999998 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.914160+0000", 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020526999999999999 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.694 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.066940+0000", 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15278029100000001 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.067018+0000", 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7902000000000001e-05 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.067053+0000", 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4527000000000003e-05 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.695 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34888 9.7 9:e39c71e1:::benchmark_data_smithi043_126002_object34887:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:28.283231+0000", 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 53.313935477999998, 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15870710099999999, 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34888 2024-09-21T11:50:21.696 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.283231+0000", 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.283231+0000", 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.283232+0000", 2024-09-21T11:50:21.697 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.114e-06 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.283272+0000", 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0136000000000003e-05 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.283283+0000", 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1048e-05 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.698 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.283292+0000", 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.283419+0000", 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000126785 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.283554+0000", 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000135434 2024-09-21T11:50:21.699 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.283705+0000", 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015075200000000001 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.291305+0000", 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0076004310000000004 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.700 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.291320+0000", 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4688999999999999e-05 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.441830+0000", 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15051009300000001 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.441886+0000", 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6443000000000002e-05 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.701 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.441938+0000", 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1845000000000003e-05 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34892 9.7 9:e59c7ff4:::benchmark_data_smithi043_126002_object34891:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:28.285535+0000", 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 53.311631781000003, 2024-09-21T11:50:21.702 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.16313436100000001, 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34892 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.285535+0000", 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.703 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.285535+0000", 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.285535+0000", 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0299999999999999e-07 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.286258+0000", 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072206699999999996 2024-09-21T11:50:21.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.286270+0000", 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2046e-05 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.286279+0000", 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4469999999999995e-06 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.286878+0000", 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00059889199999999996 2024-09-21T11:50:21.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287573+0000", 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00069488099999999995 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287693+0000", 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000120318 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.291324+0000", 2024-09-21T11:50:21.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003631118 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.291869+0000", 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00054515900000000003 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.448580+0000", 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.156710878 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.448627+0000", 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6350999999999997e-05 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.448669+0000", 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2601000000000001e-05 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34894 9.7 9:e3fb6252:::benchmark_data_smithi043_126002_object34893:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:28.286307+0000", 2024-09-21T11:50:21.708 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 53.310860011999999, 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.16860682499999999, 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34894 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.286307+0000", 2024-09-21T11:50:21.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.286307+0000", 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.286307+0000", 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8700000000000005e-07 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.286347+0000", 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0043999999999998e-05 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.286354+0000", 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8589999999999999e-06 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.286360+0000", 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7010000000000001e-06 2024-09-21T11:50:21.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.289526+0000", 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003166079 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.289647+0000", 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012053900000000001 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:21.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.289779+0000", 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013239600000000001 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.291732+0000", 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019526579999999999 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292047+0000", 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000315371 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.454816+0000", 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.16276892500000001 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.454873+0000", 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7256000000000002e-05 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.454913+0000", 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0210000000000003e-05 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.714 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:39499 9.7 9:e0ea1f65:::benchmark_data_smithi043_126002_object39498:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e255)", 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:31.199092+0000", 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.398075020999997, 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.144981952, 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39499 2024-09-21T11:50:21.715 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199092+0000", 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199092+0000", 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.716 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199093+0000", 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1599999999999999e-06 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199154+0000", 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1262999999999999e-05 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199163+0000", 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3689999999999996e-06 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.717 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199170+0000", 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9689999999999997e-06 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199218+0000", 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7759000000000001e-05 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199334+0000", 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011579200000000001 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.718 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199461+0000", 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012742499999999999 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.201444+0000", 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019823449999999999 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.202955+0000", 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015108579999999999 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.719 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.344004+0000", 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.141049182 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.344040+0000", 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6319999999999998e-05 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.344074+0000", 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3510000000000003e-05 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.720 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:39504 9.f 9:f1317196:::benchmark_data_smithi043_126002_object39503:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e255)", 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:31.199762+0000", 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 50.397404270999999, 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14432993699999999, 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39504 2024-09-21T11:50:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199762+0000", 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199762+0000", 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199763+0000", 2024-09-21T11:50:21.722 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4199999999999995e-07 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199977+0000", 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021384799999999999 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199989+0000", 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1931e-05 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.199996+0000", 2024-09-21T11:50:21.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0249999999999997e-06 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.200042+0000", 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6520000000000002e-05 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.200170+0000", 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012803900000000001 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:21.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.200306+0000", 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013551399999999999 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.203265+0000", 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0029593760000000001 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.205674+0000", 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024087739999999998 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:21.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.344024+0000", 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.138349532 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.344054+0000", 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0196e-05 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.344092+0000", 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8439999999999998e-05 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.726 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.727 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:21.727 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:21.727 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:50:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:21 smithi043 ceph-mon[29710]: pgmap v499: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 842 B/s rd, 55 MiB/s wr, 885 op/s 2024-09-21T11:50:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1649366755' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:21 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2857124672' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:50:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:21 smithi043 ceph-mon[36454]: pgmap v499: 148 pgs: 148 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 842 B/s rd, 55 MiB/s wr, 885 op/s 2024-09-21T11:50:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:21 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1649366755' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:21 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:21 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2857124672' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:50:21.766 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:21.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:21.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:21.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:21.904 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:21.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:21.905 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:22.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:22.045 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:22.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:22.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:22.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:22.189 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:22.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:22.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:22.282 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 69 16 105435 105419 95.4475 78.3125 0.00939766 0.0104725 2024-09-21T11:50:22.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:22.316 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:22.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:22.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:22.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:22.457 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:22.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:22.458 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:22.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:22.597 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:22.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:22.598 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:22.699 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:22.738 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:22.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:22.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:22 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:22 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:50:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2857124672' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:50:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:22 smithi043 ceph-mon[29710]: osdmap e275: 8 total, 8 up, 8 in 2024-09-21T11:50:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:22 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:22 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:50:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2857124672' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:50:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:22 smithi043 ceph-mon[36454]: osdmap e275: 8 total, 8 up, 8 in 2024-09-21T11:50:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:22 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:22 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:50:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2857124672' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:50:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:22 smithi175 ceph-mon[32613]: osdmap e275: 8 total, 8 up, 8 in 2024-09-21T11:50:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:22.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:22.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:22.872 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:22.873 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:22.904 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T11:50:22.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:50:22.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:23.034 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:23.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:23.035 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:23.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:23.168 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:23.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:23.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:23.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:50:23.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:23.282 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 70 16 106587 106571 95.1124 72 0.0107115 0.0105094 2024-09-21T11:50:23.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:23.314 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:23.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:23.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:23.454 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:23.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:23.455 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:23 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:50:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:23 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:50:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:23 smithi043 ceph-mon[36454]: pgmap v501: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-21T11:50:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:23 smithi043 ceph-mon[36454]: osdmap e276: 8 total, 8 up, 8 in 2024-09-21T11:50:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:23 smithi043 ceph-mon[36454]: 9.6 scrub starts 2024-09-21T11:50:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:23 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:50:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:23 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:50:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:23 smithi043 ceph-mon[29710]: pgmap v501: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-21T11:50:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:23 smithi043 ceph-mon[29710]: osdmap e276: 8 total, 8 up, 8 in 2024-09-21T11:50:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:23 smithi043 ceph-mon[29710]: 9.6 scrub starts 2024-09-21T11:50:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:23 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:50:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:23 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:50:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:23 smithi175 ceph-mon[32613]: pgmap v501: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-21T11:50:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:23 smithi175 ceph-mon[32613]: osdmap e276: 8 total, 8 up, 8 in 2024-09-21T11:50:23.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:23 smithi175 ceph-mon[32613]: 9.6 scrub starts 2024-09-21T11:50:23.556 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:23.600 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:23.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:23.601 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:23.658 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:23.673 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:23.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:23.718 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:23.731 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:23.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:23.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:23.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:23.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:23.887 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:23.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:24.031 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:24.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:24.032 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:24.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:24.182 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:24.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:24.183 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:24.274 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25565.0:107612 9.b e276/247)", 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:24.271880+0000", 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.000213179, 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00023644700000000001, 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "queued for pg", 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:24.271880+0000", 2024-09-21T11:50:24.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:24.271880+0000", 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:24.271882+0000", 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.486e-06 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:24.271927+0000", 2024-09-21T11:50:24.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5296000000000001e-05 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:24.271951+0000", 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3867e-05 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:24.271960+0000", 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2e-06 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:24.277 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:50:24.278 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T11:50:24.278 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:50:24.282 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 71 15 107621 107606 94.684 64.6875 0.0134726 0.010556 2024-09-21T11:50:24.283 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:24.332 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:24.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:24.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:24.434 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:24.467 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:24.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:24.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:24.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:24.613 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:24.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:24.615 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:24.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:24.742 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:50:24.743 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:50:24.772 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:24.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:24.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:24.788 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:50:24.789 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:50:24.835 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:50:24.835 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:50:24.874 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:50:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:50:24.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:24.920 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:50:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:50:24.923 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:24.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:24.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:24.970 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:50:24.971 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:50:25.019 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:50:25.019 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:50:25.026 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:25.052 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:25.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:25.053 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:25.060 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:50:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:50:25.105 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:50:25.154 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:25.189 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:25.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:25.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:25.282 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 72 16 108554 108538 94.178 58.25 0.0154405 0.0106131 2024-09-21T11:50:25.291 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:25.322 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:25.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:25.323 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:25.352 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:50:25.356 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:50:25.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:25 smithi175 ceph-mon[32613]: 9.9 scrub starts 2024-09-21T11:50:25.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:25 smithi175 ceph-mon[32613]: pgmap v503: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 59 MiB/s wr, 944 op/s 2024-09-21T11:50:25.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:25 smithi175 ceph-mon[32613]: 4.a scrub starts 2024-09-21T11:50:25.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:25 smithi175 ceph-mon[32613]: 9.8 scrub starts 2024-09-21T11:50:25.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:25 smithi175 ceph-mon[32613]: 4.a scrub ok 2024-09-21T11:50:25.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:25 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3820908489' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:25.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:25 smithi175 ceph-mon[32613]: 9.6 scrub ok 2024-09-21T11:50:25.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:25 smithi175 ceph-mon[32613]: 6.1d scrub starts 2024-09-21T11:50:25.378 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:25 smithi175 ceph-mon[32613]: 6.1d scrub ok 2024-09-21T11:50:25.424 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:25.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:25.476 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:25.477 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:50:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:50:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[36454]: 9.9 scrub starts 2024-09-21T11:50:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[36454]: pgmap v503: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 59 MiB/s wr, 944 op/s 2024-09-21T11:50:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[36454]: 4.a scrub starts 2024-09-21T11:50:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[36454]: 9.8 scrub starts 2024-09-21T11:50:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[36454]: 4.a scrub ok 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3820908489' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[36454]: 9.6 scrub ok 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[36454]: 6.1d scrub starts 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[36454]: 6.1d scrub ok 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[29710]: 9.9 scrub starts 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[29710]: pgmap v503: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 59 MiB/s wr, 944 op/s 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[29710]: 4.a scrub starts 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[29710]: 9.8 scrub starts 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[29710]: 4.a scrub ok 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3820908489' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[29710]: 9.6 scrub ok 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[29710]: 6.1d scrub starts 2024-09-21T11:50:25.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:25 smithi043 ceph-mon[29710]: 6.1d scrub ok 2024-09-21T11:50:25.496 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:25.578 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:25.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:50:25.614 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:25.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:25.615 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:25.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:25.793 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:25.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:25.794 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:25.895 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:25.927 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:25.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:25.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:26.028 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:26.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:26.064 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:26.065 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:26.152 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-09-21T11:50:23.969671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6836460059,"num_objects":104506,"num_object_clones":0,"num_object_copies":313518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104506,"num_whiteouts":0,"num_read":1877,"num_read_kb":1734,"num_write":105142,"num_write_kb":6677321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":596864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83652,"ondisk_log_size":83652,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":23054916,"kb_used_data":19913588,"kb_used_omap":12,"kb_used_meta":3141171,"kb_avail":726906300,"statfs":{"total":767960285184,"available":744352051200,"internally_reserved":0,"allocated":20391514112,"data_stored":20388561431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":3216559666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,11,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":371326976,"num_objects":5666,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5666,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5666,"num_write_kb":362624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002684"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":464,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.364397+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:50:21.364397+0000","last_peered":"2024-09-21T11:50:21.364397+0000","last_clean":"2024-09-21T11:50:21.364397+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:50:21.364397+0000","last_undegraded":"2024-09-21T11:50:21.364397+0000","last_fullsized":"2024-09-21T11:50:21.364397+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.789381+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":533,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.330224+0000","last_change":"2024-09-21T11:49:59.177549+0000","last_active":"2024-09-21T11:50:18.330224+0000","last_peered":"2024-09-21T11:50:18.330224+0000","last_clean":"2024-09-21T11:50:18.330224+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:50:18.330224+0000","last_undegraded":"2024-09-21T11:50:18.330224+0000","last_fullsized":"2024-09-21T11:50:18.330224+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:49:59.177497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:59.177497+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:01.997293+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":155,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.336920+0000","last_change":"2024-09-21T11:47:03.372676+0000","last_active":"2024-09-21T11:50:18.336920+0000","last_peered":"2024-09-21T11:50:18.336920+0000","last_clean":"2024-09-21T11:50:18.336920+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:50:18.336920+0000","last_undegraded":"2024-09-21T11:50:18.336920+0000","last_fullsized":"2024-09-21T11:50:18.336920+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.778217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":480,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.330953+0000","last_change":"2024-09-21T11:49:08.053201+0000","last_active":"2024-09-21T11:50:18.330953+0000","last_peered":"2024-09-21T11:50:18.330953+0000","last_clean":"2024-09-21T11:50:18.330953+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:50:18.330953+0000","last_undegraded":"2024-09-21T11:50:18.330953+0000","last_fullsized":"2024-09-21T11:50:18.330953+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.733987+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":460,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.354062+0000","last_change":"2024-09-21T11:49:09.143672+0000","last_active":"2024-09-21T11:50:21.354062+0000","last_peered":"2024-09-21T11:50:21.354062+0000","last_clean":"2024-09-21T11:50:21.354062+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:50:21.354062+0000","last_undegraded":"2024-09-21T11:50:21.354062+0000","last_fullsized":"2024-09-21T11:50:21.354062+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:09.143598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:09.143598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":457,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351705+0000","last_change":"2024-09-21T11:46:55.804484+0000","last_active":"2024-09-21T11:50:21.351705+0000","last_peered":"2024-09-21T11:50:21.351705+0000","last_clean":"2024-09-21T11:50:21.351705+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:50:21.351705+0000","last_undegraded":"2024-09-21T11:50:21.351705+0000","last_fullsized":"2024-09-21T11:50:21.351705+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.804419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.804419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:43.180856+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":503,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.353861+0000","last_change":"2024-09-21T11:50:03.300848+0000","last_active":"2024-09-21T11:50:21.353861+0000","last_peered":"2024-09-21T11:50:21.353861+0000","last_clean":"2024-09-21T11:50:21.353861+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:50:21.353861+0000","last_undegraded":"2024-09-21T11:50:21.353861+0000","last_fullsized":"2024-09-21T11:50:21.353861+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:50:03.300772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:03.300772+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.852364+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":476,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.330139+0000","last_change":"2024-09-21T11:47:03.371710+0000","last_active":"2024-09-21T11:50:18.330139+0000","last_peered":"2024-09-21T11:50:18.330139+0000","last_clean":"2024-09-21T11:50:18.330139+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:50:18.330139+0000","last_undegraded":"2024-09-21T11:50:18.330139+0000","last_fullsized":"2024-09-21T11:50:18.330139+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:29.162811+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":461,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.346298+0000","last_change":"2024-09-21T11:48:47.077320+0000","last_active":"2024-09-21T11:50:18.346298+0000","last_peered":"2024-09-21T11:50:18.346298+0000","last_clean":"2024-09-21T11:50:18.346298+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:50:18.346298+0000","last_undegraded":"2024-09-21T11:50:18.346298+0000","last_fullsized":"2024-09-21T11:50:18.346298+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.077259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.077259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:13.765851+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":499,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.331211+0000","last_change":"2024-09-21T11:49:15.028297+0000","last_active":"2024-09-21T11:50:18.331211+0000","last_peered":"2024-09-21T11:50:18.331211+0000","last_clean":"2024-09-21T11:50:18.331211+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:50:18.331211+0000","last_undegraded":"2024-09-21T11:50:18.331211+0000","last_fullsized":"2024-09-21T11:50:18.331211+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:49:15.028208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:15.028208+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.470742+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":461,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344136+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:50:18.344136+0000","last_peered":"2024-09-21T11:50:18.344136+0000","last_clean":"2024-09-21T11:50:18.344136+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:50:18.344136+0000","last_undegraded":"2024-09-21T11:50:18.344136+0000","last_fullsized":"2024-09-21T11:50:18.344136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.828789+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":473,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.363334+0000","last_change":"2024-09-21T11:48:44.523906+0000","last_active":"2024-09-21T11:50:21.363334+0000","last_peered":"2024-09-21T11:50:21.363334+0000","last_clean":"2024-09-21T11:50:21.363334+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:50:21.363334+0000","last_undegraded":"2024-09-21T11:50:21.363334+0000","last_fullsized":"2024-09-21T11:50:21.363334+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:44.523826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.523826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:11.691589+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":483,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344005+0000","last_change":"2024-09-21T11:47:03.385245+0000","last_active":"2024-09-21T11:50:18.344005+0000","last_peered":"2024-09-21T11:50:18.344005+0000","last_clean":"2024-09-21T11:50:18.344005+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:50:18.344005+0000","last_undegraded":"2024-09-21T11:50:18.344005+0000","last_fullsized":"2024-09-21T11:50:18.344005+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:25.947771+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":513,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.331560+0000","last_change":"2024-09-21T11:49:14.010678+0000","last_active":"2024-09-21T11:50:18.331560+0000","last_peered":"2024-09-21T11:50:18.331560+0000","last_clean":"2024-09-21T11:50:18.331560+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:50:18.331560+0000","last_undegraded":"2024-09-21T11:50:18.331560+0000","last_fullsized":"2024-09-21T11:50:18.331560+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.823398+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":475,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341152+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:50:18.341152+0000","last_peered":"2024-09-21T11:50:18.341152+0000","last_clean":"2024-09-21T11:50:18.341152+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:50:18.341152+0000","last_undegraded":"2024-09-21T11:50:18.341152+0000","last_fullsized":"2024-09-21T11:50:18.341152+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:45.139772+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":471,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344467+0000","last_change":"2024-09-21T11:47:03.380856+0000","last_active":"2024-09-21T11:50:18.344467+0000","last_peered":"2024-09-21T11:50:18.344467+0000","last_clean":"2024-09-21T11:50:18.344467+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:50:18.344467+0000","last_undegraded":"2024-09-21T11:50:18.344467+0000","last_fullsized":"2024-09-21T11:50:18.344467+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:13.288522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:13.288522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:34.960084+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":467,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.332310+0000","last_change":"2024-09-21T11:46:52.035524+0000","last_active":"2024-09-21T11:50:18.332310+0000","last_peered":"2024-09-21T11:50:18.332310+0000","last_clean":"2024-09-21T11:50:18.332310+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:50:18.332310+0000","last_undegraded":"2024-09-21T11:50:18.332310+0000","last_fullsized":"2024-09-21T11:50:18.332310+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.035461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.035461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:31.176135+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":455,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.364297+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:50:21.364297+0000","last_peered":"2024-09-21T11:50:21.364297+0000","last_clean":"2024-09-21T11:50:21.364297+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:50:21.364297+0000","last_undegraded":"2024-09-21T11:50:21.364297+0000","last_fullsized":"2024-09-21T11:50:21.364297+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:31.706363+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":588,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.355946+0000","last_change":"2024-09-21T11:48:18.120641+0000","last_active":"2024-09-21T11:50:21.355946+0000","last_peered":"2024-09-21T11:50:21.355946+0000","last_clean":"2024-09-21T11:50:21.355946+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:50:21.355946+0000","last_undegraded":"2024-09-21T11:50:21.355946+0000","last_fullsized":"2024-09-21T11:50:21.355946+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:48:18.120532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:18.120532+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:25.018806+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":163,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344546+0000","last_change":"2024-09-21T11:49:16.745001+0000","last_active":"2024-09-21T11:50:18.344546+0000","last_peered":"2024-09-21T11:50:18.344546+0000","last_clean":"2024-09-21T11:50:18.344546+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:50:18.344546+0000","last_undegraded":"2024-09-21T11:50:18.344546+0000","last_fullsized":"2024-09-21T11:50:18.344546+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:16.744931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:16.744931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.501502+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":460,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351940+0000","last_change":"2024-09-21T11:50:06.400378+0000","last_active":"2024-09-21T11:50:21.351940+0000","last_peered":"2024-09-21T11:50:21.351940+0000","last_clean":"2024-09-21T11:50:21.351940+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:50:21.351940+0000","last_undegraded":"2024-09-21T11:50:21.351940+0000","last_fullsized":"2024-09-21T11:50:21.351940+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:06.400307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.400307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.097684+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":484,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.331342+0000","last_change":"2024-09-21T11:49:50.122297+0000","last_active":"2024-09-21T11:50:18.331342+0000","last_peered":"2024-09-21T11:50:18.331342+0000","last_clean":"2024-09-21T11:50:18.331342+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:50:18.331342+0000","last_undegraded":"2024-09-21T11:50:18.331342+0000","last_fullsized":"2024-09-21T11:50:18.331342+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.122227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.122227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:55.903276+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":530,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351472+0000","last_change":"2024-09-21T11:47:03.873307+0000","last_active":"2024-09-21T11:50:21.351472+0000","last_peered":"2024-09-21T11:50:21.351472+0000","last_clean":"2024-09-21T11:50:21.351472+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:50:21.351472+0000","last_undegraded":"2024-09-21T11:50:21.351472+0000","last_fullsized":"2024-09-21T11:50:21.351472+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:47:03.873239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.873239+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.476375+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":462,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.332008+0000","last_change":"2024-09-21T11:48:51.931885+0000","last_active":"2024-09-21T11:50:18.332008+0000","last_peered":"2024-09-21T11:50:18.332008+0000","last_clean":"2024-09-21T11:50:18.332008+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:50:18.332008+0000","last_undegraded":"2024-09-21T11:50:18.332008+0000","last_fullsized":"2024-09-21T11:50:18.332008+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:51.931832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:51.931832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:13.265721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":466,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.342956+0000","last_change":"2024-09-21T11:46:47.611118+0000","last_active":"2024-09-21T11:50:18.342956+0000","last_peered":"2024-09-21T11:50:18.342956+0000","last_clean":"2024-09-21T11:50:18.342956+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:50:18.342956+0000","last_undegraded":"2024-09-21T11:50:18.342956+0000","last_fullsized":"2024-09-21T11:50:18.342956+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:46:47.611042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:47.611042+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:26.405632+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":167,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.337048+0000","last_change":"2024-09-21T11:50:02.920313+0000","last_active":"2024-09-21T11:50:18.337048+0000","last_peered":"2024-09-21T11:50:18.337048+0000","last_clean":"2024-09-21T11:50:18.337048+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:50:18.337048+0000","last_undegraded":"2024-09-21T11:50:18.337048+0000","last_fullsized":"2024-09-21T11:50:18.337048+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:02.920229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.920229+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:20.782582+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":506,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.364028+0000","last_change":"2024-09-21T11:49:58.702620+0000","last_active":"2024-09-21T11:50:21.364028+0000","last_peered":"2024-09-21T11:50:21.364028+0000","last_clean":"2024-09-21T11:50:21.364028+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:50:21.364028+0000","last_undegraded":"2024-09-21T11:50:21.364028+0000","last_fullsized":"2024-09-21T11:50:21.364028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:49:58.702544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.702544+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.427898+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":613,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341444+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:50:18.341444+0000","last_peered":"2024-09-21T11:50:18.341444+0000","last_clean":"2024-09-21T11:50:18.341444+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:50:18.341444+0000","last_undegraded":"2024-09-21T11:50:18.341444+0000","last_fullsized":"2024-09-21T11:50:18.341444+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.417310+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"275'6614","reported_seq":6681,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.941854+0000","last_change":"2024-09-21T11:49:08.748763+0000","last_active":"2024-09-21T11:50:21.941854+0000","last_peered":"2024-09-21T11:50:21.941854+0000","last_clean":"2024-09-21T11:50:21.941854+0000","last_became_active":"2024-09-21T11:49:08.748359+0000","last_became_peered":"2024-09-21T11:49:08.748359+0000","last_unstale":"2024-09-21T11:50:21.941854+0000","last_undegraded":"2024-09-21T11:50:21.941854+0000","last_fullsized":"2024-09-21T11:50:21.941854+0000","mapping_epoch":247,"log_start":"253'1700","ondisk_log_start":"253'1700","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4914,"log_dups_size":0,"ondisk_log_size":4914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.761336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":433455104,"num_objects":6614,"num_object_clones":0,"num_object_copies":19842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6614,"num_write_kb":423296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":197,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344320+0000","last_change":"2024-09-21T11:49:15.740456+0000","last_active":"2024-09-21T11:50:18.344320+0000","last_peered":"2024-09-21T11:50:18.344320+0000","last_clean":"2024-09-21T11:50:18.344320+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:50:18.344320+0000","last_undegraded":"2024-09-21T11:50:18.344320+0000","last_fullsized":"2024-09-21T11:50:18.344320+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:49:15.739582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:15.739582+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.931869+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":494,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351642+0000","last_change":"2024-09-21T11:49:47.205216+0000","last_active":"2024-09-21T11:50:21.351642+0000","last_peered":"2024-09-21T11:50:21.351642+0000","last_clean":"2024-09-21T11:50:21.351642+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:50:21.351642+0000","last_undegraded":"2024-09-21T11:50:21.351642+0000","last_fullsized":"2024-09-21T11:50:21.351642+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.204889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.204889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:52.456856+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":587,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.332428+0000","last_change":"2024-09-21T11:47:01.933697+0000","last_active":"2024-09-21T11:50:18.332428+0000","last_peered":"2024-09-21T11:50:18.332428+0000","last_clean":"2024-09-21T11:50:18.332428+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:50:18.332428+0000","last_undegraded":"2024-09-21T11:50:18.332428+0000","last_fullsized":"2024-09-21T11:50:18.332428+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:01.933648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:01.933648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:11.347125+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":471,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.345021+0000","last_change":"2024-09-21T11:49:20.990003+0000","last_active":"2024-09-21T11:50:18.345021+0000","last_peered":"2024-09-21T11:50:18.345021+0000","last_clean":"2024-09-21T11:50:18.345021+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:50:18.345021+0000","last_undegraded":"2024-09-21T11:50:18.345021+0000","last_fullsized":"2024-09-21T11:50:18.345021+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:20.989895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:20.989895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:23.707729+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"274'6434","reported_seq":6499,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:20.944077+0000","last_change":"2024-09-21T11:49:08.520005+0000","last_active":"2024-09-21T11:50:20.944077+0000","last_peered":"2024-09-21T11:50:20.944077+0000","last_clean":"2024-09-21T11:50:20.944077+0000","last_became_active":"2024-09-21T11:49:08.519622+0000","last_became_peered":"2024-09-21T11:49:08.519622+0000","last_unstale":"2024-09-21T11:50:20.944077+0000","last_undegraded":"2024-09-21T11:50:20.944077+0000","last_fullsized":"2024-09-21T11:50:20.944077+0000","mapping_epoch":247,"log_start":"252'1600","ondisk_log_start":"252'1600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4834,"log_dups_size":0,"ondisk_log_size":4834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421658624,"num_objects":6434,"num_object_clones":0,"num_object_copies":19302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6434,"num_write_kb":411776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":527,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351274+0000","last_change":"2024-09-21T11:49:52.209526+0000","last_active":"2024-09-21T11:50:21.351274+0000","last_peered":"2024-09-21T11:50:21.351274+0000","last_clean":"2024-09-21T11:50:21.351274+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:50:21.351274+0000","last_undegraded":"2024-09-21T11:50:21.351274+0000","last_fullsized":"2024-09-21T11:50:21.351274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:49:52.209433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:52.209433+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.566988+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":480,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.352397+0000","last_change":"2024-09-21T11:49:55.238709+0000","last_active":"2024-09-21T11:50:21.352397+0000","last_peered":"2024-09-21T11:50:21.352397+0000","last_clean":"2024-09-21T11:50:21.352397+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:50:21.352397+0000","last_undegraded":"2024-09-21T11:50:21.352397+0000","last_fullsized":"2024-09-21T11:50:21.352397+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:55.238302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:55.238302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:06.703143+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":462,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.330400+0000","last_change":"2024-09-21T11:48:47.933145+0000","last_active":"2024-09-21T11:50:18.330400+0000","last_peered":"2024-09-21T11:50:18.330400+0000","last_clean":"2024-09-21T11:50:18.330400+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:50:18.330400+0000","last_undegraded":"2024-09-21T11:50:18.330400+0000","last_fullsized":"2024-09-21T11:50:18.330400+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.933099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.933099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.352011+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":450,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.331127+0000","last_change":"2024-09-21T11:46:24.717355+0000","last_active":"2024-09-21T11:50:18.331127+0000","last_peered":"2024-09-21T11:50:18.331127+0000","last_clean":"2024-09-21T11:50:18.331127+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:50:18.331127+0000","last_undegraded":"2024-09-21T11:50:18.331127+0000","last_fullsized":"2024-09-21T11:50:18.331127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.717285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.717285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.284555+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"275'6570","reported_seq":6637,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.931631+0000","last_change":"2024-09-21T11:49:08.609939+0000","last_active":"2024-09-21T11:50:21.931631+0000","last_peered":"2024-09-21T11:50:21.931631+0000","last_clean":"2024-09-21T11:50:21.931631+0000","last_became_active":"2024-09-21T11:49:08.609618+0000","last_became_peered":"2024-09-21T11:49:08.609618+0000","last_unstale":"2024-09-21T11:50:21.931631+0000","last_undegraded":"2024-09-21T11:50:21.931631+0000","last_fullsized":"2024-09-21T11:50:21.931631+0000","mapping_epoch":247,"log_start":"253'1700","ondisk_log_start":"253'1700","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4870,"log_dups_size":0,"ondisk_log_size":4870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.213949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430571520,"num_objects":6570,"num_object_clones":0,"num_object_copies":19710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6570,"num_write_kb":420480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":524,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.330633+0000","last_change":"2024-09-21T11:49:10.053574+0000","last_active":"2024-09-21T11:50:18.330633+0000","last_peered":"2024-09-21T11:50:18.330633+0000","last_clean":"2024-09-21T11:50:18.330633+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:50:18.330633+0000","last_undegraded":"2024-09-21T11:50:18.330633+0000","last_fullsized":"2024-09-21T11:50:18.330633+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:49:10.053515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:10.053515+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:34.201371+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":502,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.364457+0000","last_change":"2024-09-21T11:49:23.515285+0000","last_active":"2024-09-21T11:50:21.364457+0000","last_peered":"2024-09-21T11:50:21.364457+0000","last_clean":"2024-09-21T11:50:21.364457+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:50:21.364457+0000","last_undegraded":"2024-09-21T11:50:21.364457+0000","last_fullsized":"2024-09-21T11:50:21.364457+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:49:23.515206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.515206+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.949694+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":621,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351058+0000","last_change":"2024-09-21T11:49:14.198995+0000","last_active":"2024-09-21T11:50:21.351058+0000","last_peered":"2024-09-21T11:50:21.351058+0000","last_clean":"2024-09-21T11:50:21.351058+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:50:21.351058+0000","last_undegraded":"2024-09-21T11:50:21.351058+0000","last_fullsized":"2024-09-21T11:50:21.351058+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:49:14.198804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.198804+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.344589+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":460,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341033+0000","last_change":"2024-09-21T11:49:12.548705+0000","last_active":"2024-09-21T11:50:18.341033+0000","last_peered":"2024-09-21T11:50:18.341033+0000","last_clean":"2024-09-21T11:50:18.341033+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:50:18.341033+0000","last_undegraded":"2024-09-21T11:50:18.341033+0000","last_fullsized":"2024-09-21T11:50:18.341033+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:12.548650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:12.548650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:22.749399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"274'6478","reported_seq":6543,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.013051+0000","last_change":"2024-09-21T11:49:08.518112+0000","last_active":"2024-09-21T11:50:21.013051+0000","last_peered":"2024-09-21T11:50:21.013051+0000","last_clean":"2024-09-21T11:50:21.013051+0000","last_became_active":"2024-09-21T11:49:08.517344+0000","last_became_peered":"2024-09-21T11:49:08.517344+0000","last_unstale":"2024-09-21T11:50:21.013051+0000","last_undegraded":"2024-09-21T11:50:21.013051+0000","last_fullsized":"2024-09-21T11:50:21.013051+0000","mapping_epoch":247,"log_start":"252'1200","ondisk_log_start":"252'1200","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5278,"log_dups_size":0,"ondisk_log_size":5278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:20.684739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":424542208,"num_objects":6478,"num_object_clones":0,"num_object_copies":19434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6478,"num_write_kb":414592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":182,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.340648+0000","last_change":"2024-09-21T11:47:03.398638+0000","last_active":"2024-09-21T11:50:18.340648+0000","last_peered":"2024-09-21T11:50:18.340648+0000","last_clean":"2024-09-21T11:50:18.340648+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:50:18.340648+0000","last_undegraded":"2024-09-21T11:50:18.340648+0000","last_fullsized":"2024-09-21T11:50:18.340648+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:12.227121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.227121+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:06.869295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":485,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.346368+0000","last_change":"2024-09-21T11:49:09.924450+0000","last_active":"2024-09-21T11:50:18.346368+0000","last_peered":"2024-09-21T11:50:18.346368+0000","last_clean":"2024-09-21T11:50:18.346368+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:50:18.346368+0000","last_undegraded":"2024-09-21T11:50:18.346368+0000","last_fullsized":"2024-09-21T11:50:18.346368+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:09.924400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:09.924400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:29.238919+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":583,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.363918+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:50:21.363918+0000","last_peered":"2024-09-21T11:50:21.363918+0000","last_clean":"2024-09-21T11:50:21.363918+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:50:21.363918+0000","last_undegraded":"2024-09-21T11:50:21.363918+0000","last_fullsized":"2024-09-21T11:50:21.363918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:28.738852+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":460,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.363485+0000","last_change":"2024-09-21T11:49:07.464269+0000","last_active":"2024-09-21T11:50:21.363485+0000","last_peered":"2024-09-21T11:50:21.363485+0000","last_clean":"2024-09-21T11:50:21.363485+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:50:21.363485+0000","last_undegraded":"2024-09-21T11:50:21.363485+0000","last_fullsized":"2024-09-21T11:50:21.363485+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.464204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.464204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:13.689854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"274'6483","reported_seq":6548,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:20.938382+0000","last_change":"2024-09-21T11:49:08.609660+0000","last_active":"2024-09-21T11:50:20.938382+0000","last_peered":"2024-09-21T11:50:20.938382+0000","last_clean":"2024-09-21T11:50:20.938382+0000","last_became_active":"2024-09-21T11:49:08.609226+0000","last_became_peered":"2024-09-21T11:49:08.609226+0000","last_unstale":"2024-09-21T11:50:20.938382+0000","last_undegraded":"2024-09-21T11:50:20.938382+0000","last_fullsized":"2024-09-21T11:50:20.938382+0000","mapping_epoch":247,"log_start":"253'1700","ondisk_log_start":"253'1700","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4783,"log_dups_size":0,"ondisk_log_size":4783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:28.968012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":424869888,"num_objects":6483,"num_object_clones":0,"num_object_copies":19449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6483,"num_write_kb":414912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":510,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.364149+0000","last_change":"2024-09-21T11:47:03.750064+0000","last_active":"2024-09-21T11:50:21.364149+0000","last_peered":"2024-09-21T11:50:21.364149+0000","last_clean":"2024-09-21T11:50:21.364149+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:50:21.364149+0000","last_undegraded":"2024-09-21T11:50:21.364149+0000","last_fullsized":"2024-09-21T11:50:21.364149+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:47:03.749962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.749962+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.702349+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":163,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.350372+0000","last_change":"2024-09-21T11:49:59.954903+0000","last_active":"2024-09-21T11:50:18.350372+0000","last_peered":"2024-09-21T11:50:18.350372+0000","last_clean":"2024-09-21T11:50:18.350372+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:50:18.350372+0000","last_undegraded":"2024-09-21T11:50:18.350372+0000","last_fullsized":"2024-09-21T11:50:18.350372+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:59.954835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:59.954835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:15.305841+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":453,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.338662+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:50:18.338662+0000","last_peered":"2024-09-21T11:50:18.338662+0000","last_clean":"2024-09-21T11:50:18.338662+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:50:18.338662+0000","last_undegraded":"2024-09-21T11:50:18.338662+0000","last_fullsized":"2024-09-21T11:50:18.338662+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.352185+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":155,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.336815+0000","last_change":"2024-09-21T11:47:03.379157+0000","last_active":"2024-09-21T11:50:18.336815+0000","last_peered":"2024-09-21T11:50:18.336815+0000","last_clean":"2024-09-21T11:50:18.336815+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:50:18.336815+0000","last_undegraded":"2024-09-21T11:50:18.336815+0000","last_fullsized":"2024-09-21T11:50:18.336815+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:32.828402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"275'6488","reported_seq":6555,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.731877+0000","last_change":"2024-09-21T11:49:08.516652+0000","last_active":"2024-09-21T11:50:21.731877+0000","last_peered":"2024-09-21T11:50:21.731877+0000","last_clean":"2024-09-21T11:50:21.731877+0000","last_became_active":"2024-09-21T11:49:08.516355+0000","last_became_peered":"2024-09-21T11:49:08.516355+0000","last_unstale":"2024-09-21T11:50:21.731877+0000","last_undegraded":"2024-09-21T11:50:21.731877+0000","last_fullsized":"2024-09-21T11:50:21.731877+0000","mapping_epoch":247,"log_start":"251'400","ondisk_log_start":"251'400","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":6088,"log_dups_size":0,"ondisk_log_size":6088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":425197568,"num_objects":6488,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6488,"num_write_kb":415232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":235,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.350315+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:50:18.350315+0000","last_peered":"2024-09-21T11:50:18.350315+0000","last_clean":"2024-09-21T11:50:18.350315+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:50:18.350315+0000","last_undegraded":"2024-09-21T11:50:18.350315+0000","last_fullsized":"2024-09-21T11:50:18.350315+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:39.781729+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":460,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.331888+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:50:18.331888+0000","last_peered":"2024-09-21T11:50:18.331888+0000","last_clean":"2024-09-21T11:50:18.331888+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:50:18.331888+0000","last_undegraded":"2024-09-21T11:50:18.331888+0000","last_fullsized":"2024-09-21T11:50:18.331888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:41.126102+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":493,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344163+0000","last_change":"2024-09-21T11:47:20.000195+0000","last_active":"2024-09-21T11:50:18.344163+0000","last_peered":"2024-09-21T11:50:18.344163+0000","last_clean":"2024-09-21T11:50:18.344163+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:50:18.344163+0000","last_undegraded":"2024-09-21T11:50:18.344163+0000","last_fullsized":"2024-09-21T11:50:18.344163+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:47:20.000051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.000051+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:11.774530+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":469,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.363963+0000","last_change":"2024-09-21T11:47:03.385173+0000","last_active":"2024-09-21T11:50:21.363963+0000","last_peered":"2024-09-21T11:50:21.363963+0000","last_clean":"2024-09-21T11:50:21.363963+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:50:21.363963+0000","last_undegraded":"2024-09-21T11:50:21.363963+0000","last_fullsized":"2024-09-21T11:50:21.363963+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:32.712434+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":444,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351428+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:50:21.351428+0000","last_peered":"2024-09-21T11:50:21.351428+0000","last_clean":"2024-09-21T11:50:21.351428+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:50:21.351428+0000","last_undegraded":"2024-09-21T11:50:21.351428+0000","last_fullsized":"2024-09-21T11:50:21.351428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.205651+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"274'6559","reported_seq":6624,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.058416+0000","last_change":"2024-09-21T11:49:08.518054+0000","last_active":"2024-09-21T11:50:21.058416+0000","last_peered":"2024-09-21T11:50:21.058416+0000","last_clean":"2024-09-21T11:50:21.058416+0000","last_became_active":"2024-09-21T11:49:08.517758+0000","last_became_peered":"2024-09-21T11:49:08.517758+0000","last_unstale":"2024-09-21T11:50:21.058416+0000","last_undegraded":"2024-09-21T11:50:21.058416+0000","last_fullsized":"2024-09-21T11:50:21.058416+0000","mapping_epoch":247,"log_start":"252'1200","ondisk_log_start":"252'1200","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.722671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429850624,"num_objects":6559,"num_object_clones":0,"num_object_copies":19677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6559,"num_write_kb":419776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":108,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.356389+0000","last_change":"2024-09-21T11:48:33.075081+0000","last_active":"2024-09-21T11:50:21.356389+0000","last_peered":"2024-09-21T11:50:21.356389+0000","last_clean":"2024-09-21T11:50:21.356389+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:50:21.356389+0000","last_undegraded":"2024-09-21T11:50:21.356389+0000","last_fullsized":"2024-09-21T11:50:21.356389+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.996705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":159,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.343231+0000","last_change":"2024-09-21T11:47:03.391283+0000","last_active":"2024-09-21T11:50:18.343231+0000","last_peered":"2024-09-21T11:50:18.343231+0000","last_clean":"2024-09-21T11:50:18.343231+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:50:18.343231+0000","last_undegraded":"2024-09-21T11:50:18.343231+0000","last_fullsized":"2024-09-21T11:50:18.343231+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:31.825843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":489,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.355371+0000","last_change":"2024-09-21T11:48:48.994249+0000","last_active":"2024-09-21T11:50:21.355371+0000","last_peered":"2024-09-21T11:50:21.355371+0000","last_clean":"2024-09-21T11:50:21.355371+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:50:21.355371+0000","last_undegraded":"2024-09-21T11:50:21.355371+0000","last_fullsized":"2024-09-21T11:50:21.355371+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:48:48.994172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:48:48.994172+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:15.864789+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":472,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.356192+0000","last_change":"2024-09-21T11:49:47.008541+0000","last_active":"2024-09-21T11:50:21.356192+0000","last_peered":"2024-09-21T11:50:21.356192+0000","last_clean":"2024-09-21T11:50:21.356192+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:50:21.356192+0000","last_undegraded":"2024-09-21T11:50:21.356192+0000","last_fullsized":"2024-09-21T11:50:21.356192+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.008369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.008369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:53.561004+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":465,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341183+0000","last_change":"2024-09-21T11:49:54.388131+0000","last_active":"2024-09-21T11:50:18.341183+0000","last_peered":"2024-09-21T11:50:18.341183+0000","last_clean":"2024-09-21T11:50:18.341183+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:50:18.341183+0000","last_undegraded":"2024-09-21T11:50:18.341183+0000","last_fullsized":"2024-09-21T11:50:18.341183+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.388068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.388068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:57.353255+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"275'6619","reported_seq":6686,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.756394+0000","last_change":"2024-09-21T11:49:08.515501+0000","last_active":"2024-09-21T11:50:21.756394+0000","last_peered":"2024-09-21T11:50:21.756394+0000","last_clean":"2024-09-21T11:50:21.756394+0000","last_became_active":"2024-09-21T11:49:08.515204+0000","last_became_peered":"2024-09-21T11:49:08.515204+0000","last_unstale":"2024-09-21T11:50:21.756394+0000","last_undegraded":"2024-09-21T11:50:21.756394+0000","last_fullsized":"2024-09-21T11:50:21.756394+0000","mapping_epoch":247,"log_start":"252'600","ondisk_log_start":"252'600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":6019,"log_dups_size":0,"ondisk_log_size":6019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.697074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":433782784,"num_objects":6619,"num_object_clones":0,"num_object_copies":19857,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6619,"num_write_kb":423616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":537,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.364348+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:50:21.364348+0000","last_peered":"2024-09-21T11:50:21.364348+0000","last_clean":"2024-09-21T11:50:21.364348+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:50:21.364348+0000","last_undegraded":"2024-09-21T11:50:21.364348+0000","last_fullsized":"2024-09-21T11:50:21.364348+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:34.714346+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":480,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.363593+0000","last_change":"2024-09-21T11:49:16.415641+0000","last_active":"2024-09-21T11:50:21.363593+0000","last_peered":"2024-09-21T11:50:21.363593+0000","last_clean":"2024-09-21T11:50:21.363593+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:50:21.363593+0000","last_undegraded":"2024-09-21T11:50:21.363593+0000","last_fullsized":"2024-09-21T11:50:21.363593+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:16.415597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:16.415597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:45.957747+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1133,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:19.616272+0000","last_change":"2024-09-21T11:49:51.434402+0000","last_active":"2024-09-21T11:50:19.616272+0000","last_peered":"2024-09-21T11:50:19.616272+0000","last_clean":"2024-09-21T11:50:19.616272+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:50:19.616272+0000","last_undegraded":"2024-09-21T11:50:19.616272+0000","last_fullsized":"2024-09-21T11:50:19.616272+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:51.434333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:49:51.434333+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:02.759046+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":612,"num_read_kb":607,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":475,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.345628+0000","last_change":"2024-09-21T11:48:42.042870+0000","last_active":"2024-09-21T11:50:18.345628+0000","last_peered":"2024-09-21T11:50:18.345628+0000","last_clean":"2024-09-21T11:50:18.345628+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:50:18.345628+0000","last_undegraded":"2024-09-21T11:50:18.345628+0000","last_fullsized":"2024-09-21T11:50:18.345628+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:42.042814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:42.042814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.767298+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":163,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.349937+0000","last_change":"2024-09-21T11:49:54.996327+0000","last_active":"2024-09-21T11:50:18.349937+0000","last_peered":"2024-09-21T11:50:18.349937+0000","last_clean":"2024-09-21T11:50:18.349937+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:50:18.349937+0000","last_undegraded":"2024-09-21T11:50:18.349937+0000","last_fullsized":"2024-09-21T11:50:18.349937+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.996240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.996240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.233955+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"275'6490","reported_seq":6557,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.937866+0000","last_change":"2024-09-21T11:49:08.516921+0000","last_active":"2024-09-21T11:50:21.937866+0000","last_peered":"2024-09-21T11:50:21.937866+0000","last_clean":"2024-09-21T11:50:21.937866+0000","last_became_active":"2024-09-21T11:49:08.516627+0000","last_became_peered":"2024-09-21T11:49:08.516627+0000","last_unstale":"2024-09-21T11:50:21.937866+0000","last_undegraded":"2024-09-21T11:50:21.937866+0000","last_fullsized":"2024-09-21T11:50:21.937866+0000","mapping_epoch":247,"log_start":"252'1600","ondisk_log_start":"252'1600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4890,"log_dups_size":0,"ondisk_log_size":4890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":425328640,"num_objects":6490,"num_object_clones":0,"num_object_copies":19470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6490,"num_write_kb":415360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":630,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341033+0000","last_change":"2024-09-21T11:49:18.420485+0000","last_active":"2024-09-21T11:50:18.341033+0000","last_peered":"2024-09-21T11:50:18.341033+0000","last_clean":"2024-09-21T11:50:18.341033+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:50:18.341033+0000","last_undegraded":"2024-09-21T11:50:18.341033+0000","last_fullsized":"2024-09-21T11:50:18.341033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:49:18.420396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:18.420396+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.826435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":472,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.353289+0000","last_change":"2024-09-21T11:46:57.833589+0000","last_active":"2024-09-21T11:50:21.353289+0000","last_peered":"2024-09-21T11:50:21.353289+0000","last_clean":"2024-09-21T11:50:21.353289+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:50:21.353289+0000","last_undegraded":"2024-09-21T11:50:21.353289+0000","last_fullsized":"2024-09-21T11:50:21.353289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:57.833329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.833329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.227963+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":525,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.340198+0000","last_change":"2024-09-21T11:46:58.645189+0000","last_active":"2024-09-21T11:50:18.340198+0000","last_peered":"2024-09-21T11:50:18.340198+0000","last_clean":"2024-09-21T11:50:18.340198+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:50:18.340198+0000","last_undegraded":"2024-09-21T11:50:18.340198+0000","last_fullsized":"2024-09-21T11:50:18.340198+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:46:58.645122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.645122+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.471456+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":469,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.342918+0000","last_change":"2024-09-21T11:49:06.639805+0000","last_active":"2024-09-21T11:50:18.342918+0000","last_peered":"2024-09-21T11:50:18.342918+0000","last_clean":"2024-09-21T11:50:18.342918+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:50:18.342918+0000","last_undegraded":"2024-09-21T11:50:18.342918+0000","last_fullsized":"2024-09-21T11:50:18.342918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:06.639732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:06.639732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:16.438605+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":457,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.345088+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:50:18.345088+0000","last_peered":"2024-09-21T11:50:18.345088+0000","last_clean":"2024-09-21T11:50:18.345088+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:50:18.345088+0000","last_undegraded":"2024-09-21T11:50:18.345088+0000","last_fullsized":"2024-09-21T11:50:18.345088+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.925430+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"274'6426","reported_seq":6491,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:20.921384+0000","last_change":"2024-09-21T11:49:08.519327+0000","last_active":"2024-09-21T11:50:20.921384+0000","last_peered":"2024-09-21T11:50:20.921384+0000","last_clean":"2024-09-21T11:50:20.921384+0000","last_became_active":"2024-09-21T11:49:08.519011+0000","last_became_peered":"2024-09-21T11:49:08.519011+0000","last_unstale":"2024-09-21T11:50:20.921384+0000","last_undegraded":"2024-09-21T11:50:20.921384+0000","last_fullsized":"2024-09-21T11:50:20.921384+0000","mapping_epoch":247,"log_start":"252'1600","ondisk_log_start":"252'1600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4826,"log_dups_size":0,"ondisk_log_size":4826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.190157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421134336,"num_objects":6426,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6426,"num_write_kb":411264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":527,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.332486+0000","last_change":"2024-09-21T11:48:05.072998+0000","last_active":"2024-09-21T11:50:18.332486+0000","last_peered":"2024-09-21T11:50:18.332486+0000","last_clean":"2024-09-21T11:50:18.332486+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:50:18.332486+0000","last_undegraded":"2024-09-21T11:50:18.332486+0000","last_fullsized":"2024-09-21T11:50:18.332486+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:05.072910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.072910+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.126516+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":494,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.330745+0000","last_change":"2024-09-21T11:49:49.165213+0000","last_active":"2024-09-21T11:50:18.330745+0000","last_peered":"2024-09-21T11:50:18.330745+0000","last_clean":"2024-09-21T11:50:18.330745+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:50:18.330745+0000","last_undegraded":"2024-09-21T11:50:18.330745+0000","last_fullsized":"2024-09-21T11:50:18.330745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.165146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.165146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:04.935623+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":461,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344873+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:50:18.344873+0000","last_peered":"2024-09-21T11:50:18.344873+0000","last_clean":"2024-09-21T11:50:18.344873+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:50:18.344873+0000","last_undegraded":"2024-09-21T11:50:18.344873+0000","last_fullsized":"2024-09-21T11:50:18.344873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.928709+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":459,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351702+0000","last_change":"2024-09-21T11:48:23.285482+0000","last_active":"2024-09-21T11:50:21.351702+0000","last_peered":"2024-09-21T11:50:21.351702+0000","last_clean":"2024-09-21T11:50:21.351702+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:50:21.351702+0000","last_undegraded":"2024-09-21T11:50:21.351702+0000","last_fullsized":"2024-09-21T11:50:21.351702+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.285283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.285283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.489355+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"274'6437","reported_seq":6502,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.073290+0000","last_change":"2024-09-21T11:49:08.748461+0000","last_active":"2024-09-21T11:50:21.073290+0000","last_peered":"2024-09-21T11:50:21.073290+0000","last_clean":"2024-09-21T11:50:21.073290+0000","last_became_active":"2024-09-21T11:49:08.748155+0000","last_became_peered":"2024-09-21T11:49:08.748155+0000","last_unstale":"2024-09-21T11:50:21.073290+0000","last_undegraded":"2024-09-21T11:50:21.073290+0000","last_fullsized":"2024-09-21T11:50:21.073290+0000","mapping_epoch":247,"log_start":"253'1700","ondisk_log_start":"253'1700","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4737,"log_dups_size":0,"ondisk_log_size":4737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.862263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421855232,"num_objects":6437,"num_object_clones":0,"num_object_copies":19311,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6437,"num_write_kb":411968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":598,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.332468+0000","last_change":"2024-09-21T11:49:09.037775+0000","last_active":"2024-09-21T11:50:18.332468+0000","last_peered":"2024-09-21T11:50:18.332468+0000","last_clean":"2024-09-21T11:50:18.332468+0000","last_became_active":"2024-09-21T11:48:33.022321+0000","last_became_peered":"2024-09-21T11:48:33.022321+0000","last_unstale":"2024-09-21T11:50:18.332468+0000","last_undegraded":"2024-09-21T11:50:18.332468+0000","last_fullsized":"2024-09-21T11:50:18.332468+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:49:09.037713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:09.037713+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:23.218918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":491,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.338872+0000","last_change":"2024-09-21T11:47:03.365071+0000","last_active":"2024-09-21T11:50:18.338872+0000","last_peered":"2024-09-21T11:50:18.338872+0000","last_clean":"2024-09-21T11:50:18.338872+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:50:18.338872+0000","last_undegraded":"2024-09-21T11:50:18.338872+0000","last_fullsized":"2024-09-21T11:50:18.338872+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.692429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.692429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.349348+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":155,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.349844+0000","last_change":"2024-09-21T11:47:03.375108+0000","last_active":"2024-09-21T11:50:18.349844+0000","last_peered":"2024-09-21T11:50:18.349844+0000","last_clean":"2024-09-21T11:50:18.349844+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:50:18.349844+0000","last_undegraded":"2024-09-21T11:50:18.349844+0000","last_fullsized":"2024-09-21T11:50:18.349844+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:50.008635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:50.008635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.928506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":468,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.356544+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:50:21.356544+0000","last_peered":"2024-09-21T11:50:21.356544+0000","last_clean":"2024-09-21T11:50:21.356544+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:50:21.356544+0000","last_undegraded":"2024-09-21T11:50:21.356544+0000","last_fullsized":"2024-09-21T11:50:21.356544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:42.937389+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"275'6594","reported_seq":6661,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.901969+0000","last_change":"2024-09-21T11:49:08.517565+0000","last_active":"2024-09-21T11:50:21.901969+0000","last_peered":"2024-09-21T11:50:21.901969+0000","last_clean":"2024-09-21T11:50:21.901969+0000","last_became_active":"2024-09-21T11:49:08.517011+0000","last_became_peered":"2024-09-21T11:49:08.517011+0000","last_unstale":"2024-09-21T11:50:21.901969+0000","last_undegraded":"2024-09-21T11:50:21.901969+0000","last_fullsized":"2024-09-21T11:50:21.901969+0000","mapping_epoch":247,"log_start":"252'1700","ondisk_log_start":"252'1700","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4894,"log_dups_size":0,"ondisk_log_size":4894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.617618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432144384,"num_objects":6594,"num_object_clones":0,"num_object_copies":19782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6594,"num_write_kb":422016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":512,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.332096+0000","last_change":"2024-09-21T11:50:06.225159+0000","last_active":"2024-09-21T11:50:18.332096+0000","last_peered":"2024-09-21T11:50:18.332096+0000","last_clean":"2024-09-21T11:50:18.332096+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:50:18.332096+0000","last_undegraded":"2024-09-21T11:50:18.332096+0000","last_fullsized":"2024-09-21T11:50:18.332096+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:32.807084+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":456,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.354484+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:50:21.354484+0000","last_peered":"2024-09-21T11:50:21.354484+0000","last_clean":"2024-09-21T11:50:21.354484+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:50:21.354484+0000","last_undegraded":"2024-09-21T11:50:21.354484+0000","last_fullsized":"2024-09-21T11:50:21.354484+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:28.151163+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":613,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.356133+0000","last_change":"2024-09-21T11:50:02.975387+0000","last_active":"2024-09-21T11:50:21.356133+0000","last_peered":"2024-09-21T11:50:21.356133+0000","last_clean":"2024-09-21T11:50:21.356133+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:50:21.356133+0000","last_undegraded":"2024-09-21T11:50:21.356133+0000","last_fullsized":"2024-09-21T11:50:21.356133+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:50:02.975313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.975313+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:25.444093+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":464,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.332228+0000","last_change":"2024-09-21T11:48:07.992586+0000","last_active":"2024-09-21T11:50:18.332228+0000","last_peered":"2024-09-21T11:50:18.332228+0000","last_clean":"2024-09-21T11:50:18.332228+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:50:18.332228+0000","last_undegraded":"2024-09-21T11:50:18.332228+0000","last_fullsized":"2024-09-21T11:50:18.332228+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.992517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.992517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:42.078128+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"274'6509","reported_seq":6573,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.061745+0000","last_change":"2024-09-21T11:49:08.748457+0000","last_active":"2024-09-21T11:50:21.061745+0000","last_peered":"2024-09-21T11:50:21.061745+0000","last_clean":"2024-09-21T11:50:21.061745+0000","last_became_active":"2024-09-21T11:49:08.748171+0000","last_became_peered":"2024-09-21T11:49:08.748171+0000","last_unstale":"2024-09-21T11:50:21.061745+0000","last_undegraded":"2024-09-21T11:50:21.061745+0000","last_fullsized":"2024-09-21T11:50:21.061745+0000","mapping_epoch":247,"log_start":"252'1200","ondisk_log_start":"252'1200","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5309,"log_dups_size":0,"ondisk_log_size":5309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426573824,"num_objects":6509,"num_object_clones":0,"num_object_copies":19527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6509,"num_write_kb":416576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":518,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.364059+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:50:21.364059+0000","last_peered":"2024-09-21T11:50:21.364059+0000","last_clean":"2024-09-21T11:50:21.364059+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:50:21.364059+0000","last_undegraded":"2024-09-21T11:50:21.364059+0000","last_fullsized":"2024-09-21T11:50:21.364059+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:41.803003+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":477,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344390+0000","last_change":"2024-09-21T11:49:23.000190+0000","last_active":"2024-09-21T11:50:18.344390+0000","last_peered":"2024-09-21T11:50:18.344390+0000","last_clean":"2024-09-21T11:50:18.344390+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:50:18.344390+0000","last_undegraded":"2024-09-21T11:50:18.344390+0000","last_fullsized":"2024-09-21T11:50:18.344390+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.000138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.000138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:42.451760+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":464,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.355526+0000","last_change":"2024-09-21T11:48:49.993307+0000","last_active":"2024-09-21T11:50:21.355526+0000","last_peered":"2024-09-21T11:50:21.355526+0000","last_clean":"2024-09-21T11:50:21.355526+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:50:21.355526+0000","last_undegraded":"2024-09-21T11:50:21.355526+0000","last_fullsized":"2024-09-21T11:50:21.355526+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.993230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.993230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.951895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":443,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.363461+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:50:21.363461+0000","last_peered":"2024-09-21T11:50:21.363461+0000","last_clean":"2024-09-21T11:50:21.363461+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:50:21.363461+0000","last_undegraded":"2024-09-21T11:50:21.363461+0000","last_fullsized":"2024-09-21T11:50:21.363461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.764486+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"274'6580","reported_seq":6645,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.042904+0000","last_change":"2024-09-21T11:49:08.517663+0000","last_active":"2024-09-21T11:50:21.042904+0000","last_peered":"2024-09-21T11:50:21.042904+0000","last_clean":"2024-09-21T11:50:21.042904+0000","last_became_active":"2024-09-21T11:49:08.516975+0000","last_became_peered":"2024-09-21T11:49:08.516975+0000","last_unstale":"2024-09-21T11:50:21.042904+0000","last_undegraded":"2024-09-21T11:50:21.042904+0000","last_fullsized":"2024-09-21T11:50:21.042904+0000","mapping_epoch":247,"log_start":"252'1300","ondisk_log_start":"252'1300","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5280,"log_dups_size":0,"ondisk_log_size":5280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:24.243398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":431226880,"num_objects":6580,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6580,"num_write_kb":421120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":582,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.338788+0000","last_change":"2024-09-21T11:48:53.117184+0000","last_active":"2024-09-21T11:50:18.338788+0000","last_peered":"2024-09-21T11:50:18.338788+0000","last_clean":"2024-09-21T11:50:18.338788+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:50:18.338788+0000","last_undegraded":"2024-09-21T11:50:18.338788+0000","last_fullsized":"2024-09-21T11:50:18.338788+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:48:53.117090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:53.117090+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":477,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341580+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:50:18.341580+0000","last_peered":"2024-09-21T11:50:18.341580+0000","last_clean":"2024-09-21T11:50:18.341580+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:50:18.341580+0000","last_undegraded":"2024-09-21T11:50:18.341580+0000","last_fullsized":"2024-09-21T11:50:18.341580+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:04.681745+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":472,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.356713+0000","last_change":"2024-09-21T11:46:16.775013+0000","last_active":"2024-09-21T11:50:21.356713+0000","last_peered":"2024-09-21T11:50:21.356713+0000","last_clean":"2024-09-21T11:50:21.356713+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:50:21.356713+0000","last_undegraded":"2024-09-21T11:50:21.356713+0000","last_fullsized":"2024-09-21T11:50:21.356713+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:16.774939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.774939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:29.020969+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":457,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.340099+0000","last_change":"2024-09-21T11:46:59.704981+0000","last_active":"2024-09-21T11:50:18.340099+0000","last_peered":"2024-09-21T11:50:18.340099+0000","last_clean":"2024-09-21T11:50:18.340099+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:50:18.340099+0000","last_undegraded":"2024-09-21T11:50:18.340099+0000","last_fullsized":"2024-09-21T11:50:18.340099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:31.444921+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"274'6414","reported_seq":6479,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.064002+0000","last_change":"2024-09-21T11:49:08.610149+0000","last_active":"2024-09-21T11:50:21.064002+0000","last_peered":"2024-09-21T11:50:21.064002+0000","last_clean":"2024-09-21T11:50:21.064002+0000","last_became_active":"2024-09-21T11:49:08.609829+0000","last_became_peered":"2024-09-21T11:49:08.609829+0000","last_unstale":"2024-09-21T11:50:21.064002+0000","last_undegraded":"2024-09-21T11:50:21.064002+0000","last_fullsized":"2024-09-21T11:50:21.064002+0000","mapping_epoch":247,"log_start":"253'1700","ondisk_log_start":"253'1700","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.242703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420347904,"num_objects":6414,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6414,"num_write_kb":410496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":552,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.330509+0000","last_change":"2024-09-21T11:48:04.111199+0000","last_active":"2024-09-21T11:50:18.330509+0000","last_peered":"2024-09-21T11:50:18.330509+0000","last_clean":"2024-09-21T11:50:18.330509+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:50:18.330509+0000","last_undegraded":"2024-09-21T11:50:18.330509+0000","last_fullsized":"2024-09-21T11:50:18.330509+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:48:04.111115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.111115+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:25.112149+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":507,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341620+0000","last_change":"2024-09-21T11:49:08.611218+0000","last_active":"2024-09-21T11:50:18.341620+0000","last_peered":"2024-09-21T11:50:18.341620+0000","last_clean":"2024-09-21T11:50:18.341620+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:50:18.341620+0000","last_undegraded":"2024-09-21T11:50:18.341620+0000","last_fullsized":"2024-09-21T11:50:18.341620+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.611159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.611159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.323039+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":494,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.357919+0000","last_change":"2024-09-21T11:49:44.985594+0000","last_active":"2024-09-21T11:50:21.357919+0000","last_peered":"2024-09-21T11:50:21.357919+0000","last_clean":"2024-09-21T11:50:21.357919+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:50:21.357919+0000","last_undegraded":"2024-09-21T11:50:21.357919+0000","last_fullsized":"2024-09-21T11:50:21.357919+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:44.985234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:44.985234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:50.892123+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":474,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.358126+0000","last_change":"2024-09-21T11:49:21.071757+0000","last_active":"2024-09-21T11:50:21.358126+0000","last_peered":"2024-09-21T11:50:21.358126+0000","last_clean":"2024-09-21T11:50:21.358126+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:50:21.358126+0000","last_undegraded":"2024-09-21T11:50:21.358126+0000","last_fullsized":"2024-09-21T11:50:21.358126+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:21.071680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:21.071680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:25.890021+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"275'6612","reported_seq":6678,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.755587+0000","last_change":"2024-09-21T11:49:08.517117+0000","last_active":"2024-09-21T11:50:21.755587+0000","last_peered":"2024-09-21T11:50:21.755587+0000","last_clean":"2024-09-21T11:50:21.755587+0000","last_became_active":"2024-09-21T11:49:08.516494+0000","last_became_peered":"2024-09-21T11:49:08.516494+0000","last_unstale":"2024-09-21T11:50:21.755587+0000","last_undegraded":"2024-09-21T11:50:21.755587+0000","last_fullsized":"2024-09-21T11:50:21.755587+0000","mapping_epoch":247,"log_start":"252'600","ondisk_log_start":"252'600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":6012,"log_dups_size":0,"ondisk_log_size":6012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.332843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":433324032,"num_objects":6612,"num_object_clones":0,"num_object_copies":19836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6612,"num_write_kb":423168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":500,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.345394+0000","last_change":"2024-09-21T11:49:45.783231+0000","last_active":"2024-09-21T11:50:18.345394+0000","last_peered":"2024-09-21T11:50:18.345394+0000","last_clean":"2024-09-21T11:50:18.345394+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:50:18.345394+0000","last_undegraded":"2024-09-21T11:50:18.345394+0000","last_fullsized":"2024-09-21T11:50:18.345394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:49:45.783036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:45.783036+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.400285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":497,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.342632+0000","last_change":"2024-09-21T11:50:06.398255+0000","last_active":"2024-09-21T11:50:18.342632+0000","last_peered":"2024-09-21T11:50:18.342632+0000","last_clean":"2024-09-21T11:50:18.342632+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:50:18.342632+0000","last_undegraded":"2024-09-21T11:50:18.342632+0000","last_fullsized":"2024-09-21T11:50:18.342632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:50:06.398195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.398195+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:06.457914+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":594,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341971+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:50:18.341971+0000","last_peered":"2024-09-21T11:50:18.341971+0000","last_clean":"2024-09-21T11:50:18.341971+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:50:18.341971+0000","last_undegraded":"2024-09-21T11:50:18.341971+0000","last_fullsized":"2024-09-21T11:50:18.341971+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.444577+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":457,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.343884+0000","last_change":"2024-09-21T11:49:56.669610+0000","last_active":"2024-09-21T11:50:18.343884+0000","last_peered":"2024-09-21T11:50:18.343884+0000","last_clean":"2024-09-21T11:50:18.343884+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:50:18.343884+0000","last_undegraded":"2024-09-21T11:50:18.343884+0000","last_fullsized":"2024-09-21T11:50:18.343884+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:56.669547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:56.669547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:24.559358+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":477,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.340200+0000","last_change":"2024-09-21T11:48:17.129730+0000","last_active":"2024-09-21T11:50:18.340200+0000","last_peered":"2024-09-21T11:50:18.340200+0000","last_clean":"2024-09-21T11:50:18.340200+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:50:18.340200+0000","last_undegraded":"2024-09-21T11:50:18.340200+0000","last_fullsized":"2024-09-21T11:50:18.340200+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.129675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.129675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.393394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":508,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.352204+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:50:21.352204+0000","last_peered":"2024-09-21T11:50:21.352204+0000","last_clean":"2024-09-21T11:50:21.352204+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:50:21.352204+0000","last_undegraded":"2024-09-21T11:50:21.352204+0000","last_fullsized":"2024-09-21T11:50:21.352204+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:38.248209+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":483,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344772+0000","last_change":"2024-09-21T11:49:08.902428+0000","last_active":"2024-09-21T11:50:18.344772+0000","last_peered":"2024-09-21T11:50:18.344772+0000","last_clean":"2024-09-21T11:50:18.344772+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:50:18.344772+0000","last_undegraded":"2024-09-21T11:50:18.344772+0000","last_fullsized":"2024-09-21T11:50:18.344772+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.902382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.902382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.218309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":473,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341734+0000","last_change":"2024-09-21T11:49:43.464860+0000","last_active":"2024-09-21T11:50:18.341734+0000","last_peered":"2024-09-21T11:50:18.341734+0000","last_clean":"2024-09-21T11:50:18.341734+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:50:18.341734+0000","last_undegraded":"2024-09-21T11:50:18.341734+0000","last_fullsized":"2024-09-21T11:50:18.341734+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:43.464792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:43.464792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:48.801237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":501,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.364347+0000","last_change":"2024-09-21T11:46:28.973700+0000","last_active":"2024-09-21T11:50:21.364347+0000","last_peered":"2024-09-21T11:50:21.364347+0000","last_clean":"2024-09-21T11:50:21.364347+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:50:21.364347+0000","last_undegraded":"2024-09-21T11:50:21.364347+0000","last_fullsized":"2024-09-21T11:50:21.364347+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:28.973606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.973606+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.696833+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":469,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.342291+0000","last_change":"2024-09-21T11:50:02.438498+0000","last_active":"2024-09-21T11:50:18.342291+0000","last_peered":"2024-09-21T11:50:18.342291+0000","last_clean":"2024-09-21T11:50:18.342291+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:50:18.342291+0000","last_undegraded":"2024-09-21T11:50:18.342291+0000","last_fullsized":"2024-09-21T11:50:18.342291+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.436591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.436591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:19.559044+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":486,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.352026+0000","last_change":"2024-09-21T11:49:50.191966+0000","last_active":"2024-09-21T11:50:21.352026+0000","last_peered":"2024-09-21T11:50:21.352026+0000","last_clean":"2024-09-21T11:50:21.352026+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:50:21.352026+0000","last_undegraded":"2024-09-21T11:50:21.352026+0000","last_fullsized":"2024-09-21T11:50:21.352026+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:01.459336+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":476,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.355572+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:50:21.355572+0000","last_peered":"2024-09-21T11:50:21.355572+0000","last_clean":"2024-09-21T11:50:21.355572+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:50:21.355572+0000","last_undegraded":"2024-09-21T11:50:21.355572+0000","last_fullsized":"2024-09-21T11:50:21.355572+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:54.617405+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":482,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.346211+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:50:18.346211+0000","last_peered":"2024-09-21T11:50:18.346211+0000","last_clean":"2024-09-21T11:50:18.346211+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:50:18.346211+0000","last_undegraded":"2024-09-21T11:50:18.346211+0000","last_fullsized":"2024-09-21T11:50:18.346211+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:39.874682+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":479,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.363529+0000","last_change":"2024-09-21T11:49:22.528906+0000","last_active":"2024-09-21T11:50:21.363529+0000","last_peered":"2024-09-21T11:50:21.363529+0000","last_clean":"2024-09-21T11:50:21.363529+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:50:21.363529+0000","last_undegraded":"2024-09-21T11:50:21.363529+0000","last_fullsized":"2024-09-21T11:50:21.363529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:22.528831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:22.528831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.548018+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":581,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351870+0000","last_change":"2024-09-21T11:47:02.849839+0000","last_active":"2024-09-21T11:50:21.351870+0000","last_peered":"2024-09-21T11:50:21.351870+0000","last_clean":"2024-09-21T11:50:21.351870+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:50:21.351870+0000","last_undegraded":"2024-09-21T11:50:21.351870+0000","last_fullsized":"2024-09-21T11:50:21.351870+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:02.849786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:02.849786+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.365178+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":466,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.332770+0000","last_change":"2024-09-21T11:48:50.927661+0000","last_active":"2024-09-21T11:50:18.332770+0000","last_peered":"2024-09-21T11:50:18.332770+0000","last_clean":"2024-09-21T11:50:18.332770+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:50:18.332770+0000","last_undegraded":"2024-09-21T11:50:18.332770+0000","last_fullsized":"2024-09-21T11:50:18.332770+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:50.927613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.927613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":520,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.352159+0000","last_change":"2024-09-21T11:47:03.428846+0000","last_active":"2024-09-21T11:50:21.352159+0000","last_peered":"2024-09-21T11:50:21.352159+0000","last_clean":"2024-09-21T11:50:21.352159+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:50:21.352159+0000","last_undegraded":"2024-09-21T11:50:21.352159+0000","last_fullsized":"2024-09-21T11:50:21.352159+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:33.175857+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":469,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.340943+0000","last_change":"2024-09-21T11:49:58.400078+0000","last_active":"2024-09-21T11:50:18.340943+0000","last_peered":"2024-09-21T11:50:18.340943+0000","last_clean":"2024-09-21T11:50:18.340943+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:50:18.340943+0000","last_undegraded":"2024-09-21T11:50:18.340943+0000","last_fullsized":"2024-09-21T11:50:18.340943+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.400013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.400013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:58.776865+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":617,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.345183+0000","last_change":"2024-09-21T11:47:03.415793+0000","last_active":"2024-09-21T11:50:18.345183+0000","last_peered":"2024-09-21T11:50:18.345183+0000","last_clean":"2024-09-21T11:50:18.345183+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:50:18.345183+0000","last_undegraded":"2024-09-21T11:50:18.345183+0000","last_fullsized":"2024-09-21T11:50:18.345183+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:10.174971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:10.174971+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.015617+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":475,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.363917+0000","last_change":"2024-09-21T11:50:02.782565+0000","last_active":"2024-09-21T11:50:21.363917+0000","last_peered":"2024-09-21T11:50:21.363917+0000","last_clean":"2024-09-21T11:50:21.363917+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:50:21.363917+0000","last_undegraded":"2024-09-21T11:50:21.363917+0000","last_fullsized":"2024-09-21T11:50:21.363917+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.782475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.782475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:29.369093+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":562,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.331658+0000","last_change":"2024-09-21T11:48:33.033557+0000","last_active":"2024-09-21T11:50:18.331658+0000","last_peered":"2024-09-21T11:50:18.331658+0000","last_clean":"2024-09-21T11:50:18.331658+0000","last_became_active":"2024-09-21T11:48:33.022647+0000","last_became_peered":"2024-09-21T11:48:33.022647+0000","last_unstale":"2024-09-21T11:50:18.331658+0000","last_undegraded":"2024-09-21T11:50:18.331658+0000","last_fullsized":"2024-09-21T11:50:18.331658+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:54.032795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.032795+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.144714+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":462,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.332343+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:50:18.332343+0000","last_peered":"2024-09-21T11:50:18.332343+0000","last_clean":"2024-09-21T11:50:18.332343+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:50:18.332343+0000","last_undegraded":"2024-09-21T11:50:18.332343+0000","last_fullsized":"2024-09-21T11:50:18.332343+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.091776+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":163,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.340556+0000","last_change":"2024-09-21T11:49:58.034124+0000","last_active":"2024-09-21T11:50:18.340556+0000","last_peered":"2024-09-21T11:50:18.340556+0000","last_clean":"2024-09-21T11:50:18.340556+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:50:18.340556+0000","last_undegraded":"2024-09-21T11:50:18.340556+0000","last_fullsized":"2024-09-21T11:50:18.340556+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.034057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.034057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:05.285593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":452,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.353997+0000","last_change":"2024-09-21T11:50:02.329020+0000","last_active":"2024-09-21T11:50:21.353997+0000","last_peered":"2024-09-21T11:50:21.353997+0000","last_clean":"2024-09-21T11:50:21.353997+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:50:21.353997+0000","last_undegraded":"2024-09-21T11:50:21.353997+0000","last_fullsized":"2024-09-21T11:50:21.353997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.328926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.328926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.336948+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":174,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.342971+0000","last_change":"2024-09-21T11:47:03.404473+0000","last_active":"2024-09-21T11:50:18.342971+0000","last_peered":"2024-09-21T11:50:18.342971+0000","last_clean":"2024-09-21T11:50:18.342971+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:50:18.342971+0000","last_undegraded":"2024-09-21T11:50:18.342971+0000","last_fullsized":"2024-09-21T11:50:18.342971+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:26.649189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:26.649189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.922345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":477,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341461+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:50:18.341461+0000","last_peered":"2024-09-21T11:50:18.341461+0000","last_clean":"2024-09-21T11:50:18.341461+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:50:18.341461+0000","last_undegraded":"2024-09-21T11:50:18.341461+0000","last_fullsized":"2024-09-21T11:50:18.341461+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:47.576218+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":486,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.354208+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:50:21.354208+0000","last_peered":"2024-09-21T11:50:21.354208+0000","last_clean":"2024-09-21T11:50:21.354208+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:50:21.354208+0000","last_undegraded":"2024-09-21T11:50:21.354208+0000","last_fullsized":"2024-09-21T11:50:21.354208+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.838895+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":490,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.363188+0000","last_change":"2024-09-21T11:48:50.487842+0000","last_active":"2024-09-21T11:50:21.363188+0000","last_peered":"2024-09-21T11:50:21.363188+0000","last_clean":"2024-09-21T11:50:21.363188+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:50:21.363188+0000","last_undegraded":"2024-09-21T11:50:21.363188+0000","last_fullsized":"2024-09-21T11:50:21.363188+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:48:50.487786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.487786+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.708437+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":499,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.353612+0000","last_change":"2024-09-21T11:49:07.148305+0000","last_active":"2024-09-21T11:50:21.353612+0000","last_peered":"2024-09-21T11:50:21.353612+0000","last_clean":"2024-09-21T11:50:21.353612+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:50:21.353612+0000","last_undegraded":"2024-09-21T11:50:21.353612+0000","last_fullsized":"2024-09-21T11:50:21.353612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:49:07.148241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.148241+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:19.446103+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":483,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344847+0000","last_change":"2024-09-21T11:49:12.981959+0000","last_active":"2024-09-21T11:50:18.344847+0000","last_peered":"2024-09-21T11:50:18.344847+0000","last_clean":"2024-09-21T11:50:18.344847+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:50:18.344847+0000","last_undegraded":"2024-09-21T11:50:18.344847+0000","last_fullsized":"2024-09-21T11:50:18.344847+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:12.981878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:12.981878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:34.068456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":453,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.339576+0000","last_change":"2024-09-21T11:48:44.639778+0000","last_active":"2024-09-21T11:50:18.339576+0000","last_peered":"2024-09-21T11:50:18.339576+0000","last_clean":"2024-09-21T11:50:18.339576+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:50:18.339576+0000","last_undegraded":"2024-09-21T11:50:18.339576+0000","last_fullsized":"2024-09-21T11:50:18.339576+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:44.639638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.639638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.393642+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":460,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.354366+0000","last_change":"2024-09-21T11:49:08.171420+0000","last_active":"2024-09-21T11:50:21.354366+0000","last_peered":"2024-09-21T11:50:21.354366+0000","last_clean":"2024-09-21T11:50:21.354366+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:50:21.354366+0000","last_undegraded":"2024-09-21T11:50:21.354366+0000","last_fullsized":"2024-09-21T11:50:21.354366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.171357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.171357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:24.012559+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":519,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351165+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:50:21.351165+0000","last_peered":"2024-09-21T11:50:21.351165+0000","last_clean":"2024-09-21T11:50:21.351165+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:50:21.351165+0000","last_undegraded":"2024-09-21T11:50:21.351165+0000","last_fullsized":"2024-09-21T11:50:21.351165+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.196992+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":463,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.364442+0000","last_change":"2024-09-21T11:42:27.512392+0000","last_active":"2024-09-21T11:50:21.364442+0000","last_peered":"2024-09-21T11:50:21.364442+0000","last_clean":"2024-09-21T11:50:21.364442+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:50:21.364442+0000","last_undegraded":"2024-09-21T11:50:21.364442+0000","last_fullsized":"2024-09-21T11:50:21.364442+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:27.511729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.511729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:24.557879+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":461,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.338594+0000","last_change":"2024-09-21T11:48:09.104826+0000","last_active":"2024-09-21T11:50:18.338594+0000","last_peered":"2024-09-21T11:50:18.338594+0000","last_clean":"2024-09-21T11:50:18.338594+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:50:18.338594+0000","last_undegraded":"2024-09-21T11:50:18.338594+0000","last_fullsized":"2024-09-21T11:50:18.338594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:09.104767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.104767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.382787+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":457,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341064+0000","last_change":"2024-09-21T11:50:05.419390+0000","last_active":"2024-09-21T11:50:18.341064+0000","last_peered":"2024-09-21T11:50:18.341064+0000","last_clean":"2024-09-21T11:50:18.341064+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:50:18.341064+0000","last_undegraded":"2024-09-21T11:50:18.341064+0000","last_fullsized":"2024-09-21T11:50:18.341064+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.419329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.419329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:15.874052+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":163,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.344415+0000","last_change":"2024-09-21T11:49:08.636912+0000","last_active":"2024-09-21T11:50:18.344415+0000","last_peered":"2024-09-21T11:50:18.344415+0000","last_clean":"2024-09-21T11:50:18.344415+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:50:18.344415+0000","last_undegraded":"2024-09-21T11:50:18.344415+0000","last_fullsized":"2024-09-21T11:50:18.344415+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.636860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.636860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:22.892875+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":486,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.351533+0000","last_change":"2024-09-21T11:49:12.177359+0000","last_active":"2024-09-21T11:50:21.351533+0000","last_peered":"2024-09-21T11:50:21.351533+0000","last_clean":"2024-09-21T11:50:21.351533+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:50:21.351533+0000","last_undegraded":"2024-09-21T11:50:21.351533+0000","last_fullsized":"2024-09-21T11:50:21.351533+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:12.177303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:12.177303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":550,"reported_epoch":274,"state":"active+clean","last_fresh":"2024-09-21T11:50:18.341487+0000","last_change":"2024-09-21T11:48:45.147257+0000","last_active":"2024-09-21T11:50:18.341487+0000","last_peered":"2024-09-21T11:50:18.341487+0000","last_clean":"2024-09-21T11:50:18.341487+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:50:18.341487+0000","last_undegraded":"2024-09-21T11:50:18.341487+0000","last_fullsized":"2024-09-21T11:50:18.341487+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:45.147190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:45.147190+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.355453+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":480,"reported_epoch":275,"state":"active+clean","last_fresh":"2024-09-21T11:50:21.364121+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:50:21.364121+0000","last_peered":"2024-09-21T11:50:21.364121+0000","last_clean":"2024-09-21T11:50:21.364121+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:50:21.364121+0000","last_undegraded":"2024-09-21T11:50:21.364121+0000","last_fullsized":"2024-09-21T11:50:21.364121+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.689301+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":6835863552,"num_objects":104307,"num_object_clones":0,"num_object_copies":312921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104307,"num_write_kb":6675648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20476592128,"data_stored":20476592128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82807,"ondisk_log_size":82807,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1122,"num_read_kb":879,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":612,"num_read_kb":607,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234113,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4122540,"kb_used_data":3722072,"kb_used_omap":1,"kb_used_meta":400446,"kb_avail":89622612,"statfs":{"total":95995035648,"available":91773554688,"internally_reserved":0,"allocated":3811401728,"data_stored":3811037736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":410057162},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463046,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1557084,"kb_used_data":1241292,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":92188068,"statfs":{"total":95995035648,"available":94400581632,"internally_reserved":0,"allocated":1271083008,"data_stored":1270731648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":323353030},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724683,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4125420,"kb_used_data":3733276,"kb_used_omap":1,"kb_used_meta":392126,"kb_avail":89619732,"statfs":{"total":95995035648,"available":91770605568,"internally_reserved":0,"allocated":3822874624,"data_stored":3822505747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":401537478},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920913,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1532020,"kb_used_data":1234272,"kb_used_omap":1,"kb_used_meta":297726,"kb_avail":92213132,"statfs":{"total":95995035648,"available":94426247168,"internally_reserved":0,"allocated":1263894528,"data_stored":1263522313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":304871877},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117143,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3837680,"kb_used_data":3333084,"kb_used_omap":1,"kb_used_meta":504574,"kb_avail":89907472,"statfs":{"total":95995035648,"available":92065251328,"internally_reserved":0,"allocated":3413078016,"data_stored":3412715677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":516684231},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411485,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1691748,"kb_used_data":1245460,"kb_used_omap":1,"kb_used_meta":446270,"kb_avail":92053404,"statfs":{"total":95995035648,"available":94262685696,"internally_reserved":0,"allocated":1275351040,"data_stored":1274990649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":456980933},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4151428,"kb_used_data":3727348,"kb_used_omap":1,"kb_used_meta":424062,"kb_avail":89593724,"statfs":{"total":95995035648,"available":91743973376,"internally_reserved":0,"allocated":3816804352,"data_stored":3816411743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":434239942},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738538,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2036996,"kb_used_data":1676784,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":91708156,"statfs":{"total":95995035648,"available":93909151744,"internally_reserved":0,"allocated":1717026816,"data_stored":1716645918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":368835013},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1715798016,"data_stored":1715798016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3821469696,"data_stored":3821469696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1282867200,"data_stored":1282867200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3436707840,"data_stored":3436707840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1269891072,"data_stored":1269891072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3838246912,"data_stored":3838246912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1279066112,"data_stored":1279066112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3832545280,"data_stored":3832545280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:50:26.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:50:26.194 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:26.224 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:26.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:26.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:26.282 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 73 16 109488 109472 93.6877 58.375 0.0123618 0.0106688 2024-09-21T11:50:26.326 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:26.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:26.363 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:26.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:26.453 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:50:26.453 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:50:26.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:26.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:26.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:26.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:26.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[36454]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:50:26.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T11:50:26.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[36454]: 3.17 scrub starts 2024-09-21T11:50:26.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[36454]: 3.17 scrub ok 2024-09-21T11:50:26.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T11:50:26.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[36454]: 4.1c scrub starts 2024-09-21T11:50:26.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[36454]: 4.1c scrub ok 2024-09-21T11:50:26.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[29710]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:50:26.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T11:50:26.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[29710]: 3.17 scrub starts 2024-09-21T11:50:26.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[29710]: 3.17 scrub ok 2024-09-21T11:50:26.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T11:50:26.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[29710]: 4.1c scrub starts 2024-09-21T11:50:26.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:26 smithi043 ceph-mon[29710]: 4.1c scrub ok 2024-09-21T11:50:26.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:26 smithi175 ceph-mon[32613]: from='client.25459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:50:26.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:26 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T11:50:26.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:26 smithi175 ceph-mon[32613]: 3.17 scrub starts 2024-09-21T11:50:26.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:26 smithi175 ceph-mon[32613]: 3.17 scrub ok 2024-09-21T11:50:26.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:26 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T11:50:26.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:26 smithi175 ceph-mon[32613]: 4.1c scrub starts 2024-09-21T11:50:26.695 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:26 smithi175 ceph-mon[32613]: 4.1c scrub ok 2024-09-21T11:50:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:26.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:26.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:26.725 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:26.776 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:26.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:26.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:26.837 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:26.837 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:26.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:26.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:26.980 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:26.981 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:27.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:27.123 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:27.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:27.124 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:27.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:27.264 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:27.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:27.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:27.273 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:50:27.282 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 74 16 110541 110525 93.311 65.8125 0.00758608 0.0107124 2024-09-21T11:50:27.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:27.402 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:27.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:27.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:27.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:27 smithi175 ceph-mon[32613]: pgmap v504: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 670 B/s rd, 46 MiB/s wr, 742 op/s 2024-09-21T11:50:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:27 smithi175 ceph-mon[32613]: 9.d scrub starts 2024-09-21T11:50:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:27 smithi175 ceph-mon[32613]: 9.9 scrub ok 2024-09-21T11:50:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:27 smithi175 ceph-mon[32613]: 6.1b scrub starts 2024-09-21T11:50:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:27 smithi175 ceph-mon[32613]: 6.1b scrub ok 2024-09-21T11:50:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:27.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:27.546 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:27.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:27.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:27 smithi043 ceph-mon[36454]: pgmap v504: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 670 B/s rd, 46 MiB/s wr, 742 op/s 2024-09-21T11:50:27.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:27 smithi043 ceph-mon[36454]: 9.d scrub starts 2024-09-21T11:50:27.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:27 smithi043 ceph-mon[36454]: 9.9 scrub ok 2024-09-21T11:50:27.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:27 smithi043 ceph-mon[36454]: 6.1b scrub starts 2024-09-21T11:50:27.664 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:27 smithi043 ceph-mon[36454]: 6.1b scrub ok 2024-09-21T11:50:27.666 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:27 smithi043 ceph-mon[29710]: pgmap v504: 148 pgs: 148 active+clean; 6.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 670 B/s rd, 46 MiB/s wr, 742 op/s 2024-09-21T11:50:27.666 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:27 smithi043 ceph-mon[29710]: 9.d scrub starts 2024-09-21T11:50:27.666 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:27 smithi043 ceph-mon[29710]: 9.9 scrub ok 2024-09-21T11:50:27.666 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:27 smithi043 ceph-mon[29710]: 6.1b scrub starts 2024-09-21T11:50:27.666 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:27 smithi043 ceph-mon[29710]: 6.1b scrub ok 2024-09-21T11:50:27.679 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:27.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:27.685 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:27.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:27.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:27.832 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:27.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:27.994 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:28.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:28.129 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:28.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:28.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:28.195 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:50:28.196 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:50:28.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:28.275 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:28.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:28.276 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:28.282 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 75 16 111493 111477 92.8603 59.5 0.0112563 0.0107642 2024-09-21T11:50:28.377 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:28.410 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:50:28.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:28.411 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:28.411 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:28.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[36454]: 9.8 scrub ok 2024-09-21T11:50:28.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[36454]: 6.1c scrub starts 2024-09-21T11:50:28.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[36454]: 6.1c scrub ok 2024-09-21T11:50:28.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[36454]: 3.1c scrub starts 2024-09-21T11:50:28.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[36454]: 3.1c scrub ok 2024-09-21T11:50:28.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3258244003' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:28.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[36454]: 9.4 scrub starts 2024-09-21T11:50:28.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[29710]: 9.8 scrub ok 2024-09-21T11:50:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[29710]: 6.1c scrub starts 2024-09-21T11:50:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[29710]: 6.1c scrub ok 2024-09-21T11:50:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[29710]: 3.1c scrub starts 2024-09-21T11:50:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[29710]: 3.1c scrub ok 2024-09-21T11:50:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3258244003' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:28.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:28 smithi043 ceph-mon[29710]: 9.4 scrub starts 2024-09-21T11:50:28.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:28.549 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:28.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:28.550 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:28.650 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:28.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:50:28.689 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:28.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:28.690 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:28 smithi175 ceph-mon[32613]: 9.8 scrub ok 2024-09-21T11:50:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:28 smithi175 ceph-mon[32613]: 6.1c scrub starts 2024-09-21T11:50:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:28 smithi175 ceph-mon[32613]: 6.1c scrub ok 2024-09-21T11:50:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:28 smithi175 ceph-mon[32613]: 3.1c scrub starts 2024-09-21T11:50:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:28 smithi175 ceph-mon[32613]: 3.1c scrub ok 2024-09-21T11:50:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:28 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3258244003' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:28 smithi175 ceph-mon[32613]: 9.4 scrub starts 2024-09-21T11:50:28.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:28.819 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:28.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:28.822 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:28.922 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:28.946 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:28.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:28.947 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:29.048 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:29.084 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:29.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:29.086 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:29.186 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:29.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:29.228 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:29.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:29.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 76 16 112475 112459 92.4461 61.375 0.0153389 0.0108121 2024-09-21T11:50:29.330 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:29.373 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:29.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:29.374 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:29.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:29.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:29.514 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:29.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:29.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:29.657 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:29.657 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:29.679 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:50:29.680 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:50:29.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:29 smithi175 ceph-mon[32613]: pgmap v505: 148 pgs: 148 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-21T11:50:29.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:29 smithi175 ceph-mon[32613]: from='client.25471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:50:29.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:29 smithi175 ceph-mon[32613]: 9.d scrub ok 2024-09-21T11:50:29.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:29 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:50:29.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:29 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:50:29.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3258244003' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:50:29.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:29 smithi175 ceph-mon[32613]: osdmap e277: 8 total, 8 up, 8 in 2024-09-21T11:50:29.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:29 smithi175 ceph-mon[32613]: 3.6 scrub starts 2024-09-21T11:50:29.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:29 smithi175 ceph-mon[32613]: 3.6 scrub ok 2024-09-21T11:50:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[29710]: pgmap v505: 148 pgs: 148 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-21T11:50:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[29710]: from='client.25471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:50:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[29710]: 9.d scrub ok 2024-09-21T11:50:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:50:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:50:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3258244003' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:50:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[29710]: osdmap e277: 8 total, 8 up, 8 in 2024-09-21T11:50:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[29710]: 3.6 scrub starts 2024-09-21T11:50:29.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[29710]: 3.6 scrub ok 2024-09-21T11:50:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[36454]: pgmap v505: 148 pgs: 148 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-21T11:50:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[36454]: from='client.25471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:50:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[36454]: 9.d scrub ok 2024-09-21T11:50:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:50:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:50:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3258244003' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:50:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[36454]: osdmap e277: 8 total, 8 up, 8 in 2024-09-21T11:50:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[36454]: 3.6 scrub starts 2024-09-21T11:50:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:29 smithi043 ceph-mon[36454]: 3.6 scrub ok 2024-09-21T11:50:29.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:29.793 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:29.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:29.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:29.895 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:29.916 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:29.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:29.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:30.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:30.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:30.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:30.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:30.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:30.204 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:30.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 77 16 113462 113446 92.0466 61.6875 0.0245085 0.0108571 2024-09-21T11:50:30.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:30.345 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:30.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:30.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:30.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:30.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:30.482 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:30.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:30.534 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:50:30.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:30.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:30.627 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:30.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:30.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:30 smithi043 ceph-mon[29710]: 4.5 scrub starts 2024-09-21T11:50:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:30 smithi043 ceph-mon[29710]: 4.5 scrub ok 2024-09-21T11:50:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:30 smithi043 ceph-mon[29710]: 5.3 scrub starts 2024-09-21T11:50:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:30 smithi043 ceph-mon[29710]: 5.3 scrub ok 2024-09-21T11:50:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:30 smithi043 ceph-mon[29710]: 9.7 scrub starts 2024-09-21T11:50:30.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:30 smithi043 ceph-mon[36454]: 4.5 scrub starts 2024-09-21T11:50:30.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:30 smithi043 ceph-mon[36454]: 4.5 scrub ok 2024-09-21T11:50:30.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:30 smithi043 ceph-mon[36454]: 5.3 scrub starts 2024-09-21T11:50:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:30 smithi043 ceph-mon[36454]: 5.3 scrub ok 2024-09-21T11:50:30.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:30 smithi043 ceph-mon[36454]: 9.7 scrub starts 2024-09-21T11:50:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:30.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:30.751 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:30.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:30 smithi175 ceph-mon[32613]: 4.5 scrub starts 2024-09-21T11:50:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:30 smithi175 ceph-mon[32613]: 4.5 scrub ok 2024-09-21T11:50:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:30 smithi175 ceph-mon[32613]: 5.3 scrub starts 2024-09-21T11:50:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:30 smithi175 ceph-mon[32613]: 5.3 scrub ok 2024-09-21T11:50:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:30 smithi175 ceph-mon[32613]: 9.7 scrub starts 2024-09-21T11:50:30.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:30.890 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:30.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:30.891 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:30.911 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:30.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:31.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:31.029 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:31.130 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:31.167 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:31.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:31.168 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:31.269 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:31.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 78 16 114420 114404 91.6342 59.875 0.0321117 0.0109078 2024-09-21T11:50:31.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:31.309 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:31.309 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:31.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:31.451 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:31.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:31.452 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:31.514 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:50:31.514 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:50:31.553 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:31.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:31.585 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:31.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:31.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:31.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:31.717 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:31.718 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[36454]: pgmap v507: 148 pgs: 148 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 42 MiB/s wr, 664 op/s 2024-09-21T11:50:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[36454]: 9.4 scrub ok 2024-09-21T11:50:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[36454]: 5.15 scrub starts 2024-09-21T11:50:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[36454]: 5.15 scrub ok 2024-09-21T11:50:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[36454]: 5.1d scrub starts 2024-09-21T11:50:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[36454]: 5.1d scrub ok 2024-09-21T11:50:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[29710]: pgmap v507: 148 pgs: 148 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 42 MiB/s wr, 664 op/s 2024-09-21T11:50:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[29710]: 9.4 scrub ok 2024-09-21T11:50:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[29710]: 5.15 scrub starts 2024-09-21T11:50:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[29710]: 5.15 scrub ok 2024-09-21T11:50:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[29710]: 5.1d scrub starts 2024-09-21T11:50:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[29710]: 5.1d scrub ok 2024-09-21T11:50:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:31 smithi175 ceph-mon[32613]: pgmap v507: 148 pgs: 148 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 42 MiB/s wr, 664 op/s 2024-09-21T11:50:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:31 smithi175 ceph-mon[32613]: 9.4 scrub ok 2024-09-21T11:50:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:31 smithi175 ceph-mon[32613]: 5.15 scrub starts 2024-09-21T11:50:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:31 smithi175 ceph-mon[32613]: 5.15 scrub ok 2024-09-21T11:50:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:31 smithi175 ceph-mon[32613]: 5.1d scrub starts 2024-09-21T11:50:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:31 smithi175 ceph-mon[32613]: 5.1d scrub ok 2024-09-21T11:50:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:31.762 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:50:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:50:31.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:31.860 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:31.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:31.861 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:31.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:31.994 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:31.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:31.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:32.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:32.134 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:32.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:32.135 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:32.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:32.272 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:32.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:32.273 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:32.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 79 16 115526 115510 91.3493 69.125 0.00624198 0.0109421 2024-09-21T11:50:32.373 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:32.398 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:32.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:32.399 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:32.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:32.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:32.537 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:32.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:32.638 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:32.658 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:32.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:32.659 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: 5.18 scrub starts 2024-09-21T11:50:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: 5.18 scrub ok 2024-09-21T11:50:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: 6.3 scrub starts 2024-09-21T11:50:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: 6.3 scrub ok 2024-09-21T11:50:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: 3.10 scrub starts 2024-09-21T11:50:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: 3.10 scrub ok 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: 4.3 scrub starts 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: 4.3 scrub ok 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: 5.b scrub starts 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[29710]: 5.b scrub ok 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: 5.18 scrub starts 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: 5.18 scrub ok 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: 6.3 scrub starts 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: 6.3 scrub ok 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: 3.10 scrub starts 2024-09-21T11:50:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: 3.10 scrub ok 2024-09-21T11:50:32.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: 4.3 scrub starts 2024-09-21T11:50:32.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: 4.3 scrub ok 2024-09-21T11:50:32.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: 5.b scrub starts 2024-09-21T11:50:32.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:32 smithi043 ceph-mon[36454]: 5.b scrub ok 2024-09-21T11:50:32.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: 5.18 scrub starts 2024-09-21T11:50:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: 5.18 scrub ok 2024-09-21T11:50:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: 6.3 scrub starts 2024-09-21T11:50:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: 6.3 scrub ok 2024-09-21T11:50:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: 3.10 scrub starts 2024-09-21T11:50:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: 3.10 scrub ok 2024-09-21T11:50:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: 4.3 scrub starts 2024-09-21T11:50:32.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: 4.3 scrub ok 2024-09-21T11:50:32.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: 5.b scrub starts 2024-09-21T11:50:32.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:32 smithi175 ceph-mon[32613]: 5.b scrub ok 2024-09-21T11:50:32.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:32.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:32.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:32.944 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:32.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:32.945 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:33.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:33.067 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:33.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:33.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:33.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:33.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:33.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:33.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:50:33.282342+0000 min lat: 0.00137184 max lat: 0.17025 avg lat: 0.0109874 2024-09-21T11:50:33.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:50:33.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 80 16 116466 116450 90.9418 58.75 0.00416563 0.0109874 2024-09-21T11:50:33.293 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:33.314 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:33.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:33.315 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:33.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:33.459 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:33.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:33.460 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:33.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:33.583 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:33.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:33.584 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:33.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:33.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:33.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:33 smithi043 ceph-mon[29710]: pgmap v508: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.0 KiB/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-21T11:50:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:33 smithi043 ceph-mon[29710]: 6.1e scrub starts 2024-09-21T11:50:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:33 smithi043 ceph-mon[29710]: 6.1e scrub ok 2024-09-21T11:50:33.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:33 smithi043 ceph-mon[29710]: 9.7 scrub ok 2024-09-21T11:50:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:33 smithi043 ceph-mon[36454]: pgmap v508: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.0 KiB/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-21T11:50:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:33 smithi043 ceph-mon[36454]: 6.1e scrub starts 2024-09-21T11:50:33.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:33 smithi043 ceph-mon[36454]: 6.1e scrub ok 2024-09-21T11:50:33.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:33 smithi043 ceph-mon[36454]: 9.7 scrub ok 2024-09-21T11:50:33.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:33 smithi175 ceph-mon[32613]: pgmap v508: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.0 KiB/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-21T11:50:33.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:33 smithi175 ceph-mon[32613]: 6.1e scrub starts 2024-09-21T11:50:33.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:33 smithi175 ceph-mon[32613]: 6.1e scrub ok 2024-09-21T11:50:33.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:33 smithi175 ceph-mon[32613]: 9.7 scrub ok 2024-09-21T11:50:33.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:33.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:33.867 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:33.867 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:33.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:33.999 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:33.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:34.000 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:34.075 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-09-21T11:50:27.971016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7184783899,"num_objects":109821,"num_object_clones":0,"num_object_copies":329463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109821,"num_whiteouts":0,"num_read":1882,"num_read_kb":1739,"num_write":110457,"num_write_kb":7017481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":596864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83767,"ondisk_log_size":83767,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":24160220,"kb_used_data":20953692,"kb_used_omap":12,"kb_used_meta":3206387,"kb_avail":725800996,"statfs":{"total":767960285184,"available":743220219904,"internally_reserved":0,"allocated":21456580608,"data_stored":21453553601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":3283340850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":719650816,"num_objects":10981,"num_object_clones":0,"num_object_copies":32943,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10981,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10981,"num_write_kb":702784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002737"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":466,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357793+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:50:22.357793+0000","last_peered":"2024-09-21T11:50:22.357793+0000","last_clean":"2024-09-21T11:50:22.357793+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:50:22.357793+0000","last_undegraded":"2024-09-21T11:50:22.357793+0000","last_fullsized":"2024-09-21T11:50:22.357793+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.789381+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":537,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354362+0000","last_change":"2024-09-21T11:49:59.177549+0000","last_active":"2024-09-21T11:50:22.354362+0000","last_peered":"2024-09-21T11:50:22.354362+0000","last_clean":"2024-09-21T11:50:22.354362+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:50:22.354362+0000","last_undegraded":"2024-09-21T11:50:22.354362+0000","last_fullsized":"2024-09-21T11:50:22.354362+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:49:59.177497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:59.177497+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:01.997293+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":159,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.364576+0000","last_change":"2024-09-21T11:47:03.372676+0000","last_active":"2024-09-21T11:50:22.364576+0000","last_peered":"2024-09-21T11:50:22.364576+0000","last_clean":"2024-09-21T11:50:22.364576+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:50:22.364576+0000","last_undegraded":"2024-09-21T11:50:22.364576+0000","last_fullsized":"2024-09-21T11:50:22.364576+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.050152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.050152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.778217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":484,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355155+0000","last_change":"2024-09-21T11:49:08.053201+0000","last_active":"2024-09-21T11:50:22.355155+0000","last_peered":"2024-09-21T11:50:22.355155+0000","last_clean":"2024-09-21T11:50:22.355155+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:50:22.355155+0000","last_undegraded":"2024-09-21T11:50:22.355155+0000","last_fullsized":"2024-09-21T11:50:22.355155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.733987+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":470,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.427803+0000","last_change":"2024-09-21T11:50:26.427803+0000","last_active":"2024-09-21T11:50:26.427803+0000","last_peered":"2024-09-21T11:50:26.427803+0000","last_clean":"2024-09-21T11:50:26.427803+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:50:26.427803+0000","last_undegraded":"2024-09-21T11:50:26.427803+0000","last_fullsized":"2024-09-21T11:50:26.427803+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:26.427733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.427733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:34.278269+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":459,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361233+0000","last_change":"2024-09-21T11:46:55.804484+0000","last_active":"2024-09-21T11:50:22.361233+0000","last_peered":"2024-09-21T11:50:22.361233+0000","last_clean":"2024-09-21T11:50:22.361233+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:50:22.361233+0000","last_undegraded":"2024-09-21T11:50:22.361233+0000","last_fullsized":"2024-09-21T11:50:22.361233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.804419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.804419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:43.180856+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":505,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.362015+0000","last_change":"2024-09-21T11:50:03.300848+0000","last_active":"2024-09-21T11:50:22.362015+0000","last_peered":"2024-09-21T11:50:22.362015+0000","last_clean":"2024-09-21T11:50:22.362015+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:50:22.362015+0000","last_undegraded":"2024-09-21T11:50:22.362015+0000","last_fullsized":"2024-09-21T11:50:22.362015+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:50:03.300772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:03.300772+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.852364+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":480,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354268+0000","last_change":"2024-09-21T11:47:03.371710+0000","last_active":"2024-09-21T11:50:22.354268+0000","last_peered":"2024-09-21T11:50:22.354268+0000","last_clean":"2024-09-21T11:50:22.354268+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:50:22.354268+0000","last_undegraded":"2024-09-21T11:50:22.354268+0000","last_fullsized":"2024-09-21T11:50:22.354268+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:08.781842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:45:08.781842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:29.162811+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":465,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.366999+0000","last_change":"2024-09-21T11:48:47.077320+0000","last_active":"2024-09-21T11:50:22.366999+0000","last_peered":"2024-09-21T11:50:22.366999+0000","last_clean":"2024-09-21T11:50:22.366999+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:50:22.366999+0000","last_undegraded":"2024-09-21T11:50:22.366999+0000","last_fullsized":"2024-09-21T11:50:22.366999+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.077259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.077259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:13.765851+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":503,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355448+0000","last_change":"2024-09-21T11:49:15.028297+0000","last_active":"2024-09-21T11:50:22.355448+0000","last_peered":"2024-09-21T11:50:22.355448+0000","last_clean":"2024-09-21T11:50:22.355448+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:50:22.355448+0000","last_undegraded":"2024-09-21T11:50:22.355448+0000","last_fullsized":"2024-09-21T11:50:22.355448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:49:15.028208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:15.028208+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.470742+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":465,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.362433+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:50:22.362433+0000","last_peered":"2024-09-21T11:50:22.362433+0000","last_clean":"2024-09-21T11:50:22.362433+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:50:22.362433+0000","last_undegraded":"2024-09-21T11:50:22.362433+0000","last_fullsized":"2024-09-21T11:50:22.362433+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.828789+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":475,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357103+0000","last_change":"2024-09-21T11:48:44.523906+0000","last_active":"2024-09-21T11:50:22.357103+0000","last_peered":"2024-09-21T11:50:22.357103+0000","last_clean":"2024-09-21T11:50:22.357103+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:50:22.357103+0000","last_undegraded":"2024-09-21T11:50:22.357103+0000","last_fullsized":"2024-09-21T11:50:22.357103+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:44.523826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.523826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:11.691589+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":487,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.363782+0000","last_change":"2024-09-21T11:47:03.385245+0000","last_active":"2024-09-21T11:50:22.363782+0000","last_peered":"2024-09-21T11:50:22.363782+0000","last_clean":"2024-09-21T11:50:22.363782+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:50:22.363782+0000","last_undegraded":"2024-09-21T11:50:22.363782+0000","last_fullsized":"2024-09-21T11:50:22.363782+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.172734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.172734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:25.947771+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":517,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355703+0000","last_change":"2024-09-21T11:49:14.010678+0000","last_active":"2024-09-21T11:50:22.355703+0000","last_peered":"2024-09-21T11:50:22.355703+0000","last_clean":"2024-09-21T11:50:22.355703+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:50:22.355703+0000","last_undegraded":"2024-09-21T11:50:22.355703+0000","last_fullsized":"2024-09-21T11:50:22.355703+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.823398+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":479,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.364509+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:50:22.364509+0000","last_peered":"2024-09-21T11:50:22.364509+0000","last_clean":"2024-09-21T11:50:22.364509+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:50:22.364509+0000","last_undegraded":"2024-09-21T11:50:22.364509+0000","last_fullsized":"2024-09-21T11:50:22.364509+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:45.139772+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":475,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361458+0000","last_change":"2024-09-21T11:47:03.380856+0000","last_active":"2024-09-21T11:50:22.361458+0000","last_peered":"2024-09-21T11:50:22.361458+0000","last_clean":"2024-09-21T11:50:22.361458+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:50:22.361458+0000","last_undegraded":"2024-09-21T11:50:22.361458+0000","last_fullsized":"2024-09-21T11:50:22.361458+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:13.288522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:13.288522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:34.960084+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":471,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.358324+0000","last_change":"2024-09-21T11:46:52.035524+0000","last_active":"2024-09-21T11:50:22.358324+0000","last_peered":"2024-09-21T11:50:22.358324+0000","last_clean":"2024-09-21T11:50:22.358324+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:50:22.358324+0000","last_undegraded":"2024-09-21T11:50:22.358324+0000","last_fullsized":"2024-09-21T11:50:22.358324+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:52.035461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:52.035461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:31.176135+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":457,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357985+0000","last_change":"2024-09-21T11:43:32.639077+0000","last_active":"2024-09-21T11:50:22.357985+0000","last_peered":"2024-09-21T11:50:22.357985+0000","last_clean":"2024-09-21T11:50:22.357985+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:50:22.357985+0000","last_undegraded":"2024-09-21T11:50:22.357985+0000","last_fullsized":"2024-09-21T11:50:22.357985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:32.639005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:43:32.639005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:31.706363+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":598,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:25.966805+0000","last_change":"2024-09-21T11:50:25.966805+0000","last_active":"2024-09-21T11:50:25.966805+0000","last_peered":"2024-09-21T11:50:25.966805+0000","last_clean":"2024-09-21T11:50:25.966805+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:50:25.966805+0000","last_undegraded":"2024-09-21T11:50:25.966805+0000","last_fullsized":"2024-09-21T11:50:25.966805+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:50:25.966740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.966740+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.945863+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":167,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365397+0000","last_change":"2024-09-21T11:49:16.745001+0000","last_active":"2024-09-21T11:50:22.365397+0000","last_peered":"2024-09-21T11:50:22.365397+0000","last_clean":"2024-09-21T11:50:22.365397+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:50:22.365397+0000","last_undegraded":"2024-09-21T11:50:22.365397+0000","last_fullsized":"2024-09-21T11:50:22.365397+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:16.744931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:16.744931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.501502+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":462,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361723+0000","last_change":"2024-09-21T11:50:06.400378+0000","last_active":"2024-09-21T11:50:22.361723+0000","last_peered":"2024-09-21T11:50:22.361723+0000","last_clean":"2024-09-21T11:50:22.361723+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:50:22.361723+0000","last_undegraded":"2024-09-21T11:50:22.361723+0000","last_fullsized":"2024-09-21T11:50:22.361723+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:06.400307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.400307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.097684+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":488,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355568+0000","last_change":"2024-09-21T11:49:50.122297+0000","last_active":"2024-09-21T11:50:22.355568+0000","last_peered":"2024-09-21T11:50:22.355568+0000","last_clean":"2024-09-21T11:50:22.355568+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:50:22.355568+0000","last_undegraded":"2024-09-21T11:50:22.355568+0000","last_fullsized":"2024-09-21T11:50:22.355568+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.122227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.122227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:55.903276+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":532,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361387+0000","last_change":"2024-09-21T11:47:03.873307+0000","last_active":"2024-09-21T11:50:22.361387+0000","last_peered":"2024-09-21T11:50:22.361387+0000","last_clean":"2024-09-21T11:50:22.361387+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:50:22.361387+0000","last_undegraded":"2024-09-21T11:50:22.361387+0000","last_fullsized":"2024-09-21T11:50:22.361387+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:47:03.873239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.873239+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.476375+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":466,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355393+0000","last_change":"2024-09-21T11:48:51.931885+0000","last_active":"2024-09-21T11:50:22.355393+0000","last_peered":"2024-09-21T11:50:22.355393+0000","last_clean":"2024-09-21T11:50:22.355393+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:50:22.355393+0000","last_undegraded":"2024-09-21T11:50:22.355393+0000","last_fullsized":"2024-09-21T11:50:22.355393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:51.931832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:51.931832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:13.265721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":470,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361785+0000","last_change":"2024-09-21T11:46:47.611118+0000","last_active":"2024-09-21T11:50:22.361785+0000","last_peered":"2024-09-21T11:50:22.361785+0000","last_clean":"2024-09-21T11:50:22.361785+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:50:22.361785+0000","last_undegraded":"2024-09-21T11:50:22.361785+0000","last_fullsized":"2024-09-21T11:50:22.361785+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:46:47.611042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:47.611042+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:26.405632+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":171,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.364705+0000","last_change":"2024-09-21T11:50:02.920313+0000","last_active":"2024-09-21T11:50:22.364705+0000","last_peered":"2024-09-21T11:50:22.364705+0000","last_clean":"2024-09-21T11:50:22.364705+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:50:22.364705+0000","last_undegraded":"2024-09-21T11:50:22.364705+0000","last_fullsized":"2024-09-21T11:50:22.364705+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:02.920229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.920229+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:20.782582+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":508,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357748+0000","last_change":"2024-09-21T11:49:58.702620+0000","last_active":"2024-09-21T11:50:22.357748+0000","last_peered":"2024-09-21T11:50:22.357748+0000","last_clean":"2024-09-21T11:50:22.357748+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:50:22.357748+0000","last_undegraded":"2024-09-21T11:50:22.357748+0000","last_fullsized":"2024-09-21T11:50:22.357748+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:49:58.702544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.702544+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.427898+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":619,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:25.494268+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:50:25.494268+0000","last_peered":"2024-09-21T11:50:25.494268+0000","last_clean":"2024-09-21T11:50:25.494268+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:50:25.494268+0000","last_undegraded":"2024-09-21T11:50:25.494268+0000","last_fullsized":"2024-09-21T11:50:25.494268+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.417310+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"276'6942","reported_seq":7011,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.956294+0000","last_change":"2024-09-21T11:49:08.748763+0000","last_active":"2024-09-21T11:50:26.956294+0000","last_peered":"2024-09-21T11:50:26.956294+0000","last_clean":"2024-09-21T11:50:26.956294+0000","last_became_active":"2024-09-21T11:49:08.748359+0000","last_became_peered":"2024-09-21T11:49:08.748359+0000","last_unstale":"2024-09-21T11:50:26.956294+0000","last_undegraded":"2024-09-21T11:50:26.956294+0000","last_fullsized":"2024-09-21T11:50:26.956294+0000","mapping_epoch":247,"log_start":"253'2100","ondisk_log_start":"253'2100","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4842,"log_dups_size":0,"ondisk_log_size":4842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.761336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454950912,"num_objects":6942,"num_object_clones":0,"num_object_copies":20826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6942,"num_write_kb":444288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":201,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365540+0000","last_change":"2024-09-21T11:49:15.740456+0000","last_active":"2024-09-21T11:50:22.365540+0000","last_peered":"2024-09-21T11:50:22.365540+0000","last_clean":"2024-09-21T11:50:22.365540+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:50:22.365540+0000","last_undegraded":"2024-09-21T11:50:22.365540+0000","last_fullsized":"2024-09-21T11:50:22.365540+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:49:15.739582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:15.739582+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.931869+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":496,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.366802+0000","last_change":"2024-09-21T11:49:47.205216+0000","last_active":"2024-09-21T11:50:22.366802+0000","last_peered":"2024-09-21T11:50:22.366802+0000","last_clean":"2024-09-21T11:50:22.366802+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:50:22.366802+0000","last_undegraded":"2024-09-21T11:50:22.366802+0000","last_fullsized":"2024-09-21T11:50:22.366802+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.204889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.204889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:52.456856+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":592,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.358448+0000","last_change":"2024-09-21T11:47:01.933697+0000","last_active":"2024-09-21T11:50:22.358448+0000","last_peered":"2024-09-21T11:50:22.358448+0000","last_clean":"2024-09-21T11:50:22.358448+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:50:22.358448+0000","last_undegraded":"2024-09-21T11:50:22.358448+0000","last_fullsized":"2024-09-21T11:50:22.358448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:01.933648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:01.933648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:11.347125+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":483,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:25.708598+0000","last_change":"2024-09-21T11:50:25.708598+0000","last_active":"2024-09-21T11:50:25.708598+0000","last_peered":"2024-09-21T11:50:25.708598+0000","last_clean":"2024-09-21T11:50:25.708598+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:50:25.708598+0000","last_undegraded":"2024-09-21T11:50:25.708598+0000","last_fullsized":"2024-09-21T11:50:25.708598+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:28.981755+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"276'6780","reported_seq":6849,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:25.922995+0000","last_change":"2024-09-21T11:49:08.520005+0000","last_active":"2024-09-21T11:50:25.922995+0000","last_peered":"2024-09-21T11:50:25.922995+0000","last_clean":"2024-09-21T11:50:25.922995+0000","last_became_active":"2024-09-21T11:49:08.519622+0000","last_became_peered":"2024-09-21T11:49:08.519622+0000","last_unstale":"2024-09-21T11:50:25.922995+0000","last_undegraded":"2024-09-21T11:50:25.922995+0000","last_fullsized":"2024-09-21T11:50:25.922995+0000","mapping_epoch":247,"log_start":"253'2000","ondisk_log_start":"253'2000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4780,"log_dups_size":0,"ondisk_log_size":4780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":444334080,"num_objects":6780,"num_object_clones":0,"num_object_copies":20340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6780,"num_write_kb":433920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":529,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365908+0000","last_change":"2024-09-21T11:49:52.209526+0000","last_active":"2024-09-21T11:50:22.365908+0000","last_peered":"2024-09-21T11:50:22.365908+0000","last_clean":"2024-09-21T11:50:22.365908+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:50:22.365908+0000","last_undegraded":"2024-09-21T11:50:22.365908+0000","last_fullsized":"2024-09-21T11:50:22.365908+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:49:52.209433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:52.209433+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.566988+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":482,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.362048+0000","last_change":"2024-09-21T11:49:55.238709+0000","last_active":"2024-09-21T11:50:22.362048+0000","last_peered":"2024-09-21T11:50:22.362048+0000","last_clean":"2024-09-21T11:50:22.362048+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:50:22.362048+0000","last_undegraded":"2024-09-21T11:50:22.362048+0000","last_fullsized":"2024-09-21T11:50:22.362048+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:55.238302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:55.238302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:06.703143+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":466,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354567+0000","last_change":"2024-09-21T11:48:47.933145+0000","last_active":"2024-09-21T11:50:22.354567+0000","last_peered":"2024-09-21T11:50:22.354567+0000","last_clean":"2024-09-21T11:50:22.354567+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:50:22.354567+0000","last_undegraded":"2024-09-21T11:50:22.354567+0000","last_fullsized":"2024-09-21T11:50:22.354567+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.933099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.933099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.352011+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":454,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355321+0000","last_change":"2024-09-21T11:46:24.717355+0000","last_active":"2024-09-21T11:50:22.355321+0000","last_peered":"2024-09-21T11:50:22.355321+0000","last_clean":"2024-09-21T11:50:22.355321+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:50:22.355321+0000","last_undegraded":"2024-09-21T11:50:22.355321+0000","last_fullsized":"2024-09-21T11:50:22.355321+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.717285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.717285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.284555+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"276'6871","reported_seq":6940,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.962844+0000","last_change":"2024-09-21T11:49:08.609939+0000","last_active":"2024-09-21T11:50:26.962844+0000","last_peered":"2024-09-21T11:50:26.962844+0000","last_clean":"2024-09-21T11:50:26.962844+0000","last_became_active":"2024-09-21T11:49:08.609618+0000","last_became_peered":"2024-09-21T11:49:08.609618+0000","last_unstale":"2024-09-21T11:50:26.962844+0000","last_undegraded":"2024-09-21T11:50:26.962844+0000","last_fullsized":"2024-09-21T11:50:26.962844+0000","mapping_epoch":247,"log_start":"253'2000","ondisk_log_start":"253'2000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4871,"log_dups_size":0,"ondisk_log_size":4871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.213949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450297856,"num_objects":6871,"num_object_clones":0,"num_object_copies":20613,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6871,"num_write_kb":439744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":529,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354849+0000","last_change":"2024-09-21T11:49:10.053574+0000","last_active":"2024-09-21T11:50:22.354849+0000","last_peered":"2024-09-21T11:50:22.354849+0000","last_clean":"2024-09-21T11:50:22.354849+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:50:22.354849+0000","last_undegraded":"2024-09-21T11:50:22.354849+0000","last_fullsized":"2024-09-21T11:50:22.354849+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-21T11:49:10.053515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:10.053515+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:34.201371+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":504,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.358175+0000","last_change":"2024-09-21T11:49:23.515285+0000","last_active":"2024-09-21T11:50:22.358175+0000","last_peered":"2024-09-21T11:50:22.358175+0000","last_clean":"2024-09-21T11:50:22.358175+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:50:22.358175+0000","last_undegraded":"2024-09-21T11:50:22.358175+0000","last_fullsized":"2024-09-21T11:50:22.358175+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:49:23.515206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.515206+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.949694+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":624,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365779+0000","last_change":"2024-09-21T11:49:14.198995+0000","last_active":"2024-09-21T11:50:22.365779+0000","last_peered":"2024-09-21T11:50:22.365779+0000","last_clean":"2024-09-21T11:50:22.365779+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:50:22.365779+0000","last_undegraded":"2024-09-21T11:50:22.365779+0000","last_fullsized":"2024-09-21T11:50:22.365779+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:49:14.198804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.198804+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.344589+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":464,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.366547+0000","last_change":"2024-09-21T11:49:12.548705+0000","last_active":"2024-09-21T11:50:22.366547+0000","last_peered":"2024-09-21T11:50:22.366547+0000","last_clean":"2024-09-21T11:50:22.366547+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:50:22.366547+0000","last_undegraded":"2024-09-21T11:50:22.366547+0000","last_fullsized":"2024-09-21T11:50:22.366547+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:12.548650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:12.548650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:22.749399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"276'6809","reported_seq":6877,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.060633+0000","last_change":"2024-09-21T11:49:08.518112+0000","last_active":"2024-09-21T11:50:26.060633+0000","last_peered":"2024-09-21T11:50:26.060633+0000","last_clean":"2024-09-21T11:50:26.060633+0000","last_became_active":"2024-09-21T11:49:08.517344+0000","last_became_peered":"2024-09-21T11:49:08.517344+0000","last_unstale":"2024-09-21T11:50:26.060633+0000","last_undegraded":"2024-09-21T11:50:26.060633+0000","last_fullsized":"2024-09-21T11:50:26.060633+0000","mapping_epoch":247,"log_start":"252'1500","ondisk_log_start":"252'1500","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5309,"log_dups_size":0,"ondisk_log_size":5309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:20.684739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446234624,"num_objects":6809,"num_object_clones":0,"num_object_copies":20427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6809,"num_write_kb":435776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":186,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.369250+0000","last_change":"2024-09-21T11:47:03.398638+0000","last_active":"2024-09-21T11:50:22.369250+0000","last_peered":"2024-09-21T11:50:22.369250+0000","last_clean":"2024-09-21T11:50:22.369250+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:50:22.369250+0000","last_undegraded":"2024-09-21T11:50:22.369250+0000","last_fullsized":"2024-09-21T11:50:22.369250+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:12.227121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.227121+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:06.869295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":490,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.367095+0000","last_change":"2024-09-21T11:49:09.924450+0000","last_active":"2024-09-21T11:50:22.367095+0000","last_peered":"2024-09-21T11:50:22.367095+0000","last_clean":"2024-09-21T11:50:22.367095+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:50:22.367095+0000","last_undegraded":"2024-09-21T11:50:22.367095+0000","last_fullsized":"2024-09-21T11:50:22.367095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:09.924400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:09.924400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:29.238919+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":586,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357515+0000","last_change":"2024-09-21T11:42:19.440343+0000","last_active":"2024-09-21T11:50:22.357515+0000","last_peered":"2024-09-21T11:50:22.357515+0000","last_clean":"2024-09-21T11:50:22.357515+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:50:22.357515+0000","last_undegraded":"2024-09-21T11:50:22.357515+0000","last_fullsized":"2024-09-21T11:50:22.357515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:42:19.440252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:42:19.440252+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:28.738852+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":462,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357189+0000","last_change":"2024-09-21T11:49:07.464269+0000","last_active":"2024-09-21T11:50:22.357189+0000","last_peered":"2024-09-21T11:50:22.357189+0000","last_clean":"2024-09-21T11:50:22.357189+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:50:22.357189+0000","last_undegraded":"2024-09-21T11:50:22.357189+0000","last_fullsized":"2024-09-21T11:50:22.357189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.464204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.464204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:13.689854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"276'6839","reported_seq":6908,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:25.922736+0000","last_change":"2024-09-21T11:49:08.609660+0000","last_active":"2024-09-21T11:50:25.922736+0000","last_peered":"2024-09-21T11:50:25.922736+0000","last_clean":"2024-09-21T11:50:25.922736+0000","last_became_active":"2024-09-21T11:49:08.609226+0000","last_became_peered":"2024-09-21T11:49:08.609226+0000","last_unstale":"2024-09-21T11:50:25.922736+0000","last_undegraded":"2024-09-21T11:50:25.922736+0000","last_fullsized":"2024-09-21T11:50:25.922736+0000","mapping_epoch":247,"log_start":"253'2000","ondisk_log_start":"253'2000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:28.968012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":448200704,"num_objects":6839,"num_object_clones":0,"num_object_copies":20517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6839,"num_write_kb":437696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":512,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357777+0000","last_change":"2024-09-21T11:47:03.750064+0000","last_active":"2024-09-21T11:50:22.357777+0000","last_peered":"2024-09-21T11:50:22.357777+0000","last_clean":"2024-09-21T11:50:22.357777+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:50:22.357777+0000","last_undegraded":"2024-09-21T11:50:22.357777+0000","last_fullsized":"2024-09-21T11:50:22.357777+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-21T11:47:03.749962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.749962+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.702349+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":167,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365712+0000","last_change":"2024-09-21T11:49:59.954903+0000","last_active":"2024-09-21T11:50:22.365712+0000","last_peered":"2024-09-21T11:50:22.365712+0000","last_clean":"2024-09-21T11:50:22.365712+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:50:22.365712+0000","last_undegraded":"2024-09-21T11:50:22.365712+0000","last_fullsized":"2024-09-21T11:50:22.365712+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:59.954835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:59.954835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:15.305841+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":457,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.359432+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:50:22.359432+0000","last_peered":"2024-09-21T11:50:22.359432+0000","last_clean":"2024-09-21T11:50:22.359432+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:50:22.359432+0000","last_undegraded":"2024-09-21T11:50:22.359432+0000","last_fullsized":"2024-09-21T11:50:22.359432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.352185+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":159,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.364488+0000","last_change":"2024-09-21T11:47:03.379157+0000","last_active":"2024-09-21T11:50:22.364488+0000","last_peered":"2024-09-21T11:50:22.364488+0000","last_clean":"2024-09-21T11:50:22.364488+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:50:22.364488+0000","last_undegraded":"2024-09-21T11:50:22.364488+0000","last_fullsized":"2024-09-21T11:50:22.364488+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:43:37.819857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.819857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:32.828402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"276'6809","reported_seq":6952,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.752591+0000","last_change":"2024-09-21T11:50:26.274463+0000","last_active":"2024-09-21T11:50:26.752591+0000","last_peered":"2024-09-21T11:50:26.752591+0000","last_clean":"2024-09-21T11:50:26.752591+0000","last_became_active":"2024-09-21T11:49:08.516355+0000","last_became_peered":"2024-09-21T11:49:08.516355+0000","last_unstale":"2024-09-21T11:50:26.752591+0000","last_undegraded":"2024-09-21T11:50:26.752591+0000","last_fullsized":"2024-09-21T11:50:26.752591+0000","mapping_epoch":247,"log_start":"252'800","ondisk_log_start":"252'800","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6771","last_scrub_stamp":"2024-09-21T11:50:26.274165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.274165+0000","objects_scrubbed":6698,"log_size":6009,"log_dups_size":0,"ondisk_log_size":6009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:39.313808+0000","scrub_duration":2483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446234624,"num_objects":6809,"num_object_clones":0,"num_object_copies":20427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6809,"num_write_kb":435776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":240,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365661+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:50:22.365661+0000","last_peered":"2024-09-21T11:50:22.365661+0000","last_clean":"2024-09-21T11:50:22.365661+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:50:22.365661+0000","last_undegraded":"2024-09-21T11:50:22.365661+0000","last_fullsized":"2024-09-21T11:50:22.365661+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:39.781729+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":465,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.356038+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:50:22.356038+0000","last_peered":"2024-09-21T11:50:22.356038+0000","last_clean":"2024-09-21T11:50:22.356038+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:50:22.356038+0000","last_undegraded":"2024-09-21T11:50:22.356038+0000","last_fullsized":"2024-09-21T11:50:22.356038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:41.126102+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":498,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365126+0000","last_change":"2024-09-21T11:47:20.000195+0000","last_active":"2024-09-21T11:50:22.365126+0000","last_peered":"2024-09-21T11:50:22.365126+0000","last_clean":"2024-09-21T11:50:22.365126+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:50:22.365126+0000","last_undegraded":"2024-09-21T11:50:22.365126+0000","last_fullsized":"2024-09-21T11:50:22.365126+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:47:20.000051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.000051+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:11.774530+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":471,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.358463+0000","last_change":"2024-09-21T11:47:03.385173+0000","last_active":"2024-09-21T11:50:22.358463+0000","last_peered":"2024-09-21T11:50:22.358463+0000","last_clean":"2024-09-21T11:50:22.358463+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:50:22.358463+0000","last_undegraded":"2024-09-21T11:50:22.358463+0000","last_fullsized":"2024-09-21T11:50:22.358463+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:44:41.300618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:44:41.300618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:32.712434+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":446,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361026+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:50:22.361026+0000","last_peered":"2024-09-21T11:50:22.361026+0000","last_clean":"2024-09-21T11:50:22.361026+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:50:22.361026+0000","last_undegraded":"2024-09-21T11:50:22.361026+0000","last_fullsized":"2024-09-21T11:50:22.361026+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.205651+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"276'6884","reported_seq":6953,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.029148+0000","last_change":"2024-09-21T11:49:08.518054+0000","last_active":"2024-09-21T11:50:26.029148+0000","last_peered":"2024-09-21T11:50:26.029148+0000","last_clean":"2024-09-21T11:50:26.029148+0000","last_became_active":"2024-09-21T11:49:08.517758+0000","last_became_peered":"2024-09-21T11:49:08.517758+0000","last_unstale":"2024-09-21T11:50:26.029148+0000","last_undegraded":"2024-09-21T11:50:26.029148+0000","last_fullsized":"2024-09-21T11:50:26.029148+0000","mapping_epoch":247,"log_start":"252'1600","ondisk_log_start":"252'1600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.722671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451149824,"num_objects":6884,"num_object_clones":0,"num_object_copies":20652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6884,"num_write_kb":440576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":110,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355979+0000","last_change":"2024-09-21T11:48:33.075081+0000","last_active":"2024-09-21T11:50:22.355979+0000","last_peered":"2024-09-21T11:50:22.355979+0000","last_clean":"2024-09-21T11:50:22.355979+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:50:22.355979+0000","last_undegraded":"2024-09-21T11:50:22.355979+0000","last_fullsized":"2024-09-21T11:50:22.355979+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.996705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":163,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.366149+0000","last_change":"2024-09-21T11:47:03.391283+0000","last_active":"2024-09-21T11:50:22.366149+0000","last_peered":"2024-09-21T11:50:22.366149+0000","last_clean":"2024-09-21T11:50:22.366149+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:50:22.366149+0000","last_undegraded":"2024-09-21T11:50:22.366149+0000","last_fullsized":"2024-09-21T11:50:22.366149+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:27.226869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:27.226869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:31.825843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":492,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354186+0000","last_change":"2024-09-21T11:48:48.994249+0000","last_active":"2024-09-21T11:50:22.354186+0000","last_peered":"2024-09-21T11:50:22.354186+0000","last_clean":"2024-09-21T11:50:22.354186+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:50:22.354186+0000","last_undegraded":"2024-09-21T11:50:22.354186+0000","last_fullsized":"2024-09-21T11:50:22.354186+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:48:48.994172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:48:48.994172+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:15.864789+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":474,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354532+0000","last_change":"2024-09-21T11:49:47.008541+0000","last_active":"2024-09-21T11:50:22.354532+0000","last_peered":"2024-09-21T11:50:22.354532+0000","last_clean":"2024-09-21T11:50:22.354532+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:50:22.354532+0000","last_undegraded":"2024-09-21T11:50:22.354532+0000","last_fullsized":"2024-09-21T11:50:22.354532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.008369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.008369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:53.561004+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":469,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.367422+0000","last_change":"2024-09-21T11:49:54.388131+0000","last_active":"2024-09-21T11:50:22.367422+0000","last_peered":"2024-09-21T11:50:22.367422+0000","last_clean":"2024-09-21T11:50:22.367422+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:50:22.367422+0000","last_undegraded":"2024-09-21T11:50:22.367422+0000","last_fullsized":"2024-09-21T11:50:22.367422+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.388068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.388068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:57.353255+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"276'6955","reported_seq":7024,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.751656+0000","last_change":"2024-09-21T11:49:08.515501+0000","last_active":"2024-09-21T11:50:26.751656+0000","last_peered":"2024-09-21T11:50:26.751656+0000","last_clean":"2024-09-21T11:50:26.751656+0000","last_became_active":"2024-09-21T11:49:08.515204+0000","last_became_peered":"2024-09-21T11:49:08.515204+0000","last_unstale":"2024-09-21T11:50:26.751656+0000","last_undegraded":"2024-09-21T11:50:26.751656+0000","last_fullsized":"2024-09-21T11:50:26.751656+0000","mapping_epoch":247,"log_start":"252'900","ondisk_log_start":"252'900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":6055,"log_dups_size":0,"ondisk_log_size":6055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.697074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455802880,"num_objects":6955,"num_object_clones":0,"num_object_copies":20865,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6955,"num_write_kb":445120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":539,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357847+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:50:22.357847+0000","last_peered":"2024-09-21T11:50:22.357847+0000","last_clean":"2024-09-21T11:50:22.357847+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:50:22.357847+0000","last_undegraded":"2024-09-21T11:50:22.357847+0000","last_fullsized":"2024-09-21T11:50:22.357847+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:34.714346+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":482,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357098+0000","last_change":"2024-09-21T11:49:16.415641+0000","last_active":"2024-09-21T11:50:22.357098+0000","last_peered":"2024-09-21T11:50:22.357098+0000","last_clean":"2024-09-21T11:50:22.357098+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:50:22.357098+0000","last_undegraded":"2024-09-21T11:50:22.357098+0000","last_fullsized":"2024-09-21T11:50:22.357098+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:16.415597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:16.415597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:45.957747+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1142,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:24.624323+0000","last_change":"2024-09-21T11:49:51.434402+0000","last_active":"2024-09-21T11:50:24.624323+0000","last_peered":"2024-09-21T11:50:24.624323+0000","last_clean":"2024-09-21T11:50:24.624323+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:50:24.624323+0000","last_undegraded":"2024-09-21T11:50:24.624323+0000","last_fullsized":"2024-09-21T11:50:24.624323+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:51.434333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:49:51.434333+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:02.759046+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":617,"num_read_kb":612,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":479,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.368588+0000","last_change":"2024-09-21T11:48:42.042870+0000","last_active":"2024-09-21T11:50:22.368588+0000","last_peered":"2024-09-21T11:50:22.368588+0000","last_clean":"2024-09-21T11:50:22.368588+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:50:22.368588+0000","last_undegraded":"2024-09-21T11:50:22.368588+0000","last_fullsized":"2024-09-21T11:50:22.368588+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:42.042814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:42.042814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.767298+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":167,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365493+0000","last_change":"2024-09-21T11:49:54.996327+0000","last_active":"2024-09-21T11:50:22.365493+0000","last_peered":"2024-09-21T11:50:22.365493+0000","last_clean":"2024-09-21T11:50:22.365493+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:50:22.365493+0000","last_undegraded":"2024-09-21T11:50:22.365493+0000","last_fullsized":"2024-09-21T11:50:22.365493+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.996240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.996240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.233955+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"276'6826","reported_seq":6972,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.940799+0000","last_change":"2024-09-21T11:50:26.726878+0000","last_active":"2024-09-21T11:50:26.940799+0000","last_peered":"2024-09-21T11:50:26.940799+0000","last_clean":"2024-09-21T11:50:26.940799+0000","last_became_active":"2024-09-21T11:49:08.516627+0000","last_became_peered":"2024-09-21T11:49:08.516627+0000","last_unstale":"2024-09-21T11:50:26.940799+0000","last_undegraded":"2024-09-21T11:50:26.940799+0000","last_fullsized":"2024-09-21T11:50:26.940799+0000","mapping_epoch":247,"log_start":"253'1900","ondisk_log_start":"253'1900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6809","last_scrub_stamp":"2024-09-21T11:50:26.726538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.726538+0000","objects_scrubbed":6753,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:32.214810+0000","scrub_duration":2384,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":447348736,"num_objects":6826,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6826,"num_write_kb":436864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":635,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.360149+0000","last_change":"2024-09-21T11:49:18.420485+0000","last_active":"2024-09-21T11:50:22.360149+0000","last_peered":"2024-09-21T11:50:22.360149+0000","last_clean":"2024-09-21T11:50:22.360149+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:50:22.360149+0000","last_undegraded":"2024-09-21T11:50:22.360149+0000","last_fullsized":"2024-09-21T11:50:22.360149+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-21T11:49:18.420396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:18.420396+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.826435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":475,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361654+0000","last_change":"2024-09-21T11:46:57.833589+0000","last_active":"2024-09-21T11:50:22.361654+0000","last_peered":"2024-09-21T11:50:22.361654+0000","last_clean":"2024-09-21T11:50:22.361654+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:50:22.361654+0000","last_undegraded":"2024-09-21T11:50:22.361654+0000","last_fullsized":"2024-09-21T11:50:22.361654+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:57.833329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.833329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.227963+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":529,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361557+0000","last_change":"2024-09-21T11:46:58.645189+0000","last_active":"2024-09-21T11:50:22.361557+0000","last_peered":"2024-09-21T11:50:22.361557+0000","last_clean":"2024-09-21T11:50:22.361557+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:50:22.361557+0000","last_undegraded":"2024-09-21T11:50:22.361557+0000","last_fullsized":"2024-09-21T11:50:22.361557+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:46:58.645122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:46:58.645122+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.471456+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":473,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361920+0000","last_change":"2024-09-21T11:49:06.639805+0000","last_active":"2024-09-21T11:50:22.361920+0000","last_peered":"2024-09-21T11:50:22.361920+0000","last_clean":"2024-09-21T11:50:22.361920+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:50:22.361920+0000","last_undegraded":"2024-09-21T11:50:22.361920+0000","last_fullsized":"2024-09-21T11:50:22.361920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:06.639732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:06.639732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:16.438605+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":461,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.368446+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:50:22.368446+0000","last_peered":"2024-09-21T11:50:22.368446+0000","last_clean":"2024-09-21T11:50:22.368446+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:50:22.368446+0000","last_undegraded":"2024-09-21T11:50:22.368446+0000","last_fullsized":"2024-09-21T11:50:22.368446+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.925430+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"276'6750","reported_seq":6819,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:25.915465+0000","last_change":"2024-09-21T11:49:08.519327+0000","last_active":"2024-09-21T11:50:25.915465+0000","last_peered":"2024-09-21T11:50:25.915465+0000","last_clean":"2024-09-21T11:50:25.915465+0000","last_became_active":"2024-09-21T11:49:08.519011+0000","last_became_peered":"2024-09-21T11:49:08.519011+0000","last_unstale":"2024-09-21T11:50:25.915465+0000","last_undegraded":"2024-09-21T11:50:25.915465+0000","last_fullsized":"2024-09-21T11:50:25.915465+0000","mapping_epoch":247,"log_start":"253'1900","ondisk_log_start":"253'1900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4850,"log_dups_size":0,"ondisk_log_size":4850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.190157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":442368000,"num_objects":6750,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6750,"num_write_kb":432000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":531,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.358497+0000","last_change":"2024-09-21T11:48:05.072998+0000","last_active":"2024-09-21T11:50:22.358497+0000","last_peered":"2024-09-21T11:50:22.358497+0000","last_clean":"2024-09-21T11:50:22.358497+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:50:22.358497+0000","last_undegraded":"2024-09-21T11:50:22.358497+0000","last_fullsized":"2024-09-21T11:50:22.358497+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:05.072910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:05.072910+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.126516+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":498,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354949+0000","last_change":"2024-09-21T11:49:49.165213+0000","last_active":"2024-09-21T11:50:22.354949+0000","last_peered":"2024-09-21T11:50:22.354949+0000","last_clean":"2024-09-21T11:50:22.354949+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:50:22.354949+0000","last_undegraded":"2024-09-21T11:50:22.354949+0000","last_fullsized":"2024-09-21T11:50:22.354949+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.165146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.165146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:04.935623+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":465,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365948+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:50:22.365948+0000","last_peered":"2024-09-21T11:50:22.365948+0000","last_clean":"2024-09-21T11:50:22.365948+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:50:22.365948+0000","last_undegraded":"2024-09-21T11:50:22.365948+0000","last_fullsized":"2024-09-21T11:50:22.365948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.928709+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":461,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.360732+0000","last_change":"2024-09-21T11:48:23.285482+0000","last_active":"2024-09-21T11:50:22.360732+0000","last_peered":"2024-09-21T11:50:22.360732+0000","last_clean":"2024-09-21T11:50:22.360732+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:50:22.360732+0000","last_undegraded":"2024-09-21T11:50:22.360732+0000","last_fullsized":"2024-09-21T11:50:22.360732+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.285283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.285283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.489355+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"276'6774","reported_seq":6843,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.056240+0000","last_change":"2024-09-21T11:49:08.748461+0000","last_active":"2024-09-21T11:50:26.056240+0000","last_peered":"2024-09-21T11:50:26.056240+0000","last_clean":"2024-09-21T11:50:26.056240+0000","last_became_active":"2024-09-21T11:49:08.748155+0000","last_became_peered":"2024-09-21T11:49:08.748155+0000","last_unstale":"2024-09-21T11:50:26.056240+0000","last_undegraded":"2024-09-21T11:50:26.056240+0000","last_fullsized":"2024-09-21T11:50:26.056240+0000","mapping_epoch":247,"log_start":"253'2000","ondisk_log_start":"253'2000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4774,"log_dups_size":0,"ondisk_log_size":4774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.862263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":443940864,"num_objects":6774,"num_object_clones":0,"num_object_copies":20322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6774,"num_write_kb":433536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":602,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.356443+0000","last_change":"2024-09-21T11:49:09.037775+0000","last_active":"2024-09-21T11:50:22.356443+0000","last_peered":"2024-09-21T11:50:22.356443+0000","last_clean":"2024-09-21T11:50:22.356443+0000","last_became_active":"2024-09-21T11:48:33.022321+0000","last_became_peered":"2024-09-21T11:48:33.022321+0000","last_unstale":"2024-09-21T11:50:22.356443+0000","last_undegraded":"2024-09-21T11:50:22.356443+0000","last_fullsized":"2024-09-21T11:50:22.356443+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-21T11:49:09.037713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:09.037713+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:23.218918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":495,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.359765+0000","last_change":"2024-09-21T11:47:03.365071+0000","last_active":"2024-09-21T11:50:22.359765+0000","last_peered":"2024-09-21T11:50:22.359765+0000","last_clean":"2024-09-21T11:50:22.359765+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:50:22.359765+0000","last_undegraded":"2024-09-21T11:50:22.359765+0000","last_fullsized":"2024-09-21T11:50:22.359765+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.692429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.692429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.349348+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":159,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365098+0000","last_change":"2024-09-21T11:47:03.375108+0000","last_active":"2024-09-21T11:50:22.365098+0000","last_peered":"2024-09-21T11:50:22.365098+0000","last_clean":"2024-09-21T11:50:22.365098+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:50:22.365098+0000","last_undegraded":"2024-09-21T11:50:22.365098+0000","last_fullsized":"2024-09-21T11:50:22.365098+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:50.008635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:50.008635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.928506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":470,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.356151+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:50:22.356151+0000","last_peered":"2024-09-21T11:50:22.356151+0000","last_clean":"2024-09-21T11:50:22.356151+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:50:22.356151+0000","last_undegraded":"2024-09-21T11:50:22.356151+0000","last_fullsized":"2024-09-21T11:50:22.356151+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:42.937389+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"276'6936","reported_seq":7005,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.931809+0000","last_change":"2024-09-21T11:49:08.517565+0000","last_active":"2024-09-21T11:50:26.931809+0000","last_peered":"2024-09-21T11:50:26.931809+0000","last_clean":"2024-09-21T11:50:26.931809+0000","last_became_active":"2024-09-21T11:49:08.517011+0000","last_became_peered":"2024-09-21T11:49:08.517011+0000","last_unstale":"2024-09-21T11:50:26.931809+0000","last_undegraded":"2024-09-21T11:50:26.931809+0000","last_fullsized":"2024-09-21T11:50:26.931809+0000","mapping_epoch":247,"log_start":"253'2000","ondisk_log_start":"253'2000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4936,"log_dups_size":0,"ondisk_log_size":4936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.617618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454557696,"num_objects":6936,"num_object_clones":0,"num_object_copies":20808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6936,"num_write_kb":443904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":516,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355068+0000","last_change":"2024-09-21T11:50:06.225159+0000","last_active":"2024-09-21T11:50:22.355068+0000","last_peered":"2024-09-21T11:50:22.355068+0000","last_clean":"2024-09-21T11:50:22.355068+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:50:22.355068+0000","last_undegraded":"2024-09-21T11:50:22.355068+0000","last_fullsized":"2024-09-21T11:50:22.355068+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:32.807084+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":458,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361116+0000","last_change":"2024-09-21T11:42:28.431024+0000","last_active":"2024-09-21T11:50:22.361116+0000","last_peered":"2024-09-21T11:50:22.361116+0000","last_clean":"2024-09-21T11:50:22.361116+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:50:22.361116+0000","last_undegraded":"2024-09-21T11:50:22.361116+0000","last_fullsized":"2024-09-21T11:50:22.361116+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:28.430956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:28.430956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:28.151163+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":616,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355518+0000","last_change":"2024-09-21T11:50:02.975387+0000","last_active":"2024-09-21T11:50:22.355518+0000","last_peered":"2024-09-21T11:50:22.355518+0000","last_clean":"2024-09-21T11:50:22.355518+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:50:22.355518+0000","last_undegraded":"2024-09-21T11:50:22.355518+0000","last_fullsized":"2024-09-21T11:50:22.355518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:50:02.975313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.975313+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:25.444093+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":468,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.358220+0000","last_change":"2024-09-21T11:48:07.992586+0000","last_active":"2024-09-21T11:50:22.358220+0000","last_peered":"2024-09-21T11:50:22.358220+0000","last_clean":"2024-09-21T11:50:22.358220+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:50:22.358220+0000","last_undegraded":"2024-09-21T11:50:22.358220+0000","last_fullsized":"2024-09-21T11:50:22.358220+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.992517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.992517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:42.078128+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"276'6849","reported_seq":6993,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.037035+0000","last_change":"2024-09-21T11:50:24.709591+0000","last_active":"2024-09-21T11:50:26.037035+0000","last_peered":"2024-09-21T11:50:26.037035+0000","last_clean":"2024-09-21T11:50:26.037035+0000","last_became_active":"2024-09-21T11:49:08.748171+0000","last_became_peered":"2024-09-21T11:49:08.748171+0000","last_unstale":"2024-09-21T11:50:26.037035+0000","last_undegraded":"2024-09-21T11:50:26.037035+0000","last_fullsized":"2024-09-21T11:50:26.037035+0000","mapping_epoch":247,"log_start":"252'1500","ondisk_log_start":"252'1500","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6768","last_scrub_stamp":"2024-09-21T11:50:24.709386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.709386+0000","objects_scrubbed":6700,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:51.612675+0000","scrub_duration":2008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":448856064,"num_objects":6849,"num_object_clones":0,"num_object_copies":20547,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6849,"num_write_kb":438336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":520,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357683+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:50:22.357683+0000","last_peered":"2024-09-21T11:50:22.357683+0000","last_clean":"2024-09-21T11:50:22.357683+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:50:22.357683+0000","last_undegraded":"2024-09-21T11:50:22.357683+0000","last_fullsized":"2024-09-21T11:50:22.357683+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:41.803003+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":481,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.363923+0000","last_change":"2024-09-21T11:49:23.000190+0000","last_active":"2024-09-21T11:50:22.363923+0000","last_peered":"2024-09-21T11:50:22.363923+0000","last_clean":"2024-09-21T11:50:22.363923+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:50:22.363923+0000","last_undegraded":"2024-09-21T11:50:22.363923+0000","last_fullsized":"2024-09-21T11:50:22.363923+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.000138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.000138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:42.451760+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":466,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355545+0000","last_change":"2024-09-21T11:48:49.993307+0000","last_active":"2024-09-21T11:50:22.355545+0000","last_peered":"2024-09-21T11:50:22.355545+0000","last_clean":"2024-09-21T11:50:22.355545+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:50:22.355545+0000","last_undegraded":"2024-09-21T11:50:22.355545+0000","last_fullsized":"2024-09-21T11:50:22.355545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.993230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.993230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.951895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":445,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357007+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:50:22.357007+0000","last_peered":"2024-09-21T11:50:22.357007+0000","last_clean":"2024-09-21T11:50:22.357007+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:50:22.357007+0000","last_undegraded":"2024-09-21T11:50:22.357007+0000","last_fullsized":"2024-09-21T11:50:22.357007+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.764486+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"276'6914","reported_seq":6983,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.060856+0000","last_change":"2024-09-21T11:49:08.517663+0000","last_active":"2024-09-21T11:50:26.060856+0000","last_peered":"2024-09-21T11:50:26.060856+0000","last_clean":"2024-09-21T11:50:26.060856+0000","last_became_active":"2024-09-21T11:49:08.516975+0000","last_became_peered":"2024-09-21T11:49:08.516975+0000","last_unstale":"2024-09-21T11:50:26.060856+0000","last_undegraded":"2024-09-21T11:50:26.060856+0000","last_fullsized":"2024-09-21T11:50:26.060856+0000","mapping_epoch":247,"log_start":"252'1600","ondisk_log_start":"252'1600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5314,"log_dups_size":0,"ondisk_log_size":5314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453115904,"num_objects":6914,"num_object_clones":0,"num_object_copies":20742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6914,"num_write_kb":442496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":594,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:24.359357+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:50:24.359357+0000","last_peered":"2024-09-21T11:50:24.359357+0000","last_clean":"2024-09-21T11:50:24.359357+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:50:24.359357+0000","last_undegraded":"2024-09-21T11:50:24.359357+0000","last_fullsized":"2024-09-21T11:50:24.359357+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:45.721984+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":481,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.367969+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:50:22.367969+0000","last_peered":"2024-09-21T11:50:22.367969+0000","last_clean":"2024-09-21T11:50:22.367969+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:50:22.367969+0000","last_undegraded":"2024-09-21T11:50:22.367969+0000","last_fullsized":"2024-09-21T11:50:22.367969+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:04.681745+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":474,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354792+0000","last_change":"2024-09-21T11:46:16.775013+0000","last_active":"2024-09-21T11:50:22.354792+0000","last_peered":"2024-09-21T11:50:22.354792+0000","last_clean":"2024-09-21T11:50:22.354792+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:50:22.354792+0000","last_undegraded":"2024-09-21T11:50:22.354792+0000","last_fullsized":"2024-09-21T11:50:22.354792+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:16.774939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:16.774939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:29.020969+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":461,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361481+0000","last_change":"2024-09-21T11:46:59.704981+0000","last_active":"2024-09-21T11:50:22.361481+0000","last_peered":"2024-09-21T11:50:22.361481+0000","last_clean":"2024-09-21T11:50:22.361481+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:50:22.361481+0000","last_undegraded":"2024-09-21T11:50:22.361481+0000","last_fullsized":"2024-09-21T11:50:22.361481+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:59.704805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:59.704805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:31.444921+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"276'6730","reported_seq":6799,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.055462+0000","last_change":"2024-09-21T11:49:08.610149+0000","last_active":"2024-09-21T11:50:26.055462+0000","last_peered":"2024-09-21T11:50:26.055462+0000","last_clean":"2024-09-21T11:50:26.055462+0000","last_became_active":"2024-09-21T11:49:08.609829+0000","last_became_peered":"2024-09-21T11:49:08.609829+0000","last_unstale":"2024-09-21T11:50:26.055462+0000","last_undegraded":"2024-09-21T11:50:26.055462+0000","last_fullsized":"2024-09-21T11:50:26.055462+0000","mapping_epoch":247,"log_start":"253'2000","ondisk_log_start":"253'2000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4730,"log_dups_size":0,"ondisk_log_size":4730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.242703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":441057280,"num_objects":6730,"num_object_clones":0,"num_object_copies":20190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6730,"num_write_kb":430720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":556,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354679+0000","last_change":"2024-09-21T11:48:04.111199+0000","last_active":"2024-09-21T11:50:22.354679+0000","last_peered":"2024-09-21T11:50:22.354679+0000","last_clean":"2024-09-21T11:50:22.354679+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:50:22.354679+0000","last_undegraded":"2024-09-21T11:50:22.354679+0000","last_fullsized":"2024-09-21T11:50:22.354679+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:48:04.111115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:04.111115+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:25.112149+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":511,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.367799+0000","last_change":"2024-09-21T11:49:08.611218+0000","last_active":"2024-09-21T11:50:22.367799+0000","last_peered":"2024-09-21T11:50:22.367799+0000","last_clean":"2024-09-21T11:50:22.367799+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:50:22.367799+0000","last_undegraded":"2024-09-21T11:50:22.367799+0000","last_fullsized":"2024-09-21T11:50:22.367799+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.611159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.611159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:36.323039+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":496,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354526+0000","last_change":"2024-09-21T11:49:44.985594+0000","last_active":"2024-09-21T11:50:22.354526+0000","last_peered":"2024-09-21T11:50:22.354526+0000","last_clean":"2024-09-21T11:50:22.354526+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:50:22.354526+0000","last_undegraded":"2024-09-21T11:50:22.354526+0000","last_fullsized":"2024-09-21T11:50:22.354526+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:44.985234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:44.985234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:50.892123+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":476,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.360752+0000","last_change":"2024-09-21T11:49:21.071757+0000","last_active":"2024-09-21T11:50:22.360752+0000","last_peered":"2024-09-21T11:50:22.360752+0000","last_clean":"2024-09-21T11:50:22.360752+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:50:22.360752+0000","last_undegraded":"2024-09-21T11:50:22.360752+0000","last_fullsized":"2024-09-21T11:50:22.360752+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:21.071680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:21.071680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:25.890021+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"276'6954","reported_seq":7023,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:26.759314+0000","last_change":"2024-09-21T11:49:08.517117+0000","last_active":"2024-09-21T11:50:26.759314+0000","last_peered":"2024-09-21T11:50:26.759314+0000","last_clean":"2024-09-21T11:50:26.759314+0000","last_became_active":"2024-09-21T11:49:08.516494+0000","last_became_peered":"2024-09-21T11:49:08.516494+0000","last_unstale":"2024-09-21T11:50:26.759314+0000","last_undegraded":"2024-09-21T11:50:26.759314+0000","last_fullsized":"2024-09-21T11:50:26.759314+0000","mapping_epoch":247,"log_start":"252'900","ondisk_log_start":"252'900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":6054,"log_dups_size":0,"ondisk_log_size":6054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.332843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455737344,"num_objects":6954,"num_object_clones":0,"num_object_copies":20862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6954,"num_write_kb":445056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":504,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.364518+0000","last_change":"2024-09-21T11:49:45.783231+0000","last_active":"2024-09-21T11:50:22.364518+0000","last_peered":"2024-09-21T11:50:22.364518+0000","last_clean":"2024-09-21T11:50:22.364518+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:50:22.364518+0000","last_undegraded":"2024-09-21T11:50:22.364518+0000","last_fullsized":"2024-09-21T11:50:22.364518+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:49:45.783036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:45.783036+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.400285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":501,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361673+0000","last_change":"2024-09-21T11:50:06.398255+0000","last_active":"2024-09-21T11:50:22.361673+0000","last_peered":"2024-09-21T11:50:22.361673+0000","last_clean":"2024-09-21T11:50:22.361673+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:50:22.361673+0000","last_undegraded":"2024-09-21T11:50:22.361673+0000","last_fullsized":"2024-09-21T11:50:22.361673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:50:06.398195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.398195+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:06.457914+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":599,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365257+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:50:22.365257+0000","last_peered":"2024-09-21T11:50:22.365257+0000","last_clean":"2024-09-21T11:50:22.365257+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:50:22.365257+0000","last_undegraded":"2024-09-21T11:50:22.365257+0000","last_fullsized":"2024-09-21T11:50:22.365257+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:37.444577+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":461,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.364789+0000","last_change":"2024-09-21T11:49:56.669610+0000","last_active":"2024-09-21T11:50:22.364789+0000","last_peered":"2024-09-21T11:50:22.364789+0000","last_clean":"2024-09-21T11:50:22.364789+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:50:22.364789+0000","last_undegraded":"2024-09-21T11:50:22.364789+0000","last_fullsized":"2024-09-21T11:50:22.364789+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:56.669547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:56.669547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:24.559358+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":481,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.360198+0000","last_change":"2024-09-21T11:48:17.129730+0000","last_active":"2024-09-21T11:50:22.360198+0000","last_peered":"2024-09-21T11:50:22.360198+0000","last_clean":"2024-09-21T11:50:22.360198+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:50:22.360198+0000","last_undegraded":"2024-09-21T11:50:22.360198+0000","last_fullsized":"2024-09-21T11:50:22.360198+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:17.129675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:17.129675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.393394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":510,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361886+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:50:22.361886+0000","last_peered":"2024-09-21T11:50:22.361886+0000","last_clean":"2024-09-21T11:50:22.361886+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:50:22.361886+0000","last_undegraded":"2024-09-21T11:50:22.361886+0000","last_fullsized":"2024-09-21T11:50:22.361886+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:38.248209+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":487,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361827+0000","last_change":"2024-09-21T11:49:08.902428+0000","last_active":"2024-09-21T11:50:22.361827+0000","last_peered":"2024-09-21T11:50:22.361827+0000","last_clean":"2024-09-21T11:50:22.361827+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:50:22.361827+0000","last_undegraded":"2024-09-21T11:50:22.361827+0000","last_fullsized":"2024-09-21T11:50:22.361827+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.902382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.902382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.218309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":477,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.367300+0000","last_change":"2024-09-21T11:49:43.464860+0000","last_active":"2024-09-21T11:50:22.367300+0000","last_peered":"2024-09-21T11:50:22.367300+0000","last_clean":"2024-09-21T11:50:22.367300+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:50:22.367300+0000","last_undegraded":"2024-09-21T11:50:22.367300+0000","last_fullsized":"2024-09-21T11:50:22.367300+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:43.464792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:43.464792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:48.801237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":503,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.358083+0000","last_change":"2024-09-21T11:46:28.973700+0000","last_active":"2024-09-21T11:50:22.358083+0000","last_peered":"2024-09-21T11:50:22.358083+0000","last_clean":"2024-09-21T11:50:22.358083+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:50:22.358083+0000","last_undegraded":"2024-09-21T11:50:22.358083+0000","last_fullsized":"2024-09-21T11:50:22.358083+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:28.973606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.973606+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.696833+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":473,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365187+0000","last_change":"2024-09-21T11:50:02.438498+0000","last_active":"2024-09-21T11:50:22.365187+0000","last_peered":"2024-09-21T11:50:22.365187+0000","last_clean":"2024-09-21T11:50:22.365187+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:50:22.365187+0000","last_undegraded":"2024-09-21T11:50:22.365187+0000","last_fullsized":"2024-09-21T11:50:22.365187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.436591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.436591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:19.559044+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":488,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361789+0000","last_change":"2024-09-21T11:49:50.191966+0000","last_active":"2024-09-21T11:50:22.361789+0000","last_peered":"2024-09-21T11:50:22.361789+0000","last_clean":"2024-09-21T11:50:22.361789+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:50:22.361789+0000","last_undegraded":"2024-09-21T11:50:22.361789+0000","last_fullsized":"2024-09-21T11:50:22.361789+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:01.459336+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":478,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.354952+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:50:22.354952+0000","last_peered":"2024-09-21T11:50:22.354952+0000","last_clean":"2024-09-21T11:50:22.354952+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:50:22.354952+0000","last_undegraded":"2024-09-21T11:50:22.354952+0000","last_fullsized":"2024-09-21T11:50:22.354952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:54.617405+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":486,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.366902+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:50:22.366902+0000","last_peered":"2024-09-21T11:50:22.366902+0000","last_clean":"2024-09-21T11:50:22.366902+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:50:22.366902+0000","last_undegraded":"2024-09-21T11:50:22.366902+0000","last_fullsized":"2024-09-21T11:50:22.366902+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:39.874682+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":481,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357017+0000","last_change":"2024-09-21T11:49:22.528906+0000","last_active":"2024-09-21T11:50:22.357017+0000","last_peered":"2024-09-21T11:50:22.357017+0000","last_clean":"2024-09-21T11:50:22.357017+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:50:22.357017+0000","last_undegraded":"2024-09-21T11:50:22.357017+0000","last_fullsized":"2024-09-21T11:50:22.357017+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:22.528831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:22.528831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.548018+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":584,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.360917+0000","last_change":"2024-09-21T11:47:02.849839+0000","last_active":"2024-09-21T11:50:22.360917+0000","last_peered":"2024-09-21T11:50:22.360917+0000","last_clean":"2024-09-21T11:50:22.360917+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:50:22.360917+0000","last_undegraded":"2024-09-21T11:50:22.360917+0000","last_fullsized":"2024-09-21T11:50:22.360917+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:02.849786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:02.849786+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.365178+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":470,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.356724+0000","last_change":"2024-09-21T11:48:50.927661+0000","last_active":"2024-09-21T11:50:22.356724+0000","last_peered":"2024-09-21T11:50:22.356724+0000","last_clean":"2024-09-21T11:50:22.356724+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:50:22.356724+0000","last_undegraded":"2024-09-21T11:50:22.356724+0000","last_fullsized":"2024-09-21T11:50:22.356724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:50.927613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.927613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:20.385159+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":522,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361073+0000","last_change":"2024-09-21T11:47:03.428846+0000","last_active":"2024-09-21T11:50:22.361073+0000","last_peered":"2024-09-21T11:50:22.361073+0000","last_clean":"2024-09-21T11:50:22.361073+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:50:22.361073+0000","last_undegraded":"2024-09-21T11:50:22.361073+0000","last_fullsized":"2024-09-21T11:50:22.361073+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:42:24.563953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:42:24.563953+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:33.175857+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":473,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.366984+0000","last_change":"2024-09-21T11:49:58.400078+0000","last_active":"2024-09-21T11:50:22.366984+0000","last_peered":"2024-09-21T11:50:22.366984+0000","last_clean":"2024-09-21T11:50:22.366984+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:50:22.366984+0000","last_undegraded":"2024-09-21T11:50:22.366984+0000","last_fullsized":"2024-09-21T11:50:22.366984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.400013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.400013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:58.776865+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":622,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.366141+0000","last_change":"2024-09-21T11:47:03.415793+0000","last_active":"2024-09-21T11:50:22.366141+0000","last_peered":"2024-09-21T11:50:22.366141+0000","last_clean":"2024-09-21T11:50:22.366141+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:50:22.366141+0000","last_undegraded":"2024-09-21T11:50:22.366141+0000","last_fullsized":"2024-09-21T11:50:22.366141+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'9","last_scrub_stamp":"2024-09-21T11:46:10.174971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:10.174971+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:30.015617+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":477,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357997+0000","last_change":"2024-09-21T11:50:02.782565+0000","last_active":"2024-09-21T11:50:22.357997+0000","last_peered":"2024-09-21T11:50:22.357997+0000","last_clean":"2024-09-21T11:50:22.357997+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:50:22.357997+0000","last_undegraded":"2024-09-21T11:50:22.357997+0000","last_fullsized":"2024-09-21T11:50:22.357997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.782475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.782475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:29.369093+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":566,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355791+0000","last_change":"2024-09-21T11:48:33.033557+0000","last_active":"2024-09-21T11:50:22.355791+0000","last_peered":"2024-09-21T11:50:22.355791+0000","last_clean":"2024-09-21T11:50:22.355791+0000","last_became_active":"2024-09-21T11:48:33.022647+0000","last_became_peered":"2024-09-21T11:48:33.022647+0000","last_unstale":"2024-09-21T11:50:22.355791+0000","last_undegraded":"2024-09-21T11:50:22.355791+0000","last_fullsized":"2024-09-21T11:50:22.355791+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:46:54.032795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.032795+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.144714+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":466,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.355330+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:50:22.355330+0000","last_peered":"2024-09-21T11:50:22.355330+0000","last_clean":"2024-09-21T11:50:22.355330+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:50:22.355330+0000","last_undegraded":"2024-09-21T11:50:22.355330+0000","last_fullsized":"2024-09-21T11:50:22.355330+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.091776+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":167,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.369154+0000","last_change":"2024-09-21T11:49:58.034124+0000","last_active":"2024-09-21T11:50:22.369154+0000","last_peered":"2024-09-21T11:50:22.369154+0000","last_clean":"2024-09-21T11:50:22.369154+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:50:22.369154+0000","last_undegraded":"2024-09-21T11:50:22.369154+0000","last_fullsized":"2024-09-21T11:50:22.369154+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.034057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.034057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:05.285593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":454,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361528+0000","last_change":"2024-09-21T11:50:02.329020+0000","last_active":"2024-09-21T11:50:22.361528+0000","last_peered":"2024-09-21T11:50:22.361528+0000","last_clean":"2024-09-21T11:50:22.361528+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:50:22.361528+0000","last_undegraded":"2024-09-21T11:50:22.361528+0000","last_fullsized":"2024-09-21T11:50:22.361528+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.328926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.328926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.336948+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":178,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.369382+0000","last_change":"2024-09-21T11:47:03.404473+0000","last_active":"2024-09-21T11:50:22.369382+0000","last_peered":"2024-09-21T11:50:22.369382+0000","last_clean":"2024-09-21T11:50:22.369382+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:50:22.369382+0000","last_undegraded":"2024-09-21T11:50:22.369382+0000","last_fullsized":"2024-09-21T11:50:22.369382+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:26.649189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:26.649189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.922345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":481,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.367834+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:50:22.367834+0000","last_peered":"2024-09-21T11:50:22.367834+0000","last_clean":"2024-09-21T11:50:22.367834+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:50:22.367834+0000","last_undegraded":"2024-09-21T11:50:22.367834+0000","last_fullsized":"2024-09-21T11:50:22.367834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:47.576218+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":488,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.360854+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:50:22.360854+0000","last_peered":"2024-09-21T11:50:22.360854+0000","last_clean":"2024-09-21T11:50:22.360854+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:50:22.360854+0000","last_undegraded":"2024-09-21T11:50:22.360854+0000","last_fullsized":"2024-09-21T11:50:22.360854+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.838895+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":492,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.356882+0000","last_change":"2024-09-21T11:48:50.487842+0000","last_active":"2024-09-21T11:50:22.356882+0000","last_peered":"2024-09-21T11:50:22.356882+0000","last_clean":"2024-09-21T11:50:22.356882+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:50:22.356882+0000","last_undegraded":"2024-09-21T11:50:22.356882+0000","last_fullsized":"2024-09-21T11:50:22.356882+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:48:50.487786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.487786+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.708437+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":501,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361755+0000","last_change":"2024-09-21T11:49:07.148305+0000","last_active":"2024-09-21T11:50:22.361755+0000","last_peered":"2024-09-21T11:50:22.361755+0000","last_clean":"2024-09-21T11:50:22.361755+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:50:22.361755+0000","last_undegraded":"2024-09-21T11:50:22.361755+0000","last_fullsized":"2024-09-21T11:50:22.361755+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:49:07.148241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.148241+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:19.446103+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":487,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361908+0000","last_change":"2024-09-21T11:49:12.981959+0000","last_active":"2024-09-21T11:50:22.361908+0000","last_peered":"2024-09-21T11:50:22.361908+0000","last_clean":"2024-09-21T11:50:22.361908+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:50:22.361908+0000","last_undegraded":"2024-09-21T11:50:22.361908+0000","last_fullsized":"2024-09-21T11:50:22.361908+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:12.981878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:12.981878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:34.068456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":457,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361120+0000","last_change":"2024-09-21T11:48:44.639778+0000","last_active":"2024-09-21T11:50:22.361120+0000","last_peered":"2024-09-21T11:50:22.361120+0000","last_clean":"2024-09-21T11:50:22.361120+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:50:22.361120+0000","last_undegraded":"2024-09-21T11:50:22.361120+0000","last_fullsized":"2024-09-21T11:50:22.361120+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:44.639638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.639638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.393642+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":462,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.361020+0000","last_change":"2024-09-21T11:49:08.171420+0000","last_active":"2024-09-21T11:50:22.361020+0000","last_peered":"2024-09-21T11:50:22.361020+0000","last_clean":"2024-09-21T11:50:22.361020+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:50:22.361020+0000","last_undegraded":"2024-09-21T11:50:22.361020+0000","last_fullsized":"2024-09-21T11:50:22.361020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.171357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.171357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:24.012559+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":521,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.365826+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:50:22.365826+0000","last_peered":"2024-09-21T11:50:22.365826+0000","last_clean":"2024-09-21T11:50:22.365826+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:50:22.365826+0000","last_undegraded":"2024-09-21T11:50:22.365826+0000","last_fullsized":"2024-09-21T11:50:22.365826+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:27.196992+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":473,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:25.708558+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:50:25.708558+0000","last_peered":"2024-09-21T11:50:25.708558+0000","last_clean":"2024-09-21T11:50:25.708558+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:50:25.708558+0000","last_undegraded":"2024-09-21T11:50:25.708558+0000","last_fullsized":"2024-09-21T11:50:25.708558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:34.752247+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":465,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.359355+0000","last_change":"2024-09-21T11:48:09.104826+0000","last_active":"2024-09-21T11:50:22.359355+0000","last_peered":"2024-09-21T11:50:22.359355+0000","last_clean":"2024-09-21T11:50:22.359355+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:50:22.359355+0000","last_undegraded":"2024-09-21T11:50:22.359355+0000","last_fullsized":"2024-09-21T11:50:22.359355+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:09.104767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:09.104767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:35.382787+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":461,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.364421+0000","last_change":"2024-09-21T11:50:05.419390+0000","last_active":"2024-09-21T11:50:22.364421+0000","last_peered":"2024-09-21T11:50:22.364421+0000","last_clean":"2024-09-21T11:50:22.364421+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:50:22.364421+0000","last_undegraded":"2024-09-21T11:50:22.364421+0000","last_fullsized":"2024-09-21T11:50:22.364421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.419329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.419329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:15.874052+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":175,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:25.045141+0000","last_change":"2024-09-21T11:50:25.045141+0000","last_active":"2024-09-21T11:50:25.045141+0000","last_peered":"2024-09-21T11:50:25.045141+0000","last_clean":"2024-09-21T11:50:25.045141+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:50:25.045141+0000","last_undegraded":"2024-09-21T11:50:25.045141+0000","last_fullsized":"2024-09-21T11:50:25.045141+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.045074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.045074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.164898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":488,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.366797+0000","last_change":"2024-09-21T11:49:12.177359+0000","last_active":"2024-09-21T11:50:22.366797+0000","last_peered":"2024-09-21T11:50:22.366797+0000","last_clean":"2024-09-21T11:50:22.366797+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:50:22.366797+0000","last_undegraded":"2024-09-21T11:50:22.366797+0000","last_fullsized":"2024-09-21T11:50:22.366797+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:12.177303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:12.177303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":555,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.360827+0000","last_change":"2024-09-21T11:48:45.147257+0000","last_active":"2024-09-21T11:50:22.360827+0000","last_peered":"2024-09-21T11:50:22.360827+0000","last_clean":"2024-09-21T11:50:22.360827+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:50:22.360827+0000","last_undegraded":"2024-09-21T11:50:22.360827+0000","last_fullsized":"2024-09-21T11:50:22.360827+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:45.147190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:45.147190+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.355453+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":482,"reported_epoch":276,"state":"active+clean","last_fresh":"2024-09-21T11:50:22.357660+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:50:22.357660+0000","last_peered":"2024-09-21T11:50:22.357660+0000","last_clean":"2024-09-21T11:50:22.357660+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:50:22.357660+0000","last_undegraded":"2024-09-21T11:50:22.357660+0000","last_fullsized":"2024-09-21T11:50:22.357660+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.689301+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":7184187392,"num_objects":109622,"num_object_clones":0,"num_object_copies":328866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109622,"num_write_kb":7015808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21527068672,"data_stored":21527068672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82922,"ondisk_log_size":82922,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1122,"num_read_kb":879,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":617,"num_read_kb":612,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234114,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4307136,"kb_used_data":3914992,"kb_used_omap":1,"kb_used_meta":392126,"kb_avail":89438016,"statfs":{"total":95995035648,"available":91584528384,"internally_reserved":0,"allocated":4008951808,"data_stored":4008577192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":401537482},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463047,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1628596,"kb_used_data":1308644,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":92116556,"statfs":{"total":95995035648,"available":94327353344,"internally_reserved":0,"allocated":1340051456,"data_stored":1339689472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":327612870},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724684,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4372612,"kb_used_data":3931892,"kb_used_omap":1,"kb_used_meta":440702,"kb_avail":89372540,"statfs":{"total":95995035648,"available":91517480960,"internally_reserved":0,"allocated":4026257408,"data_stored":4025877907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":451279302},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920914,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1599624,"kb_used_data":1297720,"kb_used_omap":1,"kb_used_meta":301886,"kb_avail":92145528,"statfs":{"total":95995035648,"available":94357020672,"internally_reserved":0,"allocated":1328865280,"data_stored":1328482441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309131717},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117144,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4003320,"kb_used_data":3502888,"kb_used_omap":1,"kb_used_meta":500414,"kb_avail":89741832,"statfs":{"total":95995035648,"available":91895635968,"internally_reserved":0,"allocated":3586957312,"data_stored":3586589682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":512424391},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411486,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1762236,"kb_used_data":1311788,"kb_used_omap":1,"kb_used_meta":450430,"kb_avail":91982916,"statfs":{"total":95995035648,"available":94190505984,"internally_reserved":0,"allocated":1343270912,"data_stored":1342899897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":461240773},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575013,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4360668,"kb_used_data":3924108,"kb_used_omap":1,"kb_used_meta":436542,"kb_avail":89384484,"statfs":{"total":95995035648,"available":91529711616,"internally_reserved":0,"allocated":4018286592,"data_stored":4017883359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":447019462},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738539,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2126028,"kb_used_data":1761660,"kb_used_omap":1,"kb_used_meta":364350,"kb_avail":91619124,"statfs":{"total":95995035648,"available":93817982976,"internally_reserved":0,"allocated":1803939840,"data_stored":1803553651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":373094853},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1804533760,"data_stored":1804533760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4018601984,"data_stored":4018601984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1348337664,"data_stored":1348337664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3605331968,"data_stored":3605331968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1334706176,"data_stored":1334706176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4037935104,"data_stored":4037935104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1346830336,"data_stored":1346830336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4030791680,"data_stored":4030791680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:50:34.081 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-21T11:50:34.081 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:50:34.081 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-21T11:50:34.081 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T11:50:34.081 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T11:50:34.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 config set filestore_inject_stall 3 2024-09-21T11:50:34.094 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:34.100 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:34.129 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:34.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:34.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:34.231 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:34.253 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:34.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:34.254 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:34.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 81 16 117479 117463 90.6008 63.3125 0.0922504 0.0110243 2024-09-21T11:50:34.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:34.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:50:34.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:34.408 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:34.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:34.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:34.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:34.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:34.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:34.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:34.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:34 smithi043 ceph-mon[29710]: 3.0 scrub starts 2024-09-21T11:50:34.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:34 smithi043 ceph-mon[29710]: 3.0 scrub ok 2024-09-21T11:50:34.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:34 smithi043 ceph-mon[36454]: 3.0 scrub starts 2024-09-21T11:50:34.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:34 smithi043 ceph-mon[36454]: 3.0 scrub ok 2024-09-21T11:50:34.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:34 smithi175 ceph-mon[32613]: 3.0 scrub starts 2024-09-21T11:50:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:34 smithi175 ceph-mon[32613]: 3.0 scrub ok 2024-09-21T11:50:34.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:34.819 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:34.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:34.820 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:34.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:34.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:34.962 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:34.962 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:35.063 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:35.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:35.095 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:35.096 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:35.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:35.220 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:35.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:35.221 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:35.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:50:35.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 82 16 118418 118402 90.2115 58.6875 0.00330623 0.0110661 2024-09-21T11:50:35.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:35.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:35.351 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:35.351 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:35.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:35.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:35.491 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:35.491 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:35.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:50:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:50:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[36454]: pgmap v509: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-21T11:50:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[36454]: 4.9 scrub starts 2024-09-21T11:50:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[36454]: 4.9 scrub ok 2024-09-21T11:50:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[36454]: 6.8 scrub starts 2024-09-21T11:50:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[36454]: 6.8 scrub ok 2024-09-21T11:50:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T11:50:35.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T11:50:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[29710]: pgmap v509: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-21T11:50:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[29710]: 4.9 scrub starts 2024-09-21T11:50:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[29710]: 4.9 scrub ok 2024-09-21T11:50:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[29710]: 6.8 scrub starts 2024-09-21T11:50:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[29710]: 6.8 scrub ok 2024-09-21T11:50:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T11:50:35.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:35 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T11:50:35.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:35 smithi175 ceph-mon[32613]: pgmap v509: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-21T11:50:35.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:35 smithi175 ceph-mon[32613]: 4.9 scrub starts 2024-09-21T11:50:35.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:35 smithi175 ceph-mon[32613]: 4.9 scrub ok 2024-09-21T11:50:35.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:35 smithi175 ceph-mon[32613]: 6.8 scrub starts 2024-09-21T11:50:35.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:35 smithi175 ceph-mon[32613]: 6.8 scrub ok 2024-09-21T11:50:35.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:35 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T11:50:35.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:35 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T11:50:35.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:35.604 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:35.634 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:35.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:35.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:35.735 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:35.768 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:35.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:35.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:35.796 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:35.815 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:50:35.815 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:50:35.862 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:50:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:50:35.869 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:35.894 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:50:35.894 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:50:35.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:35.914 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:35.914 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:35.922 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:50:35.922 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:50:35.957 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:50:35.958 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:50:35.991 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:50:35.991 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:50:36.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:36.025 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:50:36.025 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:50:36.059 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:36.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:36.060 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:36.067 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:50:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:50:36.161 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:36.184 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:36.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:36.185 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:36.204 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:50:36.204 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:50:36.204 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:50:36.204 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:50:36.204 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:50:36.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 83 16 119328 119312 89.8099 56.875 0.00786292 0.0111299 2024-09-21T11:50:36.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:36.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:36.332 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:36.332 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:36.433 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:36.472 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:36.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:36.473 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:36.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:36.605 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:36.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:36.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:36.707 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:36.728 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:36.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:36.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:36.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[36454]: 9.3 scrub starts 2024-09-21T11:50:36.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[36454]: 4.13 scrub starts 2024-09-21T11:50:36.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[36454]: 4.13 scrub ok 2024-09-21T11:50:36.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[36454]: 3.b scrub starts 2024-09-21T11:50:36.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[36454]: 3.b scrub ok 2024-09-21T11:50:36.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[36454]: 3.1b scrub starts 2024-09-21T11:50:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[36454]: 3.1b scrub ok 2024-09-21T11:50:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/545619043' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[36454]: 6.1 scrub starts 2024-09-21T11:50:36.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[36454]: 6.1 scrub ok 2024-09-21T11:50:36.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[29710]: 9.3 scrub starts 2024-09-21T11:50:36.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[29710]: 4.13 scrub starts 2024-09-21T11:50:36.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[29710]: 4.13 scrub ok 2024-09-21T11:50:36.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[29710]: 3.b scrub starts 2024-09-21T11:50:36.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[29710]: 3.b scrub ok 2024-09-21T11:50:36.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[29710]: 3.1b scrub starts 2024-09-21T11:50:36.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[29710]: 3.1b scrub ok 2024-09-21T11:50:36.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/545619043' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:36.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[29710]: 6.1 scrub starts 2024-09-21T11:50:36.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:36 smithi043 ceph-mon[29710]: 6.1 scrub ok 2024-09-21T11:50:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:36 smithi175 ceph-mon[32613]: 9.3 scrub starts 2024-09-21T11:50:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:36 smithi175 ceph-mon[32613]: 4.13 scrub starts 2024-09-21T11:50:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:36 smithi175 ceph-mon[32613]: 4.13 scrub ok 2024-09-21T11:50:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:36 smithi175 ceph-mon[32613]: 3.b scrub starts 2024-09-21T11:50:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:36 smithi175 ceph-mon[32613]: 3.b scrub ok 2024-09-21T11:50:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:36 smithi175 ceph-mon[32613]: 3.1b scrub starts 2024-09-21T11:50:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:36 smithi175 ceph-mon[32613]: 3.1b scrub ok 2024-09-21T11:50:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:36 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/545619043' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:36 smithi175 ceph-mon[32613]: 6.1 scrub starts 2024-09-21T11:50:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:36 smithi175 ceph-mon[32613]: 6.1 scrub ok 2024-09-21T11:50:36.830 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:36.852 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:36.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:36.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:36.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:50:36.954 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:36.981 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:36.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:36.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:37.083 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:37.127 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:37.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:37.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:37.145 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:50:37.145 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "" 2024-09-21T11:50:37.145 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:50:37.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:37.281 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:37.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:37.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:37.283 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:50:37.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 84 16 120104 120088 89.3181 48.5 0.00688123 0.0111914 2024-09-21T11:50:37.383 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:37.392 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:37.411 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:37.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:37.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:37.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:37.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:37.545 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:37.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:37.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:37.680 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:37.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:37.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[36454]: pgmap v510: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-21T11:50:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[36454]: 4.d scrub starts 2024-09-21T11:50:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[36454]: 4.d scrub ok 2024-09-21T11:50:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[36454]: 6.19 scrub starts 2024-09-21T11:50:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[36454]: 6.19 scrub ok 2024-09-21T11:50:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[36454]: 9.0 scrub starts 2024-09-21T11:50:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[29710]: pgmap v510: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-21T11:50:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[29710]: 4.d scrub starts 2024-09-21T11:50:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[29710]: 4.d scrub ok 2024-09-21T11:50:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[29710]: 6.19 scrub starts 2024-09-21T11:50:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[29710]: 6.19 scrub ok 2024-09-21T11:50:37.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:37 smithi043 ceph-mon[29710]: 9.0 scrub starts 2024-09-21T11:50:37.743 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:37 smithi175 ceph-mon[32613]: pgmap v510: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-21T11:50:37.743 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:37 smithi175 ceph-mon[32613]: 4.d scrub starts 2024-09-21T11:50:37.743 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:37 smithi175 ceph-mon[32613]: 4.d scrub ok 2024-09-21T11:50:37.743 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:37 smithi175 ceph-mon[32613]: 6.19 scrub starts 2024-09-21T11:50:37.744 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:37 smithi175 ceph-mon[32613]: 6.19 scrub ok 2024-09-21T11:50:37.744 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:37 smithi175 ceph-mon[32613]: 9.0 scrub starts 2024-09-21T11:50:37.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:37.825 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:37.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:37.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:37.908 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:50:37.908 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:50:37.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:50:37.908 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:50:37.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.908 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:99033 7.5 7:a309294f:::benchmark_data_smithi043_70646_object99032:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:50:37.908 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:26.337954+0000", 2024-09-21T11:50:37.908 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 491.56549617399997, 2024-09-21T11:50:37.908 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10807878899999999, 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99033 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337954+0000", 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337954+0000", 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337955+0000", 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8500000000000001e-06 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337993+0000", 2024-09-21T11:50:37.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7715000000000001e-05 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338011+0000", 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7676000000000002e-05 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338020+0000", 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9209999999999998e-06 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338081+0000", 2024-09-21T11:50:37.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0928999999999999e-05 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338172+0000", 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0969000000000003e-05 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338260+0000", 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8417000000000004e-05 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.340226+0000", 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001966069 2024-09-21T11:50:37.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.433908+0000", 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093681581999999999 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.445962+0000", 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012053895 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.446001+0000", 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9795000000000001e-05 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.446032+0000", 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0970999999999999e-05 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123054 7.5 7:a4098129:::benchmark_data_smithi043_70646_object5646:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:50:37.914 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.880861+0000", 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 462.022588826, 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.098889820000000003, 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123054 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880861+0000", 2024-09-21T11:50:37.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880861+0000", 2024-09-21T11:50:37.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880863+0000", 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6300000000000001e-06 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880867+0000", 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3239999999999996e-06 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880875+0000", 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0639999999999994e-06 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880882+0000", 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2139999999999999e-06 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.880946+0000", 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3366000000000003e-05 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.881054+0000", 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010853000000000001 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:50:37.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.881162+0000", 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010778 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.885528+0000", 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043657339999999996 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.889186+0000", 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036579680000000002 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979646+0000", 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090460503999999997 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979705+0000", 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9070999999999997e-05 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.979751+0000", 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5634999999999999e-05 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.920 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126210 7.5 7:a6687ff4:::benchmark_data_smithi043_70646_object8802:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.192178+0000", 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 458.711271909, 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.099158476999999995, 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.921 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:50:37.922 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:50:37.922 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126210 2024-09-21T11:50:37.922 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.922 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.922 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.922 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.922 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192178+0000", 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192178+0000", 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192180+0000", 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.807e-06 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.923 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192185+0000", 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1250000000000001e-06 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192197+0000", 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2527000000000001e-05 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192205+0000", 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0709999999999995e-06 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192287+0000", 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1976000000000003e-05 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192415+0000", 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127317 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.192525+0000", 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110297 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194639+0000", 2024-09-21T11:50:37.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021140999999999998 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.200841+0000", 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062023790000000001 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291238+0000", 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090396746 2024-09-21T11:50:37.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291292+0000", 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3792000000000003e-05 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291336+0000", 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4339999999999999e-05 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220189 7.5 7:acec303d:::benchmark_data_smithi043_70646_object102781:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.204953+0000", 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 371.69849653900002, 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.102016905, 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220189 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.928 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204953+0000", 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204953+0000", 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204954+0000", 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0049999999999999e-06 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204959+0000", 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.296e-06 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204968+0000", 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7329999999999998e-06 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.204974+0000", 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.276e-06 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205021+0000", 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7154000000000003e-05 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205112+0000", 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1334000000000003e-05 2024-09-21T11:50:37.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.205186+0000", 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4011999999999995e-05 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.206522+0000", 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001336292 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.206787+0000", 2024-09-21T11:50:37.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026466100000000002 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306842+0000", 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.100054927 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306920+0000", 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8430999999999995e-05 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.306970+0000", 2024-09-21T11:50:37.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9784000000000003e-05 2024-09-21T11:50:37.934 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.934 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.934 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.934 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221210 7.5 7:a52e1ee7:::benchmark_data_smithi043_70646_object103802:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:50:37.934 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.174153+0000", 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 370.72929721000003, 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.110515855, 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221210 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174153+0000", 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174153+0000", 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174154+0000", 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3820000000000001e-06 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174159+0000", 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.746e-06 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174171+0000", 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2434e-05 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174178+0000", 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8700000000000003e-06 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174742+0000", 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00056385099999999998 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174850+0000", 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108383 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.174961+0000", 2024-09-21T11:50:37.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110818 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.177355+0000", 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002393825 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.178344+0000", 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00098920699999999998 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.284555+0000", 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10621093 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.284625+0000", 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9937000000000005e-05 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.284668+0000", 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3472e-05 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221223 7.5 7:a91eb7cb:::benchmark_data_smithi043_70646_object103815:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.184471+0000", 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 370.71897843699998, 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10880833400000001, 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221223 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184471+0000", 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184471+0000", 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184473+0000", 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3999999999999999e-06 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184477+0000", 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.33e-06 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184487+0000", 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7510000000000003e-06 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184494+0000", 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9059999999999996e-06 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184572+0000", 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8236000000000001e-05 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184735+0000", 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016326399999999999 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,2", 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.184852+0000", 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117142 2024-09-21T11:50:37.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.187351+0000", 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024981640000000002 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.189227+0000", 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018760070000000001 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293189+0000", 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.103962165 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293240+0000", 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1685999999999999e-05 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293280+0000", 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9283000000000003e-05 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.946 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:30570 9.3 9:cd491152:::benchmark_data_smithi043_126002_object30569:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:25.575481+0000", 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 72.327968756999994, 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.121148906, 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 30570 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.947 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.575481+0000", 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.575481+0000", 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.575482+0000", 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2640000000000001e-06 2024-09-21T11:50:37.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.575514+0000", 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2119000000000002e-05 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.575532+0000", 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7601999999999999e-05 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.575539+0000", 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3300000000000001e-06 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:37.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.575608+0000", 2024-09-21T11:50:37.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8602000000000005e-05 2024-09-21T11:50:37.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.575747+0000", 2024-09-21T11:50:37.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013883400000000001 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.575861+0000", 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011383399999999999 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.577029+0000", 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011685199999999999 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.696486+0000", 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.119457363 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.696523+0000", 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6470000000000001e-05 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.696588+0000", 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5252999999999993e-05 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:25.696630+0000", 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1715000000000003e-05 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:32116 9.3 9:ca366b6e:::benchmark_data_smithi043_126002_object32115:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:26.524402+0000", 2024-09-21T11:50:37.954 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 71.379047557000007, 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.108213595, 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 32116 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524402+0000", 2024-09-21T11:50:37.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524402+0000", 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524404+0000", 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3090000000000001e-06 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524435+0000", 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1134999999999999e-05 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524443+0000", 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4470000000000006e-06 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524450+0000", 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6590000000000001e-06 2024-09-21T11:50:37.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524507+0000", 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6737000000000002e-05 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524627+0000", 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012024 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524748+0000", 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000121662 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.527333+0000", 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025844259999999999 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.527794+0000", 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046133099999999999 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.632541+0000", 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.104746637 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.632576+0000", 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4764000000000001e-05 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.632616+0000", 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0247999999999998e-05 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:32119 9.3 9:ce191138:::benchmark_data_smithi043_126002_object32118:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:26.524532+0000", 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 71.378917954000002, 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10827461100000001, 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.961 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 32119 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524532+0000", 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524532+0000", 2024-09-21T11:50:37.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524532+0000", 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6499999999999999e-07 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524560+0000", 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7212999999999998e-05 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524566+0000", 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3810000000000004e-06 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.524572+0000", 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0190000000000003e-06 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.525895+0000", 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001322634 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.526021+0000", 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012598899999999999 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.526147+0000", 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125995 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.527761+0000", 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001614435 2024-09-21T11:50:37.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.528071+0000", 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030945099999999998 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.632721+0000", 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.104650141 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.632769+0000", 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8136999999999997e-05 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:26.632806+0000", 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7651000000000003e-05 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:33300 9.3 9:c428ed35:::benchmark_data_smithi043_126002_object33299:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.237332+0000", 2024-09-21T11:50:37.967 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.666117904999993, 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.134383526, 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 33300 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237332+0000", 2024-09-21T11:50:37.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237332+0000", 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237333+0000", 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237363+0000", 2024-09-21T11:50:37.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9663e-05 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237371+0000", 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8739999999999995e-06 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237376+0000", 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7790000000000001e-06 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237863+0000", 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00048618499999999998 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237990+0000", 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127274 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238162+0000", 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017167199999999999 2024-09-21T11:50:37.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.240852+0000", 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026904239999999999 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.370262+0000", 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.129410105 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.371631+0000", 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013692159999999999 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.371670+0000", 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9162999999999998e-05 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.371715+0000", 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4941000000000002e-05 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.973 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:33309 9.d 9:b0d5d949:::benchmark_data_smithi043_126002_object33308:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.241419+0000", 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 70.662030829000003, 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13374207499999999, 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:37.974 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 33309 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.241419+0000", 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.241419+0000", 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.241421+0000", 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8050000000000001e-06 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.241494+0000", 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3392999999999993e-05 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.241506+0000", 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1590000000000001e-05 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.241513+0000", 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4970000000000004e-06 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.241569+0000", 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6150999999999998e-05 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.241699+0000", 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129445 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.241843+0000", 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014403699999999999 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.243103+0000", 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001259694 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.243499+0000", 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00039663900000000001 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.375065+0000", 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.131565499 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.375117+0000", 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2460999999999998e-05 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.375161+0000", 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3863999999999999e-05 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34897 9.d 9:bfc2b959:::benchmark_data_smithi043_126002_object34896:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:28.287213+0000", 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.616236803999996, 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14583664900000001, 2024-09-21T11:50:37.980 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34897 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287213+0000", 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287213+0000", 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287214+0000", 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0100000000000004e-07 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287243+0000", 2024-09-21T11:50:37.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9388999999999999e-05 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287255+0000", 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1763000000000001e-05 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287264+0000", 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7669999999999992e-06 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.288881+0000", 2024-09-21T11:50:37.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001617451 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.288998+0000", 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117126 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.289810+0000", 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00081188099999999997 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292344+0000", 2024-09-21T11:50:37.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025342720000000002 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292451+0000", 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106952 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.432926+0000", 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.140474458 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.432998+0000", 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1766000000000004e-05 2024-09-21T11:50:37.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.433050+0000", 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2023e-05 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34906 9.3 9:c91cddb2:::benchmark_data_smithi043_126002_object34905:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:28.292233+0000", 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.611216662999993, 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15032242500000001, 2024-09-21T11:50:37.986 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.987 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.987 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.987 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.987 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:37.987 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:37.987 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34906 2024-09-21T11:50:37.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.987 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292233+0000", 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292233+0000", 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292234+0000", 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2950000000000001e-06 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292286+0000", 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1756999999999999e-05 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292296+0000", 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0210999999999999e-05 2024-09-21T11:50:37.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292302+0000", 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3820000000000003e-06 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292354+0000", 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2340999999999999e-05 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292454+0000", 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100006 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292577+0000", 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122676 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293904+0000", 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013268150000000001 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.434222+0000", 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14031828800000001 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.442495+0000", 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0082727500000000006 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.442532+0000", 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7456999999999997e-05 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.442556+0000", 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3447000000000001e-05 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.995 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34907 9.d 9:bf3be697:::benchmark_data_smithi043_126002_object34906:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:28.292968+0000", 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 69.610482262000005, 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14137534800000001, 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34907 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:37.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292968+0000", 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292968+0000", 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292968+0000", 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7000000000000003e-07 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293002+0000", 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3735999999999997e-05 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293012+0000", 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9559999999999992e-06 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293019+0000", 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5869999999999998e-06 2024-09-21T11:50:37.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293062+0000", 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3631000000000003e-05 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293145+0000", 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2669000000000004e-05 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293228+0000", 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3251000000000006e-05 2024-09-21T11:50:37.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.294386+0000", 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001157822 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.295168+0000", 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00078156199999999997 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.434254+0000", 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13908669700000001 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.434303+0000", 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8146999999999998e-05 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.434343+0000", 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.032e-05 2024-09-21T11:50:38.001 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:36071 9.3 9:cc6f22ec:::benchmark_data_smithi043_126002_object36070:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:28.993426+0000", 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 68.910023952000003, 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11349991199999999, 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:38.002 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 36071 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.993426+0000", 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.993426+0000", 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.993427+0000", 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3340000000000001e-06 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.993449+0000", 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2237000000000001e-05 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:38.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.993457+0000", 2024-09-21T11:50:38.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5170000000000002e-06 2024-09-21T11:50:38.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:38.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.993461+0000", 2024-09-21T11:50:38.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5410000000000002e-06 2024-09-21T11:50:38.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:38.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.993509+0000", 2024-09-21T11:50:38.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7945999999999998e-05 2024-09-21T11:50:38.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:38.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.993638+0000", 2024-09-21T11:50:38.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012886899999999999 2024-09-21T11:50:38.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.993788+0000", 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015014999999999999 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.995064+0000", 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012754520000000001 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.995854+0000", 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079043600000000005 2024-09-21T11:50:38.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.106860+0000", 2024-09-21T11:50:38.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.111006141 2024-09-21T11:50:38.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:38.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.106896+0000", 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5318999999999998e-05 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.106926+0000", 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9969999999999999e-05 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:37430 9.3 9:c97f41d7:::benchmark_data_smithi043_126002_object37429:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:29.911376+0000", 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 67.992074083000006, 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.110905425, 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:38.010 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 37430 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.911376+0000", 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.911376+0000", 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:38.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.911377+0000", 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2899999999999999e-06 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.911412+0000", 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4514999999999997e-05 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.911420+0000", 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1359999999999997e-06 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.911426+0000", 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1979999999999998e-06 2024-09-21T11:50:38.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.911484+0000", 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7908000000000002e-05 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.911604+0000", 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000120025 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.911730+0000", 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012598199999999999 2024-09-21T11:50:38.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.915465+0000", 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003735141 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.915519+0000", 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4601999999999997e-05 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:30.021029+0000", 2024-09-21T11:50:38.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.105509464 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:30.022235+0000", 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001206082 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:30.022281+0000", 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6081999999999997e-05 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:37432 9.3 9:ce1d622a:::benchmark_data_smithi043_126002_object37431:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:29.912732+0000", 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 67.990717735999993, 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.109680834, 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 37432 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.016 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.912732+0000", 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.912732+0000", 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.912733+0000", 2024-09-21T11:50:38.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.015e-06 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.912766+0000", 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3422e-05 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.912778+0000", 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.181e-05 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:38.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.912786+0000", 2024-09-21T11:50:38.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4120000000000004e-06 2024-09-21T11:50:38.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:38.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.913082+0000", 2024-09-21T11:50:38.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029585599999999999 2024-09-21T11:50:38.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.913204+0000", 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122213 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.913337+0000", 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013278899999999999 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.915501+0000", 2024-09-21T11:50:38.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002164539 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:29.915525+0000", 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4235000000000001e-05 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:30.022330+0000", 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.106804963 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:30.022375+0000", 2024-09-21T11:50:38.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4214000000000001e-05 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:30.022413+0000", 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8365999999999998e-05 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:39514 9.3 9:c09b283c:::benchmark_data_smithi043_126002_object39513:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e255)", 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:31.204766+0000", 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 66.698683962000004, 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14268297999999999, 2024-09-21T11:50:38.022 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39514 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204766+0000", 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.023 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204766+0000", 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204767+0000", 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5000000000000001e-07 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204836+0000", 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8992000000000006e-05 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204843+0000", 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5410000000000003e-06 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204848+0000", 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7060000000000003e-06 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204893+0000", 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5222000000000002e-05 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204990+0000", 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6620000000000007e-05 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.205093+0000", 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010343799999999999 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.206689+0000", 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015956360000000001 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.207481+0000", 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079241399999999997 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.347380+0000", 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.139898839 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.347409+0000", 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8858999999999999e-05 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.347449+0000", 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9762999999999999e-05 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:119798 9.3 9:c29d2bd1:::benchmark_data_smithi043_126002_object119797:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:36.882211+0000", 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.0212385930000001, 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15405127199999999, 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:38.028 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 119798 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882211+0000", 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882211+0000", 2024-09-21T11:50:38.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882213+0000", 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0779999999999998e-06 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882247+0000", 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3668999999999999e-05 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:38.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882260+0000", 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2597e-05 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882269+0000", 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-06 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.893113+0000", 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010843939 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:38.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.893287+0000", 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000174079 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.893502+0000", 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021474999999999999 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.902743+0000", 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092404410000000003 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.904342+0000", 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015994970000000001 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.036159+0000", 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13181647899999999 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.036209+0000", 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0541000000000003e-05 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.036262+0000", 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3276000000000003e-05 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:119810 9.3 9:c8160ff6:::benchmark_data_smithi043_126002_object119809:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:36.892458+0000", 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 1.0109916430000001, 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.16747954200000001, 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:50:38.034 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 119810 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892458+0000", 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.036 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:50:38.036 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892458+0000", 2024-09-21T11:50:38.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:50:38.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:50:38.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892460+0000", 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7179999999999999e-06 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892523+0000", 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3162000000000003e-05 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892534+0000", 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1422e-05 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892541+0000", 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9890000000000003e-06 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.896348+0000", 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0038067639999999998 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.896494+0000", 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000145592 2024-09-21T11:50:38.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.896641+0000", 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014765899999999999 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.904387+0000", 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0077451430000000003 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.905317+0000", 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00093077800000000005 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.059807+0000", 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15448937300000001 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.059867+0000", 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0757000000000001e-05 2024-09-21T11:50:38.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:50:38.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:50:38.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:50:38.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.059938+0000", 2024-09-21T11:50:38.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0185000000000006e-05 2024-09-21T11:50:38.042 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.042 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:38.042 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.042 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:50:38.042 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:50:38.042 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:50:38.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:38.043 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:38.043 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:38.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:38.183 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:38.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:38.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:38.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 85 16 121244 121228 89.1055 71.25 0.0159908 0.011218 2024-09-21T11:50:38.285 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:38.316 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:38.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:38.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:38.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:38.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:38.460 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:38.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:38.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:38.603 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:38.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:38.604 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:38.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:38.738 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:38.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:38.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:38 smithi043 ceph-mon[29710]: 4.e scrub starts 2024-09-21T11:50:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:38 smithi043 ceph-mon[29710]: 4.e scrub ok 2024-09-21T11:50:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:38 smithi043 ceph-mon[29710]: 9.3 scrub ok 2024-09-21T11:50:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:38 smithi043 ceph-mon[29710]: 4.14 scrub starts 2024-09-21T11:50:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:38 smithi043 ceph-mon[29710]: 4.14 scrub ok 2024-09-21T11:50:38.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:38 smithi043 ceph-mon[36454]: 4.e scrub starts 2024-09-21T11:50:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:38 smithi043 ceph-mon[36454]: 4.e scrub ok 2024-09-21T11:50:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:38 smithi043 ceph-mon[36454]: 9.3 scrub ok 2024-09-21T11:50:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:38 smithi043 ceph-mon[36454]: 4.14 scrub starts 2024-09-21T11:50:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:38 smithi043 ceph-mon[36454]: 4.14 scrub ok 2024-09-21T11:50:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:38 smithi175 ceph-mon[32613]: 4.e scrub starts 2024-09-21T11:50:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:38 smithi175 ceph-mon[32613]: 4.e scrub ok 2024-09-21T11:50:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:38 smithi175 ceph-mon[32613]: 9.3 scrub ok 2024-09-21T11:50:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:38 smithi175 ceph-mon[32613]: 4.14 scrub starts 2024-09-21T11:50:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:38 smithi175 ceph-mon[32613]: 4.14 scrub ok 2024-09-21T11:50:38.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:38.873 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:38.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:38.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:38.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:39.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:39.013 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:39.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:50:39.114 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:39.166 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:39.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:39.168 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:39.269 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:39.283 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 86 16 122299 122283 88.8361 65.9375 0.0178236 0.0112519 2024-09-21T11:50:39.305 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:39.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:39.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:39.407 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:39.434 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:39.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:39.435 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:39.535 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:39.562 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:39.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:39.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:39.607 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:39.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:39.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:39.719 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:39.719 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[36454]: 5.5 scrub starts 2024-09-21T11:50:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[36454]: 5.5 scrub ok 2024-09-21T11:50:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[36454]: pgmap v511: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 979 op/s 2024-09-21T11:50:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[36454]: 4.1 scrub starts 2024-09-21T11:50:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[36454]: 4.1 scrub ok 2024-09-21T11:50:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[36454]: 5.11 scrub starts 2024-09-21T11:50:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[36454]: 5.11 scrub ok 2024-09-21T11:50:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[36454]: 5.19 scrub starts 2024-09-21T11:50:39.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[36454]: 5.19 scrub ok 2024-09-21T11:50:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[29710]: 5.5 scrub starts 2024-09-21T11:50:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[29710]: 5.5 scrub ok 2024-09-21T11:50:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[29710]: pgmap v511: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 979 op/s 2024-09-21T11:50:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[29710]: 4.1 scrub starts 2024-09-21T11:50:39.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[29710]: 4.1 scrub ok 2024-09-21T11:50:39.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[29710]: 5.11 scrub starts 2024-09-21T11:50:39.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[29710]: 5.11 scrub ok 2024-09-21T11:50:39.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[29710]: 5.19 scrub starts 2024-09-21T11:50:39.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:39 smithi043 ceph-mon[29710]: 5.19 scrub ok 2024-09-21T11:50:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:39 smithi175 ceph-mon[32613]: 5.5 scrub starts 2024-09-21T11:50:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:39 smithi175 ceph-mon[32613]: 5.5 scrub ok 2024-09-21T11:50:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:39 smithi175 ceph-mon[32613]: pgmap v511: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 61 MiB/s wr, 979 op/s 2024-09-21T11:50:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:39 smithi175 ceph-mon[32613]: 4.1 scrub starts 2024-09-21T11:50:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:39 smithi175 ceph-mon[32613]: 4.1 scrub ok 2024-09-21T11:50:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:39 smithi175 ceph-mon[32613]: 5.11 scrub starts 2024-09-21T11:50:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:39 smithi175 ceph-mon[32613]: 5.11 scrub ok 2024-09-21T11:50:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:39 smithi175 ceph-mon[32613]: 5.19 scrub starts 2024-09-21T11:50:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:39 smithi175 ceph-mon[32613]: 5.19 scrub ok 2024-09-21T11:50:39.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:39.858 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:39.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:39.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:39.960 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:39.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:39.995 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:39.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:40.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:50:40.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:40.161 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:40.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:40.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:40.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:40.284 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 87 16 123468 123452 88.6547 73.0625 0.0131865 0.0112749 2024-09-21T11:50:40.308 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:40.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:40.309 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:40.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:40.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:40.447 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:40.447 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:40.493 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:50:40.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:40.587 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:40.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:40.588 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:40.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:40.723 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:40.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:40.724 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[36454]: 9.0 scrub ok 2024-09-21T11:50:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[36454]: 4.1a scrub starts 2024-09-21T11:50:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[36454]: 4.1a scrub ok 2024-09-21T11:50:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:50:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:50:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/589847568' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[29710]: 9.0 scrub ok 2024-09-21T11:50:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[29710]: 4.1a scrub starts 2024-09-21T11:50:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[29710]: 4.1a scrub ok 2024-09-21T11:50:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:50:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:50:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/589847568' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:40.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:40 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:40.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:40 smithi175 ceph-mon[32613]: 9.0 scrub ok 2024-09-21T11:50:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:40 smithi175 ceph-mon[32613]: 4.1a scrub starts 2024-09-21T11:50:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:40 smithi175 ceph-mon[32613]: 4.1a scrub ok 2024-09-21T11:50:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:40 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:50:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:40 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:50:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:40 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/589847568' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:40 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:50:40.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:40.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:40.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:40.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:40.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:40.998 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:40.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:41.099 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:41.130 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:41.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:41.131 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:41.232 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:41.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:41.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:41.284 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 88 16 124615 124599 88.4618 71.6875 0.0345591 0.0113 2024-09-21T11:50:41.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:41.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:41.422 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:41.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:41.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:41.562 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:41.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:41.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:41.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:41.702 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:41.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:41.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:41 smithi043 ceph-mon[29710]: pgmap v512: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 884 B/s rd, 53 MiB/s wr, 846 op/s 2024-09-21T11:50:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:41 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:41 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:50:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:41 smithi043 ceph-mon[29710]: osdmap e278: 8 total, 8 up, 8 in 2024-09-21T11:50:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:41 smithi043 ceph-mon[36454]: pgmap v512: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 884 B/s rd, 53 MiB/s wr, 846 op/s 2024-09-21T11:50:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:41 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:41 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:50:41.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:41 smithi043 ceph-mon[36454]: osdmap e278: 8 total, 8 up, 8 in 2024-09-21T11:50:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:41 smithi175 ceph-mon[32613]: pgmap v512: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 884 B/s rd, 53 MiB/s wr, 846 op/s 2024-09-21T11:50:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:41 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:41 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:50:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:41 smithi175 ceph-mon[32613]: osdmap e278: 8 total, 8 up, 8 in 2024-09-21T11:50:41.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:50:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:50:41.804 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:41.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:41.836 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:41.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:41.937 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:41.972 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:41.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:41.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:42.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:42.116 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:42.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:42.117 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:42.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:42.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:42.259 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:42.284 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 89 16 125826 125810 88.3183 75.6875 0.00636925 0.0113182 2024-09-21T11:50:42.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:42.374 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:124522 9.8 e278/247 9:1725295d:::benchmark_data_smithi043_126002_object124521:head v 278'7750, mlcod=278'7748)", 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:41.205885+0000", 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0048329130000000003, 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0048635659999999997, 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:41.205885+0000", 2024-09-21T11:50:42.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:41.205885+0000", 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:41.205886+0000", 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4e-07 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:41.205910+0000", 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3309e-05 2024-09-21T11:50:42.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:41.205920+0000", 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0210000000000001e-05 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:41.205927+0000", 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6959999999999999e-06 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:41.206002+0000", 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5891999999999998e-05 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:42.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:42.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:42.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:41.206022+0000", 2024-09-21T11:50:42.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9412e-05 2024-09-21T11:50:42.378 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:42.378 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:42.378 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:42.378 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:42.378 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:50:42.378 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 1 2024-09-21T11:50:42.378 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:50:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:42.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:42.398 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:42.499 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:42.530 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:42.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:42.531 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:42.632 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:42.671 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:42.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:42.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:42.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:42.774 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:42.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:42.816 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:42.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:42.954 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:42.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:42.955 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:43.056 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:43.086 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:43.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:43.087 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:43.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:43.228 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:43.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:43.229 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:43.284 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Total time run: 90.0093 2024-09-21T11:50:43.284 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Total writes made: 127063 2024-09-21T11:50:43.284 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Write size: 65536 2024-09-21T11:50:43.284 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Object size: 65536 2024-09-21T11:50:43.284 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Bandwidth (MB/sec): 88.2291 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev Bandwidth: 31.6768 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max bandwidth (MB/sec): 197.5 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min bandwidth (MB/sec): 48.5 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Average IOPS: 1411 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev IOPS: 506.828 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max IOPS: 3160 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min IOPS: 776 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Average Latency(s): 0.01133 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev Latency(s): 0.00937882 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max latency(s): 0.219963 2024-09-21T11:50:43.285 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min latency(s): 0.00137184 2024-09-21T11:50:43.289 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Cleaning up (deleting benchmark objects) 2024-09-21T11:50:43.330 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:43.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:43.372 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:43.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:43.473 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:43.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:43.502 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:43.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:43.555 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:50:43.555 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:50:43.556 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_2 2024-09-21T11:50:43.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:50:43.604 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:43.634 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:43.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:43.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:43.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:43.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:43 smithi175 ceph-mon[32613]: pgmap v514: 148 pgs: 148 active+clean; 7.5 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 55 MiB/s wr, 879 op/s 2024-09-21T11:50:43.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:43.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:43.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:43 smithi043 ceph-mon[29710]: pgmap v514: 148 pgs: 148 active+clean; 7.5 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 55 MiB/s wr, 879 op/s 2024-09-21T11:50:43.783 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:43 smithi043 ceph-mon[36454]: pgmap v514: 148 pgs: 148 active+clean; 7.5 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1023 B/s rd, 55 MiB/s wr, 879 op/s 2024-09-21T11:50:43.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:50:43.876 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:43.888 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:43.902 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:43.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:43.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:44.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:44.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:44.068 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:44.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:44.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:44.194 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:44.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:44.195 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:44.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:44.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:44.347 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:44.448 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:44.473 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:44.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:44.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:44.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:44.600 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:44.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:44.601 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:44.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:44.729 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:44.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:44.730 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:44.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:44.886 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:44.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:44.888 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:44.918 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:44.921 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:44.933 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:50:44.934 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:50:44.973 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:50:44.973 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:50:44.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:45.008 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:50:45.009 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:50:45.016 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:45.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:45.018 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:45.043 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:50:45.043 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:50:45.084 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:50:45.084 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:50:45.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:50:45.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:45.125 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:50:45.125 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:50:45.145 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:45.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:45.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:45.159 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:50:45.159 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:50:45.197 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:50:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:50:45.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:45.273 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:45.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:45.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:45.374 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:45.401 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:45.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:45.402 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:45.499 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:50:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:50:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:50:45.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:45.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:45.553 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:45.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:45.686 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:45.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:45.687 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:45 smithi175 ceph-mon[32613]: pgmap v515: 148 pgs: 148 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-21T11:50:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1182478162' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:45.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:45.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:45 smithi043 ceph-mon[36454]: pgmap v515: 148 pgs: 148 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-21T11:50:45.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1182478162' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:45.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:45 smithi043 ceph-mon[29710]: pgmap v515: 148 pgs: 148 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-21T11:50:45.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1182478162' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:45.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:45.827 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:45.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:45.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:45.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:46.068 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:46.090 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:50:46.092 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:50:46.109 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:46.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:46.110 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:46.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:46.249 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:46.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:46.250 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:46.351 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:46.386 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:46.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:46.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:46.488 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:46.514 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:46.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:46.515 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:46.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:46.650 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:46.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:46.651 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:46.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:46.786 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:46.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:46.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:46.888 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:46.916 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:46.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:46.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:47.018 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:47.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:47.047 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:47.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:47.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:47.192 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:47.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:47.193 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:47.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:47.327 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:47.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:47.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:47.405 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-21T11:50:45.981774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8181651995,"num_objects":125032,"num_object_clones":0,"num_object_copies":375096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125032,"num_whiteouts":0,"num_read":1897,"num_read_kb":1754,"num_write":125668,"num_write_kb":7990985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":596864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83778,"ondisk_log_size":83778,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":27108816,"kb_used_data":23860612,"kb_used_omap":12,"kb_used_meta":3248051,"kb_avail":722852400,"statfs":{"total":767960285184,"available":740200857600,"internally_reserved":0,"allocated":24433266688,"data_stored":24430164907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":3326004786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,3,4,5,4,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672071680,"num_objects":10255,"num_object_clones":0,"num_object_copies":30765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10255,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10255,"num_write_kb":656320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003565"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":470,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.490896+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:50:40.490896+0000","last_peered":"2024-09-21T11:50:40.490896+0000","last_clean":"2024-09-21T11:50:40.490896+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:50:40.490896+0000","last_undegraded":"2024-09-21T11:50:40.490896+0000","last_fullsized":"2024-09-21T11:50:40.490896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.789381+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":541,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.496886+0000","last_change":"2024-09-21T11:49:59.177549+0000","last_active":"2024-09-21T11:50:40.496886+0000","last_peered":"2024-09-21T11:50:40.496886+0000","last_clean":"2024-09-21T11:50:40.496886+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:50:40.496886+0000","last_undegraded":"2024-09-21T11:50:40.496886+0000","last_fullsized":"2024-09-21T11:50:40.496886+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:49:59.177497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:59.177497+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:01.997293+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":171,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507067+0000","last_change":"2024-09-21T11:50:39.017249+0000","last_active":"2024-09-21T11:50:40.507067+0000","last_peered":"2024-09-21T11:50:40.507067+0000","last_clean":"2024-09-21T11:50:40.507067+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:50:40.507067+0000","last_undegraded":"2024-09-21T11:50:40.507067+0000","last_fullsized":"2024-09-21T11:50:40.507067+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:39.017173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.017173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:55.424271+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":488,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497080+0000","last_change":"2024-09-21T11:49:08.053201+0000","last_active":"2024-09-21T11:50:40.497080+0000","last_peered":"2024-09-21T11:50:40.497080+0000","last_clean":"2024-09-21T11:50:40.497080+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:50:40.497080+0000","last_undegraded":"2024-09-21T11:50:40.497080+0000","last_fullsized":"2024-09-21T11:50:40.497080+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":474,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.505981+0000","last_change":"2024-09-21T11:50:26.427803+0000","last_active":"2024-09-21T11:50:40.505981+0000","last_peered":"2024-09-21T11:50:40.505981+0000","last_clean":"2024-09-21T11:50:40.505981+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:50:40.505981+0000","last_undegraded":"2024-09-21T11:50:40.505981+0000","last_fullsized":"2024-09-21T11:50:40.505981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:26.427733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.427733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:34.278269+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":463,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506808+0000","last_change":"2024-09-21T11:46:55.804484+0000","last_active":"2024-09-21T11:50:40.506808+0000","last_peered":"2024-09-21T11:50:40.506808+0000","last_clean":"2024-09-21T11:50:40.506808+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:50:40.506808+0000","last_undegraded":"2024-09-21T11:50:40.506808+0000","last_fullsized":"2024-09-21T11:50:40.506808+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:55.804419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:55.804419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:43.180856+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":509,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506226+0000","last_change":"2024-09-21T11:50:03.300848+0000","last_active":"2024-09-21T11:50:40.506226+0000","last_peered":"2024-09-21T11:50:40.506226+0000","last_clean":"2024-09-21T11:50:40.506226+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:50:40.506226+0000","last_undegraded":"2024-09-21T11:50:40.506226+0000","last_fullsized":"2024-09-21T11:50:40.506226+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:50:03.300772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:03.300772+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.852364+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":492,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.496804+0000","last_change":"2024-09-21T11:50:31.279422+0000","last_active":"2024-09-21T11:50:40.496804+0000","last_peered":"2024-09-21T11:50:40.496804+0000","last_clean":"2024-09-21T11:50:40.496804+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:50:40.496804+0000","last_undegraded":"2024-09-21T11:50:40.496804+0000","last_fullsized":"2024-09-21T11:50:40.496804+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.279369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.279369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:42.707926+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":469,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493151+0000","last_change":"2024-09-21T11:48:47.077320+0000","last_active":"2024-09-21T11:50:40.493151+0000","last_peered":"2024-09-21T11:50:40.493151+0000","last_clean":"2024-09-21T11:50:40.493151+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:50:40.493151+0000","last_undegraded":"2024-09-21T11:50:40.493151+0000","last_fullsized":"2024-09-21T11:50:40.493151+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.077259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.077259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:13.765851+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":515,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497322+0000","last_change":"2024-09-21T11:50:39.302183+0000","last_active":"2024-09-21T11:50:40.497322+0000","last_peered":"2024-09-21T11:50:40.497322+0000","last_clean":"2024-09-21T11:50:40.497322+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:50:40.497322+0000","last_undegraded":"2024-09-21T11:50:40.497322+0000","last_fullsized":"2024-09-21T11:50:40.497322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:50:39.302102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.302102+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:08.488427+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":469,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.492402+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:50:40.492402+0000","last_peered":"2024-09-21T11:50:40.492402+0000","last_clean":"2024-09-21T11:50:40.492402+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:50:40.492402+0000","last_undegraded":"2024-09-21T11:50:40.492402+0000","last_fullsized":"2024-09-21T11:50:40.492402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.828789+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":479,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493443+0000","last_change":"2024-09-21T11:48:44.523906+0000","last_active":"2024-09-21T11:50:40.493443+0000","last_peered":"2024-09-21T11:50:40.493443+0000","last_clean":"2024-09-21T11:50:40.493443+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:50:40.493443+0000","last_undegraded":"2024-09-21T11:50:40.493443+0000","last_fullsized":"2024-09-21T11:50:40.493443+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:44.523826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.523826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:11.691589+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":499,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.492529+0000","last_change":"2024-09-21T11:50:27.746458+0000","last_active":"2024-09-21T11:50:40.492529+0000","last_peered":"2024-09-21T11:50:40.492529+0000","last_clean":"2024-09-21T11:50:40.492529+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:50:40.492529+0000","last_undegraded":"2024-09-21T11:50:40.492529+0000","last_fullsized":"2024-09-21T11:50:40.492529+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:27.746396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.746396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:48.701384+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":521,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497132+0000","last_change":"2024-09-21T11:49:14.010678+0000","last_active":"2024-09-21T11:50:40.497132+0000","last_peered":"2024-09-21T11:50:40.497132+0000","last_clean":"2024-09-21T11:50:40.497132+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:50:40.497132+0000","last_undegraded":"2024-09-21T11:50:40.497132+0000","last_fullsized":"2024-09-21T11:50:40.497132+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:40.823398+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":483,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.491472+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:50:40.491472+0000","last_peered":"2024-09-21T11:50:40.491472+0000","last_clean":"2024-09-21T11:50:40.491472+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:50:40.491472+0000","last_undegraded":"2024-09-21T11:50:40.491472+0000","last_fullsized":"2024-09-21T11:50:40.491472+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:45.139772+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":487,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.492303+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:50:40.492303+0000","last_peered":"2024-09-21T11:50:40.492303+0000","last_clean":"2024-09-21T11:50:40.492303+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:50:40.492303+0000","last_undegraded":"2024-09-21T11:50:40.492303+0000","last_fullsized":"2024-09-21T11:50:40.492303+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:43.301832+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":483,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.498130+0000","last_change":"2024-09-21T11:50:32.302938+0000","last_active":"2024-09-21T11:50:40.498130+0000","last_peered":"2024-09-21T11:50:40.498130+0000","last_clean":"2024-09-21T11:50:40.498130+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:50:40.498130+0000","last_undegraded":"2024-09-21T11:50:40.498130+0000","last_fullsized":"2024-09-21T11:50:40.498130+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:32.302842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.302842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:55.927870+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":469,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.494620+0000","last_change":"2024-09-21T11:50:35.718093+0000","last_active":"2024-09-21T11:50:40.494620+0000","last_peered":"2024-09-21T11:50:40.494620+0000","last_clean":"2024-09-21T11:50:40.494620+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:50:40.494620+0000","last_undegraded":"2024-09-21T11:50:40.494620+0000","last_fullsized":"2024-09-21T11:50:40.494620+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:35.717991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.717991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.422206+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":602,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.500822+0000","last_change":"2024-09-21T11:50:25.966805+0000","last_active":"2024-09-21T11:50:40.500822+0000","last_peered":"2024-09-21T11:50:40.500822+0000","last_clean":"2024-09-21T11:50:40.500822+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:50:40.500822+0000","last_undegraded":"2024-09-21T11:50:40.500822+0000","last_fullsized":"2024-09-21T11:50:40.500822+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:50:25.966740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.966740+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.945863+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":179,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507931+0000","last_change":"2024-09-21T11:50:31.012243+0000","last_active":"2024-09-21T11:50:40.507931+0000","last_peered":"2024-09-21T11:50:40.507931+0000","last_clean":"2024-09-21T11:50:40.507931+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:50:40.507931+0000","last_undegraded":"2024-09-21T11:50:40.507931+0000","last_fullsized":"2024-09-21T11:50:40.507931+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.012177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.012177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.366361+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":467,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507055+0000","last_change":"2024-09-21T11:50:06.400378+0000","last_active":"2024-09-21T11:50:40.507055+0000","last_peered":"2024-09-21T11:50:40.507055+0000","last_clean":"2024-09-21T11:50:40.507055+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:50:40.507055+0000","last_undegraded":"2024-09-21T11:50:40.507055+0000","last_fullsized":"2024-09-21T11:50:40.507055+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:06.400307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.400307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.097684+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":492,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497445+0000","last_change":"2024-09-21T11:49:50.122297+0000","last_active":"2024-09-21T11:50:40.497445+0000","last_peered":"2024-09-21T11:50:40.497445+0000","last_clean":"2024-09-21T11:50:40.497445+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:50:40.497445+0000","last_undegraded":"2024-09-21T11:50:40.497445+0000","last_fullsized":"2024-09-21T11:50:40.497445+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.122227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.122227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:55.903276+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":536,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506631+0000","last_change":"2024-09-21T11:47:03.873307+0000","last_active":"2024-09-21T11:50:40.506631+0000","last_peered":"2024-09-21T11:50:40.506631+0000","last_clean":"2024-09-21T11:50:40.506631+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:50:40.506631+0000","last_undegraded":"2024-09-21T11:50:40.506631+0000","last_fullsized":"2024-09-21T11:50:40.506631+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:47:03.873239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:47:03.873239+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.476375+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":470,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.498396+0000","last_change":"2024-09-21T11:48:51.931885+0000","last_active":"2024-09-21T11:50:40.498396+0000","last_peered":"2024-09-21T11:50:40.498396+0000","last_clean":"2024-09-21T11:50:40.498396+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:50:40.498396+0000","last_undegraded":"2024-09-21T11:50:40.498396+0000","last_fullsized":"2024-09-21T11:50:40.498396+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:51.931832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:51.931832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:13.265721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":482,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.490039+0000","last_change":"2024-09-21T11:50:27.321818+0000","last_active":"2024-09-21T11:50:40.490039+0000","last_peered":"2024-09-21T11:50:40.490039+0000","last_clean":"2024-09-21T11:50:40.490039+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:50:40.490039+0000","last_undegraded":"2024-09-21T11:50:40.490039+0000","last_fullsized":"2024-09-21T11:50:40.490039+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:37.464853+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":175,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507271+0000","last_change":"2024-09-21T11:50:02.920313+0000","last_active":"2024-09-21T11:50:40.507271+0000","last_peered":"2024-09-21T11:50:40.507271+0000","last_clean":"2024-09-21T11:50:40.507271+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:50:40.507271+0000","last_undegraded":"2024-09-21T11:50:40.507271+0000","last_fullsized":"2024-09-21T11:50:40.507271+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:02.920229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.920229+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:20.782582+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":512,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493476+0000","last_change":"2024-09-21T11:49:58.702620+0000","last_active":"2024-09-21T11:50:40.493476+0000","last_peered":"2024-09-21T11:50:40.493476+0000","last_clean":"2024-09-21T11:50:40.493476+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:50:40.493476+0000","last_undegraded":"2024-09-21T11:50:40.493476+0000","last_fullsized":"2024-09-21T11:50:40.493476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:49:58.702544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.702544+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.427898+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":626,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.498122+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:50:40.498122+0000","last_peered":"2024-09-21T11:50:40.498122+0000","last_clean":"2024-09-21T11:50:40.498122+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:50:40.498122+0000","last_undegraded":"2024-09-21T11:50:40.498122+0000","last_fullsized":"2024-09-21T11:50:40.498122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"278'7877","reported_seq":7950,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:42.042254+0000","last_change":"2024-09-21T11:49:08.748763+0000","last_active":"2024-09-21T11:50:42.042254+0000","last_peered":"2024-09-21T11:50:42.042254+0000","last_clean":"2024-09-21T11:50:42.042254+0000","last_became_active":"2024-09-21T11:49:08.748359+0000","last_became_peered":"2024-09-21T11:49:08.748359+0000","last_unstale":"2024-09-21T11:50:42.042254+0000","last_undegraded":"2024-09-21T11:50:42.042254+0000","last_fullsized":"2024-09-21T11:50:42.042254+0000","mapping_epoch":247,"log_start":"255'3000","ondisk_log_start":"255'3000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4877,"log_dups_size":0,"ondisk_log_size":4877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516227072,"num_objects":7877,"num_object_clones":0,"num_object_copies":23631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7877,"num_write_kb":504128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":205,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507933+0000","last_change":"2024-09-21T11:49:15.740456+0000","last_active":"2024-09-21T11:50:40.507933+0000","last_peered":"2024-09-21T11:50:40.507933+0000","last_clean":"2024-09-21T11:50:40.507933+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:50:40.507933+0000","last_undegraded":"2024-09-21T11:50:40.507933+0000","last_fullsized":"2024-09-21T11:50:40.507933+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:49:15.739582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:15.739582+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.931869+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":500,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508525+0000","last_change":"2024-09-21T11:49:47.205216+0000","last_active":"2024-09-21T11:50:40.508525+0000","last_peered":"2024-09-21T11:50:40.508525+0000","last_clean":"2024-09-21T11:50:40.508525+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:50:40.508525+0000","last_undegraded":"2024-09-21T11:50:40.508525+0000","last_fullsized":"2024-09-21T11:50:40.508525+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.204889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.204889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:52.456856+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":600,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.498794+0000","last_change":"2024-09-21T11:47:01.933697+0000","last_active":"2024-09-21T11:50:40.498794+0000","last_peered":"2024-09-21T11:50:40.498794+0000","last_clean":"2024-09-21T11:50:40.498794+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:50:40.498794+0000","last_undegraded":"2024-09-21T11:50:40.498794+0000","last_fullsized":"2024-09-21T11:50:40.498794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:01.933648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:01.933648+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:11.347125+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":487,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.492595+0000","last_change":"2024-09-21T11:50:25.708598+0000","last_active":"2024-09-21T11:50:40.492595+0000","last_peered":"2024-09-21T11:50:40.492595+0000","last_clean":"2024-09-21T11:50:40.492595+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:50:40.492595+0000","last_undegraded":"2024-09-21T11:50:40.492595+0000","last_fullsized":"2024-09-21T11:50:40.492595+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:28.981755+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"278'7742","reported_seq":7892,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.015717+0000","last_change":"2024-09-21T11:50:28.231461+0000","last_active":"2024-09-21T11:50:41.015717+0000","last_peered":"2024-09-21T11:50:41.015717+0000","last_clean":"2024-09-21T11:50:41.015717+0000","last_became_active":"2024-09-21T11:49:08.519622+0000","last_became_peered":"2024-09-21T11:49:08.519622+0000","last_unstale":"2024-09-21T11:50:41.015717+0000","last_undegraded":"2024-09-21T11:50:41.015717+0000","last_fullsized":"2024-09-21T11:50:41.015717+0000","mapping_epoch":247,"log_start":"255'2900","ondisk_log_start":"255'2900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6919","last_scrub_stamp":"2024-09-21T11:50:28.231378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.231378+0000","objects_scrubbed":6856,"log_size":4842,"log_dups_size":0,"ondisk_log_size":4842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.288046+0000","scrub_duration":1964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":507379712,"num_objects":7742,"num_object_clones":0,"num_object_copies":23226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7742,"num_write_kb":495488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":533,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508621+0000","last_change":"2024-09-21T11:49:52.209526+0000","last_active":"2024-09-21T11:50:40.508621+0000","last_peered":"2024-09-21T11:50:40.508621+0000","last_clean":"2024-09-21T11:50:40.508621+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:50:40.508621+0000","last_undegraded":"2024-09-21T11:50:40.508621+0000","last_fullsized":"2024-09-21T11:50:40.508621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:49:52.209433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:52.209433+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.566988+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":486,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508168+0000","last_change":"2024-09-21T11:49:55.238709+0000","last_active":"2024-09-21T11:50:40.508168+0000","last_peered":"2024-09-21T11:50:40.508168+0000","last_clean":"2024-09-21T11:50:40.508168+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:50:40.508168+0000","last_undegraded":"2024-09-21T11:50:40.508168+0000","last_fullsized":"2024-09-21T11:50:40.508168+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:55.238302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:55.238302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:06.703143+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":471,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497066+0000","last_change":"2024-09-21T11:48:47.933145+0000","last_active":"2024-09-21T11:50:40.497066+0000","last_peered":"2024-09-21T11:50:40.497066+0000","last_clean":"2024-09-21T11:50:40.497066+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:50:40.497066+0000","last_undegraded":"2024-09-21T11:50:40.497066+0000","last_fullsized":"2024-09-21T11:50:40.497066+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.933099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.933099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.352011+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":459,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497244+0000","last_change":"2024-09-21T11:46:24.717355+0000","last_active":"2024-09-21T11:50:40.497244+0000","last_peered":"2024-09-21T11:50:40.497244+0000","last_clean":"2024-09-21T11:50:40.497244+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:50:40.497244+0000","last_undegraded":"2024-09-21T11:50:40.497244+0000","last_fullsized":"2024-09-21T11:50:40.497244+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:24.717285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.717285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.284555+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"278'7865","reported_seq":7938,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:42.059456+0000","last_change":"2024-09-21T11:49:08.609939+0000","last_active":"2024-09-21T11:50:42.059456+0000","last_peered":"2024-09-21T11:50:42.059456+0000","last_clean":"2024-09-21T11:50:42.059456+0000","last_became_active":"2024-09-21T11:49:08.609618+0000","last_became_peered":"2024-09-21T11:49:08.609618+0000","last_unstale":"2024-09-21T11:50:42.059456+0000","last_undegraded":"2024-09-21T11:50:42.059456+0000","last_fullsized":"2024-09-21T11:50:42.059456+0000","mapping_epoch":247,"log_start":"255'3000","ondisk_log_start":"255'3000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4865,"log_dups_size":0,"ondisk_log_size":4865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":515440640,"num_objects":7865,"num_object_clones":0,"num_object_copies":23595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7865,"num_write_kb":503360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":541,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497332+0000","last_change":"2024-09-21T11:50:38.340031+0000","last_active":"2024-09-21T11:50:40.497332+0000","last_peered":"2024-09-21T11:50:40.497332+0000","last_clean":"2024-09-21T11:50:40.497332+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:50:40.497332+0000","last_undegraded":"2024-09-21T11:50:40.497332+0000","last_fullsized":"2024-09-21T11:50:40.497332+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:02.032406+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":516,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493940+0000","last_change":"2024-09-21T11:50:28.657341+0000","last_active":"2024-09-21T11:50:40.493940+0000","last_peered":"2024-09-21T11:50:40.493940+0000","last_clean":"2024-09-21T11:50:40.493940+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:50:40.493940+0000","last_undegraded":"2024-09-21T11:50:40.493940+0000","last_fullsized":"2024-09-21T11:50:40.493940+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:28.657261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.657261+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:46.131391+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":631,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506998+0000","last_change":"2024-09-21T11:49:14.198995+0000","last_active":"2024-09-21T11:50:40.506998+0000","last_peered":"2024-09-21T11:50:40.506998+0000","last_clean":"2024-09-21T11:50:40.506998+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:50:40.506998+0000","last_undegraded":"2024-09-21T11:50:40.506998+0000","last_fullsized":"2024-09-21T11:50:40.506998+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:49:14.198804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.198804+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":477,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.488134+0000","last_change":"2024-09-21T11:50:31.395647+0000","last_active":"2024-09-21T11:50:40.488134+0000","last_peered":"2024-09-21T11:50:40.488134+0000","last_clean":"2024-09-21T11:50:40.488134+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:50:40.488134+0000","last_undegraded":"2024-09-21T11:50:40.488134+0000","last_fullsized":"2024-09-21T11:50:40.488134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.395586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.395586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:44.653224+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"278'7755","reported_seq":7828,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.084599+0000","last_change":"2024-09-21T11:49:08.518112+0000","last_active":"2024-09-21T11:50:41.084599+0000","last_peered":"2024-09-21T11:50:41.084599+0000","last_clean":"2024-09-21T11:50:41.084599+0000","last_became_active":"2024-09-21T11:49:08.517344+0000","last_became_peered":"2024-09-21T11:49:08.517344+0000","last_unstale":"2024-09-21T11:50:41.084599+0000","last_undegraded":"2024-09-21T11:50:41.084599+0000","last_fullsized":"2024-09-21T11:50:41.084599+0000","mapping_epoch":247,"log_start":"255'2400","ondisk_log_start":"255'2400","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5355,"log_dups_size":0,"ondisk_log_size":5355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:20.684739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":508231680,"num_objects":7755,"num_object_clones":0,"num_object_copies":23265,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7755,"num_write_kb":496320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":190,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508572+0000","last_change":"2024-09-21T11:47:03.398638+0000","last_active":"2024-09-21T11:50:40.508572+0000","last_peered":"2024-09-21T11:50:40.508572+0000","last_clean":"2024-09-21T11:50:40.508572+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:50:40.508572+0000","last_undegraded":"2024-09-21T11:50:40.508572+0000","last_fullsized":"2024-09-21T11:50:40.508572+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:46:12.227121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:12.227121+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:06.869295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":502,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493231+0000","last_change":"2024-09-21T11:50:34.667913+0000","last_active":"2024-09-21T11:50:40.493231+0000","last_peered":"2024-09-21T11:50:40.493231+0000","last_clean":"2024-09-21T11:50:40.493231+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:50:40.493231+0000","last_undegraded":"2024-09-21T11:50:40.493231+0000","last_fullsized":"2024-09-21T11:50:40.493231+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:34.667863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.667863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:37.575353+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":602,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.494246+0000","last_change":"2024-09-21T11:50:29.676678+0000","last_active":"2024-09-21T11:50:40.494246+0000","last_peered":"2024-09-21T11:50:40.494246+0000","last_clean":"2024-09-21T11:50:40.494246+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:50:40.494246+0000","last_undegraded":"2024-09-21T11:50:40.494246+0000","last_fullsized":"2024-09-21T11:50:40.494246+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:50:29.676621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:29.676621+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:35.343100+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":466,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493595+0000","last_change":"2024-09-21T11:49:07.464269+0000","last_active":"2024-09-21T11:50:40.493595+0000","last_peered":"2024-09-21T11:50:40.493595+0000","last_clean":"2024-09-21T11:50:40.493595+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:50:40.493595+0000","last_undegraded":"2024-09-21T11:50:40.493595+0000","last_fullsized":"2024-09-21T11:50:40.493595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.464204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.464204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:13.689854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"278'7783","reported_seq":7854,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.014310+0000","last_change":"2024-09-21T11:49:08.609660+0000","last_active":"2024-09-21T11:50:41.014310+0000","last_peered":"2024-09-21T11:50:41.014310+0000","last_clean":"2024-09-21T11:50:41.014310+0000","last_became_active":"2024-09-21T11:49:08.609226+0000","last_became_peered":"2024-09-21T11:49:08.609226+0000","last_unstale":"2024-09-21T11:50:41.014310+0000","last_undegraded":"2024-09-21T11:50:41.014310+0000","last_fullsized":"2024-09-21T11:50:41.014310+0000","mapping_epoch":247,"log_start":"256'3000","ondisk_log_start":"256'3000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4783,"log_dups_size":0,"ondisk_log_size":4783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":510066688,"num_objects":7783,"num_object_clones":0,"num_object_copies":23349,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7783,"num_write_kb":498112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":525,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.494627+0000","last_change":"2024-09-21T11:50:31.701109+0000","last_active":"2024-09-21T11:50:40.494627+0000","last_peered":"2024-09-21T11:50:40.494627+0000","last_clean":"2024-09-21T11:50:40.494627+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:50:40.494627+0000","last_undegraded":"2024-09-21T11:50:40.494627+0000","last_fullsized":"2024-09-21T11:50:40.494627+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:50:31.701038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.701038+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:43.743270+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":171,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508264+0000","last_change":"2024-09-21T11:49:59.954903+0000","last_active":"2024-09-21T11:50:40.508264+0000","last_peered":"2024-09-21T11:50:40.508264+0000","last_clean":"2024-09-21T11:50:40.508264+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:50:40.508264+0000","last_undegraded":"2024-09-21T11:50:40.508264+0000","last_fullsized":"2024-09-21T11:50:40.508264+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:59.954835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:59.954835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:15.305841+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":462,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497166+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:50:40.497166+0000","last_peered":"2024-09-21T11:50:40.497166+0000","last_clean":"2024-09-21T11:50:40.497166+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:50:40.497166+0000","last_undegraded":"2024-09-21T11:50:40.497166+0000","last_fullsized":"2024-09-21T11:50:40.497166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":171,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506974+0000","last_change":"2024-09-21T11:50:36.027903+0000","last_active":"2024-09-21T11:50:40.506974+0000","last_peered":"2024-09-21T11:50:40.506974+0000","last_clean":"2024-09-21T11:50:40.506974+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:50:40.506974+0000","last_undegraded":"2024-09-21T11:50:40.506974+0000","last_fullsized":"2024-09-21T11:50:40.506974+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.027823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.027823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:48.339916+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"278'7747","reported_seq":7894,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.857189+0000","last_change":"2024-09-21T11:50:26.274463+0000","last_active":"2024-09-21T11:50:41.857189+0000","last_peered":"2024-09-21T11:50:41.857189+0000","last_clean":"2024-09-21T11:50:41.857189+0000","last_became_active":"2024-09-21T11:49:08.516355+0000","last_became_peered":"2024-09-21T11:49:08.516355+0000","last_unstale":"2024-09-21T11:50:41.857189+0000","last_undegraded":"2024-09-21T11:50:41.857189+0000","last_fullsized":"2024-09-21T11:50:41.857189+0000","mapping_epoch":247,"log_start":"253'1700","ondisk_log_start":"253'1700","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6771","last_scrub_stamp":"2024-09-21T11:50:26.274165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.274165+0000","objects_scrubbed":6698,"log_size":6047,"log_dups_size":0,"ondisk_log_size":6047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:39.313808+0000","scrub_duration":2483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":507707392,"num_objects":7747,"num_object_clones":0,"num_object_copies":23241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7747,"num_write_kb":495808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":247,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508195+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:50:40.508195+0000","last_peered":"2024-09-21T11:50:40.508195+0000","last_clean":"2024-09-21T11:50:40.508195+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:50:40.508195+0000","last_undegraded":"2024-09-21T11:50:40.508195+0000","last_fullsized":"2024-09-21T11:50:40.508195+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":469,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497444+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:50:40.497444+0000","last_peered":"2024-09-21T11:50:40.497444+0000","last_clean":"2024-09-21T11:50:40.497444+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:50:40.497444+0000","last_undegraded":"2024-09-21T11:50:40.497444+0000","last_fullsized":"2024-09-21T11:50:40.497444+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:41.126102+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":502,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.492848+0000","last_change":"2024-09-21T11:47:20.000195+0000","last_active":"2024-09-21T11:50:40.492848+0000","last_peered":"2024-09-21T11:50:40.492848+0000","last_clean":"2024-09-21T11:50:40.492848+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:50:40.492848+0000","last_undegraded":"2024-09-21T11:50:40.492848+0000","last_fullsized":"2024-09-21T11:50:40.492848+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:47:20.000051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.000051+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:11.774530+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":484,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.495065+0000","last_change":"2024-09-21T11:50:37.858401+0000","last_active":"2024-09-21T11:50:40.495065+0000","last_peered":"2024-09-21T11:50:40.495065+0000","last_clean":"2024-09-21T11:50:40.495065+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:50:40.495065+0000","last_undegraded":"2024-09-21T11:50:40.495065+0000","last_fullsized":"2024-09-21T11:50:40.495065+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:49.922881+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":450,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506605+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:50:40.506605+0000","last_peered":"2024-09-21T11:50:40.506605+0000","last_clean":"2024-09-21T11:50:40.506605+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:50:40.506605+0000","last_undegraded":"2024-09-21T11:50:40.506605+0000","last_fullsized":"2024-09-21T11:50:40.506605+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"278'7806","reported_seq":7879,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.084091+0000","last_change":"2024-09-21T11:49:08.518054+0000","last_active":"2024-09-21T11:50:41.084091+0000","last_peered":"2024-09-21T11:50:41.084091+0000","last_clean":"2024-09-21T11:50:41.084091+0000","last_became_active":"2024-09-21T11:49:08.517758+0000","last_became_peered":"2024-09-21T11:49:08.517758+0000","last_unstale":"2024-09-21T11:50:41.084091+0000","last_undegraded":"2024-09-21T11:50:41.084091+0000","last_fullsized":"2024-09-21T11:50:41.084091+0000","mapping_epoch":247,"log_start":"255'2500","ondisk_log_start":"255'2500","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.722671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":511574016,"num_objects":7806,"num_object_clones":0,"num_object_copies":23418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7806,"num_write_kb":499584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":114,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.500157+0000","last_change":"2024-09-21T11:48:33.075081+0000","last_active":"2024-09-21T11:50:40.500157+0000","last_peered":"2024-09-21T11:50:40.500157+0000","last_clean":"2024-09-21T11:50:40.500157+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:50:40.500157+0000","last_undegraded":"2024-09-21T11:50:40.500157+0000","last_fullsized":"2024-09-21T11:50:40.500157+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":175,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506106+0000","last_change":"2024-09-21T11:50:34.087998+0000","last_active":"2024-09-21T11:50:40.506106+0000","last_peered":"2024-09-21T11:50:40.506106+0000","last_clean":"2024-09-21T11:50:40.506106+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:50:40.506106+0000","last_undegraded":"2024-09-21T11:50:40.506106+0000","last_fullsized":"2024-09-21T11:50:40.506106+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:34.087799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.087799+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:43.032931+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":496,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.499915+0000","last_change":"2024-09-21T11:48:48.994249+0000","last_active":"2024-09-21T11:50:40.499915+0000","last_peered":"2024-09-21T11:50:40.499915+0000","last_clean":"2024-09-21T11:50:40.499915+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:50:40.499915+0000","last_undegraded":"2024-09-21T11:50:40.499915+0000","last_fullsized":"2024-09-21T11:50:40.499915+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:48:48.994172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:48:48.994172+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:15.864789+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":478,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.502048+0000","last_change":"2024-09-21T11:49:47.008541+0000","last_active":"2024-09-21T11:50:40.502048+0000","last_peered":"2024-09-21T11:50:40.502048+0000","last_clean":"2024-09-21T11:50:40.502048+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:50:40.502048+0000","last_undegraded":"2024-09-21T11:50:40.502048+0000","last_fullsized":"2024-09-21T11:50:40.502048+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.008369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.008369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:53.561004+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":474,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.488495+0000","last_change":"2024-09-21T11:49:54.388131+0000","last_active":"2024-09-21T11:50:40.488495+0000","last_peered":"2024-09-21T11:50:40.488495+0000","last_clean":"2024-09-21T11:50:40.488495+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:50:40.488495+0000","last_undegraded":"2024-09-21T11:50:40.488495+0000","last_fullsized":"2024-09-21T11:50:40.488495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.388068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.388068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:57.353255+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"278'7936","reported_seq":8009,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.838541+0000","last_change":"2024-09-21T11:49:08.515501+0000","last_active":"2024-09-21T11:50:41.838541+0000","last_peered":"2024-09-21T11:50:41.838541+0000","last_clean":"2024-09-21T11:50:41.838541+0000","last_became_active":"2024-09-21T11:49:08.515204+0000","last_became_peered":"2024-09-21T11:49:08.515204+0000","last_unstale":"2024-09-21T11:50:41.838541+0000","last_undegraded":"2024-09-21T11:50:41.838541+0000","last_fullsized":"2024-09-21T11:50:41.838541+0000","mapping_epoch":247,"log_start":"253'1900","ondisk_log_start":"253'1900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":6036,"log_dups_size":0,"ondisk_log_size":6036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.697074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520093696,"num_objects":7936,"num_object_clones":0,"num_object_copies":23808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7936,"num_write_kb":507904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":543,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493564+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:50:40.493564+0000","last_peered":"2024-09-21T11:50:40.493564+0000","last_clean":"2024-09-21T11:50:40.493564+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:50:40.493564+0000","last_undegraded":"2024-09-21T11:50:40.493564+0000","last_fullsized":"2024-09-21T11:50:40.493564+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":486,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.490826+0000","last_change":"2024-09-21T11:49:16.415641+0000","last_active":"2024-09-21T11:50:40.490826+0000","last_peered":"2024-09-21T11:50:40.490826+0000","last_clean":"2024-09-21T11:50:40.490826+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:50:40.490826+0000","last_undegraded":"2024-09-21T11:50:40.490826+0000","last_fullsized":"2024-09-21T11:50:40.490826+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:16.415597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:16.415597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:45.957747+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1161,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.487767+0000","last_change":"2024-09-21T11:49:51.434402+0000","last_active":"2024-09-21T11:50:40.487767+0000","last_peered":"2024-09-21T11:50:40.487767+0000","last_clean":"2024-09-21T11:50:40.487767+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:50:40.487767+0000","last_undegraded":"2024-09-21T11:50:40.487767+0000","last_fullsized":"2024-09-21T11:50:40.487767+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:51.434333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:49:51.434333+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:02.759046+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":632,"num_read_kb":627,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":483,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493283+0000","last_change":"2024-09-21T11:48:42.042870+0000","last_active":"2024-09-21T11:50:40.493283+0000","last_peered":"2024-09-21T11:50:40.493283+0000","last_clean":"2024-09-21T11:50:40.493283+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:50:40.493283+0000","last_undegraded":"2024-09-21T11:50:40.493283+0000","last_fullsized":"2024-09-21T11:50:40.493283+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:42.042814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:42.042814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.767298+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":171,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507824+0000","last_change":"2024-09-21T11:49:54.996327+0000","last_active":"2024-09-21T11:50:40.507824+0000","last_peered":"2024-09-21T11:50:40.507824+0000","last_clean":"2024-09-21T11:50:40.507824+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:50:40.507824+0000","last_undegraded":"2024-09-21T11:50:40.507824+0000","last_fullsized":"2024-09-21T11:50:40.507824+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.996240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.996240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.233955+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"278'7821","reported_seq":7971,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:42.046458+0000","last_change":"2024-09-21T11:50:26.726878+0000","last_active":"2024-09-21T11:50:42.046458+0000","last_peered":"2024-09-21T11:50:42.046458+0000","last_clean":"2024-09-21T11:50:42.046458+0000","last_became_active":"2024-09-21T11:49:08.516627+0000","last_became_peered":"2024-09-21T11:49:08.516627+0000","last_unstale":"2024-09-21T11:50:42.046458+0000","last_undegraded":"2024-09-21T11:50:42.046458+0000","last_fullsized":"2024-09-21T11:50:42.046458+0000","mapping_epoch":247,"log_start":"255'2900","ondisk_log_start":"255'2900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6809","last_scrub_stamp":"2024-09-21T11:50:26.726538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.726538+0000","objects_scrubbed":6753,"log_size":4921,"log_dups_size":0,"ondisk_log_size":4921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:32.214810+0000","scrub_duration":2384,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":512557056,"num_objects":7821,"num_object_clones":0,"num_object_copies":23463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7821,"num_write_kb":500544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":651,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497754+0000","last_change":"2024-09-21T11:50:29.389640+0000","last_active":"2024-09-21T11:50:40.497754+0000","last_peered":"2024-09-21T11:50:40.497754+0000","last_clean":"2024-09-21T11:50:40.497754+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:50:40.497754+0000","last_undegraded":"2024-09-21T11:50:40.497754+0000","last_fullsized":"2024-09-21T11:50:40.497754+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:29.389555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:29.389555+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:31.107690+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":479,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507067+0000","last_change":"2024-09-21T11:46:57.833589+0000","last_active":"2024-09-21T11:50:40.507067+0000","last_peered":"2024-09-21T11:50:40.507067+0000","last_clean":"2024-09-21T11:50:40.507067+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:50:40.507067+0000","last_undegraded":"2024-09-21T11:50:40.507067+0000","last_fullsized":"2024-09-21T11:50:40.507067+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:57.833329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:57.833329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":541,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.487842+0000","last_change":"2024-09-21T11:50:39.362320+0000","last_active":"2024-09-21T11:50:40.487842+0000","last_peered":"2024-09-21T11:50:40.487842+0000","last_clean":"2024-09-21T11:50:40.487842+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:50:40.487842+0000","last_undegraded":"2024-09-21T11:50:40.487842+0000","last_fullsized":"2024-09-21T11:50:40.487842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:50:39.361647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.361647+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.251982+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":478,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.490187+0000","last_change":"2024-09-21T11:49:06.639805+0000","last_active":"2024-09-21T11:50:40.490187+0000","last_peered":"2024-09-21T11:50:40.490187+0000","last_clean":"2024-09-21T11:50:40.490187+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:50:40.490187+0000","last_undegraded":"2024-09-21T11:50:40.490187+0000","last_fullsized":"2024-09-21T11:50:40.490187+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:06.639732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:06.639732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:16.438605+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":465,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493117+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:50:40.493117+0000","last_peered":"2024-09-21T11:50:40.493117+0000","last_clean":"2024-09-21T11:50:40.493117+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:50:40.493117+0000","last_undegraded":"2024-09-21T11:50:40.493117+0000","last_fullsized":"2024-09-21T11:50:40.493117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"278'7638","reported_seq":7793,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.016185+0000","last_change":"2024-09-21T11:50:37.410553+0000","last_active":"2024-09-21T11:50:41.016185+0000","last_peered":"2024-09-21T11:50:41.016185+0000","last_clean":"2024-09-21T11:50:41.016185+0000","last_became_active":"2024-09-21T11:49:08.519011+0000","last_became_peered":"2024-09-21T11:49:08.519011+0000","last_unstale":"2024-09-21T11:50:41.016185+0000","last_undegraded":"2024-09-21T11:50:41.016185+0000","last_fullsized":"2024-09-21T11:50:41.016185+0000","mapping_epoch":247,"log_start":"255'2800","ondisk_log_start":"255'2800","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7398","last_scrub_stamp":"2024-09-21T11:50:37.410305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.410305+0000","objects_scrubbed":7345,"log_size":4838,"log_dups_size":0,"ondisk_log_size":4838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:02.268662+0000","scrub_duration":2058,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":500563968,"num_objects":7638,"num_object_clones":0,"num_object_copies":22914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7638,"num_write_kb":488832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":543,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.498331+0000","last_change":"2024-09-21T11:50:37.352894+0000","last_active":"2024-09-21T11:50:40.498331+0000","last_peered":"2024-09-21T11:50:40.498331+0000","last_clean":"2024-09-21T11:50:40.498331+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:50:40.498331+0000","last_undegraded":"2024-09-21T11:50:40.498331+0000","last_fullsized":"2024-09-21T11:50:40.498331+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:37.352834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.352834+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:59.843673+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":502,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497460+0000","last_change":"2024-09-21T11:49:49.165213+0000","last_active":"2024-09-21T11:50:40.497460+0000","last_peered":"2024-09-21T11:50:40.497460+0000","last_clean":"2024-09-21T11:50:40.497460+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:50:40.497460+0000","last_undegraded":"2024-09-21T11:50:40.497460+0000","last_fullsized":"2024-09-21T11:50:40.497460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.165146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.165146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:04.935623+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":469,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493038+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:50:40.493038+0000","last_peered":"2024-09-21T11:50:40.493038+0000","last_clean":"2024-09-21T11:50:40.493038+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:50:40.493038+0000","last_undegraded":"2024-09-21T11:50:40.493038+0000","last_fullsized":"2024-09-21T11:50:40.493038+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":465,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507723+0000","last_change":"2024-09-21T11:48:23.285482+0000","last_active":"2024-09-21T11:50:40.507723+0000","last_peered":"2024-09-21T11:50:40.507723+0000","last_clean":"2024-09-21T11:50:40.507723+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:50:40.507723+0000","last_undegraded":"2024-09-21T11:50:40.507723+0000","last_fullsized":"2024-09-21T11:50:40.507723+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.285283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.285283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.489355+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"278'7716","reported_seq":7872,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.152646+0000","last_change":"2024-09-21T11:50:39.224005+0000","last_active":"2024-09-21T11:50:41.152646+0000","last_peered":"2024-09-21T11:50:41.152646+0000","last_clean":"2024-09-21T11:50:41.152646+0000","last_became_active":"2024-09-21T11:49:08.748155+0000","last_became_peered":"2024-09-21T11:49:08.748155+0000","last_unstale":"2024-09-21T11:50:41.152646+0000","last_undegraded":"2024-09-21T11:50:41.152646+0000","last_fullsized":"2024-09-21T11:50:41.152646+0000","mapping_epoch":247,"log_start":"256'3000","ondisk_log_start":"256'3000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7577","last_scrub_stamp":"2024-09-21T11:50:39.223848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.223848+0000","objects_scrubbed":7518,"log_size":4716,"log_dups_size":0,"ondisk_log_size":4716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:42.031062+0000","scrub_duration":2220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":505675776,"num_objects":7716,"num_object_clones":0,"num_object_copies":23148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7716,"num_write_kb":493824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":614,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497863+0000","last_change":"2024-09-21T11:50:36.386009+0000","last_active":"2024-09-21T11:50:40.497863+0000","last_peered":"2024-09-21T11:50:40.497863+0000","last_clean":"2024-09-21T11:50:40.497863+0000","last_became_active":"2024-09-21T11:48:33.022321+0000","last_became_peered":"2024-09-21T11:48:33.022321+0000","last_unstale":"2024-09-21T11:50:40.497863+0000","last_undegraded":"2024-09-21T11:50:40.497863+0000","last_fullsized":"2024-09-21T11:50:40.497863+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:50:36.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.385902+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:37.670911+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":499,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497481+0000","last_change":"2024-09-21T11:47:03.365071+0000","last_active":"2024-09-21T11:50:40.497481+0000","last_peered":"2024-09-21T11:50:40.497481+0000","last_clean":"2024-09-21T11:50:40.497481+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:50:40.497481+0000","last_undegraded":"2024-09-21T11:50:40.497481+0000","last_fullsized":"2024-09-21T11:50:40.497481+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:17.692429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:46:17.692429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.349348+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":163,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507670+0000","last_change":"2024-09-21T11:47:03.375108+0000","last_active":"2024-09-21T11:50:40.507670+0000","last_peered":"2024-09-21T11:50:40.507670+0000","last_clean":"2024-09-21T11:50:40.507670+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:50:40.507670+0000","last_undegraded":"2024-09-21T11:50:40.507670+0000","last_fullsized":"2024-09-21T11:50:40.507670+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:50.008635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:50.008635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.928506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":474,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.500346+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:50:40.500346+0000","last_peered":"2024-09-21T11:50:40.500346+0000","last_clean":"2024-09-21T11:50:40.500346+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:50:40.500346+0000","last_undegraded":"2024-09-21T11:50:40.500346+0000","last_fullsized":"2024-09-21T11:50:40.500346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:42.937389+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"278'7874","reported_seq":7947,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:42.040963+0000","last_change":"2024-09-21T11:49:08.517565+0000","last_active":"2024-09-21T11:50:42.040963+0000","last_peered":"2024-09-21T11:50:42.040963+0000","last_clean":"2024-09-21T11:50:42.040963+0000","last_became_active":"2024-09-21T11:49:08.517011+0000","last_became_peered":"2024-09-21T11:49:08.517011+0000","last_unstale":"2024-09-21T11:50:42.040963+0000","last_undegraded":"2024-09-21T11:50:42.040963+0000","last_fullsized":"2024-09-21T11:50:42.040963+0000","mapping_epoch":247,"log_start":"255'3000","ondisk_log_start":"255'3000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4874,"log_dups_size":0,"ondisk_log_size":4874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516030464,"num_objects":7874,"num_object_clones":0,"num_object_copies":23622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7874,"num_write_kb":503936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":520,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.498480+0000","last_change":"2024-09-21T11:50:06.225159+0000","last_active":"2024-09-21T11:50:40.498480+0000","last_peered":"2024-09-21T11:50:40.498480+0000","last_clean":"2024-09-21T11:50:40.498480+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:50:40.498480+0000","last_undegraded":"2024-09-21T11:50:40.498480+0000","last_fullsized":"2024-09-21T11:50:40.498480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:32.807084+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":470,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506480+0000","last_change":"2024-09-21T11:50:35.410162+0000","last_active":"2024-09-21T11:50:40.506480+0000","last_peered":"2024-09-21T11:50:40.506480+0000","last_clean":"2024-09-21T11:50:40.506480+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:50:40.506480+0000","last_undegraded":"2024-09-21T11:50:40.506480+0000","last_fullsized":"2024-09-21T11:50:40.506480+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:35.410095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.410095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:41.088475+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":623,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.501024+0000","last_change":"2024-09-21T11:50:02.975387+0000","last_active":"2024-09-21T11:50:40.501024+0000","last_peered":"2024-09-21T11:50:40.501024+0000","last_clean":"2024-09-21T11:50:40.501024+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:50:40.501024+0000","last_undegraded":"2024-09-21T11:50:40.501024+0000","last_fullsized":"2024-09-21T11:50:40.501024+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:50:02.975313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.975313+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:25.444093+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":472,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.498043+0000","last_change":"2024-09-21T11:48:07.992586+0000","last_active":"2024-09-21T11:50:40.498043+0000","last_peered":"2024-09-21T11:50:40.498043+0000","last_clean":"2024-09-21T11:50:40.498043+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:50:40.498043+0000","last_undegraded":"2024-09-21T11:50:40.498043+0000","last_fullsized":"2024-09-21T11:50:40.498043+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.992517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.992517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:42.078128+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"278'7731","reported_seq":7879,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.054815+0000","last_change":"2024-09-21T11:50:24.709591+0000","last_active":"2024-09-21T11:50:41.054815+0000","last_peered":"2024-09-21T11:50:41.054815+0000","last_clean":"2024-09-21T11:50:41.054815+0000","last_became_active":"2024-09-21T11:49:08.748171+0000","last_became_peered":"2024-09-21T11:49:08.748171+0000","last_unstale":"2024-09-21T11:50:41.054815+0000","last_undegraded":"2024-09-21T11:50:41.054815+0000","last_fullsized":"2024-09-21T11:50:41.054815+0000","mapping_epoch":247,"log_start":"255'2400","ondisk_log_start":"255'2400","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6768","last_scrub_stamp":"2024-09-21T11:50:24.709386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.709386+0000","objects_scrubbed":6700,"log_size":5331,"log_dups_size":0,"ondisk_log_size":5331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:51.612675+0000","scrub_duration":2008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":506658816,"num_objects":7731,"num_object_clones":0,"num_object_copies":23193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7731,"num_write_kb":494784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":524,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.494303+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:50:40.494303+0000","last_peered":"2024-09-21T11:50:40.494303+0000","last_clean":"2024-09-21T11:50:40.494303+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:50:40.494303+0000","last_undegraded":"2024-09-21T11:50:40.494303+0000","last_fullsized":"2024-09-21T11:50:40.494303+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:41.803003+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":485,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.492585+0000","last_change":"2024-09-21T11:49:23.000190+0000","last_active":"2024-09-21T11:50:40.492585+0000","last_peered":"2024-09-21T11:50:40.492585+0000","last_clean":"2024-09-21T11:50:40.492585+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:50:40.492585+0000","last_undegraded":"2024-09-21T11:50:40.492585+0000","last_fullsized":"2024-09-21T11:50:40.492585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.000138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.000138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:42.451760+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":470,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.500550+0000","last_change":"2024-09-21T11:48:49.993307+0000","last_active":"2024-09-21T11:50:40.500550+0000","last_peered":"2024-09-21T11:50:40.500550+0000","last_clean":"2024-09-21T11:50:40.500550+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:50:40.500550+0000","last_undegraded":"2024-09-21T11:50:40.500550+0000","last_fullsized":"2024-09-21T11:50:40.500550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.993230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.993230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.951895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":449,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.490738+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:50:40.490738+0000","last_peered":"2024-09-21T11:50:40.490738+0000","last_clean":"2024-09-21T11:50:40.490738+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:50:40.490738+0000","last_undegraded":"2024-09-21T11:50:40.490738+0000","last_fullsized":"2024-09-21T11:50:40.490738+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"278'7898","reported_seq":8053,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.095419+0000","last_change":"2024-09-21T11:50:32.751942+0000","last_active":"2024-09-21T11:50:41.095419+0000","last_peered":"2024-09-21T11:50:41.095419+0000","last_clean":"2024-09-21T11:50:41.095419+0000","last_became_active":"2024-09-21T11:49:08.516975+0000","last_became_peered":"2024-09-21T11:49:08.516975+0000","last_unstale":"2024-09-21T11:50:41.095419+0000","last_undegraded":"2024-09-21T11:50:41.095419+0000","last_fullsized":"2024-09-21T11:50:41.095419+0000","mapping_epoch":247,"log_start":"255'2600","ondisk_log_start":"255'2600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7356","last_scrub_stamp":"2024-09-21T11:50:32.751687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.751687+0000","objects_scrubbed":7220,"log_size":5298,"log_dups_size":0,"ondisk_log_size":5298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:42.366185+0000","scrub_duration":3036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":517603328,"num_objects":7898,"num_object_clones":0,"num_object_copies":23694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7898,"num_write_kb":505472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":598,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497300+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:50:40.497300+0000","last_peered":"2024-09-21T11:50:40.497300+0000","last_clean":"2024-09-21T11:50:40.497300+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:50:40.497300+0000","last_undegraded":"2024-09-21T11:50:40.497300+0000","last_fullsized":"2024-09-21T11:50:40.497300+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:45.721984+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":485,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.488875+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:50:40.488875+0000","last_peered":"2024-09-21T11:50:40.488875+0000","last_clean":"2024-09-21T11:50:40.488875+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:50:40.488875+0000","last_undegraded":"2024-09-21T11:50:40.488875+0000","last_fullsized":"2024-09-21T11:50:40.488875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:04.681745+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":486,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.500235+0000","last_change":"2024-09-21T11:50:31.946936+0000","last_active":"2024-09-21T11:50:40.500235+0000","last_peered":"2024-09-21T11:50:40.500235+0000","last_clean":"2024-09-21T11:50:40.500235+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:50:40.500235+0000","last_undegraded":"2024-09-21T11:50:40.500235+0000","last_fullsized":"2024-09-21T11:50:40.500235+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.946859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.946859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:43.718312+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":473,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.487738+0000","last_change":"2024-09-21T11:50:34.385662+0000","last_active":"2024-09-21T11:50:40.487738+0000","last_peered":"2024-09-21T11:50:40.487738+0000","last_clean":"2024-09-21T11:50:40.487738+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:50:40.487738+0000","last_undegraded":"2024-09-21T11:50:40.487738+0000","last_fullsized":"2024-09-21T11:50:40.487738+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:34.385563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.385563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:43.597782+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"278'7677","reported_seq":7827,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.136599+0000","last_change":"2024-09-21T11:50:30.668171+0000","last_active":"2024-09-21T11:50:41.136599+0000","last_peered":"2024-09-21T11:50:41.136599+0000","last_clean":"2024-09-21T11:50:41.136599+0000","last_became_active":"2024-09-21T11:49:08.609829+0000","last_became_peered":"2024-09-21T11:49:08.609829+0000","last_unstale":"2024-09-21T11:50:41.136599+0000","last_undegraded":"2024-09-21T11:50:41.136599+0000","last_fullsized":"2024-09-21T11:50:41.136599+0000","mapping_epoch":247,"log_start":"255'2900","ondisk_log_start":"255'2900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7027","last_scrub_stamp":"2024-09-21T11:50:30.668043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:30.668043+0000","objects_scrubbed":6939,"log_size":4777,"log_dups_size":0,"ondisk_log_size":4777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:37.139540+0000","scrub_duration":2651,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":503119872,"num_objects":7677,"num_object_clones":0,"num_object_copies":23031,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7677,"num_write_kb":491328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":568,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497195+0000","last_change":"2024-09-21T11:50:34.362829+0000","last_active":"2024-09-21T11:50:40.497195+0000","last_peered":"2024-09-21T11:50:40.497195+0000","last_clean":"2024-09-21T11:50:40.497195+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:50:40.497195+0000","last_undegraded":"2024-09-21T11:50:40.497195+0000","last_fullsized":"2024-09-21T11:50:40.497195+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:39.880745+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":515,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.489008+0000","last_change":"2024-09-21T11:49:08.611218+0000","last_active":"2024-09-21T11:50:40.489008+0000","last_peered":"2024-09-21T11:50:40.489008+0000","last_clean":"2024-09-21T11:50:40.489008+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:50:40.489008+0000","last_undegraded":"2024-09-21T11:50:40.489008+0000","last_fullsized":"2024-09-21T11:50:40.489008+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.611159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.611159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":500,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.500387+0000","last_change":"2024-09-21T11:49:44.985594+0000","last_active":"2024-09-21T11:50:40.500387+0000","last_peered":"2024-09-21T11:50:40.500387+0000","last_clean":"2024-09-21T11:50:40.500387+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:50:40.500387+0000","last_undegraded":"2024-09-21T11:50:40.500387+0000","last_fullsized":"2024-09-21T11:50:40.500387+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:44.985234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:44.985234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:50.892123+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":480,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506073+0000","last_change":"2024-09-21T11:49:21.071757+0000","last_active":"2024-09-21T11:50:40.506073+0000","last_peered":"2024-09-21T11:50:40.506073+0000","last_clean":"2024-09-21T11:50:40.506073+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:50:40.506073+0000","last_undegraded":"2024-09-21T11:50:40.506073+0000","last_fullsized":"2024-09-21T11:50:40.506073+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:21.071680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:21.071680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"278'7967","reported_seq":8040,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:41.849969+0000","last_change":"2024-09-21T11:49:08.517117+0000","last_active":"2024-09-21T11:50:41.849969+0000","last_peered":"2024-09-21T11:50:41.849969+0000","last_clean":"2024-09-21T11:50:41.849969+0000","last_became_active":"2024-09-21T11:49:08.516494+0000","last_became_peered":"2024-09-21T11:49:08.516494+0000","last_unstale":"2024-09-21T11:50:41.849969+0000","last_undegraded":"2024-09-21T11:50:41.849969+0000","last_fullsized":"2024-09-21T11:50:41.849969+0000","mapping_epoch":247,"log_start":"253'1900","ondisk_log_start":"253'1900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522125312,"num_objects":7967,"num_object_clones":0,"num_object_copies":23901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7967,"num_write_kb":509888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":508,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.492198+0000","last_change":"2024-09-21T11:49:45.783231+0000","last_active":"2024-09-21T11:50:40.492198+0000","last_peered":"2024-09-21T11:50:40.492198+0000","last_clean":"2024-09-21T11:50:40.492198+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:50:40.492198+0000","last_undegraded":"2024-09-21T11:50:40.492198+0000","last_fullsized":"2024-09-21T11:50:40.492198+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:49:45.783036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:45.783036+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.400285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":505,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.489137+0000","last_change":"2024-09-21T11:50:06.398255+0000","last_active":"2024-09-21T11:50:40.489137+0000","last_peered":"2024-09-21T11:50:40.489137+0000","last_clean":"2024-09-21T11:50:40.489137+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:50:40.489137+0000","last_undegraded":"2024-09-21T11:50:40.489137+0000","last_fullsized":"2024-09-21T11:50:40.489137+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:50:06.398195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.398195+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:06.457914+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":606,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.488101+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:50:40.488101+0000","last_peered":"2024-09-21T11:50:40.488101+0000","last_clean":"2024-09-21T11:50:40.488101+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:50:40.488101+0000","last_undegraded":"2024-09-21T11:50:40.488101+0000","last_fullsized":"2024-09-21T11:50:40.488101+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":465,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.492575+0000","last_change":"2024-09-21T11:49:56.669610+0000","last_active":"2024-09-21T11:50:40.492575+0000","last_peered":"2024-09-21T11:50:40.492575+0000","last_clean":"2024-09-21T11:50:40.492575+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:50:40.492575+0000","last_undegraded":"2024-09-21T11:50:40.492575+0000","last_fullsized":"2024-09-21T11:50:40.492575+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:56.669547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:56.669547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:24.559358+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":493,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.498717+0000","last_change":"2024-09-21T11:50:31.417131+0000","last_active":"2024-09-21T11:50:40.498717+0000","last_peered":"2024-09-21T11:50:40.498717+0000","last_clean":"2024-09-21T11:50:40.498717+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:50:40.498717+0000","last_undegraded":"2024-09-21T11:50:40.498717+0000","last_fullsized":"2024-09-21T11:50:40.498717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.416857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.416857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:35.529094+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":514,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508022+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:50:40.508022+0000","last_peered":"2024-09-21T11:50:40.508022+0000","last_clean":"2024-09-21T11:50:40.508022+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:50:40.508022+0000","last_undegraded":"2024-09-21T11:50:40.508022+0000","last_fullsized":"2024-09-21T11:50:40.508022+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":491,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.492957+0000","last_change":"2024-09-21T11:49:08.902428+0000","last_active":"2024-09-21T11:50:40.492957+0000","last_peered":"2024-09-21T11:50:40.492957+0000","last_clean":"2024-09-21T11:50:40.492957+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:50:40.492957+0000","last_undegraded":"2024-09-21T11:50:40.492957+0000","last_fullsized":"2024-09-21T11:50:40.492957+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.902382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.902382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":481,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.488140+0000","last_change":"2024-09-21T11:49:43.464860+0000","last_active":"2024-09-21T11:50:40.488140+0000","last_peered":"2024-09-21T11:50:40.488140+0000","last_clean":"2024-09-21T11:50:40.488140+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:50:40.488140+0000","last_undegraded":"2024-09-21T11:50:40.488140+0000","last_fullsized":"2024-09-21T11:50:40.488140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:43.464792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:43.464792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:48.801237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":507,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493850+0000","last_change":"2024-09-21T11:46:28.973700+0000","last_active":"2024-09-21T11:50:40.493850+0000","last_peered":"2024-09-21T11:50:40.493850+0000","last_clean":"2024-09-21T11:50:40.493850+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:50:40.493850+0000","last_undegraded":"2024-09-21T11:50:40.493850+0000","last_fullsized":"2024-09-21T11:50:40.493850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:28.973606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.973606+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.696833+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":477,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.488605+0000","last_change":"2024-09-21T11:50:02.438498+0000","last_active":"2024-09-21T11:50:40.488605+0000","last_peered":"2024-09-21T11:50:40.488605+0000","last_clean":"2024-09-21T11:50:40.488605+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:50:40.488605+0000","last_undegraded":"2024-09-21T11:50:40.488605+0000","last_fullsized":"2024-09-21T11:50:40.488605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.436591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.436591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:19.559044+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":492,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507273+0000","last_change":"2024-09-21T11:49:50.191966+0000","last_active":"2024-09-21T11:50:40.507273+0000","last_peered":"2024-09-21T11:50:40.507273+0000","last_clean":"2024-09-21T11:50:40.507273+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:50:40.507273+0000","last_undegraded":"2024-09-21T11:50:40.507273+0000","last_fullsized":"2024-09-21T11:50:40.507273+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:01.459336+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":482,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.500366+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:50:40.500366+0000","last_peered":"2024-09-21T11:50:40.500366+0000","last_clean":"2024-09-21T11:50:40.500366+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:50:40.500366+0000","last_undegraded":"2024-09-21T11:50:40.500366+0000","last_fullsized":"2024-09-21T11:50:40.500366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:54.617405+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":490,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493146+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:50:40.493146+0000","last_peered":"2024-09-21T11:50:40.493146+0000","last_clean":"2024-09-21T11:50:40.493146+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:50:40.493146+0000","last_undegraded":"2024-09-21T11:50:40.493146+0000","last_fullsized":"2024-09-21T11:50:40.493146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":485,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.494280+0000","last_change":"2024-09-21T11:49:22.528906+0000","last_active":"2024-09-21T11:50:40.494280+0000","last_peered":"2024-09-21T11:50:40.494280+0000","last_clean":"2024-09-21T11:50:40.494280+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:50:40.494280+0000","last_undegraded":"2024-09-21T11:50:40.494280+0000","last_fullsized":"2024-09-21T11:50:40.494280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:22.528831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:22.528831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":591,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508026+0000","last_change":"2024-09-21T11:47:02.849839+0000","last_active":"2024-09-21T11:50:40.508026+0000","last_peered":"2024-09-21T11:50:40.508026+0000","last_clean":"2024-09-21T11:50:40.508026+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:50:40.508026+0000","last_undegraded":"2024-09-21T11:50:40.508026+0000","last_fullsized":"2024-09-21T11:50:40.508026+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:02.849786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:02.849786+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.365178+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":474,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497657+0000","last_change":"2024-09-21T11:48:50.927661+0000","last_active":"2024-09-21T11:50:40.497657+0000","last_peered":"2024-09-21T11:50:40.497657+0000","last_clean":"2024-09-21T11:50:40.497657+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:50:40.497657+0000","last_undegraded":"2024-09-21T11:50:40.497657+0000","last_fullsized":"2024-09-21T11:50:40.497657+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:50.927613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.927613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:20.385159+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":534,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508208+0000","last_change":"2024-09-21T11:50:37.433622+0000","last_active":"2024-09-21T11:50:40.508208+0000","last_peered":"2024-09-21T11:50:40.508208+0000","last_clean":"2024-09-21T11:50:40.508208+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:50:40.508208+0000","last_undegraded":"2024-09-21T11:50:40.508208+0000","last_fullsized":"2024-09-21T11:50:40.508208+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:50:37.432212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.432212+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:42.400352+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":477,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.488749+0000","last_change":"2024-09-21T11:49:58.400078+0000","last_active":"2024-09-21T11:50:40.488749+0000","last_peered":"2024-09-21T11:50:40.488749+0000","last_clean":"2024-09-21T11:50:40.488749+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:50:40.488749+0000","last_undegraded":"2024-09-21T11:50:40.488749+0000","last_fullsized":"2024-09-21T11:50:40.488749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.400013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.400013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:58.776865+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":637,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493017+0000","last_change":"2024-09-21T11:50:30.703282+0000","last_active":"2024-09-21T11:50:40.493017+0000","last_peered":"2024-09-21T11:50:40.493017+0000","last_clean":"2024-09-21T11:50:40.493017+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:50:40.493017+0000","last_undegraded":"2024-09-21T11:50:40.493017+0000","last_fullsized":"2024-09-21T11:50:40.493017+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:50:30.703227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:30.703227+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:57.509405+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":481,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493216+0000","last_change":"2024-09-21T11:50:02.782565+0000","last_active":"2024-09-21T11:50:40.493216+0000","last_peered":"2024-09-21T11:50:40.493216+0000","last_clean":"2024-09-21T11:50:40.493216+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:50:40.493216+0000","last_undegraded":"2024-09-21T11:50:40.493216+0000","last_fullsized":"2024-09-21T11:50:40.493216+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.782475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.782475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:29.369093+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":578,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497220+0000","last_change":"2024-09-21T11:50:35.363920+0000","last_active":"2024-09-21T11:50:40.497220+0000","last_peered":"2024-09-21T11:50:40.497220+0000","last_clean":"2024-09-21T11:50:40.497220+0000","last_became_active":"2024-09-21T11:48:33.022647+0000","last_became_peered":"2024-09-21T11:48:33.022647+0000","last_unstale":"2024-09-21T11:50:40.497220+0000","last_undegraded":"2024-09-21T11:50:40.497220+0000","last_fullsized":"2024-09-21T11:50:40.497220+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.878372+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":470,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.498701+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:50:40.498701+0000","last_peered":"2024-09-21T11:50:40.498701+0000","last_clean":"2024-09-21T11:50:40.498701+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:50:40.498701+0000","last_undegraded":"2024-09-21T11:50:40.498701+0000","last_fullsized":"2024-09-21T11:50:40.498701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":171,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508487+0000","last_change":"2024-09-21T11:49:58.034124+0000","last_active":"2024-09-21T11:50:40.508487+0000","last_peered":"2024-09-21T11:50:40.508487+0000","last_clean":"2024-09-21T11:50:40.508487+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:50:40.508487+0000","last_undegraded":"2024-09-21T11:50:40.508487+0000","last_fullsized":"2024-09-21T11:50:40.508487+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.034057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.034057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:05.285593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":458,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.505747+0000","last_change":"2024-09-21T11:50:02.329020+0000","last_active":"2024-09-21T11:50:40.505747+0000","last_peered":"2024-09-21T11:50:40.505747+0000","last_clean":"2024-09-21T11:50:40.505747+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:50:40.505747+0000","last_undegraded":"2024-09-21T11:50:40.505747+0000","last_fullsized":"2024-09-21T11:50:40.505747+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.328926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.328926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.336948+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":182,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.511749+0000","last_change":"2024-09-21T11:47:03.404473+0000","last_active":"2024-09-21T11:50:40.511749+0000","last_peered":"2024-09-21T11:50:40.511749+0000","last_clean":"2024-09-21T11:50:40.511749+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:50:40.511749+0000","last_undegraded":"2024-09-21T11:50:40.511749+0000","last_fullsized":"2024-09-21T11:50:40.511749+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:26.649189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:26.649189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.922345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":485,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.488754+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:50:40.488754+0000","last_peered":"2024-09-21T11:50:40.488754+0000","last_clean":"2024-09-21T11:50:40.488754+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:50:40.488754+0000","last_undegraded":"2024-09-21T11:50:40.488754+0000","last_fullsized":"2024-09-21T11:50:40.488754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:50:47.576218+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":492,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506159+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:50:40.506159+0000","last_peered":"2024-09-21T11:50:40.506159+0000","last_clean":"2024-09-21T11:50:40.506159+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:50:40.506159+0000","last_undegraded":"2024-09-21T11:50:40.506159+0000","last_fullsized":"2024-09-21T11:50:40.506159+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.838895+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":496,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493241+0000","last_change":"2024-09-21T11:48:50.487842+0000","last_active":"2024-09-21T11:50:40.493241+0000","last_peered":"2024-09-21T11:50:40.493241+0000","last_clean":"2024-09-21T11:50:40.493241+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:50:40.493241+0000","last_undegraded":"2024-09-21T11:50:40.493241+0000","last_fullsized":"2024-09-21T11:50:40.493241+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:48:50.487786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.487786+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.708437+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":505,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.505989+0000","last_change":"2024-09-21T11:49:07.148305+0000","last_active":"2024-09-21T11:50:40.505989+0000","last_peered":"2024-09-21T11:50:40.505989+0000","last_clean":"2024-09-21T11:50:40.505989+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:50:40.505989+0000","last_undegraded":"2024-09-21T11:50:40.505989+0000","last_fullsized":"2024-09-21T11:50:40.505989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:49:07.148241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.148241+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:19.446103+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":491,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493011+0000","last_change":"2024-09-21T11:49:12.981959+0000","last_active":"2024-09-21T11:50:40.493011+0000","last_peered":"2024-09-21T11:50:40.493011+0000","last_clean":"2024-09-21T11:50:40.493011+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:50:40.493011+0000","last_undegraded":"2024-09-21T11:50:40.493011+0000","last_fullsized":"2024-09-21T11:50:40.493011+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:12.981878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:12.981878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":461,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.487546+0000","last_change":"2024-09-21T11:48:44.639778+0000","last_active":"2024-09-21T11:50:40.487546+0000","last_peered":"2024-09-21T11:50:40.487546+0000","last_clean":"2024-09-21T11:50:40.487546+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:50:40.487546+0000","last_undegraded":"2024-09-21T11:50:40.487546+0000","last_fullsized":"2024-09-21T11:50:40.487546+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:44.639638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:44.639638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:08.393642+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":466,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.506384+0000","last_change":"2024-09-21T11:49:08.171420+0000","last_active":"2024-09-21T11:50:40.506384+0000","last_peered":"2024-09-21T11:50:40.506384+0000","last_clean":"2024-09-21T11:50:40.506384+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:50:40.506384+0000","last_undegraded":"2024-09-21T11:50:40.506384+0000","last_fullsized":"2024-09-21T11:50:40.506384+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.171357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.171357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":525,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507067+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:50:40.507067+0000","last_peered":"2024-09-21T11:50:40.507067+0000","last_clean":"2024-09-21T11:50:40.507067+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:50:40.507067+0000","last_undegraded":"2024-09-21T11:50:40.507067+0000","last_fullsized":"2024-09-21T11:50:40.507067+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":477,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.490988+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:50:40.490988+0000","last_peered":"2024-09-21T11:50:40.490988+0000","last_clean":"2024-09-21T11:50:40.490988+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:50:40.490988+0000","last_undegraded":"2024-09-21T11:50:40.490988+0000","last_fullsized":"2024-09-21T11:50:40.490988+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:34.752247+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":477,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.497644+0000","last_change":"2024-09-21T11:50:38.405183+0000","last_active":"2024-09-21T11:50:40.497644+0000","last_peered":"2024-09-21T11:50:40.497644+0000","last_clean":"2024-09-21T11:50:40.497644+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:50:40.497644+0000","last_undegraded":"2024-09-21T11:50:40.497644+0000","last_fullsized":"2024-09-21T11:50:40.497644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:38.405118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.405118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:51.232325+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":465,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.491389+0000","last_change":"2024-09-21T11:50:05.419390+0000","last_active":"2024-09-21T11:50:40.491389+0000","last_peered":"2024-09-21T11:50:40.491389+0000","last_clean":"2024-09-21T11:50:40.491389+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:50:40.491389+0000","last_undegraded":"2024-09-21T11:50:40.491389+0000","last_fullsized":"2024-09-21T11:50:40.491389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.419329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.419329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:15.874052+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":179,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.507748+0000","last_change":"2024-09-21T11:50:25.045141+0000","last_active":"2024-09-21T11:50:40.507748+0000","last_peered":"2024-09-21T11:50:40.507748+0000","last_clean":"2024-09-21T11:50:40.507748+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:50:40.507748+0000","last_undegraded":"2024-09-21T11:50:40.507748+0000","last_fullsized":"2024-09-21T11:50:40.507748+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.045074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.045074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.164898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":492,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.508453+0000","last_change":"2024-09-21T11:49:12.177359+0000","last_active":"2024-09-21T11:50:40.508453+0000","last_peered":"2024-09-21T11:50:40.508453+0000","last_clean":"2024-09-21T11:50:40.508453+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:50:40.508453+0000","last_undegraded":"2024-09-21T11:50:40.508453+0000","last_fullsized":"2024-09-21T11:50:40.508453+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:12.177303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:12.177303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":559,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.498074+0000","last_change":"2024-09-21T11:48:45.147257+0000","last_active":"2024-09-21T11:50:40.498074+0000","last_peered":"2024-09-21T11:50:40.498074+0000","last_clean":"2024-09-21T11:50:40.498074+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:50:40.498074+0000","last_undegraded":"2024-09-21T11:50:40.498074+0000","last_fullsized":"2024-09-21T11:50:40.498074+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:48:45.147190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:45.147190+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.355453+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":486,"reported_epoch":278,"state":"active+clean","last_fresh":"2024-09-21T11:50:40.493393+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:50:40.493393+0000","last_peered":"2024-09-21T11:50:40.493393+0000","last_clean":"2024-09-21T11:50:40.493393+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:50:40.493393+0000","last_undegraded":"2024-09-21T11:50:40.493393+0000","last_fullsized":"2024-09-21T11:50:40.493393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.689301+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":8181055488,"num_objects":124833,"num_object_clones":0,"num_object_copies":374499,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124833,"num_write_kb":7989312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24508104704,"data_stored":24508104704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82933,"ondisk_log_size":82933,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1122,"num_read_kb":879,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":632,"num_read_kb":627,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234117,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4865996,"kb_used_data":4455804,"kb_used_omap":1,"kb_used_meta":410174,"kb_avail":88879156,"statfs":{"total":95995035648,"available":91012255744,"internally_reserved":0,"allocated":4562743296,"data_stored":4562363339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":420018634},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463050,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1805696,"kb_used_data":1489904,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":91939456,"statfs":{"total":95995035648,"available":94146002944,"internally_reserved":0,"allocated":1525661696,"data_stored":1525294371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":323353030},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724687,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4909792,"kb_used_data":4473228,"kb_used_omap":1,"kb_used_meta":436542,"kb_avail":88835360,"statfs":{"total":95995035648,"available":90967408640,"internally_reserved":0,"allocated":4580585472,"data_stored":4580195289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":447019462},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920918,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1795296,"kb_used_data":1493392,"kb_used_omap":1,"kb_used_meta":301886,"kb_avail":91949856,"statfs":{"total":95995035648,"available":94156652544,"internally_reserved":0,"allocated":1529233408,"data_stored":1528839887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309131717},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117147,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4500368,"kb_used_data":3991616,"kb_used_omap":1,"kb_used_meta":508734,"kb_avail":89244784,"statfs":{"total":95995035648,"available":91386658816,"internally_reserved":0,"allocated":4087414784,"data_stored":4087036472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":520944071},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411489,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1927640,"kb_used_data":1485508,"kb_used_omap":1,"kb_used_meta":442110,"kb_avail":91817512,"statfs":{"total":95995035648,"available":94021132288,"internally_reserved":0,"allocated":1521160192,"data_stored":1520778495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":452721093},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575016,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4907320,"kb_used_data":4456868,"kb_used_omap":1,"kb_used_meta":450430,"kb_avail":88837832,"statfs":{"total":95995035648,"available":90969939968,"internally_reserved":0,"allocated":4563832832,"data_stored":4563418917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":461240774},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738542,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2396708,"kb_used_data":2014292,"kb_used_omap":1,"kb_used_meta":382398,"kb_avail":91348444,"statfs":{"total":95995035648,"available":93540806656,"internally_reserved":0,"allocated":2062635008,"data_stored":2062238137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":391576005},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,1,1,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2061303808,"data_stored":2061303808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4567203840,"data_stored":4567203840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1529020416,"data_stored":1529020416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4109762560,"data_stored":4109762560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1518534656,"data_stored":1518534656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4598661120,"data_stored":4598661120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1532887040,"data_stored":1532887040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4590731264,"data_stored":4590731264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:50:47.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool set unique_pool_2 pgp_num 16 2024-09-21T11:50:47.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:47.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:47 smithi043 ceph-mon[29710]: pgmap v516: 148 pgs: 148 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-21T11:50:47.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:47 smithi043 ceph-mon[29710]: from='client.15825 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:50:47.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:47 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:47.457 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:47 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:47.457 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:47 smithi043 ceph-mon[36454]: pgmap v516: 148 pgs: 148 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-21T11:50:47.457 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:47 smithi043 ceph-mon[36454]: from='client.15825 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:50:47.457 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:47 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:47.457 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:47 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:47.501 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:47.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:47.502 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:47.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:47 smithi175 ceph-mon[32613]: pgmap v516: 148 pgs: 148 active+clean; 7.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1023 B/s rd, 64 MiB/s wr, 1.03k op/s 2024-09-21T11:50:47.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:47 smithi175 ceph-mon[32613]: from='client.15825 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:50:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:47 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:47 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:50:47.603 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:47.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:47.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:47.753 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:50:47.753 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:50:47.757 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:47.782 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:47.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:47.783 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:47.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:47.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:47.930 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:47.930 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:48.031 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:48.062 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:48.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:48.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:48.164 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:48.192 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:48.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:48.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:48.294 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:48.326 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:48.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:48.327 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:48.427 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:48.458 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:48.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:48.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:48.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:50:48.561 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:48.585 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:48.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:48.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:48.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:48.709 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:48.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:48.710 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:48.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:48.855 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:48.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:48.856 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:48.957 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:48.989 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:48.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:48.990 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:49.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:49.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:49.127 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:49.228 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:49.233 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:50:49.256 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:49.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:49.257 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:49.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:49.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:49.397 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:49.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:49 smithi043 ceph-mon[36454]: pgmap v517: 148 pgs: 148 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 48 MiB/s wr, 1.13k op/s 2024-09-21T11:50:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3142375817' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:49.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:49 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:49 smithi043 ceph-mon[29710]: pgmap v517: 148 pgs: 148 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 48 MiB/s wr, 1.13k op/s 2024-09-21T11:50:49.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3142375817' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:49.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:49 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:49.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:49.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:49 smithi175 ceph-mon[32613]: pgmap v517: 148 pgs: 148 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 48 MiB/s wr, 1.13k op/s 2024-09-21T11:50:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3142375817' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:49 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:50:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:49.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:49.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:49.637 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:49.664 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:49.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:49.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:49.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:49.798 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:49.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:49.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:49.900 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:49.934 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:49.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:49.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:50.035 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:50.075 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:50.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:50.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:50.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:50.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:50.201 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:50.301 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:50.323 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:50.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:50.324 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:50.336 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:50 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:50.336 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:50 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:50:50.336 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:50 smithi175 ceph-mon[32613]: osdmap e279: 8 total, 8 up, 8 in 2024-09-21T11:50:50.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:50.463 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:50.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:50.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:50 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:50 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:50:50.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:50 smithi043 ceph-mon[29710]: osdmap e279: 8 total, 8 up, 8 in 2024-09-21T11:50:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:50 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:50 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:50:50.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:50 smithi043 ceph-mon[36454]: osdmap e279: 8 total, 8 up, 8 in 2024-09-21T11:50:50.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:50.588 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:50.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:50.589 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:50.690 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:50.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:50.728 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:50.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:50.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:50.865 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:50.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:50.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:50.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:51.000 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:51.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:51.001 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:51.102 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:51.105 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:50:51.138 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:51.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:51.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:51.240 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:51.272 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:51.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:51.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:51.321 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:51.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:51.415 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:51.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:51.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:51.459 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:51 smithi043 ceph-mon[36454]: pgmap v519: 148 pgs: 148 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 1.19k op/s 2024-09-21T11:50:51.459 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:51 smithi043 ceph-mon[29710]: pgmap v519: 148 pgs: 148 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 1.19k op/s 2024-09-21T11:50:51.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:51 smithi175 ceph-mon[32613]: pgmap v519: 148 pgs: 148 active+clean; 7.5 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 MiB/s wr, 1.19k op/s 2024-09-21T11:50:51.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:50:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:50:51.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:51.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:51.553 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:51.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:51.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:51.692 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:51.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:51.693 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:51.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:51.822 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:51.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:51.823 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:51.924 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:51.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:51.956 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:51.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:52.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:52.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:52.086 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:52.086 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:52.187 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:52.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:52.224 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:52.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:52.325 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:52.350 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:52.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:52.351 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:52.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:52.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:52 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:50:52.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:52 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:50:52.367 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:52.367 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:52 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:50:52.367 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:52 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:50:52.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:52.455 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:52.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:50:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:52 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:50:52.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:52 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:50:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:52.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:52.535 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:52.636 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:52.663 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:52.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:52.664 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:52.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:52.807 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:52.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:52.808 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:52.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:52.951 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:52.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:52.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:53.053 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:53.074 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:53.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:53.075 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:53.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:53.211 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:53.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:53.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:53.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 9 pgp_num to 16 2024-09-21T11:50:53.337 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:53.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:53.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:53.438 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:53.464 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:53.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:53.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:53.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:53 smithi043 ceph-mon[36454]: pgmap v520: 148 pgs: 148 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-21T11:50:53.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:53 smithi043 ceph-mon[36454]: osdmap e280: 8 total, 8 up, 8 in 2024-09-21T11:50:53.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:53 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3459711631' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:50:53.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:53 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:50:53.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:53 smithi043 ceph-mon[29710]: pgmap v520: 148 pgs: 148 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-21T11:50:53.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:53 smithi043 ceph-mon[29710]: osdmap e280: 8 total, 8 up, 8 in 2024-09-21T11:50:53.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:53 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3459711631' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:50:53.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:53 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:50:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:53 smithi175 ceph-mon[32613]: pgmap v520: 148 pgs: 148 active+clean; 7.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 23 MiB/s wr, 1.17k op/s 2024-09-21T11:50:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:53 smithi175 ceph-mon[32613]: osdmap e280: 8 total, 8 up, 8 in 2024-09-21T11:50:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:53 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3459711631' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:50:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:53 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"} : dispatch 2024-09-21T11:50:53.566 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:53.600 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:53.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:53.601 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:53.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:53.735 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:53.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:53.736 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:53.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:50:53.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:53.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:53.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:53.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:54.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:54.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:54.083 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:54.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:54.110 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:50:54.110 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:50:54.152 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:50:54.152 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:50:54.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:54.189 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:50:54.190 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:50:54.242 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:50:54.242 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:50:54.253 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:54.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:54.254 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:54.300 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:50:54.300 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:50:54.346 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:50:54.346 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:50:54.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:54.381 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:50:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:50:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:50:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:50:54.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43882 8.c 8:35a2a8fe:::benchmark_data_smithi043_97042_object43881:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:50:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:50:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 333.40195517400002, 2024-09-21T11:50:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14113139399999999, 2024-09-21T11:50:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:54.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43882 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973964+0000", 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.28e-06 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973985+0000", 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1178e-05 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973994+0000", 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0119999999999994e-06 2024-09-21T11:50:54.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974001+0000", 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4570000000000002e-06 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974047+0000", 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6393000000000001e-05 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974162+0000", 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001149 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974245+0000", 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2937999999999998e-05 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.975846+0000", 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001600944 2024-09-21T11:50:54.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976044+0000", 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019846299999999999 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114984+0000", 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13893909600000001 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115046+0000", 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2924999999999998e-05 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115094+0000", 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7808e-05 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43886 8.e 8:7c60ce26:::benchmark_data_smithi043_97042_object43885:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 333.39904264500001, 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13826519300000001, 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43886 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:50:54.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:50:54.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976877+0000", 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976898+0000", 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1528000000000001e-05 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976910+0000", 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1565e-05 2024-09-21T11:50:54.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976918+0000", 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7090000000000002e-06 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976966+0000", 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8168999999999999e-05 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.977045+0000", 2024-09-21T11:50:54.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9594999999999999e-05 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.977134+0000", 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8233999999999996e-05 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.978395+0000", 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012616459999999999 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.978816+0000", 2024-09-21T11:50:54.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000420522 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981299+0000", 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024827949999999999 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981332+0000", 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3321999999999997e-05 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115141+0000", 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133808599 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43890 8.6 8:68dae88a:::benchmark_data_smithi043_97042_object43889:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 333.39591460899999, 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13554245000000001, 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43890 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980005+0000", 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2470000000000001e-06 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980022+0000", 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7380999999999998e-05 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980030+0000", 2024-09-21T11:50:54.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999995e-06 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980035+0000", 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7090000000000001e-06 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980083+0000", 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7709e-05 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980191+0000", 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010809799999999999 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980327+0000", 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013611600000000001 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115211+0000", 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13488383300000001 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115245+0000", 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4292000000000003e-05 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115423+0000", 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017810499999999999 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115476+0000", 2024-09-21T11:50:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2747000000000002e-05 2024-09-21T11:50:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115546+0000", 2024-09-21T11:50:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0159000000000004e-05 2024-09-21T11:50:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:43886 8.e e172/160 ondisk, result = 0)", 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 333.39469464400003, 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133923551, 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:50:54.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:50:54.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981224+0000", 2024-09-21T11:50:54.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5899999999999996e-07 2024-09-21T11:50:54.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981227+0000", 2024-09-21T11:50:54.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8609999999999998e-06 2024-09-21T11:50:54.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981234+0000", 2024-09-21T11:50:54.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4039999999999998e-06 2024-09-21T11:50:54.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981239+0000", 2024-09-21T11:50:54.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3939999999999998e-06 2024-09-21T11:50:54.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981291+0000", 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2352000000000003e-05 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981296+0000", 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6079999999999998e-06 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115147+0000", 2024-09-21T11:50:54.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133851373 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:46239 8.1 8:8e86b97b:::benchmark_data_smithi043_97042_object46238:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 331.78516293899997, 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.144299073, 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46239 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:50:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590756+0000", 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590781+0000", 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5000000000000001e-05 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711205+0000", 2024-09-21T11:50:54.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.120423856 2024-09-21T11:50:54.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711223+0000", 2024-09-21T11:50:54.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.751e-05 2024-09-21T11:50:54.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711326+0000", 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103274 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711483+0000", 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015652 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711616+0000", 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000133518 2024-09-21T11:50:54.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.715084+0000", 2024-09-21T11:50:54.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034678579999999999 2024-09-21T11:50:54.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.715128+0000", 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3659000000000003e-05 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.734947+0000", 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019819593999999999 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.735006+0000", 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8415000000000002e-05 2024-09-21T11:50:54.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.735054+0000", 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8513000000000001e-05 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47773 8.6 8:6c5bd339:::benchmark_data_smithi043_97042_object47772:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 330.67188221599997, 2024-09-21T11:50:54.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.141128856, 2024-09-21T11:50:54.416 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.416 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.416 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47773 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704037+0000", 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-06 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704057+0000", 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0608999999999999e-05 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704068+0000", 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0611000000000001e-05 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704077+0000", 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1710000000000008e-06 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704131+0000", 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3525e-05 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704244+0000", 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113169 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704377+0000", 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013258899999999999 2024-09-21T11:50:54.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.706707+0000", 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023301260000000001 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.706973+0000", 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026666 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845071+0000", 2024-09-21T11:50:54.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13809718700000001 2024-09-21T11:50:54.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845116+0000", 2024-09-21T11:50:54.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5581999999999998e-05 2024-09-21T11:50:54.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845165+0000", 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8603999999999997e-05 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47778 8.e 8:7e5bf474:::benchmark_data_smithi043_97042_object47777:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 330.668437273, 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13783190200000001, 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47778 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707482+0000", 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2750000000000001e-06 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707504+0000", 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1684000000000001e-05 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707515+0000", 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0844e-05 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707522+0000", 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6090000000000003e-06 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707577+0000", 2024-09-21T11:50:54.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4703000000000001e-05 2024-09-21T11:50:54.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707687+0000", 2024-09-21T11:50:54.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110474 2024-09-21T11:50:54.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:50:54.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707808+0000", 2024-09-21T11:50:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012113299999999999 2024-09-21T11:50:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845066+0000", 2024-09-21T11:50:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.137257564 2024-09-21T11:50:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845181+0000", 2024-09-21T11:50:54.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011519 2024-09-21T11:50:54.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845218+0000", 2024-09-21T11:50:54.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6356000000000002e-05 2024-09-21T11:50:54.431 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:50:54.432 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:50:54.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845277+0000", 2024-09-21T11:50:54.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9357000000000001e-05 2024-09-21T11:50:54.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845313+0000", 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5713000000000002e-05 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47781 8.6 8:69458834:::benchmark_data_smithi043_97042_object47780:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 330.66762677600002, 2024-09-21T11:50:54.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13727709099999999, 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47781 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:50:54.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708293+0000", 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2440000000000001e-06 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708313+0000", 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0846e-05 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708324+0000", 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0563e-05 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708331+0000", 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0820000000000001e-06 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708421+0000", 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0422999999999998e-05 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708530+0000", 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108286 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708651+0000", 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120821 2024-09-21T11:50:54.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845324+0000", 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.136673447 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845358+0000", 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3760999999999997e-05 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845478+0000", 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120512 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845531+0000", 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2714999999999999e-05 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845568+0000", 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7391000000000002e-05 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47784 8.6 8:65eeb18a:::benchmark_data_smithi043_97042_object47783:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 330.66574239699997, 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13626728099999999, 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47784 2024-09-21T11:50:54.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710177+0000", 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.229e-06 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710199+0000", 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469000000000001e-05 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710210+0000", 2024-09-21T11:50:54.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0387000000000001e-05 2024-09-21T11:50:54.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710217+0000", 2024-09-21T11:50:54.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8709999999999999e-06 2024-09-21T11:50:54.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710448+0000", 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023145800000000001 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710539+0000", 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0539999999999997e-05 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:50:54.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710618+0000", 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9515e-05 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845369+0000", 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134750694 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845618+0000", 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024913600000000001 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846364+0000", 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074579699999999997 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846408+0000", 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4425000000000002e-05 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846443+0000", 2024-09-21T11:50:54.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4759999999999999e-05 2024-09-21T11:50:54.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.447 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.447 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47773 8.6 e173/160 ondisk, result = 0)", 2024-09-21T11:50:54.447 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-21T11:50:54.447 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[29710]: osdmap e281: 8 total, 8 up, 8 in 2024-09-21T11:50:54.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:50:54.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:50:54.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3866934520' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:54.448 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:54.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-21T11:50:54.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[36454]: osdmap e281: 8 total, 8 up, 8 in 2024-09-21T11:50:54.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:50:54.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:50:54.448 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3866934520' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:54.449 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:54 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:54.449 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:50:54.449 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 330.66567875800001, 2024-09-21T11:50:54.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134942961, 2024-09-21T11:50:54.449 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.449 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.449 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:54.449 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710240+0000", 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5600000000000001e-07 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710242+0000", 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2239999999999998e-06 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710248+0000", 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5949999999999998e-06 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710252+0000", 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3950000000000003e-06 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845062+0000", 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13480961599999999 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845067+0000", 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1019999999999996e-06 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845182+0000", 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115573 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47778 8.e e173/160 ondisk, result = 0)", 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 330.66563097199997, 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134785869, 2024-09-21T11:50:54.453 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710290+0000", 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5339999999999998e-06 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710296+0000", 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0970000000000002e-06 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710301+0000", 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7809999999999996e-06 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845053+0000", 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13475179800000001 2024-09-21T11:50:54.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845061+0000", 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845073+0000", 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2425999999999999e-05 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.458 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.458 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47781 8.6 e173/160 ondisk, result = 0)", 2024-09-21T11:50:54.458 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:50:54.458 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 330.66496053100002, 2024-09-21T11:50:54.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13441215100000001, 2024-09-21T11:50:54.458 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:54.459 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:50:54.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:50:54.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:50:54.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-21T11:50:54.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710962+0000", 2024-09-21T11:50:54.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.692e-06 2024-09-21T11:50:54.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842829+0000", 2024-09-21T11:50:54.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13186700100000001 2024-09-21T11:50:54.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842844+0000", 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5281000000000002e-05 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845347+0000", 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002502659 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845353+0000", 2024-09-21T11:50:54.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6510000000000004e-06 2024-09-21T11:50:54.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845370+0000", 2024-09-21T11:50:54.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6214000000000001e-05 2024-09-21T11:50:54.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.464 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.464 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47792 8.e 8:71619300:::benchmark_data_smithi043_97042_object47791:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 330.66273300099999, 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14195813500000001, 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47792 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713186+0000", 2024-09-21T11:50:54.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-21T11:50:54.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:54.467 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:54.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:54.468 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:50:54.468 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:50:54.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713211+0000", 2024-09-21T11:50:54.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4343999999999999e-05 2024-09-21T11:50:54.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.843341+0000", 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13012984499999999 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.843353+0000", 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2554e-05 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845654+0000", 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023008730000000002 2024-09-21T11:50:54.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845787+0000", 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013336700000000001 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845921+0000", 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000134031 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.853059+0000", 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071370510000000002 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.853133+0000", 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4765000000000007e-05 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855050+0000", 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019165880000000001 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855099+0000", 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8680000000000001e-05 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855143+0000", 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4626000000000003e-05 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.474 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:66142 8.1 8:87e3d573:::benchmark_data_smithi043_97042_object66141:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:50:54.474 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:50:54.474 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 318.44969261099999, 2024-09-21T11:50:54.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13618391299999999, 2024-09-21T11:50:54.474 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.474 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 66142 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926226+0000", 2024-09-21T11:50:54.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3500000000000005e-07 2024-09-21T11:50:54.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926267+0000", 2024-09-21T11:50:54.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0932e-05 2024-09-21T11:50:54.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926277+0000", 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8609999999999993e-06 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926283+0000", 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4010000000000002e-06 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926387+0000", 2024-09-21T11:50:54.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103999 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926630+0000", 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000242938 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926754+0000", 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123433 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.930232+0000", 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034784540000000002 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.930829+0000", 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059640099999999998 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.059677+0000", 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12884799699999999 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.059734+0000", 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7191999999999997e-05 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.062409+0000", 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026754700000000001 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119715 9.b 9:d24acb6c:::benchmark_data_smithi043_126002_object119714:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:50:54.480 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.724272251999999, 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.205727302, 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119715 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651647+0000", 2024-09-21T11:50:54.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651670+0000", 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2603999999999999e-05 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651679+0000", 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2490000000000007e-06 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651719+0000", 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0123999999999997e-05 2024-09-21T11:50:54.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652107+0000", 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00038846399999999999 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652254+0000", 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014700100000000001 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:50:54.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652440+0000", 2024-09-21T11:50:54.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018522499999999999 2024-09-21T11:50:54.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851244+0000", 2024-09-21T11:50:54.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.198804392 2024-09-21T11:50:54.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851269+0000", 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5381999999999999e-05 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857251+0000", 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059817589999999997 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857317+0000", 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5874e-05 2024-09-21T11:50:54.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857373+0000", 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6014000000000003e-05 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119721 9.5 9:a09f84b6:::benchmark_data_smithi043_126002_object119720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.719494962999999, 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20659565899999999, 2024-09-21T11:50:54.487 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119721 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656425+0000", 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.725e-06 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656448+0000", 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3238999999999999e-05 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656461+0000", 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.331e-05 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656470+0000", 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7940000000000008e-06 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656563+0000", 2024-09-21T11:50:54.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2993999999999998e-05 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656754+0000", 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019041800000000001 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656917+0000", 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016343600000000001 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.856079+0000", 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.199162268 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.858867+0000", 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002787253 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.862926+0000", 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040595980000000002 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.862979+0000", 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3369999999999999e-05 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.863019+0000", 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9254000000000001e-05 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119722 9.9 9:9c70bd2d:::benchmark_data_smithi043_126002_object119721:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:50:54.493 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.719421654000001, 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.21088224999999999, 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119722 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:50:54.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:50:54.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656497+0000", 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4499999999999997e-07 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656518+0000", 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1294e-05 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656526+0000", 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7570000000000004e-06 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656532+0000", 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656639+0000", 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010705999999999999 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656753+0000", 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011366099999999999 2024-09-21T11:50:54.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657088+0000", 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00033480000000000001 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.860812+0000", 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.203723983 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.860856+0000", 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3983999999999998e-05 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867254+0000", 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063985939999999996 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867337+0000", 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2649000000000001e-05 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867379+0000", 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1854000000000003e-05 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:119715 9.b e277/247 ondisk, result = 0)", 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.719356311999999, 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19469081599999999, 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.500 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656563+0000", 2024-09-21T11:50:54.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2699999999999998e-07 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656565+0000", 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5900000000000002e-06 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656572+0000", 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8730000000000001e-06 2024-09-21T11:50:54.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656577+0000", 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3020000000000002e-06 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851228+0000", 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19465021800000001 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851237+0000", 2024-09-21T11:50:54.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2639999999999997e-06 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851253+0000", 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5741999999999999e-05 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:119715 9.b e277/247 ondisk, result = 0)", 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.718557230000002, 2024-09-21T11:50:54.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20004322699999999, 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:50:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657362+0000", 2024-09-21T11:50:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2799999999999995e-07 2024-09-21T11:50:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657398+0000", 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6180000000000003e-05 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851962+0000", 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19456380700000001 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851969+0000", 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2470000000000002e-06 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857236+0000", 2024-09-21T11:50:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052674760000000001 2024-09-21T11:50:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857245+0000", 2024-09-21T11:50:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3350000000000007e-06 2024-09-21T11:50:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857404+0000", 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015935400000000001 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119726 9.9 9:9360eb09:::benchmark_data_smithi043_126002_object119725:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 17.716755907, 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20832704299999999, 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:54.509 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119726 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:54.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659164+0000", 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2450000000000001e-06 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659190+0000", 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5653999999999999e-05 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:54.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851168+0000", 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.191977544 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851185+0000", 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7118000000000001e-05 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854231+0000", 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003045889 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854382+0000", 2024-09-21T11:50:54.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015179700000000001 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854513+0000", 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013041499999999999 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867241+0000", 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012728013 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867388+0000", 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147128 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867435+0000", 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6724000000000002e-05 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867467+0000", 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1986000000000002e-05 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:54.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:50:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867489+0000", 2024-09-21T11:50:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2529999999999999e-05 2024-09-21T11:50:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:54.515 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:54.515 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:50:54.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:54.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:54.614 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:54.615 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:54.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:54.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:54.755 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:54.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:54 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num", "val": "16"}]': finished 2024-09-21T11:50:54.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:54 smithi175 ceph-mon[32613]: osdmap e281: 8 total, 8 up, 8 in 2024-09-21T11:50:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:54 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:50:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:54 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:50:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:54 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3866934520' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:50:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:54 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:50:54.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:54.895 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:54.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:54.896 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:54.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:55.038 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:55.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:55.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:55.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:55.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:55.193 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:55.294 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:55.319 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:55.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:55.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:55.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:55.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:55.463 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:55.463 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:50:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: pgmap v523: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 973 op/s 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: osdmap e282: 8 total, 8 up, 8 in 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi175 to 3248M 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:55.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: pgmap v523: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 973 op/s 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: osdmap e282: 8 total, 8 up, 8 in 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi175 to 3248M 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:55 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:55.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:55.608 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:55.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:55.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:55.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:50:55.752 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:55.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:55.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:50:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: pgmap v523: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 973 op/s 2024-09-21T11:50:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: osdmap e282: 8 total, 8 up, 8 in 2024-09-21T11:50:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi175 to 3248M 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T11:50:55.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:55 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:55.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:55.890 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:55.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:55.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:55.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:56.034 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:56.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:56.035 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:56.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:56.174 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:56.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:56.175 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:56.276 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:56.311 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:56.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:56.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:56.413 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:56.446 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:56.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:56.447 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:56.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:56.584 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:56.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:56.585 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:56.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:56.701 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:50:56.701 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:50:56.701 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[29710]: Adjusting osd_memory_target on smithi043 to 3249M 2024-09-21T11:50:56.701 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:56.701 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:56.701 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:56.702 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[29710]: osdmap e283: 8 total, 8 up, 8 in 2024-09-21T11:50:56.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:50:56.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:50:56.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[36454]: Adjusting osd_memory_target on smithi043 to 3249M 2024-09-21T11:50:56.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:56.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:56.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:56.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:56 smithi043 ceph-mon[36454]: osdmap e283: 8 total, 8 up, 8 in 2024-09-21T11:50:56.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:56.721 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:56.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:56 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:50:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:56 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:50:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:56 smithi175 ceph-mon[32613]: Adjusting osd_memory_target on smithi043 to 3249M 2024-09-21T11:50:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:56 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:50:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:56 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:50:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:56 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:50:56.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:56 smithi175 ceph-mon[32613]: osdmap e283: 8 total, 8 up, 8 in 2024-09-21T11:50:56.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:56.859 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:56.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:56.860 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:56.961 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:56.998 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:56.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:56.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:57.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:50:57.099 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:57.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:57.135 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:57.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:57.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:50:57.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:57.269 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:57.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:50:57.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:57.394 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:57.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:57.395 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:57.495 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:57.528 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:57.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:57.529 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:57.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:57.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:57.672 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:57.681 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:57 smithi043 ceph-mon[29710]: pgmap v526: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 365 op/s 2024-09-21T11:50:57.681 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:57 smithi043 ceph-mon[36454]: pgmap v526: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 365 op/s 2024-09-21T11:50:57.772 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:50:57.796 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:57.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:57.797 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:57.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:57.927 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:57.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:57.928 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:58.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:57 smithi175 ceph-mon[32613]: pgmap v526: 148 pgs: 148 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 365 op/s 2024-09-21T11:50:58.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:58.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:58.074 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:58.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:58.175 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:58.201 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:58.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:58.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:58.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:58.342 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:50:58.353 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:58.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:58.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:58.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:58.465 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:50:58.465 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:50:58.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.465 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:144585 9.4 9:22d26960:::benchmark_data_smithi043_126002_object17519:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e284)", 2024-09-21T11:50:58.465 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:58.450451+0000", 2024-09-21T11:50:58.465 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012313279, 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012343781999999999, 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 144585 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:58.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.450451+0000", 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.450451+0000", 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.450452+0000", 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6789999999999999e-06 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.450458+0000", 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2580000000000003e-06 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.450473+0000", 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5446000000000001e-05 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.450489+0000", 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.611e-05 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.450614+0000", 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012530899999999999 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.450724+0000", 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011008100000000001 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.450844+0000", 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119117 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.454094+0000", 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032504669999999999 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.454106+0000", 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2245e-05 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.470 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:144588 9.a e284/247 9:5b7239ee:::benchmark_data_smithi043_126002_object17522:head v 284'9072, mlcod=284'9070)", 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:58.456989+0000", 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0057750889999999997, 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058892240000000002, 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.456989+0000", 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.456989+0000", 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.456990+0000", 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2950000000000001e-06 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.456997+0000", 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0339999999999999e-06 2024-09-21T11:50:58.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457011+0000", 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4387e-05 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457019+0000", 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3919999999999997e-06 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457152+0000", 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013288199999999999 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457186+0000", 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4504e-05 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:144591 9.3 e284/247 9:cd1bb8dd:::benchmark_data_smithi043_126002_object17525:head v 284'8924, mlcod=284'8922)", 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:58.457574+0000", 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0051902240000000002, 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0053542490000000002, 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:50:58.474 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457574+0000", 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457574+0000", 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457574+0000", 2024-09-21T11:50:58.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4000000000000001e-07 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457579+0000", 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.369e-06 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457587+0000", 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3780000000000001e-06 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457594+0000", 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6789999999999999e-06 2024-09-21T11:50:58.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457688+0000", 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4272999999999997e-05 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.457713+0000", 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5415e-05 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.462891+0000", 2024-09-21T11:50:58.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051778659999999997 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:144590 9.6 e284/247 9:66324851:::benchmark_data_smithi043_126002_object17524:head v 284'8993, mlcod=284'8991)", 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:58.458331+0000", 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.004432432, 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0046518180000000003, 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:58.478 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:58.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:58.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.458331+0000", 2024-09-21T11:50:58.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:58.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.458331+0000", 2024-09-21T11:50:58.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.458332+0000", 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6899999999999996e-07 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.458337+0000", 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0089999999999998e-06 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:58.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.458350+0000", 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2359e-05 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.458356+0000", 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0399999999999998e-06 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.458785+0000", 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042902899999999999 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.458803+0000", 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8858000000000001e-05 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:144594 9.3 e284/247 9:c6f32d2c:::benchmark_data_smithi043_126002_object17528:head v 284'8925, mlcod=284'8922)", 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:58.459598+0000", 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0031663059999999998, 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003440065, 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:58.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.459598+0000", 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.459598+0000", 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.459598+0000", 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8899999999999998e-07 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.459604+0000", 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4820000000000002e-06 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.459617+0000", 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3351e-05 2024-09-21T11:50:58.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.459624+0000", 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7970000000000003e-06 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.460480+0000", 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00085600100000000001 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.460499+0000", 2024-09-21T11:50:58.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8576999999999998e-05 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:144593 9.2 e284/247 9:4bf77d1f:::benchmark_data_smithi043_126002_object17527:head v 284'9072, mlcod=284'9070)", 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:58.461981+0000", 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00078286999999999996, 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011094379999999999, 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:50:58.486 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.461981+0000", 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.461981+0000", 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.461982+0000", 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.64e-07 2024-09-21T11:50:58.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.461989+0000", 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6429999999999998e-06 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.462003+0000", 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.471e-05 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.462010+0000", 2024-09-21T11:50:58.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.109e-06 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.462205+0000", 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019464300000000001 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:58.462261+0000", 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5928999999999999e-05 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:50:58.489 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 6 2024-09-21T11:50:58.490 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:50:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:58.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:58.500 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:58.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:50:58.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:58.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:58.708 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:58.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:58.809 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:58.838 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:58.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:58.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:58.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:50:58.985 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:58.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:58.986 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:50:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:50:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:50:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1294818297' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:50:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[29710]: osdmap e284: 8 total, 8 up, 8 in 2024-09-21T11:50:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:50:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:50:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1294818297' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:50:58.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:58 smithi043 ceph-mon[36454]: osdmap e284: 8 total, 8 up, 8 in 2024-09-21T11:50:59.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:58 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:50:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:58 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:50:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:58 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1294818297' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:58 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:50:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:58 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:50:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:58 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:50:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:58 smithi175 ceph-mon[32613]: osdmap e284: 8 total, 8 up, 8 in 2024-09-21T11:50:59.086 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:50:59.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:59.116 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:59.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:50:59.217 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:50:59.245 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:59.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:59.247 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:50:59.347 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:59.369 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:59.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:59.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:59.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:59.510 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:59.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:59.511 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:59.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:50:59.650 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:59.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:59.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:50:59.751 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:50:59.772 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:50:59.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:59.774 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:50:59.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:50:59.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:50:59.915 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:50:59.915 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:50:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:59 smithi043 ceph-mon[36454]: pgmap v527: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 890 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-21T11:50:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:59 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T11:50:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:50:59 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T11:50:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:59 smithi043 ceph-mon[29710]: pgmap v527: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 890 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-21T11:50:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:59 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T11:50:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:50:59 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T11:51:00.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:59 smithi175 ceph-mon[32613]: pgmap v527: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 890 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-21T11:51:00.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:59 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T11:51:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:50:59 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T11:51:00.016 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:00.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:00.051 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:00.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:00.152 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:00.181 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:00.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:00.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:00.253 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:51:00.253 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:51:00.254 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-21T11:51:00.254 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T11:51:00.254 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T11:51:00.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-09-21T11:51:00.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:00.323 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:00.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:00.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:00.425 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:00.465 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:00.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:00.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:00.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:00.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:00.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:00.635 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:51:00.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:00.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:00.757 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:00.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:00.858 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:00.894 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:00.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:00.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:00.938 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:00 smithi175 ceph-mon[32613]: 3.12 scrub starts 2024-09-21T11:51:00.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:00 smithi175 ceph-mon[32613]: 3.12 scrub ok 2024-09-21T11:51:00.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:00 smithi043 ceph-mon[36454]: 3.12 scrub starts 2024-09-21T11:51:00.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:00 smithi043 ceph-mon[36454]: 3.12 scrub ok 2024-09-21T11:51:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:00 smithi043 ceph-mon[29710]: 3.12 scrub starts 2024-09-21T11:51:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:00 smithi043 ceph-mon[29710]: 3.12 scrub ok 2024-09-21T11:51:00.996 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:01.040 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:01.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:01.041 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:01.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:01.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:01.184 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:01.184 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:01.246 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:51:01.246 INFO:teuthology.orchestra.run.smithi175.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T11:51:01.246 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:51:01.285 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:01.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:01.323 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:01.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:01.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:01.458 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:01.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:01.460 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:01.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:51:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:51:01.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:01.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:01.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:01.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:01.696 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:01.741 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:01.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:01.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:01.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:01.879 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:01.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:01.880 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:01.923 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[36454]: 3.2 scrub starts 2024-09-21T11:51:01.923 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[36454]: 3.2 scrub ok 2024-09-21T11:51:01.923 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[36454]: pgmap v529: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 980 op/s 2024-09-21T11:51:01.923 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[36454]: 9.a scrub starts 2024-09-21T11:51:01.923 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:01.923 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:01.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[29710]: 3.2 scrub starts 2024-09-21T11:51:01.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[29710]: 3.2 scrub ok 2024-09-21T11:51:01.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[29710]: pgmap v529: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 980 op/s 2024-09-21T11:51:01.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[29710]: 9.a scrub starts 2024-09-21T11:51:01.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:01.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:01 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:01.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:01 smithi175 ceph-mon[32613]: 3.2 scrub starts 2024-09-21T11:51:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:01 smithi175 ceph-mon[32613]: 3.2 scrub ok 2024-09-21T11:51:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:01 smithi175 ceph-mon[32613]: pgmap v529: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 980 op/s 2024-09-21T11:51:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:01 smithi175 ceph-mon[32613]: 9.a scrub starts 2024-09-21T11:51:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:01 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:02.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:02.013 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:02.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:02.154 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:02.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:02.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:02.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:02.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:02.283 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:02.284 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:02.384 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:02.420 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:02.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:02.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:02.522 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:02.560 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:02.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:02.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:02.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:02.698 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:02.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:02.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:02.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:02 smithi175 ceph-mon[32613]: 3.16 scrub starts 2024-09-21T11:51:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:02 smithi175 ceph-mon[32613]: 3.16 scrub ok 2024-09-21T11:51:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:02 smithi175 ceph-mon[32613]: pgmap v530: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-21T11:51:02.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:02.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:02.855 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:02.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:51:02.870 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:02.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:02 smithi043 ceph-mon[29710]: 3.16 scrub starts 2024-09-21T11:51:02.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:02 smithi043 ceph-mon[29710]: 3.16 scrub ok 2024-09-21T11:51:02.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:02 smithi043 ceph-mon[29710]: pgmap v530: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-21T11:51:02.924 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:02 smithi043 ceph-mon[36454]: 3.16 scrub starts 2024-09-21T11:51:02.924 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:02 smithi043 ceph-mon[36454]: 3.16 scrub ok 2024-09-21T11:51:02.925 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:02 smithi043 ceph-mon[36454]: pgmap v530: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-21T11:51:02.956 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:03.016 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:03.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:03.017 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:03.118 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:03.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:03.173 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:03.273 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:03.299 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:03.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:03.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:03.402 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:03.431 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:03.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:03.431 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:03.532 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:03.546 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:51:03.546 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:51:03.556 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:03.564 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:03.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:03.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:03.589 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:51:03.589 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:51:03.623 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:51:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:51:03.663 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:51:03.663 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:51:03.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:03.698 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:51:03.698 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:51:03.715 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:03.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:03.716 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:03.733 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:51:03.733 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:51:03.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[29710]: 3.1e scrub starts 2024-09-21T11:51:03.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[29710]: 3.1e scrub ok 2024-09-21T11:51:03.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[29710]: 9.a scrub ok 2024-09-21T11:51:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[29710]: 4.18 scrub starts 2024-09-21T11:51:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[29710]: 4.18 scrub ok 2024-09-21T11:51:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3137516770' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[36454]: 3.1e scrub starts 2024-09-21T11:51:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[36454]: 3.1e scrub ok 2024-09-21T11:51:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[36454]: 9.a scrub ok 2024-09-21T11:51:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[36454]: 4.18 scrub starts 2024-09-21T11:51:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[36454]: 4.18 scrub ok 2024-09-21T11:51:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:03 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3137516770' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:03.768 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:51:03.768 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:51:03.808 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:51:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:51:03.817 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:03.855 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:03.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:03.856 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:03.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:03.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:51:03.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:03.992 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:04.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:03 smithi175 ceph-mon[32613]: 3.1e scrub starts 2024-09-21T11:51:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:03 smithi175 ceph-mon[32613]: 3.1e scrub ok 2024-09-21T11:51:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:03 smithi175 ceph-mon[32613]: 9.a scrub ok 2024-09-21T11:51:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:03 smithi175 ceph-mon[32613]: 4.18 scrub starts 2024-09-21T11:51:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:03 smithi175 ceph-mon[32613]: 4.18 scrub ok 2024-09-21T11:51:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:03 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3137516770' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:04.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:04.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:04.138 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:04.139 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:04.239 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:04.271 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:04.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:04.272 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:51:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:51:04.373 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:04.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:04.394 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:04.394 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:04.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:04.536 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:04.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:04.537 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:04.638 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:04.668 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:04.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:04.669 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:04.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:04.810 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:04.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:04.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:04.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:04.945 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:04.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:04.947 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:04 smithi043 ceph-mon[36454]: 9.1 scrub starts 2024-09-21T11:51:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:04 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T11:51:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:04 smithi043 ceph-mon[36454]: 9.c scrub starts 2024-09-21T11:51:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:04 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T11:51:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:04 smithi043 ceph-mon[36454]: pgmap v531: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-21T11:51:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:04 smithi043 ceph-mon[29710]: 9.1 scrub starts 2024-09-21T11:51:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:04 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T11:51:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:04 smithi043 ceph-mon[29710]: 9.c scrub starts 2024-09-21T11:51:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:04 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T11:51:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:04 smithi043 ceph-mon[29710]: pgmap v531: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-21T11:51:05.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:04 smithi175 ceph-mon[32613]: 9.1 scrub starts 2024-09-21T11:51:05.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:04 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T11:51:05.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:04 smithi175 ceph-mon[32613]: 9.c scrub starts 2024-09-21T11:51:05.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:04 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T11:51:05.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:04 smithi175 ceph-mon[32613]: pgmap v531: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-21T11:51:05.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:05.077 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:05.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:05.078 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:05.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:05.201 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:05.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:05.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:05.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:05.333 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:05.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:05.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:05.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:05.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:05.479 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:05.479 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:05.490 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:51:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:51:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:51:05.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:05.605 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:05.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:05.606 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:05.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:05.746 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:05.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:05.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:05.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:05.892 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:05.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:05.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:05 smithi043 ceph-mon[36454]: 3.18 scrub starts 2024-09-21T11:51:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:05 smithi043 ceph-mon[36454]: 3.18 scrub ok 2024-09-21T11:51:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:05 smithi043 ceph-mon[36454]: 9.1 scrub ok 2024-09-21T11:51:05.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:05 smithi043 ceph-mon[29710]: 3.18 scrub starts 2024-09-21T11:51:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:05 smithi043 ceph-mon[29710]: 3.18 scrub ok 2024-09-21T11:51:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:05 smithi043 ceph-mon[29710]: 9.1 scrub ok 2024-09-21T11:51:05.995 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:06.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:05 smithi175 ceph-mon[32613]: 3.18 scrub starts 2024-09-21T11:51:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:05 smithi175 ceph-mon[32613]: 3.18 scrub ok 2024-09-21T11:51:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:05 smithi175 ceph-mon[32613]: 9.1 scrub ok 2024-09-21T11:51:06.033 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:06.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:06.034 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:06.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:06.176 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:06.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:06.177 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:06.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:06.317 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:06.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:06.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:06.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:06.457 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:06.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:06.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:06.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:06.598 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:06.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:06.599 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:06.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:06.726 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:06.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:06.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:06.827 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:06.860 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:06.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:06.861 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:06.951 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:06 smithi043 ceph-mon[29710]: 9.c scrub ok 2024-09-21T11:51:06.951 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:06 smithi043 ceph-mon[29710]: 6.13 scrub starts 2024-09-21T11:51:06.951 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:06 smithi043 ceph-mon[29710]: 6.13 scrub ok 2024-09-21T11:51:06.951 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:06 smithi043 ceph-mon[29710]: pgmap v532: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-21T11:51:06.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:06 smithi043 ceph-mon[36454]: 9.c scrub ok 2024-09-21T11:51:06.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:06 smithi043 ceph-mon[36454]: 6.13 scrub starts 2024-09-21T11:51:06.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:06 smithi043 ceph-mon[36454]: 6.13 scrub ok 2024-09-21T11:51:06.951 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:06 smithi043 ceph-mon[36454]: pgmap v532: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-21T11:51:06.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:07.000 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:07.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:07.001 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:07.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:06 smithi175 ceph-mon[32613]: 9.c scrub ok 2024-09-21T11:51:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:06 smithi175 ceph-mon[32613]: 6.13 scrub starts 2024-09-21T11:51:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:06 smithi175 ceph-mon[32613]: 6.13 scrub ok 2024-09-21T11:51:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:06 smithi175 ceph-mon[32613]: pgmap v532: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.5 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-21T11:51:07.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:07.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:07.141 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:07.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:07.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:07.272 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:07.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:07.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:07.414 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:07.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:07.415 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:07.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:07.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:51:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:07.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:07.599 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:07.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:07.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:07.741 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:07.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:07 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T11:51:07.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:07 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T11:51:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:07 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T11:51:07.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:07 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T11:51:07.842 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:07.883 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:07.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:07.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:07.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:08.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:07 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T11:51:08.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:07 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T11:51:08.014 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:08.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:08.015 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:08.116 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:08.143 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:08.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:08.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:08.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:08.283 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:08.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:08.284 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:08.376 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:51:08.377 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:51:08.377 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T11:51:08.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-21T11:51:08.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:08.412 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:08.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:08.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:08.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:08.544 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:08.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:08.545 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:08.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:08.672 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:08.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:08.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:08.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:51:08.774 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:08.800 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:08.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:08.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:08.902 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:08.930 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:08.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:08.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:08.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:08 smithi043 ceph-mon[29710]: pgmap v533: 148 pgs: 148 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:51:08.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:08 smithi043 ceph-mon[29710]: 4.2 scrub starts 2024-09-21T11:51:08.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:08 smithi043 ceph-mon[29710]: 4.2 scrub ok 2024-09-21T11:51:08.947 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:08 smithi043 ceph-mon[36454]: pgmap v533: 148 pgs: 148 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:51:08.947 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:08 smithi043 ceph-mon[36454]: 4.2 scrub starts 2024-09-21T11:51:08.947 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:08 smithi043 ceph-mon[36454]: 4.2 scrub ok 2024-09-21T11:51:08.950 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:08.965 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:09.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:08 smithi175 ceph-mon[32613]: pgmap v533: 148 pgs: 148 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:51:09.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:08 smithi175 ceph-mon[32613]: 4.2 scrub starts 2024-09-21T11:51:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:08 smithi175 ceph-mon[32613]: 4.2 scrub ok 2024-09-21T11:51:09.032 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:09.062 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:09.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:09.063 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:09.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:09.202 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:09.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:09.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:09.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:09.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:09.348 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:09.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:09.491 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:09.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:09.492 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:09.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:09.624 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:09.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:09.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:09.684 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:51:09.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:09.767 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:09.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:09.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:09.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:09.896 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:09.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:09.897 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:09.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[36454]: 5.14 scrub starts 2024-09-21T11:51:09.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[36454]: 5.14 scrub ok 2024-09-21T11:51:09.932 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3986732033' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:09.932 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:09.932 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[36454]: 6.2 scrub starts 2024-09-21T11:51:09.932 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[36454]: 6.2 scrub ok 2024-09-21T11:51:09.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[29710]: 5.14 scrub starts 2024-09-21T11:51:09.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[29710]: 5.14 scrub ok 2024-09-21T11:51:09.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3986732033' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:09.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:09.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[29710]: 6.2 scrub starts 2024-09-21T11:51:09.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:09 smithi043 ceph-mon[29710]: 6.2 scrub ok 2024-09-21T11:51:09.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:10.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:09 smithi175 ceph-mon[32613]: 5.14 scrub starts 2024-09-21T11:51:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:09 smithi175 ceph-mon[32613]: 5.14 scrub ok 2024-09-21T11:51:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3986732033' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:09 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:09 smithi175 ceph-mon[32613]: 6.2 scrub starts 2024-09-21T11:51:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:09 smithi175 ceph-mon[32613]: 6.2 scrub ok 2024-09-21T11:51:10.032 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:10.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:10.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:10.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:10.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:10.172 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:10.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:10.235 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:51:10.235 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:51:10.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:10.314 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:10.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:10.315 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:10.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:10.457 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:10.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:10.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:10.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:10.592 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:10.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:10.593 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:10.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:10.729 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:10.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:10.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:10.832 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:10.862 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:10.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:10.863 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:10.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:51:10.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:51:10.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:51:10.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:51:10.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: 6.b scrub starts 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: 6.b scrub ok 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: osdmap e285: 8 total, 8 up, 8 in 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: pgmap v535: 148 pgs: 148 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: 5.12 scrub starts 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: 5.12 scrub ok 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: 5.10 scrub starts 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[36454]: 5.10 scrub ok 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:51:10.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:51:10.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:51:10.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: 6.b scrub starts 2024-09-21T11:51:10.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: 6.b scrub ok 2024-09-21T11:51:10.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:51:10.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: osdmap e285: 8 total, 8 up, 8 in 2024-09-21T11:51:10.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: pgmap v535: 148 pgs: 148 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:51:10.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: 5.12 scrub starts 2024-09-21T11:51:10.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: 5.12 scrub ok 2024-09-21T11:51:10.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: 5.10 scrub starts 2024-09-21T11:51:10.947 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:10 smithi043 ceph-mon[29710]: 5.10 scrub ok 2024-09-21T11:51:10.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:11.006 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:11.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:11.007 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:11.010 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=cleanup t=2024-09-21T11:51:10.696441172Z level=info msg="Completed cleanup jobs" duration=8.868846ms 2024-09-21T11:51:11.011 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=plugins.update.checker t=2024-09-21T11:51:10.841253198Z level=info msg="Update check succeeded" duration=73.47827ms 2024-09-21T11:51:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:51:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:51:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:51:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:51:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: 6.b scrub starts 2024-09-21T11:51:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: 6.b scrub ok 2024-09-21T11:51:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:51:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: osdmap e285: 8 total, 8 up, 8 in 2024-09-21T11:51:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: pgmap v535: 148 pgs: 148 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:51:11.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: 5.12 scrub starts 2024-09-21T11:51:11.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: 5.12 scrub ok 2024-09-21T11:51:11.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: 5.10 scrub starts 2024-09-21T11:51:11.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:10 smithi175 ceph-mon[32613]: 5.10 scrub ok 2024-09-21T11:51:11.107 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:11.140 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:11.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:11.141 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:11.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:11.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:11.283 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:11.283 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:11.368 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 285, 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "fsid": "4f2c5252-780d-11ef-baf6-efdc52797490", 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "created": "2024-09-21T11:34:07.249195+0000", 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "modified": "2024-09-21T11:51:09.657758+0000", 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "last_up_change": "2024-09-21T11:39:49.206441+0000", 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "last_in_change": "2024-09-21T11:47:02.337266+0000", 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_num": 5799936, 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_set": [ 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "pglog_hardlimit", 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "purged_snapdirs", 2024-09-21T11:51:11.369 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_deletes", 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "sortbitwise" 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_version": 18, 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_snapshot": "", 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_max": 9, 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd": 8, 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "require_min_compat_client": "luminous", 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "min_compat_client": "luminous", 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "require_osd_release": "squid", 2024-09-21T11:51:11.370 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_crimson": false, 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "pools": [ 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 1, 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".mgr", 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:37:26.023430+0000", 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:11.371 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 1, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 1, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 1, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 1, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 1, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:11.372 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "253", 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:11.373 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:11.374 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:11.375 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_max": 32, 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_min": 1 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": {} 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 7.8899998664855957, 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 7.8899998664855957, 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.37999999523162842, 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 3, 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 3, 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:51:11.376 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 2, 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "datapool", 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:58.262982+0000", 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 8193, 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:11.377 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 3, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 3, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 3, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 3, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 3, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:11.378 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "61", 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 2, 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 61, 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:11.379 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:11.380 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:11.381 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd": {} 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.6500000953674316, 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.6500000953674316, 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.87999999523162842, 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:51:11.382 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 3, 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".rgw.root", 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:59.229530+0000", 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:11.384 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:11.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "60", 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.5, 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.5, 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1, 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.5, 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.5, 2024-09-21T11:51:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 4, 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.log", 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:00.380059+0000", 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:51:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "62", 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.25, 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.25, 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1, 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2.25, 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2.25, 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:51:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 5, 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.control", 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:02.330041+0000", 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "64", 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.25, 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.25, 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1, 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.25, 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.25, 2024-09-21T11:51:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 6, 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.meta", 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:04.395413+0000", 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "66", 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_bias": 4 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.75, 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.75, 2024-09-21T11:51:11.407 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1, 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.75, 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.75, 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 9, 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "unique_pool_2", 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:49:06.645451+0000", 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:11.408 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 16, 2024-09-21T11:51:11.409 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 16, 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 16, 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 16, 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 16, 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.410 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "281", 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:11.411 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:11.412 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "rados": {} 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2, 2024-09-21T11:51:11.413 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1, 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2, 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2, 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 1, 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 1 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4", 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:11.414 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 8, 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 247, 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.415 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6802", 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6804", 2024-09-21T11:51:11.416 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6803", 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6801/817279193", 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6802/817279193", 2024-09-21T11:51:11.417 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6804/817279193", 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6803/817279193", 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607", 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:11.418 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 13, 2024-09-21T11:51:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 247, 2024-09-21T11:51:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:11.419 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6805", 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.420 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6806", 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6808", 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6807", 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6805/263756171", 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6806/263756171", 2024-09-21T11:51:11.422 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6808/263756171", 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6807/263756171", 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce", 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:11.423 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 18, 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 235, 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6809", 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.424 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6810", 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.425 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6812", 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6811", 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.426 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6809/687577863", 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6810/687577863", 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6812/687577863", 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6811/687577863", 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:51:11.427 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d", 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 27, 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 247, 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6813", 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6814", 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6816", 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.430 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6815", 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6813/129493158", 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6814/129493158", 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6816/129493158", 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6815/129493158", 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c", 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 33, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 235, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:11.432 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6800", 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6801", 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.433 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6803", 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6802", 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6800/1323992640", 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6801/1323992640", 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6803/1323992640", 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6802/1323992640", 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "4d647321-71a1-412b-bbf5-921441923a65", 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 39, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 247, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:11.436 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6804", 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6805", 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:51:11.437 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6807", 2024-09-21T11:51:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:51:11.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:11.439 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6806", 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6804/1163208458", 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6805/1163208458", 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6807/1163208458", 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6806/1163208458", 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:11.440 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354", 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 47, 2024-09-21T11:51:11.441 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 202, 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6808", 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:11.442 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6809", 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6811", 2024-09-21T11:51:11.443 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6810", 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.444 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6808/932395625", 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6809/932395625", 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6811/932395625", 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6810/932395625", 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22", 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 54, 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 247, 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6812", 2024-09-21T11:51:11.446 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6813", 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.447 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6815", 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:11.448 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6814", 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6812/3676713275", 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6813/3676713275", 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6815/3676713275", 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6814/3676713275", 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.449 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_xinfo": [ 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:50:53.391361+0000", 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:51:11.450 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:50:51.433218+0000", 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:11.451 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:50:57.338498+0000", 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:50:55.261169+0000", 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:11.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:51:09.401036+0000", 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:11.453 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:50:11.044290+0000", 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:11.454 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:50:21.846564+0000", 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:49:59.777405+0000", 2024-09-21T11:51:11.455 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap": [ 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.c", 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: 3, 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: 7, 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: 1 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_items": [ 2024-09-21T11:51:11.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.4", 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.7", 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.457 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.d", 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.13", 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.16", 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:11.459 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_primaries": [], 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_temp": [], 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_temp": [], 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklist": { 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2419845767": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3249046048": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2518306973": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3115176321": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:51:11.460 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/2583179759": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/4225358170": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/347827280": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/1513342881": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/1113684867": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3345448895": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2454875216": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/911630798": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/983028036": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/4145025121": "2024-09-22T11:40:29.670859+0000" 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "range_blocklist": {}, 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profiles": { 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "default": { 2024-09-21T11:51:11.461 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-failure-domain": "osd", 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "k": "2", 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "m": "1", 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "plugin": "jerasure", 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "technique": "reed_sol_van" 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps_queue": [], 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "new_removed_snaps": [], 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "new_purged_snaps": [], 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_node_flags": {}, 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "device_class_flags": {}, 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode": { 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_enabled": false, 2024-09-21T11:51:11.462 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_bucket_count": 0, 2024-09-21T11:51:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "degraded_stretch_mode": 0, 2024-09-21T11:51:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "recovering_stretch_mode": 0, 2024-09-21T11:51:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_bucket": 0 2024-09-21T11:51:11.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:11.463 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:51:11.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:11.464 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:11.464 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:11.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:11.600 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:11.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:11.601 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:11.686 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:51:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:51:11.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:11.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:51:11.747 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:11.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:11.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:11.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:11.894 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:11.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:11.895 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:11.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[36454]: 4.16 scrub starts 2024-09-21T11:51:11.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[36454]: 4.16 scrub ok 2024-09-21T11:51:11.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:51:11.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:51:11.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3962403634' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:51:11.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[29710]: 4.16 scrub starts 2024-09-21T11:51:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[29710]: 4.16 scrub ok 2024-09-21T11:51:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:51:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:51:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3962403634' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:51:11.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:11 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:11.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:12.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:11 smithi175 ceph-mon[32613]: 4.16 scrub starts 2024-09-21T11:51:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:11 smithi175 ceph-mon[32613]: 4.16 scrub ok 2024-09-21T11:51:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:11 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:51:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:11 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:51:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:11 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3962403634' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:51:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:11 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:12.023 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:12.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:12.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:12.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:12.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:12.160 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:12.160 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:12.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:12.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:12.300 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:12.300 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:12.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:12.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:12.443 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:12.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:12.579 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:12.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:12.580 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:12.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:12.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[29710]: osdmap e286: 8 total, 8 up, 8 in 2024-09-21T11:51:12.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[29710]: 5.e scrub starts 2024-09-21T11:51:12.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[29710]: 5.e scrub ok 2024-09-21T11:51:12.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[29710]: pgmap v537: 148 pgs: 148 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:51:12.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[29710]: 6.4 scrub starts 2024-09-21T11:51:12.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[29710]: 6.4 scrub ok 2024-09-21T11:51:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:12.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:12.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:12.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:51:12.796 INFO:tasks.thrashosds.thrasher:j is {'epoch': 285, 'fsid': '4f2c5252-780d-11ef-baf6-efdc52797490', 'created': '2024-09-21T11:34:07.249195+0000', 'modified': '2024-09-21T11:51:09.657758+0000', 'last_up_change': '2024-09-21T11:39:49.206441+0000', 'last_in_change': '2024-09-21T11:47:02.337266+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 9, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T11:37:26.023430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-21T11:39:58.262982+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-21T11:39:59.229530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-21T11:40:00.380059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-21T11:40:02.330041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-21T11:40:04.395413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'unique_pool_2', 'create_time': '2024-09-21T11:49:06.645451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '281', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '5229029d-4fd6-46b6-8325-a3e004df09b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 247, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6801', 'nonce': 817279193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 817279193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 817279193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6803', 'nonce': 817279193}]}, 'public_addr': '172.21.15.43:6801/817279193', 'cluster_addr': '172.21.15.43:6802/817279193', 'heartbeat_back_addr': '172.21.15.43:6804/817279193', 'heartbeat_front_addr': '172.21.15.43:6803/817279193', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a2a7b58b-ba99-4383-a6d3-3b19388b2607', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 247, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6805', 'nonce': 263756171}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 263756171}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 263756171}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6807', 'nonce': 263756171}]}, 'public_addr': '172.21.15.43:6805/263756171', 'cluster_addr': '172.21.15.43:6806/263756171', 'heartbeat_back_addr': '172.21.15.43:6808/263756171', 'heartbeat_front_addr': '172.21.15.43:6807/263756171', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c24a60b5-5713-492b-82ee-576ea7a743ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 235, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6809', 'nonce': 687577863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 687577863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 687577863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6811', 'nonce': 687577863}]}, 'public_addr': '172.21.15.43:6809/687577863', 'cluster_addr': '172.21.15.43:6810/687577863', 'heartbeat_back_addr': '172.21.15.43:6812/687577863', 'heartbeat_front_addr': '172.21.15.43:6811/687577863', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c5d43cf8-1c72-4d7d-ae41-e8136b47177d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 247, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6813', 'nonce': 129493158}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 129493158}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 129493158}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6815', 'nonce': 129493158}]}, 'public_addr': '172.21.15.43:6813/129493158', 'cluster_addr': '172.21.15.43:6814/129493158', 'heartbeat_back_addr': '172.21.15.43:6816/129493158', 'heartbeat_front_addr': '172.21.15.43:6815/129493158', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '79ee85c5-afdc-4d68-a253-89b7b0eb652c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 33, 'up_thru': 235, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 1323992640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6801', 'nonce': 1323992640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6803', 'nonce': 1323992640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 1323992640}]}, 'public_addr': '172.21.15.175:6800/1323992640', 'cluster_addr': '172.21.15.175:6801/1323992640', 'heartbeat_back_addr': '172.21.15.175:6803/1323992640', 'heartbeat_front_addr': '172.21.15.175:6802/1323992640', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4d647321-71a1-412b-bbf5-921441923a65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 247, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 1163208458}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6805', 'nonce': 1163208458}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6807', 'nonce': 1163208458}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 1163208458}]}, 'public_addr': '172.21.15.175:6804/1163208458', 'cluster_addr': '172.21.15.175:6805/1163208458', 'heartbeat_back_addr': '172.21.15.175:6807/1163208458', 'heartbeat_front_addr': '172.21.15.175:6806/1163208458', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3c33bea7-94b5-46c1-9905-81b4612a2354', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 47, 'up_thru': 202, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 932395625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6809', 'nonce': 932395625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6811', 'nonce': 932395625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 932395625}]}, 'public_addr': '172.21.15.175:6808/932395625', 'cluster_addr': '172.21.15.175:6809/932395625', 'heartbeat_back_addr': '172.21.15.175:6811/932395625', 'heartbeat_front_addr': '172.21.15.175:6810/932395625', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '41bc9817-d9d1-4c11-94ff-7de332af6d22', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 54, 'up_thru': 247, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 3676713275}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6813', 'nonce': 3676713275}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6815', 'nonce': 3676713275}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 3676713275}]}, 'public_addr': '172.21.15.175:6812/3676713275', 'cluster_addr': '172.21.15.175:6813/3676713275', 'heartbeat_back_addr': '172.21.15.175:6815/3676713275', 'heartbeat_front_addr': '172.21.15.175:6814/3676713275', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:50:53.391361+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:50:51.433218+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:50:57.338498+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:50:55.261169+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:51:09.401036+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:50:11.044290+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:50:21.846564+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:49:59.777405+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '6.c', 'osds': [3, 7, 1]}], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.7', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.16', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.43:0/2419845767': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3249046048': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/2518306973': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/3115176321': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:6800/2583179759': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/4225358170': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/347827280': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/1513342881': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/1113684867': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3345448895': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/2454875216': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/911630798': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/983028036': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/4145025121': '2024-09-22T11:40:29.670859+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T11:51:12.796 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:51:12.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:12.823 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:12.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:12.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:12.905 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:12.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[36454]: osdmap e286: 8 total, 8 up, 8 in 2024-09-21T11:51:12.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[36454]: 5.e scrub starts 2024-09-21T11:51:12.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[36454]: 5.e scrub ok 2024-09-21T11:51:12.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[36454]: pgmap v537: 148 pgs: 148 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:51:12.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[36454]: 6.4 scrub starts 2024-09-21T11:51:12.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:12 smithi043 ceph-mon[36454]: 6.4 scrub ok 2024-09-21T11:51:13.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:13.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:12 smithi175 ceph-mon[32613]: osdmap e286: 8 total, 8 up, 8 in 2024-09-21T11:51:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:12 smithi175 ceph-mon[32613]: 5.e scrub starts 2024-09-21T11:51:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:12 smithi175 ceph-mon[32613]: 5.e scrub ok 2024-09-21T11:51:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:12 smithi175 ceph-mon[32613]: pgmap v537: 148 pgs: 148 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-21T11:51:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:12 smithi175 ceph-mon[32613]: 6.4 scrub starts 2024-09-21T11:51:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:12 smithi175 ceph-mon[32613]: 6.4 scrub ok 2024-09-21T11:51:13.036 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:13.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:13.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:13.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:13.191 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:13.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:13.192 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:13.292 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:13.323 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:13.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:13.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:13.425 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:13.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:13.447 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:13.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:13.547 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:13.581 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:13.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:13.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:13.682 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:13.703 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:13.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:13.704 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:13.805 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:13.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:13.827 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:13.827 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:13.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:13.956 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:13.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:13.957 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:13.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:13 smithi043 ceph-mon[36454]: 2.2 scrub starts 2024-09-21T11:51:13.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:13 smithi043 ceph-mon[36454]: 2.2 scrub ok 2024-09-21T11:51:13.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:13 smithi043 ceph-mon[36454]: 4.1d scrub starts 2024-09-21T11:51:13.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:13 smithi043 ceph-mon[36454]: 4.1d scrub ok 2024-09-21T11:51:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:13 smithi043 ceph-mon[29710]: 2.2 scrub starts 2024-09-21T11:51:13.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:13 smithi043 ceph-mon[29710]: 2.2 scrub ok 2024-09-21T11:51:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:13 smithi043 ceph-mon[29710]: 4.1d scrub starts 2024-09-21T11:51:13.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:13 smithi043 ceph-mon[29710]: 4.1d scrub ok 2024-09-21T11:51:14.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:13 smithi175 ceph-mon[32613]: 2.2 scrub starts 2024-09-21T11:51:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:13 smithi175 ceph-mon[32613]: 2.2 scrub ok 2024-09-21T11:51:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:13 smithi175 ceph-mon[32613]: 4.1d scrub starts 2024-09-21T11:51:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:13 smithi175 ceph-mon[32613]: 4.1d scrub ok 2024-09-21T11:51:14.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:14.106 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:14.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:14.107 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:14.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:14.240 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:14.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:14.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:14.342 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:14.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:14.379 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:14.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:14.480 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:14.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:14.509 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:14.509 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:14.610 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:14.639 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:14.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:14.640 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:14.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:14.769 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:14.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:14.770 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:14.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:14.914 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:14.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:14.915 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:14 smithi043 ceph-mon[29710]: 6.18 scrub starts 2024-09-21T11:51:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:14 smithi043 ceph-mon[29710]: 6.18 scrub ok 2024-09-21T11:51:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:14 smithi043 ceph-mon[29710]: 9.2 scrub starts 2024-09-21T11:51:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:14 smithi043 ceph-mon[29710]: pgmap v538: 148 pgs: 148 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-21T11:51:14.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:14 smithi043 ceph-mon[36454]: 6.18 scrub starts 2024-09-21T11:51:14.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:14 smithi043 ceph-mon[36454]: 6.18 scrub ok 2024-09-21T11:51:14.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:14 smithi043 ceph-mon[36454]: 9.2 scrub starts 2024-09-21T11:51:14.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:14 smithi043 ceph-mon[36454]: pgmap v538: 148 pgs: 148 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-21T11:51:15.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:14 smithi175 ceph-mon[32613]: 6.18 scrub starts 2024-09-21T11:51:15.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:14 smithi175 ceph-mon[32613]: 6.18 scrub ok 2024-09-21T11:51:15.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:14 smithi175 ceph-mon[32613]: 9.2 scrub starts 2024-09-21T11:51:15.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:14 smithi175 ceph-mon[32613]: pgmap v538: 148 pgs: 148 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-21T11:51:15.016 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:15.044 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:15.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:15.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:15.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:15.173 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:15.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:15.174 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:15.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:15.327 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:15.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:15.327 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:15.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:15.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:15.458 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:15.458 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:15.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:51:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:51:15.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:15.600 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:15.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:15.601 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:15.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:15.730 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:15.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:15.731 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:15.832 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:15.860 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:15.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:15.861 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:15.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:15.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:15.991 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:15.991 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[36454]: 6.0 scrub starts 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[36454]: 6.0 scrub ok 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[36454]: 3.a scrub starts 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[36454]: 3.a scrub ok 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[36454]: 4.12 scrub starts 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[36454]: 4.12 scrub ok 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[29710]: 6.0 scrub starts 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[29710]: 6.0 scrub ok 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[29710]: 3.a scrub starts 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[29710]: 3.a scrub ok 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[29710]: 4.12 scrub starts 2024-09-21T11:51:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:15 smithi043 ceph-mon[29710]: 4.12 scrub ok 2024-09-21T11:51:16.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:15 smithi175 ceph-mon[32613]: 6.0 scrub starts 2024-09-21T11:51:16.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:15 smithi175 ceph-mon[32613]: 6.0 scrub ok 2024-09-21T11:51:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:15 smithi175 ceph-mon[32613]: 3.a scrub starts 2024-09-21T11:51:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:15 smithi175 ceph-mon[32613]: 3.a scrub ok 2024-09-21T11:51:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:15 smithi175 ceph-mon[32613]: 4.12 scrub starts 2024-09-21T11:51:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:15 smithi175 ceph-mon[32613]: 4.12 scrub ok 2024-09-21T11:51:16.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:16.118 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:16.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:16.119 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:16.220 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:16.247 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:16.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:16.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:16.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:16.390 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:16.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:16.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:16.421 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:51:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:51:16.463 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:51:16.463 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:51:16.491 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:16.502 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:51:16.502 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:51:16.517 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:16.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:16.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:16.541 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:51:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:51:16.577 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:51:16.577 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:51:16.610 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:51:16.610 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:51:16.619 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:16.649 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:51:16.650 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:51:16.651 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:16.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:16.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:16.683 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:51:16.683 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:51:16.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:16.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:16.785 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:16.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:16.886 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:16.927 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:16.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:16.928 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:51:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:51:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 3.1d scrub starts 2024-09-21T11:51:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 3.1d scrub ok 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 2.1 scrub starts 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 2.1 scrub ok 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 5.1c scrub starts 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 5.1c scrub ok 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: pgmap v539: 148 pgs: 148 active+clean; 5.8 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 782 op/s 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 9.2 scrub ok 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 5.c scrub starts 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: 5.c scrub ok 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1342334050' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 3.1d scrub starts 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 3.1d scrub ok 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 2.1 scrub starts 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 2.1 scrub ok 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 5.1c scrub starts 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 5.1c scrub ok 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: pgmap v539: 148 pgs: 148 active+clean; 5.8 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 782 op/s 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 9.2 scrub ok 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 5.c scrub starts 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: 5.c scrub ok 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:16.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:16 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1342334050' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:51:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:51:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 3.1d scrub starts 2024-09-21T11:51:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 3.1d scrub ok 2024-09-21T11:51:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 2.1 scrub starts 2024-09-21T11:51:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 2.1 scrub ok 2024-09-21T11:51:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 5.1c scrub starts 2024-09-21T11:51:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 5.1c scrub ok 2024-09-21T11:51:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: pgmap v539: 148 pgs: 148 active+clean; 5.8 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 782 op/s 2024-09-21T11:51:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 9.2 scrub ok 2024-09-21T11:51:17.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 5.c scrub starts 2024-09-21T11:51:17.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: 5.c scrub ok 2024-09-21T11:51:17.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:51:17.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:17.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:16 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1342334050' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:17.029 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:17.059 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:17.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:17.060 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:17.161 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:17.186 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:17.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:17.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:17.287 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:17.317 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:17.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:17.318 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:17.418 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:17.452 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:17.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:17.453 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:17.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:17.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:17.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:17.678 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:17.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:17.710 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:17.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:17.811 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:17.845 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:17.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:17.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:17.946 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:17.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:17.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:17.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:18.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:18.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:18.112 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:18.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:18.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:18.233 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:18.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:18.234 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:18.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:18.359 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:18.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:18.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:18.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:18.485 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:18.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:18.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:18.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[36454]: 6.10 scrub starts 2024-09-21T11:51:18.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[36454]: 6.10 scrub ok 2024-09-21T11:51:18.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[36454]: 5.7 scrub starts 2024-09-21T11:51:18.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[36454]: 5.7 scrub ok 2024-09-21T11:51:18.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[36454]: 5.1 scrub starts 2024-09-21T11:51:18.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[36454]: 5.1 scrub ok 2024-09-21T11:51:18.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[29710]: 6.10 scrub starts 2024-09-21T11:51:18.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[29710]: 6.10 scrub ok 2024-09-21T11:51:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[29710]: 5.7 scrub starts 2024-09-21T11:51:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[29710]: 5.7 scrub ok 2024-09-21T11:51:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[29710]: 5.1 scrub starts 2024-09-21T11:51:18.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:18 smithi043 ceph-mon[29710]: 5.1 scrub ok 2024-09-21T11:51:18.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:18.624 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:18.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:18.625 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:18.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:18 smithi175 ceph-mon[32613]: 6.10 scrub starts 2024-09-21T11:51:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:18 smithi175 ceph-mon[32613]: 6.10 scrub ok 2024-09-21T11:51:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:18 smithi175 ceph-mon[32613]: 5.7 scrub starts 2024-09-21T11:51:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:18 smithi175 ceph-mon[32613]: 5.7 scrub ok 2024-09-21T11:51:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:18 smithi175 ceph-mon[32613]: 5.1 scrub starts 2024-09-21T11:51:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:18 smithi175 ceph-mon[32613]: 5.1 scrub ok 2024-09-21T11:51:18.762 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:18.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:18.764 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:18.864 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:18.893 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:18.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:18.894 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:18.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:19.022 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:19.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:19.023 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:19.124 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:19.157 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:19.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:19.158 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:19.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:19.306 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:19.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:19.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:19.390 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:51:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:51:19.391 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-21T11:51:19.391 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:51:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:19.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:19.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:19 smithi043 ceph-mon[29710]: pgmap v540: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.40k op/s 2024-09-21T11:51:19.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:19 smithi043 ceph-mon[29710]: 5.4 scrub starts 2024-09-21T11:51:19.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:19 smithi043 ceph-mon[29710]: 5.4 scrub ok 2024-09-21T11:51:19.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:19 smithi043 ceph-mon[36454]: pgmap v540: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.40k op/s 2024-09-21T11:51:19.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:19 smithi043 ceph-mon[36454]: 5.4 scrub starts 2024-09-21T11:51:19.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:19 smithi043 ceph-mon[36454]: 5.4 scrub ok 2024-09-21T11:51:19.482 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:19.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:19.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:19.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:19.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:19.637 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:19.637 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:19.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:19 smithi175 ceph-mon[32613]: pgmap v540: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.40k op/s 2024-09-21T11:51:19.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:19 smithi175 ceph-mon[32613]: 5.4 scrub starts 2024-09-21T11:51:19.662 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:19 smithi175 ceph-mon[32613]: 5.4 scrub ok 2024-09-21T11:51:19.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:19.760 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:19.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:19.761 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:19.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:19.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:19.901 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:20.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:20.036 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:20.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:20.037 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:20.138 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:20.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:20.177 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:20.177 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:20.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:20.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:51:20.313 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:20.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:20.315 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:20.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:20.452 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:20.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:20.453 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:20.454 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:20 smithi175 ceph-mon[32613]: 6.c scrub starts 2024-09-21T11:51:20.454 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:20 smithi175 ceph-mon[32613]: 6.c scrub ok 2024-09-21T11:51:20.454 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:20 smithi175 ceph-mon[32613]: 4.8 scrub starts 2024-09-21T11:51:20.454 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:20 smithi175 ceph-mon[32613]: 4.8 scrub ok 2024-09-21T11:51:20.454 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:20 smithi175 ceph-mon[32613]: 4.f scrub starts 2024-09-21T11:51:20.454 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:20 smithi175 ceph-mon[32613]: 4.f scrub ok 2024-09-21T11:51:20.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:20.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[29710]: 6.c scrub starts 2024-09-21T11:51:20.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[29710]: 6.c scrub ok 2024-09-21T11:51:20.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[29710]: 4.8 scrub starts 2024-09-21T11:51:20.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[29710]: 4.8 scrub ok 2024-09-21T11:51:20.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[29710]: 4.f scrub starts 2024-09-21T11:51:20.574 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[29710]: 4.f scrub ok 2024-09-21T11:51:20.574 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[36454]: 6.c scrub starts 2024-09-21T11:51:20.574 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[36454]: 6.c scrub ok 2024-09-21T11:51:20.574 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[36454]: 4.8 scrub starts 2024-09-21T11:51:20.574 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[36454]: 4.8 scrub ok 2024-09-21T11:51:20.574 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[36454]: 4.f scrub starts 2024-09-21T11:51:20.574 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:20 smithi043 ceph-mon[36454]: 4.f scrub ok 2024-09-21T11:51:20.602 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:20.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:20.603 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:20.704 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:20.738 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:20.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:20.739 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:20.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:20.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:20.874 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:20.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:20.956 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:51:20.975 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:21.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:21.042 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:21.143 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:21.179 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:21.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:21.180 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:21.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:21.311 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:21.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:21.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:21.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:21.467 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:21.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:21.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:21.569 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:21.599 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:21.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:21.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:21.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:21.735 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:21.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:21.736 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[36454]: 4.11 scrub starts 2024-09-21T11:51:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[36454]: 4.11 scrub ok 2024-09-21T11:51:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[36454]: pgmap v541: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:51:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[36454]: 6.17 scrub starts 2024-09-21T11:51:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[36454]: 6.17 scrub ok 2024-09-21T11:51:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[36454]: 3.1 scrub starts 2024-09-21T11:51:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[36454]: 3.1 scrub ok 2024-09-21T11:51:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[29710]: 4.11 scrub starts 2024-09-21T11:51:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[29710]: 4.11 scrub ok 2024-09-21T11:51:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[29710]: pgmap v541: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:51:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[29710]: 6.17 scrub starts 2024-09-21T11:51:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[29710]: 6.17 scrub ok 2024-09-21T11:51:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[29710]: 3.1 scrub starts 2024-09-21T11:51:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:21 smithi043 ceph-mon[29710]: 3.1 scrub ok 2024-09-21T11:51:21.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:21 smithi175 ceph-mon[32613]: 4.11 scrub starts 2024-09-21T11:51:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:21 smithi175 ceph-mon[32613]: 4.11 scrub ok 2024-09-21T11:51:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:21 smithi175 ceph-mon[32613]: pgmap v541: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:51:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:21 smithi175 ceph-mon[32613]: 6.17 scrub starts 2024-09-21T11:51:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:21 smithi175 ceph-mon[32613]: 6.17 scrub ok 2024-09-21T11:51:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:21 smithi175 ceph-mon[32613]: 3.1 scrub starts 2024-09-21T11:51:21.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:21 smithi175 ceph-mon[32613]: 3.1 scrub ok 2024-09-21T11:51:21.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:51:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:51:21.786 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:51:21.789 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:51:21.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:21.891 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:21.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:21.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:21.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:22.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:22.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:22.054 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:22.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:22.183 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:22.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:22.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:22.284 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:22.321 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:22.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:22.322 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:22.390 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:51:22.414 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:22 smithi043 ceph-mon[29710]: 5.0 scrub starts 2024-09-21T11:51:22.414 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:22 smithi043 ceph-mon[29710]: 5.0 scrub ok 2024-09-21T11:51:22.414 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:22.414 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2095740137' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:51:22.414 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:22 smithi043 ceph-mon[36454]: 5.0 scrub starts 2024-09-21T11:51:22.414 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:22 smithi043 ceph-mon[36454]: 5.0 scrub ok 2024-09-21T11:51:22.414 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:22.414 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2095740137' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:51:22.423 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:22.451 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:22.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:22.452 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:22.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:22.577 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":541,"stamp":"2024-09-21T11:51:19.993147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5847128627,"num_objects":89411,"num_object_clones":0,"num_object_copies":268233,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89411,"num_whiteouts":0,"num_read":1933,"num_read_kb":1790,"num_write":165751,"num_write_kb":8133706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":596864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83961,"ondisk_log_size":83961,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20713896,"kb_used_data":17272424,"kb_used_omap":12,"kb_used_meta":3441011,"kb_avail":729247320,"statfs":{"total":767960285184,"available":746749255680,"internally_reserved":0,"allocated":17686962176,"data_stored":17684009305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":3523595826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-764936192,"num_objects":-11672,"num_object_clones":0,"num_object_copies":-35016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-11672,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11672,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002760"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":486,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.715445+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:51:11.715445+0000","last_peered":"2024-09-21T11:51:11.715445+0000","last_clean":"2024-09-21T11:51:11.715445+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:51:11.715445+0000","last_undegraded":"2024-09-21T11:51:11.715445+0000","last_fullsized":"2024-09-21T11:51:11.715445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:48.904875+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":565,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.695320+0000","last_change":"2024-09-21T11:51:02.926687+0000","last_active":"2024-09-21T11:51:11.695320+0000","last_peered":"2024-09-21T11:51:11.695320+0000","last_clean":"2024-09-21T11:51:11.695320+0000","last_became_active":"2024-09-21T11:47:03.370839+0000","last_became_peered":"2024-09-21T11:47:03.370839+0000","last_unstale":"2024-09-21T11:51:11.695320+0000","last_undegraded":"2024-09-21T11:51:11.695320+0000","last_fullsized":"2024-09-21T11:51:11.695320+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:02.926630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:02.926630+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:17.462119+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":187,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.697544+0000","last_change":"2024-09-21T11:50:39.017249+0000","last_active":"2024-09-21T11:51:11.697544+0000","last_peered":"2024-09-21T11:51:11.697544+0000","last_clean":"2024-09-21T11:51:11.697544+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:51:11.697544+0000","last_undegraded":"2024-09-21T11:51:11.697544+0000","last_fullsized":"2024-09-21T11:51:11.697544+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:39.017173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.017173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:55.424271+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":504,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.695816+0000","last_change":"2024-09-21T11:49:08.053201+0000","last_active":"2024-09-21T11:51:11.695816+0000","last_peered":"2024-09-21T11:51:11.695816+0000","last_clean":"2024-09-21T11:51:11.695816+0000","last_became_active":"2024-09-21T11:47:03.369249+0000","last_became_peered":"2024-09-21T11:47:03.369249+0000","last_unstale":"2024-09-21T11:51:11.695816+0000","last_undegraded":"2024-09-21T11:51:11.695816+0000","last_fullsized":"2024-09-21T11:51:11.695816+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:46.222835+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":490,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702224+0000","last_change":"2024-09-21T11:50:26.427803+0000","last_active":"2024-09-21T11:51:11.702224+0000","last_peered":"2024-09-21T11:51:11.702224+0000","last_clean":"2024-09-21T11:51:11.702224+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:51:11.702224+0000","last_undegraded":"2024-09-21T11:51:11.702224+0000","last_fullsized":"2024-09-21T11:51:11.702224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:26.427733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.427733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:34.278269+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":487,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.701441+0000","last_change":"2024-09-21T11:51:01.667426+0000","last_active":"2024-09-21T11:51:11.701441+0000","last_peered":"2024-09-21T11:51:11.701441+0000","last_clean":"2024-09-21T11:51:11.701441+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:51:11.701441+0000","last_undegraded":"2024-09-21T11:51:11.701441+0000","last_fullsized":"2024-09-21T11:51:11.701441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.667366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.667366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:02.624646+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":525,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.704601+0000","last_change":"2024-09-21T11:50:03.300848+0000","last_active":"2024-09-21T11:51:11.704601+0000","last_peered":"2024-09-21T11:51:11.704601+0000","last_clean":"2024-09-21T11:51:11.704601+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:51:11.704601+0000","last_undegraded":"2024-09-21T11:51:11.704601+0000","last_fullsized":"2024-09-21T11:51:11.704601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:50:03.300772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:03.300772+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.852364+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":508,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.695117+0000","last_change":"2024-09-21T11:50:31.279422+0000","last_active":"2024-09-21T11:51:11.695117+0000","last_peered":"2024-09-21T11:51:11.695117+0000","last_clean":"2024-09-21T11:51:11.695117+0000","last_became_active":"2024-09-21T11:47:03.369790+0000","last_became_peered":"2024-09-21T11:47:03.369790+0000","last_unstale":"2024-09-21T11:51:11.695117+0000","last_undegraded":"2024-09-21T11:51:11.695117+0000","last_fullsized":"2024-09-21T11:51:11.695117+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.279369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.279369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:42.707926+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":493,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:15.585553+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:51:15.585553+0000","last_peered":"2024-09-21T11:51:15.585553+0000","last_clean":"2024-09-21T11:51:15.585553+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:51:15.585553+0000","last_undegraded":"2024-09-21T11:51:15.585553+0000","last_fullsized":"2024-09-21T11:51:15.585553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:16.581299+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":531,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696008+0000","last_change":"2024-09-21T11:50:39.302183+0000","last_active":"2024-09-21T11:51:11.696008+0000","last_peered":"2024-09-21T11:51:11.696008+0000","last_clean":"2024-09-21T11:51:11.696008+0000","last_became_active":"2024-09-21T11:40:02.313911+0000","last_became_peered":"2024-09-21T11:40:02.313911+0000","last_unstale":"2024-09-21T11:51:11.696008+0000","last_undegraded":"2024-09-21T11:51:11.696008+0000","last_fullsized":"2024-09-21T11:51:11.696008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:50:39.302102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.302102+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:08.488427+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":485,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702313+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:51:11.702313+0000","last_peered":"2024-09-21T11:51:11.702313+0000","last_clean":"2024-09-21T11:51:11.702313+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:51:11.702313+0000","last_undegraded":"2024-09-21T11:51:11.702313+0000","last_fullsized":"2024-09-21T11:51:11.702313+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:42.620193+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":503,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:12.982402+0000","last_change":"2024-09-21T11:51:12.982402+0000","last_active":"2024-09-21T11:51:12.982402+0000","last_peered":"2024-09-21T11:51:12.982402+0000","last_clean":"2024-09-21T11:51:12.982402+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:51:12.982402+0000","last_undegraded":"2024-09-21T11:51:12.982402+0000","last_fullsized":"2024-09-21T11:51:12.982402+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:12.982081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.982081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:31.517051+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":515,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702732+0000","last_change":"2024-09-21T11:50:27.746458+0000","last_active":"2024-09-21T11:51:11.702732+0000","last_peered":"2024-09-21T11:51:11.702732+0000","last_clean":"2024-09-21T11:51:11.702732+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:51:11.702732+0000","last_undegraded":"2024-09-21T11:51:11.702732+0000","last_fullsized":"2024-09-21T11:51:11.702732+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:27.746396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.746396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:48.701384+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":537,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.695894+0000","last_change":"2024-09-21T11:49:14.010678+0000","last_active":"2024-09-21T11:51:11.695894+0000","last_peered":"2024-09-21T11:51:11.695894+0000","last_clean":"2024-09-21T11:51:11.695894+0000","last_became_active":"2024-09-21T11:47:03.371864+0000","last_became_peered":"2024-09-21T11:47:03.371864+0000","last_unstale":"2024-09-21T11:51:11.695894+0000","last_undegraded":"2024-09-21T11:51:11.695894+0000","last_fullsized":"2024-09-21T11:51:11.695894+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:47.204897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":499,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.698818+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:51:11.698818+0000","last_peered":"2024-09-21T11:51:11.698818+0000","last_clean":"2024-09-21T11:51:11.698818+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:51:11.698818+0000","last_undegraded":"2024-09-21T11:51:11.698818+0000","last_fullsized":"2024-09-21T11:51:11.698818+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.398415+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":503,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702346+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:51:11.702346+0000","last_peered":"2024-09-21T11:51:11.702346+0000","last_clean":"2024-09-21T11:51:11.702346+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:51:11.702346+0000","last_undegraded":"2024-09-21T11:51:11.702346+0000","last_fullsized":"2024-09-21T11:51:11.702346+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:43.301832+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":499,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696317+0000","last_change":"2024-09-21T11:50:32.302938+0000","last_active":"2024-09-21T11:51:11.696317+0000","last_peered":"2024-09-21T11:51:11.696317+0000","last_clean":"2024-09-21T11:51:11.696317+0000","last_became_active":"2024-09-21T11:40:06.487643+0000","last_became_peered":"2024-09-21T11:40:06.487643+0000","last_unstale":"2024-09-21T11:51:11.696317+0000","last_undegraded":"2024-09-21T11:51:11.696317+0000","last_fullsized":"2024-09-21T11:51:11.696317+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:32.302842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.302842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:55.927870+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":485,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.714543+0000","last_change":"2024-09-21T11:50:35.718093+0000","last_active":"2024-09-21T11:51:11.714543+0000","last_peered":"2024-09-21T11:51:11.714543+0000","last_clean":"2024-09-21T11:51:11.714543+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:51:11.714543+0000","last_undegraded":"2024-09-21T11:51:11.714543+0000","last_fullsized":"2024-09-21T11:51:11.714543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:35.717991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.717991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.422206+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":618,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.709083+0000","last_change":"2024-09-21T11:50:25.966805+0000","last_active":"2024-09-21T11:51:11.709083+0000","last_peered":"2024-09-21T11:51:11.709083+0000","last_clean":"2024-09-21T11:51:11.709083+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:51:11.709083+0000","last_undegraded":"2024-09-21T11:51:11.709083+0000","last_fullsized":"2024-09-21T11:51:11.709083+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:50:25.966740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.966740+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.945863+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":195,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.699204+0000","last_change":"2024-09-21T11:50:31.012243+0000","last_active":"2024-09-21T11:51:11.699204+0000","last_peered":"2024-09-21T11:51:11.699204+0000","last_clean":"2024-09-21T11:51:11.699204+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:51:11.699204+0000","last_undegraded":"2024-09-21T11:51:11.699204+0000","last_fullsized":"2024-09-21T11:51:11.699204+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.012177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.012177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.366361+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":483,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702570+0000","last_change":"2024-09-21T11:50:06.400378+0000","last_active":"2024-09-21T11:51:11.702570+0000","last_peered":"2024-09-21T11:51:11.702570+0000","last_clean":"2024-09-21T11:51:11.702570+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:51:11.702570+0000","last_undegraded":"2024-09-21T11:51:11.702570+0000","last_fullsized":"2024-09-21T11:51:11.702570+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:06.400307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.400307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":508,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696302+0000","last_change":"2024-09-21T11:49:50.122297+0000","last_active":"2024-09-21T11:51:11.696302+0000","last_peered":"2024-09-21T11:51:11.696302+0000","last_clean":"2024-09-21T11:51:11.696302+0000","last_became_active":"2024-09-21T11:47:03.369636+0000","last_became_peered":"2024-09-21T11:47:03.369636+0000","last_unstale":"2024-09-21T11:51:11.696302+0000","last_undegraded":"2024-09-21T11:51:11.696302+0000","last_fullsized":"2024-09-21T11:51:11.696302+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.122227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.122227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:57.023598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":560,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:12.652049+0000","last_change":"2024-09-21T11:51:12.652049+0000","last_active":"2024-09-21T11:51:12.652049+0000","last_peered":"2024-09-21T11:51:12.652049+0000","last_clean":"2024-09-21T11:51:12.652049+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:51:12.652049+0000","last_undegraded":"2024-09-21T11:51:12.652049+0000","last_fullsized":"2024-09-21T11:51:12.652049+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:51:12.651967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.651967+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:24.851481+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":494,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:15.908421+0000","last_change":"2024-09-21T11:51:15.908421+0000","last_active":"2024-09-21T11:51:15.908421+0000","last_peered":"2024-09-21T11:51:15.908421+0000","last_clean":"2024-09-21T11:51:15.908421+0000","last_became_active":"2024-09-21T11:40:04.328544+0000","last_became_peered":"2024-09-21T11:40:04.328544+0000","last_unstale":"2024-09-21T11:51:15.908421+0000","last_undegraded":"2024-09-21T11:51:15.908421+0000","last_fullsized":"2024-09-21T11:51:15.908421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.908339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.908339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:28.891936+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":498,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.700911+0000","last_change":"2024-09-21T11:50:27.321818+0000","last_active":"2024-09-21T11:51:11.700911+0000","last_peered":"2024-09-21T11:51:11.700911+0000","last_clean":"2024-09-21T11:51:11.700911+0000","last_became_active":"2024-09-21T11:40:06.340023+0000","last_became_peered":"2024-09-21T11:40:06.340023+0000","last_unstale":"2024-09-21T11:51:11.700911+0000","last_undegraded":"2024-09-21T11:51:11.700911+0000","last_fullsized":"2024-09-21T11:51:11.700911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:37.464853+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":191,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.697731+0000","last_change":"2024-09-21T11:50:02.920313+0000","last_active":"2024-09-21T11:51:11.697731+0000","last_peered":"2024-09-21T11:51:11.697731+0000","last_clean":"2024-09-21T11:51:11.697731+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:51:11.697731+0000","last_undegraded":"2024-09-21T11:51:11.697731+0000","last_fullsized":"2024-09-21T11:51:11.697731+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:02.920229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.920229+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:20.782582+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":528,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713652+0000","last_change":"2024-09-21T11:49:58.702620+0000","last_active":"2024-09-21T11:51:11.713652+0000","last_peered":"2024-09-21T11:51:11.713652+0000","last_clean":"2024-09-21T11:51:11.713652+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:51:11.713652+0000","last_undegraded":"2024-09-21T11:51:11.713652+0000","last_fullsized":"2024-09-21T11:51:11.713652+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:49:58.702544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.702544+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.427898+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":649,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.708926+0000","last_change":"2024-09-21T11:46:22.616426+0000","last_active":"2024-09-21T11:51:11.708926+0000","last_peered":"2024-09-21T11:51:11.708926+0000","last_clean":"2024-09-21T11:51:11.708926+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:51:11.708926+0000","last_undegraded":"2024-09-21T11:51:11.708926+0000","last_fullsized":"2024-09-21T11:51:11.708926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:46:22.616354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:22.616354+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:44.445344+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"286'10397","reported_seq":10552,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:17.130571+0000","last_change":"2024-09-21T11:51:16.011591+0000","last_active":"2024-09-21T11:51:17.130571+0000","last_peered":"2024-09-21T11:51:17.130571+0000","last_clean":"2024-09-21T11:51:17.130571+0000","last_became_active":"2024-09-21T11:49:08.748359+0000","last_became_peered":"2024-09-21T11:49:08.748359+0000","last_unstale":"2024-09-21T11:51:17.130571+0000","last_undegraded":"2024-09-21T11:51:17.130571+0000","last_fullsized":"2024-09-21T11:51:17.130571+0000","mapping_epoch":247,"log_start":"267'5500","ondisk_log_start":"267'5500","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"286'10317","last_scrub_stamp":"2024-09-21T11:51:16.011166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:51:16.011166+0000","objects_scrubbed":5697,"log_size":4897,"log_dups_size":0,"ondisk_log_size":4897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:32.495145+0000","scrub_duration":2370,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362872832,"num_objects":5537,"num_object_clones":0,"num_object_copies":16611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10397,"num_write_kb":509888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":221,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.703339+0000","last_change":"2024-09-21T11:49:15.740456+0000","last_active":"2024-09-21T11:51:11.703339+0000","last_peered":"2024-09-21T11:51:11.703339+0000","last_clean":"2024-09-21T11:51:11.703339+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:51:11.703339+0000","last_undegraded":"2024-09-21T11:51:11.703339+0000","last_fullsized":"2024-09-21T11:51:11.703339+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:49:15.739582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:15.739582+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.931869+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":517,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.701958+0000","last_change":"2024-09-21T11:49:47.205216+0000","last_active":"2024-09-21T11:51:11.701958+0000","last_peered":"2024-09-21T11:51:11.701958+0000","last_clean":"2024-09-21T11:51:11.701958+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:51:11.701958+0000","last_undegraded":"2024-09-21T11:51:11.701958+0000","last_fullsized":"2024-09-21T11:51:11.701958+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.204889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.204889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":630,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.932591+0000","last_change":"2024-09-21T11:51:11.932529+0000","last_active":"2024-09-21T11:51:11.932591+0000","last_peered":"2024-09-21T11:51:11.932591+0000","last_clean":"2024-09-21T11:51:11.932591+0000","last_became_active":"2024-09-21T11:40:04.342306+0000","last_became_peered":"2024-09-21T11:40:04.342306+0000","last_unstale":"2024-09-21T11:51:11.932591+0000","last_undegraded":"2024-09-21T11:51:11.932591+0000","last_fullsized":"2024-09-21T11:51:11.932591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:11.932479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:11.932479+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:26.439218+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":503,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.703326+0000","last_change":"2024-09-21T11:50:25.708598+0000","last_active":"2024-09-21T11:51:11.703326+0000","last_peered":"2024-09-21T11:51:11.703326+0000","last_clean":"2024-09-21T11:51:11.703326+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:51:11.703326+0000","last_undegraded":"2024-09-21T11:51:11.703326+0000","last_fullsized":"2024-09-21T11:51:11.703326+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:28.981755+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"286'10195","reported_seq":10363,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.030976+0000","last_change":"2024-09-21T11:50:28.231461+0000","last_active":"2024-09-21T11:51:16.030976+0000","last_peered":"2024-09-21T11:51:16.030976+0000","last_clean":"2024-09-21T11:51:16.030976+0000","last_became_active":"2024-09-21T11:49:08.519622+0000","last_became_peered":"2024-09-21T11:49:08.519622+0000","last_unstale":"2024-09-21T11:51:16.030976+0000","last_undegraded":"2024-09-21T11:51:16.030976+0000","last_fullsized":"2024-09-21T11:51:16.030976+0000","mapping_epoch":247,"log_start":"267'5400","ondisk_log_start":"267'5400","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6919","last_scrub_stamp":"2024-09-21T11:50:28.231378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.231378+0000","objects_scrubbed":6856,"log_size":4795,"log_dups_size":0,"ondisk_log_size":4795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:31.482523+0000","scrub_duration":1964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368836632,"num_objects":5629,"num_object_clones":0,"num_object_copies":16887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10195,"num_write_kb":506305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":549,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.701639+0000","last_change":"2024-09-21T11:49:52.209526+0000","last_active":"2024-09-21T11:51:11.701639+0000","last_peered":"2024-09-21T11:51:11.701639+0000","last_clean":"2024-09-21T11:51:11.701639+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:51:11.701639+0000","last_undegraded":"2024-09-21T11:51:11.701639+0000","last_fullsized":"2024-09-21T11:51:11.701639+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:49:52.209433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:52.209433+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":503,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713249+0000","last_change":"2024-09-21T11:49:55.238709+0000","last_active":"2024-09-21T11:51:11.713249+0000","last_peered":"2024-09-21T11:51:11.713249+0000","last_clean":"2024-09-21T11:51:11.713249+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:51:11.713249+0000","last_undegraded":"2024-09-21T11:51:11.713249+0000","last_fullsized":"2024-09-21T11:51:11.713249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:55.238302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:55.238302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":487,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.695427+0000","last_change":"2024-09-21T11:48:47.933145+0000","last_active":"2024-09-21T11:51:11.695427+0000","last_peered":"2024-09-21T11:51:11.695427+0000","last_clean":"2024-09-21T11:51:11.695427+0000","last_became_active":"2024-09-21T11:40:04.329900+0000","last_became_peered":"2024-09-21T11:40:04.329900+0000","last_unstale":"2024-09-21T11:51:11.695427+0000","last_undegraded":"2024-09-21T11:51:11.695427+0000","last_fullsized":"2024-09-21T11:51:11.695427+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:47.933099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:47.933099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.352011+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":483,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.695908+0000","last_change":"2024-09-21T11:51:08.978057+0000","last_active":"2024-09-21T11:51:11.695908+0000","last_peered":"2024-09-21T11:51:11.695908+0000","last_clean":"2024-09-21T11:51:11.695908+0000","last_became_active":"2024-09-21T11:40:06.343447+0000","last_became_peered":"2024-09-21T11:40:06.343447+0000","last_unstale":"2024-09-21T11:51:11.695908+0000","last_undegraded":"2024-09-21T11:51:11.695908+0000","last_fullsized":"2024-09-21T11:51:11.695908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:08.977984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.977984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:35.351161+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"286'10338","reported_seq":10500,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:17.112787+0000","last_change":"2024-09-21T11:51:05.618601+0000","last_active":"2024-09-21T11:51:17.112787+0000","last_peered":"2024-09-21T11:51:17.112787+0000","last_clean":"2024-09-21T11:51:17.112787+0000","last_became_active":"2024-09-21T11:49:08.609618+0000","last_became_peered":"2024-09-21T11:49:08.609618+0000","last_unstale":"2024-09-21T11:51:17.112787+0000","last_undegraded":"2024-09-21T11:51:17.112787+0000","last_fullsized":"2024-09-21T11:51:17.112787+0000","mapping_epoch":247,"log_start":"267'5400","ondisk_log_start":"267'5400","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"284'9506","last_scrub_stamp":"2024-09-21T11:51:05.618420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.618420+0000","objects_scrubbed":6457,"log_size":4938,"log_dups_size":0,"ondisk_log_size":4938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:17.769660+0000","scrub_duration":1992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":365690880,"num_objects":5580,"num_object_clones":0,"num_object_copies":16740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10338,"num_write_kb":509376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":557,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.695640+0000","last_change":"2024-09-21T11:50:38.340031+0000","last_active":"2024-09-21T11:51:11.695640+0000","last_peered":"2024-09-21T11:51:11.695640+0000","last_clean":"2024-09-21T11:51:11.695640+0000","last_became_active":"2024-09-21T11:40:02.312231+0000","last_became_peered":"2024-09-21T11:40:02.312231+0000","last_unstale":"2024-09-21T11:51:11.695640+0000","last_undegraded":"2024-09-21T11:51:11.695640+0000","last_fullsized":"2024-09-21T11:51:11.695640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:02.032406+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":533,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713946+0000","last_change":"2024-09-21T11:50:28.657341+0000","last_active":"2024-09-21T11:51:11.713946+0000","last_peered":"2024-09-21T11:51:11.713946+0000","last_clean":"2024-09-21T11:51:11.713946+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:51:11.713946+0000","last_undegraded":"2024-09-21T11:51:11.713946+0000","last_fullsized":"2024-09-21T11:51:11.713946+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:28.657261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.657261+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:46.131391+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":654,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.701488+0000","last_change":"2024-09-21T11:49:14.198995+0000","last_active":"2024-09-21T11:51:11.701488+0000","last_peered":"2024-09-21T11:51:11.701488+0000","last_clean":"2024-09-21T11:51:11.701488+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:51:11.701488+0000","last_undegraded":"2024-09-21T11:51:11.701488+0000","last_fullsized":"2024-09-21T11:51:11.701488+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:49:14.198804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.198804+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":493,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.697573+0000","last_change":"2024-09-21T11:50:31.395647+0000","last_active":"2024-09-21T11:51:11.697573+0000","last_peered":"2024-09-21T11:51:11.697573+0000","last_clean":"2024-09-21T11:51:11.697573+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:51:11.697573+0000","last_undegraded":"2024-09-21T11:51:11.697573+0000","last_fullsized":"2024-09-21T11:51:11.697573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.395586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.395586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:44.653224+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"286'10244","reported_seq":10334,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.181186+0000","last_change":"2024-09-21T11:49:08.518112+0000","last_active":"2024-09-21T11:51:16.181186+0000","last_peered":"2024-09-21T11:51:16.181186+0000","last_clean":"2024-09-21T11:51:16.181186+0000","last_became_active":"2024-09-21T11:49:08.517344+0000","last_became_peered":"2024-09-21T11:49:08.517344+0000","last_unstale":"2024-09-21T11:51:16.181186+0000","last_undegraded":"2024-09-21T11:51:16.181186+0000","last_fullsized":"2024-09-21T11:51:16.181186+0000","mapping_epoch":247,"log_start":"265'4900","ondisk_log_start":"265'4900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:53.624008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":366739456,"num_objects":5596,"num_object_clones":0,"num_object_copies":16788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10244,"num_write_kb":506880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":215,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.699828+0000","last_change":"2024-09-21T11:51:08.137774+0000","last_active":"2024-09-21T11:51:11.699828+0000","last_peered":"2024-09-21T11:51:11.699828+0000","last_clean":"2024-09-21T11:51:11.699828+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:51:11.699828+0000","last_undegraded":"2024-09-21T11:51:11.699828+0000","last_fullsized":"2024-09-21T11:51:11.699828+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:51:08.137700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.137700+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:15.672786+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":518,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.706088+0000","last_change":"2024-09-21T11:50:34.667913+0000","last_active":"2024-09-21T11:51:11.706088+0000","last_peered":"2024-09-21T11:51:11.706088+0000","last_clean":"2024-09-21T11:51:11.706088+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:51:11.706088+0000","last_undegraded":"2024-09-21T11:51:11.706088+0000","last_fullsized":"2024-09-21T11:51:11.706088+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:34.667863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.667863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:37.575353+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":625,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713813+0000","last_change":"2024-09-21T11:50:29.676678+0000","last_active":"2024-09-21T11:51:11.713813+0000","last_peered":"2024-09-21T11:51:11.713813+0000","last_clean":"2024-09-21T11:51:11.713813+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:51:11.713813+0000","last_undegraded":"2024-09-21T11:51:11.713813+0000","last_fullsized":"2024-09-21T11:51:11.713813+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:50:29.676621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:29.676621+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:35.343100+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":490,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:13.963333+0000","last_change":"2024-09-21T11:51:13.963333+0000","last_active":"2024-09-21T11:51:13.963333+0000","last_peered":"2024-09-21T11:51:13.963333+0000","last_clean":"2024-09-21T11:51:13.963333+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:51:13.963333+0000","last_undegraded":"2024-09-21T11:51:13.963333+0000","last_fullsized":"2024-09-21T11:51:13.963333+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:13.963251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:13.963251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:22.452371+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"286'10213","reported_seq":10303,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.060415+0000","last_change":"2024-09-21T11:49:08.609660+0000","last_active":"2024-09-21T11:51:16.060415+0000","last_peered":"2024-09-21T11:51:16.060415+0000","last_clean":"2024-09-21T11:51:16.060415+0000","last_became_active":"2024-09-21T11:49:08.609226+0000","last_became_peered":"2024-09-21T11:49:08.609226+0000","last_unstale":"2024-09-21T11:51:16.060415+0000","last_undegraded":"2024-09-21T11:51:16.060415+0000","last_fullsized":"2024-09-21T11:51:16.060415+0000","mapping_epoch":247,"log_start":"267'5400","ondisk_log_start":"267'5400","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4813,"log_dups_size":0,"ondisk_log_size":4813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:54.452806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":372178944,"num_objects":5679,"num_object_clones":0,"num_object_copies":17037,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10213,"num_write_kb":508544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":541,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.714291+0000","last_change":"2024-09-21T11:50:31.701109+0000","last_active":"2024-09-21T11:51:11.714291+0000","last_peered":"2024-09-21T11:51:11.714291+0000","last_clean":"2024-09-21T11:51:11.714291+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:51:11.714291+0000","last_undegraded":"2024-09-21T11:51:11.714291+0000","last_fullsized":"2024-09-21T11:51:11.714291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:50:31.701038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.701038+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:43.743270+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":188,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.704551+0000","last_change":"2024-09-21T11:49:59.954903+0000","last_active":"2024-09-21T11:51:11.704551+0000","last_peered":"2024-09-21T11:51:11.704551+0000","last_clean":"2024-09-21T11:51:11.704551+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:51:11.704551+0000","last_undegraded":"2024-09-21T11:51:11.704551+0000","last_fullsized":"2024-09-21T11:51:11.704551+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:59.954835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:59.954835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":478,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.707507+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:51:11.707507+0000","last_peered":"2024-09-21T11:51:11.707507+0000","last_clean":"2024-09-21T11:51:11.707507+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:51:11.707507+0000","last_undegraded":"2024-09-21T11:51:11.707507+0000","last_fullsized":"2024-09-21T11:51:11.707507+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.361694+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":188,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.697491+0000","last_change":"2024-09-21T11:50:36.027903+0000","last_active":"2024-09-21T11:51:11.697491+0000","last_peered":"2024-09-21T11:51:11.697491+0000","last_clean":"2024-09-21T11:51:11.697491+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:51:11.697491+0000","last_undegraded":"2024-09-21T11:51:11.697491+0000","last_fullsized":"2024-09-21T11:51:11.697491+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.027823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.027823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:48.339916+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"286'10230","reported_seq":10395,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.877540+0000","last_change":"2024-09-21T11:50:26.274463+0000","last_active":"2024-09-21T11:51:16.877540+0000","last_peered":"2024-09-21T11:51:16.877540+0000","last_clean":"2024-09-21T11:51:16.877540+0000","last_became_active":"2024-09-21T11:49:08.516355+0000","last_became_peered":"2024-09-21T11:49:08.516355+0000","last_unstale":"2024-09-21T11:51:16.877540+0000","last_undegraded":"2024-09-21T11:51:16.877540+0000","last_fullsized":"2024-09-21T11:51:16.877540+0000","mapping_epoch":247,"log_start":"261'4200","ondisk_log_start":"261'4200","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6771","last_scrub_stamp":"2024-09-21T11:50:26.274165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.274165+0000","objects_scrubbed":6698,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.909290+0000","scrub_duration":2483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360841216,"num_objects":5506,"num_object_clones":0,"num_object_copies":16518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10230,"num_write_kb":503552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":271,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:15.514589+0000","last_change":"2024-09-21T11:47:03.413948+0000","last_active":"2024-09-21T11:51:15.514589+0000","last_peered":"2024-09-21T11:51:15.514589+0000","last_clean":"2024-09-21T11:51:15.514589+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:51:15.514589+0000","last_undegraded":"2024-09-21T11:51:15.514589+0000","last_fullsized":"2024-09-21T11:51:15.514589+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"140'30","last_scrub_stamp":"2024-09-21T11:43:37.416879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:43:37.416879+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:44.029969+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":485,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696282+0000","last_change":"2024-09-21T11:42:53.097813+0000","last_active":"2024-09-21T11:51:11.696282+0000","last_peered":"2024-09-21T11:51:11.696282+0000","last_clean":"2024-09-21T11:51:11.696282+0000","last_became_active":"2024-09-21T11:40:00.313833+0000","last_became_peered":"2024-09-21T11:40:00.313833+0000","last_unstale":"2024-09-21T11:51:11.696282+0000","last_undegraded":"2024-09-21T11:51:11.696282+0000","last_fullsized":"2024-09-21T11:51:11.696282+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:42:53.097760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:42:53.097760+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:41.314220+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":526,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:12.562026+0000","last_change":"2024-09-21T11:51:12.562026+0000","last_active":"2024-09-21T11:51:12.562026+0000","last_peered":"2024-09-21T11:51:12.562026+0000","last_clean":"2024-09-21T11:51:12.562026+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:51:12.562026+0000","last_undegraded":"2024-09-21T11:51:12.562026+0000","last_fullsized":"2024-09-21T11:51:12.562026+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:51:12.561961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.561961+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:19.756001+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":500,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.715391+0000","last_change":"2024-09-21T11:50:37.858401+0000","last_active":"2024-09-21T11:51:11.715391+0000","last_peered":"2024-09-21T11:51:11.715391+0000","last_clean":"2024-09-21T11:51:11.715391+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:51:11.715391+0000","last_undegraded":"2024-09-21T11:51:11.715391+0000","last_fullsized":"2024-09-21T11:51:11.715391+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:49.922881+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":467,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.701217+0000","last_change":"2024-09-21T11:42:29.388338+0000","last_active":"2024-09-21T11:51:11.701217+0000","last_peered":"2024-09-21T11:51:11.701217+0000","last_clean":"2024-09-21T11:51:11.701217+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:51:11.701217+0000","last_undegraded":"2024-09-21T11:51:11.701217+0000","last_fullsized":"2024-09-21T11:51:11.701217+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:42:29.388276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:42:29.388276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:44.532264+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"286'10305","reported_seq":10470,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.188465+0000","last_change":"2024-09-21T11:51:02.732433+0000","last_active":"2024-09-21T11:51:16.188465+0000","last_peered":"2024-09-21T11:51:16.188465+0000","last_clean":"2024-09-21T11:51:16.188465+0000","last_became_active":"2024-09-21T11:49:08.517758+0000","last_became_peered":"2024-09-21T11:49:08.517758+0000","last_unstale":"2024-09-21T11:51:16.188465+0000","last_undegraded":"2024-09-21T11:51:16.188465+0000","last_fullsized":"2024-09-21T11:51:16.188465+0000","mapping_epoch":247,"log_start":"265'5000","ondisk_log_start":"265'5000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"284'9390","last_scrub_stamp":"2024-09-21T11:51:02.732224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:51:02.732224+0000","objects_scrubbed":6621,"log_size":5305,"log_dups_size":0,"ondisk_log_size":5305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:14.953202+0000","scrub_duration":2062,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369033216,"num_objects":5631,"num_object_clones":0,"num_object_copies":16893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10305,"num_write_kb":509952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":131,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.709443+0000","last_change":"2024-09-21T11:48:33.075081+0000","last_active":"2024-09-21T11:51:11.709443+0000","last_peered":"2024-09-21T11:51:11.709443+0000","last_clean":"2024-09-21T11:51:11.709443+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:51:11.709443+0000","last_undegraded":"2024-09-21T11:51:11.709443+0000","last_fullsized":"2024-09-21T11:51:11.709443+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-21T11:41:07.206557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:41:07.206557+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:40.858990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":192,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.698490+0000","last_change":"2024-09-21T11:50:34.087998+0000","last_active":"2024-09-21T11:51:11.698490+0000","last_peered":"2024-09-21T11:51:11.698490+0000","last_clean":"2024-09-21T11:51:11.698490+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:51:11.698490+0000","last_undegraded":"2024-09-21T11:51:11.698490+0000","last_fullsized":"2024-09-21T11:51:11.698490+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:34.087799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.087799+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:43.032931+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":520,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:15.878501+0000","last_change":"2024-09-21T11:51:15.878501+0000","last_active":"2024-09-21T11:51:15.878501+0000","last_peered":"2024-09-21T11:51:15.878501+0000","last_clean":"2024-09-21T11:51:15.878501+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:51:15.878501+0000","last_undegraded":"2024-09-21T11:51:15.878501+0000","last_fullsized":"2024-09-21T11:51:15.878501+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:15.878417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.878417+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:17.080384+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":495,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.709708+0000","last_change":"2024-09-21T11:49:47.008541+0000","last_active":"2024-09-21T11:51:11.709708+0000","last_peered":"2024-09-21T11:51:11.709708+0000","last_clean":"2024-09-21T11:51:11.709708+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:51:11.709708+0000","last_undegraded":"2024-09-21T11:51:11.709708+0000","last_fullsized":"2024-09-21T11:51:11.709708+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.008369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.008369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:54.670178+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":490,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.706371+0000","last_change":"2024-09-21T11:49:54.388131+0000","last_active":"2024-09-21T11:51:11.706371+0000","last_peered":"2024-09-21T11:51:11.706371+0000","last_clean":"2024-09-21T11:51:11.706371+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:51:11.706371+0000","last_undegraded":"2024-09-21T11:51:11.706371+0000","last_fullsized":"2024-09-21T11:51:11.706371+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.388068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.388068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"286'10461","reported_seq":10551,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.914591+0000","last_change":"2024-09-21T11:49:08.515501+0000","last_active":"2024-09-21T11:51:16.914591+0000","last_peered":"2024-09-21T11:51:16.914591+0000","last_clean":"2024-09-21T11:51:16.914591+0000","last_became_active":"2024-09-21T11:49:08.515204+0000","last_became_peered":"2024-09-21T11:49:08.515204+0000","last_unstale":"2024-09-21T11:51:16.914591+0000","last_undegraded":"2024-09-21T11:51:16.914591+0000","last_fullsized":"2024-09-21T11:51:16.914591+0000","mapping_epoch":247,"log_start":"261'4400","ondisk_log_start":"261'4400","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":6061,"log_dups_size":0,"ondisk_log_size":6061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:57.840535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369950720,"num_objects":5645,"num_object_clones":0,"num_object_copies":16935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10461,"num_write_kb":515392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":560,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713749+0000","last_change":"2024-09-21T11:47:03.421296+0000","last_active":"2024-09-21T11:51:11.713749+0000","last_peered":"2024-09-21T11:51:11.713749+0000","last_clean":"2024-09-21T11:51:11.713749+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:51:11.713749+0000","last_undegraded":"2024-09-21T11:51:11.713749+0000","last_fullsized":"2024-09-21T11:51:11.713749+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-21T11:45:09.370085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:45:09.370085+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:40.832128+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":502,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.714044+0000","last_change":"2024-09-21T11:49:16.415641+0000","last_active":"2024-09-21T11:51:11.714044+0000","last_peered":"2024-09-21T11:51:11.714044+0000","last_clean":"2024-09-21T11:51:11.714044+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:51:11.714044+0000","last_undegraded":"2024-09-21T11:51:11.714044+0000","last_fullsized":"2024-09-21T11:51:11.714044+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:16.415597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:16.415597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1212,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:14.700536+0000","last_change":"2024-09-21T11:49:51.434402+0000","last_active":"2024-09-21T11:51:14.700536+0000","last_peered":"2024-09-21T11:51:14.700536+0000","last_clean":"2024-09-21T11:51:14.700536+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:51:14.700536+0000","last_undegraded":"2024-09-21T11:51:14.700536+0000","last_fullsized":"2024-09-21T11:51:14.700536+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:51.434333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:49:51.434333+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":667,"num_read_kb":662,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":500,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.704323+0000","last_change":"2024-09-21T11:48:42.042870+0000","last_active":"2024-09-21T11:51:11.704323+0000","last_peered":"2024-09-21T11:51:11.704323+0000","last_clean":"2024-09-21T11:51:11.704323+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:51:11.704323+0000","last_undegraded":"2024-09-21T11:51:11.704323+0000","last_fullsized":"2024-09-21T11:51:11.704323+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:42.042814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:42.042814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.767298+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":196,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:12.151537+0000","last_change":"2024-09-21T11:51:12.151470+0000","last_active":"2024-09-21T11:51:12.151537+0000","last_peered":"2024-09-21T11:51:12.151537+0000","last_clean":"2024-09-21T11:51:12.151537+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:51:12.151537+0000","last_undegraded":"2024-09-21T11:51:12.151537+0000","last_fullsized":"2024-09-21T11:51:12.151537+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:12.151404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.151404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:30.877629+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"286'10326","reported_seq":10493,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:17.064143+0000","last_change":"2024-09-21T11:50:26.726878+0000","last_active":"2024-09-21T11:51:17.064143+0000","last_peered":"2024-09-21T11:51:17.064143+0000","last_clean":"2024-09-21T11:51:17.064143+0000","last_became_active":"2024-09-21T11:49:08.516627+0000","last_became_peered":"2024-09-21T11:49:08.516627+0000","last_unstale":"2024-09-21T11:51:17.064143+0000","last_undegraded":"2024-09-21T11:51:17.064143+0000","last_fullsized":"2024-09-21T11:51:17.064143+0000","mapping_epoch":247,"log_start":"268'5400","ondisk_log_start":"268'5400","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6809","last_scrub_stamp":"2024-09-21T11:50:26.726538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.726538+0000","objects_scrubbed":6753,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:53.175169+0000","scrub_duration":2384,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362676224,"num_objects":5534,"num_object_clones":0,"num_object_copies":16602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10326,"num_write_kb":507520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":674,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.708578+0000","last_change":"2024-09-21T11:50:29.389640+0000","last_active":"2024-09-21T11:51:11.708578+0000","last_peered":"2024-09-21T11:51:11.708578+0000","last_clean":"2024-09-21T11:51:11.708578+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:51:11.708578+0000","last_undegraded":"2024-09-21T11:51:11.708578+0000","last_fullsized":"2024-09-21T11:51:11.708578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:29.389555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:29.389555+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:31.107690+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":503,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702036+0000","last_change":"2024-09-21T11:50:59.680952+0000","last_active":"2024-09-21T11:51:11.702036+0000","last_peered":"2024-09-21T11:51:11.702036+0000","last_clean":"2024-09-21T11:51:11.702036+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:51:11.702036+0000","last_undegraded":"2024-09-21T11:51:11.702036+0000","last_fullsized":"2024-09-21T11:51:11.702036+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:59.680755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:59.680755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:08.366731+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":557,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.697552+0000","last_change":"2024-09-21T11:50:39.362320+0000","last_active":"2024-09-21T11:51:11.697552+0000","last_peered":"2024-09-21T11:51:11.697552+0000","last_clean":"2024-09-21T11:51:11.697552+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:51:11.697552+0000","last_undegraded":"2024-09-21T11:51:11.697552+0000","last_fullsized":"2024-09-21T11:51:11.697552+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:50:39.361647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.361647+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.251982+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":494,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.701068+0000","last_change":"2024-09-21T11:49:06.639805+0000","last_active":"2024-09-21T11:51:11.701068+0000","last_peered":"2024-09-21T11:51:11.701068+0000","last_clean":"2024-09-21T11:51:11.701068+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:51:11.701068+0000","last_undegraded":"2024-09-21T11:51:11.701068+0000","last_fullsized":"2024-09-21T11:51:11.701068+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:06.639732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:06.639732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:16.438605+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":482,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.704201+0000","last_change":"2024-09-21T11:46:14.276584+0000","last_active":"2024-09-21T11:51:11.704201+0000","last_peered":"2024-09-21T11:51:11.704201+0000","last_clean":"2024-09-21T11:51:11.704201+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:51:11.704201+0000","last_undegraded":"2024-09-21T11:51:11.704201+0000","last_fullsized":"2024-09-21T11:51:11.704201+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:14.276505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:14.276505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:40.652662+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"286'10144","reported_seq":10316,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.041332+0000","last_change":"2024-09-21T11:50:37.410553+0000","last_active":"2024-09-21T11:51:16.041332+0000","last_peered":"2024-09-21T11:51:16.041332+0000","last_clean":"2024-09-21T11:51:16.041332+0000","last_became_active":"2024-09-21T11:49:08.519011+0000","last_became_peered":"2024-09-21T11:49:08.519011+0000","last_unstale":"2024-09-21T11:51:16.041332+0000","last_undegraded":"2024-09-21T11:51:16.041332+0000","last_fullsized":"2024-09-21T11:51:16.041332+0000","mapping_epoch":247,"log_start":"267'5300","ondisk_log_start":"267'5300","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7398","last_scrub_stamp":"2024-09-21T11:50:37.410305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.410305+0000","objects_scrubbed":7345,"log_size":4844,"log_dups_size":0,"ondisk_log_size":4844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.828393+0000","scrub_duration":2058,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361234432,"num_objects":5512,"num_object_clones":0,"num_object_copies":16536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10144,"num_write_kb":500992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":560,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696505+0000","last_change":"2024-09-21T11:50:37.352894+0000","last_active":"2024-09-21T11:51:11.696505+0000","last_peered":"2024-09-21T11:51:11.696505+0000","last_clean":"2024-09-21T11:51:11.696505+0000","last_became_active":"2024-09-21T11:47:03.370727+0000","last_became_peered":"2024-09-21T11:47:03.370727+0000","last_unstale":"2024-09-21T11:51:11.696505+0000","last_undegraded":"2024-09-21T11:51:11.696505+0000","last_fullsized":"2024-09-21T11:51:11.696505+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:37.352834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.352834+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:59.843673+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":527,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.695710+0000","last_change":"2024-09-21T11:51:06.913098+0000","last_active":"2024-09-21T11:51:11.695710+0000","last_peered":"2024-09-21T11:51:11.695710+0000","last_clean":"2024-09-21T11:51:11.695710+0000","last_became_active":"2024-09-21T11:40:00.311784+0000","last_became_peered":"2024-09-21T11:40:00.311784+0000","last_unstale":"2024-09-21T11:51:11.695710+0000","last_undegraded":"2024-09-21T11:51:11.695710+0000","last_fullsized":"2024-09-21T11:51:11.695710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:06.912982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:06.912982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:29.108241+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":485,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.704124+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:51:11.704124+0000","last_peered":"2024-09-21T11:51:11.704124+0000","last_clean":"2024-09-21T11:51:11.704124+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:51:11.704124+0000","last_undegraded":"2024-09-21T11:51:11.704124+0000","last_fullsized":"2024-09-21T11:51:11.704124+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:44.619467+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":481,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702202+0000","last_change":"2024-09-21T11:48:23.285482+0000","last_active":"2024-09-21T11:51:11.702202+0000","last_peered":"2024-09-21T11:51:11.702202+0000","last_clean":"2024-09-21T11:51:11.702202+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:51:11.702202+0000","last_undegraded":"2024-09-21T11:51:11.702202+0000","last_fullsized":"2024-09-21T11:51:11.702202+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.285283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.285283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.489355+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"286'10184","reported_seq":10358,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.207544+0000","last_change":"2024-09-21T11:50:39.224005+0000","last_active":"2024-09-21T11:51:16.207544+0000","last_peered":"2024-09-21T11:51:16.207544+0000","last_clean":"2024-09-21T11:51:16.207544+0000","last_became_active":"2024-09-21T11:49:08.748155+0000","last_became_peered":"2024-09-21T11:49:08.748155+0000","last_unstale":"2024-09-21T11:51:16.207544+0000","last_undegraded":"2024-09-21T11:51:16.207544+0000","last_fullsized":"2024-09-21T11:51:16.207544+0000","mapping_epoch":247,"log_start":"267'5400","ondisk_log_start":"267'5400","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7577","last_scrub_stamp":"2024-09-21T11:50:39.223848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.223848+0000","objects_scrubbed":7518,"log_size":4784,"log_dups_size":0,"ondisk_log_size":4784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.351968+0000","scrub_duration":2220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363593728,"num_objects":5548,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10184,"num_write_kb":503424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":631,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696848+0000","last_change":"2024-09-21T11:50:36.386009+0000","last_active":"2024-09-21T11:51:11.696848+0000","last_peered":"2024-09-21T11:51:11.696848+0000","last_clean":"2024-09-21T11:51:11.696848+0000","last_became_active":"2024-09-21T11:48:33.022321+0000","last_became_peered":"2024-09-21T11:48:33.022321+0000","last_unstale":"2024-09-21T11:51:11.696848+0000","last_undegraded":"2024-09-21T11:51:11.696848+0000","last_fullsized":"2024-09-21T11:51:11.696848+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:50:36.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.385902+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:37.670911+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":524,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:14.312046+0000","last_change":"2024-09-21T11:51:14.312046+0000","last_active":"2024-09-21T11:51:14.312046+0000","last_peered":"2024-09-21T11:51:14.312046+0000","last_clean":"2024-09-21T11:51:14.312046+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:51:14.312046+0000","last_undegraded":"2024-09-21T11:51:14.312046+0000","last_fullsized":"2024-09-21T11:51:14.312046+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:14.311953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:14.311953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:25.751618+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":187,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.151389+0000","last_change":"2024-09-21T11:51:16.151389+0000","last_active":"2024-09-21T11:51:16.151389+0000","last_peered":"2024-09-21T11:51:16.151389+0000","last_clean":"2024-09-21T11:51:16.151389+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:51:16.151389+0000","last_undegraded":"2024-09-21T11:51:16.151389+0000","last_fullsized":"2024-09-21T11:51:16.151389+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:16.151330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:16.151330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:43.557799+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":490,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.709589+0000","last_change":"2024-09-21T11:46:20.746726+0000","last_active":"2024-09-21T11:51:11.709589+0000","last_peered":"2024-09-21T11:51:11.709589+0000","last_clean":"2024-09-21T11:51:11.709589+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:51:11.709589+0000","last_undegraded":"2024-09-21T11:51:11.709589+0000","last_fullsized":"2024-09-21T11:51:11.709589+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:20.741763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:20.741763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:44.758863+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"286'10453","reported_seq":10617,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:17.126283+0000","last_change":"2024-09-21T11:51:05.135848+0000","last_active":"2024-09-21T11:51:17.126283+0000","last_peered":"2024-09-21T11:51:17.126283+0000","last_clean":"2024-09-21T11:51:17.126283+0000","last_became_active":"2024-09-21T11:49:08.517011+0000","last_became_peered":"2024-09-21T11:49:08.517011+0000","last_unstale":"2024-09-21T11:51:17.126283+0000","last_undegraded":"2024-09-21T11:51:17.126283+0000","last_fullsized":"2024-09-21T11:51:17.126283+0000","mapping_epoch":247,"log_start":"269'5600","ondisk_log_start":"269'5600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"284'9566","last_scrub_stamp":"2024-09-21T11:51:05.135728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.135728+0000","objects_scrubbed":6439,"log_size":4853,"log_dups_size":0,"ondisk_log_size":4853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:35.110511+0000","scrub_duration":2470,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358023168,"num_objects":5463,"num_object_clones":0,"num_object_copies":16389,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10453,"num_write_kb":509312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":537,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696434+0000","last_change":"2024-09-21T11:50:06.225159+0000","last_active":"2024-09-21T11:51:11.696434+0000","last_peered":"2024-09-21T11:51:11.696434+0000","last_clean":"2024-09-21T11:51:11.696434+0000","last_became_active":"2024-09-21T11:40:02.316886+0000","last_became_peered":"2024-09-21T11:40:02.316886+0000","last_unstale":"2024-09-21T11:51:11.696434+0000","last_undegraded":"2024-09-21T11:51:11.696434+0000","last_fullsized":"2024-09-21T11:51:11.696434+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:32.807084+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":487,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702785+0000","last_change":"2024-09-21T11:50:35.410162+0000","last_active":"2024-09-21T11:51:11.702785+0000","last_peered":"2024-09-21T11:51:11.702785+0000","last_clean":"2024-09-21T11:51:11.702785+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:51:11.702785+0000","last_undegraded":"2024-09-21T11:51:11.702785+0000","last_fullsized":"2024-09-21T11:51:11.702785+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:35.410095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.410095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:41.088475+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":646,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.709275+0000","last_change":"2024-09-21T11:50:02.975387+0000","last_active":"2024-09-21T11:51:11.709275+0000","last_peered":"2024-09-21T11:51:11.709275+0000","last_clean":"2024-09-21T11:51:11.709275+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:51:11.709275+0000","last_undegraded":"2024-09-21T11:51:11.709275+0000","last_fullsized":"2024-09-21T11:51:11.709275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:50:02.975313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.975313+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:25.444093+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":488,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696235+0000","last_change":"2024-09-21T11:48:07.992586+0000","last_active":"2024-09-21T11:51:11.696235+0000","last_peered":"2024-09-21T11:51:11.696235+0000","last_clean":"2024-09-21T11:51:11.696235+0000","last_became_active":"2024-09-21T11:47:03.368822+0000","last_became_peered":"2024-09-21T11:47:03.368822+0000","last_unstale":"2024-09-21T11:51:11.696235+0000","last_undegraded":"2024-09-21T11:51:11.696235+0000","last_fullsized":"2024-09-21T11:51:11.696235+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:07.992517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:07.992517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:45.236026+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"286'10246","reported_seq":10411,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.149807+0000","last_change":"2024-09-21T11:50:24.709591+0000","last_active":"2024-09-21T11:51:16.149807+0000","last_peered":"2024-09-21T11:51:16.149807+0000","last_clean":"2024-09-21T11:51:16.149807+0000","last_became_active":"2024-09-21T11:49:08.748171+0000","last_became_peered":"2024-09-21T11:49:08.748171+0000","last_unstale":"2024-09-21T11:51:16.149807+0000","last_undegraded":"2024-09-21T11:51:16.149807+0000","last_fullsized":"2024-09-21T11:51:16.149807+0000","mapping_epoch":247,"log_start":"265'4900","ondisk_log_start":"265'4900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6768","last_scrub_stamp":"2024-09-21T11:50:24.709386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.709386+0000","objects_scrubbed":6700,"log_size":5346,"log_dups_size":0,"ondisk_log_size":5346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:27.416703+0000","scrub_duration":2008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":365297664,"num_objects":5574,"num_object_clones":0,"num_object_copies":16722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10246,"num_write_kb":506240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":541,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.714221+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:51:11.714221+0000","last_peered":"2024-09-21T11:51:11.714221+0000","last_clean":"2024-09-21T11:51:11.714221+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:51:11.714221+0000","last_undegraded":"2024-09-21T11:51:11.714221+0000","last_fullsized":"2024-09-21T11:51:11.714221+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.887029+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":510,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702902+0000","last_change":"2024-09-21T11:51:03.627990+0000","last_active":"2024-09-21T11:51:11.702902+0000","last_peered":"2024-09-21T11:51:11.702902+0000","last_clean":"2024-09-21T11:51:11.702902+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:51:11.702902+0000","last_undegraded":"2024-09-21T11:51:11.702902+0000","last_fullsized":"2024-09-21T11:51:11.702902+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:03.627923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:03.627923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:14.026807+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":487,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.709191+0000","last_change":"2024-09-21T11:48:49.993307+0000","last_active":"2024-09-21T11:51:11.709191+0000","last_peered":"2024-09-21T11:51:11.709191+0000","last_clean":"2024-09-21T11:51:11.709191+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:51:11.709191+0000","last_undegraded":"2024-09-21T11:51:11.709191+0000","last_fullsized":"2024-09-21T11:51:11.709191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.993230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.993230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.951895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":466,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713832+0000","last_change":"2024-09-21T11:46:54.686975+0000","last_active":"2024-09-21T11:51:11.713832+0000","last_peered":"2024-09-21T11:51:11.713832+0000","last_clean":"2024-09-21T11:51:11.713832+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:51:11.713832+0000","last_undegraded":"2024-09-21T11:51:11.713832+0000","last_fullsized":"2024-09-21T11:51:11.713832+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:54.686820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:46:54.686820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:47.933953+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"286'10448","reported_seq":10619,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.170030+0000","last_change":"2024-09-21T11:50:32.751942+0000","last_active":"2024-09-21T11:51:16.170030+0000","last_peered":"2024-09-21T11:51:16.170030+0000","last_clean":"2024-09-21T11:51:16.170030+0000","last_became_active":"2024-09-21T11:49:08.516975+0000","last_became_peered":"2024-09-21T11:49:08.516975+0000","last_unstale":"2024-09-21T11:51:16.170030+0000","last_undegraded":"2024-09-21T11:51:16.170030+0000","last_fullsized":"2024-09-21T11:51:16.170030+0000","mapping_epoch":247,"log_start":"266'5100","ondisk_log_start":"266'5100","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7356","last_scrub_stamp":"2024-09-21T11:50:32.751687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.751687+0000","objects_scrubbed":7220,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.722147+0000","scrub_duration":3036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":371458048,"num_objects":5668,"num_object_clones":0,"num_object_copies":17004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10448,"num_write_kb":515712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":615,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.707658+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:51:11.707658+0000","last_peered":"2024-09-21T11:51:11.707658+0000","last_clean":"2024-09-21T11:51:11.707658+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:51:11.707658+0000","last_undegraded":"2024-09-21T11:51:11.707658+0000","last_fullsized":"2024-09-21T11:51:11.707658+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:45.721984+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":501,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.707603+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:51:11.707603+0000","last_peered":"2024-09-21T11:51:11.707603+0000","last_clean":"2024-09-21T11:51:11.707603+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:51:11.707603+0000","last_undegraded":"2024-09-21T11:51:11.707603+0000","last_fullsized":"2024-09-21T11:51:11.707603+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":502,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.708667+0000","last_change":"2024-09-21T11:50:31.946936+0000","last_active":"2024-09-21T11:51:11.708667+0000","last_peered":"2024-09-21T11:51:11.708667+0000","last_clean":"2024-09-21T11:51:11.708667+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:51:11.708667+0000","last_undegraded":"2024-09-21T11:51:11.708667+0000","last_fullsized":"2024-09-21T11:51:11.708667+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.946859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.946859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:43.718312+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":490,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.697476+0000","last_change":"2024-09-21T11:50:34.385662+0000","last_active":"2024-09-21T11:51:11.697476+0000","last_peered":"2024-09-21T11:51:11.697476+0000","last_clean":"2024-09-21T11:51:11.697476+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:51:11.697476+0000","last_undegraded":"2024-09-21T11:51:11.697476+0000","last_fullsized":"2024-09-21T11:51:11.697476+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:34.385563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.385563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:43.597782+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"286'10163","reported_seq":10329,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.200441+0000","last_change":"2024-09-21T11:50:30.668171+0000","last_active":"2024-09-21T11:51:16.200441+0000","last_peered":"2024-09-21T11:51:16.200441+0000","last_clean":"2024-09-21T11:51:16.200441+0000","last_became_active":"2024-09-21T11:49:08.609829+0000","last_became_peered":"2024-09-21T11:49:08.609829+0000","last_unstale":"2024-09-21T11:51:16.200441+0000","last_undegraded":"2024-09-21T11:51:16.200441+0000","last_fullsized":"2024-09-21T11:51:16.200441+0000","mapping_epoch":247,"log_start":"267'5400","ondisk_log_start":"267'5400","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7027","last_scrub_stamp":"2024-09-21T11:50:30.668043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:30.668043+0000","objects_scrubbed":6939,"log_size":4763,"log_dups_size":0,"ondisk_log_size":4763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:54.716375+0000","scrub_duration":2651,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361431040,"num_objects":5515,"num_object_clones":0,"num_object_copies":16545,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10163,"num_write_kb":501696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":585,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.695550+0000","last_change":"2024-09-21T11:50:34.362829+0000","last_active":"2024-09-21T11:51:11.695550+0000","last_peered":"2024-09-21T11:51:11.695550+0000","last_clean":"2024-09-21T11:51:11.695550+0000","last_became_active":"2024-09-21T11:47:03.370090+0000","last_became_peered":"2024-09-21T11:47:03.370090+0000","last_unstale":"2024-09-21T11:51:11.695550+0000","last_undegraded":"2024-09-21T11:51:11.695550+0000","last_fullsized":"2024-09-21T11:51:11.695550+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:39.880745+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":531,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.701624+0000","last_change":"2024-09-21T11:49:08.611218+0000","last_active":"2024-09-21T11:51:11.701624+0000","last_peered":"2024-09-21T11:51:11.701624+0000","last_clean":"2024-09-21T11:51:11.701624+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:51:11.701624+0000","last_undegraded":"2024-09-21T11:51:11.701624+0000","last_fullsized":"2024-09-21T11:51:11.701624+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.611159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.611159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:46.366590+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":517,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.718194+0000","last_change":"2024-09-21T11:49:44.985594+0000","last_active":"2024-09-21T11:51:11.718194+0000","last_peered":"2024-09-21T11:51:11.718194+0000","last_clean":"2024-09-21T11:51:11.718194+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:51:11.718194+0000","last_undegraded":"2024-09-21T11:51:11.718194+0000","last_fullsized":"2024-09-21T11:51:11.718194+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:44.985234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:44.985234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:51.744078+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":504,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702327+0000","last_change":"2024-09-21T11:51:09.660956+0000","last_active":"2024-09-21T11:51:11.702327+0000","last_peered":"2024-09-21T11:51:11.702327+0000","last_clean":"2024-09-21T11:51:11.702327+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:51:11.702327+0000","last_undegraded":"2024-09-21T11:51:11.702327+0000","last_fullsized":"2024-09-21T11:51:11.702327+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:09.660880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.660880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:28.267025+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"286'10569","reported_seq":10660,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:16.928853+0000","last_change":"2024-09-21T11:49:08.517117+0000","last_active":"2024-09-21T11:51:16.928853+0000","last_peered":"2024-09-21T11:51:16.928853+0000","last_clean":"2024-09-21T11:51:16.928853+0000","last_became_active":"2024-09-21T11:49:08.516494+0000","last_became_peered":"2024-09-21T11:49:08.516494+0000","last_unstale":"2024-09-21T11:51:16.928853+0000","last_undegraded":"2024-09-21T11:51:16.928853+0000","last_fullsized":"2024-09-21T11:51:16.928853+0000","mapping_epoch":247,"log_start":"262'4500","ondisk_log_start":"262'4500","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.866266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":366673920,"num_objects":5595,"num_object_clones":0,"num_object_copies":16785,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10569,"num_write_kb":517248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":525,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702028+0000","last_change":"2024-09-21T11:49:45.783231+0000","last_active":"2024-09-21T11:51:11.702028+0000","last_peered":"2024-09-21T11:51:11.702028+0000","last_clean":"2024-09-21T11:51:11.702028+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:51:11.702028+0000","last_undegraded":"2024-09-21T11:51:11.702028+0000","last_fullsized":"2024-09-21T11:51:11.702028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-21T11:49:45.783036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:45.783036+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:14.400285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":521,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.701626+0000","last_change":"2024-09-21T11:50:06.398255+0000","last_active":"2024-09-21T11:51:11.701626+0000","last_peered":"2024-09-21T11:51:11.701626+0000","last_clean":"2024-09-21T11:51:11.701626+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:51:11.701626+0000","last_undegraded":"2024-09-21T11:51:11.701626+0000","last_fullsized":"2024-09-21T11:51:11.701626+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:50:06.398195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.398195+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":630,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.700225+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:51:11.700225+0000","last_peered":"2024-09-21T11:51:11.700225+0000","last_clean":"2024-09-21T11:51:11.700225+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:51:11.700225+0000","last_undegraded":"2024-09-21T11:51:11.700225+0000","last_fullsized":"2024-09-21T11:51:11.700225+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:42.344108+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":482,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702105+0000","last_change":"2024-09-21T11:49:56.669610+0000","last_active":"2024-09-21T11:51:11.702105+0000","last_peered":"2024-09-21T11:51:11.702105+0000","last_clean":"2024-09-21T11:51:11.702105+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:51:11.702105+0000","last_undegraded":"2024-09-21T11:51:11.702105+0000","last_fullsized":"2024-09-21T11:51:11.702105+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:56.669547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:56.669547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:24.559358+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":509,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.707842+0000","last_change":"2024-09-21T11:50:31.417131+0000","last_active":"2024-09-21T11:51:11.707842+0000","last_peered":"2024-09-21T11:51:11.707842+0000","last_clean":"2024-09-21T11:51:11.707842+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:51:11.707842+0000","last_undegraded":"2024-09-21T11:51:11.707842+0000","last_fullsized":"2024-09-21T11:51:11.707842+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.416857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.416857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:35.529094+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":530,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713073+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:51:11.713073+0000","last_peered":"2024-09-21T11:51:11.713073+0000","last_clean":"2024-09-21T11:51:11.713073+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:51:11.713073+0000","last_undegraded":"2024-09-21T11:51:11.713073+0000","last_fullsized":"2024-09-21T11:51:11.713073+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.460407+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":515,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.703522+0000","last_change":"2024-09-21T11:50:58.694445+0000","last_active":"2024-09-21T11:51:11.703522+0000","last_peered":"2024-09-21T11:51:11.703522+0000","last_clean":"2024-09-21T11:51:11.703522+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:51:11.703522+0000","last_undegraded":"2024-09-21T11:51:11.703522+0000","last_fullsized":"2024-09-21T11:51:11.703522+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:58.694382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:58.694382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:07.087368+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":497,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.706279+0000","last_change":"2024-09-21T11:49:43.464860+0000","last_active":"2024-09-21T11:51:11.706279+0000","last_peered":"2024-09-21T11:51:11.706279+0000","last_clean":"2024-09-21T11:51:11.706279+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:51:11.706279+0000","last_undegraded":"2024-09-21T11:51:11.706279+0000","last_fullsized":"2024-09-21T11:51:11.706279+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:43.464792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:43.464792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":531,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713845+0000","last_change":"2024-09-21T11:51:09.955823+0000","last_active":"2024-09-21T11:51:11.713845+0000","last_peered":"2024-09-21T11:51:11.713845+0000","last_clean":"2024-09-21T11:51:11.713845+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:51:11.713845+0000","last_undegraded":"2024-09-21T11:51:11.713845+0000","last_fullsized":"2024-09-21T11:51:11.713845+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:09.955744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.955744+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:15.698416+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":493,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702550+0000","last_change":"2024-09-21T11:50:02.438498+0000","last_active":"2024-09-21T11:51:11.702550+0000","last_peered":"2024-09-21T11:51:11.702550+0000","last_clean":"2024-09-21T11:51:11.702550+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:51:11.702550+0000","last_undegraded":"2024-09-21T11:51:11.702550+0000","last_fullsized":"2024-09-21T11:51:11.702550+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.436591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.436591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:19.559044+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":508,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702670+0000","last_change":"2024-09-21T11:49:50.191966+0000","last_active":"2024-09-21T11:51:11.702670+0000","last_peered":"2024-09-21T11:51:11.702670+0000","last_clean":"2024-09-21T11:51:11.702670+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:51:11.702670+0000","last_undegraded":"2024-09-21T11:51:11.702670+0000","last_fullsized":"2024-09-21T11:51:11.702670+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":498,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.708753+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:51:11.708753+0000","last_peered":"2024-09-21T11:51:11.708753+0000","last_clean":"2024-09-21T11:51:11.708753+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:51:11.708753+0000","last_undegraded":"2024-09-21T11:51:11.708753+0000","last_fullsized":"2024-09-21T11:51:11.708753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:55.647487+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":506,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.705916+0000","last_change":"2024-09-21T11:46:24.193356+0000","last_active":"2024-09-21T11:51:11.705916+0000","last_peered":"2024-09-21T11:51:11.705916+0000","last_clean":"2024-09-21T11:51:11.705916+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:51:11.705916+0000","last_undegraded":"2024-09-21T11:51:11.705916+0000","last_fullsized":"2024-09-21T11:51:11.705916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:46:24.193225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:24.193225+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:41.642993+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":509,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713219+0000","last_change":"2024-09-21T11:50:58.890596+0000","last_active":"2024-09-21T11:51:11.713219+0000","last_peered":"2024-09-21T11:51:11.713219+0000","last_clean":"2024-09-21T11:51:11.713219+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:51:11.713219+0000","last_undegraded":"2024-09-21T11:51:11.713219+0000","last_fullsized":"2024-09-21T11:51:11.713219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:58.890528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:58.890528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:23.788125+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":622,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702913+0000","last_change":"2024-09-21T11:51:08.629172+0000","last_active":"2024-09-21T11:51:11.702913+0000","last_peered":"2024-09-21T11:51:11.702913+0000","last_clean":"2024-09-21T11:51:11.702913+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:51:11.702913+0000","last_undegraded":"2024-09-21T11:51:11.702913+0000","last_fullsized":"2024-09-21T11:51:11.702913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:08.628986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.628986+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:27.310184+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":490,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696296+0000","last_change":"2024-09-21T11:48:50.927661+0000","last_active":"2024-09-21T11:51:11.696296+0000","last_peered":"2024-09-21T11:51:11.696296+0000","last_clean":"2024-09-21T11:51:11.696296+0000","last_became_active":"2024-09-21T11:40:06.346415+0000","last_became_peered":"2024-09-21T11:40:06.346415+0000","last_unstale":"2024-09-21T11:51:11.696296+0000","last_undegraded":"2024-09-21T11:51:11.696296+0000","last_fullsized":"2024-09-21T11:51:11.696296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:50.927613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.927613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:20.385159+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":550,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702515+0000","last_change":"2024-09-21T11:50:37.433622+0000","last_active":"2024-09-21T11:51:11.702515+0000","last_peered":"2024-09-21T11:51:11.702515+0000","last_clean":"2024-09-21T11:51:11.702515+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:51:11.702515+0000","last_undegraded":"2024-09-21T11:51:11.702515+0000","last_fullsized":"2024-09-21T11:51:11.702515+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:50:37.432212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.432212+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:42.400352+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":493,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.697837+0000","last_change":"2024-09-21T11:49:58.400078+0000","last_active":"2024-09-21T11:51:11.697837+0000","last_peered":"2024-09-21T11:51:11.697837+0000","last_clean":"2024-09-21T11:51:11.697837+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:51:11.697837+0000","last_undegraded":"2024-09-21T11:51:11.697837+0000","last_fullsized":"2024-09-21T11:51:11.697837+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.400013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.400013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":660,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.704570+0000","last_change":"2024-09-21T11:50:30.703282+0000","last_active":"2024-09-21T11:51:11.704570+0000","last_peered":"2024-09-21T11:51:11.704570+0000","last_clean":"2024-09-21T11:51:11.704570+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:51:11.704570+0000","last_undegraded":"2024-09-21T11:51:11.704570+0000","last_fullsized":"2024-09-21T11:51:11.704570+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:50:30.703227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:30.703227+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:57.509405+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":497,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713412+0000","last_change":"2024-09-21T11:50:02.782565+0000","last_active":"2024-09-21T11:51:11.713412+0000","last_peered":"2024-09-21T11:51:11.713412+0000","last_clean":"2024-09-21T11:51:11.713412+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:51:11.713412+0000","last_undegraded":"2024-09-21T11:51:11.713412+0000","last_fullsized":"2024-09-21T11:51:11.713412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.782475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.782475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:29.369093+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":594,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696018+0000","last_change":"2024-09-21T11:50:35.363920+0000","last_active":"2024-09-21T11:51:11.696018+0000","last_peered":"2024-09-21T11:51:11.696018+0000","last_clean":"2024-09-21T11:51:11.696018+0000","last_became_active":"2024-09-21T11:48:33.022647+0000","last_became_peered":"2024-09-21T11:48:33.022647+0000","last_unstale":"2024-09-21T11:51:11.696018+0000","last_undegraded":"2024-09-21T11:51:11.696018+0000","last_fullsized":"2024-09-21T11:51:11.696018+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.878372+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":486,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696710+0000","last_change":"2024-09-21T11:45:36.642466+0000","last_active":"2024-09-21T11:51:11.696710+0000","last_peered":"2024-09-21T11:51:11.696710+0000","last_clean":"2024-09-21T11:51:11.696710+0000","last_became_active":"2024-09-21T11:40:00.317484+0000","last_became_peered":"2024-09-21T11:40:00.317484+0000","last_unstale":"2024-09-21T11:51:11.696710+0000","last_undegraded":"2024-09-21T11:51:11.696710+0000","last_fullsized":"2024-09-21T11:51:11.696710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:42.266277+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":195,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.699676+0000","last_change":"2024-09-21T11:51:10.143792+0000","last_active":"2024-09-21T11:51:11.699676+0000","last_peered":"2024-09-21T11:51:11.699676+0000","last_clean":"2024-09-21T11:51:11.699676+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:51:11.699676+0000","last_undegraded":"2024-09-21T11:51:11.699676+0000","last_fullsized":"2024-09-21T11:51:11.699676+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.141936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.141936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:18.171087+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":474,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.704043+0000","last_change":"2024-09-21T11:50:02.329020+0000","last_active":"2024-09-21T11:51:11.704043+0000","last_peered":"2024-09-21T11:51:11.704043+0000","last_clean":"2024-09-21T11:51:11.704043+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:51:11.704043+0000","last_undegraded":"2024-09-21T11:51:11.704043+0000","last_fullsized":"2024-09-21T11:51:11.704043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.328926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.328926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:12.336948+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":206,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:15.200506+0000","last_change":"2024-09-21T11:51:15.200506+0000","last_active":"2024-09-21T11:51:15.200506+0000","last_peered":"2024-09-21T11:51:15.200506+0000","last_clean":"2024-09-21T11:51:15.200506+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:51:15.200506+0000","last_undegraded":"2024-09-21T11:51:15.200506+0000","last_fullsized":"2024-09-21T11:51:15.200506+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:15.200430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.200430+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:29.878537+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":501,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.707402+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:51:11.707402+0000","last_peered":"2024-09-21T11:51:11.707402+0000","last_clean":"2024-09-21T11:51:11.707402+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:51:11.707402+0000","last_undegraded":"2024-09-21T11:51:11.707402+0000","last_fullsized":"2024-09-21T11:51:11.707402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.443409+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":508,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702426+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:51:11.702426+0000","last_peered":"2024-09-21T11:51:11.702426+0000","last_clean":"2024-09-21T11:51:11.702426+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:51:11.702426+0000","last_undegraded":"2024-09-21T11:51:11.702426+0000","last_fullsized":"2024-09-21T11:51:11.702426+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:18.838895+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":513,"reported_epoch":286,"state":"active+clean+scrubbing","last_fresh":"2024-09-21T11:51:11.712935+0000","last_change":"2024-09-21T11:51:16.937924+0000","last_active":"2024-09-21T11:51:11.712935+0000","last_peered":"2024-09-21T11:51:11.712935+0000","last_clean":"2024-09-21T11:51:11.712935+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:51:11.712935+0000","last_undegraded":"2024-09-21T11:51:11.712935+0000","last_fullsized":"2024-09-21T11:51:11.712935+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:48:50.487786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:48:50.487786+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":521,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.704293+0000","last_change":"2024-09-21T11:49:07.148305+0000","last_active":"2024-09-21T11:51:11.704293+0000","last_peered":"2024-09-21T11:51:11.704293+0000","last_clean":"2024-09-21T11:51:11.704293+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:51:11.704293+0000","last_undegraded":"2024-09-21T11:51:11.704293+0000","last_fullsized":"2024-09-21T11:51:11.704293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-21T11:49:07.148241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.148241+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:19.446103+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":515,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.703568+0000","last_change":"2024-09-21T11:51:01.690172+0000","last_active":"2024-09-21T11:51:11.703568+0000","last_peered":"2024-09-21T11:51:11.703568+0000","last_clean":"2024-09-21T11:51:11.703568+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:51:11.703568+0000","last_undegraded":"2024-09-21T11:51:11.703568+0000","last_fullsized":"2024-09-21T11:51:11.703568+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.686885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.686885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:08.926819+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":485,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.696973+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:51:11.696973+0000","last_peered":"2024-09-21T11:51:11.696973+0000","last_clean":"2024-09-21T11:51:11.696973+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:51:11.696973+0000","last_undegraded":"2024-09-21T11:51:11.696973+0000","last_fullsized":"2024-09-21T11:51:11.696973+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:39.200896+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":490,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.702596+0000","last_change":"2024-09-21T11:51:05.711964+0000","last_active":"2024-09-21T11:51:11.702596+0000","last_peered":"2024-09-21T11:51:11.702596+0000","last_clean":"2024-09-21T11:51:11.702596+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:51:11.702596+0000","last_undegraded":"2024-09-21T11:51:11.702596+0000","last_fullsized":"2024-09-21T11:51:11.702596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:05.711768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.711768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:19.424825+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":541,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.701539+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:51:11.701539+0000","last_peered":"2024-09-21T11:51:11.701539+0000","last_clean":"2024-09-21T11:51:11.701539+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:51:11.701539+0000","last_undegraded":"2024-09-21T11:51:11.701539+0000","last_fullsized":"2024-09-21T11:51:11.701539+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:46.491919+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":493,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.714297+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:51:11.714297+0000","last_peered":"2024-09-21T11:51:11.714297+0000","last_clean":"2024-09-21T11:51:11.714297+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:51:11.714297+0000","last_undegraded":"2024-09-21T11:51:11.714297+0000","last_fullsized":"2024-09-21T11:51:11.714297+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:34.752247+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":493,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.707564+0000","last_change":"2024-09-21T11:50:38.405183+0000","last_active":"2024-09-21T11:51:11.707564+0000","last_peered":"2024-09-21T11:51:11.707564+0000","last_clean":"2024-09-21T11:51:11.707564+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:51:11.707564+0000","last_undegraded":"2024-09-21T11:51:11.707564+0000","last_fullsized":"2024-09-21T11:51:11.707564+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:38.405118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.405118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:51.232325+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":481,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.698740+0000","last_change":"2024-09-21T11:50:05.419390+0000","last_active":"2024-09-21T11:51:11.698740+0000","last_peered":"2024-09-21T11:51:11.698740+0000","last_clean":"2024-09-21T11:51:11.698740+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:51:11.698740+0000","last_undegraded":"2024-09-21T11:51:11.698740+0000","last_fullsized":"2024-09-21T11:51:11.698740+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.419329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.419329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:15.874052+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":195,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.699044+0000","last_change":"2024-09-21T11:50:25.045141+0000","last_active":"2024-09-21T11:51:11.699044+0000","last_peered":"2024-09-21T11:51:11.699044+0000","last_clean":"2024-09-21T11:51:11.699044+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:51:11.699044+0000","last_undegraded":"2024-09-21T11:51:11.699044+0000","last_fullsized":"2024-09-21T11:51:11.699044+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.045074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.045074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.164898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":516,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.701879+0000","last_change":"2024-09-21T11:51:04.656939+0000","last_active":"2024-09-21T11:51:11.701879+0000","last_peered":"2024-09-21T11:51:11.701879+0000","last_clean":"2024-09-21T11:51:11.701879+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:51:11.701879+0000","last_undegraded":"2024-09-21T11:51:11.701879+0000","last_fullsized":"2024-09-21T11:51:11.701879+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:04.656870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:04.656870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:13.464686+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":583,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:15.333035+0000","last_change":"2024-09-21T11:51:15.333035+0000","last_active":"2024-09-21T11:51:15.333035+0000","last_peered":"2024-09-21T11:51:15.333035+0000","last_clean":"2024-09-21T11:51:15.333035+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:51:15.333035+0000","last_undegraded":"2024-09-21T11:51:15.333035+0000","last_fullsized":"2024-09-21T11:51:15.333035+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:51:15.332952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.332952+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:19.341819+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":502,"reported_epoch":286,"state":"active+clean","last_fresh":"2024-09-21T11:51:11.713562+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:51:11.713562+0000","last_peered":"2024-09-21T11:51:11.713562+0000","last_clean":"2024-09-21T11:51:11.713562+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:51:11.713562+0000","last_undegraded":"2024-09-21T11:51:11.713562+0000","last_fullsized":"2024-09-21T11:51:11.713562+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:17.689301+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":5846532120,"num_objects":89212,"num_object_clones":0,"num_object_copies":267636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164916,"num_write_kb":8132033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17577947136,"data_stored":17577934920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83116,"ondisk_log_size":83116,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1122,"num_read_kb":879,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":667,"num_read_kb":662,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234124,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3733228,"kb_used_data":3268892,"kb_used_omap":1,"kb_used_meta":464318,"kb_avail":90011924,"statfs":{"total":95995035648,"available":92172210176,"internally_reserved":0,"allocated":3347345408,"data_stored":3346977514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":475462090},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463057,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1434716,"kb_used_data":1088396,"kb_used_omap":1,"kb_used_meta":346302,"kb_avail":92310436,"statfs":{"total":95995035648,"available":94525886464,"internally_reserved":0,"allocated":1114517504,"data_stored":1114166314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":354613702},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724694,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3721136,"kb_used_data":3266528,"kb_used_omap":1,"kb_used_meta":454590,"kb_avail":90024016,"statfs":{"total":95995035648,"available":92184592384,"internally_reserved":0,"allocated":3344924672,"data_stored":3344551893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":465500614},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920924,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1392560,"kb_used_data":1082336,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":92352592,"statfs":{"total":95995035648,"available":94569054208,"internally_reserved":0,"allocated":1108312064,"data_stored":1107940019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":317651397},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117154,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3375456,"kb_used_data":2843024,"kb_used_omap":1,"kb_used_meta":532350,"kb_avail":90369696,"statfs":{"total":95995035648,"available":92538568704,"internally_reserved":0,"allocated":2911256576,"data_stored":2910899740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":545126855},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411496,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1551972,"kb_used_data":1075156,"kb_used_omap":1,"kb_used_meta":476798,"kb_avail":92193180,"statfs":{"total":95995035648,"available":94405816320,"internally_reserved":0,"allocated":1100959744,"data_stored":1100599523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":488241605},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575023,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3684104,"kb_used_data":3219512,"kb_used_omap":1,"kb_used_meta":464318,"kb_avail":90061048,"statfs":{"total":95995035648,"available":92222513152,"internally_reserved":0,"allocated":3296780288,"data_stored":3296383777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":475462086},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738549,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1820724,"kb_used_data":1428580,"kb_used_omap":1,"kb_used_meta":392126,"kb_avail":91924428,"statfs":{"total":95995035648,"available":94130614272,"internally_reserved":0,"allocated":1462865920,"data_stored":1462490525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":401537477},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1461452800,"data_stored":1461452800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3292467200,"data_stored":3292463128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1092354048,"data_stored":1092354048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2890596352,"data_stored":2890596352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1107034112,"data_stored":1107034112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3316912128,"data_stored":3316908056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1102905344,"data_stored":1102905344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3314225152,"data_stored":3314221080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:51:22.583 INFO:tasks.thrashosds.thrasher:Setting 6.1c to [0, 7, 2] 2024-09-21T11:51:22.583 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '6.1c', '0', '7', '2'] 2024-09-21T11:51:22.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pg-upmap 6.1c 0 7 2 2024-09-21T11:51:22.598 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:22.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:22.599 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:22.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:22.744 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:22.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:22.745 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:22 smithi175 ceph-mon[32613]: 5.0 scrub starts 2024-09-21T11:51:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:22 smithi175 ceph-mon[32613]: 5.0 scrub ok 2024-09-21T11:51:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2095740137' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:51:22.846 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:22.877 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:22.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:22.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:22.979 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:23.007 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:23.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:23.008 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:23.108 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:23.136 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:23.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:23.137 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:23.238 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:23.265 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:23.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:23.267 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:23.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:23.405 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:23.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:23.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:23 smithi043 ceph-mon[36454]: from='client.25733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:51:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:23 smithi043 ceph-mon[36454]: pgmap v542: 148 pgs: 148 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 994 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-21T11:51:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:23 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2095740137' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:51:23.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:23 smithi043 ceph-mon[36454]: osdmap e287: 8 total, 8 up, 8 in 2024-09-21T11:51:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:23 smithi043 ceph-mon[29710]: from='client.25733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:51:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:23 smithi043 ceph-mon[29710]: pgmap v542: 148 pgs: 148 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 994 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-21T11:51:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:23 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2095740137' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:51:23.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:23 smithi043 ceph-mon[29710]: osdmap e287: 8 total, 8 up, 8 in 2024-09-21T11:51:23.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:23.542 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:23.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:23.543 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:23.644 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:23.671 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:23.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:23.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:23.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:23 smithi175 ceph-mon[32613]: from='client.25733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:51:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:23 smithi175 ceph-mon[32613]: pgmap v542: 148 pgs: 148 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 994 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-21T11:51:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:23 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2095740137' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:51:23.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:23 smithi175 ceph-mon[32613]: osdmap e287: 8 total, 8 up, 8 in 2024-09-21T11:51:23.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:23.805 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:23.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:23.806 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:23.907 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:23.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:23.935 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:23.935 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:23.940 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:51:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:51:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:51:23.940 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:51:23.940 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:51:24.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:24.066 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:24.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:24.067 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:24.168 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:24.193 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:24.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:24.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:24.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:24.341 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:24.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:24.343 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:24.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:24.495 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:24.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:24.496 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:24.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:24.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:24.636 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:24.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:24.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:24.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:24.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:24.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:24.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:25.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:25.045 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:25.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:25.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:25.146 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:25.182 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:25.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:25.183 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:25.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:51:25.283 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:25.312 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:25.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:25.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:25.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:25.459 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:25.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:25.460 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:51:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:51:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:51:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:25 smithi043 ceph-mon[36454]: pgmap v544: 148 pgs: 148 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:51:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:25 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1635991902' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:25 smithi043 ceph-mon[29710]: pgmap v544: 148 pgs: 148 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:51:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:25 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1635991902' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:25.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:25.595 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:25.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:25.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:25.697 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:25.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:25.735 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:25.736 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:25 smithi175 ceph-mon[32613]: pgmap v544: 148 pgs: 148 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:51:25.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:25 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1635991902' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:25.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:25.880 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:25.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:25.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:25.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:26.020 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:26.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:26.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:26.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:26.143 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:26.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:26.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:26.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:26.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:26.285 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:26.285 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:26.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:26.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:26.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:26.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:26.561 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:26.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:26.561 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:26.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:26.684 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:26.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:26.685 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:26 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:51:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:26 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:51:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:26 smithi043 ceph-mon[36454]: osdmap e288: 8 total, 8 up, 8 in 2024-09-21T11:51:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:26 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:51:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:26 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:51:26.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:26 smithi043 ceph-mon[29710]: osdmap e288: 8 total, 8 up, 8 in 2024-09-21T11:51:26.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:26 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:51:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:26 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:51:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:26 smithi175 ceph-mon[32613]: osdmap e288: 8 total, 8 up, 8 in 2024-09-21T11:51:26.786 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:26.817 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:26.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:26.818 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:26.918 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:26.959 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:26.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:26.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:27.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:27.092 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:27.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:27.093 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:27.194 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:27.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:27.228 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:27.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:27.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:27.358 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:27.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:27.359 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:27.460 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:27.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:51:27.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:27.506 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:27.506 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:51:27.591 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:51:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:51:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:51:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:51:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:51:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:51:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:51:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:51:27.592 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:51:27.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:27.637 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:27.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:27.639 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:27.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:27.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:27 smithi043 ceph-mon[29710]: pgmap v546: 148 pgs: 148 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 677 op/s 2024-09-21T11:51:27.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:27 smithi043 ceph-mon[36454]: pgmap v546: 148 pgs: 148 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 677 op/s 2024-09-21T11:51:27.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:27 smithi175 ceph-mon[32613]: pgmap v546: 148 pgs: 148 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 677 op/s 2024-09-21T11:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:27.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:27.785 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:27.820 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:27.886 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:27.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:27.917 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:27.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:28.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:28.056 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:28.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:28.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:28.158 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:28.187 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:28.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:28.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:28.289 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:28.311 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:28.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:28.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:28.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:28.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:28.450 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:28.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:28.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:28.588 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:28.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:28.589 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:28.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:28.727 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:28.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:28.728 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:28.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:28.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:28.871 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:28.879 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:28.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:29.005 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:29.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:29.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:29.107 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:29.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:29.141 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:29.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:29.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:29.284 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:29.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:29.285 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:29.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:29.429 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:29.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:29.430 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:29.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:29.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:29.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:29.661 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:29 smithi043 ceph-mon[29710]: pgmap v547: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-21T11:51:29.661 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:29 smithi043 ceph-mon[36454]: pgmap v547: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-21T11:51:29.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:29.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:29.725 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:29.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:29 smithi175 ceph-mon[32613]: pgmap v547: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-21T11:51:29.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:29.880 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:29.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:29.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:29.982 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:30.015 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:30.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:30.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:30.102 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:30.117 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:30.141 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:30.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:30.142 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:30.243 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:30.275 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:30.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:30.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:30.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:30.418 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:30.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:30.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:set 6.1c pg_upmap mapping to [0,7,2] 2024-09-21T11:51:30.520 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:30.548 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:30.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:30.549 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:30.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:30.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:30.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:30.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:30 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3106498465' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.1c", "id": [0, 7, 2]} : dispatch 2024-09-21T11:51:30.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:30 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3106498465' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.1c", "id": [0, 7, 2]} : dispatch 2024-09-21T11:51:30.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:30 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3106498465' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.1c", "id": [0, 7, 2]} : dispatch 2024-09-21T11:51:30.795 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:30.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:30.834 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:30.834 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:30.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:51:30.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:30.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:30.999 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:30.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:31.002 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:51:31.002 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:51:31.100 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:31.130 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:31.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:31.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:31.232 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:31.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:31.264 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:31.264 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:31.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:31.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:31.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:31.460 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:51:31.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:31.547 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:31.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:31.548 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:31.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:31.684 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:31.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:31.685 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[29710]: pgmap v548: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 939 op/s 2024-09-21T11:51:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3106498465' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.1c", "id": [0, 7, 2]}]': finished 2024-09-21T11:51:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[29710]: osdmap e289: 8 total, 8 up, 8 in 2024-09-21T11:51:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1732729249' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:51:31.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[36454]: pgmap v548: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 939 op/s 2024-09-21T11:51:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3106498465' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.1c", "id": [0, 7, 2]}]': finished 2024-09-21T11:51:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[36454]: osdmap e289: 8 total, 8 up, 8 in 2024-09-21T11:51:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1732729249' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:51:31.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:31.760 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:51:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:51:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:31 smithi175 ceph-mon[32613]: pgmap v548: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 939 op/s 2024-09-21T11:51:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3106498465' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.1c", "id": [0, 7, 2]}]': finished 2024-09-21T11:51:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:31 smithi175 ceph-mon[32613]: osdmap e289: 8 total, 8 up, 8 in 2024-09-21T11:51:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1732729249' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:31 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:51:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:31.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:31.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:31.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:31.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:31.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:31.954 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:31.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:32.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:32.096 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:32.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:32.097 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:32.198 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:32.229 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:32.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:32.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:32.331 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:32.366 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:32.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:32.367 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:32.467 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:32.497 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:32.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:32.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:32.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:51:32.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:32.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:32.645 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:32.645 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:32.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:32.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:32 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:32.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:32 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:51:32.704 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:32 smithi043 ceph-mon[29710]: osdmap e290: 8 total, 8 up, 8 in 2024-09-21T11:51:32.704 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:32.704 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:32 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:32.704 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:32 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:51:32.704 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:32 smithi043 ceph-mon[36454]: osdmap e290: 8 total, 8 up, 8 in 2024-09-21T11:51:32.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:32.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:32 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:32 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:51:32.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:32 smithi175 ceph-mon[32613]: osdmap e290: 8 total, 8 up, 8 in 2024-09-21T11:51:32.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:32.787 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:32.787 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:32.888 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:32.919 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:32.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:32.920 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:33.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:33.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:33.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:33.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:33.181 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:33.187 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:33.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:33.188 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:33.289 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:33.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:33.333 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:33.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:33.434 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:33.466 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:33.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:33.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:33.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:33.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:33.615 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:33.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:33 smithi043 ceph-mon[29710]: pgmap v551: 148 pgs: 148 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.60k op/s; 60 B/s, 0 objects/s recovering 2024-09-21T11:51:33.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:33 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:51:33.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:33 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:51:33.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:33 smithi043 ceph-mon[29710]: osdmap e291: 8 total, 8 up, 8 in 2024-09-21T11:51:33.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:33 smithi043 ceph-mon[36454]: pgmap v551: 148 pgs: 148 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.60k op/s; 60 B/s, 0 objects/s recovering 2024-09-21T11:51:33.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:33 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:51:33.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:33 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:51:33.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:33 smithi043 ceph-mon[36454]: osdmap e291: 8 total, 8 up, 8 in 2024-09-21T11:51:33.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:33.745 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:33.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:33.747 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:33.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:33 smithi175 ceph-mon[32613]: pgmap v551: 148 pgs: 148 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.60k op/s; 60 B/s, 0 objects/s recovering 2024-09-21T11:51:33.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:33 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:51:33.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:33 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:51:33.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:33 smithi175 ceph-mon[32613]: osdmap e291: 8 total, 8 up, 8 in 2024-09-21T11:51:33.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:33.888 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:33.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:33.890 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:33.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:34.011 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:34.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:34.012 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:34.113 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:34.145 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:34.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:34.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:34.227 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:51:34.227 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:51:34.227 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:51:34.227 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:51:34.227 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:51:34.227 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:51:34.227 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:51:34.227 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:51:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:51:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:51:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:51:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:51:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:51:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:51:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:51:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:51:34.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:34.283 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:34.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:34.284 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:34.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:34.531 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:34.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:34.532 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:34.632 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:34.668 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:34.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:34.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:34.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1443319845' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:34.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1443319845' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:34.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1443319845' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:34.770 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:34.801 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:34.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:34.802 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:34.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:34.923 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:34.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:34.924 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:35.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:35.062 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:35.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:35.063 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:35.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:35.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:51:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:35.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:35.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:35.303 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:35.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:35.338 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:35.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:35.482 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:35.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:35.483 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:35 smithi043 ceph-mon[29710]: pgmap v553: 148 pgs: 148 active+clean; 4.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 999 op/s; 67 B/s, 0 objects/s recovering 2024-09-21T11:51:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:51:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:51:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:51:35.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:35 smithi043 ceph-mon[36454]: pgmap v553: 148 pgs: 148 active+clean; 4.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 999 op/s; 67 B/s, 0 objects/s recovering 2024-09-21T11:51:35.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:35.606 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:35.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:35.607 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:35.708 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:35.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:35.742 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:35.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:35.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:35 smithi175 ceph-mon[32613]: pgmap v553: 148 pgs: 148 active+clean; 4.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 999 op/s; 67 B/s, 0 objects/s recovering 2024-09-21T11:51:35.844 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:35.877 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:35.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:35.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:35.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:36.015 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:36.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:36.018 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:36.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:36.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:36.168 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:36.268 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:36.299 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:36.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:36.300 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:36.400 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:36.421 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:36.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:36.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:36.523 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:36.554 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:36.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:36.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:36.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:36.689 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:36.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:36.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:36.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:36.806 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:51:36.806 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:51:36.806 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T11:51:36.806 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.000000 2024-09-21T11:51:36.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 4 0 2024-09-21T11:51:36.823 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:36.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:36.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:36.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:36.976 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:36.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:36.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:37.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:37.107 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:37.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:37.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:37.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:37.286 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:37.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:37.287 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:37.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:37.423 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:37.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:37.424 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:37.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:37.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:37.556 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:37.556 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:37.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:37 smithi043 ceph-mon[29710]: pgmap v554: 148 pgs: 148 active+clean; 4.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 999 op/s; 67 B/s, 0 objects/s recovering 2024-09-21T11:51:37.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:37 smithi043 ceph-mon[36454]: pgmap v554: 148 pgs: 148 active+clean; 4.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 999 op/s; 67 B/s, 0 objects/s recovering 2024-09-21T11:51:37.626 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:37.657 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:37.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:37.707 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:37.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:37.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:37 smithi175 ceph-mon[32613]: pgmap v554: 148 pgs: 148 active+clean; 4.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 999 op/s; 67 B/s, 0 objects/s recovering 2024-09-21T11:51:37.790 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:51:37.790 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:51:37.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.790 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:189870 9.8 9:162c791d:::benchmark_data_smithi043_126002_object62804:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.772985+0000", 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012373535, 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012405354, 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189870 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.791 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.772985+0000", 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.772985+0000", 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.772986+0000", 2024-09-21T11:51:37.792 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.314e-06 2024-09-21T11:51:37.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.772991+0000", 2024-09-21T11:51:37.793 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4569999999999998e-06 2024-09-21T11:51:37.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.793 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.793 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773001+0000", 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0380999999999999e-05 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773009+0000", 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5329999999999997e-06 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773048+0000", 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8868000000000001e-05 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:51:37.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773171+0000", 2024-09-21T11:51:37.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012299899999999999 2024-09-21T11:51:37.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:37.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773311+0000", 2024-09-21T11:51:37.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014082499999999999 2024-09-21T11:51:37.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.776800+0000", 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034887820000000002 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:189871 9.8 9:15cb6d39:::benchmark_data_smithi043_126002_object62805:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.773022+0000", 2024-09-21T11:51:37.796 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012336260999999999, 2024-09-21T11:51:37.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012441366000000001, 2024-09-21T11:51:37.797 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:51:37.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:51:37.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:51:37.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:51:37.797 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189871 2024-09-21T11:51:37.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.797 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773022+0000", 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773022+0000", 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773023+0000", 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4600000000000005e-07 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773025+0000", 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6680000000000001e-06 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773034+0000", 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.952e-06 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.773042+0000", 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0590000000000003e-06 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.776828+0000", 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003785844 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.776927+0000", 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8837000000000003e-05 2024-09-21T11:51:37.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.777030+0000", 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103226 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:189875 9.8 9:18d44d51:::benchmark_data_smithi043_126002_object62809:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.774060+0000", 2024-09-21T11:51:37.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011298551, 2024-09-21T11:51:37.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011465476, 2024-09-21T11:51:37.802 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:51:37.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:51:37.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:51:37.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:51:37.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189875 2024-09-21T11:51:37.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.803 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.774060+0000", 2024-09-21T11:51:37.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.803 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.803 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.774060+0000", 2024-09-21T11:51:37.803 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.774060+0000", 2024-09-21T11:51:37.804 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4700000000000002e-07 2024-09-21T11:51:37.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.804 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.804 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.774063+0000", 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6819999999999999e-06 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.774070+0000", 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5740000000000001e-06 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.774075+0000", 2024-09-21T11:51:37.805 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0499999999999999e-06 2024-09-21T11:51:37.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:51:37.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.782060+0000", 2024-09-21T11:51:37.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079850330000000008 2024-09-21T11:51:37.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.806 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:51:37.806 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.782167+0000", 2024-09-21T11:51:37.806 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106989 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.782278+0000", 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111637 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.807 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:189877 9.8 9.3184eb8 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:51:37.808 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.775320+0000", 2024-09-21T11:51:37.808 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010038371000000001, 2024-09-21T11:51:37.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010263414, 2024-09-21T11:51:37.808 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.808 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.808 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:51:37.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:51:37.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:51:37.808 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:51:37.808 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189877 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.775320+0000", 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.775320+0000", 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.775321+0000", 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9000000000000005e-07 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.775325+0000", 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3340000000000003e-06 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.775340+0000", 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4199000000000001e-05 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.775347+0000", 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9909999999999996e-06 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:189881 9.c 9:3e6e8398:::benchmark_data_smithi043_126002_object62815:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.779835+0000", 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.005523048, 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057968710000000003, 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189881 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779835+0000", 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779835+0000", 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779837+0000", 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4759999999999999e-06 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779842+0000", 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8980000000000004e-06 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779853+0000", 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0812000000000001e-05 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779861+0000", 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0900000000000005e-06 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.780013+0000", 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015197600000000001 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.780111+0000", 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8770000000000005e-05 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,7", 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.780211+0000", 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9234999999999994e-05 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.816 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:189882 9.c 9:3d0a4598:::benchmark_data_smithi043_126002_object62816:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:51:37.816 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.779880+0000", 2024-09-21T11:51:37.816 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0054783219999999999, 2024-09-21T11:51:37.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058136660000000003, 2024-09-21T11:51:37.817 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.817 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.817 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:51:37.817 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:51:37.817 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:51:37.817 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189882 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779880+0000", 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779880+0000", 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779881+0000", 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4000000000000002e-07 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779883+0000", 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6970000000000002e-06 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779890+0000", 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2720000000000003e-06 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779895+0000", 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3489999999999999e-06 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783224+0000", 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033293110000000002 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783334+0000", 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010919 2024-09-21T11:51:37.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 4,7", 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783437+0000", 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102973 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:189871 9.8 e291/247)", 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.779913+0000", 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0054454359999999997, 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058352359999999997, 2024-09-21T11:51:37.821 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779913+0000", 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779913+0000", 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779914+0000", 2024-09-21T11:51:37.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4799999999999999e-07 2024-09-21T11:51:37.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:37.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779916+0000", 2024-09-21T11:51:37.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.323e-06 2024-09-21T11:51:37.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779922+0000", 2024-09-21T11:51:37.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9599999999999997e-06 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.779926+0000", 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5299999999999998e-06 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:189867 9.8 e291/247)", 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.781213+0000", 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0041456089999999998, 2024-09-21T11:51:37.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0045729209999999998, 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781213+0000", 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781213+0000", 2024-09-21T11:51:37.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781214+0000", 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9800000000000003e-07 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781218+0000", 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4830000000000001e-06 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781228+0000", 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6260000000000007e-06 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781235+0000", 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9650000000000002e-06 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.829 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:189870 9.8 e291/247)", 2024-09-21T11:51:37.829 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.781247+0000", 2024-09-21T11:51:37.829 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0041110369999999997, 2024-09-21T11:51:37.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0045736370000000002, 2024-09-21T11:51:37.829 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:51:37.829 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781247+0000", 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781247+0000", 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781248+0000", 2024-09-21T11:51:37.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2000000000000001e-07 2024-09-21T11:51:37.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781250+0000", 2024-09-21T11:51:37.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3810000000000002e-06 2024-09-21T11:51:37.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781266+0000", 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6263e-05 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781272+0000", 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0950000000000003e-06 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.833 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:189880 9.4 e291/247 9:249a2d35:::benchmark_data_smithi043_126002_object62814:head v 291'11731, mlcod=291'11729)", 2024-09-21T11:51:37.833 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.781952+0000", 2024-09-21T11:51:37.833 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0034069590000000002, 2024-09-21T11:51:37.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039097230000000004, 2024-09-21T11:51:37.833 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.833 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.833 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:51:37.833 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781952+0000", 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781952+0000", 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781952+0000", 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9100000000000003e-07 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781958+0000", 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0039999999999996e-06 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.835 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781969+0000", 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0277e-05 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.781975+0000", 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8369999999999999e-06 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:51:37.836 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.782732+0000", 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00075667200000000001 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.782751+0000", 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8519999999999999e-05 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.837 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.838 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:189887 9.2 9:47b064c8:::benchmark_data_smithi043_126002_object62821:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:51:37.838 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.783382+0000", 2024-09-21T11:51:37.838 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0019768939999999999, 2024-09-21T11:51:37.838 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025280530000000002, 2024-09-21T11:51:37.838 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.838 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.838 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:51:37.838 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:51:37.838 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:51:37.838 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:51:37.839 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189887 2024-09-21T11:51:37.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.839 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.839 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.839 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783382+0000", 2024-09-21T11:51:37.839 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.839 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783382+0000", 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783383+0000", 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.392e-06 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.840 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783388+0000", 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6970000000000001e-06 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783399+0000", 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1754e-05 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.841 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783407+0000", 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3000000000000004e-06 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783480+0000", 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3639000000000004e-05 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783631+0000", 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015067999999999999 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.783741+0000", 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011009 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:189875 9.8 e291/247)", 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.785133+0000", 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.000225701, 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00083131000000000004, 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.843 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785133+0000", 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785133+0000", 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.844 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785134+0000", 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6499999999999998e-07 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785138+0000", 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2549999999999999e-06 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785147+0000", 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9879999999999993e-06 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.845 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785153+0000", 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.251e-06 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:189890 9.c 9.6d90111c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:37.785784+0000", 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 4294967295.9995742, 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021804999999999999, 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:51:37.846 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 189890 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785784+0000", 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:51:37.847 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785784+0000", 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785786+0000", 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.311e-06 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785790+0000", 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4059999999999998e-06 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:51:37.848 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785801+0000", 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0923e-05 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:37.785808+0000", 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9210000000000003e-06 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 13 2024-09-21T11:51:37.849 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:51:37.887 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:37.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:37.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:37.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:38.018 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:38.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:38.019 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:38.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:38.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:38.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:38.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:38.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:38.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:38.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:38.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:38.444 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:38.444 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:38.544 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:38.579 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:38.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:38.580 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:38.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:38.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:38.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:38.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:38.862 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:38.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:38.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:38.964 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:38.994 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:38.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:38.995 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:39.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:51:39.096 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:39.099 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:51:39.121 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:39.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:39.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:39.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:39.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:39.342 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:39.342 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:39.386 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:39.443 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:39.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:39.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:39.470 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:39.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:39.596 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:39.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:39.597 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:39.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:39.738 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:39.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:39.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:39.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:39 smithi043 ceph-mon[29710]: pgmap v555: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.59k op/s; 53 B/s, 0 objects/s recovering 2024-09-21T11:51:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:39 smithi043 ceph-mon[36454]: pgmap v555: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.59k op/s; 53 B/s, 0 objects/s recovering 2024-09-21T11:51:39.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:39 smithi175 ceph-mon[32613]: pgmap v555: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.59k op/s; 53 B/s, 0 objects/s recovering 2024-09-21T11:51:39.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:39.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:39.874 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:39.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:39.975 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:40.011 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:40.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:40.012 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:40.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:40.149 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:40.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:40.150 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:40.251 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:40.285 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:40.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:40.286 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:40.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:40.424 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:40.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:40.425 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:40.502 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:51:40.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:40.571 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:40.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:40.572 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:40.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:40.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:40.718 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:40.718 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:40.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:40 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2095354422' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:51:40.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:40 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:51:40.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:40.860 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:40.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:40.861 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:40.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:40.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2095354422' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:51:40.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:40 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:51:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:40 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2095354422' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:51:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:40 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:51:40.999 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:41.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:41.000 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:41.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:41.141 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:41.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:41.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:41.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:41.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:41.284 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:41.284 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:41.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:41.412 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:41.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:41.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:41.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:41.545 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:41.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:41.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:41.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:41.675 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:41.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:41.676 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:41.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[36454]: pgmap v556: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 599 B/s rd, 0 B/s wr, 1.02k op/s; 47 B/s, 0 objects/s recovering 2024-09-21T11:51:41.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:41.678 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:51:41.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[36454]: osdmap e292: 8 total, 8 up, 8 in 2024-09-21T11:51:41.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[36454]: 4.7 scrub starts 2024-09-21T11:51:41.679 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[36454]: 4.7 scrub ok 2024-09-21T11:51:41.680 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[29710]: pgmap v556: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 599 B/s rd, 0 B/s wr, 1.02k op/s; 47 B/s, 0 objects/s recovering 2024-09-21T11:51:41.680 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:41.680 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:51:41.680 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[29710]: osdmap e292: 8 total, 8 up, 8 in 2024-09-21T11:51:41.680 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[29710]: 4.7 scrub starts 2024-09-21T11:51:41.680 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:41 smithi043 ceph-mon[29710]: 4.7 scrub ok 2024-09-21T11:51:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:41 smithi175 ceph-mon[32613]: pgmap v556: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 599 B/s rd, 0 B/s wr, 1.02k op/s; 47 B/s, 0 objects/s recovering 2024-09-21T11:51:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:41 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:41 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:51:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:41 smithi175 ceph-mon[32613]: osdmap e292: 8 total, 8 up, 8 in 2024-09-21T11:51:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:41 smithi175 ceph-mon[32613]: 4.7 scrub starts 2024-09-21T11:51:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:41 smithi175 ceph-mon[32613]: 4.7 scrub ok 2024-09-21T11:51:41.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:51:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:51:41.765 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:41.776 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:41.801 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:41.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:41.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:41.827 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:41.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:41.931 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:41.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:41.932 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:42.032 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:42.060 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:42.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:42.061 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:42.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:42.192 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:42.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:42.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:42.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:42.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:42.332 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:42.332 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:42.433 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:42.462 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:42.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:42.463 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:42.520 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.4 primary-affinity to 0 (802) 2024-09-21T11:51:42.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:42.587 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:42.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:42.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:42.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:42.723 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:42.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:42.724 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:42.725 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:51:42.725 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:51:42.762 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:51:42.763 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:51:42.803 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:51:42.804 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:51:42.804 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:42.804 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1321882583' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 0.0} : dispatch 2024-09-21T11:51:42.804 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:42 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 0.0} : dispatch 2024-09-21T11:51:42.804 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:42.804 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1321882583' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 0.0} : dispatch 2024-09-21T11:51:42.805 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:42 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 0.0} : dispatch 2024-09-21T11:51:42.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:42.840 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:51:42.840 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:51:42.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:42.855 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:42.856 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:42.872 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:51:42.873 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:51:42.908 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:51:42.908 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:51:42.946 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:51:42.947 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:51:42.956 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:42.981 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:51:42.981 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:51:42.984 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:42.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:42.985 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:43.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:43.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1321882583' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 0.0} : dispatch 2024-09-21T11:51:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:42 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 0.0} : dispatch 2024-09-21T11:51:43.086 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:43.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:43.124 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:43.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:43.261 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:43.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:43.263 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:43.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:43.401 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:43.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:43.402 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:43.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:51:43.502 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:51:43.502 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:51:43.502 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:51:43.502 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:51:43.503 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:51:43.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:43.550 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:43.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:43.551 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:43.652 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:43.681 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:43.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:43.682 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[36454]: pgmap v558: 148 pgs: 148 active+clean; 3.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.11k op/s 2024-09-21T11:51:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[36454]: 3.11 scrub starts 2024-09-21T11:51:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[36454]: 3.11 scrub ok 2024-09-21T11:51:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]': finished 2024-09-21T11:51:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[36454]: osdmap e293: 8 total, 8 up, 8 in 2024-09-21T11:51:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2080452143' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[36454]: osdmap e294: 8 total, 8 up, 8 in 2024-09-21T11:51:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[29710]: pgmap v558: 148 pgs: 148 active+clean; 3.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.11k op/s 2024-09-21T11:51:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[29710]: 3.11 scrub starts 2024-09-21T11:51:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[29710]: 3.11 scrub ok 2024-09-21T11:51:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]': finished 2024-09-21T11:51:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[29710]: osdmap e293: 8 total, 8 up, 8 in 2024-09-21T11:51:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2080452143' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:43 smithi043 ceph-mon[29710]: osdmap e294: 8 total, 8 up, 8 in 2024-09-21T11:51:43.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:43.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:43.819 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:43.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:43.919 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:43.950 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:43.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:43.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:43 smithi175 ceph-mon[32613]: pgmap v558: 148 pgs: 148 active+clean; 3.7 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.11k op/s 2024-09-21T11:51:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:43 smithi175 ceph-mon[32613]: 3.11 scrub starts 2024-09-21T11:51:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:43 smithi175 ceph-mon[32613]: 3.11 scrub ok 2024-09-21T11:51:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:43 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 0.0}]': finished 2024-09-21T11:51:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:43 smithi175 ceph-mon[32613]: osdmap e293: 8 total, 8 up, 8 in 2024-09-21T11:51:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:43 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2080452143' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:43 smithi175 ceph-mon[32613]: osdmap e294: 8 total, 8 up, 8 in 2024-09-21T11:51:44.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:44.089 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:44.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:44.090 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:44.178 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:51:44.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:44.285 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:44.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:44.286 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:44.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:44.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:44.424 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:44.424 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:44.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:44.557 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:44.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:44.558 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:44.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:44.693 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:44.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:44.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:44.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:44.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:44.835 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:44.936 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:44.967 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:44.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:44.968 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[36454]: 4.6 scrub starts 2024-09-21T11:51:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[36454]: 4.6 scrub ok 2024-09-21T11:51:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[36454]: 3.0 scrub starts 2024-09-21T11:51:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[36454]: 3.0 scrub ok 2024-09-21T11:51:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[36454]: 4.19 scrub starts 2024-09-21T11:51:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[36454]: 4.19 scrub ok 2024-09-21T11:51:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[29710]: 4.6 scrub starts 2024-09-21T11:51:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[29710]: 4.6 scrub ok 2024-09-21T11:51:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[29710]: 3.0 scrub starts 2024-09-21T11:51:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[29710]: 3.0 scrub ok 2024-09-21T11:51:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[29710]: 4.19 scrub starts 2024-09-21T11:51:44.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:44 smithi043 ceph-mon[29710]: 4.19 scrub ok 2024-09-21T11:51:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:44 smithi175 ceph-mon[32613]: 4.6 scrub starts 2024-09-21T11:51:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:44 smithi175 ceph-mon[32613]: 4.6 scrub ok 2024-09-21T11:51:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:44 smithi175 ceph-mon[32613]: 3.0 scrub starts 2024-09-21T11:51:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:44 smithi175 ceph-mon[32613]: 3.0 scrub ok 2024-09-21T11:51:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:44 smithi175 ceph-mon[32613]: 4.19 scrub starts 2024-09-21T11:51:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:44 smithi175 ceph-mon[32613]: 4.19 scrub ok 2024-09-21T11:51:45.069 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:45.097 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:45.098 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:45.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:45.099 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:45.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:45.244 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:45.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:45.245 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:45.292 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:51:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:51:45.345 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:45.375 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:45.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:45.376 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:45.477 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:45.515 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:45.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:45.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:45.617 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:45.649 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:45.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:45.650 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:45.721 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:45.750 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:45.781 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:45.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:45.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:45.862 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-mon[36454]: pgmap v561: 148 pgs: 148 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:51:45.862 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-mon[36454]: 3.3 scrub starts 2024-09-21T11:51:45.862 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-mon[36454]: 3.3 scrub ok 2024-09-21T11:51:45.862 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-mon[36454]: 5.1f scrub starts 2024-09-21T11:51:45.863 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-mon[36454]: 5.1f scrub ok 2024-09-21T11:51:45.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-mon[29710]: pgmap v561: 148 pgs: 148 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:51:45.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-mon[29710]: 3.3 scrub starts 2024-09-21T11:51:45.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-mon[29710]: 3.3 scrub ok 2024-09-21T11:51:45.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-mon[29710]: 5.1f scrub starts 2024-09-21T11:51:45.863 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:45 smithi043 ceph-mon[29710]: 5.1f scrub ok 2024-09-21T11:51:45.883 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:45.914 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:45.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:45.915 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:45 smithi175 ceph-mon[32613]: pgmap v561: 148 pgs: 148 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:51:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:45 smithi175 ceph-mon[32613]: 3.3 scrub starts 2024-09-21T11:51:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:45 smithi175 ceph-mon[32613]: 3.3 scrub ok 2024-09-21T11:51:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:45 smithi175 ceph-mon[32613]: 5.1f scrub starts 2024-09-21T11:51:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:45 smithi175 ceph-mon[32613]: 5.1f scrub ok 2024-09-21T11:51:46.015 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:46.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:46.043 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:46.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:46.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:46.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:46.185 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:46.186 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:46.287 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:46.321 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:46.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:46.323 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:51:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:51:46.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:46.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:46.458 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:46.458 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:46.559 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:51:46.591 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:46.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:46.592 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:46.693 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:46.725 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:46.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:46.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:46.827 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:46.862 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:46.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:46.863 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:46.882 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:46 smithi043 ceph-mon[36454]: 4.3 scrub starts 2024-09-21T11:51:46.882 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:46 smithi043 ceph-mon[36454]: 4.3 scrub ok 2024-09-21T11:51:46.882 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:46 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1079734384' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:46.883 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:46 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:46.883 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:46.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:46 smithi043 ceph-mon[29710]: 4.3 scrub starts 2024-09-21T11:51:46.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:46 smithi043 ceph-mon[29710]: 4.3 scrub ok 2024-09-21T11:51:46.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:46 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1079734384' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:46.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:46 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:46.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:46.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:47.000 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:47.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:47.001 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:47.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:46 smithi175 ceph-mon[32613]: 4.3 scrub starts 2024-09-21T11:51:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:46 smithi175 ceph-mon[32613]: 4.3 scrub ok 2024-09-21T11:51:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:46 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1079734384' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:46 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:51:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:51:47.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:51:47.102 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:47.122 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:47.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:47.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:47.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:47.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:47.271 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:47.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:47.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:47.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:47.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:47.579 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:47.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:47.580 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:47.682 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:47.710 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:47.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:47.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:47.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:51:47.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:47.856 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:47.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:47.857 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:47.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[29710]: pgmap v562: 148 pgs: 148 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:51:47.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:51:47.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:51:47.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:51:47.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[29710]: osdmap e295: 8 total, 8 up, 8 in 2024-09-21T11:51:47.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[29710]: 6.7 scrub starts 2024-09-21T11:51:47.913 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[29710]: 6.7 scrub ok 2024-09-21T11:51:47.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[36454]: pgmap v562: 148 pgs: 148 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:51:47.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:51:47.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:51:47.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:51:47.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[36454]: osdmap e295: 8 total, 8 up, 8 in 2024-09-21T11:51:47.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[36454]: 6.7 scrub starts 2024-09-21T11:51:47.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:47 smithi043 ceph-mon[36454]: 6.7 scrub ok 2024-09-21T11:51:47.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:48.000 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:48.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:48.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:48.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:47 smithi175 ceph-mon[32613]: pgmap v562: 148 pgs: 148 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:51:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:47 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:51:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:47 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:51:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:47 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:51:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:47 smithi175 ceph-mon[32613]: osdmap e295: 8 total, 8 up, 8 in 2024-09-21T11:51:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:47 smithi175 ceph-mon[32613]: 6.7 scrub starts 2024-09-21T11:51:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:47 smithi175 ceph-mon[32613]: 6.7 scrub ok 2024-09-21T11:51:48.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:48.147 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:48.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:48.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:48.168 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:51:48.249 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:48.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:48.284 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:48.284 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:48.378 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:48.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:48.418 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:51:48.419 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:51:48.419 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T11:51:48.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-21T11:51:48.419 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:48.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:48.420 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:48.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:48.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:48.553 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:48.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:48.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:48.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:48 smithi043 ceph-mon[29710]: 4.4 scrub starts 2024-09-21T11:51:48.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:48 smithi043 ceph-mon[29710]: 4.4 scrub ok 2024-09-21T11:51:48.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:48 smithi043 ceph-mon[29710]: 6.f scrub starts 2024-09-21T11:51:48.688 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:48 smithi043 ceph-mon[29710]: 6.f scrub ok 2024-09-21T11:51:48.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:48 smithi043 ceph-mon[36454]: 4.4 scrub starts 2024-09-21T11:51:48.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:48 smithi043 ceph-mon[36454]: 4.4 scrub ok 2024-09-21T11:51:48.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:48 smithi043 ceph-mon[36454]: 6.f scrub starts 2024-09-21T11:51:48.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:48 smithi043 ceph-mon[36454]: 6.f scrub ok 2024-09-21T11:51:48.696 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:48.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:48.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:48.782 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:51:48.782 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:51:48.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:48.835 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:48.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:48.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:48.937 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:48.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:48.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:48.975 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:48 smithi175 ceph-mon[32613]: 4.4 scrub starts 2024-09-21T11:51:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:48 smithi175 ceph-mon[32613]: 4.4 scrub ok 2024-09-21T11:51:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:48 smithi175 ceph-mon[32613]: 6.f scrub starts 2024-09-21T11:51:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:48 smithi175 ceph-mon[32613]: 6.f scrub ok 2024-09-21T11:51:49.030 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:51:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:51:49.064 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:51:49.065 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:51:49.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:49.104 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:51:49.104 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:51:49.107 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:49.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:49.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:49.133 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:49.147 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:51:49.147 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:51:49.178 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:51:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:51:49.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:49.215 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:51:49.215 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:51:49.254 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:51:49.254 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:51:49.257 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:49.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:49.258 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:49.287 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:51:49.287 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:51:49.359 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:49.398 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:49.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:49.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:49.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:49.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:49.544 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:49.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:49.645 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:49.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:49.686 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:49.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:49 smithi175 ceph-mon[32613]: pgmap v564: 148 pgs: 148 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 1.29k op/s 2024-09-21T11:51:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:49 smithi175 ceph-mon[32613]: 5.b scrub starts 2024-09-21T11:51:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:49 smithi175 ceph-mon[32613]: 5.b scrub ok 2024-09-21T11:51:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1002021611' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:49.787 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:49.823 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:49.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:49.824 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:49.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:49 smithi043 ceph-mon[29710]: pgmap v564: 148 pgs: 148 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 1.29k op/s 2024-09-21T11:51:49.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:49 smithi043 ceph-mon[29710]: 5.b scrub starts 2024-09-21T11:51:49.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:49 smithi043 ceph-mon[29710]: 5.b scrub ok 2024-09-21T11:51:49.852 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1002021611' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:49.852 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:49 smithi043 ceph-mon[36454]: pgmap v564: 148 pgs: 148 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 1.29k op/s 2024-09-21T11:51:49.852 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:49 smithi043 ceph-mon[36454]: 5.b scrub starts 2024-09-21T11:51:49.852 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:49 smithi043 ceph-mon[36454]: 5.b scrub ok 2024-09-21T11:51:49.853 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1002021611' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:51:49.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:49.959 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:49.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:49.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:49.966 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:51:50.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:50.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:50.101 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:50.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:50.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:50.236 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:50.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:50.237 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:50.252 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:51:50.252 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 295, 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "fsid": "4f2c5252-780d-11ef-baf6-efdc52797490", 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "created": "2024-09-21T11:34:07.249195+0000", 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "modified": "2024-09-21T11:51:46.551502+0000", 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "last_up_change": "2024-09-21T11:39:49.206441+0000", 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "last_in_change": "2024-09-21T11:47:02.337266+0000", 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_num": 5799936, 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_set": [ 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "pglog_hardlimit", 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "purged_snapdirs", 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_deletes", 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: "sortbitwise" 2024-09-21T11:51:50.253 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_version": 18, 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_snapshot": "", 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_max": 9, 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd": 8, 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "require_min_compat_client": "luminous", 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "min_compat_client": "luminous", 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "require_osd_release": "squid", 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_crimson": false, 2024-09-21T11:51:50.254 INFO:teuthology.orchestra.run.smithi043.stdout: "pools": [ 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 1, 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".mgr", 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:37:26.023430+0000", 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:50.255 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 1, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 1, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 1, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 1, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 1, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:50.256 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "253", 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:50.257 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:50.258 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:50.259 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_max": 32, 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_min": 1 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": {} 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 6.9800000190734863, 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 6.9800000190734863, 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.43000000715255737, 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 3, 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 3, 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T11:51:50.260 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 2, 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "datapool", 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:58.262982+0000", 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 8193, 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:50.261 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 3, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 3, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 3, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 3, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 3, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:50.262 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "61", 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 2, 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 61, 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:50.263 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:50.264 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:50.265 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd": {} 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.3299999237060547, 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.3299999237060547, 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1, 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T11:51:50.266 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 3, 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".rgw.root", 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:59.229530+0000", 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:50.268 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "60", 2024-09-21T11:51:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:50.269 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:50.270 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:50.271 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.5399999618530273, 2024-09-21T11:51:50.272 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.5399999618530273, 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.75, 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.75, 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 4, 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.log", 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:00.380059+0000", 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:50.273 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:50.274 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "62", 2024-09-21T11:51:50.275 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:50.276 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:50.277 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.278 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.190000057220459, 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.190000057220459, 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2.5, 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2.5, 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 5, 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.control", 2024-09-21T11:51:50.279 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:02.330041+0000", 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:50.280 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "64", 2024-09-21T11:51:50.281 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:50.282 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.3200000524520874, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.3200000524520874, 2024-09-21T11:51:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.5, 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.5, 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 6, 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.meta", 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:04.395413+0000", 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:51:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "66", 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_bias": 4 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.75, 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.75, 2024-09-21T11:51:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2, 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2, 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 9, 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "unique_pool_2", 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:49:06.645451+0000", 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:51:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 16, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 16, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 16, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 16, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 16, 2024-09-21T11:51:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "281", 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:51:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:51:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:51:50.295 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rados": {} 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.75, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.75, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.875, 2024-09-21T11:51:50.296 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4", 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:50.297 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 8, 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 293, 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:50.298 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6802", 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6804", 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:51:50.299 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6803", 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6801/817279193", 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6802/817279193", 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6804/817279193", 2024-09-21T11:51:50.300 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6803/817279193", 2024-09-21T11:51:50.301 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:50.301 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:50.301 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607", 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 13, 2024-09-21T11:51:50.302 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 293, 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6805", 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.303 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6806", 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6808", 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6807", 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6805/263756171", 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6806/263756171", 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6808/263756171", 2024-09-21T11:51:50.305 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6807/263756171", 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce", 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:50.306 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 18, 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 293, 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6809", 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.307 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6810", 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.308 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:50.309 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.309 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.309 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.309 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6812", 2024-09-21T11:51:50.309 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:51:50.309 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.309 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.309 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6811", 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6809/687577863", 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6810/687577863", 2024-09-21T11:51:50.310 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6812/687577863", 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6811/687577863", 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d", 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:50.311 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 27, 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 293, 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6813", 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.312 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6814", 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6816", 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:51:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6815", 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6813/129493158", 2024-09-21T11:51:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6814/129493158", 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6816/129493158", 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6815/129493158", 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c", 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 0, 2024-09-21T11:51:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 33, 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 235, 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6800", 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6801", 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6803", 2024-09-21T11:51:50.317 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6802", 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6800/1323992640", 2024-09-21T11:51:50.318 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6801/1323992640", 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6803/1323992640", 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6802/1323992640", 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "4d647321-71a1-412b-bbf5-921441923a65", 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:50.319 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 39, 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 293, 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6804", 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6805", 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.321 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6807", 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6806", 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6804/1163208458", 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6805/1163208458", 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6807/1163208458", 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6806/1163208458", 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354", 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:50.323 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 47, 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 293, 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6808", 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6809", 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6811", 2024-09-21T11:51:50.325 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6810", 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6808/932395625", 2024-09-21T11:51:50.326 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6809/932395625", 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6811/932395625", 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6810/932395625", 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22", 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:51:50.327 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 54, 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 293, 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6812", 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6813", 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6815", 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6814", 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6812/3676713275", 2024-09-21T11:51:50.330 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6813/3676713275", 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6815/3676713275", 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6814/3676713275", 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_xinfo": [ 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:50.331 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:50:53.391361+0000", 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:50:51.433218+0000", 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:50.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:50:57.338498+0000", 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:51:50.333 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:50:55.261169+0000", 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:51:09.401036+0000", 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:51:32.285902+0000", 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:51:25.018577+0000", 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:51:50.336 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:51:11.122943+0000", 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap": [ 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.c", 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: 3, 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: 7, 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: 1 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.1c", 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: 7, 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: 2 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_items": [ 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.4", 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:51:50.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.7", 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.d", 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.13", 2024-09-21T11:51:50.340 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:51:50.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:51:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:51:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.341 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.16", 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_primaries": [], 2024-09-21T11:51:50.342 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_temp": [], 2024-09-21T11:51:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_temp": [], 2024-09-21T11:51:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklist": { 2024-09-21T11:51:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2419845767": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:51:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3249046048": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:51:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2518306973": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:51:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3115176321": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:51:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/2583179759": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:51:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/4225358170": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:51:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/347827280": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:51:50.343 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/1513342881": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:51:50.345 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/1113684867": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:51:50.345 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3345448895": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:51:50.345 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2454875216": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:51:50.345 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/911630798": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:51:50.345 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/983028036": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/4145025121": "2024-09-22T11:40:29.670859+0000" 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "range_blocklist": {}, 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profiles": { 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "default": { 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-failure-domain": "osd", 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "k": "2", 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "m": "1", 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "plugin": "jerasure", 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "technique": "reed_sol_van" 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps_queue": [], 2024-09-21T11:51:50.346 INFO:teuthology.orchestra.run.smithi043.stdout: "new_removed_snaps": [], 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout: "new_purged_snaps": [], 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_node_flags": {}, 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout: "device_class_flags": {}, 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode": { 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_enabled": false, 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_bucket_count": 0, 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout: "degraded_stretch_mode": 0, 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout: "recovering_stretch_mode": 0, 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_bucket": 0 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:51:50.347 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:51:50.380 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:51:50.381 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:50.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:50.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:50.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:51:50.484 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:50.525 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:50.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:50.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:50.626 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:50.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:50.661 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:50.661 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:50.762 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:50.800 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:50.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:50.801 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:50.849 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:50 smithi175 ceph-mon[32613]: 6.6 scrub starts 2024-09-21T11:51:50.849 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:50 smithi175 ceph-mon[32613]: 6.6 scrub ok 2024-09-21T11:51:50.849 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:50 smithi175 ceph-mon[32613]: 4.15 scrub starts 2024-09-21T11:51:50.849 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:50 smithi175 ceph-mon[32613]: 4.15 scrub ok 2024-09-21T11:51:50.849 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2964482357' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:51:50.849 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:50 smithi175 ceph-mon[32613]: 6.e scrub starts 2024-09-21T11:51:50.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[29710]: 6.6 scrub starts 2024-09-21T11:51:50.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[29710]: 6.6 scrub ok 2024-09-21T11:51:50.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[29710]: 4.15 scrub starts 2024-09-21T11:51:50.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[29710]: 4.15 scrub ok 2024-09-21T11:51:50.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2964482357' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:51:50.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[29710]: 6.e scrub starts 2024-09-21T11:51:50.856 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[36454]: 6.6 scrub starts 2024-09-21T11:51:50.856 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[36454]: 6.6 scrub ok 2024-09-21T11:51:50.856 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[36454]: 4.15 scrub starts 2024-09-21T11:51:50.857 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[36454]: 4.15 scrub ok 2024-09-21T11:51:50.857 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2964482357' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:51:50.857 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:50 smithi043 ceph-mon[36454]: 6.e scrub starts 2024-09-21T11:51:50.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:50.942 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:50.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:50.943 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:51.004 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:51:51.005 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:51:51.005 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:51:51.005 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:51:51.005 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:51:51.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:51.079 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:51.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:51.080 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:51.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:51.215 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:51.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:51.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:51.269 INFO:tasks.thrashosds.thrasher:j is {'epoch': 295, 'fsid': '4f2c5252-780d-11ef-baf6-efdc52797490', 'created': '2024-09-21T11:34:07.249195+0000', 'modified': '2024-09-21T11:51:46.551502+0000', 'last_up_change': '2024-09-21T11:39:49.206441+0000', 'last_in_change': '2024-09-21T11:47:02.337266+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 9, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T11:37:26.023430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.980000019073486, 'score_stable': 6.980000019073486, 'optimal_score': 0.4300000071525574, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-21T11:39:58.262982+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 1, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-21T11:39:59.229530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-21T11:40:00.380059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.190000057220459, 'score_stable': 2.190000057220459, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-21T11:40:02.330041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-21T11:40:04.395413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 9, 'pool_name': 'unique_pool_2', 'create_time': '2024-09-21T11:49:06.645451+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '281', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '5229029d-4fd6-46b6-8325-a3e004df09b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6801', 'nonce': 817279193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 817279193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 817279193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6803', 'nonce': 817279193}]}, 'public_addr': '172.21.15.43:6801/817279193', 'cluster_addr': '172.21.15.43:6802/817279193', 'heartbeat_back_addr': '172.21.15.43:6804/817279193', 'heartbeat_front_addr': '172.21.15.43:6803/817279193', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a2a7b58b-ba99-4383-a6d3-3b19388b2607', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6805', 'nonce': 263756171}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 263756171}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 263756171}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6807', 'nonce': 263756171}]}, 'public_addr': '172.21.15.43:6805/263756171', 'cluster_addr': '172.21.15.43:6806/263756171', 'heartbeat_back_addr': '172.21.15.43:6808/263756171', 'heartbeat_front_addr': '172.21.15.43:6807/263756171', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c24a60b5-5713-492b-82ee-576ea7a743ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6809', 'nonce': 687577863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 687577863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 687577863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6811', 'nonce': 687577863}]}, 'public_addr': '172.21.15.43:6809/687577863', 'cluster_addr': '172.21.15.43:6810/687577863', 'heartbeat_back_addr': '172.21.15.43:6812/687577863', 'heartbeat_front_addr': '172.21.15.43:6811/687577863', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c5d43cf8-1c72-4d7d-ae41-e8136b47177d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6813', 'nonce': 129493158}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 129493158}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 129493158}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6815', 'nonce': 129493158}]}, 'public_addr': '172.21.15.43:6813/129493158', 'cluster_addr': '172.21.15.43:6814/129493158', 'heartbeat_back_addr': '172.21.15.43:6816/129493158', 'heartbeat_front_addr': '172.21.15.43:6815/129493158', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '79ee85c5-afdc-4d68-a253-89b7b0eb652c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 33, 'up_thru': 235, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 1323992640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6801', 'nonce': 1323992640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6803', 'nonce': 1323992640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 1323992640}]}, 'public_addr': '172.21.15.175:6800/1323992640', 'cluster_addr': '172.21.15.175:6801/1323992640', 'heartbeat_back_addr': '172.21.15.175:6803/1323992640', 'heartbeat_front_addr': '172.21.15.175:6802/1323992640', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4d647321-71a1-412b-bbf5-921441923a65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 1163208458}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6805', 'nonce': 1163208458}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6807', 'nonce': 1163208458}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 1163208458}]}, 'public_addr': '172.21.15.175:6804/1163208458', 'cluster_addr': '172.21.15.175:6805/1163208458', 'heartbeat_back_addr': '172.21.15.175:6807/1163208458', 'heartbeat_front_addr': '172.21.15.175:6806/1163208458', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3c33bea7-94b5-46c1-9905-81b4612a2354', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 47, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 932395625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6809', 'nonce': 932395625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6811', 'nonce': 932395625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 932395625}]}, 'public_addr': '172.21.15.175:6808/932395625', 'cluster_addr': '172.21.15.175:6809/932395625', 'heartbeat_back_addr': '172.21.15.175:6811/932395625', 'heartbeat_front_addr': '172.21.15.175:6810/932395625', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '41bc9817-d9d1-4c11-94ff-7de332af6d22', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 54, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 3676713275}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6813', 'nonce': 3676713275}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6815', 'nonce': 3676713275}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 3676713275}]}, 'public_addr': '172.21.15.175:6812/3676713275', 'cluster_addr': '172.21.15.175:6813/3676713275', 'heartbeat_back_addr': '172.21.15.175:6815/3676713275', 'heartbeat_front_addr': '172.21.15.175:6814/3676713275', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:50:53.391361+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:50:51.433218+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:50:57.338498+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:50:55.261169+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:51:09.401036+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:51:32.285902+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:51:25.018577+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:51:11.122943+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '6.c', 'osds': [3, 7, 1]}, {'pgid': '6.1c', 'osds': [0, 7, 2]}], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.7', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.16', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.43:0/2419845767': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3249046048': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/2518306973': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/3115176321': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:6800/2583179759': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/4225358170': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/347827280': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/1513342881': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/1113684867': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3345448895': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/2454875216': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/911630798': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/983028036': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/4145025121': '2024-09-22T11:40:29.670859+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T11:51:51.269 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:51:51.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:51.356 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:51.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:51.370 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:51.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:51.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:51.530 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:51.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:51.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:51.632 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:51.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:51.674 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:51.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:51.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:51:51.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:51 smithi175 ceph-mon[32613]: pgmap v565: 148 pgs: 148 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 829 op/s 2024-09-21T11:51:51.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:51 smithi175 ceph-mon[32613]: 6.9 scrub starts 2024-09-21T11:51:51.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:51 smithi175 ceph-mon[32613]: 6.9 scrub ok 2024-09-21T11:51:51.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:51 smithi175 ceph-mon[32613]: 6.e scrub ok 2024-09-21T11:51:51.725 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:51:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:51:51.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:51.811 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:51.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:51.812 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:51.853 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:51 smithi043 ceph-mon[29710]: pgmap v565: 148 pgs: 148 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 829 op/s 2024-09-21T11:51:51.853 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:51 smithi043 ceph-mon[29710]: 6.9 scrub starts 2024-09-21T11:51:51.853 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:51 smithi043 ceph-mon[29710]: 6.9 scrub ok 2024-09-21T11:51:51.853 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:51 smithi043 ceph-mon[29710]: 6.e scrub ok 2024-09-21T11:51:51.853 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:51 smithi043 ceph-mon[36454]: pgmap v565: 148 pgs: 148 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 829 op/s 2024-09-21T11:51:51.853 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:51 smithi043 ceph-mon[36454]: 6.9 scrub starts 2024-09-21T11:51:51.853 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:51 smithi043 ceph-mon[36454]: 6.9 scrub ok 2024-09-21T11:51:51.853 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:51 smithi043 ceph-mon[36454]: 6.e scrub ok 2024-09-21T11:51:51.913 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:51.968 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:51.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:51.969 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:52.052 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:52.063 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:51:52.069 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:52.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:52.109 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:52.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:52.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:52.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:52.259 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:52.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:52.360 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:52.390 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:52.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:52.391 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:52.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:51:52.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:52.522 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:52.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:52.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:51:52.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:52.673 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:52.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:52.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:52.723 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:51:52.723 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:51:52.723 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:51:52.723 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:51:52.723 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.723 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126212 7.1 7:86c8b838:::benchmark_data_smithi043_70646_object8804:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.194683+0000", 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 533.52317572200002, 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.097383589000000007, 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126212 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.724 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194683+0000", 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194683+0000", 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194685+0000", 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9920000000000002e-06 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.725 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194691+0000", 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4809999999999997e-06 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194702+0000", 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1559999999999999e-05 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194711+0000", 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5250000000000005e-06 2024-09-21T11:51:52.726 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194786+0000", 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4846999999999997e-05 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194894+0000", 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010852400000000001 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.195001+0000", 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106406 2024-09-21T11:51:52.727 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197371+0000", 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002370367 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197897+0000", 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00052575600000000005 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291967+0000", 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.094070592999999994 2024-09-21T11:51:52.728 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.292026+0000", 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8671000000000001e-05 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.292067+0000", 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0867e-05 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:185199 7.1 7:848115c2:::benchmark_data_smithi043_70646_object67791:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-21T11:51:52.729 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:43:57.851664+0000", 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 474.86619455200002, 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092637500999999997, 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 185199 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.730 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851664+0000", 2024-09-21T11:51:52.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.731 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.731 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851664+0000", 2024-09-21T11:51:52.731 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.731 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.731 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851666+0000", 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2249999999999999e-06 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851671+0000", 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8919999999999999e-06 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851686+0000", 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5675000000000002e-05 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.732 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851694+0000", 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9489999999999996e-06 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851811+0000", 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116335 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.851954+0000", 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143941 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.733 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.852069+0000", 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011415499999999999 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.854468+0000", 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023996629999999998 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.854814+0000", 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00034581899999999998 2024-09-21T11:51:52.734 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.944168+0000", 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.089353558999999999 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.944259+0000", 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0932000000000003e-05 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:43:57.944302+0000", 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3356000000000003e-05 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.735 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221225 7.1 7:81c1cf62:::benchmark_data_smithi043_70646_object103817:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.185689+0000", 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 445.53217001500002, 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092272482000000003, 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221225 2024-09-21T11:51:52.736 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.737 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185689+0000", 2024-09-21T11:51:52.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.737 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.737 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.737 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.737 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185689+0000", 2024-09-21T11:51:52.737 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185690+0000", 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4079999999999999e-06 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185696+0000", 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3789999999999997e-06 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.738 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185708+0000", 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2383e-05 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185714+0000", 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5709999999999997e-06 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185775+0000", 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1477999999999997e-05 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.739 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.185891+0000", 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011618800000000001 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,6", 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186010+0000", 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118808 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.188391+0000", 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023805369999999998 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.740 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.189115+0000", 2024-09-21T11:51:52.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00072425500000000004 2024-09-21T11:51:52.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.741 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.741 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.277871+0000", 2024-09-21T11:51:52.741 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.088755954999999997 2024-09-21T11:51:52.741 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.741 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.277921+0000", 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9827999999999998e-05 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.277961+0000", 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0692000000000003e-05 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.742 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.743 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.743 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.743 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43899 8.2 8:43c05007:::benchmark_data_smithi043_97042_object43898:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:51:52.743 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.986744+0000", 2024-09-21T11:51:52.743 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 391.73111521999999, 2024-09-21T11:51:52.743 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.132095406, 2024-09-21T11:51:52.743 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.743 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43899 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986744+0000", 2024-09-21T11:51:52.744 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986744+0000", 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986745+0000", 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.632e-06 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986800+0000", 2024-09-21T11:51:52.745 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4793999999999997e-05 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986816+0000", 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5444e-05 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986824+0000", 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5799999999999992e-06 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.746 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986889+0000", 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4658000000000003e-05 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987030+0000", 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014126500000000001 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987181+0000", 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015071799999999999 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989750+0000", 2024-09-21T11:51:52.747 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025688159999999998 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.991302+0000", 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015523080000000001 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118726+0000", 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12742430799999999 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118787+0000", 2024-09-21T11:51:52.748 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1141000000000003e-05 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118839+0000", 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1742e-05 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43905 8.2 8:49886794:::benchmark_data_smithi043_97042_object43904:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.987297+0000", 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 391.73056158899999, 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13219620100000001, 2024-09-21T11:51:52.749 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.750 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.750 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.750 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.750 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.750 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.750 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43905 2024-09-21T11:51:52.750 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.750 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.750 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.750 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987297+0000", 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987297+0000", 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.751 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987298+0000", 2024-09-21T11:51:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:52 smithi043 ceph-mon[36454]: 3.14 scrub starts 2024-09-21T11:51:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:52 smithi043 ceph-mon[36454]: 3.14 scrub ok 2024-09-21T11:51:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:52 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/612895913' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:51:52.752 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:52 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:51:52.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:52.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:52 smithi043 ceph-mon[29710]: 3.14 scrub starts 2024-09-21T11:51:52.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:52 smithi043 ceph-mon[29710]: 3.14 scrub ok 2024-09-21T11:51:52.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:52 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/612895913' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:51:52.753 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:52 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:51:52.753 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9999999999999996e-07 2024-09-21T11:51:52.753 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.753 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.753 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987377+0000", 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8947000000000002e-05 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987387+0000", 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4960000000000002e-06 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987394+0000", 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3159999999999999e-06 2024-09-21T11:51:52.754 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.988450+0000", 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001055802 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.988542+0000", 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1652999999999998e-05 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.988627+0000", 2024-09-21T11:51:52.755 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5554000000000001e-05 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989839+0000", 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001212288 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.991597+0000", 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001757303 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.756 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.119404+0000", 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12780698200000001 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.119460+0000", 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.622e-05 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.119494+0000", 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3739999999999999e-05 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.757 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46235 8.2 8:4e6f27b7:::benchmark_data_smithi043_97042_object46234:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.589574+0000", 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 390.12828537000001, 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.126942478, 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.758 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46235 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589574+0000", 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589574+0000", 2024-09-21T11:51:52.759 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589585+0000", 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0998999999999999e-05 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589616+0000", 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1123e-05 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.760 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589627+0000", 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1486e-05 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589634+0000", 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0740000000000004e-06 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589713+0000", 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8753999999999998e-05 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.761 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589860+0000", 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014660000000000001 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590003+0000", 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143516 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596149+0000", 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0061461579999999997 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.762 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596193+0000", 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3176000000000001e-05 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716435+0000", 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.120242746 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716472+0000", 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6868e-05 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.763 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716516+0000", 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3977999999999998e-05 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46237 8.2 8:4ddff5f6:::benchmark_data_smithi043_97042_object46236:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.590497+0000", 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 390.12736196399999, 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.126123824, 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.764 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46237 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590497+0000", 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.765 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590497+0000", 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590498+0000", 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0729999999999999e-06 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590524+0000", 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6284000000000001e-05 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.766 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590535+0000", 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0913999999999999e-05 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590545+0000", 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0054999999999999e-05 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.591661+0000", 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001115324 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.767 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.591765+0000", 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104269 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.591884+0000", 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118842 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.768 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596154+0000", 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0042705729999999997 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596233+0000", 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8449999999999996e-05 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716553+0000", 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12031977000000001 2024-09-21T11:51:52.769 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716582+0000", 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9628000000000002e-05 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716621+0000", 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8642e-05 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.770 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46241 8.2 8:41f5ae22:::benchmark_data_smithi043_97042_object46240:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.590600+0000", 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 390.12725951300001, 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.126117386, 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46241 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.771 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590600+0000", 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590600+0000", 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590600+0000", 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1500000000000004e-07 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.772 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590739+0000", 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013872799999999999 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590746+0000", 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5830000000000001e-06 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590752+0000", 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5099999999999998e-06 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.773 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.593857+0000", 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003104674 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.593998+0000", 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000141125 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.594116+0000", 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118461 2024-09-21T11:51:52.774 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596202+0000", 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020862720000000001 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598936+0000", 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002733666 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716656+0000", 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.117719485 2024-09-21T11:51:52.775 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716685+0000", 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9683e-05 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716717+0000", 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1584000000000001e-05 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.776 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46255 8.2 8:4a6e54a1:::benchmark_data_smithi043_97042_object46254:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.598672+0000", 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 390.11918679000001, 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.118134724, 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46255 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.777 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598672+0000", 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598672+0000", 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598674+0000", 2024-09-21T11:51:52.778 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6640000000000001e-06 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598702+0000", 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7993000000000001e-05 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598714+0000", 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2078e-05 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598721+0000", 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9099999999999999e-06 2024-09-21T11:51:52.779 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598969+0000", 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000248006 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.599102+0000", 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133526 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.599254+0000", 2024-09-21T11:51:52.780 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000151302 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.601110+0000", 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001856749 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.601489+0000", 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037851399999999999 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716744+0000", 2024-09-21T11:51:52.781 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.115254947 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716771+0000", 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7053e-05 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716807+0000", 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5982000000000002e-05 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.782 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47783 8.2 8:4b213eb9:::benchmark_data_smithi043_97042_object47782:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.709555+0000", 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 389.00830414900003, 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.135854843, 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47783 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.783 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709555+0000", 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709555+0000", 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709556+0000", 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4160000000000001e-06 2024-09-21T11:51:52.784 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709667+0000", 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110912 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709676+0000", 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6349999999999995e-06 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709682+0000", 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8649999999999998e-06 2024-09-21T11:51:52.785 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.710257+0000", 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057514299999999997 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.710363+0000", 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010655200000000001 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.710488+0000", 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124943 2024-09-21T11:51:52.786 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712767+0000", 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022785959999999999 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712779+0000", 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2417e-05 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845284+0000", 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.132504967 2024-09-21T11:51:52.787 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845364+0000", 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9629000000000006e-05 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845410+0000", 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5768e-05 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.788 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:51215 8.2 8:470c6f96:::benchmark_data_smithi043_97042_object51214:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:25.913685+0000", 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 386.80417406800001, 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.098319711000000004, 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 51215 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.789 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913685+0000", 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913685+0000", 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913686+0000", 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1829999999999999e-06 2024-09-21T11:51:52.790 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913793+0000", 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106716 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913802+0000", 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5389999999999996e-06 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.913809+0000", 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7139999999999996e-06 2024-09-21T11:51:52.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.919718+0000", 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0059089989999999998 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.919851+0000", 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000132526 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.920234+0000", 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000383397 2024-09-21T11:51:52.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.925843+0000", 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0056085889999999998 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.925917+0000", 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4525999999999998e-05 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.011831+0000", 2024-09-21T11:51:52.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:52.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.085914040999999997 2024-09-21T11:51:52.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.011952+0000", 2024-09-21T11:51:52.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000120442 2024-09-21T11:51:52.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.012005+0000", 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3038999999999998e-05 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:51227 8.2 8:4082683c:::benchmark_data_smithi043_97042_object51226:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:25.921381+0000", 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 386.79647794300001, 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.094778826999999996, 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.796 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 51227 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921381+0000", 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921381+0000", 2024-09-21T11:51:52.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921382+0000", 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9000000000000005e-07 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921406+0000", 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3465e-05 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921473+0000", 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7885999999999993e-05 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.921482+0000", 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3610000000000001e-06 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.924030+0000", 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025484309999999999 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.924134+0000", 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104109 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.924253+0000", 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118822 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.925849+0000", 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015963360000000001 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:25.927356+0000", 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015062750000000001 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.016021+0000", 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.088665620000000001 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.016114+0000", 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2369999999999998e-05 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:26.016160+0000", 2024-09-21T11:51:52.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6162000000000003e-05 2024-09-21T11:51:52.802 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.802 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.802 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.802 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66128 8.2 8:4c9d601c:::benchmark_data_smithi043_97042_object66127:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:51:52.802 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.918161+0000", 2024-09-21T11:51:52.802 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 376.79969771600003, 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13079232700000001, 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66128 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918161+0000", 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918161+0000", 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918162+0000", 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0610000000000001e-06 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.804 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918211+0000", 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9125000000000001e-05 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918221+0000", 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.183e-06 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918227+0000", 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5470000000000002e-06 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919404+0000", 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011771520000000001 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919510+0000", 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106103 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919632+0000", 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012106400000000001 2024-09-21T11:51:52.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.921625+0000", 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019929919999999999 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.923993+0000", 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023686279999999998 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048774+0000", 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.124780935 2024-09-21T11:51:52.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048871+0000", 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6514000000000005e-05 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048954+0000", 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3022999999999994e-05 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.808 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:68279 8.2 8:4095089f:::benchmark_data_smithi043_97042_object68278:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e178)", 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:37.675190+0000", 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 375.04266868899998, 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.098842746999999995, 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 68279 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675190+0000", 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675190+0000", 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675192+0000", 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4130000000000001e-06 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.810 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675440+0000", 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024801000000000001 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675452+0000", 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1754e-05 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675459+0000", 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4120000000000004e-06 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675521+0000", 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2074000000000002e-05 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675614+0000", 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3327999999999998e-05 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.675723+0000", 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108779 2024-09-21T11:51:52.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.677638+0000", 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001914397 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680355+0000", 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027178079999999999 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773903+0000", 2024-09-21T11:51:52.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093547443999999993 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773991+0000", 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8202999999999996e-05 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.774033+0000", 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2125000000000001e-05 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:68271 8.0 e178/160 8:076b09a0:::benchmark_data_smithi043_97042_object68270:head v 178'4262, mlcod=178'4258)", 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:37.680051+0000", 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 375.03780796299998, 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093011187999999995, 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680051+0000", 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680051+0000", 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680052+0000", 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4199999999999995e-07 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.816 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680119+0000", 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7389000000000001e-05 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680126+0000", 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1069999999999999e-06 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680131+0000", 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5310000000000003e-06 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680178+0000", 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7292000000000001e-05 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.680191+0000", 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2585999999999999e-05 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.772967+0000", 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092776485000000006 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.818 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773062+0000", 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5056000000000006e-05 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:68278 8.0 e178/160 8:0020ddcf:::benchmark_data_smithi043_97042_object68277:head v 178'4263, mlcod=178'4260)", 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:37.681594+0000", 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 375.03626524600003, 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.095449811999999995, 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.819 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681594+0000", 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681594+0000", 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681594+0000", 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1999999999999999e-07 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681624+0000", 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9839e-05 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681634+0000", 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8099999999999992e-06 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.681641+0000", 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5610000000000004e-06 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.770591+0000", 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.088950076000000003 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.770613+0000", 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2410999999999999e-05 2024-09-21T11:51:52.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.776994+0000", 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0063813350000000001 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.777044+0000", 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9159999999999997e-05 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.823 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.15150.0:68285 8.0 e178/160 8:0330324f:::benchmark_data_smithi043_97042_object68284:head v 178'4264, mlcod=178'4260)", 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:37.685442+0000", 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 375.03241672299998, 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093833114999999995, 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.685442+0000", 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.685442+0000", 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.685443+0000", 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1500000000000005e-07 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.685495+0000", 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2326e-05 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.768117+0000", 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.082621662999999998 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.768127+0000", 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0193999999999999e-05 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773140+0000", 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005013399 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.773161+0000", 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0126999999999999e-05 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.779234+0000", 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0060736840000000002 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:37.779275+0000", 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1207e-05 2024-09-21T11:51:52.827 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:123646 8.2 8:4e3911e1:::benchmark_data_smithi043_97042_object123645:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:21.826953+0000", 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 330.89090647699999, 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11253259, 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.828 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123646 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.826953+0000", 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.826953+0000", 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.829 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.826954+0000", 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.398e-06 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.826983+0000", 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8931000000000001e-05 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.826993+0000", 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0366e-05 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827000+0000", 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9639999999999997e-06 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827273+0000", 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000272916 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827397+0000", 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012345800000000001 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827532+0000", 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013517600000000001 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.830113+0000", 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025809880000000002 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.836834+0000", 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.006721744 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939366+0000", 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.102531452 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939426+0000", 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0059999999999998e-05 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939485+0000", 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9137e-05 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:123647 8.2 8:40a5b330:::benchmark_data_smithi043_97042_object123646:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:21.827047+0000", 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 330.890811746, 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11257774, 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123647 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827047+0000", 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827047+0000", 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827048+0000", 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2399999999999998e-07 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827087+0000", 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9209999999999999e-05 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827095+0000", 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3950000000000003e-06 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.827100+0000", 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3759999999999999e-06 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.830139+0000", 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030388669999999998 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.830263+0000", 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124721 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.830393+0000", 2024-09-21T11:51:52.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129195 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833041+0000", 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002648451 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.839629+0000", 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0065879090000000003 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939529+0000", 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.099900177000000007 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939588+0000", 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8683e-05 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.939625+0000", 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7132000000000003e-05 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.839 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:123655 8.2 8:4f314857:::benchmark_data_smithi043_97042_object123654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:46:21.833269+0000", 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 330.88459033800001, 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.109746765, 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:51:52.840 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123655 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833269+0000", 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833269+0000", 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:52.841 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833270+0000", 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1880000000000001e-06 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833325+0000", 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5247000000000001e-05 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833335+0000", 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5410000000000006e-06 2024-09-21T11:51:52.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833340+0000", 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.293e-06 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833393+0000", 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3057999999999999e-05 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833521+0000", 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012788099999999999 2024-09-21T11:51:52.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.833672+0000", 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015077299999999999 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.837990+0000", 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043184779999999997 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.839933+0000", 2024-09-21T11:51:52.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019426389999999999 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.942944+0000", 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.103011493 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.942987+0000", 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2731999999999997e-05 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:46:21.943015+0000", 2024-09-21T11:51:52.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8442e-05 2024-09-21T11:51:52.846 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.846 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.846 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.846 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:52.846 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:52.846 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:51:52.878 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:52.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:52.879 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:52.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:53.003 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:53.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:53.004 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:53.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:51:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:52 smithi175 ceph-mon[32613]: 3.14 scrub starts 2024-09-21T11:51:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:52 smithi175 ceph-mon[32613]: 3.14 scrub ok 2024-09-21T11:51:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:52 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/612895913' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:51:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:52 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:51:53.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:53.142 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:53.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:53.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:53.243 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:53.275 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:53.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:53.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:53.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:53.415 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:53.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:53.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:53.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:53.549 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:53.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:53.550 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:53.651 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:53.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:53.692 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:53.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:53.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:53.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:53.824 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:53.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:53.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:53.953 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:53.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:53.954 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[36454]: pgmap v566: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-21T11:51:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[36454]: 5.2 scrub starts 2024-09-21T11:51:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[36454]: 5.2 scrub ok 2024-09-21T11:51:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:51:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[36454]: osdmap e296: 8 total, 8 up, 8 in 2024-09-21T11:51:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[29710]: pgmap v566: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-21T11:51:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[29710]: 5.2 scrub starts 2024-09-21T11:51:53.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[29710]: 5.2 scrub ok 2024-09-21T11:51:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:51:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:53 smithi043 ceph-mon[29710]: osdmap e296: 8 total, 8 up, 8 in 2024-09-21T11:51:54.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:53 smithi175 ceph-mon[32613]: pgmap v566: 148 pgs: 148 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.25k op/s 2024-09-21T11:51:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:53 smithi175 ceph-mon[32613]: 5.2 scrub starts 2024-09-21T11:51:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:53 smithi175 ceph-mon[32613]: 5.2 scrub ok 2024-09-21T11:51:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:53 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:51:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:53 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:51:54.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:53 smithi175 ceph-mon[32613]: osdmap e296: 8 total, 8 up, 8 in 2024-09-21T11:51:54.054 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:54.085 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:54.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:54.086 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:54.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:54.224 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:54.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:54.225 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:54.326 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:54.353 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:54.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:54.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:54.455 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:54.477 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:54.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:54.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:54.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:51:54.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:54.610 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:54.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:54.611 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:54.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:54.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:54.749 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:54.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:54.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:54.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:54.876 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:54.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:54.915 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:51:54.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:55.017 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:55.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:55.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:55.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:55.157 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:55.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:55.158 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:55.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:55.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:55.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:55.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:55.427 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:55.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:55.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211066 9.a 9:51f636b8:::benchmark_data_smithi043_126002_object84000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.041382374999999999, 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.041405026999999997, 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:51:55.455 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211066 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:51:55.456 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411975+0000", 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.401e-06 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411979+0000", 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4889999999999997e-06 2024-09-21T11:51:55.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411984+0000", 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2530000000000004e-06 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412132+0000", 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014723399999999999 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412228+0000", 2024-09-21T11:51:55.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6713999999999996e-05 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,7", 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412367+0000", 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013867400000000001 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415401+0000", 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030337630000000001 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.421153+0000", 2024-09-21T11:51:55.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057522909999999997 2024-09-21T11:51:55.460 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:55.460 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:51:55.460 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:55.460 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:51:55.460 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:51:55.460 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T11:51:55.460 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:51:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:51:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:51:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:51:55.528 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:55.549 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:55.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:55.550 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:55.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:55.673 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:55.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:55.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:55.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:55.811 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:55.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:55.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:55.912 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:55.945 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:55.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:55.946 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:55.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:55 smithi043 ceph-mon[29710]: pgmap v568: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.51k op/s 2024-09-21T11:51:55.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:55 smithi043 ceph-mon[36454]: pgmap v568: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.51k op/s 2024-09-21T11:51:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:55 smithi175 ceph-mon[32613]: pgmap v568: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.51k op/s 2024-09-21T11:51:56.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:56.069 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:56.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:56.070 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:56.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:51:56.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:56.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:56.243 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:56.243 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:56.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:56.373 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:56.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:56.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:56.474 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:56.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:56.518 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:56.519 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:56.582 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:51:56.619 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:56.668 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:56.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:56.669 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:56.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:56 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:56.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:56 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:56.770 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:56.797 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:56.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:56.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:56.881 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:56 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:51:56.899 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:56.949 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:56.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:56.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:56.999 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:51:57.002 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:51:57.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:51:57.073 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:57.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:57.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:51:57.140 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:51:57.140 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:51:57.140 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:51:57.140 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:51:57.140 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:51:57.174 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:57.207 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:57.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:57.208 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:57.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:57.342 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:57.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:57.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:57.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:57.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:57.475 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:57.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:57.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:57.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:57.638 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:57.638 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:57.739 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:57.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:57 smithi175 ceph-mon[32613]: pgmap v569: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 909 op/s 2024-09-21T11:51:57.770 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:57.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:57.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:57.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:51:57.872 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:57.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:57.917 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:57.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:57.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:57 smithi043 ceph-mon[29710]: pgmap v569: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 909 op/s 2024-09-21T11:51:57.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:57 smithi043 ceph-mon[36454]: pgmap v569: 148 pgs: 148 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 909 op/s 2024-09-21T11:51:58.018 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:58.062 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:58.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:58.063 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:58.106 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":569,"stamp":"2024-09-21T11:51:56.006447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3111066163,"num_objects":47662,"num_object_clones":0,"num_object_copies":142986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47662,"num_whiteouts":0,"num_read":1952,"num_read_kb":1809,"num_write":207500,"num_write_kb":8133706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":597267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83761,"ondisk_log_size":83761,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":12723524,"kb_used_data":9204228,"kb_used_omap":12,"kb_used_meta":3518771,"kb_avail":737237692,"statfs":{"total":767960285184,"available":754931396608,"internally_reserved":0,"allocated":9425129472,"data_stored":9421853815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":3603222066},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,5,8,7,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-562561024,"num_objects":-8584,"num_object_clones":0,"num_object_copies":-25752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8584,"num_whiteouts":0,"num_read":-5,"num_read_kb":-5,"num_write":8584,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.441582"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":504,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.584543+0000","last_change":"2024-09-21T11:48:14.197852+0000","last_active":"2024-09-21T11:51:46.584543+0000","last_peered":"2024-09-21T11:51:46.584543+0000","last_clean":"2024-09-21T11:51:46.584543+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:51:46.584543+0000","last_undegraded":"2024-09-21T11:51:46.584543+0000","last_fullsized":"2024-09-21T11:51:46.584543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:14.197775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:48:14.197775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:48.904875+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":536,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572262+0000","last_change":"2024-09-21T11:51:43.527338+0000","last_active":"2024-09-21T11:51:46.572262+0000","last_peered":"2024-09-21T11:51:46.572262+0000","last_clean":"2024-09-21T11:51:46.572262+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:51:46.572262+0000","last_undegraded":"2024-09-21T11:51:46.572262+0000","last_fullsized":"2024-09-21T11:51:46.572262+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:02.926630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:02.926630+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:08.234762+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":205,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.567993+0000","last_change":"2024-09-21T11:50:39.017249+0000","last_active":"2024-09-21T11:51:46.567993+0000","last_peered":"2024-09-21T11:51:46.567993+0000","last_clean":"2024-09-21T11:51:46.567993+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:51:46.567993+0000","last_undegraded":"2024-09-21T11:51:46.567993+0000","last_fullsized":"2024-09-21T11:51:46.567993+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:39.017173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.017173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:55.424271+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":325,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575948+0000","last_change":"2024-09-21T11:51:43.528594+0000","last_active":"2024-09-21T11:51:46.575948+0000","last_peered":"2024-09-21T11:51:46.575948+0000","last_clean":"2024-09-21T11:51:46.575948+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:51:46.575948+0000","last_undegraded":"2024-09-21T11:51:46.575948+0000","last_fullsized":"2024-09-21T11:51:46.575948+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":508,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572676+0000","last_change":"2024-09-21T11:50:26.427803+0000","last_active":"2024-09-21T11:51:46.572676+0000","last_peered":"2024-09-21T11:51:46.572676+0000","last_clean":"2024-09-21T11:51:46.572676+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:51:46.572676+0000","last_undegraded":"2024-09-21T11:51:46.572676+0000","last_fullsized":"2024-09-21T11:51:46.572676+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:26.427733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.427733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":505,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572789+0000","last_change":"2024-09-21T11:51:01.667426+0000","last_active":"2024-09-21T11:51:46.572789+0000","last_peered":"2024-09-21T11:51:46.572789+0000","last_clean":"2024-09-21T11:51:46.572789+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:51:46.572789+0000","last_undegraded":"2024-09-21T11:51:46.572789+0000","last_fullsized":"2024-09-21T11:51:46.572789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.667366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.667366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:02.624646+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":551,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.576892+0000","last_change":"2024-09-21T11:51:43.409002+0000","last_active":"2024-09-21T11:51:46.576892+0000","last_peered":"2024-09-21T11:51:46.576892+0000","last_clean":"2024-09-21T11:51:46.576892+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:51:46.576892+0000","last_undegraded":"2024-09-21T11:51:46.576892+0000","last_fullsized":"2024-09-21T11:51:46.576892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:51:43.408804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.408804+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:45.694729+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":337,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572353+0000","last_change":"2024-09-21T11:51:43.527225+0000","last_active":"2024-09-21T11:51:46.572353+0000","last_peered":"2024-09-21T11:51:46.572353+0000","last_clean":"2024-09-21T11:51:46.572353+0000","last_became_active":"2024-09-21T11:51:43.526945+0000","last_became_peered":"2024-09-21T11:51:43.526945+0000","last_unstale":"2024-09-21T11:51:46.572353+0000","last_undegraded":"2024-09-21T11:51:46.572353+0000","last_fullsized":"2024-09-21T11:51:46.572353+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.279369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.279369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":511,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.580261+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:51:46.580261+0000","last_peered":"2024-09-21T11:51:46.580261+0000","last_clean":"2024-09-21T11:51:46.580261+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:51:46.580261+0000","last_undegraded":"2024-09-21T11:51:46.580261+0000","last_fullsized":"2024-09-21T11:51:46.580261+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:16.581299+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":502,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.577712+0000","last_change":"2024-09-21T11:51:43.548520+0000","last_active":"2024-09-21T11:51:46.577712+0000","last_peered":"2024-09-21T11:51:46.577712+0000","last_clean":"2024-09-21T11:51:46.577712+0000","last_became_active":"2024-09-21T11:51:43.545021+0000","last_became_peered":"2024-09-21T11:51:43.545021+0000","last_unstale":"2024-09-21T11:51:46.577712+0000","last_undegraded":"2024-09-21T11:51:46.577712+0000","last_fullsized":"2024-09-21T11:51:46.577712+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:50:39.302102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.302102+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:00.832261+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":503,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.576255+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:51:46.576255+0000","last_peered":"2024-09-21T11:51:46.576255+0000","last_clean":"2024-09-21T11:51:46.576255+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:51:46.576255+0000","last_undegraded":"2024-09-21T11:51:46.576255+0000","last_fullsized":"2024-09-21T11:51:46.576255+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":521,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578463+0000","last_change":"2024-09-21T11:51:12.982402+0000","last_active":"2024-09-21T11:51:46.578463+0000","last_peered":"2024-09-21T11:51:46.578463+0000","last_clean":"2024-09-21T11:51:46.578463+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:51:46.578463+0000","last_undegraded":"2024-09-21T11:51:46.578463+0000","last_fullsized":"2024-09-21T11:51:46.578463+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:12.982081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.982081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:31.517051+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":533,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.576307+0000","last_change":"2024-09-21T11:50:27.746458+0000","last_active":"2024-09-21T11:51:46.576307+0000","last_peered":"2024-09-21T11:51:46.576307+0000","last_clean":"2024-09-21T11:51:46.576307+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:51:46.576307+0000","last_undegraded":"2024-09-21T11:51:46.576307+0000","last_fullsized":"2024-09-21T11:51:46.576307+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:27.746396+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.746396+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:48.701384+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":516,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572960+0000","last_change":"2024-09-21T11:51:43.547924+0000","last_active":"2024-09-21T11:51:46.572960+0000","last_peered":"2024-09-21T11:51:46.572960+0000","last_clean":"2024-09-21T11:51:46.572960+0000","last_became_active":"2024-09-21T11:51:43.545191+0000","last_became_peered":"2024-09-21T11:51:43.545191+0000","last_unstale":"2024-09-21T11:51:46.572960+0000","last_undegraded":"2024-09-21T11:51:46.572960+0000","last_fullsized":"2024-09-21T11:51:46.572960+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":517,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573817+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:51:46.573817+0000","last_peered":"2024-09-21T11:51:46.573817+0000","last_clean":"2024-09-21T11:51:46.573817+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:51:46.573817+0000","last_undegraded":"2024-09-21T11:51:46.573817+0000","last_fullsized":"2024-09-21T11:51:46.573817+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.398415+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":521,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575896+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:51:46.575896+0000","last_peered":"2024-09-21T11:51:46.575896+0000","last_clean":"2024-09-21T11:51:46.575896+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:51:46.575896+0000","last_undegraded":"2024-09-21T11:51:46.575896+0000","last_fullsized":"2024-09-21T11:51:46.575896+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":15,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.570898+0000","last_change":"2024-09-21T11:51:43.524599+0000","last_active":"2024-09-21T11:51:46.570898+0000","last_peered":"2024-09-21T11:51:46.570898+0000","last_clean":"2024-09-21T11:51:46.570898+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:51:46.570898+0000","last_undegraded":"2024-09-21T11:51:46.570898+0000","last_fullsized":"2024-09-21T11:51:46.570898+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:32.302842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.302842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":503,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.580090+0000","last_change":"2024-09-21T11:50:35.718093+0000","last_active":"2024-09-21T11:51:46.580090+0000","last_peered":"2024-09-21T11:51:46.580090+0000","last_clean":"2024-09-21T11:51:46.580090+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:51:46.580090+0000","last_undegraded":"2024-09-21T11:51:46.580090+0000","last_fullsized":"2024-09-21T11:51:46.580090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:35.717991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.717991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.422206+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":636,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573433+0000","last_change":"2024-09-21T11:50:25.966805+0000","last_active":"2024-09-21T11:51:46.573433+0000","last_peered":"2024-09-21T11:51:46.573433+0000","last_clean":"2024-09-21T11:51:46.573433+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:51:46.573433+0000","last_undegraded":"2024-09-21T11:51:46.573433+0000","last_fullsized":"2024-09-21T11:51:46.573433+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:50:25.966740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.966740+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.945863+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":213,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.568377+0000","last_change":"2024-09-21T11:50:31.012243+0000","last_active":"2024-09-21T11:51:46.568377+0000","last_peered":"2024-09-21T11:51:46.568377+0000","last_clean":"2024-09-21T11:51:46.568377+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:51:46.568377+0000","last_undegraded":"2024-09-21T11:51:46.568377+0000","last_fullsized":"2024-09-21T11:51:46.568377+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.012177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.012177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.366361+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":501,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573809+0000","last_change":"2024-09-21T11:50:06.400378+0000","last_active":"2024-09-21T11:51:46.573809+0000","last_peered":"2024-09-21T11:51:46.573809+0000","last_clean":"2024-09-21T11:51:46.573809+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:51:46.573809+0000","last_undegraded":"2024-09-21T11:51:46.573809+0000","last_fullsized":"2024-09-21T11:51:46.573809+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:06.400307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.400307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:29.569363+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":329,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574191+0000","last_change":"2024-09-21T11:51:43.543408+0000","last_active":"2024-09-21T11:51:46.574191+0000","last_peered":"2024-09-21T11:51:46.574191+0000","last_clean":"2024-09-21T11:51:46.574191+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:51:46.574191+0000","last_undegraded":"2024-09-21T11:51:46.574191+0000","last_fullsized":"2024-09-21T11:51:46.574191+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.122227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.122227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":578,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573125+0000","last_change":"2024-09-21T11:51:12.652049+0000","last_active":"2024-09-21T11:51:46.573125+0000","last_peered":"2024-09-21T11:51:46.573125+0000","last_clean":"2024-09-21T11:51:46.573125+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:51:46.573125+0000","last_undegraded":"2024-09-21T11:51:46.573125+0000","last_fullsized":"2024-09-21T11:51:46.573125+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:51:12.651967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.651967+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:24.851481+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575426+0000","last_change":"2024-09-21T11:51:43.549287+0000","last_active":"2024-09-21T11:51:46.575426+0000","last_peered":"2024-09-21T11:51:46.575426+0000","last_clean":"2024-09-21T11:51:46.575426+0000","last_became_active":"2024-09-21T11:51:43.548726+0000","last_became_peered":"2024-09-21T11:51:43.548726+0000","last_unstale":"2024-09-21T11:51:46.575426+0000","last_undegraded":"2024-09-21T11:51:46.575426+0000","last_fullsized":"2024-09-21T11:51:46.575426+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.908339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.908339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:45.695969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":26,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.581204+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:51:46.581204+0000","last_peered":"2024-09-21T11:51:46.581204+0000","last_clean":"2024-09-21T11:51:46.581204+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:51:46.581204+0000","last_undegraded":"2024-09-21T11:51:46.581204+0000","last_fullsized":"2024-09-21T11:51:46.581204+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:49.526433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":209,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.568169+0000","last_change":"2024-09-21T11:50:02.920313+0000","last_active":"2024-09-21T11:51:46.568169+0000","last_peered":"2024-09-21T11:51:46.568169+0000","last_clean":"2024-09-21T11:51:46.568169+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:51:46.568169+0000","last_undegraded":"2024-09-21T11:51:46.568169+0000","last_fullsized":"2024-09-21T11:51:46.568169+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:02.920229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.920229+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:26.221890+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":546,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.584996+0000","last_change":"2024-09-21T11:49:58.702620+0000","last_active":"2024-09-21T11:51:46.584996+0000","last_peered":"2024-09-21T11:51:46.584996+0000","last_clean":"2024-09-21T11:51:46.584996+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:51:46.584996+0000","last_undegraded":"2024-09-21T11:51:46.584996+0000","last_fullsized":"2024-09-21T11:51:46.584996+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:49:58.702544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.702544+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:25.870515+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":682,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.571217+0000","last_change":"2024-09-21T11:51:44.478918+0000","last_active":"2024-09-21T11:51:46.571217+0000","last_peered":"2024-09-21T11:51:46.571217+0000","last_clean":"2024-09-21T11:51:46.571217+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:51:46.571217+0000","last_undegraded":"2024-09-21T11:51:46.571217+0000","last_fullsized":"2024-09-21T11:51:46.571217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:44.478855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.478855+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:51.098456+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"295'13031","reported_seq":13205,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:52.236398+0000","last_change":"2024-09-21T11:51:16.011591+0000","last_active":"2024-09-21T11:51:52.236398+0000","last_peered":"2024-09-21T11:51:52.236398+0000","last_clean":"2024-09-21T11:51:52.236398+0000","last_became_active":"2024-09-21T11:49:08.748359+0000","last_became_peered":"2024-09-21T11:49:08.748359+0000","last_unstale":"2024-09-21T11:51:52.236398+0000","last_undegraded":"2024-09-21T11:51:52.236398+0000","last_fullsized":"2024-09-21T11:51:52.236398+0000","mapping_epoch":247,"log_start":"278'8100","ondisk_log_start":"278'8100","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"286'10317","last_scrub_stamp":"2024-09-21T11:51:16.011166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:51:16.011166+0000","objects_scrubbed":5697,"log_size":4931,"log_dups_size":0,"ondisk_log_size":4931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:32.495145+0000","scrub_duration":2370,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190251008,"num_objects":2903,"num_object_clones":0,"num_object_copies":8709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13031,"num_write_kb":509888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":248,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.568372+0000","last_change":"2024-09-21T11:51:20.237081+0000","last_active":"2024-09-21T11:51:46.568372+0000","last_peered":"2024-09-21T11:51:46.568372+0000","last_clean":"2024-09-21T11:51:46.568372+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:51:46.568372+0000","last_undegraded":"2024-09-21T11:51:46.568372+0000","last_fullsized":"2024-09-21T11:51:46.568372+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:51:20.236816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.236816+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:36.925607+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":535,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574121+0000","last_change":"2024-09-21T11:49:47.205216+0000","last_active":"2024-09-21T11:51:46.574121+0000","last_peered":"2024-09-21T11:51:46.574121+0000","last_clean":"2024-09-21T11:51:46.574121+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:51:46.574121+0000","last_undegraded":"2024-09-21T11:51:46.574121+0000","last_fullsized":"2024-09-21T11:51:46.574121+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.204889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.204889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:23.639717+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"294'9","reported_seq":44,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.577941+0000","last_change":"2024-09-21T11:51:43.526079+0000","last_active":"2024-09-21T11:51:46.577941+0000","last_peered":"2024-09-21T11:51:46.577941+0000","last_clean":"2024-09-21T11:51:46.577941+0000","last_became_active":"2024-09-21T11:51:43.525566+0000","last_became_peered":"2024-09-21T11:51:43.525566+0000","last_unstale":"2024-09-21T11:51:46.577941+0000","last_undegraded":"2024-09-21T11:51:46.577941+0000","last_fullsized":"2024-09-21T11:51:46.577941+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:11.932479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:11.932479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:18.443038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":522,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578744+0000","last_change":"2024-09-21T11:50:25.708598+0000","last_active":"2024-09-21T11:51:46.578744+0000","last_peered":"2024-09-21T11:51:46.578744+0000","last_clean":"2024-09-21T11:51:46.578744+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:51:46.578744+0000","last_undegraded":"2024-09-21T11:51:46.578744+0000","last_fullsized":"2024-09-21T11:51:46.578744+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:33.641612+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"295'12841","reported_seq":13028,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.094529+0000","last_change":"2024-09-21T11:50:28.231461+0000","last_active":"2024-09-21T11:51:51.094529+0000","last_peered":"2024-09-21T11:51:51.094529+0000","last_clean":"2024-09-21T11:51:51.094529+0000","last_became_active":"2024-09-21T11:49:08.519622+0000","last_became_peered":"2024-09-21T11:49:08.519622+0000","last_unstale":"2024-09-21T11:51:51.094529+0000","last_undegraded":"2024-09-21T11:51:51.094529+0000","last_fullsized":"2024-09-21T11:51:51.094529+0000","mapping_epoch":247,"log_start":"278'8000","ondisk_log_start":"278'8000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6919","last_scrub_stamp":"2024-09-21T11:50:28.231378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.231378+0000","objects_scrubbed":6856,"log_size":4841,"log_dups_size":0,"ondisk_log_size":4841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195428376,"num_objects":2983,"num_object_clones":0,"num_object_copies":8949,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12841,"num_write_kb":506305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":567,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573792+0000","last_change":"2024-09-21T11:49:52.209526+0000","last_active":"2024-09-21T11:51:46.573792+0000","last_peered":"2024-09-21T11:51:46.573792+0000","last_clean":"2024-09-21T11:51:46.573792+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:51:46.573792+0000","last_undegraded":"2024-09-21T11:51:46.573792+0000","last_fullsized":"2024-09-21T11:51:46.573792+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-21T11:49:52.209433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:52.209433+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:31.527581+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":521,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574274+0000","last_change":"2024-09-21T11:49:55.238709+0000","last_active":"2024-09-21T11:51:46.574274+0000","last_peered":"2024-09-21T11:51:46.574274+0000","last_clean":"2024-09-21T11:51:46.574274+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:51:46.574274+0000","last_undegraded":"2024-09-21T11:51:46.574274+0000","last_fullsized":"2024-09-21T11:51:46.574274+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:55.238302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:55.238302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:27.582532+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.576465+0000","last_change":"2024-09-21T11:51:43.542323+0000","last_active":"2024-09-21T11:51:46.576465+0000","last_peered":"2024-09-21T11:51:46.576465+0000","last_clean":"2024-09-21T11:51:46.576465+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:51:46.576465+0000","last_undegraded":"2024-09-21T11:51:46.576465+0000","last_fullsized":"2024-09-21T11:51:46.576465+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.940335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.940335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:35.027733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.577613+0000","last_change":"2024-09-21T11:51:43.548783+0000","last_active":"2024-09-21T11:51:46.577613+0000","last_peered":"2024-09-21T11:51:46.577613+0000","last_clean":"2024-09-21T11:51:46.577613+0000","last_became_active":"2024-09-21T11:51:43.545884+0000","last_became_peered":"2024-09-21T11:51:43.545884+0000","last_unstale":"2024-09-21T11:51:46.577613+0000","last_undegraded":"2024-09-21T11:51:46.577613+0000","last_fullsized":"2024-09-21T11:51:46.577613+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:08.977984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.977984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:11.274400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"295'12985","reported_seq":13165,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:52.229903+0000","last_change":"2024-09-21T11:51:05.618601+0000","last_active":"2024-09-21T11:51:52.229903+0000","last_peered":"2024-09-21T11:51:52.229903+0000","last_clean":"2024-09-21T11:51:52.229903+0000","last_became_active":"2024-09-21T11:49:08.609618+0000","last_became_peered":"2024-09-21T11:49:08.609618+0000","last_unstale":"2024-09-21T11:51:52.229903+0000","last_undegraded":"2024-09-21T11:51:52.229903+0000","last_fullsized":"2024-09-21T11:51:52.229903+0000","mapping_epoch":247,"log_start":"278'8100","ondisk_log_start":"278'8100","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"284'9506","last_scrub_stamp":"2024-09-21T11:51:05.618420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.618420+0000","objects_scrubbed":6457,"log_size":4885,"log_dups_size":0,"ondisk_log_size":4885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:17.769660+0000","scrub_duration":1992,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192217088,"num_objects":2933,"num_object_clones":0,"num_object_copies":8799,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12985,"num_write_kb":509376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":527,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.576544+0000","last_change":"2024-09-21T11:51:43.525478+0000","last_active":"2024-09-21T11:51:46.576544+0000","last_peered":"2024-09-21T11:51:46.576544+0000","last_clean":"2024-09-21T11:51:46.576544+0000","last_became_active":"2024-09-21T11:51:43.523852+0000","last_became_peered":"2024-09-21T11:51:43.523852+0000","last_unstale":"2024-09-21T11:51:46.576544+0000","last_undegraded":"2024-09-21T11:51:46.576544+0000","last_fullsized":"2024-09-21T11:51:46.576544+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":551,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.585276+0000","last_change":"2024-09-21T11:50:28.657341+0000","last_active":"2024-09-21T11:51:46.585276+0000","last_peered":"2024-09-21T11:51:46.585276+0000","last_clean":"2024-09-21T11:51:46.585276+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:51:46.585276+0000","last_undegraded":"2024-09-21T11:51:46.585276+0000","last_fullsized":"2024-09-21T11:51:46.585276+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:28.657261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.657261+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":687,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573502+0000","last_change":"2024-09-21T11:51:20.675165+0000","last_active":"2024-09-21T11:51:46.573502+0000","last_peered":"2024-09-21T11:51:46.573502+0000","last_clean":"2024-09-21T11:51:46.573502+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:51:46.573502+0000","last_undegraded":"2024-09-21T11:51:46.573502+0000","last_fullsized":"2024-09-21T11:51:46.573502+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:51:20.675101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.675101+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:30.892012+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":511,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574208+0000","last_change":"2024-09-21T11:50:31.395647+0000","last_active":"2024-09-21T11:51:46.574208+0000","last_peered":"2024-09-21T11:51:46.574208+0000","last_clean":"2024-09-21T11:51:46.574208+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:51:46.574208+0000","last_undegraded":"2024-09-21T11:51:46.574208+0000","last_fullsized":"2024-09-21T11:51:46.574208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.395586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.395586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"295'12872","reported_seq":12981,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.229673+0000","last_change":"2024-09-21T11:49:08.518112+0000","last_active":"2024-09-21T11:51:51.229673+0000","last_peered":"2024-09-21T11:51:51.229673+0000","last_clean":"2024-09-21T11:51:51.229673+0000","last_became_active":"2024-09-21T11:49:08.517344+0000","last_became_peered":"2024-09-21T11:49:08.517344+0000","last_unstale":"2024-09-21T11:51:51.229673+0000","last_undegraded":"2024-09-21T11:51:51.229673+0000","last_fullsized":"2024-09-21T11:51:51.229673+0000","mapping_epoch":247,"log_start":"277'7500","ondisk_log_start":"277'7500","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5372,"log_dups_size":0,"ondisk_log_size":5372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:53.624008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194510848,"num_objects":2968,"num_object_clones":0,"num_object_copies":8904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12872,"num_write_kb":506880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":233,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574498+0000","last_change":"2024-09-21T11:51:08.137774+0000","last_active":"2024-09-21T11:51:46.574498+0000","last_peered":"2024-09-21T11:51:46.574498+0000","last_clean":"2024-09-21T11:51:46.574498+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:51:46.574498+0000","last_undegraded":"2024-09-21T11:51:46.574498+0000","last_fullsized":"2024-09-21T11:51:46.574498+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:51:08.137700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.137700+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:15.672786+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":536,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.580340+0000","last_change":"2024-09-21T11:50:34.667913+0000","last_active":"2024-09-21T11:51:46.580340+0000","last_peered":"2024-09-21T11:51:46.580340+0000","last_clean":"2024-09-21T11:51:46.580340+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:51:46.580340+0000","last_undegraded":"2024-09-21T11:51:46.580340+0000","last_fullsized":"2024-09-21T11:51:46.580340+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:34.667863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.667863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:37.637487+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":650,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.579556+0000","last_change":"2024-09-21T11:50:29.676678+0000","last_active":"2024-09-21T11:51:46.579556+0000","last_peered":"2024-09-21T11:51:46.579556+0000","last_clean":"2024-09-21T11:51:46.579556+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:51:46.579556+0000","last_undegraded":"2024-09-21T11:51:46.579556+0000","last_fullsized":"2024-09-21T11:51:46.579556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:50:29.676621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:29.676621+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:36.003404+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":508,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578621+0000","last_change":"2024-09-21T11:51:13.963333+0000","last_active":"2024-09-21T11:51:46.578621+0000","last_peered":"2024-09-21T11:51:46.578621+0000","last_clean":"2024-09-21T11:51:46.578621+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:51:46.578621+0000","last_undegraded":"2024-09-21T11:51:46.578621+0000","last_fullsized":"2024-09-21T11:51:46.578621+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:13.963251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:13.963251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:22.452371+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"295'12842","reported_seq":12951,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.081977+0000","last_change":"2024-09-21T11:49:08.609660+0000","last_active":"2024-09-21T11:51:51.081977+0000","last_peered":"2024-09-21T11:51:51.081977+0000","last_clean":"2024-09-21T11:51:51.081977+0000","last_became_active":"2024-09-21T11:49:08.609226+0000","last_became_peered":"2024-09-21T11:49:08.609226+0000","last_unstale":"2024-09-21T11:51:51.081977+0000","last_undegraded":"2024-09-21T11:51:51.081977+0000","last_fullsized":"2024-09-21T11:51:51.081977+0000","mapping_epoch":247,"log_start":"278'8000","ondisk_log_start":"278'8000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":4842,"log_dups_size":0,"ondisk_log_size":4842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:54.452806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199884800,"num_objects":3050,"num_object_clones":0,"num_object_copies":9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12842,"num_write_kb":508544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":567,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.579809+0000","last_change":"2024-09-21T11:51:45.114831+0000","last_active":"2024-09-21T11:51:46.579809+0000","last_peered":"2024-09-21T11:51:46.579809+0000","last_clean":"2024-09-21T11:51:46.579809+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:51:46.579809+0000","last_undegraded":"2024-09-21T11:51:46.579809+0000","last_fullsized":"2024-09-21T11:51:46.579809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:51:45.114143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:45.114143+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:12.367746+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":206,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.568462+0000","last_change":"2024-09-21T11:49:59.954903+0000","last_active":"2024-09-21T11:51:46.568462+0000","last_peered":"2024-09-21T11:51:46.568462+0000","last_clean":"2024-09-21T11:51:46.568462+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:51:46.568462+0000","last_undegraded":"2024-09-21T11:51:46.568462+0000","last_fullsized":"2024-09-21T11:51:46.568462+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:59.954835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:59.954835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:24.288696+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":496,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.570807+0000","last_change":"2024-09-21T11:48:10.144870+0000","last_active":"2024-09-21T11:51:46.570807+0000","last_peered":"2024-09-21T11:51:46.570807+0000","last_clean":"2024-09-21T11:51:46.570807+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:51:46.570807+0000","last_undegraded":"2024-09-21T11:51:46.570807+0000","last_fullsized":"2024-09-21T11:51:46.570807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:10.144810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:10.144810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.361694+0000","scrub_duration":77,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":206,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.567904+0000","last_change":"2024-09-21T11:50:36.027903+0000","last_active":"2024-09-21T11:51:46.567904+0000","last_peered":"2024-09-21T11:51:46.567904+0000","last_clean":"2024-09-21T11:51:46.567904+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:51:46.567904+0000","last_undegraded":"2024-09-21T11:51:46.567904+0000","last_fullsized":"2024-09-21T11:51:46.567904+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.027823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.027823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:48.339916+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"295'12843","reported_seq":13027,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.977919+0000","last_change":"2024-09-21T11:50:26.274463+0000","last_active":"2024-09-21T11:51:51.977919+0000","last_peered":"2024-09-21T11:51:51.977919+0000","last_clean":"2024-09-21T11:51:51.977919+0000","last_became_active":"2024-09-21T11:49:08.516355+0000","last_became_peered":"2024-09-21T11:49:08.516355+0000","last_unstale":"2024-09-21T11:51:51.977919+0000","last_undegraded":"2024-09-21T11:51:51.977919+0000","last_fullsized":"2024-09-21T11:51:51.977919+0000","mapping_epoch":247,"log_start":"277'6921","ondisk_log_start":"277'6921","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6771","last_scrub_stamp":"2024-09-21T11:50:26.274165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.274165+0000","objects_scrubbed":6698,"log_size":5922,"log_dups_size":0,"ondisk_log_size":5922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.909290+0000","scrub_duration":2483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189595648,"num_objects":2893,"num_object_clones":0,"num_object_copies":8679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12843,"num_write_kb":503552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":304,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:47.300045+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:51:47.300045+0000","last_peered":"2024-09-21T11:51:47.300045+0000","last_clean":"2024-09-21T11:51:47.300045+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:51:47.300045+0000","last_undegraded":"2024-09-21T11:51:47.300045+0000","last_fullsized":"2024-09-21T11:51:47.300045+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:07.053796+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":32,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.579933+0000","last_change":"2024-09-21T11:51:44.078378+0000","last_active":"2024-09-21T11:51:46.579933+0000","last_peered":"2024-09-21T11:51:46.579933+0000","last_clean":"2024-09-21T11:51:46.579933+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:51:46.579933+0000","last_undegraded":"2024-09-21T11:51:46.579933+0000","last_fullsized":"2024-09-21T11:51:46.579933+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:44.078299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.078299+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:45.213194+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":544,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.576385+0000","last_change":"2024-09-21T11:51:12.562026+0000","last_active":"2024-09-21T11:51:46.576385+0000","last_peered":"2024-09-21T11:51:46.576385+0000","last_clean":"2024-09-21T11:51:46.576385+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:51:46.576385+0000","last_undegraded":"2024-09-21T11:51:46.576385+0000","last_fullsized":"2024-09-21T11:51:46.576385+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:51:12.561961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.561961+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:19.756001+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":518,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.579240+0000","last_change":"2024-09-21T11:50:37.858401+0000","last_active":"2024-09-21T11:51:46.579240+0000","last_peered":"2024-09-21T11:51:46.579240+0000","last_clean":"2024-09-21T11:51:46.579240+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:51:46.579240+0000","last_undegraded":"2024-09-21T11:51:46.579240+0000","last_fullsized":"2024-09-21T11:51:46.579240+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:49.922881+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":493,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:49.357728+0000","last_change":"2024-09-21T11:51:49.357728+0000","last_active":"2024-09-21T11:51:49.357728+0000","last_peered":"2024-09-21T11:51:49.357728+0000","last_clean":"2024-09-21T11:51:49.357728+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:51:49.357728+0000","last_undegraded":"2024-09-21T11:51:49.357728+0000","last_fullsized":"2024-09-21T11:51:49.357728+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:49.357659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:49.357659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:50.134472+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"295'12975","reported_seq":13158,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.196977+0000","last_change":"2024-09-21T11:51:02.732433+0000","last_active":"2024-09-21T11:51:51.196977+0000","last_peered":"2024-09-21T11:51:51.196977+0000","last_clean":"2024-09-21T11:51:51.196977+0000","last_became_active":"2024-09-21T11:49:08.517758+0000","last_became_peered":"2024-09-21T11:49:08.517758+0000","last_unstale":"2024-09-21T11:51:51.196977+0000","last_undegraded":"2024-09-21T11:51:51.196977+0000","last_fullsized":"2024-09-21T11:51:51.196977+0000","mapping_epoch":247,"log_start":"277'7600","ondisk_log_start":"277'7600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"284'9390","last_scrub_stamp":"2024-09-21T11:51:02.732224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:51:02.732224+0000","objects_scrubbed":6621,"log_size":5375,"log_dups_size":0,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:14.953202+0000","scrub_duration":2062,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194052096,"num_objects":2961,"num_object_clones":0,"num_object_copies":8883,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12975,"num_write_kb":509952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":157,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573043+0000","last_change":"2024-09-21T11:51:41.006710+0000","last_active":"2024-09-21T11:51:46.573043+0000","last_peered":"2024-09-21T11:51:46.573043+0000","last_clean":"2024-09-21T11:51:46.573043+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:51:46.573043+0000","last_undegraded":"2024-09-21T11:51:46.573043+0000","last_fullsized":"2024-09-21T11:51:46.573043+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:51:41.006622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:41.006622+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:04.673833+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":218,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.569064+0000","last_change":"2024-09-21T11:51:43.346789+0000","last_active":"2024-09-21T11:51:46.569064+0000","last_peered":"2024-09-21T11:51:46.569064+0000","last_clean":"2024-09-21T11:51:46.569064+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:51:46.569064+0000","last_undegraded":"2024-09-21T11:51:46.569064+0000","last_fullsized":"2024-09-21T11:51:46.569064+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:43.346718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.346718+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:57.180696+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":538,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572661+0000","last_change":"2024-09-21T11:51:15.878501+0000","last_active":"2024-09-21T11:51:46.572661+0000","last_peered":"2024-09-21T11:51:46.572661+0000","last_clean":"2024-09-21T11:51:46.572661+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:51:46.572661+0000","last_undegraded":"2024-09-21T11:51:46.572661+0000","last_fullsized":"2024-09-21T11:51:46.572661+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:15.878417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.878417+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:17.080384+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":513,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578832+0000","last_change":"2024-09-21T11:49:47.008541+0000","last_active":"2024-09-21T11:51:46.578832+0000","last_peered":"2024-09-21T11:51:46.578832+0000","last_clean":"2024-09-21T11:51:46.578832+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:51:46.578832+0000","last_undegraded":"2024-09-21T11:51:46.578832+0000","last_fullsized":"2024-09-21T11:51:46.578832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.008369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.008369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:54.670178+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":508,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575402+0000","last_change":"2024-09-21T11:49:54.388131+0000","last_active":"2024-09-21T11:51:46.575402+0000","last_peered":"2024-09-21T11:51:46.575402+0000","last_clean":"2024-09-21T11:51:46.575402+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:51:46.575402+0000","last_undegraded":"2024-09-21T11:51:46.575402+0000","last_fullsized":"2024-09-21T11:51:46.575402+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.388068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.388068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:25.210516+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"295'13091","reported_seq":13200,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.972670+0000","last_change":"2024-09-21T11:49:08.515501+0000","last_active":"2024-09-21T11:51:51.972670+0000","last_peered":"2024-09-21T11:51:51.972670+0000","last_clean":"2024-09-21T11:51:51.972670+0000","last_became_active":"2024-09-21T11:49:08.515204+0000","last_became_peered":"2024-09-21T11:49:08.515204+0000","last_unstale":"2024-09-21T11:51:51.972670+0000","last_undegraded":"2024-09-21T11:51:51.972670+0000","last_fullsized":"2024-09-21T11:51:51.972670+0000","mapping_epoch":247,"log_start":"277'7173","ondisk_log_start":"277'7173","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5918,"log_dups_size":0,"ondisk_log_size":5918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:57.840535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197591040,"num_objects":3015,"num_object_clones":0,"num_object_copies":9045,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13091,"num_write_kb":515392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":586,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.585089+0000","last_change":"2024-09-21T11:51:43.069869+0000","last_active":"2024-09-21T11:51:46.585089+0000","last_peered":"2024-09-21T11:51:46.585089+0000","last_clean":"2024-09-21T11:51:46.585089+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:51:46.585089+0000","last_undegraded":"2024-09-21T11:51:46.585089+0000","last_fullsized":"2024-09-21T11:51:46.585089+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:51:43.069788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.069788+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:04.681540+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":528,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.579141+0000","last_change":"2024-09-21T11:51:20.858177+0000","last_active":"2024-09-21T11:51:46.579141+0000","last_peered":"2024-09-21T11:51:46.579141+0000","last_clean":"2024-09-21T11:51:46.579141+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:51:46.579141+0000","last_undegraded":"2024-09-21T11:51:46.579141+0000","last_fullsized":"2024-09-21T11:51:46.579141+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.858109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.858109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:43.341698+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1265,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:49.750205+0000","last_change":"2024-09-21T11:49:51.434402+0000","last_active":"2024-09-21T11:51:49.750205+0000","last_peered":"2024-09-21T11:51:49.750205+0000","last_clean":"2024-09-21T11:51:49.750205+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:51:49.750205+0000","last_undegraded":"2024-09-21T11:51:49.750205+0000","last_fullsized":"2024-09-21T11:51:49.750205+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:51.434333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:49:51.434333+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:27.193688+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":702,"num_read_kb":697,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":526,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578666+0000","last_change":"2024-09-21T11:51:17.645920+0000","last_active":"2024-09-21T11:51:46.578666+0000","last_peered":"2024-09-21T11:51:46.578666+0000","last_clean":"2024-09-21T11:51:46.578666+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:51:46.578666+0000","last_undegraded":"2024-09-21T11:51:46.578666+0000","last_fullsized":"2024-09-21T11:51:46.578666+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.645823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.645823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:32.277022+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":214,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.568301+0000","last_change":"2024-09-21T11:51:12.151470+0000","last_active":"2024-09-21T11:51:46.568301+0000","last_peered":"2024-09-21T11:51:46.568301+0000","last_clean":"2024-09-21T11:51:46.568301+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:51:46.568301+0000","last_undegraded":"2024-09-21T11:51:46.568301+0000","last_fullsized":"2024-09-21T11:51:46.568301+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:12.151404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.151404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:30.877629+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"295'12902","reported_seq":13088,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:52.174564+0000","last_change":"2024-09-21T11:50:26.726878+0000","last_active":"2024-09-21T11:51:52.174564+0000","last_peered":"2024-09-21T11:51:52.174564+0000","last_clean":"2024-09-21T11:51:52.174564+0000","last_became_active":"2024-09-21T11:49:08.516627+0000","last_became_peered":"2024-09-21T11:49:08.516627+0000","last_unstale":"2024-09-21T11:51:52.174564+0000","last_undegraded":"2024-09-21T11:51:52.174564+0000","last_fullsized":"2024-09-21T11:51:52.174564+0000","mapping_epoch":247,"log_start":"278'8000","ondisk_log_start":"278'8000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6809","last_scrub_stamp":"2024-09-21T11:50:26.726538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:26.726538+0000","objects_scrubbed":6753,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:53.175169+0000","scrub_duration":2384,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":193855488,"num_objects":2958,"num_object_clones":0,"num_object_copies":8874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12902,"num_write_kb":507520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":699,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.571324+0000","last_change":"2024-09-21T11:50:29.389640+0000","last_active":"2024-09-21T11:51:46.571324+0000","last_peered":"2024-09-21T11:51:46.571324+0000","last_clean":"2024-09-21T11:51:46.571324+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:51:46.571324+0000","last_undegraded":"2024-09-21T11:51:46.571324+0000","last_fullsized":"2024-09-21T11:51:46.571324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:29.389555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:29.389555+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:32.354365+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":521,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578386+0000","last_change":"2024-09-21T11:50:59.680952+0000","last_active":"2024-09-21T11:51:46.578386+0000","last_peered":"2024-09-21T11:51:46.578386+0000","last_clean":"2024-09-21T11:51:46.578386+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:51:46.578386+0000","last_undegraded":"2024-09-21T11:51:46.578386+0000","last_fullsized":"2024-09-21T11:51:46.578386+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:59.680755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:59.680755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:08.366731+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":575,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573701+0000","last_change":"2024-09-21T11:50:39.362320+0000","last_active":"2024-09-21T11:51:46.573701+0000","last_peered":"2024-09-21T11:51:46.573701+0000","last_clean":"2024-09-21T11:51:46.573701+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:51:46.573701+0000","last_undegraded":"2024-09-21T11:51:46.573701+0000","last_fullsized":"2024-09-21T11:51:46.573701+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:50:39.361647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.361647+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.251982+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":520,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574364+0000","last_change":"2024-09-21T11:51:18.318095+0000","last_active":"2024-09-21T11:51:46.574364+0000","last_peered":"2024-09-21T11:51:46.574364+0000","last_clean":"2024-09-21T11:51:46.574364+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:51:46.574364+0000","last_undegraded":"2024-09-21T11:51:46.574364+0000","last_fullsized":"2024-09-21T11:51:46.574364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.317950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.317950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:41.186001+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":508,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.607389+0000","last_change":"2024-09-21T11:51:46.607389+0000","last_active":"2024-09-21T11:51:46.607389+0000","last_peered":"2024-09-21T11:51:46.607389+0000","last_clean":"2024-09-21T11:51:46.607389+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:51:46.607389+0000","last_undegraded":"2024-09-21T11:51:46.607389+0000","last_fullsized":"2024-09-21T11:51:46.607389+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:46.607326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:46.607326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:02.049939+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"295'12695","reported_seq":12885,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.085128+0000","last_change":"2024-09-21T11:50:37.410553+0000","last_active":"2024-09-21T11:51:51.085128+0000","last_peered":"2024-09-21T11:51:51.085128+0000","last_clean":"2024-09-21T11:51:51.085128+0000","last_became_active":"2024-09-21T11:49:08.519011+0000","last_became_peered":"2024-09-21T11:49:08.519011+0000","last_unstale":"2024-09-21T11:51:51.085128+0000","last_undegraded":"2024-09-21T11:51:51.085128+0000","last_fullsized":"2024-09-21T11:51:51.085128+0000","mapping_epoch":247,"log_start":"278'7900","ondisk_log_start":"278'7900","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7398","last_scrub_stamp":"2024-09-21T11:50:37.410305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.410305+0000","objects_scrubbed":7345,"log_size":4795,"log_dups_size":0,"ondisk_log_size":4795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2058,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194052096,"num_objects":2961,"num_object_clones":0,"num_object_copies":8883,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12695,"num_write_kb":500992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":530,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.571320+0000","last_change":"2024-09-21T11:51:43.526788+0000","last_active":"2024-09-21T11:51:46.571320+0000","last_peered":"2024-09-21T11:51:46.571320+0000","last_clean":"2024-09-21T11:51:46.571320+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:51:46.571320+0000","last_undegraded":"2024-09-21T11:51:46.571320+0000","last_fullsized":"2024-09-21T11:51:46.571320+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:37.352834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.352834+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572490+0000","last_change":"2024-09-21T11:51:43.532348+0000","last_active":"2024-09-21T11:51:46.572490+0000","last_peered":"2024-09-21T11:51:46.572490+0000","last_clean":"2024-09-21T11:51:46.572490+0000","last_became_active":"2024-09-21T11:51:43.532027+0000","last_became_peered":"2024-09-21T11:51:43.532027+0000","last_unstale":"2024-09-21T11:51:46.572490+0000","last_undegraded":"2024-09-21T11:51:46.572490+0000","last_fullsized":"2024-09-21T11:51:46.572490+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:06.912982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:06.912982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:12.292897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":504,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578291+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:51:46.578291+0000","last_peered":"2024-09-21T11:51:46.578291+0000","last_clean":"2024-09-21T11:51:46.578291+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:51:46.578291+0000","last_undegraded":"2024-09-21T11:51:46.578291+0000","last_fullsized":"2024-09-21T11:51:46.578291+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":508,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575353+0000","last_change":"2024-09-21T11:51:18.664721+0000","last_active":"2024-09-21T11:51:46.575353+0000","last_peered":"2024-09-21T11:51:46.575353+0000","last_clean":"2024-09-21T11:51:46.575353+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:51:46.575353+0000","last_undegraded":"2024-09-21T11:51:46.575353+0000","last_fullsized":"2024-09-21T11:51:46.575353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.664635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.664635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:36.173356+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"295'12798","reported_seq":12991,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.266982+0000","last_change":"2024-09-21T11:50:39.224005+0000","last_active":"2024-09-21T11:51:51.266982+0000","last_peered":"2024-09-21T11:51:51.266982+0000","last_clean":"2024-09-21T11:51:51.266982+0000","last_became_active":"2024-09-21T11:49:08.748155+0000","last_became_peered":"2024-09-21T11:49:08.748155+0000","last_unstale":"2024-09-21T11:51:51.266982+0000","last_undegraded":"2024-09-21T11:51:51.266982+0000","last_fullsized":"2024-09-21T11:51:51.266982+0000","mapping_epoch":247,"log_start":"278'8100","ondisk_log_start":"278'8100","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7577","last_scrub_stamp":"2024-09-21T11:50:39.223848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:39.223848+0000","objects_scrubbed":7518,"log_size":4698,"log_dups_size":0,"ondisk_log_size":4698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.351968+0000","scrub_duration":2220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192282624,"num_objects":2934,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":503424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":601,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573441+0000","last_change":"2024-09-21T11:51:43.533280+0000","last_active":"2024-09-21T11:51:46.573441+0000","last_peered":"2024-09-21T11:51:46.573441+0000","last_clean":"2024-09-21T11:51:46.573441+0000","last_became_active":"2024-09-21T11:51:43.525941+0000","last_became_peered":"2024-09-21T11:51:43.525941+0000","last_unstale":"2024-09-21T11:51:46.573441+0000","last_undegraded":"2024-09-21T11:51:46.573441+0000","last_fullsized":"2024-09-21T11:51:46.573441+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:50:36.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.385902+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:58.047958+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":542,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.571195+0000","last_change":"2024-09-21T11:51:14.312046+0000","last_active":"2024-09-21T11:51:46.571195+0000","last_peered":"2024-09-21T11:51:46.571195+0000","last_clean":"2024-09-21T11:51:46.571195+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:51:46.571195+0000","last_undegraded":"2024-09-21T11:51:46.571195+0000","last_fullsized":"2024-09-21T11:51:46.571195+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:14.311953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:14.311953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:25.751618+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":206,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.568266+0000","last_change":"2024-09-21T11:51:16.151389+0000","last_active":"2024-09-21T11:51:46.568266+0000","last_peered":"2024-09-21T11:51:46.568266+0000","last_clean":"2024-09-21T11:51:46.568266+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:51:46.568266+0000","last_undegraded":"2024-09-21T11:51:46.568266+0000","last_fullsized":"2024-09-21T11:51:46.568266+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:16.151330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:16.151330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:43.557799+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":517,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:47.944684+0000","last_change":"2024-09-21T11:51:47.944684+0000","last_active":"2024-09-21T11:51:47.944684+0000","last_peered":"2024-09-21T11:51:47.944684+0000","last_clean":"2024-09-21T11:51:47.944684+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:51:47.944684+0000","last_undegraded":"2024-09-21T11:51:47.944684+0000","last_fullsized":"2024-09-21T11:51:47.944684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:47.944609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.944609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:56.751528+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"295'13027","reported_seq":13209,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:52.218943+0000","last_change":"2024-09-21T11:51:05.135848+0000","last_active":"2024-09-21T11:51:52.218943+0000","last_peered":"2024-09-21T11:51:52.218943+0000","last_clean":"2024-09-21T11:51:52.218943+0000","last_became_active":"2024-09-21T11:49:08.517011+0000","last_became_peered":"2024-09-21T11:49:08.517011+0000","last_unstale":"2024-09-21T11:51:52.218943+0000","last_undegraded":"2024-09-21T11:51:52.218943+0000","last_fullsized":"2024-09-21T11:51:52.218943+0000","mapping_epoch":247,"log_start":"278'8100","ondisk_log_start":"278'8100","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"284'9566","last_scrub_stamp":"2024-09-21T11:51:05.135728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.135728+0000","objects_scrubbed":6439,"log_size":4927,"log_dups_size":0,"ondisk_log_size":4927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:35.110511+0000","scrub_duration":2470,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189333504,"num_objects":2889,"num_object_clones":0,"num_object_copies":8667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13027,"num_write_kb":509312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":515,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575591+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:51:46.575591+0000","last_peered":"2024-09-21T11:51:46.575591+0000","last_clean":"2024-09-21T11:51:46.575591+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:51:46.575591+0000","last_undegraded":"2024-09-21T11:51:46.575591+0000","last_fullsized":"2024-09-21T11:51:46.575591+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":505,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573009+0000","last_change":"2024-09-21T11:50:35.410162+0000","last_active":"2024-09-21T11:51:46.573009+0000","last_peered":"2024-09-21T11:51:46.573009+0000","last_clean":"2024-09-21T11:51:46.573009+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:51:46.573009+0000","last_undegraded":"2024-09-21T11:51:46.573009+0000","last_fullsized":"2024-09-21T11:51:46.573009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:35.410095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.410095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":672,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573387+0000","last_change":"2024-09-21T11:50:02.975387+0000","last_active":"2024-09-21T11:51:46.573387+0000","last_peered":"2024-09-21T11:51:46.573387+0000","last_clean":"2024-09-21T11:51:46.573387+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:51:46.573387+0000","last_undegraded":"2024-09-21T11:51:46.573387+0000","last_fullsized":"2024-09-21T11:51:46.573387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:50:02.975313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.975313+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:26.845768+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":326,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:50.279512+0000","last_change":"2024-09-21T11:51:50.279512+0000","last_active":"2024-09-21T11:51:50.279512+0000","last_peered":"2024-09-21T11:51:50.279512+0000","last_clean":"2024-09-21T11:51:50.279512+0000","last_became_active":"2024-09-21T11:51:43.543072+0000","last_became_peered":"2024-09-21T11:51:43.543072+0000","last_unstale":"2024-09-21T11:51:50.279512+0000","last_undegraded":"2024-09-21T11:51:50.279512+0000","last_fullsized":"2024-09-21T11:51:50.279512+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:50.279238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:50.279238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:50.684532+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"295'12875","reported_seq":13059,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.202155+0000","last_change":"2024-09-21T11:50:24.709591+0000","last_active":"2024-09-21T11:51:51.202155+0000","last_peered":"2024-09-21T11:51:51.202155+0000","last_clean":"2024-09-21T11:51:51.202155+0000","last_became_active":"2024-09-21T11:49:08.748171+0000","last_became_peered":"2024-09-21T11:49:08.748171+0000","last_unstale":"2024-09-21T11:51:51.202155+0000","last_undegraded":"2024-09-21T11:51:51.202155+0000","last_fullsized":"2024-09-21T11:51:51.202155+0000","mapping_epoch":247,"log_start":"277'7500","ondisk_log_start":"277'7500","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'6768","last_scrub_stamp":"2024-09-21T11:50:24.709386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.709386+0000","objects_scrubbed":6700,"log_size":5375,"log_dups_size":0,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:27.666178+0000","scrub_duration":2008,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":193003520,"num_objects":2945,"num_object_clones":0,"num_object_copies":8835,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12875,"num_write_kb":506240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":559,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.579712+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:51:46.579712+0000","last_peered":"2024-09-21T11:51:46.579712+0000","last_clean":"2024-09-21T11:51:46.579712+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:51:46.579712+0000","last_undegraded":"2024-09-21T11:51:46.579712+0000","last_fullsized":"2024-09-21T11:51:46.579712+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.887029+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":528,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.576456+0000","last_change":"2024-09-21T11:51:03.627990+0000","last_active":"2024-09-21T11:51:46.576456+0000","last_peered":"2024-09-21T11:51:46.576456+0000","last_clean":"2024-09-21T11:51:46.576456+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:51:46.576456+0000","last_undegraded":"2024-09-21T11:51:46.576456+0000","last_fullsized":"2024-09-21T11:51:46.576456+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:03.627923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:03.627923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:14.026807+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":505,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572633+0000","last_change":"2024-09-21T11:48:49.993307+0000","last_active":"2024-09-21T11:51:46.572633+0000","last_peered":"2024-09-21T11:51:46.572633+0000","last_clean":"2024-09-21T11:51:46.572633+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:51:46.572633+0000","last_undegraded":"2024-09-21T11:51:46.572633+0000","last_fullsized":"2024-09-21T11:51:46.572633+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.993230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.993230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:22.863750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":492,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:50.032271+0000","last_change":"2024-09-21T11:51:50.032271+0000","last_active":"2024-09-21T11:51:50.032271+0000","last_peered":"2024-09-21T11:51:50.032271+0000","last_clean":"2024-09-21T11:51:50.032271+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:51:50.032271+0000","last_undegraded":"2024-09-21T11:51:50.032271+0000","last_fullsized":"2024-09-21T11:51:50.032271+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:50.032193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:50.032193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:18.671132+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"295'13043","reported_seq":13233,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.220763+0000","last_change":"2024-09-21T11:50:32.751942+0000","last_active":"2024-09-21T11:51:51.220763+0000","last_peered":"2024-09-21T11:51:51.220763+0000","last_clean":"2024-09-21T11:51:51.220763+0000","last_became_active":"2024-09-21T11:49:08.516975+0000","last_became_peered":"2024-09-21T11:49:08.516975+0000","last_unstale":"2024-09-21T11:51:51.220763+0000","last_undegraded":"2024-09-21T11:51:51.220763+0000","last_fullsized":"2024-09-21T11:51:51.220763+0000","mapping_epoch":247,"log_start":"277'7600","ondisk_log_start":"277'7600","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7356","last_scrub_stamp":"2024-09-21T11:50:32.751687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.751687+0000","objects_scrubbed":7220,"log_size":5443,"log_dups_size":0,"ondisk_log_size":5443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.722147+0000","scrub_duration":3036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201392128,"num_objects":3073,"num_object_clones":0,"num_object_copies":9219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13043,"num_write_kb":515712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":633,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.570985+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:51:46.570985+0000","last_peered":"2024-09-21T11:51:46.570985+0000","last_clean":"2024-09-21T11:51:46.570985+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:51:46.570985+0000","last_undegraded":"2024-09-21T11:51:46.570985+0000","last_fullsized":"2024-09-21T11:51:46.570985+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":520,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575873+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:51:46.575873+0000","last_peered":"2024-09-21T11:51:46.575873+0000","last_clean":"2024-09-21T11:51:46.575873+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:51:46.575873+0000","last_undegraded":"2024-09-21T11:51:46.575873+0000","last_fullsized":"2024-09-21T11:51:46.575873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:30.208249+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":529,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:48.941202+0000","last_change":"2024-09-21T11:51:48.941202+0000","last_active":"2024-09-21T11:51:48.941202+0000","last_peered":"2024-09-21T11:51:48.941202+0000","last_clean":"2024-09-21T11:51:48.941202+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:51:48.941202+0000","last_undegraded":"2024-09-21T11:51:48.941202+0000","last_fullsized":"2024-09-21T11:51:48.941202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:48.941128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:48.941128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:52.822804+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":508,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573613+0000","last_change":"2024-09-21T11:50:34.385662+0000","last_active":"2024-09-21T11:51:46.573613+0000","last_peered":"2024-09-21T11:51:46.573613+0000","last_clean":"2024-09-21T11:51:46.573613+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:51:46.573613+0000","last_undegraded":"2024-09-21T11:51:46.573613+0000","last_fullsized":"2024-09-21T11:51:46.573613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:34.385563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.385563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"295'12753","reported_seq":12940,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.266577+0000","last_change":"2024-09-21T11:50:30.668171+0000","last_active":"2024-09-21T11:51:51.266577+0000","last_peered":"2024-09-21T11:51:51.266577+0000","last_clean":"2024-09-21T11:51:51.266577+0000","last_became_active":"2024-09-21T11:49:08.609829+0000","last_became_peered":"2024-09-21T11:49:08.609829+0000","last_unstale":"2024-09-21T11:51:51.266577+0000","last_undegraded":"2024-09-21T11:51:51.266577+0000","last_fullsized":"2024-09-21T11:51:51.266577+0000","mapping_epoch":247,"log_start":"278'8000","ondisk_log_start":"278'8000","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"277'7027","last_scrub_stamp":"2024-09-21T11:50:30.668043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:50:30.668043+0000","objects_scrubbed":6939,"log_size":4753,"log_dups_size":0,"ondisk_log_size":4753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:54.716375+0000","scrub_duration":2651,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191692800,"num_objects":2925,"num_object_clones":0,"num_object_copies":8775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12753,"num_write_kb":501696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":555,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.568077+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:51:46.568077+0000","last_peered":"2024-09-21T11:51:46.568077+0000","last_clean":"2024-09-21T11:51:46.568077+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:51:46.568077+0000","last_undegraded":"2024-09-21T11:51:46.568077+0000","last_fullsized":"2024-09-21T11:51:46.568077+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":550,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574559+0000","last_change":"2024-09-21T11:49:08.611218+0000","last_active":"2024-09-21T11:51:46.574559+0000","last_peered":"2024-09-21T11:51:46.574559+0000","last_clean":"2024-09-21T11:51:46.574559+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:51:46.574559+0000","last_undegraded":"2024-09-21T11:51:46.574559+0000","last_fullsized":"2024-09-21T11:51:46.574559+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.611159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.611159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":535,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573753+0000","last_change":"2024-09-21T11:49:44.985594+0000","last_active":"2024-09-21T11:51:46.573753+0000","last_peered":"2024-09-21T11:51:46.573753+0000","last_clean":"2024-09-21T11:51:46.573753+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:51:46.573753+0000","last_undegraded":"2024-09-21T11:51:46.573753+0000","last_fullsized":"2024-09-21T11:51:46.573753+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:44.985234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:44.985234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:51.744078+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":523,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572766+0000","last_change":"2024-09-21T11:51:09.660956+0000","last_active":"2024-09-21T11:51:46.572766+0000","last_peered":"2024-09-21T11:51:46.572766+0000","last_clean":"2024-09-21T11:51:46.572766+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:51:46.572766+0000","last_undegraded":"2024-09-21T11:51:46.572766+0000","last_fullsized":"2024-09-21T11:51:46.572766+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:09.660880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.660880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:28.267025+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"295'13092","reported_seq":13201,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:51.930603+0000","last_change":"2024-09-21T11:49:08.517117+0000","last_active":"2024-09-21T11:51:51.930603+0000","last_peered":"2024-09-21T11:51:51.930603+0000","last_clean":"2024-09-21T11:51:51.930603+0000","last_became_active":"2024-09-21T11:49:08.516494+0000","last_became_peered":"2024-09-21T11:49:08.516494+0000","last_unstale":"2024-09-21T11:51:51.930603+0000","last_undegraded":"2024-09-21T11:51:51.930603+0000","last_fullsized":"2024-09-21T11:51:51.930603+0000","mapping_epoch":247,"log_start":"277'7156","ondisk_log_start":"277'7156","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:49:07.498602+0000","last_clean_scrub_stamp":"2024-09-21T11:49:07.498602+0000","objects_scrubbed":0,"log_size":5936,"log_dups_size":0,"ondisk_log_size":5936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:56.866266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201326592,"num_objects":3072,"num_object_clones":0,"num_object_copies":9216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13092,"num_write_kb":517248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":551,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575642+0000","last_change":"2024-09-21T11:51:19.638791+0000","last_active":"2024-09-21T11:51:46.575642+0000","last_peered":"2024-09-21T11:51:46.575642+0000","last_clean":"2024-09-21T11:51:46.575642+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:51:46.575642+0000","last_undegraded":"2024-09-21T11:51:46.575642+0000","last_fullsized":"2024-09-21T11:51:46.575642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:51:19.638734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:19.638734+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:36.977249+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":540,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575652+0000","last_change":"2024-09-21T11:50:06.398255+0000","last_active":"2024-09-21T11:51:46.575652+0000","last_peered":"2024-09-21T11:51:46.575652+0000","last_clean":"2024-09-21T11:51:46.575652+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:51:46.575652+0000","last_undegraded":"2024-09-21T11:51:46.575652+0000","last_fullsized":"2024-09-21T11:51:46.575652+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:50:06.398195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.398195+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:38.228164+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":655,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574004+0000","last_change":"2024-09-21T11:47:20.387576+0000","last_active":"2024-09-21T11:51:46.574004+0000","last_peered":"2024-09-21T11:51:46.574004+0000","last_clean":"2024-09-21T11:51:46.574004+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:51:46.574004+0000","last_undegraded":"2024-09-21T11:51:46.574004+0000","last_fullsized":"2024-09-21T11:51:46.574004+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:47:20.387495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:47:20.387495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":500,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.576059+0000","last_change":"2024-09-21T11:49:56.669610+0000","last_active":"2024-09-21T11:51:46.576059+0000","last_peered":"2024-09-21T11:51:46.576059+0000","last_clean":"2024-09-21T11:51:46.576059+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:51:46.576059+0000","last_undegraded":"2024-09-21T11:51:46.576059+0000","last_fullsized":"2024-09-21T11:51:46.576059+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:56.669547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:56.669547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:24.651050+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":528,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572529+0000","last_change":"2024-09-21T11:50:31.417131+0000","last_active":"2024-09-21T11:51:46.572529+0000","last_peered":"2024-09-21T11:51:46.572529+0000","last_clean":"2024-09-21T11:51:46.572529+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:51:46.572529+0000","last_undegraded":"2024-09-21T11:51:46.572529+0000","last_fullsized":"2024-09-21T11:51:46.572529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.416857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.416857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:37.430140+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":548,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574116+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:51:46.574116+0000","last_peered":"2024-09-21T11:51:46.574116+0000","last_clean":"2024-09-21T11:51:46.574116+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:51:46.574116+0000","last_undegraded":"2024-09-21T11:51:46.574116+0000","last_fullsized":"2024-09-21T11:51:46.574116+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:50.460407+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":533,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578036+0000","last_change":"2024-09-21T11:50:58.694445+0000","last_active":"2024-09-21T11:51:46.578036+0000","last_peered":"2024-09-21T11:51:46.578036+0000","last_clean":"2024-09-21T11:51:46.578036+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:51:46.578036+0000","last_undegraded":"2024-09-21T11:51:46.578036+0000","last_fullsized":"2024-09-21T11:51:46.578036+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:58.694382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:58.694382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:07.087368+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":515,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575310+0000","last_change":"2024-09-21T11:49:43.464860+0000","last_active":"2024-09-21T11:51:46.575310+0000","last_peered":"2024-09-21T11:51:46.575310+0000","last_clean":"2024-09-21T11:51:46.575310+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:51:46.575310+0000","last_undegraded":"2024-09-21T11:51:46.575310+0000","last_fullsized":"2024-09-21T11:51:46.575310+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:43.464792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:43.464792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:24.209462+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":549,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.585177+0000","last_change":"2024-09-21T11:51:09.955823+0000","last_active":"2024-09-21T11:51:46.585177+0000","last_peered":"2024-09-21T11:51:46.585177+0000","last_clean":"2024-09-21T11:51:46.585177+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:51:46.585177+0000","last_undegraded":"2024-09-21T11:51:46.585177+0000","last_fullsized":"2024-09-21T11:51:46.585177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:09.955744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.955744+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:15.698416+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":520,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574109+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:51:46.574109+0000","last_peered":"2024-09-21T11:51:46.574109+0000","last_clean":"2024-09-21T11:51:46.574109+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:51:46.574109+0000","last_undegraded":"2024-09-21T11:51:46.574109+0000","last_fullsized":"2024-09-21T11:51:46.574109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:51.705484+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":526,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573908+0000","last_change":"2024-09-21T11:49:50.191966+0000","last_active":"2024-09-21T11:51:46.573908+0000","last_peered":"2024-09-21T11:51:46.573908+0000","last_clean":"2024-09-21T11:51:46.573908+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:51:46.573908+0000","last_undegraded":"2024-09-21T11:51:46.573908+0000","last_fullsized":"2024-09-21T11:51:46.573908+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:26.610988+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":517,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573057+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:51:46.573057+0000","last_peered":"2024-09-21T11:51:46.573057+0000","last_clean":"2024-09-21T11:51:46.573057+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:51:46.573057+0000","last_undegraded":"2024-09-21T11:51:46.573057+0000","last_fullsized":"2024-09-21T11:51:46.573057+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:55.647487+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":532,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:49.643640+0000","last_change":"2024-09-21T11:51:49.643640+0000","last_active":"2024-09-21T11:51:49.643640+0000","last_peered":"2024-09-21T11:51:49.643640+0000","last_clean":"2024-09-21T11:51:49.643640+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:51:49.643640+0000","last_undegraded":"2024-09-21T11:51:49.643640+0000","last_fullsized":"2024-09-21T11:51:49.643640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:49.643424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:49.643424+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:54.847960+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":528,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.579730+0000","last_change":"2024-09-21T11:50:58.890596+0000","last_active":"2024-09-21T11:51:46.579730+0000","last_peered":"2024-09-21T11:51:46.579730+0000","last_clean":"2024-09-21T11:51:46.579730+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:51:46.579730+0000","last_undegraded":"2024-09-21T11:51:46.579730+0000","last_fullsized":"2024-09-21T11:51:46.579730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:58.890528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:58.890528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:23.788125+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":648,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:50.515984+0000","last_change":"2024-09-21T11:51:08.629172+0000","last_active":"2024-09-21T11:51:50.515984+0000","last_peered":"2024-09-21T11:51:50.515984+0000","last_clean":"2024-09-21T11:51:50.515984+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:51:50.515984+0000","last_undegraded":"2024-09-21T11:51:50.515984+0000","last_fullsized":"2024-09-21T11:51:50.515984+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:08.628986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.628986+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:27.310184+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573147+0000","last_change":"2024-09-21T11:51:43.531318+0000","last_active":"2024-09-21T11:51:46.573147+0000","last_peered":"2024-09-21T11:51:46.573147+0000","last_clean":"2024-09-21T11:51:46.573147+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:51:46.573147+0000","last_undegraded":"2024-09-21T11:51:46.573147+0000","last_fullsized":"2024-09-21T11:51:46.573147+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.826834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.826834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:49.809107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":569,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575595+0000","last_change":"2024-09-21T11:50:37.433622+0000","last_active":"2024-09-21T11:51:46.575595+0000","last_peered":"2024-09-21T11:51:46.575595+0000","last_clean":"2024-09-21T11:51:46.575595+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:51:46.575595+0000","last_undegraded":"2024-09-21T11:51:46.575595+0000","last_fullsized":"2024-09-21T11:51:46.575595+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:50:37.432212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.432212+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":512,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574337+0000","last_change":"2024-09-21T11:49:58.400078+0000","last_active":"2024-09-21T11:51:46.574337+0000","last_peered":"2024-09-21T11:51:46.574337+0000","last_clean":"2024-09-21T11:51:46.574337+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:51:46.574337+0000","last_undegraded":"2024-09-21T11:51:46.574337+0000","last_fullsized":"2024-09-21T11:51:46.574337+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.400013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.400013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:26.182640+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":685,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578925+0000","last_change":"2024-09-21T11:50:30.703282+0000","last_active":"2024-09-21T11:51:46.578925+0000","last_peered":"2024-09-21T11:51:46.578925+0000","last_clean":"2024-09-21T11:51:46.578925+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:51:46.578925+0000","last_undegraded":"2024-09-21T11:51:46.578925+0000","last_fullsized":"2024-09-21T11:51:46.578925+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:50:30.703227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:30.703227+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:57.509405+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":515,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.584743+0000","last_change":"2024-09-21T11:50:02.782565+0000","last_active":"2024-09-21T11:51:46.584743+0000","last_peered":"2024-09-21T11:51:46.584743+0000","last_clean":"2024-09-21T11:51:46.584743+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:51:46.584743+0000","last_undegraded":"2024-09-21T11:51:46.584743+0000","last_fullsized":"2024-09-21T11:51:46.584743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.782475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.782475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:31.978204+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":565,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.571102+0000","last_change":"2024-09-21T11:51:43.533694+0000","last_active":"2024-09-21T11:51:46.571102+0000","last_peered":"2024-09-21T11:51:46.571102+0000","last_clean":"2024-09-21T11:51:46.571102+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:51:46.571102+0000","last_undegraded":"2024-09-21T11:51:46.571102+0000","last_fullsized":"2024-09-21T11:51:46.571102+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574565+0000","last_change":"2024-09-21T11:51:43.523598+0000","last_active":"2024-09-21T11:51:46.574565+0000","last_peered":"2024-09-21T11:51:46.574565+0000","last_clean":"2024-09-21T11:51:46.574565+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:51:46.574565+0000","last_undegraded":"2024-09-21T11:51:46.574565+0000","last_fullsized":"2024-09-21T11:51:46.574565+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:45:36.642282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:45:36.642282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":214,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574797+0000","last_change":"2024-09-21T11:51:10.143792+0000","last_active":"2024-09-21T11:51:46.574797+0000","last_peered":"2024-09-21T11:51:46.574797+0000","last_clean":"2024-09-21T11:51:46.574797+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:51:46.574797+0000","last_undegraded":"2024-09-21T11:51:46.574797+0000","last_fullsized":"2024-09-21T11:51:46.574797+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.141936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.141936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:18.171087+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":493,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.576359+0000","last_change":"2024-09-21T11:50:02.329020+0000","last_active":"2024-09-21T11:51:46.576359+0000","last_peered":"2024-09-21T11:51:46.576359+0000","last_clean":"2024-09-21T11:51:46.576359+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:51:46.576359+0000","last_undegraded":"2024-09-21T11:51:46.576359+0000","last_fullsized":"2024-09-21T11:51:46.576359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.328926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.328926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:38.528250+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":225,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574652+0000","last_change":"2024-09-21T11:51:15.200506+0000","last_active":"2024-09-21T11:51:46.574652+0000","last_peered":"2024-09-21T11:51:46.574652+0000","last_clean":"2024-09-21T11:51:46.574652+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:51:46.574652+0000","last_undegraded":"2024-09-21T11:51:46.574652+0000","last_fullsized":"2024-09-21T11:51:46.574652+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:15.200430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.200430+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:29.878537+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":519,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.575695+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:51:46.575695+0000","last_peered":"2024-09-21T11:51:46.575695+0000","last_clean":"2024-09-21T11:51:46.575695+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:51:46.575695+0000","last_undegraded":"2024-09-21T11:51:46.575695+0000","last_fullsized":"2024-09-21T11:51:46.575695+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.443409+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":527,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572887+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:51:46.572887+0000","last_peered":"2024-09-21T11:51:46.572887+0000","last_clean":"2024-09-21T11:51:46.572887+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:51:46.572887+0000","last_undegraded":"2024-09-21T11:51:46.572887+0000","last_fullsized":"2024-09-21T11:51:46.572887+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":539,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578354+0000","last_change":"2024-09-21T11:51:16.962933+0000","last_active":"2024-09-21T11:51:46.578354+0000","last_peered":"2024-09-21T11:51:46.578354+0000","last_clean":"2024-09-21T11:51:46.578354+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:51:46.578354+0000","last_undegraded":"2024-09-21T11:51:46.578354+0000","last_fullsized":"2024-09-21T11:51:46.578354+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:51:16.962860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:16.962860+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:37.516532+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":548,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.576787+0000","last_change":"2024-09-21T11:51:19.708885+0000","last_active":"2024-09-21T11:51:46.576787+0000","last_peered":"2024-09-21T11:51:46.576787+0000","last_clean":"2024-09-21T11:51:46.576787+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:51:46.576787+0000","last_undegraded":"2024-09-21T11:51:46.576787+0000","last_fullsized":"2024-09-21T11:51:46.576787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:51:19.708641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:19.708641+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:21.239576+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":533,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.578131+0000","last_change":"2024-09-21T11:51:01.690172+0000","last_active":"2024-09-21T11:51:46.578131+0000","last_peered":"2024-09-21T11:51:46.578131+0000","last_clean":"2024-09-21T11:51:46.578131+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:51:46.578131+0000","last_undegraded":"2024-09-21T11:51:46.578131+0000","last_fullsized":"2024-09-21T11:51:46.578131+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.686885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.686885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:08.926819+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":504,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573205+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:51:46.573205+0000","last_peered":"2024-09-21T11:51:46.573205+0000","last_clean":"2024-09-21T11:51:46.573205+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:51:46.573205+0000","last_undegraded":"2024-09-21T11:51:46.573205+0000","last_fullsized":"2024-09-21T11:51:46.573205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:39.200896+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":509,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.572969+0000","last_change":"2024-09-21T11:51:05.711964+0000","last_active":"2024-09-21T11:51:46.572969+0000","last_peered":"2024-09-21T11:51:46.572969+0000","last_clean":"2024-09-21T11:51:46.572969+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:51:46.572969+0000","last_undegraded":"2024-09-21T11:51:46.572969+0000","last_fullsized":"2024-09-21T11:51:46.572969+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:05.711768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.711768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:19.424825+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":560,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573737+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:51:46.573737+0000","last_peered":"2024-09-21T11:51:46.573737+0000","last_clean":"2024-09-21T11:51:46.573737+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:51:46.573737+0000","last_undegraded":"2024-09-21T11:51:46.573737+0000","last_fullsized":"2024-09-21T11:51:46.573737+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":511,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.584636+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:51:46.584636+0000","last_peered":"2024-09-21T11:51:46.584636+0000","last_clean":"2024-09-21T11:51:46.584636+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:51:46.584636+0000","last_undegraded":"2024-09-21T11:51:46.584636+0000","last_fullsized":"2024-09-21T11:51:46.584636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:34.980043+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":512,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.570807+0000","last_change":"2024-09-21T11:50:38.405183+0000","last_active":"2024-09-21T11:51:46.570807+0000","last_peered":"2024-09-21T11:51:46.570807+0000","last_clean":"2024-09-21T11:51:46.570807+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:51:46.570807+0000","last_undegraded":"2024-09-21T11:51:46.570807+0000","last_fullsized":"2024-09-21T11:51:46.570807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:38.405118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.405118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:51.232325+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":500,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.573303+0000","last_change":"2024-09-21T11:50:05.419390+0000","last_active":"2024-09-21T11:51:46.573303+0000","last_peered":"2024-09-21T11:51:46.573303+0000","last_clean":"2024-09-21T11:51:46.573303+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:51:46.573303+0000","last_undegraded":"2024-09-21T11:51:46.573303+0000","last_fullsized":"2024-09-21T11:51:46.573303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.419329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.419329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:40.266787+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":213,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.568206+0000","last_change":"2024-09-21T11:50:25.045141+0000","last_active":"2024-09-21T11:51:46.568206+0000","last_peered":"2024-09-21T11:51:46.568206+0000","last_clean":"2024-09-21T11:51:46.568206+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:51:46.568206+0000","last_undegraded":"2024-09-21T11:51:46.568206+0000","last_fullsized":"2024-09-21T11:51:46.568206+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.045074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.045074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:51:52.164898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":534,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.574042+0000","last_change":"2024-09-21T11:51:04.656939+0000","last_active":"2024-09-21T11:51:46.574042+0000","last_peered":"2024-09-21T11:51:46.574042+0000","last_clean":"2024-09-21T11:51:46.574042+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:51:46.574042+0000","last_undegraded":"2024-09-21T11:51:46.574042+0000","last_fullsized":"2024-09-21T11:51:46.574042+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:04.656870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:04.656870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:13.464686+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":601,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.571272+0000","last_change":"2024-09-21T11:51:15.333035+0000","last_active":"2024-09-21T11:51:46.571272+0000","last_peered":"2024-09-21T11:51:46.571272+0000","last_clean":"2024-09-21T11:51:46.571272+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:51:46.571272+0000","last_undegraded":"2024-09-21T11:51:46.571272+0000","last_fullsized":"2024-09-21T11:51:46.571272+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:51:15.332952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.332952+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:19.341819+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":520,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-21T11:51:46.584915+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:51:46.584915+0000","last_peered":"2024-09-21T11:51:46.584915+0000","last_clean":"2024-09-21T11:51:46.584915+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:51:46.584915+0000","last_undegraded":"2024-09-21T11:51:46.584915+0000","last_fullsized":"2024-09-21T11:51:46.584915+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:52:26.899451+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":3110469656,"num_objects":47463,"num_object_clones":0,"num_object_copies":142389,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206665,"num_write_kb":8132033,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9295179776,"data_stored":9295167560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82915,"ondisk_log_size":82915,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1113,"num_read_kb":870,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":702,"num_read_kb":697,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234131,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2280088,"kb_used_data":1775368,"kb_used_omap":1,"kb_used_meta":504574,"kb_avail":91465064,"statfs":{"total":95995035648,"available":93660225536,"internally_reserved":0,"allocated":1817976832,"data_stored":1817561305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":516684234},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463064,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":949768,"kb_used_data":585400,"kb_used_omap":1,"kb_used_meta":364350,"kb_avail":92795384,"statfs":{"total":95995035648,"available":95022473216,"internally_reserved":0,"allocated":599449600,"data_stored":599050777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":373094854},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724701,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2223768,"kb_used_data":1759432,"kb_used_omap":1,"kb_used_meta":464318,"kb_avail":91521384,"statfs":{"total":95995035648,"available":93717897216,"internally_reserved":0,"allocated":1801658368,"data_stored":1801241649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":475462086},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,4,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920932,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":811352,"kb_used_data":513608,"kb_used_omap":1,"kb_used_meta":297726,"kb_avail":92933800,"statfs":{"total":95995035648,"available":95164211200,"internally_reserved":0,"allocated":525934592,"data_stored":525573192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":304871877},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117161,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2060428,"kb_used_data":1523772,"kb_used_omap":1,"kb_used_meta":536510,"kb_avail":91684724,"statfs":{"total":95995035648,"available":93885157376,"internally_reserved":0,"allocated":1560342528,"data_stored":1559938059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":549386695},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411503,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1045328,"kb_used_data":572672,"kb_used_omap":1,"kb_used_meta":472638,"kb_avail":92699824,"statfs":{"total":95995035648,"available":94924619776,"internally_reserved":0,"allocated":586416128,"data_stored":586008274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":483981765},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575030,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2189556,"kb_used_data":1707044,"kb_used_omap":1,"kb_used_meta":482366,"kb_avail":91555596,"statfs":{"total":95995035648,"available":93752930304,"internally_reserved":0,"allocated":1748013056,"data_stored":1747568912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":493943238},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738556,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1163236,"kb_used_data":766932,"kb_used_omap":1,"kb_used_meta":396286,"kb_avail":92581916,"statfs":{"total":95995035648,"available":94803881984,"internally_reserved":0,"allocated":785338368,"data_stored":784911647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":405797317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":783482880,"data_stored":783482880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1744572416,"data_stored":1744568344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":577568768,"data_stored":577568768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1527578624,"data_stored":1527578624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":521797632,"data_stored":521797632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1773473792,"data_stored":1773469720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":590282752,"data_stored":590282752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1776422912,"data_stored":1776418840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:51:58.112 INFO:tasks.thrashosds.thrasher:Setting 6.2 to [2, 4, 5] 2024-09-21T11:51:58.113 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '6.2', '2', '4', '5'] 2024-09-21T11:51:58.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pg-upmap 6.2 2 4 5 2024-09-21T11:51:58.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:58.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:58.213 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:58.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:58.215 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:51:58.314 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:58.344 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:58.348 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:58.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:58.349 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:58.450 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:58.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:58.483 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:58.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:58.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:58.621 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:58.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:58.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:58.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:51:58.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:58.762 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:51:58.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:51:58.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:58.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:58.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:51:58.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:58 smithi043 ceph-mon[36454]: from='client.25558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:51:58.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:58 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:58.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:58 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:58.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:58 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:51:58.945 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:58 smithi043 ceph-mon[36454]: pgmap v570: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:51:58.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:58 smithi043 ceph-mon[29710]: from='client.25558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:51:58.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:58 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:58.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:58 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:58.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:58 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:51:58.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:58 smithi043 ceph-mon[29710]: pgmap v570: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:51:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:58 smithi175 ceph-mon[32613]: from='client.25558 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:51:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:58 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:51:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:58 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:51:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:58 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:51:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:58 smithi175 ceph-mon[32613]: pgmap v570: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:51:59.012 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:51:59.023 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:51:59.047 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:59.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:59.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:51:59.049 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:51:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:51:59.085 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:51:59.086 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:51:59.123 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:51:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:51:59.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:59.158 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:51:59.158 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:51:59.195 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:51:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:51:59.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:59.202 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:59.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:59.229 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:51:59.229 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:51:59.261 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:51:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:51:59.299 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:51:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:51:59.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:51:59.349 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:51:59.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:59.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:51:59.451 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:51:59.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:59.483 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:59.483 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:51:59.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:51:59.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:59.621 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:59.621 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:51:59.722 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:59.747 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:59.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:59.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:59.849 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:51:59.877 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:51:59.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:51:59.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:51:59.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:00.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:00.020 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:00.057 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:52:00.057 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:52:00.120 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:00.154 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:00.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:00.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:00.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:51:59 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/159433176' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:00.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:51:59 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/159433176' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:00.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:00.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:51:59 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/159433176' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:00.295 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:00.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:00.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:00.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:52:00.397 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:00.423 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:00.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:00.424 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:00.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:00.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:00.556 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:00.556 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:00.657 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:00.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:00.707 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:00.808 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:00.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:00.845 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:00.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:00.929 INFO:teuthology.orchestra.run.smithi043.stderr:set 6.2 pg_upmap mapping to [2,4,5] 2024-09-21T11:52:00.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:00.993 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:00.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:00.994 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:01.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:00 smithi175 ceph-mon[32613]: pgmap v571: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:52:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2754495748' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.2", "id": [2, 4, 5]} : dispatch 2024-09-21T11:52:01.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:01.129 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:01.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:01.130 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:01.231 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:01.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:00 smithi043 ceph-mon[29710]: pgmap v571: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:52:01.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2754495748' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.2", "id": [2, 4, 5]} : dispatch 2024-09-21T11:52:01.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:00 smithi043 ceph-mon[36454]: pgmap v571: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:52:01.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2754495748' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "6.2", "id": [2, 4, 5]} : dispatch 2024-09-21T11:52:01.260 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:01.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:01.261 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:01.362 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:01.390 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:01.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:01.391 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:01.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:01.527 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:01.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:01.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:01.629 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:01.633 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:52:01.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:01.657 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:01.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:01.732 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:52:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:52:01.758 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:01.804 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:01.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:01.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:01.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:01.937 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:01.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:01.938 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:01 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:52:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:01 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:52:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2754495748' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.2", "id": [2, 4, 5]}]': finished 2024-09-21T11:52:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:01 smithi175 ceph-mon[32613]: osdmap e297: 8 total, 8 up, 8 in 2024-09-21T11:52:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:01 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:02.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:02.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:02.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:02.107 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:02.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:02.219 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:02.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:02.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:02.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:52:02.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:52:02.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2754495748' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.2", "id": [2, 4, 5]}]': finished 2024-09-21T11:52:02.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[36454]: osdmap e297: 8 total, 8 up, 8 in 2024-09-21T11:52:02.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:02.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:52:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:52:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2754495748' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.2", "id": [2, 4, 5]}]': finished 2024-09-21T11:52:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[29710]: osdmap e297: 8 total, 8 up, 8 in 2024-09-21T11:52:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:02.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:01 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:02.321 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:02.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:02.351 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:02.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:02.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:52:02.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:02.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:02.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:02.604 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:02.647 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:02.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:02.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:02.729 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:52:02.729 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:52:02.749 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:02.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:02.806 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:02.806 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:02.907 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:02.939 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:02.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:02.940 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:52:03.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:03.074 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:03.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:03.075 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:03.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:03.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:03.216 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:03.216 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:03.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:02 smithi043 ceph-mon[29710]: osdmap e298: 8 total, 8 up, 8 in 2024-09-21T11:52:03.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2534012241' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:03.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:02 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:03.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:02 smithi043 ceph-mon[29710]: pgmap v574: 148 pgs: 1 peering, 147 active+clean; 2.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-21T11:52:03.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:02 smithi043 ceph-mon[36454]: osdmap e298: 8 total, 8 up, 8 in 2024-09-21T11:52:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2534012241' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:02 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:02 smithi043 ceph-mon[36454]: pgmap v574: 148 pgs: 1 peering, 147 active+clean; 2.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-21T11:52:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:02 smithi175 ceph-mon[32613]: osdmap e298: 8 total, 8 up, 8 in 2024-09-21T11:52:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2534012241' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:02 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:02 smithi175 ceph-mon[32613]: pgmap v574: 148 pgs: 1 peering, 147 active+clean; 2.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-21T11:52:03.317 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:03.358 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:03.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:03.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:03.460 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:03.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:03.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:03.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:52:03.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:03.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:03.640 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:03.640 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:03.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:03.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:03.780 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:03.780 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:03.881 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:03.927 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:03.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:03.928 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:03.934 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:04.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:04.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:04.064 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:04.165 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:04.201 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:04.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:04.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:04.221 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:03 smithi175 ceph-mon[32613]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-21T11:52:04.221 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:03 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:04.221 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:03 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:04.221 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:03 smithi175 ceph-mon[32613]: osdmap e299: 8 total, 8 up, 8 in 2024-09-21T11:52:04.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:03 smithi043 ceph-mon[29710]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-21T11:52:04.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:03 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:04.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:03 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:04.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:03 smithi043 ceph-mon[29710]: osdmap e299: 8 total, 8 up, 8 in 2024-09-21T11:52:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:03 smithi043 ceph-mon[36454]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-21T11:52:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:03 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:03 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:04.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:03 smithi043 ceph-mon[36454]: osdmap e299: 8 total, 8 up, 8 in 2024-09-21T11:52:04.303 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:04.338 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:04.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:04.339 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:04.440 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:04.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:04.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:04.471 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:04.520 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:52:04.520 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:52:04.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:04.607 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:04.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:04.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:04.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:04.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:04.753 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:04.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:04.802 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:52:04.802 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:52:04.841 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:52:04.842 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:52:04.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:04.874 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:52:04.874 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:52:04.893 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:04.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:04.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:04.906 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:52:04.907 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:52:04.936 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:52:04.937 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:52:04.973 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:52:04.974 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:52:04.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:05.007 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:52:05.007 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:52:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:05.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:05.035 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:05.041 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:52:05.041 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:52:05.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:05.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:04 smithi175 ceph-mon[32613]: pgmap v576: 148 pgs: 1 peering, 147 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 966 op/s 2024-09-21T11:52:05.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:04 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:52:05.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:04 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:52:05.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:04 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/476745760' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:05.217 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:05.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:05.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:05.219 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:52:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:04 smithi043 ceph-mon[36454]: pgmap v576: 148 pgs: 1 peering, 147 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 966 op/s 2024-09-21T11:52:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:04 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:52:05.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:04 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:52:05.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:04 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/476745760' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:05.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:04 smithi043 ceph-mon[29710]: pgmap v576: 148 pgs: 1 peering, 147 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 966 op/s 2024-09-21T11:52:05.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:04 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:52:05.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:04 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:52:05.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:04 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/476745760' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:05.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:05.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:05.359 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:05.360 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:05.460 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:52:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:52:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:52:05.497 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:05.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:05.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:05.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:05.634 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:05.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:05.644 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:05.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:05.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:05.786 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:05.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:05.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:05.888 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:05.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:05.922 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:05.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:06.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:06.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:06.062 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:06.062 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:06.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:05 smithi175 ceph-mon[32613]: osdmap e300: 8 total, 8 up, 8 in 2024-09-21T11:52:06.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:52:06.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:06.179 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:05 smithi043 ceph-mon[29710]: osdmap e300: 8 total, 8 up, 8 in 2024-09-21T11:52:06.180 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:05 smithi043 ceph-mon[36454]: osdmap e300: 8 total, 8 up, 8 in 2024-09-21T11:52:06.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:06.229 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:06.229 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:06.330 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:06.370 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:06.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:06.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:06.472 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:06.501 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:06.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:06.502 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:06.573 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:06.603 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:06.630 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:06.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:06.631 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:06.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:06.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:06.772 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:06.772 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:06.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:06.916 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:06.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:06.917 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:07.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:07.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:07.052 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:07.153 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:52:07.153 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:52:07.154 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-21T11:52:07.154 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T11:52:07.154 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T11:52:07.154 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-09-21T11:52:07.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:07.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:07.194 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:07.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:07.227 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:06 smithi175 ceph-mon[32613]: pgmap v578: 148 pgs: 1 peering, 147 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1008 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:52:07.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:06 smithi043 ceph-mon[29710]: pgmap v578: 148 pgs: 1 peering, 147 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1008 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:52:07.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:06 smithi043 ceph-mon[36454]: pgmap v578: 148 pgs: 1 peering, 147 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1008 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-21T11:52:07.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:07.331 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:07.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:07.332 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:99019 7.d 7:bdee4b66:::benchmark_data_smithi043_70646_object99018:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:26.324885+0000", 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 581.05818363100002, 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.109393896, 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.388 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99019 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324885+0000", 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.389 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324885+0000", 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324887+0000", 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8419999999999999e-06 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324939+0000", 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1867000000000003e-05 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.390 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324952+0000", 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2622999999999999e-05 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.324960+0000", 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3310000000000004e-06 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.325018+0000", 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7787000000000001e-05 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.391 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.325122+0000", 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104081 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1", 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.325256+0000", 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013424699999999999 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.327618+0000", 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023618419999999998 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.392 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.330240+0000", 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026218690000000002 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.434164+0000", 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.103923735 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.434226+0000", 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2242000000000005e-05 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.393 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.434279+0000", 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3430000000000002e-05 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:99032 7.d 7:b4661550:::benchmark_data_smithi043_70646_object99031:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:26.337848+0000", 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 581.04522083500001, 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10748263299999999, 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.394 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99032 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337848+0000", 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.395 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337848+0000", 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337849+0000", 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.331e-06 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337893+0000", 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3432e-05 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337904+0000", 2024-09-21T11:52:07.396 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1518999999999999e-05 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337912+0000", 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0369999999999995e-06 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.337974+0000", 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1711999999999995e-05 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338086+0000", 2024-09-21T11:52:07.397 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011175000000000001 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1", 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.338186+0000", 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9905e-05 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.340545+0000", 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023587730000000002 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.398 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.341117+0000", 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057252099999999999 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.445226+0000", 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10410860399999999 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.445284+0000", 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8069999999999998e-05 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.445331+0000", 2024-09-21T11:52:07.399 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6978999999999999e-05 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:99272 7.7 7:e3c50c45:::benchmark_data_smithi043_70646_object99271:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:26.706556+0000", 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 580.67651302499996, 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096941552, 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.400 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 99272 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706556+0000", 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706556+0000", 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.401 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706557+0000", 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5290000000000001e-06 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706601+0000", 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3544000000000002e-05 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706613+0000", 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1625e-05 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.402 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706619+0000", 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5919999999999997e-06 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706666+0000", 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6526999999999997e-05 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706752+0000", 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6734e-05 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.403 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.706847+0000", 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4136999999999997e-05 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.709350+0000", 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002503244 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.710741+0000", 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013909250000000001 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.404 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.803398+0000", 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092657623999999994 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.803445+0000", 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6652e-05 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:26.803497+0000", 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2419000000000001e-05 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.405 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123033 7.d 7:bdea6e43:::benchmark_data_smithi043_70646_object5625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.860948+0000", 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 551.52212058999999, 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10815586100000001, 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.406 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123033 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860948+0000", 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860948+0000", 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.407 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860950+0000", 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5680000000000001e-06 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860955+0000", 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3929999999999999e-06 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860972+0000", 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6901e-05 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.408 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.860980+0000", 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0779999999999996e-06 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.861037+0000", 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6368e-05 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.861319+0000", 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028206500000000002 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1,3", 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.862396+0000", 2024-09-21T11:52:07.409 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010775660000000001 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.867300+0000", 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0049041199999999997 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.868070+0000", 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00077001800000000003 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.410 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.872379+0000", 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043081150000000004 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.968899+0000", 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096520717000000006 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.968976+0000", 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7199000000000004e-05 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.411 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.969104+0000", 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012775300000000001 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123045 7.d 7:be75f9a0:::benchmark_data_smithi043_70646_object5637:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.874453+0000", 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 551.50861573500003, 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.100889214, 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.412 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123045 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874453+0000", 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.413 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874453+0000", 2024-09-21T11:52:07.414 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.414 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.414 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.414 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.414 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874454+0000", 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.17e-06 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874460+0000", 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2460000000000003e-06 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874468+0000", 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3049999999999993e-06 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.415 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874475+0000", 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0740000000000004e-06 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874536+0000", 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0645e-05 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874666+0000", 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000130264 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.416 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1,3", 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.874801+0000", 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013530200000000001 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879404+0000", 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004603261 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879516+0000", 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011174400000000001 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.417 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882653+0000", 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003136808 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975138+0000", 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.092484760999999999 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.418 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975214+0000", 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5873999999999999e-05 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975342+0000", 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012876000000000001 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:123052 7.d 7:b4211c57:::benchmark_data_smithi043_70646_object5644:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:55.879526+0000", 2024-09-21T11:52:07.419 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 551.50354285900005, 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096040904999999996, 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 123052 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879526+0000", 2024-09-21T11:52:07.420 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879526+0000", 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879528+0000", 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.058e-06 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879534+0000", 2024-09-21T11:52:07.421 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9290000000000003e-06 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879547+0000", 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2625e-05 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879555+0000", 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3289999999999994e-06 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879604+0000", 2024-09-21T11:52:07.422 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8906999999999998e-05 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879739+0000", 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013482199999999999 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1,3", 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.879886+0000", 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014718199999999999 2024-09-21T11:52:07.423 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.882961+0000", 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003075468 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.885347+0000", 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002385543 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.885927+0000", 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00058006400000000004 2024-09-21T11:52:07.424 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975391+0000", 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.089464284000000005 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975459+0000", 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7566999999999999e-05 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:55.975567+0000", 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108127 2024-09-21T11:52:07.425 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.426 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.426 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.426 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.426 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.426 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126201 7.d 7:bd4d612f:::benchmark_data_smithi043_70646_object8793:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:52:07.426 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.184391+0000", 2024-09-21T11:52:07.426 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 548.19867838699997, 2024-09-21T11:52:07.426 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.107348185, 2024-09-21T11:52:07.426 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.426 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126201 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184391+0000", 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.427 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184391+0000", 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184392+0000", 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5090000000000001e-06 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184397+0000", 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3839999999999996e-06 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.428 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184408+0000", 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.031e-05 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.184415+0000", 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3039999999999999e-06 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.185017+0000", 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00060169300000000004 2024-09-21T11:52:07.429 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.185146+0000", 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012897000000000001 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1,3", 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.185294+0000", 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014784299999999999 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.187915+0000", 2024-09-21T11:52:07.430 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026212169999999999 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194804+0000", 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0068889140000000003 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194847+0000", 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3689000000000001e-05 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291506+0000", 2024-09-21T11:52:07.431 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.096658620000000001 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291615+0000", 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109423 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291739+0000", 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012330899999999999 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.432 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126205 7.d 7:bf875b8c:::benchmark_data_smithi043_70646_object8797:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.189901+0000", 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 548.19316772800005, 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.102518362, 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126205 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.433 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189901+0000", 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189901+0000", 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189902+0000", 2024-09-21T11:52:07.434 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2589999999999999e-06 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189907+0000", 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.729e-06 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189915+0000", 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8860000000000004e-06 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.189921+0000", 2024-09-21T11:52:07.435 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1330000000000004e-06 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.190242+0000", 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032039900000000002 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.190365+0000", 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012315199999999999 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1,3", 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.190603+0000", 2024-09-21T11:52:07.436 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000237777 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194889+0000", 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0042860900000000002 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194975+0000", 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6694999999999996e-05 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.198044+0000", 2024-09-21T11:52:07.437 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030690719999999999 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291775+0000", 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.093730331 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.291866+0000", 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0890999999999994e-05 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.438 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.292420+0000", 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00055394799999999996 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:126213 7.7 7:efdaa55a:::benchmark_data_smithi043_70646_object8805:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122)", 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:42:59.194786+0000", 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 548.18828272300004, 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.099879320999999993, 2024-09-21T11:52:07.439 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126213 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194786+0000", 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.440 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194786+0000", 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194788+0000", 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4559999999999999e-06 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194793+0000", 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.31e-06 2024-09-21T11:52:07.441 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194804+0000", 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0886e-05 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194811+0000", 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2300000000000002e-06 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194876+0000", 2024-09-21T11:52:07.442 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4975999999999996e-05 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.194991+0000", 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114567 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.195150+0000", 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015947499999999999 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.197421+0000", 2024-09-21T11:52:07.443 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022710370000000001 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.288199+0000", 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.090777468 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.294580+0000", 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0063818929999999996 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.294623+0000", 2024-09-21T11:52:07.444 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2929000000000002e-05 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:42:59.294666+0000", 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2094000000000001e-05 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220977 7.c 7:39262562:::benchmark_data_smithi043_70646_object103569:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:52:07.445 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.889732+0000", 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 460.49333670599998, 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.128598458, 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220977 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.446 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889732+0000", 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889732+0000", 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889733+0000", 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9400000000000004e-07 2024-09-21T11:52:07.447 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889737+0000", 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.335e-06 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889747+0000", 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8079999999999999e-06 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.889754+0000", 2024-09-21T11:52:07.448 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8399999999999997e-06 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.891865+0000", 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021112990000000001 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.891983+0000", 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117707 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892088+0000", 2024-09-21T11:52:07.449 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105102 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894742+0000", 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026534750000000002 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.897215+0000", 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024734470000000001 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.450 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018256+0000", 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.121041197 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018295+0000", 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8430000000000003e-05 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018331+0000", 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5923999999999997e-05 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.451 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220983 7.7 7:e4a77ba0:::benchmark_data_smithi043_70646_object103575:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.892652+0000", 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 460.490417272, 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.125681194, 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220983 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.452 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892652+0000", 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892652+0000", 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892653+0000", 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.178e-06 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.453 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892656+0000", 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6650000000000001e-06 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892670+0000", 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3560000000000001e-05 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892677+0000", 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1579999999999999e-06 2024-09-21T11:52:07.454 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892734+0000", 2024-09-21T11:52:07.455 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6365e-05 2024-09-21T11:52:07.455 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.455 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.455 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.455 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892833+0000", 2024-09-21T11:52:07.455 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:07.456 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9402999999999997e-05 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.892935+0000", 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102348 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.895993+0000", 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003057471 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.457 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896614+0000", 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00062117100000000001 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018250+0000", 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12163622 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018298+0000", 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7837000000000003e-05 2024-09-21T11:52:07.458 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018333+0000", 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4817999999999998e-05 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220986 7.c 7:34da84d9:::benchmark_data_smithi043_70646_object103578:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.894893+0000", 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 460.488175517, 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.123588221, 2024-09-21T11:52:07.459 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220986 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894893+0000", 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.460 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894893+0000", 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894895+0000", 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.175e-06 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894899+0000", 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0199999999999996e-06 2024-09-21T11:52:07.461 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894905+0000", 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4540000000000004e-06 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894910+0000", 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4750000000000004e-06 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.894962+0000", 2024-09-21T11:52:07.462 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2067000000000001e-05 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.895060+0000", 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8333999999999997e-05 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.895158+0000", 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7813999999999996e-05 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.897402+0000", 2024-09-21T11:52:07.463 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022445289999999999 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.898115+0000", 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00071272200000000005 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018382+0000", 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.120266574 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.464 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018436+0000", 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3965999999999998e-05 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.018482+0000", 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6091000000000003e-05 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.465 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:220987 7.7 7:eba202fb:::benchmark_data_smithi043_70646_object103579:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:26.896951+0000", 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 460.48611788300002, 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12351648699999999, 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 220987 2024-09-21T11:52:07.466 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896951+0000", 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896951+0000", 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.467 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896952+0000", 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9700000000000005e-07 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896956+0000", 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3869999999999997e-06 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896967+0000", 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0363000000000001e-05 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.468 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.896980+0000", 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3407999999999999e-05 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.897040+0000", 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9947000000000001e-05 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.897148+0000", 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010828900000000001 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.469 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.897255+0000", 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010709000000000001 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.899364+0000", 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021090129999999999 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:26.901062+0000", 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016971289999999999 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.470 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.020394+0000", 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11933194699999999 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.020435+0000", 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1430000000000001e-05 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.020468+0000", 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2586999999999998e-05 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.471 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14805.0:221226 7.d 7:b01dbbaf:::benchmark_data_smithi043_70646_object103818:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:44:27.186275+0000", 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 460.19679434400001, 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.10689198599999999, 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14805", 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/539141136", 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 221226 2024-09-21T11:52:07.472 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186275+0000", 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186275+0000", 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.473 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186277+0000", 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.018e-06 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186282+0000", 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4890000000000003e-06 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186295+0000", 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2962999999999999e-05 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.474 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186303+0000", 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4619999999999999e-06 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186363+0000", 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0896000000000001e-05 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186460+0000", 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6302000000000001e-05 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.475 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.186558+0000", 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8073999999999996e-05 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.189168+0000", 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002609765 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.276314+0000", 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.087146766000000001 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.476 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293019+0000", 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.016704455 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293108+0000", 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9046000000000001e-05 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:44:27.293167+0000", 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8749999999999998e-05 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.477 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46252 8.9 8:969b922f:::benchmark_data_smithi043_97042_object46251:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.597214+0000", 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 404.78585444999999, 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.11913404700000001, 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46252 2024-09-21T11:52:07.478 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597214+0000", 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597214+0000", 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597222+0000", 2024-09-21T11:52:07.479 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8909999999999995e-06 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597262+0000", 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9393e-05 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597277+0000", 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5421e-05 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597284+0000", 2024-09-21T11:52:07.480 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1620000000000002e-06 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597351+0000", 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6948999999999999e-05 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597501+0000", 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014983499999999999 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597630+0000", 2024-09-21T11:52:07.481 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012867 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604093+0000", 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0064636370000000004 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604140+0000", 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6717e-05 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.482 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716241+0000", 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.112100726 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716302+0000", 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0639e-05 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716349+0000", 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7006999999999999e-05 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.483 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46253 8.9 8:9d2b1260:::benchmark_data_smithi043_97042_object46252:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.597498+0000", 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 404.78557050400002, 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.140088831, 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46253 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.484 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597498+0000", 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597498+0000", 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597499+0000", 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3399999999999998e-07 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.485 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597572+0000", 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3135999999999999e-05 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597580+0000", 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0830000000000004e-06 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.597588+0000", 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9969999999999998e-06 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.486 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.599249+0000", 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016605820000000001 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.599363+0000", 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011366600000000001 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.599532+0000", 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016955299999999999 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.487 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604099+0000", 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045664690000000001 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604188+0000", 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8968000000000006e-05 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737476+0000", 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.133288615 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.488 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737535+0000", 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8573000000000003e-05 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737587+0000", 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2454999999999998e-05 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46254 8.9 8:9dcdf461:::benchmark_data_smithi043_97042_object46253:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:07.489 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.598445+0000", 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 404.78462405800002, 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13929741600000001, 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46254 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.490 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598445+0000", 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598445+0000", 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598446+0000", 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9999999999999995e-07 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598516+0000", 2024-09-21T11:52:07.491 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9726000000000003e-05 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598526+0000", 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0349999999999999e-05 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.598533+0000", 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7129999999999999e-06 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.601614+0000", 2024-09-21T11:52:07.492 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030816649999999999 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.601758+0000", 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014389200000000001 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.601919+0000", 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016073800000000001 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604200+0000", 2024-09-21T11:52:07.493 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022812470000000001 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604222+0000", 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1481e-05 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737643+0000", 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.133421174 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737704+0000", 2024-09-21T11:52:07.494 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0919999999999999e-05 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737742+0000", 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8510000000000002e-05 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46259 8.9 8:9a5eb1e7:::benchmark_data_smithi043_97042_object46258:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.600174+0000", 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 404.78289527800001, 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13768232, 2024-09-21T11:52:07.495 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.496 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.496 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.496 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.496 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46259 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600174+0000", 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600174+0000", 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.497 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600175+0000", 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2899999999999999e-06 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600235+0000", 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0517999999999999e-05 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600260+0000", 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.4201000000000001e-05 2024-09-21T11:52:07.498 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.600267+0000", 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2640000000000002e-06 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.603150+0000", 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002883409 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.603264+0000", 2024-09-21T11:52:07.499 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113286 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.603398+0000", 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013435899999999999 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604295+0000", 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00089736199999999996 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.604666+0000", 2024-09-21T11:52:07.500 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000370379 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737780+0000", 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13311397799999999 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737824+0000", 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4549999999999999e-05 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.501 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.737856+0000", 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1724000000000003e-05 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47790 8.9 8:98dbfe67:::benchmark_data_smithi043_97042_object47789:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.712965+0000", 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 403.67010373900001, 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13282617399999999, 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.502 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47790 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712965+0000", 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712965+0000", 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.503 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712967+0000", 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.59e-06 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713048+0000", 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1173000000000004e-05 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713073+0000", 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5333999999999998e-05 2024-09-21T11:52:07.504 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713088+0000", 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4436e-05 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713161+0000", 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3152999999999996e-05 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713271+0000", 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000109849 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.505 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.713392+0000", 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000121477 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.715175+0000", 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017832729999999999 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.717167+0000", 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001991044 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.506 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845693+0000", 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12852650299999999 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845750+0000", 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6895999999999999e-05 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845791+0000", 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1446000000000003e-05 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.507 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66144 8.9 8:9c2c8472:::benchmark_data_smithi043_97042_object66143:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.930304+0000", 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 391.45276484700003, 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12634800199999999, 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66144 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.508 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930304+0000", 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930304+0000", 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930305+0000", 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.342e-06 2024-09-21T11:52:07.509 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930332+0000", 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6105000000000002e-05 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930341+0000", 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7739999999999999e-06 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930347+0000", 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6149999999999996e-06 2024-09-21T11:52:07.510 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930389+0000", 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1933e-05 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930471+0000", 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2466999999999995e-05 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.930559+0000", 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8022999999999994e-05 2024-09-21T11:52:07.511 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.932240+0000", 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0016807599999999999 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.933402+0000", 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011622270000000001 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.056538+0000", 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12313563700000001 2024-09-21T11:52:07.512 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.056610+0000", 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1798000000000006e-05 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.056652+0000", 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2320999999999997e-05 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:07.513 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:52:07.529 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:07.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:07.531 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:07.632 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:07.671 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:07.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:07.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:07.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:07.815 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:07.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:07.816 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:07.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:07.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:52:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:07.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:07.954 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:08.000 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:52:08.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:08.109 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:08.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:08.110 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:08.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:08.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1550375276' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:08.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:07 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:08.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1550375276' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:08.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:07 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:08.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1550375276' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:08.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:07 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:08.273 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:08.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:08.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:08.374 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:08.403 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:08.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:08.404 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:08.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:08.537 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:08.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:08.538 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:08.618 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:52:08.639 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:08.648 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:08.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:08.712 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:08.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:08.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:08.851 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:08.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:08.852 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:08.953 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:08.961 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:09.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:09.009 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:09.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:09.110 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:09.113 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:08 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:09.113 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:08 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:52:09.113 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:08 smithi175 ceph-mon[32613]: osdmap e301: 8 total, 8 up, 8 in 2024-09-21T11:52:09.113 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:08 smithi175 ceph-mon[32613]: pgmap v580: 148 pgs: 148 active+clean; 1.8 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.47k op/s 2024-09-21T11:52:09.113 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:08 smithi175 ceph-mon[32613]: 4.18 scrub starts 2024-09-21T11:52:09.114 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:08 smithi175 ceph-mon[32613]: 4.18 scrub ok 2024-09-21T11:52:09.147 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:09.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:09.148 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:52:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[36454]: osdmap e301: 8 total, 8 up, 8 in 2024-09-21T11:52:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[36454]: pgmap v580: 148 pgs: 148 active+clean; 1.8 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.47k op/s 2024-09-21T11:52:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[36454]: 4.18 scrub starts 2024-09-21T11:52:09.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[36454]: 4.18 scrub ok 2024-09-21T11:52:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:52:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[29710]: osdmap e301: 8 total, 8 up, 8 in 2024-09-21T11:52:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[29710]: pgmap v580: 148 pgs: 148 active+clean; 1.8 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.47k op/s 2024-09-21T11:52:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[29710]: 4.18 scrub starts 2024-09-21T11:52:09.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:08 smithi043 ceph-mon[29710]: 4.18 scrub ok 2024-09-21T11:52:09.249 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:09.272 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:52:09.272 INFO:teuthology.orchestra.run.smithi175.stdout: "success": "" 2024-09-21T11:52:09.272 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:52:09.280 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:09.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:09.281 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:09.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:09.412 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:09.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:09.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:09.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:09.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:09.560 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:09.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:09.668 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:52:09.668 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:52:09.705 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:52:09.705 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:52:09.705 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:09.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:09.706 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:09.736 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:52:09.736 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:52:09.770 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:52:09.770 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:52:09.805 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:52:09.806 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:52:09.807 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:09.836 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:09.843 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:09.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:09.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:09.844 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:52:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:52:09.869 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:52:09.870 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:52:09.908 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:52:09.908 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:52:09.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:09.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:09.989 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:10.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:10.124 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:10.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:10.125 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:10.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:09 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:52:10.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:09 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:52:10.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:09 smithi043 ceph-mon[36454]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-21T11:52:10.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:09 smithi043 ceph-mon[36454]: osdmap e302: 8 total, 8 up, 8 in 2024-09-21T11:52:10.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3411324933' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:10.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:09 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:52:10.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:09 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:52:10.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:09 smithi043 ceph-mon[29710]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-21T11:52:10.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:09 smithi043 ceph-mon[29710]: osdmap e302: 8 total, 8 up, 8 in 2024-09-21T11:52:10.170 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3411324933' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:10.226 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:10.258 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:10.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:10.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:09 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:52:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:09 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:52:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:09 smithi175 ceph-mon[32613]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-21T11:52:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:09 smithi175 ceph-mon[32613]: osdmap e302: 8 total, 8 up, 8 in 2024-09-21T11:52:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3411324933' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:10.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:10.401 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:10.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:10.402 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:10.503 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:10.541 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:10.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:10.542 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:10.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:10.680 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:10.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:10.681 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:10.781 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:10.820 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:10.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:10.821 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:10.922 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:10.943 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:10.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:10.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:11.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:52:11.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:11.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:11.097 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:11.097 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:11.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:11.238 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:11.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:11.239 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:11.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:10 smithi175 ceph-mon[32613]: 6.1 scrub starts 2024-09-21T11:52:11.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:10 smithi175 ceph-mon[32613]: 6.1 scrub ok 2024-09-21T11:52:11.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:10 smithi175 ceph-mon[32613]: pgmap v582: 148 pgs: 148 active+clean; 1.8 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-21T11:52:11.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:11.394 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:11.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:11.395 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:11.425 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:11.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:10 smithi043 ceph-mon[29710]: 6.1 scrub starts 2024-09-21T11:52:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:10 smithi043 ceph-mon[29710]: 6.1 scrub ok 2024-09-21T11:52:11.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:10 smithi043 ceph-mon[29710]: pgmap v582: 148 pgs: 148 active+clean; 1.8 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-21T11:52:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:11 smithi043 ceph-mon[36454]: 6.1 scrub starts 2024-09-21T11:52:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:11 smithi043 ceph-mon[36454]: 6.1 scrub ok 2024-09-21T11:52:11.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:11 smithi043 ceph-mon[36454]: pgmap v582: 148 pgs: 148 active+clean; 1.8 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-21T11:52:11.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:11.533 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:11.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:11.535 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:11.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:11.659 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:52:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:52:11.670 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:11.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:11.672 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:11.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:11.814 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:52:11.815 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:52:11.820 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:11.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:11.821 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:11.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:11.962 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:11.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:11.963 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:12.064 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:12.093 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:12.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:12.094 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:12.195 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:12.234 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:12.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:12.235 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:12 smithi043 ceph-mon[29710]: 6.1b scrub starts 2024-09-21T11:52:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:12 smithi043 ceph-mon[29710]: 6.1b scrub ok 2024-09-21T11:52:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:12 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:12 smithi043 ceph-mon[29710]: 3.1c scrub starts 2024-09-21T11:52:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:12 smithi043 ceph-mon[29710]: 3.1c scrub ok 2024-09-21T11:52:12.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:12 smithi043 ceph-mon[36454]: 6.1b scrub starts 2024-09-21T11:52:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:12 smithi043 ceph-mon[36454]: 6.1b scrub ok 2024-09-21T11:52:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:12 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:12 smithi043 ceph-mon[36454]: 3.1c scrub starts 2024-09-21T11:52:12.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:12 smithi043 ceph-mon[36454]: 3.1c scrub ok 2024-09-21T11:52:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:12 smithi175 ceph-mon[32613]: 6.1b scrub starts 2024-09-21T11:52:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:12 smithi175 ceph-mon[32613]: 6.1b scrub ok 2024-09-21T11:52:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:12 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:12 smithi175 ceph-mon[32613]: 3.1c scrub starts 2024-09-21T11:52:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:12 smithi175 ceph-mon[32613]: 3.1c scrub ok 2024-09-21T11:52:12.336 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:12.364 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:12.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:12.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:12.466 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:12.498 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:12.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:12.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:12.600 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:12.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:52:12.632 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:12.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:12.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:12.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:12.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:52:12.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:12.768 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:12.768 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:12.869 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:12.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:12.906 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:12.906 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:13.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:13.032 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:13.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:13.053 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:13.054 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:13.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:13.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:13.210 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:13.237 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:13.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:13.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:13.345 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:13.375 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:52:13.445 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:13.477 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:13.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:13.478 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:13.494 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:13 smithi175 ceph-mon[32613]: pgmap v583: 148 pgs: 148 active+clean; 1.6 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-21T11:52:13.494 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:13 smithi175 ceph-mon[32613]: 6.8 scrub starts 2024-09-21T11:52:13.494 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:13 smithi175 ceph-mon[32613]: 6.8 scrub ok 2024-09-21T11:52:13.494 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:13 smithi175 ceph-mon[32613]: 6.1d scrub starts 2024-09-21T11:52:13.494 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:13 smithi175 ceph-mon[32613]: 6.1d scrub ok 2024-09-21T11:52:13.494 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3078564716' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:13.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:13.614 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:13.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:13.616 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:13.649 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:52:13.649 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:52:13.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:13.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[36454]: pgmap v583: 148 pgs: 148 active+clean; 1.6 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-21T11:52:13.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[36454]: 6.8 scrub starts 2024-09-21T11:52:13.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[36454]: 6.8 scrub ok 2024-09-21T11:52:13.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[36454]: 6.1d scrub starts 2024-09-21T11:52:13.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[36454]: 6.1d scrub ok 2024-09-21T11:52:13.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3078564716' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:13.731 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[29710]: pgmap v583: 148 pgs: 148 active+clean; 1.6 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-21T11:52:13.731 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[29710]: 6.8 scrub starts 2024-09-21T11:52:13.732 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[29710]: 6.8 scrub ok 2024-09-21T11:52:13.732 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[29710]: 6.1d scrub starts 2024-09-21T11:52:13.732 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[29710]: 6.1d scrub ok 2024-09-21T11:52:13.732 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3078564716' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:13.746 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:13.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:13.747 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:13.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:13.883 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:13.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:13.884 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:13.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:14.023 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:14.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:14.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:14.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:14.162 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:14.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:14.163 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:14.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:14.306 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:14.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:14.307 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:14.408 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:14.433 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:14 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:52:14.433 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:14 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:52:14.433 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3078564716' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:14.434 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:14 smithi043 ceph-mon[36454]: osdmap e303: 8 total, 8 up, 8 in 2024-09-21T11:52:14.434 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:14 smithi043 ceph-mon[36454]: 9.4 scrub starts 2024-09-21T11:52:14.434 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:14 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:52:14.434 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:14 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:52:14.434 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3078564716' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:14.434 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:14 smithi043 ceph-mon[29710]: osdmap e303: 8 total, 8 up, 8 in 2024-09-21T11:52:14.434 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:14 smithi043 ceph-mon[29710]: 9.4 scrub starts 2024-09-21T11:52:14.450 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:14.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:14.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:14.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:52:14.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:14.597 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:14.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:14.598 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:14.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:14.732 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:14.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:14.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:14.743 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:52:14.747 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:52:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:14 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:52:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:14 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:52:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3078564716' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:14 smithi175 ceph-mon[32613]: osdmap e303: 8 total, 8 up, 8 in 2024-09-21T11:52:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:14 smithi175 ceph-mon[32613]: 9.4 scrub starts 2024-09-21T11:52:14.778 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:52:14.778 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:52:14.811 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:52:14.811 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:52:14.834 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:14.844 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:52:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:52:14.877 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:14.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:14.878 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:14.878 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:52:14.878 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:52:14.904 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:52:14.921 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:52:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:52:14.956 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:52:14.956 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:52:14.979 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:14.991 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:52:14.991 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:52:15.013 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:15.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:15.016 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:15.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:15.153 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:15.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:15.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:15.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:15.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:15.292 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:15.292 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:15.381 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:52:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:52:15.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:15.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:15.428 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:15.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:15.529 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:33300 9.3 9:c428ed35:::benchmark_data_smithi043_126002_object33299:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:27.237332+0000", 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 168.28785614500001, 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.134383526, 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.530 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 33300 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237332+0000", 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237332+0000", 2024-09-21T11:52:15.531 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237333+0000", 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2300000000000001e-06 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237363+0000", 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9663e-05 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237371+0000", 2024-09-21T11:52:15.532 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8739999999999995e-06 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237376+0000", 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7790000000000001e-06 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237863+0000", 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00048618499999999998 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.237990+0000", 2024-09-21T11:52:15.533 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127274 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.238162+0000", 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017167199999999999 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.240852+0000", 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026904239999999999 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.534 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.370262+0000", 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.129410105 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.371631+0000", 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013692159999999999 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.371670+0000", 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9162999999999998e-05 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:27.371715+0000", 2024-09-21T11:52:15.535 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4941000000000002e-05 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34897 9.d 9:bfc2b959:::benchmark_data_smithi043_126002_object34896:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:28.287213+0000", 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 167.237975044, 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14583664900000001, 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.536 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34897 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287213+0000", 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.537 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287213+0000", 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287214+0000", 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0100000000000004e-07 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287243+0000", 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9388999999999999e-05 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.538 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287255+0000", 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1763000000000001e-05 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.287264+0000", 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7669999999999992e-06 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.288881+0000", 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001617451 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.539 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.288998+0000", 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117126 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.289810+0000", 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00081188099999999997 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292344+0000", 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025342720000000002 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.540 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292451+0000", 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106952 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.432926+0000", 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.140474458 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.432998+0000", 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1766000000000004e-05 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.541 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.433050+0000", 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2023e-05 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34906 9.3 9:c91cddb2:::benchmark_data_smithi043_126002_object34905:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:28.292233+0000", 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 167.23295490300001, 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15032242500000001, 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.542 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.543 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.543 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.543 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.543 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34906 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292233+0000", 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292233+0000", 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.544 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.545 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292234+0000", 2024-09-21T11:52:15.545 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2950000000000001e-06 2024-09-21T11:52:15.545 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.545 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.545 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292286+0000", 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1756999999999999e-05 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292296+0000", 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0210999999999999e-05 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292302+0000", 2024-09-21T11:52:15.546 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3820000000000003e-06 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292354+0000", 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2340999999999999e-05 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292454+0000", 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100006 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.547 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292577+0000", 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122676 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293904+0000", 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0013268150000000001 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.434222+0000", 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14031828800000001 2024-09-21T11:52:15.548 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.442495+0000", 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0082727500000000006 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.442532+0000", 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7456999999999997e-05 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.549 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.442556+0000", 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3447000000000001e-05 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:34907 9.d 9:bf3be697:::benchmark_data_smithi043_126002_object34906:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:28.292968+0000", 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 167.23222050199999, 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14137534800000001, 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.550 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 34907 2024-09-21T11:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.551 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292968+0000", 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292968+0000", 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.292968+0000", 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7000000000000003e-07 2024-09-21T11:52:15.552 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293002+0000", 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3735999999999997e-05 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293012+0000", 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9559999999999992e-06 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.553 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293019+0000", 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5869999999999998e-06 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293062+0000", 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3631000000000003e-05 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293145+0000", 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2669000000000004e-05 2024-09-21T11:52:15.554 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.293228+0000", 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3251000000000006e-05 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.294386+0000", 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001157822 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.555 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.295168+0000", 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00078156199999999997 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.434254+0000", 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13908669700000001 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.434303+0000", 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8146999999999998e-05 2024-09-21T11:52:15.556 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:28.434343+0000", 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.032e-05 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:39514 9.3 9:c09b283c:::benchmark_data_smithi043_126002_object39513:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e255)", 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:49:31.204766+0000", 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 164.320422202, 2024-09-21T11:52:15.557 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14268297999999999, 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 39514 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.558 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204766+0000", 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204766+0000", 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204767+0000", 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5000000000000001e-07 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.559 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204836+0000", 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8992000000000006e-05 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204843+0000", 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5410000000000003e-06 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.560 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204848+0000", 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7060000000000003e-06 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204893+0000", 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5222000000000002e-05 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.204990+0000", 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6620000000000007e-05 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.561 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.205093+0000", 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010343799999999999 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.206689+0000", 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015956360000000001 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.562 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.207481+0000", 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079241399999999997 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.347380+0000", 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.139898839 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.347409+0000", 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8858999999999999e-05 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.563 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:49:31.347449+0000", 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9762999999999999e-05 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:119798 9.3 9:c29d2bd1:::benchmark_data_smithi043_126002_object119797:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:36.882211+0000", 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.642976833000006, 2024-09-21T11:52:15.564 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15405127199999999, 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 119798 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882211+0000", 2024-09-21T11:52:15.565 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882211+0000", 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882213+0000", 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0779999999999998e-06 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.566 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882247+0000", 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3668999999999999e-05 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882260+0000", 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2597e-05 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.882269+0000", 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9259999999999995e-06 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.567 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.893113+0000", 2024-09-21T11:52:15.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010843939 2024-09-21T11:52:15.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.568 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.568 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.893287+0000", 2024-09-21T11:52:15.568 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000174079 2024-09-21T11:52:15.568 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.568 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.569 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:52:15.569 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.893502+0000", 2024-09-21T11:52:15.569 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021474999999999999 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.902743+0000", 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092404410000000003 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.904342+0000", 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015994970000000001 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.570 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.036159+0000", 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13181647899999999 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.036209+0000", 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0541000000000003e-05 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.036262+0000", 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3276000000000003e-05 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.571 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:119810 9.3 9:c8160ff6:::benchmark_data_smithi043_126002_object119809:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:36.892458+0000", 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 98.632729882999996, 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.16747954200000001, 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 119810 2024-09-21T11:52:15.572 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892458+0000", 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892458+0000", 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.573 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892460+0000", 2024-09-21T11:52:15.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7179999999999999e-06 2024-09-21T11:52:15.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892523+0000", 2024-09-21T11:52:15.574 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3162000000000003e-05 2024-09-21T11:52:15.574 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.574 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.574 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.574 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892534+0000", 2024-09-21T11:52:15.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1422e-05 2024-09-21T11:52:15.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892541+0000", 2024-09-21T11:52:15.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9890000000000003e-06 2024-09-21T11:52:15.575 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.575 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.575 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.896348+0000", 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0038067639999999998 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.896494+0000", 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000145592 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.896641+0000", 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014765899999999999 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.904387+0000", 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0077451430000000003 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.905317+0000", 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00093077800000000005 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.059807+0000", 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15448937300000001 2024-09-21T11:52:15.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.059867+0000", 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0757000000000001e-05 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.059938+0000", 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0185000000000006e-05 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:166810 9.e 9:7b839d92:::benchmark_data_smithi043_126002_object39744:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:18.317940+0000", 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 57.207247885999998, 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.159676332, 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 166810 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.317940+0000", 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.317940+0000", 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.317942+0000", 2024-09-21T11:52:15.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.677e-06 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.317947+0000", 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0509999999999996e-06 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.317958+0000", 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1273e-05 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.317966+0000", 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4769999999999997e-06 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.318021+0000", 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5152000000000003e-05 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.318143+0000", 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012256100000000001 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.582 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.318254+0000", 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111014 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.322137+0000", 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0038828040000000001 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.322942+0000", 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00080446499999999998 2024-09-21T11:52:15.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.477512+0000", 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.154570285 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.477562+0000", 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0003000000000002e-05 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.477616+0000", 2024-09-21T11:52:15.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4570000000000001e-05 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:166816 9.3 9:cf56ab13:::benchmark_data_smithi043_126002_object39750:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:18.319216+0000", 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 57.205972453000001, 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15603789000000001, 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.585 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 166816 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.319216+0000", 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.319216+0000", 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.319217+0000", 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.128e-06 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.319221+0000", 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7879999999999997e-06 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.319232+0000", 2024-09-21T11:52:15.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.031e-05 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.319239+0000", 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0899999999999999e-06 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.319297+0000", 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8492000000000002e-05 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.319435+0000", 2024-09-21T11:52:15.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013782000000000001 2024-09-21T11:52:15.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:52:15.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.319546+0000", 2024-09-21T11:52:15.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111264 2024-09-21T11:52:15.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.325402+0000", 2024-09-21T11:52:15.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0058558400000000002 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.325463+0000", 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0272000000000002e-05 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.475169+0000", 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.149706117 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.475208+0000", 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8828999999999997e-05 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.475253+0000", 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5939999999999997e-05 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:166826 9.3 9:c85d38ec:::benchmark_data_smithi043_126002_object39760:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:18.328623+0000", 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 57.196565110999998, 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14695618299999999, 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 166826 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.593 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.328623+0000", 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.328623+0000", 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.328624+0000", 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.246e-06 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.328628+0000", 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0960000000000003e-06 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.328638+0000", 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0074e-05 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.328646+0000", 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9610000000000005e-06 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.328701+0000", 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4681e-05 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.328796+0000", 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4727000000000003e-05 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.328868+0000", 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2173000000000002e-05 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.330676+0000", 2024-09-21T11:52:15.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001808594 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.331047+0000", 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037054200000000002 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.475439+0000", 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.144392411 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.475542+0000", 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102273 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.475579+0000", 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7404999999999998e-05 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:188200 9.3 9:c98f5fb7:::benchmark_data_smithi043_126002_object61134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:52:15.599 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:36.281767+0000", 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.243421013000003, 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.16112254100000001, 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 188200 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.281767+0000", 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.281767+0000", 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.281769+0000", 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5430000000000001e-06 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.601 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.281773+0000", 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4000000000000002e-06 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.281782+0000", 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2869999999999993e-06 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.281789+0000", 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8539999999999999e-06 2024-09-21T11:52:15.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.281915+0000", 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012588700000000001 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.282034+0000", 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011875999999999999 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:52:15.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.282140+0000", 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106437 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.286985+0000", 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0048446929999999997 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.287244+0000", 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025933299999999999 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.442792+0000", 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15554828900000001 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.442839+0000", 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6520999999999997e-05 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.442890+0000", 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0537000000000001e-05 2024-09-21T11:52:15.605 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:188222 9.3 9:cc1eaa0f:::benchmark_data_smithi043_126002_object61156:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:36.300409+0000", 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 39.224779085000002, 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14562624299999999, 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.606 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 188222 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.300409+0000", 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.300409+0000", 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.300409+0000", 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9900000000000001e-07 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.300411+0000", 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0660000000000002e-06 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.300417+0000", 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7590000000000003e-06 2024-09-21T11:52:15.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.300422+0000", 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4959999999999998e-06 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.300460+0000", 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8353999999999999e-05 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.300560+0000", 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9683000000000001e-05 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.300676+0000", 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115979 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.302768+0000", 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020919770000000001 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.303106+0000", 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033813000000000002 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.445950+0000", 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14284402400000001 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.445996+0000", 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6304999999999998e-05 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.446035+0000", 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9070999999999999e-05 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211057 9.e 9:722bc66e:::benchmark_data_smithi043_126002_object83991:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.408880+0000", 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.116308257, 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.30000521899999999, 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.612 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211057 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408880+0000", 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408880+0000", 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408881+0000", 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4759999999999999e-06 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408886+0000", 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7199999999999997e-06 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408895+0000", 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0399999999999998e-06 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408902+0000", 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7490000000000001e-06 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408968+0000", 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6713999999999999e-05 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409115+0000", 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014610400000000001 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409227+0000", 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011280700000000001 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415783+0000", 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0065555459999999998 2024-09-21T11:52:15.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415844+0000", 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0736000000000003e-05 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.708802+0000", 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.292957984 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.708841+0000", 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8881999999999998e-05 2024-09-21T11:52:15.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.708885+0000", 2024-09-21T11:52:15.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4461e-05 2024-09-21T11:52:15.618 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.618 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.618 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:15.619 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:15.620 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:15.620 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211058 9.d 9:b1ac4faf:::benchmark_data_smithi043_126002_object83992:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:52:15.620 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.409106+0000", 2024-09-21T11:52:15.620 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.116081565999998, 2024-09-21T11:52:15.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29279477300000001, 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211058 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409106+0000", 2024-09-21T11:52:15.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409106+0000", 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409107+0000", 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4199999999999995e-07 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.622 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409111+0000", 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7239999999999998e-06 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409120+0000", 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7299999999999994e-06 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409126+0000", 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2840000000000003e-06 2024-09-21T11:52:15.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409187+0000", 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1270000000000001e-05 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409347+0000", 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016009800000000001 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:52:15.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409487+0000", 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140097 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.419524+0000", 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010036953 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.419667+0000", 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142695 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.701808+0000", 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.282140687 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.701849+0000", 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1121000000000002e-05 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.701901+0000", 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2472000000000002e-05 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.626 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211061 9.e 9:76133603:::benchmark_data_smithi043_126002_object83995:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.410901+0000", 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.114287499, 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.298113611, 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211061 2024-09-21T11:52:15.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410901+0000", 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410901+0000", 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410902+0000", 2024-09-21T11:52:15.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1880000000000001e-06 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410906+0000", 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.25e-06 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410922+0000", 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5665999999999999e-05 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410935+0000", 2024-09-21T11:52:15.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3487e-05 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411705+0000", 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00076967500000000005 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411820+0000", 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011558 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:52:15.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411944+0000", 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123359 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.416307+0000", 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043633209999999999 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422021+0000", 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00571434 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.708922+0000", 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.286900975 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.708971+0000", 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8937000000000003e-05 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.709014+0000", 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2833000000000001e-05 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.632 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211064 9.d 9:b03adb7e:::benchmark_data_smithi043_126002_object83998:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.410999+0000", 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.114189392, 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.30108133799999998, 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211064 2024-09-21T11:52:15.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410999+0000", 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410999+0000", 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410999+0000", 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8199999999999999e-07 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411003+0000", 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6859999999999999e-06 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411012+0000", 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0489999999999993e-06 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.635 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411019+0000", 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9539999999999998e-06 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415773+0000", 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047537710000000004 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415901+0000", 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012853800000000001 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.416029+0000", 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012789999999999999 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.419685+0000", 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036557690000000001 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.421768+0000", 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002082754 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.711984+0000", 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29021645800000001 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712036+0000", 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1657999999999999e-05 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712080+0000", 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4119000000000002e-05 2024-09-21T11:52:15.638 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211071 9.e 9:74ef5582:::benchmark_data_smithi043_126002_object84005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.418278+0000", 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.106909957999999, 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29953004500000002, 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.639 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211071 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418278+0000", 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418278+0000", 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418280+0000", 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5239999999999999e-06 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418284+0000", 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3209999999999998e-06 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418294+0000", 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8709999999999992e-06 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418301+0000", 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.328e-06 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418410+0000", 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108491 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418527+0000", 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011742499999999999 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418644+0000", 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117092 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.421227+0000", 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002582754 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422100+0000", 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00087292900000000004 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.717710+0000", 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29561067699999999 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.717764+0000", 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3575000000000001e-05 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.717808+0000", 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4057999999999997e-05 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.644 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211076 9.d 9:b6dd861d:::benchmark_data_smithi043_126002_object84010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.422179+0000", 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.103009330999999, 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29006326399999999, 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211076 2024-09-21T11:52:15.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422179+0000", 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422179+0000", 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422180+0000", 2024-09-21T11:52:15.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422184+0000", 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0749999999999999e-06 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422192+0000", 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1280000000000008e-06 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422197+0000", 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5920000000000002e-06 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422236+0000", 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9332000000000002e-05 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422315+0000", 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:52:15.648 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422403+0000", 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7632000000000004e-05 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428577+0000", 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.006174995 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428852+0000", 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027460599999999999 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712126+0000", 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28327408799999998 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712205+0000", 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8461999999999995e-05 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712242+0000", 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7295999999999997e-05 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.650 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211077 9.d 9:b5166759:::benchmark_data_smithi043_126002_object84011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.422212+0000", 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.102975544, 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.290211045, 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211077 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.651 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422212+0000", 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422212+0000", 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422213+0000", 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8599999999999998e-07 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422216+0000", 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7860000000000001e-06 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422220+0000", 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9870000000000001e-06 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422224+0000", 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6190000000000002e-06 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.424411+0000", 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021861010000000002 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.424540+0000", 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129636 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.424662+0000", 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122139 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428585+0000", 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0039231659999999996 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428888+0000", 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030278100000000001 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712282+0000", 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28339345399999999 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712387+0000", 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105746 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712424+0000", 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6044000000000003e-05 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.656 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211078 9.d 9:bb97decd:::benchmark_data_smithi043_126002_object84012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.422243+0000", 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 20.102944965999999, 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29034121800000001, 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211078 2024-09-21T11:52:15.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422243+0000", 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422243+0000", 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422244+0000", 2024-09-21T11:52:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2900000000000004e-07 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422246+0000", 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.695e-06 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422252+0000", 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4439999999999999e-06 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422255+0000", 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3100000000000001e-06 2024-09-21T11:52:15.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.426651+0000", 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043957340000000001 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.426730+0000", 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8875999999999995e-05 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.426812+0000", 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2638000000000004e-05 2024-09-21T11:52:15.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428716+0000", 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019032879999999999 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428958+0000", 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024197500000000001 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712476+0000", 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28351836899999999 2024-09-21T11:52:15.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712538+0000", 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2286999999999995e-05 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712584+0000", 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6072999999999997e-05 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:52:15.662 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:52:15.663 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:15 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T11:52:15.663 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:15 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T11:52:15.663 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:15 smithi175 ceph-mon[32613]: 6.3 scrub starts 2024-09-21T11:52:15.663 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:15 smithi175 ceph-mon[32613]: 6.3 scrub ok 2024-09-21T11:52:15.663 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:15 smithi175 ceph-mon[32613]: pgmap v585: 148 pgs: 148 active+clean; 1.5 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 849 B/s rd, 0 B/s wr, 993 op/s 2024-09-21T11:52:15.663 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1107230103' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:15.663 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:15 smithi175 ceph-mon[32613]: 9.4 scrub ok 2024-09-21T11:52:15.668 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:52:15.669 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:52:15.669 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-21T11:52:15.669 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.973777 2024-09-21T11:52:15.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 6 0.9737769815010819 2024-09-21T11:52:15.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T11:52:15.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[36454]: 6.3 scrub starts 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[36454]: 6.3 scrub ok 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[36454]: pgmap v585: 148 pgs: 148 active+clean; 1.5 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 849 B/s rd, 0 B/s wr, 993 op/s 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1107230103' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[36454]: 9.4 scrub ok 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[29710]: 6.3 scrub starts 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[29710]: 6.3 scrub ok 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[29710]: pgmap v585: 148 pgs: 148 active+clean; 1.5 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 849 B/s rd, 0 B/s wr, 993 op/s 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1107230103' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:15.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:15 smithi043 ceph-mon[29710]: 9.4 scrub ok 2024-09-21T11:52:15.721 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:15.750 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:15.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:15.751 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:15.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:15.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:15.905 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:15.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:16.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:16.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:16.068 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:16.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:16.120 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:16.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:16.199 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:16.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:16.200 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:16.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:52:16.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:16.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:16.355 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:16.355 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:16.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:16.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:16.517 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:16.517 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:16.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:52:16.618 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:16.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[29710]: 9.3 scrub starts 2024-09-21T11:52:16.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[29710]: 9.3 scrub ok 2024-09-21T11:52:16.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[29710]: 9.7 scrub starts 2024-09-21T11:52:16.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[29710]: 5.19 scrub starts 2024-09-21T11:52:16.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[29710]: 5.19 scrub ok 2024-09-21T11:52:16.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:52:16.633 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:16.634 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[36454]: 9.3 scrub starts 2024-09-21T11:52:16.634 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[36454]: 9.3 scrub ok 2024-09-21T11:52:16.634 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[36454]: 9.7 scrub starts 2024-09-21T11:52:16.634 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[36454]: 5.19 scrub starts 2024-09-21T11:52:16.634 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[36454]: 5.19 scrub ok 2024-09-21T11:52:16.634 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:52:16.634 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:16 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:16.688 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:16.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:16.689 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:16 smithi175 ceph-mon[32613]: 9.3 scrub starts 2024-09-21T11:52:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:16 smithi175 ceph-mon[32613]: 9.3 scrub ok 2024-09-21T11:52:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:16 smithi175 ceph-mon[32613]: 9.7 scrub starts 2024-09-21T11:52:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:16 smithi175 ceph-mon[32613]: 5.19 scrub starts 2024-09-21T11:52:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:16 smithi175 ceph-mon[32613]: 5.19 scrub ok 2024-09-21T11:52:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:16 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:52:16.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:16 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:16.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:16.837 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:16.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:16.838 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:16.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:16.968 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:17.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:17.010 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:17.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:17.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:17.151 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:17.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:17.152 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:17.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:17.298 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:17.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:17.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:17.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:17.403 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.6 primary-affinity to 0.973777 (8638172) 2024-09-21T11:52:17.444 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:17.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:17.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:17.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:17.584 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:17.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:17.585 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:17.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:17.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:17.715 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:17.715 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:17.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:17 smithi043 ceph-mon[36454]: pgmap v586: 148 pgs: 148 active+clean; 1.5 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 747 op/s 2024-09-21T11:52:17.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:17 smithi043 ceph-mon[36454]: 9.7 scrub ok 2024-09-21T11:52:17.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:17 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/104755838' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 0.9737769815010819} : dispatch 2024-09-21T11:52:17.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:17 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 0.9737769815010819} : dispatch 2024-09-21T11:52:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:17 smithi043 ceph-mon[29710]: pgmap v586: 148 pgs: 148 active+clean; 1.5 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 747 op/s 2024-09-21T11:52:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:17 smithi043 ceph-mon[29710]: 9.7 scrub ok 2024-09-21T11:52:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/104755838' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 0.9737769815010819} : dispatch 2024-09-21T11:52:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:17 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 0.9737769815010819} : dispatch 2024-09-21T11:52:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:17 smithi175 ceph-mon[32613]: pgmap v586: 148 pgs: 148 active+clean; 1.5 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 747 op/s 2024-09-21T11:52:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:17 smithi175 ceph-mon[32613]: 9.7 scrub ok 2024-09-21T11:52:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:17 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/104755838' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 0.9737769815010819} : dispatch 2024-09-21T11:52:17.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:17 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 0.9737769815010819} : dispatch 2024-09-21T11:52:17.790 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:17.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:17.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:17.859 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:17.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:17.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:52:17.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:18.017 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:18.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:18.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:18.118 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:18.146 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:18.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:18.147 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:18.247 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:18.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:18.291 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:18.292 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:18.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:18.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:18.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:18.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:18.435 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:52:18.529 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:18.567 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:18.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:18.568 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:18.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:18.707 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:18.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:18.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:18.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:18 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.9737769815010819}]': finished 2024-09-21T11:52:18.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:18 smithi043 ceph-mon[36454]: osdmap e304: 8 total, 8 up, 8 in 2024-09-21T11:52:18.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:18 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2013548678' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:18.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:18 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:18 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.9737769815010819}]': finished 2024-09-21T11:52:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:18 smithi043 ceph-mon[29710]: osdmap e304: 8 total, 8 up, 8 in 2024-09-21T11:52:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:18 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2013548678' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:18 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:18.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:18 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 0.9737769815010819}]': finished 2024-09-21T11:52:18.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:18 smithi175 ceph-mon[32613]: osdmap e304: 8 total, 8 up, 8 in 2024-09-21T11:52:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:18 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2013548678' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:18.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:18 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:18.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:18.847 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:18.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:18.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:18.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:18.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:18.989 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:18.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:19.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:19.120 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:19.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:19.121 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:19.222 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:19.252 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:19.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:19.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:19.353 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:19.391 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:19.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:19.392 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:19.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:19.526 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:19.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:19.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:19.627 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:19.656 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:19.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:19.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[36454]: 2.1 scrub starts 2024-09-21T11:52:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[36454]: 2.1 scrub ok 2024-09-21T11:52:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[36454]: pgmap v588: 148 pgs: 148 active+clean; 1.1 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-21T11:52:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:52:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[36454]: osdmap e305: 8 total, 8 up, 8 in 2024-09-21T11:52:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[29710]: 2.1 scrub starts 2024-09-21T11:52:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[29710]: 2.1 scrub ok 2024-09-21T11:52:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[29710]: pgmap v588: 148 pgs: 148 active+clean; 1.1 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-21T11:52:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:52:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:19 smithi043 ceph-mon[29710]: osdmap e305: 8 total, 8 up, 8 in 2024-09-21T11:52:19.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:19.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:19 smithi175 ceph-mon[32613]: 2.1 scrub starts 2024-09-21T11:52:19.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:19 smithi175 ceph-mon[32613]: 2.1 scrub ok 2024-09-21T11:52:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:19 smithi175 ceph-mon[32613]: pgmap v588: 148 pgs: 148 active+clean; 1.1 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s 2024-09-21T11:52:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:19 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:19 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:52:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:19 smithi175 ceph-mon[32613]: osdmap e305: 8 total, 8 up, 8 in 2024-09-21T11:52:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:19.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:19.802 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:19.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:19.928 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:19.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:19.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:20.030 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:20.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:20.068 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:20.069 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:20.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:20.202 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:20.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:20.203 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:20.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:20.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:20.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:20.443 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:20.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:20.481 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:20.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:20.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:20.626 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:20.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:20.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:20.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:20.761 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:20.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:20.762 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:20.863 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:20.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:20.908 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:20.909 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:21.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:21.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:21.057 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:21.057 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:21.158 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:21.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:21.199 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:21.199 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:21.300 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:21.332 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:21.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:21.334 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:21.434 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:21.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:21.477 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:21.477 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:21.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:21.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:21.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:21.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:21 smithi043 ceph-mon[36454]: pgmap v590: 148 pgs: 148 active+clean; 1.1 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 771 B/s rd, 0 B/s wr, 910 op/s 2024-09-21T11:52:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:21 smithi043 ceph-mon[29710]: pgmap v590: 148 pgs: 148 active+clean; 1.1 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 771 B/s rd, 0 B/s wr, 910 op/s 2024-09-21T11:52:21.747 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:21.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:21.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:21.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:21 smithi175 ceph-mon[32613]: pgmap v590: 148 pgs: 148 active+clean; 1.1 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 771 B/s rd, 0 B/s wr, 910 op/s 2024-09-21T11:52:21.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:52:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:52:21.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:21.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:21.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:21.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:22.030 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:22.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:22.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:22.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:22.164 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:22.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:22.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:22.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:22.300 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:22.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:22.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:22.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:22.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:22.446 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:22.514 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:243265 9.9 9:99432de1:::benchmark_data_smithi043_126002_object116199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e306)", 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:52:22.499880+0000", 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012134300000000001, 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012165553000000001, 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 243265 2024-09-21T11:52:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.499880+0000", 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.499880+0000", 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.499882+0000", 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.35e-06 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.499886+0000", 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4220000000000002e-06 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.499897+0000", 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1198e-05 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.499904+0000", 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4690000000000002e-06 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.500007+0000", 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010300499999999999 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.500111+0000", 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000104732 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:52:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.500215+0000", 2024-09-21T11:52:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001039 2024-09-21T11:52:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.507663+0000", 2024-09-21T11:52:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0074478640000000002 2024-09-21T11:52:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.507709+0000", 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6008999999999999e-05 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:243274 9.9 9:96af58c7:::benchmark_data_smithi043_126002_object116208:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e306)", 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:52:22.501394+0000", 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010620767, 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010732652000000001, 2024-09-21T11:52:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 243274 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.501394+0000", 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.501394+0000", 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.501395+0000", 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9299999999999996e-07 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.501399+0000", 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.464e-06 2024-09-21T11:52:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.501410+0000", 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0611999999999999e-05 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.501417+0000", 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3690000000000001e-06 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.507737+0000", 2024-09-21T11:52:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063204139999999999 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.507851+0000", 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113319 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.507960+0000", 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109354 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:243280 9.9 9:9edca703:::benchmark_data_smithi043_126002_object116214:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e306)", 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:52:22.506710+0000", 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0053047459999999999, 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054784689999999997, 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 243280 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506710+0000", 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506710+0000", 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506712+0000", 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9300000000000002e-06 2024-09-21T11:52:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506718+0000", 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0260000000000004e-06 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506735+0000", 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7280000000000001e-05 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506743+0000", 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7279999999999995e-06 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.511002+0000", 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042596869999999998 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.511117+0000", 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011414899999999999 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.511231+0000", 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114775 2024-09-21T11:52:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:243281 9.9 9.1166ab09 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e306)", 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:52:22.506758+0000", 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0052566430000000001, 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054876999999999999, 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 243281 2024-09-21T11:52:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506758+0000", 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506758+0000", 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506758+0000", 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1699999999999999e-07 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506761+0000", 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3970000000000001e-06 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506776+0000", 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5344000000000001e-05 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:52:22.506782+0000", 2024-09-21T11:52:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7200000000000003e-06 2024-09-21T11:52:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:52:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 4 2024-09-21T11:52:22.532 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:52:22.533 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:22.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:22.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:22.575 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:22.575 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:22.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:22.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:22.715 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:22 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:52:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:22 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:52:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:22 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:52:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:22 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:52:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:22 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:52:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:22 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:52:22.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:22.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:22.860 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:22.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:23.000 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:23.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:23.001 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:23.101 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:23.137 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:23.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:23.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:23.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:23.281 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:23.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:23.282 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:23.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:23.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:23.429 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:23.434 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:52:23.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:23 smithi043 ceph-mon[29710]: pgmap v591: 148 pgs: 148 active+clean; 914 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:52:23.512 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:23 smithi043 ceph-mon[29710]: osdmap e306: 8 total, 8 up, 8 in 2024-09-21T11:52:23.513 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:23 smithi043 ceph-mon[36454]: pgmap v591: 148 pgs: 148 active+clean; 914 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:52:23.513 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:23 smithi043 ceph-mon[36454]: osdmap e306: 8 total, 8 up, 8 in 2024-09-21T11:52:23.530 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:23.562 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:23.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:23.563 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:23.664 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:23.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:23.692 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:23.692 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:23.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:23 smithi175 ceph-mon[32613]: pgmap v591: 148 pgs: 148 active+clean; 914 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:52:23.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:23 smithi175 ceph-mon[32613]: osdmap e306: 8 total, 8 up, 8 in 2024-09-21T11:52:23.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:23.820 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:23.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:23.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:23.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:52:23.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:23.992 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:23.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:23.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:24.053 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:52:24.053 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:52:24.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:24.095 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:52:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:52:24.120 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:24.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:24.121 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:24.131 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:52:24.131 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:52:24.172 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:52:24.173 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:52:24.207 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:52:24.207 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:52:24.222 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:24.241 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:52:24.242 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:52:24.251 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:24.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:24.252 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:24.279 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:52:24.279 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:52:24.314 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:52:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:52:24.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:24.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:24.397 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:24.458 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:24 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2488920200' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:24.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:24.538 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:24.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:24.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:24.640 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:24.671 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:24.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:24.672 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:24.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:24 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2488920200' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:24.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:24 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2488920200' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:24.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:24.809 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:24.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:24.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:24.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:24.944 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:24.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:24.945 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:25.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:25.073 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:25.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:25.074 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:25.175 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:25.205 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:25.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:25.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:25.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:25.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:25.344 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:25.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:25.483 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:25.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:25.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:52:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:52:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:52:25.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:25 smithi043 ceph-mon[36454]: pgmap v593: 148 pgs: 148 active+clean; 743 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 773 B/s rd, 0 B/s wr, 915 op/s 2024-09-21T11:52:25.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:25 smithi043 ceph-mon[29710]: pgmap v593: 148 pgs: 148 active+clean; 743 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 773 B/s rd, 0 B/s wr, 915 op/s 2024-09-21T11:52:25.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:25.619 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:25.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:25.620 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:25.721 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:25.750 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:25.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:25.751 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:25.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:25 smithi175 ceph-mon[32613]: pgmap v593: 148 pgs: 148 active+clean; 743 MiB data, 5.7 GiB used, 709 GiB / 715 GiB avail; 773 B/s rd, 0 B/s wr, 915 op/s 2024-09-21T11:52:25.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:25.885 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:25.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:25.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:25.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:26.026 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:26.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:26.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:26.031 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:52:26.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 0 1 2024-09-21T11:52:26.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:26.171 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:26.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:26.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:26.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:26.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:26.315 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:26.315 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:26.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:26.443 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:26.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:26.449 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:26.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:26.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:26.587 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:26.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:26.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:26.729 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:26.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:26.730 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:26.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:26.862 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:26.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:26.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:26.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:26.999 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:27.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:27.000 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:27.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:27.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:27.139 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:27.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:27.271 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:27.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:27.272 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:27.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:27.410 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:27.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:27.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:27.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:27.551 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:27.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:27.552 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:27.652 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:27.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:27.683 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:27.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:27.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:27 smithi043 ceph-mon[29710]: pgmap v594: 148 pgs: 148 active+clean; 743 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 757 op/s 2024-09-21T11:52:27.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:27 smithi043 ceph-mon[36454]: pgmap v594: 148 pgs: 148 active+clean; 743 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 757 op/s 2024-09-21T11:52:27.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:27 smithi175 ceph-mon[32613]: pgmap v594: 148 pgs: 148 active+clean; 743 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 757 op/s 2024-09-21T11:52:27.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:27.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:27.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:27.925 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:27.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:27.953 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:27.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:28.054 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:28.085 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:28.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:28.086 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:28.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:28.223 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:28.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:28.225 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:28.325 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:28.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:28.358 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:28.359 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:28.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:28.499 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:28.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:28.500 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:28.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:28.617 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:52:28.617 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:52:28.638 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:28.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:28.639 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:28.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:28.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:28.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:28.882 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:28.887 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:28.911 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:28.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:28.913 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:29.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:29.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:29.057 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:29.158 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:29.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:29.188 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:29.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:29.289 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:29.324 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:29.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:29.326 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:29.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:52:29.427 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:29.460 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:29.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:29.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:29.483 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:29 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:52:29.483 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:29 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:52:29.483 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:29 smithi175 ceph-mon[32613]: pgmap v595: 148 pgs: 148 active+clean; 350 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-21T11:52:29.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:29.590 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:29.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:29.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:29.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:52:29.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:52:29.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[36454]: pgmap v595: 148 pgs: 148 active+clean; 350 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-21T11:52:29.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:52:29.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:52:29.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[36454]: osdmap e307: 8 total, 8 up, 8 in 2024-09-21T11:52:29.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:52:29.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:52:29.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[29710]: pgmap v595: 148 pgs: 148 active+clean; 350 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-21T11:52:29.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:52:29.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:52:29.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:29 smithi043 ceph-mon[29710]: osdmap e307: 8 total, 8 up, 8 in 2024-09-21T11:52:29.692 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:29.717 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:29.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:29.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:29.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:29 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:52:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:29 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:52:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:29 smithi175 ceph-mon[32613]: osdmap e307: 8 total, 8 up, 8 in 2024-09-21T11:52:29.819 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:29.849 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:29.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:29.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:29.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:30.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:30.010 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:30.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:30.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:30.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:30.156 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:30.156 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:30.257 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:30.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:30.285 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:30.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:30.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:30.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:30.421 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:30.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:30.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:52:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:52:30.522 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:30.547 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:30.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:30.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:30.649 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:30.675 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:30.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:30.676 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:30.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:30 smithi175 ceph-mon[32613]: osdmap e308: 8 total, 8 up, 8 in 2024-09-21T11:52:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:30 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4288817533' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:30 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:30.777 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:30.801 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:30.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:30.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:30.902 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:30.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:30 smithi043 ceph-mon[36454]: osdmap e308: 8 total, 8 up, 8 in 2024-09-21T11:52:30.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:30 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4288817533' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:30.931 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:30 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:30.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:30 smithi043 ceph-mon[29710]: osdmap e308: 8 total, 8 up, 8 in 2024-09-21T11:52:30.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:30 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4288817533' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:30.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:30 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:30.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:30.933 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:30.933 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:31.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:31.087 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:31.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:31.088 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:31.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:31.229 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:31.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:31.230 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:31.257 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Removed 127063 objects 2024-09-21T11:52:31.257 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Clean up completed and total clean up time :107.968 2024-09-21T11:52:31.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:31.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:31.354 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:31.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:31.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:31.477 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:31.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:31.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:31.510 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T11:52:31.517 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[36454]: pgmap v598: 148 pgs: 148 active+clean; 350 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 676 B/s rd, 0 B/s wr, 831 op/s 2024-09-21T11:52:31.517 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:31.517 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:31.517 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[29710]: pgmap v598: 148 pgs: 148 active+clean; 350 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 676 B/s rd, 0 B/s wr, 831 op/s 2024-09-21T11:52:31.517 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:31.517 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:31.517 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[29710]: osdmap e309: 8 total, 8 up, 8 in 2024-09-21T11:52:31.517 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3659477107' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:52:31.579 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:31.601 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:31.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:31.602 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:31.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:31.723 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:31.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:31.724 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:31.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:52:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:52:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:31 smithi175 ceph-mon[32613]: pgmap v598: 148 pgs: 148 active+clean; 350 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 676 B/s rd, 0 B/s wr, 831 op/s 2024-09-21T11:52:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:31 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:31 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:31 smithi175 ceph-mon[32613]: osdmap e309: 8 total, 8 up, 8 in 2024-09-21T11:52:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3659477107' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:52:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:31 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:52:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:52:31.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:31.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:31.847 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:31.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:31.848 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:31.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:31.974 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:31.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:31.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:52:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:52:31.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[36454]: osdmap e309: 8 total, 8 up, 8 in 2024-09-21T11:52:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3659477107' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:52:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:52:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:52:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:32.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:32.100 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:32.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:32.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:32.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:32.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:32.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:32.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:32.356 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:32.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:32.357 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:32.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:32.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:32.484 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:32.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:32.610 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:32.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:32.611 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:32.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:32.738 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:32.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:32.739 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:32.839 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:32.866 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:32.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:32.867 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:32.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:32.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:32 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:52:32.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:32 smithi043 ceph-mon[36454]: osdmap e310: 8 total, 8 up, 8 in 2024-09-21T11:52:32.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:32.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:32 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:52:32.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:32 smithi043 ceph-mon[29710]: osdmap e310: 8 total, 8 up, 8 in 2024-09-21T11:52:32.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:32.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:32.994 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:32 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:52:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:32 smithi175 ceph-mon[32613]: osdmap e310: 8 total, 8 up, 8 in 2024-09-21T11:52:33.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:33.123 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:33.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:33.124 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:33.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:33.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:33.253 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:33.353 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:33.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 2 1 2024-09-21T11:52:33.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:33.380 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:33.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:33.435 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:33.437 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:33.480 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:33.503 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:33.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:33.504 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:33.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:33.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:33 smithi043 ceph-mon[29710]: pgmap v601: 148 pgs: 148 active+clean; 141 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 836 op/s 2024-09-21T11:52:33.612 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:33 smithi043 ceph-mon[36454]: pgmap v601: 148 pgs: 148 active+clean; 141 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 836 op/s 2024-09-21T11:52:33.627 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:33.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:33.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:33.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:52:33.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:33.751 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:33.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:33.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:33.853 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:33.878 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:33.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:33.879 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:33.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:34.005 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:34.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:34.006 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:34.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:33 smithi175 ceph-mon[32613]: pgmap v601: 148 pgs: 148 active+clean; 141 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 836 op/s 2024-09-21T11:52:34.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:34.129 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:34.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:34.130 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:34.230 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:34.256 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:34.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:34.257 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:34.358 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:34.382 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:34.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:34.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43882 8.c 8:35a2a8fe:::benchmark_data_smithi043_97042_object43881:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 433.42611041200001, 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14113139399999999, 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.403 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43882 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973963+0000", 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973964+0000", 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.28e-06 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973985+0000", 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1178e-05 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.973994+0000", 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0119999999999994e-06 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974001+0000", 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4570000000000002e-06 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974047+0000", 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6393000000000001e-05 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974162+0000", 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001149 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.974245+0000", 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2937999999999998e-05 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.975846+0000", 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001600944 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976044+0000", 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019846299999999999 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.114984+0000", 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13893909600000001 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115046+0000", 2024-09-21T11:52:34.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2924999999999998e-05 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115094+0000", 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7808e-05 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43886 8.e 8:7c60ce26:::benchmark_data_smithi043_97042_object43885:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 433.423197883, 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13826519300000001, 2024-09-21T11:52:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43886 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976875+0000", 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976877+0000", 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5090000000000001e-06 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976898+0000", 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1528000000000001e-05 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976910+0000", 2024-09-21T11:52:34.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1565e-05 2024-09-21T11:52:34.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976918+0000", 2024-09-21T11:52:34.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7090000000000002e-06 2024-09-21T11:52:34.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.976966+0000", 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8168999999999999e-05 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.977045+0000", 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9594999999999999e-05 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.977134+0000", 2024-09-21T11:52:34.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8233999999999996e-05 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.978395+0000", 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012616459999999999 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.978816+0000", 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000420522 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981299+0000", 2024-09-21T11:52:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024827949999999999 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981332+0000", 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3321999999999997e-05 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115141+0000", 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133808599 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.417 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:43890 8.6 8:68dae88a:::benchmark_data_smithi043_97042_object43889:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:52:34.417 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:52:34.417 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 433.42006984699998, 2024-09-21T11:52:34.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13554245000000001, 2024-09-21T11:52:34.417 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:52:34.418 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:34.418 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 43890 2024-09-21T11:52:34.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.418 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:52:34.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980003+0000", 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980005+0000", 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2470000000000001e-06 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980022+0000", 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7380999999999998e-05 2024-09-21T11:52:34.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980030+0000", 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0539999999999995e-06 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980035+0000", 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7090000000000001e-06 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980083+0000", 2024-09-21T11:52:34.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7709e-05 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980191+0000", 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010809799999999999 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.980327+0000", 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013611600000000001 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115211+0000", 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13488383300000001 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115245+0000", 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4292000000000003e-05 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115423+0000", 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017810499999999999 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115476+0000", 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2747000000000002e-05 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115546+0000", 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0159000000000004e-05 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:43886 8.e e172/160 ondisk, result = 0)", 2024-09-21T11:52:34.423 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 433.41884988200002, 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133923551, 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981223+0000", 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981224+0000", 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5899999999999996e-07 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981227+0000", 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8609999999999998e-06 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981234+0000", 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4039999999999998e-06 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981239+0000", 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3939999999999998e-06 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981291+0000", 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2352000000000003e-05 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:20.981296+0000", 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6079999999999998e-06 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:21.115147+0000", 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.133851373 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:46239 8.1 8:8e86b97b:::benchmark_data_smithi043_97042_object46238:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:52:34.427 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 431.80931817700002, 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.144299073, 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 46239 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:52:34.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590755+0000", 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590756+0000", 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.356e-06 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.590781+0000", 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5000000000000001e-05 2024-09-21T11:52:34.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711205+0000", 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.120423856 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711223+0000", 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.751e-05 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711326+0000", 2024-09-21T11:52:34.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103274 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711483+0000", 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015652 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.711616+0000", 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000133518 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.715084+0000", 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034678579999999999 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.715128+0000", 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3659000000000003e-05 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.734947+0000", 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019819593999999999 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.735006+0000", 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8415000000000002e-05 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:22.735054+0000", 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8513000000000001e-05 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47773 8.6 8:6c5bd339:::benchmark_data_smithi043_97042_object47772:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 430.69603745400002, 2024-09-21T11:52:34.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.141128856, 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47773 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:52:34.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704036+0000", 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704037+0000", 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-06 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704057+0000", 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0608999999999999e-05 2024-09-21T11:52:34.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704068+0000", 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0611000000000001e-05 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704077+0000", 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1710000000000008e-06 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704131+0000", 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3525e-05 2024-09-21T11:52:34.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704244+0000", 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113169 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.704377+0000", 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013258899999999999 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.706707+0000", 2024-09-21T11:52:34.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023301260000000001 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.706973+0000", 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026666 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845071+0000", 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13809718700000001 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845116+0000", 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5581999999999998e-05 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845165+0000", 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8603999999999997e-05 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47778 8.e 8:7e5bf474:::benchmark_data_smithi043_97042_object47777:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 430.69259251099999, 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13783190200000001, 2024-09-21T11:52:34.439 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47778 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707481+0000", 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707482+0000", 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2750000000000001e-06 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707504+0000", 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1684000000000001e-05 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707515+0000", 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0844e-05 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707522+0000", 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6090000000000003e-06 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707577+0000", 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4703000000000001e-05 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707687+0000", 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110474 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.707808+0000", 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012113299999999999 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845066+0000", 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.137257564 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845181+0000", 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011519 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845218+0000", 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6356000000000002e-05 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845277+0000", 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9357000000000001e-05 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845313+0000", 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5713000000000002e-05 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47781 8.6 8:69458834:::benchmark_data_smithi043_97042_object47780:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 430.69178201400001, 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13727709099999999, 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.445 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47781 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708291+0000", 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708293+0000", 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2440000000000001e-06 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708313+0000", 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0846e-05 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708324+0000", 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0563e-05 2024-09-21T11:52:34.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708331+0000", 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0820000000000001e-06 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708421+0000", 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0422999999999998e-05 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708530+0000", 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108286 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.708651+0000", 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120821 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845324+0000", 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.136673447 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845358+0000", 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3760999999999997e-05 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845478+0000", 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120512 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845531+0000", 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2714999999999999e-05 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845568+0000", 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7391000000000002e-05 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47784 8.6 8:65eeb18a:::benchmark_data_smithi043_97042_object47783:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 430.68989763500002, 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13626728099999999, 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.451 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47784 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710176+0000", 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710177+0000", 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.229e-06 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710199+0000", 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2469000000000001e-05 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710210+0000", 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0387000000000001e-05 2024-09-21T11:52:34.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710217+0000", 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8709999999999999e-06 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710448+0000", 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023145800000000001 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710539+0000", 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0539999999999997e-05 2024-09-21T11:52:34.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 6,7", 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710618+0000", 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9515e-05 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845369+0000", 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134750694 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845618+0000", 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024913600000000001 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846364+0000", 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074579699999999997 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846408+0000", 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4425000000000002e-05 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.846443+0000", 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4759999999999999e-05 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47773 8.6 e173/160 ondisk, result = 0)", 2024-09-21T11:52:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:52:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 430.689833996, 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134942961, 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710239+0000", 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710240+0000", 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5600000000000001e-07 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710242+0000", 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2239999999999998e-06 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710248+0000", 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5949999999999998e-06 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710252+0000", 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3950000000000003e-06 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845062+0000", 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13480961599999999 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845067+0000", 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1019999999999996e-06 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845182+0000", 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115573 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47778 8.e e173/160 ondisk, result = 0)", 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 430.68978621000002, 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.134785869, 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710287+0000", 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.89e-07 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710290+0000", 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5339999999999998e-06 2024-09-21T11:52:34.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710296+0000", 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0970000000000002e-06 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710301+0000", 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7809999999999996e-06 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845053+0000", 2024-09-21T11:52:34.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13475179800000001 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845061+0000", 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8439999999999998e-06 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845073+0000", 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2425999999999999e-05 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.15150.0:47781 8.6 e173/160 ondisk, result = 0)", 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 430.68911576900001, 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13441215100000001, 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710958+0000", 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5300000000000004e-07 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.710962+0000", 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.692e-06 2024-09-21T11:52:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842829+0000", 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13186700100000001 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.842844+0000", 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5281000000000002e-05 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845347+0000", 2024-09-21T11:52:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002502659 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845353+0000", 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6510000000000004e-06 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845370+0000", 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6214000000000001e-05 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:47792 8.e 8:71619300:::benchmark_data_smithi043_97042_object47791:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 430.68688823899998, 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14195813500000001, 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47792 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713185+0000", 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713186+0000", 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4109999999999999e-06 2024-09-21T11:52:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.713211+0000", 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4343999999999999e-05 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.843341+0000", 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13012984499999999 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.843353+0000", 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2554e-05 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845654+0000", 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023008730000000002 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845787+0000", 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013336700000000001 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.845921+0000", 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000134031 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.853059+0000", 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0071370510000000002 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.853133+0000", 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4765000000000007e-05 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855050+0000", 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019165880000000001 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855099+0000", 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8680000000000001e-05 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:23.855143+0000", 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4626000000000003e-05 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:66142 8.1 8:87e3d573:::benchmark_data_smithi043_97042_object66141:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 418.47384784899998, 2024-09-21T11:52:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13618391299999999, 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 66142 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:52:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926225+0000", 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926226+0000", 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3500000000000005e-07 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926267+0000", 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0932e-05 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926277+0000", 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8609999999999993e-06 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926283+0000", 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4010000000000002e-06 2024-09-21T11:52:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926387+0000", 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103999 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926630+0000", 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000242938 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,5", 2024-09-21T11:52:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.926754+0000", 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123433 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.930232+0000", 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034784540000000002 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:35.930829+0000", 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00059640099999999998 2024-09-21T11:52:34.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.059677+0000", 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12884799699999999 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.059734+0000", 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7191999999999997e-05 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:45:36.062409+0000", 2024-09-21T11:52:34.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026754700000000001 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119715 9.b 9:d24acb6c:::benchmark_data_smithi043_126002_object119714:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.74842749, 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.205727302, 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.483 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119715 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651647+0000", 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651670+0000", 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2603999999999999e-05 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651679+0000", 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2490000000000007e-06 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651719+0000", 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0123999999999997e-05 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652107+0000", 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00038846399999999999 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652254+0000", 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014700100000000001 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652440+0000", 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018522499999999999 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851244+0000", 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.198804392 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851269+0000", 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5381999999999999e-05 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857251+0000", 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059817589999999997 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857317+0000", 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5874e-05 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857373+0000", 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6014000000000003e-05 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.488 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119721 9.5 9:a09f84b6:::benchmark_data_smithi043_126002_object119720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.74365020099999, 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20659565899999999, 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119721 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.489 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656425+0000", 2024-09-21T11:52:34.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.725e-06 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656448+0000", 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3238999999999999e-05 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656461+0000", 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.331e-05 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656470+0000", 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7940000000000008e-06 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656563+0000", 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2993999999999998e-05 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656754+0000", 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019041800000000001 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656917+0000", 2024-09-21T11:52:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016343600000000001 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.856079+0000", 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.199162268 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.858867+0000", 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002787253 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.862926+0000", 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040595980000000002 2024-09-21T11:52:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.862979+0000", 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3369999999999999e-05 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.863019+0000", 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9254000000000001e-05 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.494 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119722 9.9 9:9c70bd2d:::benchmark_data_smithi043_126002_object119721:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.74357689199999, 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.21088224999999999, 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119722 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656497+0000", 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4499999999999997e-07 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656518+0000", 2024-09-21T11:52:34.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1294e-05 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656526+0000", 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7570000000000004e-06 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656532+0000", 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656639+0000", 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010705999999999999 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656753+0000", 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011366099999999999 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657088+0000", 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00033480000000000001 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.860812+0000", 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.203723983 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.860856+0000", 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3983999999999998e-05 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867254+0000", 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063985939999999996 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867337+0000", 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2649000000000001e-05 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867379+0000", 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1854000000000003e-05 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:119715 9.b e277/247 ondisk, result = 0)", 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.74351154999999, 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19469081599999999, 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656563+0000", 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2699999999999998e-07 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656565+0000", 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5900000000000002e-06 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656572+0000", 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8730000000000001e-06 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656577+0000", 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3020000000000002e-06 2024-09-21T11:52:34.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851228+0000", 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19465021800000001 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851237+0000", 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2639999999999997e-06 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851253+0000", 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5741999999999999e-05 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.503 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:119715 9.b e277/247 ondisk, result = 0)", 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.74271246799999, 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20004322699999999, 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657362+0000", 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2799999999999995e-07 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657398+0000", 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6180000000000003e-05 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851962+0000", 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19456380700000001 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851969+0000", 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2470000000000002e-06 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857236+0000", 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052674760000000001 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857245+0000", 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3350000000000007e-06 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857404+0000", 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015935400000000001 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119726 9.9 9:9360eb09:::benchmark_data_smithi043_126002_object119725:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 117.740911145, 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20832704299999999, 2024-09-21T11:52:34.507 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119726 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:52:34.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659164+0000", 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2450000000000001e-06 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659190+0000", 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5653999999999999e-05 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851168+0000", 2024-09-21T11:52:34.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.191977544 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851185+0000", 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7118000000000001e-05 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854231+0000", 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003045889 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:52:34.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854382+0000", 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015179700000000001 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854513+0000", 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013041499999999999 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867241+0000", 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012728013 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867388+0000", 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147128 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867435+0000", 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6724000000000002e-05 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867467+0000", 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1986000000000002e-05 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:52:34.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867489+0000", 2024-09-21T11:52:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2529999999999999e-05 2024-09-21T11:52:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:52:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:52:34.513 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:52:34.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:34.527 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:52:34.527 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:52:34.527 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:52:34.528 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:52:34.528 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:52:34.528 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:52:34.528 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:52:34.528 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:52:34.529 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:52:34.529 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:52:34.529 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:52:34.536 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:34.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:34.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:34.539 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-21T11:52:34.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-21T11:52:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3180688097' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3180688097' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:34.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:34.660 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:34.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:34.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:34.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:34.783 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:34.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:34.784 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:34.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:34.907 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:34.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:34.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:35.008 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:35.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3180688097' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:35.033 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:35.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:35.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:35.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:35.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:35.158 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:35.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:35.286 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:35.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:35.287 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:35.353 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:35.388 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:35.413 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:35.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:35.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:52:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:52:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:52:35.515 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:35.539 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:35.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:35.540 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:35.641 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:35.666 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:35.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:35.667 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:35.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:35.792 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:35.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:35.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:35.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:35 smithi043 ceph-mon[36454]: pgmap v602: 148 pgs: 148 active+clean; 2.4 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 925 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:52:35.819 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:35 smithi043 ceph-mon[29710]: pgmap v602: 148 pgs: 148 active+clean; 2.4 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 925 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:52:35.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:35.919 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:35.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:35.920 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:35 smithi175 ceph-mon[32613]: pgmap v602: 148 pgs: 148 active+clean; 2.4 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 925 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-21T11:52:36.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:36.045 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:36.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:36.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:36.146 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:36.172 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:36.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:36.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:36.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:36.298 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:36.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:36.299 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:36.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:36.422 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:36.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:52:36.427 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:36.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:36.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:36.530 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:36.555 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:36.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:36.556 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:36.657 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:36.682 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:36.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:36.683 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:36.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:36.807 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:36.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:36.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:36.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:36.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:36.935 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:36.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:37.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:37.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:37.063 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:37.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:37.163 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:37.188 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:37.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:37.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:37.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:37.313 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:37.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:37.314 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:37.376 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:37.416 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:37.442 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:37.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:37.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:37.535 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:52:37.543 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:37.568 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:37.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:37.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:37.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:37.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:37.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:37.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:37.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:37 smithi043 ceph-mon[29710]: pgmap v603: 148 pgs: 148 active+clean; 2.4 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 784 B/s rd, 0 B/s wr, 852 op/s 2024-09-21T11:52:37.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:37 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2905183093' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:37.801 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:37 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:37.801 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:37 smithi043 ceph-mon[36454]: pgmap v603: 148 pgs: 148 active+clean; 2.4 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 784 B/s rd, 0 B/s wr, 852 op/s 2024-09-21T11:52:37.801 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:37 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2905183093' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:37.801 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:37 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:37.817 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:37.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:37.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:37.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:52:37.919 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:37.943 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:37.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:37.944 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:37 smithi175 ceph-mon[32613]: pgmap v603: 148 pgs: 148 active+clean; 2.4 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 784 B/s rd, 0 B/s wr, 852 op/s 2024-09-21T11:52:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:37 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2905183093' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:37 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:52:38.045 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:38.069 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:38.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:38.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:38.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:38.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:38.196 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:38.196 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:38.297 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:38.319 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:38.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:38.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:38.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:38.443 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:38.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:38.444 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:38.538 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'unique_pool_2' removed 2024-09-21T11:52:38.542 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T11:52:38.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:38.574 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:38.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:38.575 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:38.676 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:38.698 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:38.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:38.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:52:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[29710]: osdmap e311: 8 total, 8 up, 8 in 2024-09-21T11:52:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3315742311' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:52:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:52:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[29710]: 5.3 scrub starts 2024-09-21T11:52:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[29710]: 5.3 scrub ok 2024-09-21T11:52:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/417836586' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:52:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[29710]: 6.4 scrub starts 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[29710]: 6.4 scrub ok 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[36454]: osdmap e311: 8 total, 8 up, 8 in 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3315742311' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[36454]: 5.3 scrub starts 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[36454]: 5.3 scrub ok 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/417836586' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[36454]: 6.4 scrub starts 2024-09-21T11:52:38.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:38 smithi043 ceph-mon[36454]: 6.4 scrub ok 2024-09-21T11:52:38.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:38.821 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:38.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:38.822 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:38.923 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:38.944 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:38.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:38.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:38 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:38 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:52:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:38 smithi175 ceph-mon[32613]: osdmap e311: 8 total, 8 up, 8 in 2024-09-21T11:52:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:38 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3315742311' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:52:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:38 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:52:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:38 smithi175 ceph-mon[32613]: 5.3 scrub starts 2024-09-21T11:52:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:38 smithi175 ceph-mon[32613]: 5.3 scrub ok 2024-09-21T11:52:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:38 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/417836586' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:52:39.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:38 smithi175 ceph-mon[32613]: 6.4 scrub starts 2024-09-21T11:52:39.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:38 smithi175 ceph-mon[32613]: 6.4 scrub ok 2024-09-21T11:52:39.045 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:39.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:39.067 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:39.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:39.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:39.191 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:39.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:39.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:39.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:39.311 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-21T11:52:39.312 INFO:tasks.radosbench:Beginning radosbench... 2024-09-21T11:52:39.312 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-21T11:52:39.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool create unique_pool_3 16 2024-09-21T11:52:39.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:39.321 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:39.321 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:39.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:39.449 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:39.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:39.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:39.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:39.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[29710]: pgmap v605: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 745 op/s 2024-09-21T11:52:39.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:52:39.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/417836586' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:52:39.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[29710]: osdmap e312: 8 total, 8 up, 8 in 2024-09-21T11:52:39.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T11:52:39.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T11:52:39.558 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[29710]: 4.f scrub starts 2024-09-21T11:52:39.573 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:39.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:39.574 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:39.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:39.698 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:39.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:39.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:52:39.712 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:52:39.712 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-21T11:52:39.712 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:52:39.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:39.821 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:39.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:39.822 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:39.923 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:39.944 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:39.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:39.945 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:39.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[29710]: 4.f scrub ok 2024-09-21T11:52:39.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[36454]: pgmap v605: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 745 op/s 2024-09-21T11:52:39.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:52:39.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/417836586' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:52:39.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[36454]: osdmap e312: 8 total, 8 up, 8 in 2024-09-21T11:52:39.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T11:52:39.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T11:52:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[36454]: 4.f scrub starts 2024-09-21T11:52:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:39 smithi043 ceph-mon[36454]: 4.f scrub ok 2024-09-21T11:52:40.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:39 smithi175 ceph-mon[32613]: pgmap v605: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 745 op/s 2024-09-21T11:52:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:39 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:52:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/417836586' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:52:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:39 smithi175 ceph-mon[32613]: osdmap e312: 8 total, 8 up, 8 in 2024-09-21T11:52:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:39 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T11:52:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:39 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T11:52:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:39 smithi175 ceph-mon[32613]: 4.f scrub starts 2024-09-21T11:52:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:39 smithi175 ceph-mon[32613]: 4.f scrub ok 2024-09-21T11:52:40.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:40.068 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:40.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:40.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:40.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:40.190 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:40.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:40.191 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:40.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:40.315 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:40.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:40.317 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:40.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:40.440 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:40.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:40.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:40.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:52:40.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:52:40.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:40.597 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:40.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:40.598 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:40.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:40.723 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:40.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:40.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:40 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T11:52:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:40 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T11:52:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:40 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T11:52:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:40 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T11:52:40.825 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:40.847 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:40.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:40.848 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:40.949 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:40.970 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:40.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:40.971 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:40 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T11:52:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:40 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T11:52:41.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:41.094 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:41.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:41.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:41.196 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:41.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 3 1 2024-09-21T11:52:41.217 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:41.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:41.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:41.305 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:41.310 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:41.310 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:41.313 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:41.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:41.340 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:41.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:41.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:41.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:41.463 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:41.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:41.463 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:41.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:41.586 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:41.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:41.587 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:41.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:41.711 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:41.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:41.712 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:41.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:41 smithi043 ceph-mon[36454]: pgmap v607: 132 pgs: 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-21T11:52:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:41 smithi043 ceph-mon[29710]: pgmap v607: 132 pgs: 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-21T11:52:41.812 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:41.834 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:41.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:41.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:41.861 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:52:41.861 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:52:41.878 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:52:41.878 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:52:41.896 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:52:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:52:41.912 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:52:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:52:41.929 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:52:41.930 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:52:41.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:41.946 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:52:41.947 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:52:41.958 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:41.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:41.959 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:41.964 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:52:41.964 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:52:41.981 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:52:41.981 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:52:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:41 smithi175 ceph-mon[32613]: pgmap v607: 132 pgs: 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-21T11:52:42.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:52:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:52:42.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:42.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:42.083 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:42.083 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:42.184 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:42.205 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:42.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:42.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:42.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:42.330 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:42.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:42.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:42.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:42.453 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:42.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:42.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:42.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:42.567 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:52:42.567 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:52:42.577 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:42.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:42.578 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:42.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:42.701 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:42.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:42.702 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:42 smithi043 ceph-mon[36454]: 5.17 scrub starts 2024-09-21T11:52:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:42 smithi043 ceph-mon[36454]: 5.17 scrub ok 2024-09-21T11:52:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2392890729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:42 smithi043 ceph-mon[29710]: 5.17 scrub starts 2024-09-21T11:52:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:42 smithi043 ceph-mon[29710]: 5.17 scrub ok 2024-09-21T11:52:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2392890729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:42.803 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:42.827 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:42.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:42.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:42.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:42.953 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:42.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:42.954 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:43.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:42 smithi175 ceph-mon[32613]: 5.17 scrub starts 2024-09-21T11:52:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:42 smithi175 ceph-mon[32613]: 5.17 scrub ok 2024-09-21T11:52:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2392890729' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:43.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:43.080 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:43.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:43.082 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:43.183 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:43.208 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:43.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:43.209 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:43.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:43.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:43.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:43.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:52:43.436 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:43.462 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:43.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:43.464 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:43.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:43.589 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:43.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:43.590 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:43.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:43.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:43.714 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:43.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:43.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:43.839 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:43.839 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:43.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:43.966 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:43.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:43.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[29710]: pgmap v608: 132 pgs: 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:52:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[29710]: 3.7 scrub starts 2024-09-21T11:52:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[29710]: 3.7 scrub ok 2024-09-21T11:52:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[29710]: 3.10 scrub starts 2024-09-21T11:52:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[29710]: 3.10 scrub ok 2024-09-21T11:52:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[29710]: 3.f scrub starts 2024-09-21T11:52:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[29710]: 3.f scrub ok 2024-09-21T11:52:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[36454]: pgmap v608: 132 pgs: 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:52:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[36454]: 3.7 scrub starts 2024-09-21T11:52:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[36454]: 3.7 scrub ok 2024-09-21T11:52:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[36454]: 3.10 scrub starts 2024-09-21T11:52:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[36454]: 3.10 scrub ok 2024-09-21T11:52:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[36454]: 3.f scrub starts 2024-09-21T11:52:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:43 smithi043 ceph-mon[36454]: 3.f scrub ok 2024-09-21T11:52:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:43 smithi175 ceph-mon[32613]: pgmap v608: 132 pgs: 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:52:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:43 smithi175 ceph-mon[32613]: 3.7 scrub starts 2024-09-21T11:52:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:43 smithi175 ceph-mon[32613]: 3.7 scrub ok 2024-09-21T11:52:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:43 smithi175 ceph-mon[32613]: 3.10 scrub starts 2024-09-21T11:52:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:43 smithi175 ceph-mon[32613]: 3.10 scrub ok 2024-09-21T11:52:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:43 smithi175 ceph-mon[32613]: 3.f scrub starts 2024-09-21T11:52:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:43 smithi175 ceph-mon[32613]: 3.f scrub ok 2024-09-21T11:52:44.068 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:44.093 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:44.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:44.094 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:44.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:44.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:44.218 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:44.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:44.341 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:44.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:44.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:44.443 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:44.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:44.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:44.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:44.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:44.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:44.596 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:44.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:44.598 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'unique_pool_3' created 2024-09-21T11:52:44.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:44.719 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:44.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:44.720 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:44.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:44 smithi043 ceph-mon[29710]: 6.0 scrub starts 2024-09-21T11:52:44.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:44 smithi043 ceph-mon[29710]: 6.0 scrub ok 2024-09-21T11:52:44.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2843448326' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-21T11:52:44.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:44 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-21T11:52:44.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:44 smithi043 ceph-mon[36454]: 6.0 scrub starts 2024-09-21T11:52:44.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:44 smithi043 ceph-mon[36454]: 6.0 scrub ok 2024-09-21T11:52:44.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2843448326' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-21T11:52:44.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:44 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-21T11:52:44.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:44.843 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:44.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:44.844 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:44.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:44.967 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:44.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:44.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:44.975 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:44.976 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:44 smithi175 ceph-mon[32613]: 6.0 scrub starts 2024-09-21T11:52:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:44 smithi175 ceph-mon[32613]: 6.0 scrub ok 2024-09-21T11:52:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2843448326' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-21T11:52:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:44 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-21T11:52:45.068 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:45.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:45.094 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:45.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:45.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:45.218 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:45.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:45.219 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:45.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:45.345 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:45.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:45.346 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:45.377 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:52:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:52:45.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-21T11:52:45.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:45.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:45.471 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:45.471 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:45.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:45.598 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:45.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:45.599 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:45.599 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:52:45.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: pgmap v609: 132 pgs: 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:52:45.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: 6.1f scrub starts 2024-09-21T11:52:45.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: 6.1f scrub ok 2024-09-21T11:52:45.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-21T11:52:45.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: osdmap e313: 8 total, 8 up, 8 in 2024-09-21T11:52:45.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/363210343' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:45.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:45.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: 5.c scrub starts 2024-09-21T11:52:45.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: 5.c scrub ok 2024-09-21T11:52:45.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: 6.12 scrub starts 2024-09-21T11:52:45.672 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[36454]: 6.12 scrub ok 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: pgmap v609: 132 pgs: 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: 6.1f scrub starts 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: 6.1f scrub ok 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: osdmap e313: 8 total, 8 up, 8 in 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/363210343' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: 5.c scrub starts 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: 5.c scrub ok 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: 6.12 scrub starts 2024-09-21T11:52:45.673 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:45 smithi043 ceph-mon[29710]: 6.12 scrub ok 2024-09-21T11:52:45.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:45.724 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:45.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:45.725 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:45.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:45.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:45.851 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:45.852 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:45.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:45.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:45.977 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:45.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: pgmap v609: 132 pgs: 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:52:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: 6.1f scrub starts 2024-09-21T11:52:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: 6.1f scrub ok 2024-09-21T11:52:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-21T11:52:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: osdmap e313: 8 total, 8 up, 8 in 2024-09-21T11:52:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/363210343' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:52:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: 5.c scrub starts 2024-09-21T11:52:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: 5.c scrub ok 2024-09-21T11:52:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: 6.12 scrub starts 2024-09-21T11:52:46.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:45 smithi175 ceph-mon[32613]: 6.12 scrub ok 2024-09-21T11:52:46.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:52:46.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:46.103 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:46.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:46.105 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:46.203 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:52:46.204 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:52:46.205 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:46.230 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:46.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:46.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:46.332 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:46.358 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:46.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:46.359 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:46.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:46.484 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:46.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:46.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:46.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:46.612 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:46.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:46.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:46.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:52:46.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:52:46.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:46.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[29710]: osdmap e314: 8 total, 8 up, 8 in 2024-09-21T11:52:46.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[29710]: 3.12 scrub starts 2024-09-21T11:52:46.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[29710]: 3.12 scrub ok 2024-09-21T11:52:46.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:52:46.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:46.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:52:46.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:52:46.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:46.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[36454]: osdmap e314: 8 total, 8 up, 8 in 2024-09-21T11:52:46.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[36454]: 3.12 scrub starts 2024-09-21T11:52:46.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[36454]: 3.12 scrub ok 2024-09-21T11:52:46.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:52:46.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:46 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:46.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:46.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:46.738 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:46.738 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:46.839 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:46.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:46.866 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:46.866 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:46.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:46.991 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:46.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:46.992 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:47.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:46 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:52:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:46 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:52:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:46 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:52:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:46 smithi175 ceph-mon[32613]: osdmap e314: 8 total, 8 up, 8 in 2024-09-21T11:52:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:46 smithi175 ceph-mon[32613]: 3.12 scrub starts 2024-09-21T11:52:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:46 smithi175 ceph-mon[32613]: 3.12 scrub ok 2024-09-21T11:52:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:46 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:52:47.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:46 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:52:47.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:52:47.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:47.120 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:47.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:47.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:47.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:47.248 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:47.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:47.249 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:47.350 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:47.376 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:47.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:47.377 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:47.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:47.500 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:47.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:47.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:47.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:47.615 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T11:52:47.628 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:47.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:47.629 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:47.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:47.752 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:47.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:47.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:47.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:47.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:47.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:47.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:47.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[29710]: pgmap v612: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 683 B/s rd, 0 op/s 2024-09-21T11:52:47.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:52:47.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:52:47.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[29710]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:52:47.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[29710]: osdmap e315: 8 total, 8 up, 8 in 2024-09-21T11:52:47.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1284671846' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:52:47.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[36454]: pgmap v612: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 683 B/s rd, 0 op/s 2024-09-21T11:52:47.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:52:47.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:52:47.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[36454]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:52:47.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[36454]: osdmap e315: 8 total, 8 up, 8 in 2024-09-21T11:52:47.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1284671846' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:52:48.006 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:48.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:48.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:48.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:47 smithi175 ceph-mon[32613]: pgmap v612: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 683 B/s rd, 0 op/s 2024-09-21T11:52:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:47 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:52:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:47 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:52:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:47 smithi175 ceph-mon[32613]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:52:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:47 smithi175 ceph-mon[32613]: osdmap e315: 8 total, 8 up, 8 in 2024-09-21T11:52:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1284671846' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:52:48.024 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:48.033 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:48.108 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:48.134 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:48.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:48.135 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:48.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:48.260 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:48.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:48.261 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:48.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:48.384 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:48.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:48.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:48.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:48.508 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:48.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:48.509 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:48.610 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:48.634 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:48.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:48.635 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:48.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:48.759 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:48.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:48.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:48.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 4 1 2024-09-21T11:52:48.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:48.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:48 smithi043 ceph-mon[29710]: 4.0 scrub starts 2024-09-21T11:52:48.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:48 smithi043 ceph-mon[29710]: 4.0 scrub ok 2024-09-21T11:52:48.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:48 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1284671846' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:52:48.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:48 smithi043 ceph-mon[29710]: osdmap e316: 8 total, 8 up, 8 in 2024-09-21T11:52:48.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:48 smithi043 ceph-mon[36454]: 4.0 scrub starts 2024-09-21T11:52:48.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:48 smithi043 ceph-mon[36454]: 4.0 scrub ok 2024-09-21T11:52:48.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:48 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1284671846' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:52:48.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:48 smithi043 ceph-mon[36454]: osdmap e316: 8 total, 8 up, 8 in 2024-09-21T11:52:48.899 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:48.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:48.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:49.001 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:49.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:48 smithi175 ceph-mon[32613]: 4.0 scrub starts 2024-09-21T11:52:49.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:48 smithi175 ceph-mon[32613]: 4.0 scrub ok 2024-09-21T11:52:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:48 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1284671846' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:52:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:48 smithi175 ceph-mon[32613]: osdmap e316: 8 total, 8 up, 8 in 2024-09-21T11:52:49.027 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:49.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:49.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:49.101 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:49.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:49.154 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:49.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:49.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:49.256 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:49.281 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:49.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:49.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:49.383 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:49.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:49.409 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:49.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:49.510 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:49.535 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:49.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:49.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:49.555 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:49.568 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:52:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:52:49.588 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:52:49.588 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:52:49.604 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:52:49.605 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:52:49.623 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:52:49.623 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:52:49.627 INFO:teuthology.orchestra.run.smithi043.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-21T11:52:49.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:49.644 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:52:49.644 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:52:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:49.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:49.662 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:49.668 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:52:49.668 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:52:49.686 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:52:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:52:49.705 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:52:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:52:49.763 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:49.788 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:49.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:49.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:49.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:49.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:49 smithi175 ceph-mon[32613]: pgmap v615: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:52:49.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:49 smithi175 ceph-mon[32613]: 6.11 scrub starts 2024-09-21T11:52:49.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:49 smithi175 ceph-mon[32613]: 6.11 scrub ok 2024-09-21T11:52:49.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/19498949' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:52:49.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:49 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:52:49.913 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2128902016' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:49.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:49.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[29710]: pgmap v615: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:52:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[29710]: 6.11 scrub starts 2024-09-21T11:52:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[29710]: 6.11 scrub ok 2024-09-21T11:52:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/19498949' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:52:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:52:49.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2128902016' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[36454]: pgmap v615: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:52:49.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[36454]: 6.11 scrub starts 2024-09-21T11:52:49.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[36454]: 6.11 scrub ok 2024-09-21T11:52:49.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/19498949' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:52:49.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:52:49.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2128902016' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:50.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:50.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:52:50.039 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:50.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:50.040 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:50.141 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:50.163 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:50.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:50.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:50.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:50.286 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:50.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:50.287 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:50.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:50.411 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:50.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:50.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:50.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:50.537 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:50.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:50.538 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:50.561 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:52:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:52:50.561 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 0 2024-09-21T11:52:50.561 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:52:50.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:50.662 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:50.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:50.663 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:50.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:50.785 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:50.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:50.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:50.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:50.908 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:50.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:50.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:50.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:50 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:52:50.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:50 smithi043 ceph-mon[36454]: osdmap e317: 8 total, 8 up, 8 in 2024-09-21T11:52:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:50 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:52:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:50 smithi043 ceph-mon[29710]: osdmap e317: 8 total, 8 up, 8 in 2024-09-21T11:52:51.010 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:50 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:52:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:50 smithi175 ceph-mon[32613]: osdmap e317: 8 total, 8 up, 8 in 2024-09-21T11:52:51.030 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:51.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:51.031 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:51.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:51.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:51.160 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:51.261 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:51.286 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:51.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:51.287 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:51.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:51.409 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:51.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:51.410 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:51.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:51.533 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:51.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:51.534 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:51.630 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T11:52:51.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:51.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:51.659 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:51.724 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[36454]: pgmap v617: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[36454]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[36454]: 6.18 scrub starts 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[36454]: 6.18 scrub ok 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/829193212' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[29710]: pgmap v617: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[29710]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[29710]: 6.18 scrub starts 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[29710]: 6.18 scrub ok 2024-09-21T11:52:51.725 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:51 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/829193212' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:52:51.760 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:51.782 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:51.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:51.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:51.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:51.906 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:51.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:51.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:52.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:51 smithi175 ceph-mon[32613]: pgmap v617: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:52:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:51 smithi175 ceph-mon[32613]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:52:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:51 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:52:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:51 smithi175 ceph-mon[32613]: 6.18 scrub starts 2024-09-21T11:52:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:51 smithi175 ceph-mon[32613]: 6.18 scrub ok 2024-09-21T11:52:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:51 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/829193212' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:52:52.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:52:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:52:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:52.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:52.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:52.133 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:52.154 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:52.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:52.155 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:52.181 DEBUG:teuthology.orchestra.run.smithi043:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-21T11:52:52.184 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-21T11:52:52.184 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-21T11:52:52.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:52.278 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:52.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:52.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:52.301 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:hints = 1 2024-09-21T11:52:52.302 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-21T11:52:52.302 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Object prefix: benchmark_data_smithi043_150678 2024-09-21T11:52:52.302 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:52:52.302 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 0 0 0 0 0 0 - 0 2024-09-21T11:52:52.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:52.433 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:52.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:52.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:52.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:52:52.535 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:52.572 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:52.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:52.574 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:52.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:52.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:52.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:52.817 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:52.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:52.857 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:52.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:52.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:52.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:52 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/829193212' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:52:52.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:52 smithi043 ceph-mon[29710]: osdmap e318: 8 total, 8 up, 8 in 2024-09-21T11:52:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:52 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/829193212' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:52:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:52 smithi043 ceph-mon[36454]: osdmap e318: 8 total, 8 up, 8 in 2024-09-21T11:52:52.998 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:52.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:52.999 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:53.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:52:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:52 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/829193212' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:52:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:52 smithi175 ceph-mon[32613]: osdmap e318: 8 total, 8 up, 8 in 2024-09-21T11:52:53.100 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:53.136 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:53.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:53.137 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:53.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:53.276 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:53.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:53.277 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:53.302 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 1 16 3573 3557 222.3 222.312 0.00266817 0.00448589 2024-09-21T11:52:53.378 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:53.417 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:53.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:53.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:53.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:53.558 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:53.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:53.559 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:53.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 5 1 2024-09-21T11:52:53.649 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:53.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:53.689 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:53.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:53.690 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:53.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:53.823 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:53.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:53.824 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:53.846 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T11:52:53.846 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T11:52:53.846 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[29710]: pgmap v619: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 945 B/s rd, 0 op/s 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[29710]: 6.e scrub starts 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[29710]: 6.e scrub ok 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[36454]: pgmap v619: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 945 B/s rd, 0 op/s 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[36454]: 6.e scrub starts 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[36454]: 6.e scrub ok 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:52:53.847 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:53 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:52:53.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:53.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:53.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:53 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T11:52:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:53 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T11:52:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:53 smithi175 ceph-mon[32613]: pgmap v619: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 945 B/s rd, 0 op/s 2024-09-21T11:52:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:53 smithi175 ceph-mon[32613]: 6.e scrub starts 2024-09-21T11:52:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:53 smithi175 ceph-mon[32613]: 6.e scrub ok 2024-09-21T11:52:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:53 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:52:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:53 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:52:54.048 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:54.083 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:54.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:54.084 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:54.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:54.225 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:54.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:54.227 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:54.302 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 2 16 6698 6682 208.794 195.312 0.00295685 0.00477893 2024-09-21T11:52:54.327 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:54.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:54.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:54.469 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:54.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:54.506 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:54.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:54.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:54.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:54.649 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:54.749 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:54.782 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:54.782 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:52:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:52:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:52:54.782 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:52:54.783 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:52:54.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:54.783 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:54.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:54.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:54.918 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:54.918 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:54 smithi043 ceph-mon[36454]: 5.9 scrub starts 2024-09-21T11:52:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:54 smithi043 ceph-mon[36454]: 5.9 scrub ok 2024-09-21T11:52:54.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:54 smithi043 ceph-mon[29710]: 5.9 scrub starts 2024-09-21T11:52:54.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:54 smithi043 ceph-mon[29710]: 5.9 scrub ok 2024-09-21T11:52:55.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:54 smithi175 ceph-mon[32613]: 5.9 scrub starts 2024-09-21T11:52:55.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:54 smithi175 ceph-mon[32613]: 5.9 scrub ok 2024-09-21T11:52:55.019 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:55.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:52:55.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:55.056 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:55.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:55.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:55.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:55.199 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:55.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:55.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:55.304 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 3 16 9759 9743 202.962 191.312 0.00517111 0.00491796 2024-09-21T11:52:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:55.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:55.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:55.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:55.484 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:55.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:55.485 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:52:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:52:55.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:55.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:52:55.656 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:55.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:55.658 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:55.673 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:52:55.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:55.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:55.800 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:55.800 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:55.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:55.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[36454]: pgmap v620: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 798 B/s rd, 0 op/s 2024-09-21T11:52:55.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[36454]: 5.18 scrub starts 2024-09-21T11:52:55.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[36454]: 5.18 scrub ok 2024-09-21T11:52:55.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[36454]: 3.1f scrub starts 2024-09-21T11:52:55.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[36454]: 3.1f scrub ok 2024-09-21T11:52:55.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/981074802' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:55.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:55.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[29710]: pgmap v620: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 798 B/s rd, 0 op/s 2024-09-21T11:52:55.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[29710]: 5.18 scrub starts 2024-09-21T11:52:55.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[29710]: 5.18 scrub ok 2024-09-21T11:52:55.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[29710]: 3.1f scrub starts 2024-09-21T11:52:55.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[29710]: 3.1f scrub ok 2024-09-21T11:52:55.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/981074802' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:55.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:55 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:55.949 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:55.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:55.950 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:55 smithi175 ceph-mon[32613]: pgmap v620: 148 pgs: 148 active+clean; 583 KiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 798 B/s rd, 0 op/s 2024-09-21T11:52:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:55 smithi175 ceph-mon[32613]: 5.18 scrub starts 2024-09-21T11:52:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:55 smithi175 ceph-mon[32613]: 5.18 scrub ok 2024-09-21T11:52:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:55 smithi175 ceph-mon[32613]: 3.1f scrub starts 2024-09-21T11:52:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:55 smithi175 ceph-mon[32613]: 3.1f scrub ok 2024-09-21T11:52:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:55 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/981074802' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:55 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:52:56.051 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:56.086 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:56.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:56.087 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:56.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:56.228 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:56.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:56.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:56.303 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 4 16 12613 12597 196.811 178.375 0.00524052 0.00507395 2024-09-21T11:52:56.331 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:56.367 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:56.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:56.368 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:56.458 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:56.458 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:56.469 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:56.497 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:56.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:56.498 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:56.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:56.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:56.640 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:56.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[36454]: 4.1a scrub starts 2024-09-21T11:52:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[36454]: 4.1a scrub ok 2024-09-21T11:52:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:56.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:52:56.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[36454]: osdmap e319: 8 total, 8 up, 8 in 2024-09-21T11:52:56.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[36454]: pgmap v622: 148 pgs: 148 active+clean; 583 KiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 799 B/s rd, 0 op/s 2024-09-21T11:52:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[29710]: 4.1a scrub starts 2024-09-21T11:52:56.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[29710]: 4.1a scrub ok 2024-09-21T11:52:56.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:56.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:52:56.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[29710]: osdmap e319: 8 total, 8 up, 8 in 2024-09-21T11:52:56.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:56 smithi043 ceph-mon[29710]: pgmap v622: 148 pgs: 148 active+clean; 583 KiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 799 B/s rd, 0 op/s 2024-09-21T11:52:56.774 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:56.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:56.775 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:56.876 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:56.911 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:56.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:56.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:56 smithi175 ceph-mon[32613]: 4.1a scrub starts 2024-09-21T11:52:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:56 smithi175 ceph-mon[32613]: 4.1a scrub ok 2024-09-21T11:52:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:56 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:52:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:56 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:52:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:56 smithi175 ceph-mon[32613]: osdmap e319: 8 total, 8 up, 8 in 2024-09-21T11:52:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:56 smithi175 ceph-mon[32613]: pgmap v622: 148 pgs: 148 active+clean; 583 KiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 799 B/s rd, 0 op/s 2024-09-21T11:52:57.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:57.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:57.046 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:57.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:57.146 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:57.179 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:57.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:57.180 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:57.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:52:57.303 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 5 16 15348 15332 191.633 170.938 0.00261389 0.00521053 2024-09-21T11:52:57.310 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:57.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:57.311 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:52:57.354 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:52:57.355 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:52:57.367 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:52:57.367 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:52:57.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:57.413 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:52:57.414 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:52:57.452 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:52:57.452 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:52:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:57.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:57.457 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:57.489 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:52:57.489 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:52:57.528 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:52:57.528 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:52:57.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:57.565 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:52:57.565 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:52:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:52:57.587 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:57.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:57.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:57.602 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:52:57.602 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:52:57.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:57.728 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:57.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:57.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:57.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:57.870 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:57.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:57.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:57.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:52:57.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:57 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1470675786' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:57.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1470675786' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:58.010 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:58.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:57 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1470675786' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:52:58.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:58.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:52:58.112 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:58.146 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:58.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:58.147 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:58.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:52:58.285 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:58.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:58.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:52:58.303 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 6 16 17793 17777 185.161 152.812 0.0058489 0.00539464 2024-09-21T11:52:58.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:58.424 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:58.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:58.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:58.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:52:58.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:58.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:58.600 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:58.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:58.708 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T11:52:58.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:58.754 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:58.754 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:58.855 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:52:58.893 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:58.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:58.894 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:52:58.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:58 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/378524545' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:52:58.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:58 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:52:58.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:58 smithi043 ceph-mon[29710]: pgmap v623: 148 pgs: 148 active+clean; 783 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-21T11:52:58.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:58 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:58.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:58 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/378524545' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:52:58.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:58 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:52:58.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:58 smithi043 ceph-mon[36454]: pgmap v623: 148 pgs: 148 active+clean; 783 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-21T11:52:58.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:58 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:58.995 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:52:59.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:58 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/378524545' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:52:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:58 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:52:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:58 smithi175 ceph-mon[32613]: pgmap v623: 148 pgs: 148 active+clean; 783 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.57k op/s 2024-09-21T11:52:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:58 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:52:59.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:59.049 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:52:59.150 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:52:59.180 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:59.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:59.181 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:52:59.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:59.303 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 7 15 20207 20192 180.27 150.938 0.00386619 0.00554118 2024-09-21T11:52:59.318 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:59.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:59.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:59.420 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:59.456 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:59.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:59.457 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:59.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:59.593 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:59.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:59.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:59.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:52:59.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:59.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:52:59.836 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:52:59.869 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:52:59.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:52:59.871 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:52:59.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:52:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:59 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:52:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:59 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:52:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:59 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:52:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:52:59 smithi043 ceph-mon[29710]: osdmap e320: 8 total, 8 up, 8 in 2024-09-21T11:52:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:59 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:52:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:59 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:52:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:59 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:52:59.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:52:59 smithi043 ceph-mon[36454]: osdmap e320: 8 total, 8 up, 8 in 2024-09-21T11:53:00.009 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:59 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:53:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:59 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:53:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:59 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:53:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:52:59 smithi175 ceph-mon[32613]: osdmap e320: 8 total, 8 up, 8 in 2024-09-21T11:53:00.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:00.012 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:00.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:00.153 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:00.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:00.154 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:00.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:00.293 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:00.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:00.294 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:00.304 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 8 16 22374 22358 174.631 135.375 0.00764724 0.00571929 2024-09-21T11:53:00.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:00.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:00.429 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:00.429 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:00.530 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:00.563 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:00.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:00.564 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:00.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:00.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:00.706 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:00.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:00.807 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:00.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:00.829 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:00.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:00.930 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:00.964 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:00.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:00.965 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:00 smithi043 ceph-mon[29710]: pgmap v625: 148 pgs: 148 active+clean; 783 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 98 MiB/s wr, 1.56k op/s 2024-09-21T11:53:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:00 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:00 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:00 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:00 smithi043 ceph-mon[36454]: pgmap v625: 148 pgs: 148 active+clean; 783 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 98 MiB/s wr, 1.56k op/s 2024-09-21T11:53:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:00 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:00 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:00 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:00 smithi175 ceph-mon[32613]: pgmap v625: 148 pgs: 148 active+clean; 783 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 98 MiB/s wr, 1.56k op/s 2024-09-21T11:53:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:00 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:53:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:00 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:53:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:00 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:01.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:01.088 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:01.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:01.089 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:01.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:01.227 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:01.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:01.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:01.304 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 9 15 24346 24331 168.928 123.312 0.00378325 0.00591394 2024-09-21T11:53:01.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:01.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:01.360 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:01.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:01.494 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:01.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:01.495 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:01.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:01.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:01.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:01.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:01.760 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:53:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:53:01.779 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:01.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:01.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:01.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:01.903 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:01.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:01.904 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:02.005 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:02.040 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:02.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:02.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:02.142 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:02.175 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:02.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:02.176 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:02.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:02.299 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:02.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:02.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:02.304 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 10 16 26318 26302 164.354 123.188 0.00467531 0.00605159 2024-09-21T11:53:02.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:02.425 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:02.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:02.426 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:02.526 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:02.548 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:02.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:02.549 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:02.650 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:02.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:02.687 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:02.687 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:02.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:02 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:02.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:02 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:02.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:02.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:02 smithi043 ceph-mon[29710]: osdmap e321: 8 total, 8 up, 8 in 2024-09-21T11:53:02.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:02 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:02.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:02 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:02.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:02.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:02 smithi043 ceph-mon[36454]: osdmap e321: 8 total, 8 up, 8 in 2024-09-21T11:53:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:02 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:02 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:02.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:02 smithi175 ceph-mon[32613]: osdmap e321: 8 total, 8 up, 8 in 2024-09-21T11:53:02.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:02.825 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:02.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:02.826 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:02.927 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:02.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:02.964 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:02.964 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:03.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:03.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:03.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:03.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:03.245 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:03.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:03.246 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:03.304 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 11 16 28162 28146 159.889 115.25 0.00452657 0.00624883 2024-09-21T11:53:03.347 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:03.385 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:03.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:03.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:03.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:03.520 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:03.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:03.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:03.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:03.659 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:03.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:03.660 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:03.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:03 smithi043 ceph-mon[36454]: pgmap v626: 148 pgs: 148 active+clean; 1.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 179 MiB/s wr, 2.87k op/s 2024-09-21T11:53:03.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:03 smithi043 ceph-mon[29710]: pgmap v626: 148 pgs: 148 active+clean; 1.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 179 MiB/s wr, 2.87k op/s 2024-09-21T11:53:03.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:03.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:03 smithi175 ceph-mon[32613]: pgmap v626: 148 pgs: 148 active+clean; 1.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 179 MiB/s wr, 2.87k op/s 2024-09-21T11:53:03.795 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:03.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:03.796 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:03.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:03.924 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:03.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:03.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:04.026 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:04.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 6 1 2024-09-21T11:53:04.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:04.061 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:04.062 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:04.122 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:04.123 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:04.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:04.197 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:04.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:04.198 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:04.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:04.304 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 12 16 30189 30173 157.122 126.688 0.00469271 0.00635758 2024-09-21T11:53:04.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:04.343 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:04.344 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:04.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:04.484 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:04.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:04.485 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:04.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:04.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:04.622 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:04.623 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:04.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:04.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:04.762 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:04.763 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:04.863 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:04.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:04.903 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:04.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:04.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:53:05.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:05.036 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:05.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:05.037 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:05.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:05.178 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:05.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:05.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:05.280 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:05.306 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 13 16 32368 32352 155.495 136.188 0.00624849 0.00642505 2024-09-21T11:53:05.312 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:05.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:05.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:05.414 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:05.451 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:05.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:05.452 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:53:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:53:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:53:05.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:05 smithi043 ceph-mon[36454]: pgmap v628: 148 pgs: 148 active+clean; 1.5 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 190 MiB/s wr, 3.04k op/s 2024-09-21T11:53:05.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:05 smithi043 ceph-mon[29710]: pgmap v628: 148 pgs: 148 active+clean; 1.5 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 190 MiB/s wr, 3.04k op/s 2024-09-21T11:53:05.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:05.591 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:05.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:05.592 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:05.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:05.719 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:05.724 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:05.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:05 smithi175 ceph-mon[32613]: pgmap v628: 148 pgs: 148 active+clean; 1.5 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 190 MiB/s wr, 3.04k op/s 2024-09-21T11:53:05.762 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:05.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:05.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:05.815 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:05.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:05.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:05.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:05.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:06.029 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:06.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:06.031 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:06.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:06.163 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:06.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:06.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:06.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:06.303 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:06.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:06.304 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:06.306 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 14 16 34417 34401 153.535 128.062 0.0203472 0.00650685 2024-09-21T11:53:06.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:06.427 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:06.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:06.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:06.528 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:06.561 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:06.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:06.562 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:06.662 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:06.700 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:06.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:06.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:06.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:06.833 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:06.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:06.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:06.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:06.971 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:06.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:06.972 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:07.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:07.105 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:07.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:07.106 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:07.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:07.248 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:07.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:07.249 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:07.306 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 15 16 36305 36289 151.166 118 0.00302189 0.00660994 2024-09-21T11:53:07.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:07.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:07.392 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:07.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:07.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:07.542 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:07.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:07.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:07.597 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:07 smithi043 ceph-mon[36454]: pgmap v629: 148 pgs: 148 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-21T11:53:07.597 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:07 smithi043 ceph-mon[36454]: osdmap e322: 8 total, 8 up, 8 in 2024-09-21T11:53:07.597 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:07 smithi043 ceph-mon[29710]: pgmap v629: 148 pgs: 148 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-21T11:53:07.597 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:07 smithi043 ceph-mon[29710]: osdmap e322: 8 total, 8 up, 8 in 2024-09-21T11:53:07.644 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:07.674 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:07.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:07.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:07.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:53:07.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:07 smithi175 ceph-mon[32613]: pgmap v629: 148 pgs: 148 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-21T11:53:07.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:07 smithi175 ceph-mon[32613]: osdmap e322: 8 total, 8 up, 8 in 2024-09-21T11:53:07.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:07.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:07.824 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:07.925 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:07.950 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:07.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:07.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:08.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:08.091 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:08.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:08.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:08.194 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:08.230 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:08.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:08.231 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:08.306 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 16 16 38026 38010 148.44 107.562 0.0299424 0.00672792 2024-09-21T11:53:08.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:08.373 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:08.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:08.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:08.458 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T11:53:08.467 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:53:08.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:08.509 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:08.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:08.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:08.611 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:08.647 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:08.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:08.649 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:08.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:08 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:53:08.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:08 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:53:08.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:08 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3053740256' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:08.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:08 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:08.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:08 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2781551427' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:53:08.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:08 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:53:08.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:08 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:53:08.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:08 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3053740256' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:08.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:08 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:08.712 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:08 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2781551427' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:53:08.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:08.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:08 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:53:08.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:08 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:53:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:08 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3053740256' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:08 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:08 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2781551427' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:53:08.786 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:08.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:08.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:08.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:08.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:08.932 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:08.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:09.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:09.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:09.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:09.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:09.218 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:09.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:09.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:09.257 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:09.257 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:09.306 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 17 16 39864 39848 146.466 114.875 0.00699752 0.00682206 2024-09-21T11:53:09.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:09.358 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:09.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:09.359 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:09.460 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:09.495 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:09.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:09.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:09.597 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:09.632 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:09.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:09.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:09.733 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:09 smithi043 ceph-mon[36454]: pgmap v631: 148 pgs: 148 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 171 MiB/s wr, 2.73k op/s 2024-09-21T11:53:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:09 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:09 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2781551427' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:53:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:09 smithi043 ceph-mon[36454]: osdmap e323: 8 total, 8 up, 8 in 2024-09-21T11:53:09.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:09 smithi043 ceph-mon[29710]: pgmap v631: 148 pgs: 148 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 171 MiB/s wr, 2.73k op/s 2024-09-21T11:53:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:09 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:09 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2781551427' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:53:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:09 smithi043 ceph-mon[29710]: osdmap e323: 8 total, 8 up, 8 in 2024-09-21T11:53:09.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:09 smithi175 ceph-mon[32613]: pgmap v631: 148 pgs: 148 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 171 MiB/s wr, 2.73k op/s 2024-09-21T11:53:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:09 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:09 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2781551427' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:53:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:09 smithi175 ceph-mon[32613]: osdmap e323: 8 total, 8 up, 8 in 2024-09-21T11:53:09.766 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:09.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:09.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:09.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:09.904 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:09.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:09.905 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:10.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:10.043 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:10.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:10.045 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:10.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:53:10.146 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:10.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:10.212 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:10.212 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:10.306 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 18 16 41824 41808 145.133 122.5 0.00598313 0.006885 2024-09-21T11:53:10.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:10.348 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:10.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:10.349 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:10.450 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:10.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:10.483 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:10.483 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:10.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:10.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:10.628 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:10.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:10.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:10.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:10.770 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:10.771 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:10.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:10.908 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:10.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:10.910 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:11.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:11.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 7 1 2024-09-21T11:53:11.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:11.118 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:11.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:11.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:11.268 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:11.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:11.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:11.306 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 19 16 43603 43587 143.347 111.188 0.0109209 0.00697088 2024-09-21T11:53:11.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:11.422 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:11.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:11.423 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:11.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:11.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:11 smithi043 ceph-mon[36454]: pgmap v633: 148 pgs: 148 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 669 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-21T11:53:11.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:11 smithi043 ceph-mon[29710]: pgmap v633: 148 pgs: 148 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 669 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-21T11:53:11.569 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:11.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:11.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:11.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:11.716 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:11.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:11.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:11 smithi175 ceph-mon[32613]: pgmap v633: 148 pgs: 148 active+clean; 2.1 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 669 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-21T11:53:11.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:53:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:53:11.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:11.847 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:11.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:11.849 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:11.872 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:53:11.872 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:53:11.874 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:11.912 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:53:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:53:11.948 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:53:11.948 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:53:11.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:11.983 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:53:11.983 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:53:11.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:11.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:12.015 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:53:12.016 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:53:12.048 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:53:12.049 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:53:12.086 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:53:12.086 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:53:12.098 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:12.125 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:12.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:12.126 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:12.126 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:53:12.127 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:53:12.226 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:12.261 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:12.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:12.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:12.307 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:53:12.305915+0000 min lat: 0.00131482 max lat: 0.225397 avg lat: 0.00706329 2024-09-21T11:53:12.307 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:53:12.307 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 20 16 45294 45278 141.463 105.688 0.01375 0.00706329 2024-09-21T11:53:12.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:12.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:12.414 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:12.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:12.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:12.552 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:12.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:12.553 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:12.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:53:12.653 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:12.686 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:12.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:12.688 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:12.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:12 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:12.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:12 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3706094483' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:12.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:12 smithi043 ceph-mon[36454]: osdmap e324: 8 total, 8 up, 8 in 2024-09-21T11:53:12.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:12 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:12.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:12 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3706094483' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:12.729 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:12 smithi043 ceph-mon[29710]: osdmap e324: 8 total, 8 up, 8 in 2024-09-21T11:53:12.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:12 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:12 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3706094483' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:12.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:12 smithi175 ceph-mon[32613]: osdmap e324: 8 total, 8 up, 8 in 2024-09-21T11:53:12.788 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:12.817 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:12.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:12.818 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:12.913 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:53:12.913 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:53:12.913 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:53:12.914 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:53:12.914 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:53:12.918 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:12.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:12.964 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:12.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:13.064 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:13.094 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:13.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:13.095 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:13.196 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:13.217 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:13.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:13.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:13.307 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 21 16 46757 46741 139.081 91.4375 0.00509709 0.00717965 2024-09-21T11:53:13.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:13.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:13.353 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:13.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:13.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:13.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:13.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:13.600 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:13.634 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:13.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:13.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:13.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:13.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:13 smithi043 ceph-mon[36454]: pgmap v634: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-21T11:53:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2024744351' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:53:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:13 smithi043 ceph-mon[29710]: pgmap v634: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-21T11:53:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2024744351' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:53:13.758 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:13.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:13.759 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:13.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:13 smithi175 ceph-mon[32613]: pgmap v634: 148 pgs: 148 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-21T11:53:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2024744351' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:53:13.860 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:13.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:13.884 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:13.885 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:13.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:14.008 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:14.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:14.010 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:14.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:14.134 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:14.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:14.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:14.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:14.262 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:14.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:14.263 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:14.307 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 22 16 47827 47811 135.799 66.875 0.00623276 0.00735834 2024-09-21T11:53:14.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:14.406 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:14.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:14.407 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:14.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:14.545 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:14.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:14.547 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:14.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:14.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:14.685 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:14.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:14.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2024744351' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:53:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:14 smithi175 ceph-mon[32613]: osdmap e325: 8 total, 8 up, 8 in 2024-09-21T11:53:14.787 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:14.823 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:14.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:14.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:14.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:14.962 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:14.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:14.963 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:14.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2024744351' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:53:14.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:14 smithi043 ceph-mon[36454]: osdmap e325: 8 total, 8 up, 8 in 2024-09-21T11:53:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2024744351' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:53:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:14 smithi043 ceph-mon[29710]: osdmap e325: 8 total, 8 up, 8 in 2024-09-21T11:53:15.064 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:15.097 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:15.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:15.098 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:15.199 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:15.239 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:15.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:15.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:15.307 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 23 16 49520 49504 134.495 105.812 0.0086769 0.00743075 2024-09-21T11:53:15.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:15.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:15.374 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:15.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:15.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:53:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:53:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:53:15.514 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:15.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:15.515 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:15.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:15.655 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:15.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:15.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:15.757 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:15.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:15 smithi175 ceph-mon[32613]: pgmap v637: 148 pgs: 148 active+clean; 2.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-21T11:53:15.763 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:15 smithi043 ceph-mon[36454]: pgmap v637: 148 pgs: 148 active+clean; 2.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-21T11:53:15.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:15 smithi043 ceph-mon[29710]: pgmap v637: 148 pgs: 148 active+clean; 2.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-21T11:53:15.788 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:15.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:15.789 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:15.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T11:53:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:15.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:15.930 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:16.030 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:16.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:16.067 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:16.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:16.206 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:16.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:16.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:16.307 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 24 16 51332 51316 133.61 113.25 0.00566031 0.0074798 2024-09-21T11:53:16.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:16.342 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:16.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:16.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:16.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:16.475 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:16.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:16.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:16.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:16.618 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:16.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:16.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:16.623 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 1 1 2024-09-21T11:53:16.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:16.756 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:16.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:16.757 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:16.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:16 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:16.858 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:16.895 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:16.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:16.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:16 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:16.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:16 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:16.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:17.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:17.041 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:17.041 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:17.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:53:17.142 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:17.174 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:17.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:17.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:17.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:17.277 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:17.285 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:17.307 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 25 16 52915 52899 132.219 98.9375 0.0139024 0.00755673 2024-09-21T11:53:17.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:17.349 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:17.349 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:17.450 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:17.485 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:17.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:17.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:17.587 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:17.614 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:17.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:17.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:17.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:17.723 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:17 smithi043 ceph-mon[36454]: pgmap v638: 148 pgs: 148 active+clean; 2.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 675 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-21T11:53:17.724 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:17 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:53:17.724 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:17 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:53:17.724 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:17 smithi043 ceph-mon[36454]: osdmap e326: 8 total, 8 up, 8 in 2024-09-21T11:53:17.724 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:17 smithi043 ceph-mon[29710]: pgmap v638: 148 pgs: 148 active+clean; 2.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 675 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-21T11:53:17.724 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:17 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:53:17.724 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:17 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:53:17.724 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:17 smithi043 ceph-mon[29710]: osdmap e326: 8 total, 8 up, 8 in 2024-09-21T11:53:17.755 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:17.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:17.757 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:17.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:53:17.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:17.887 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:17.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:17.888 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:17.989 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:18.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:17 smithi175 ceph-mon[32613]: pgmap v638: 148 pgs: 148 active+clean; 2.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 675 B/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-21T11:53:18.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:17 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:53:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:17 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:53:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:17 smithi175 ceph-mon[32613]: osdmap e326: 8 total, 8 up, 8 in 2024-09-21T11:53:18.017 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:18.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:18.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:18.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:18.163 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:18.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:18.165 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:18.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:18.296 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:18.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:18.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:18.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 26 15 54354 54339 130.595 90 0.00744053 0.00765198 2024-09-21T11:53:18.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:18.424 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:18.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:18.517 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:18.517 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:18.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:18.658 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:18.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:18.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:18.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:18.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:18.806 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:18.806 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:18.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:18.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:18.952 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:18.952 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:19.016 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:19.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:19.075 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:53:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:53:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:53:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:53:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245657 8.b 8:db11300a:::benchmark_data_smithi043_97042_object108563:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:53:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.50370108200002, 2024-09-21T11:53:19.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167927969999999, 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245657 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565747+0000", 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5340000000000001e-06 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565752+0000", 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8459999999999999e-06 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565762+0000", 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6029999999999994e-06 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565769+0000", 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3019999999999997e-06 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565833+0000", 2024-09-21T11:53:19.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4139000000000004e-05 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565842+0000", 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2550000000000003e-06 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576490+0000", 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.010647965 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576618+0000", 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000128233 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576763+0000", 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000144845 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582393+0000", 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056301229999999999 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582458+0000", 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4715000000000006e-05 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582472+0000", 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3985000000000001e-05 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582509+0000", 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7063000000000001e-05 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582538+0000", 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9189000000000001e-05 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245660 8.b 8:d0b775d8:::benchmark_data_smithi043_97042_object108566:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.50077604500001, 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0141818570000001, 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.082 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245660 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0200000000000003e-07 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568673+0000", 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2029999999999999e-06 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568679+0000", 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7590000000000003e-06 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568684+0000", 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1800000000000004e-06 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568765+0000", 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0898999999999999e-05 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568774+0000", 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2e-06 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.579784+0000", 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0110101520000001 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.579907+0000", 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000122404 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.580042+0000", 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000135205 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582646+0000", 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026039240000000001 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582746+0000", 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100447 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582764+0000", 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7265000000000002e-05 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582813+0000", 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9487999999999998e-05 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582852+0000", 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9329000000000002e-05 2024-09-21T11:53:19.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245661 8.b 8:d3f46cc5:::benchmark_data_smithi043_97042_object108567:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.50074723099999, 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026528527, 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245661 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568700+0000", 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9200000000000001e-07 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568702+0000", 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4049999999999998e-06 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568708+0000", 2024-09-21T11:53:19.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0900000000000001e-06 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568715+0000", 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3319999999999997e-06 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568799+0000", 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4618999999999997e-05 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568806+0000", 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3239999999999999e-06 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582891+0000", 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0140855200000001 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.583014+0000", 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012294699999999999 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.583136+0000", 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121651 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588881+0000", 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057450959999999999 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588920+0000", 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9140999999999997e-05 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595113+0000", 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061930309999999999 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595177+0000", 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4243000000000001e-05 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595228+0000", 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0636000000000001e-05 2024-09-21T11:53:19.095 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245694 8.b 8:d4a2706b:::benchmark_data_smithi043_97042_object108600:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.471333926, 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99723669299999995, 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.096 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245694 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598114+0000", 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.203e-06 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598118+0000", 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3760000000000001e-06 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598129+0000", 2024-09-21T11:53:19.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0373999999999999e-05 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598137+0000", 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598208+0000", 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1235000000000005e-05 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598217+0000", 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0480000000000004e-06 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585700+0000", 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98748349499999999 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585809+0000", 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010897500000000001 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585910+0000", 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100326 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588958+0000", 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030481290000000001 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595096+0000", 2024-09-21T11:53:19.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.006137829 2024-09-21T11:53:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595241+0000", 2024-09-21T11:53:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014534899999999999 2024-09-21T11:53:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595298+0000", 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7256999999999997e-05 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595349+0000", 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1267999999999998e-05 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.103 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245711 8.b 8:d32e232b:::benchmark_data_smithi043_97042_object108617:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.45509205000002, 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98116202900000005, 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245711 2024-09-21T11:53:19.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614357+0000", 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9910000000000001e-06 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614362+0000", 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8119999999999996e-06 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614381+0000", 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8799000000000001e-05 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614390+0000", 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6289999999999999e-06 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614452+0000", 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2620999999999996e-05 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614462+0000", 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9869999999999994e-06 2024-09-21T11:53:19.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588986+0000", 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.974523374 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.589275+0000", 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028897299999999999 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.589384+0000", 2024-09-21T11:53:19.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109497 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595361+0000", 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059765479999999999 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595400+0000", 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8847999999999998e-05 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595444+0000", 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4270999999999997e-05 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595473+0000", 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9339000000000001e-05 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595517+0000", 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3340000000000002e-05 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.110 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245713 8.b 8:d46d9f23:::benchmark_data_smithi043_97042_object108619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.45428889499999, 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98051237099999999, 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245713 2024-09-21T11:53:19.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615159+0000", 2024-09-21T11:53:19.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.339e-06 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615164+0000", 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6659999999999999e-06 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615180+0000", 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5835e-05 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615188+0000", 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7600000000000008e-06 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615244+0000", 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5172e-05 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615252+0000", 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6969999999999999e-06 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592015+0000", 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97676251999999997 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592146+0000", 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000131094 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592299+0000", 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015273399999999999 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595406+0000", 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031073920000000001 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595425+0000", 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9119000000000001e-05 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595582+0000", 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001573 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595632+0000", 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9282e-05 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595670+0000", 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8461000000000003e-05 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245721 8.b 8:de2b861f:::benchmark_data_smithi043_97042_object108627:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.117 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.44638316599998, 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98811751400000003, 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245721 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623065+0000", 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.553e-06 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623072+0000", 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4729999999999997e-06 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623087+0000", 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5197999999999999e-05 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623095+0000", 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2369999999999992e-06 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623159+0000", 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3953000000000002e-05 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623168+0000", 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0350000000000007e-06 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595704+0000", 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97253655100000003 2024-09-21T11:53:19.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595813+0000", 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108168 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595916+0000", 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103482 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604767+0000", 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088508360000000008 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604858+0000", 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0768999999999998e-05 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611090+0000", 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0062325690000000003 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611136+0000", 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5197000000000001e-05 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611181+0000", 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5493e-05 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245797 8.b 8:d43bc7ef:::benchmark_data_smithi043_97042_object108703:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:53:19.124 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.36813914999999, 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.910005379, 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245797 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701309+0000", 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1999999999999999e-06 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701315+0000", 2024-09-21T11:53:19.126 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701326+0000", 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0956e-05 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701334+0000", 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7570000000000004e-06 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.127 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701426+0000", 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2200999999999994e-05 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701436+0000", 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0175e-05 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.598806+0000", 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.89737045500000001 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.598902+0000", 2024-09-21T11:53:19.128 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5308000000000003e-05 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.599142+0000", 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024078399999999999 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604823+0000", 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056808700000000002 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.129 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.610984+0000", 2024-09-21T11:53:19.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:19.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061603750000000001 2024-09-21T11:53:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611191+0000", 2024-09-21T11:53:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000207612 2024-09-21T11:53:19.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611273+0000", 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1924999999999999e-05 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611313+0000", 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9651999999999999e-05 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.132 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245872 8.b 8:d0b8f221:::benchmark_data_smithi043_97042_object108778:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.28079033199998, 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.82278842799999996, 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245872 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.133 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788658+0000", 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.708e-06 2024-09-21T11:53:19.134 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788664+0000", 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5140000000000001e-06 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788679+0000", 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5764e-05 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788691+0000", 2024-09-21T11:53:19.135 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.173e-05 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788774+0000", 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3029999999999996e-05 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788783+0000", 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9840000000000007e-06 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.136 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.601802+0000", 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81301899799999999 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.601917+0000", 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011464 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.602046+0000", 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000129228 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.137 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604894+0000", 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028480250000000001 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611032+0000", 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061384710000000004 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611319+0000", 2024-09-21T11:53:19.138 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028687299999999999 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611405+0000", 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5921999999999995e-05 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611445+0000", 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9541e-05 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.139 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245873 8.b 8:dccc5dce:::benchmark_data_smithi043_97042_object108779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.28073347899999, 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.82286793400000002, 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245873 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.140 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788714+0000", 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5300000000000004e-07 2024-09-21T11:53:19.141 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788717+0000", 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2849999999999999e-06 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788723+0000", 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3300000000000004e-06 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788729+0000", 2024-09-21T11:53:19.142 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3979999999999998e-06 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788805+0000", 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6068000000000004e-05 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788810+0000", 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4720000000000003e-06 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604923+0000", 2024-09-21T11:53:19.143 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81611315500000003 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.605021+0000", 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7503999999999993e-05 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.605122+0000", 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010059400000000001 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.144 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.610912+0000", 2024-09-21T11:53:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057902530000000004 2024-09-21T11:53:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.145 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611078+0000", 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016624000000000001 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611455+0000", 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037679500000000003 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611538+0000", 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3288999999999995e-05 2024-09-21T11:53:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611581+0000", 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2997999999999997e-05 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245890 8.b 8:ddfc19d9:::benchmark_data_smithi043_97042_object108796:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.805073+0000", 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.26437374400001, 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81340417600000003, 2024-09-21T11:53:19.147 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245890 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805073+0000", 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.148 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805073+0000", 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805074+0000", 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3799999999999999e-06 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805079+0000", 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9949999999999996e-06 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.149 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805090+0000", 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0535e-05 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805096+0000", 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6429999999999998e-06 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805183+0000", 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6765e-05 2024-09-21T11:53:19.150 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.805195+0000", 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1684e-05 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.607855+0000", 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.80266011599999998 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.607979+0000", 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012364099999999999 2024-09-21T11:53:19.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.608105+0000", 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012666500000000001 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611632+0000", 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035269120000000001 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611643+0000", 2024-09-21T11:53:19.152 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0995000000000001e-05 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618369+0000", 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0067257940000000002 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618427+0000", 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8192999999999997e-05 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618477+0000", 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9858000000000003e-05 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245900 8.b 8:da6e4cef:::benchmark_data_smithi043_97042_object108806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.814804+0000", 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.254642585, 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.809993615, 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.154 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245900 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814804+0000", 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814804+0000", 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814806+0000", 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.767e-06 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814811+0000", 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.814e-06 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814825+0000", 2024-09-21T11:53:19.156 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3949e-05 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814832+0000", 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6000000000000001e-06 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814903+0000", 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0462000000000004e-05 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.157 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.814913+0000", 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0161e-05 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611670+0000", 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79675690499999996 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611808+0000", 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013816800000000001 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.158 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611894+0000", 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6110000000000001e-05 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618523+0000", 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0066291659999999997 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624649+0000", 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061260459999999996 2024-09-21T11:53:19.159 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624696+0000", 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7271999999999997e-05 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624750+0000", 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3782000000000001e-05 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624798+0000", 2024-09-21T11:53:19.160 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7413000000000002e-05 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245905 8.b 8:dde0f4b3:::benchmark_data_smithi043_97042_object108811:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.819241+0000", 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.25020559900003, 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.80567893899999998, 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.161 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245905 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819241+0000", 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819241+0000", 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819243+0000", 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.277e-06 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819249+0000", 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4659999999999998e-06 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819276+0000", 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7506000000000001e-05 2024-09-21T11:53:19.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819285+0000", 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7239999999999998e-06 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819348+0000", 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2675999999999994e-05 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.819356+0000", 2024-09-21T11:53:19.164 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2470000000000008e-06 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.615392+0000", 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.796035728 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.615494+0000", 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102256 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.615604+0000", 2024-09-21T11:53:19.165 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011042799999999999 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624636+0000", 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0090319690000000008 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624655+0000", 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8525999999999998e-05 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.166 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624840+0000", 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018551700000000001 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624885+0000", 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4743999999999997e-05 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624920+0000", 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4875000000000001e-05 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.167 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245931 8.b 8:d43f52c0:::benchmark_data_smithi043_97042_object108837:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.852205+0000", 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.21724189899999, 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.77959361400000005, 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245931 2024-09-21T11:53:19.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852205+0000", 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852205+0000", 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852206+0000", 2024-09-21T11:53:19.169 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.09e-07 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852209+0000", 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.76e-06 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852220+0000", 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0198e-05 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.170 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852225+0000", 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8220000000000003e-06 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852292+0000", 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6230999999999997e-05 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.852301+0000", 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1339999999999993e-06 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.171 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618556+0000", 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76625512600000001 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618656+0000", 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010033400000000001 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.172 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.618785+0000", 2024-09-21T11:53:19.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012859199999999999 2024-09-21T11:53:19.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624659+0000", 2024-09-21T11:53:19.173 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058743650000000003 2024-09-21T11:53:19.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.173 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.173 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.173 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631628+0000", 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0069688379999999998 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631704+0000", 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6015999999999999e-05 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631756+0000", 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1678999999999998e-05 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.174 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631798+0000", 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.261e-05 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245933 8.b 8:d86bb2d6:::benchmark_data_smithi043_97042_object108839:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.855701+0000", 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.21374613500001, 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.79554628400000005, 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245933 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855701+0000", 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855701+0000", 2024-09-21T11:53:19.176 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855702+0000", 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1200000000000001e-06 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855706+0000", 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8879999999999999e-06 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855716+0000", 2024-09-21T11:53:19.177 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0241000000000001e-05 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855721+0000", 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.366e-06 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855764+0000", 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2432999999999998e-05 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.855770+0000", 2024-09-21T11:53:19.178 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3470000000000004e-06 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.621640+0000", 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.76586964800000001 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.621756+0000", 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116568 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.179 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.621882+0000", 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125795 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631670+0000", 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0097883689999999999 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631892+0000", 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000221797 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.180 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651081+0000", 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019188877 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651195+0000", 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113743 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651247+0000", 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2092000000000002e-05 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.181 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245945 8.b 8:d81ed0d8:::benchmark_data_smithi043_97042_object108851:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.892389+0000", 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 295.17705766400002, 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.75901874300000005, 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245945 2024-09-21T11:53:19.182 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892389+0000", 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892389+0000", 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892390+0000", 2024-09-21T11:53:19.183 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1629999999999999e-06 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892394+0000", 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8920000000000002e-06 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892404+0000", 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0132000000000001e-05 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.184 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892411+0000", 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9040000000000003e-06 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892458+0000", 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6913000000000003e-05 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.892468+0000", 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0227999999999999e-05 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.185 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.624952+0000", 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.73248412399999996 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.625090+0000", 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000138064 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.625205+0000", 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011459999999999999 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.186 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631842+0000", 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0066366690000000004 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.631897+0000", 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5130000000000002e-05 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651292+0000", 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.019394734 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.187 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651372+0000", 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0452999999999997e-05 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.651408+0000", 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5737e-05 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.188 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:211070 9.2 9:497dbf98:::benchmark_data_smithi043_126002_object84004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.416100+0000", 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 83.653346403, 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.294090729, 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 211070 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.189 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.416100+0000", 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.416100+0000", 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.416101+0000", 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8599999999999997e-07 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.416106+0000", 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6059999999999996e-06 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.416127+0000", 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1169999999999999e-05 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.416135+0000", 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6199999999999999e-06 2024-09-21T11:53:19.191 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.416209+0000", 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4785999999999999e-05 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.416345+0000", 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013531099999999999 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.416430+0000", 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5473e-05 2024-09-21T11:53:19.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.418994+0000", 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025639249999999999 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.419730+0000", 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00073622599999999998 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.709876+0000", 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29014589400000002 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.193 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.710012+0000", 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013541200000000001 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.710191+0000", 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017942000000000001 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:47521 10.4 e325/313 10:207ae421:::benchmark_data_smithi043_150678_object47520:head v 325'2996, mlcod=325'2993)", 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.841916+0000", 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2275304049999995, 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29470554799999998, 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841916+0000", 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.195 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841916+0000", 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841917+0000", 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.186e-06 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842185+0000", 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026752600000000003 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.196 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842197+0000", 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2103000000000001e-05 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842204+0000", 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.323e-06 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842318+0000", 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011409299999999999 2024-09-21T11:53:19.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842338+0000", 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9775999999999998e-05 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136574+0000", 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29423542600000002 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136622+0000", 2024-09-21T11:53:19.198 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8115000000000003e-05 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:47525 10.4 e325/313 10:29bd7892:::benchmark_data_smithi043_150678_object47524:head v 325'2997, mlcod=325'2993)", 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.842377+0000", 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2270696189999999, 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29427192000000002, 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.199 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842377+0000", 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842377+0000", 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.200 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842378+0000", 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9699999999999995e-07 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842420+0000", 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2030000000000002e-05 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842429+0000", 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4199999999999996e-06 2024-09-21T11:53:19.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842435+0000", 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7169999999999996e-06 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.132596+0000", 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29016065899999999 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.132614+0000", 2024-09-21T11:53:19.202 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8740999999999999e-05 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136626+0000", 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040116509999999998 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136649+0000", 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3005000000000001e-05 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.203 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:47529 10.3 e325/313 10:ce2591d9:::benchmark_data_smithi043_150678_object47528:head v 325'2939, mlcod=325'2937)", 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.842491+0000", 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.2269561360000001, 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29427761000000002, 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842491+0000", 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.204 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842491+0000", 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842491+0000", 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0599999999999999e-07 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842607+0000", 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115275 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.205 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842632+0000", 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4848e-05 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842640+0000", 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1240000000000005e-06 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.132323+0000", 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.289683831 2024-09-21T11:53:19.206 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.132368+0000", 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4163999999999999e-05 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136732+0000", 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043645289999999998 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136768+0000", 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5933000000000003e-05 2024-09-21T11:53:19.207 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.208 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.208 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:19.208 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:19.208 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:53:19.232 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:19.268 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:19.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:19.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:19.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 27 16 55687 55671 128.842 83.25 0.00810283 0.00775617 2024-09-21T11:53:19.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:19.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:19.399 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:19.399 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:19.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:53:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T11:53:19.536 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:19.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:19.537 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:19.638 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:19.675 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:19.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:19.676 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:19 smithi043 ceph-mon[29710]: pgmap v640: 148 pgs: 148 active+clean; 3.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-21T11:53:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:19 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3738762326' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:53:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:19 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/394871341' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:53:19.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:19 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:53:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:19 smithi043 ceph-mon[36454]: pgmap v640: 148 pgs: 148 active+clean; 3.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-21T11:53:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:19 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3738762326' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:53:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:19 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/394871341' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:53:19.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:19 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:53:19.776 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:19.812 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:19 smithi175 ceph-mon[32613]: pgmap v640: 148 pgs: 148 active+clean; 3.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-21T11:53:19.812 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:19 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3738762326' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:53:19.812 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:19 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/394871341' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:53:19.812 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:19 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:53:19.817 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:19.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:19.818 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:19.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:19.956 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:19.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:19.957 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:20.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:20.098 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:20.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:20.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:20.199 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:20.233 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:20.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:20.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:20.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 28 16 57291 57275 127.82 100.25 0.00400103 0.0078183 2024-09-21T11:53:20.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:20.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:20.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:20.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:20.512 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:20.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:20.513 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:20.614 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:20.647 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:20.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:20.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3738762326' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:53:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:53:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[29710]: osdmap e327: 8 total, 8 up, 8 in 2024-09-21T11:53:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[29710]: 5.19 scrub starts 2024-09-21T11:53:20.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[29710]: 5.19 scrub ok 2024-09-21T11:53:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3738762326' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:53:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:53:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[36454]: osdmap e327: 8 total, 8 up, 8 in 2024-09-21T11:53:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[36454]: 5.19 scrub starts 2024-09-21T11:53:20.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:20 smithi043 ceph-mon[36454]: 5.19 scrub ok 2024-09-21T11:53:20.748 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:20.783 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:20.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:20.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:20.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:20.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:20.922 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:20.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:20 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3738762326' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:53:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:20 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:53:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:20 smithi175 ceph-mon[32613]: osdmap e327: 8 total, 8 up, 8 in 2024-09-21T11:53:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:20 smithi175 ceph-mon[32613]: 5.19 scrub starts 2024-09-21T11:53:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:20 smithi175 ceph-mon[32613]: 5.19 scrub ok 2024-09-21T11:53:21.023 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:21.059 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:21.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:21.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:21.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:21.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:21.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:21.267 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T11:53:21.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:53:21.303 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:21.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 29 16 58936 58920 126.958 102.812 0.00847032 0.00787154 2024-09-21T11:53:21.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:21.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:21.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:21.492 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:21.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:21.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:21.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:21.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:21.627 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:21.628 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:21.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[29710]: pgmap v642: 148 pgs: 148 active+clean; 3.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 781 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:53:21.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[29710]: 4.6 scrub starts 2024-09-21T11:53:21.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[29710]: 4.6 scrub ok 2024-09-21T11:53:21.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:53:21.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:53:21.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[29710]: 2.1 scrub starts 2024-09-21T11:53:21.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[29710]: 2.1 scrub ok 2024-09-21T11:53:21.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[36454]: pgmap v642: 148 pgs: 148 active+clean; 3.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 781 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:53:21.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[36454]: 4.6 scrub starts 2024-09-21T11:53:21.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[36454]: 4.6 scrub ok 2024-09-21T11:53:21.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:53:21.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:53:21.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[36454]: 2.1 scrub starts 2024-09-21T11:53:21.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:21 smithi043 ceph-mon[36454]: 2.1 scrub ok 2024-09-21T11:53:21.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:21.777 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:21.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:21.778 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:21.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:53:21.819 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:21 smithi175 ceph-mon[32613]: pgmap v642: 148 pgs: 148 active+clean; 3.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 781 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:53:21.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:21 smithi175 ceph-mon[32613]: 4.6 scrub starts 2024-09-21T11:53:21.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:21 smithi175 ceph-mon[32613]: 4.6 scrub ok 2024-09-21T11:53:21.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:21 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:53:21.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:21 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:53:21.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:21 smithi175 ceph-mon[32613]: 2.1 scrub starts 2024-09-21T11:53:21.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:21 smithi175 ceph-mon[32613]: 2.1 scrub ok 2024-09-21T11:53:21.820 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:53:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:53:21.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:21.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:21.922 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:22.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:22.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:22.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:22.084 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:22.097 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:53:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:53:22.133 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:53:22.133 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:53:22.169 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:53:22.169 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:53:22.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:22.194 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:22.204 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:53:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:53:22.217 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:22.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:22.218 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:22.243 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:53:22.244 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:53:22.278 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:53:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:53:22.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 30 16 60381 60365 125.736 90.3125 0.00818893 0.00794838 2024-09-21T11:53:22.318 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:53:22.318 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:53:22.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:22.355 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:53:22.356 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:53:22.363 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:22.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:22.364 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:22.465 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:22.508 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:22.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:22.509 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:22.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:22.648 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:22.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:22.650 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:22.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2130675060' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:22 smithi175 ceph-mon[32613]: osdmap e328: 8 total, 8 up, 8 in 2024-09-21T11:53:22.774 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:22.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:22.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:22.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:22.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2130675060' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:22.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:22 smithi043 ceph-mon[29710]: osdmap e328: 8 total, 8 up, 8 in 2024-09-21T11:53:22.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:22.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2130675060' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:22.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:22 smithi043 ceph-mon[36454]: osdmap e328: 8 total, 8 up, 8 in 2024-09-21T11:53:22.876 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:22.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:22.910 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:22.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:23.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:23.058 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:23.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:23.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:23.125 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:23.125 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:23.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:53:23.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:23.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:23.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:23.255 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:53:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:53:23.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 31 16 61627 61611 124.192 77.875 0.00426826 0.00803371 2024-09-21T11:53:23.332 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:23.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:23.368 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:23.368 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:23.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:23.553 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:23.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:23.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:23.656 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:23.690 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:23.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:23.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[36454]: pgmap v643: 148 pgs: 148 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-21T11:53:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[36454]: 3.b scrub starts 2024-09-21T11:53:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[36454]: 3.b scrub ok 2024-09-21T11:53:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[36454]: 4.3 scrub starts 2024-09-21T11:53:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[36454]: 4.3 scrub ok 2024-09-21T11:53:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[36454]: 5.1d scrub starts 2024-09-21T11:53:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[36454]: 5.1d scrub ok 2024-09-21T11:53:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[29710]: pgmap v643: 148 pgs: 148 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-21T11:53:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[29710]: 3.b scrub starts 2024-09-21T11:53:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[29710]: 3.b scrub ok 2024-09-21T11:53:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[29710]: 4.3 scrub starts 2024-09-21T11:53:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[29710]: 4.3 scrub ok 2024-09-21T11:53:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[29710]: 5.1d scrub starts 2024-09-21T11:53:23.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:23 smithi043 ceph-mon[29710]: 5.1d scrub ok 2024-09-21T11:53:23.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:23.814 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:23.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:23.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:23.916 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:23.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:23.952 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:23.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:23 smithi175 ceph-mon[32613]: pgmap v643: 148 pgs: 148 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-21T11:53:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:23 smithi175 ceph-mon[32613]: 3.b scrub starts 2024-09-21T11:53:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:23 smithi175 ceph-mon[32613]: 3.b scrub ok 2024-09-21T11:53:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:23 smithi175 ceph-mon[32613]: 4.3 scrub starts 2024-09-21T11:53:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:23 smithi175 ceph-mon[32613]: 4.3 scrub ok 2024-09-21T11:53:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:23 smithi175 ceph-mon[32613]: 5.1d scrub starts 2024-09-21T11:53:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:23 smithi175 ceph-mon[32613]: 5.1d scrub ok 2024-09-21T11:53:24.053 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:24.086 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:24.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:24.087 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:24.116 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":644,"stamp":"2024-09-21T11:53:22.385729+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3860601371,"num_objects":59098,"num_object_clones":0,"num_object_copies":177294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59098,"num_whiteouts":0,"num_read":2041,"num_read_kb":1898,"num_write":59734,"num_write_kb":3771209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":597267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59745,"ondisk_log_size":59745,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15214128,"kb_used_data":11243996,"kb_used_omap":12,"kb_used_meta":3969971,"kb_avail":734747088,"statfs":{"total":767960285184,"available":752381018112,"internally_reserved":0,"allocated":11513851904,"data_stored":11510089569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4065250866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[12,4,9,8,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1002176512,"num_objects":15292,"num_object_clones":0,"num_object_copies":45876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15292,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":15292,"num_write_kb":978688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.352211"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":577,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554076+0000","last_change":"2024-09-21T11:52:54.792106+0000","last_active":"2024-09-21T11:53:19.554076+0000","last_peered":"2024-09-21T11:53:19.554076+0000","last_clean":"2024-09-21T11:53:19.554076+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:53:19.554076+0000","last_undegraded":"2024-09-21T11:53:19.554076+0000","last_fullsized":"2024-09-21T11:53:19.554076+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:54.792037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:54.792037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.576756+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":608,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.544585+0000","last_change":"2024-09-21T11:52:08.520781+0000","last_active":"2024-09-21T11:53:19.544585+0000","last_peered":"2024-09-21T11:53:19.544585+0000","last_clean":"2024-09-21T11:53:19.544585+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:53:19.544585+0000","last_undegraded":"2024-09-21T11:53:19.544585+0000","last_fullsized":"2024-09-21T11:53:19.544585+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:52:08.520688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:08.520688+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:17.574851+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":286,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:20.097633+0000","last_change":"2024-09-21T11:53:20.097633+0000","last_active":"2024-09-21T11:53:20.097633+0000","last_peered":"2024-09-21T11:53:20.097633+0000","last_clean":"2024-09-21T11:53:20.097633+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:53:20.097633+0000","last_undegraded":"2024-09-21T11:53:20.097633+0000","last_fullsized":"2024-09-21T11:53:20.097633+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:20.097557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.097557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:40.861158+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":389,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.534869+0000","last_change":"2024-09-21T11:51:43.528594+0000","last_active":"2024-09-21T11:53:19.534869+0000","last_peered":"2024-09-21T11:53:19.534869+0000","last_clean":"2024-09-21T11:53:19.534869+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:53:19.534869+0000","last_undegraded":"2024-09-21T11:53:19.534869+0000","last_fullsized":"2024-09-21T11:53:19.534869+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.914122+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":581,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532528+0000","last_change":"2024-09-21T11:52:11.462678+0000","last_active":"2024-09-21T11:53:19.532528+0000","last_peered":"2024-09-21T11:53:19.532528+0000","last_clean":"2024-09-21T11:53:19.532528+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:53:19.532528+0000","last_undegraded":"2024-09-21T11:53:19.532528+0000","last_fullsized":"2024-09-21T11:53:19.532528+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:11.462611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:11.462611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":570,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533616+0000","last_change":"2024-09-21T11:51:01.667426+0000","last_active":"2024-09-21T11:53:19.533616+0000","last_peered":"2024-09-21T11:53:19.533616+0000","last_clean":"2024-09-21T11:53:19.533616+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:53:19.533616+0000","last_undegraded":"2024-09-21T11:53:19.533616+0000","last_fullsized":"2024-09-21T11:53:19.533616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.667366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.667366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.200426+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":616,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532344+0000","last_change":"2024-09-21T11:51:43.409002+0000","last_active":"2024-09-21T11:53:19.532344+0000","last_peered":"2024-09-21T11:53:19.532344+0000","last_clean":"2024-09-21T11:53:19.532344+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:53:19.532344+0000","last_undegraded":"2024-09-21T11:53:19.532344+0000","last_fullsized":"2024-09-21T11:53:19.532344+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:51:43.408804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.408804+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":409,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.544740+0000","last_change":"2024-09-21T11:52:54.554185+0000","last_active":"2024-09-21T11:53:19.544740+0000","last_peered":"2024-09-21T11:53:19.544740+0000","last_clean":"2024-09-21T11:53:19.544740+0000","last_became_active":"2024-09-21T11:51:43.526945+0000","last_became_peered":"2024-09-21T11:51:43.526945+0000","last_unstale":"2024-09-21T11:53:19.544740+0000","last_undegraded":"2024-09-21T11:53:19.544740+0000","last_fullsized":"2024-09-21T11:53:19.544740+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:54.553857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:54.553857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:55.495025+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":576,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537431+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:53:19.537431+0000","last_peered":"2024-09-21T11:53:19.537431+0000","last_clean":"2024-09-21T11:53:19.537431+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:53:19.537431+0000","last_undegraded":"2024-09-21T11:53:19.537431+0000","last_fullsized":"2024-09-21T11:53:19.537431+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.870896+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":574,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532543+0000","last_change":"2024-09-21T11:52:55.214031+0000","last_active":"2024-09-21T11:53:19.532543+0000","last_peered":"2024-09-21T11:53:19.532543+0000","last_clean":"2024-09-21T11:53:19.532543+0000","last_became_active":"2024-09-21T11:51:43.545021+0000","last_became_peered":"2024-09-21T11:51:43.545021+0000","last_unstale":"2024-09-21T11:53:19.532543+0000","last_undegraded":"2024-09-21T11:53:19.532543+0000","last_fullsized":"2024-09-21T11:53:19.532543+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:52:55.213938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:55.213938+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:12.879409+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":568,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535203+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:53:19.535203+0000","last_peered":"2024-09-21T11:53:19.535203+0000","last_clean":"2024-09-21T11:53:19.535203+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:53:19.535203+0000","last_undegraded":"2024-09-21T11:53:19.535203+0000","last_fullsized":"2024-09-21T11:53:19.535203+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.824812+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":594,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551096+0000","last_change":"2024-09-21T11:52:50.731704+0000","last_active":"2024-09-21T11:53:19.551096+0000","last_peered":"2024-09-21T11:53:19.551096+0000","last_clean":"2024-09-21T11:53:19.551096+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:53:19.551096+0000","last_undegraded":"2024-09-21T11:53:19.551096+0000","last_fullsized":"2024-09-21T11:53:19.551096+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:50.731653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:50.731653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.382981+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":606,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536155+0000","last_change":"2024-09-21T11:52:11.666514+0000","last_active":"2024-09-21T11:53:19.536155+0000","last_peered":"2024-09-21T11:53:19.536155+0000","last_clean":"2024-09-21T11:53:19.536155+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:53:19.536155+0000","last_undegraded":"2024-09-21T11:53:19.536155+0000","last_fullsized":"2024-09-21T11:53:19.536155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:11.666440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:11.666440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:23.501612+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":580,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532734+0000","last_change":"2024-09-21T11:51:43.547924+0000","last_active":"2024-09-21T11:53:19.532734+0000","last_peered":"2024-09-21T11:53:19.532734+0000","last_clean":"2024-09-21T11:53:19.532734+0000","last_became_active":"2024-09-21T11:51:43.545191+0000","last_became_peered":"2024-09-21T11:51:43.545191+0000","last_unstale":"2024-09-21T11:53:19.532734+0000","last_undegraded":"2024-09-21T11:53:19.532734+0000","last_fullsized":"2024-09-21T11:53:19.532734+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.268969+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":582,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537718+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:53:19.537718+0000","last_peered":"2024-09-21T11:53:19.537718+0000","last_clean":"2024-09-21T11:53:19.537718+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:53:19.537718+0000","last_undegraded":"2024-09-21T11:53:19.537718+0000","last_fullsized":"2024-09-21T11:53:19.537718+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.104956+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":586,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536149+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:53:19.536149+0000","last_peered":"2024-09-21T11:53:19.536149+0000","last_clean":"2024-09-21T11:53:19.536149+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:53:19.536149+0000","last_undegraded":"2024-09-21T11:53:19.536149+0000","last_fullsized":"2024-09-21T11:53:19.536149+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.899330+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":79,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.539739+0000","last_change":"2024-09-21T11:51:43.524599+0000","last_active":"2024-09-21T11:53:19.539739+0000","last_peered":"2024-09-21T11:53:19.539739+0000","last_clean":"2024-09-21T11:53:19.539739+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:53:19.539739+0000","last_undegraded":"2024-09-21T11:53:19.539739+0000","last_fullsized":"2024-09-21T11:53:19.539739+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:32.302842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.302842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.577507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":568,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551860+0000","last_change":"2024-09-21T11:50:35.718093+0000","last_active":"2024-09-21T11:53:19.551860+0000","last_peered":"2024-09-21T11:53:19.551860+0000","last_clean":"2024-09-21T11:53:19.551860+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:53:19.551860+0000","last_undegraded":"2024-09-21T11:53:19.551860+0000","last_fullsized":"2024-09-21T11:53:19.551860+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:35.717991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.717991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:10.565445+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":701,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535722+0000","last_change":"2024-09-21T11:50:25.966805+0000","last_active":"2024-09-21T11:53:19.535722+0000","last_peered":"2024-09-21T11:53:19.535722+0000","last_clean":"2024-09-21T11:53:19.535722+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:53:19.535722+0000","last_undegraded":"2024-09-21T11:53:19.535722+0000","last_fullsized":"2024-09-21T11:53:19.535722+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:50:25.966740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.966740+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.907984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":278,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.552757+0000","last_change":"2024-09-21T11:50:31.012243+0000","last_active":"2024-09-21T11:53:19.552757+0000","last_peered":"2024-09-21T11:53:19.552757+0000","last_clean":"2024-09-21T11:53:19.552757+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:53:19.552757+0000","last_undegraded":"2024-09-21T11:53:19.552757+0000","last_fullsized":"2024-09-21T11:53:19.552757+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.012177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.012177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:23.104375+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":574,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533727+0000","last_change":"2024-09-21T11:52:44.264180+0000","last_active":"2024-09-21T11:53:19.533727+0000","last_peered":"2024-09-21T11:53:19.533727+0000","last_clean":"2024-09-21T11:53:19.533727+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:53:19.533727+0000","last_undegraded":"2024-09-21T11:53:19.533727+0000","last_fullsized":"2024-09-21T11:53:19.533727+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.264114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.264114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.220096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":393,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.546096+0000","last_change":"2024-09-21T11:51:43.543408+0000","last_active":"2024-09-21T11:53:19.546096+0000","last_peered":"2024-09-21T11:53:19.546096+0000","last_clean":"2024-09-21T11:53:19.546096+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:53:19.546096+0000","last_undegraded":"2024-09-21T11:53:19.546096+0000","last_fullsized":"2024-09-21T11:53:19.546096+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.122227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.122227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.038540+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":643,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533841+0000","last_change":"2024-09-21T11:51:12.652049+0000","last_active":"2024-09-21T11:53:19.533841+0000","last_peered":"2024-09-21T11:53:19.533841+0000","last_clean":"2024-09-21T11:53:19.533841+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:53:19.533841+0000","last_undegraded":"2024-09-21T11:53:19.533841+0000","last_fullsized":"2024-09-21T11:53:19.533841+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:51:12.651967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.651967+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":79,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532932+0000","last_change":"2024-09-21T11:51:43.549287+0000","last_active":"2024-09-21T11:53:19.532932+0000","last_peered":"2024-09-21T11:53:19.532932+0000","last_clean":"2024-09-21T11:53:19.532932+0000","last_became_active":"2024-09-21T11:51:43.548726+0000","last_became_peered":"2024-09-21T11:51:43.548726+0000","last_unstale":"2024-09-21T11:53:19.532932+0000","last_undegraded":"2024-09-21T11:53:19.532932+0000","last_fullsized":"2024-09-21T11:53:19.532932+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.908339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.908339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":90,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.553070+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:53:19.553070+0000","last_peered":"2024-09-21T11:53:19.553070+0000","last_clean":"2024-09-21T11:53:19.553070+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:53:19.553070+0000","last_undegraded":"2024-09-21T11:53:19.553070+0000","last_fullsized":"2024-09-21T11:53:19.553070+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.632356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":274,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.545613+0000","last_change":"2024-09-21T11:50:02.920313+0000","last_active":"2024-09-21T11:53:19.545613+0000","last_peered":"2024-09-21T11:53:19.545613+0000","last_clean":"2024-09-21T11:53:19.545613+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:53:19.545613+0000","last_undegraded":"2024-09-21T11:53:19.545613+0000","last_fullsized":"2024-09-21T11:53:19.545613+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:02.920229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.920229+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:27.127472+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":611,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554566+0000","last_change":"2024-09-21T11:49:58.702620+0000","last_active":"2024-09-21T11:53:19.554566+0000","last_peered":"2024-09-21T11:53:19.554566+0000","last_clean":"2024-09-21T11:53:19.554566+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:53:19.554566+0000","last_undegraded":"2024-09-21T11:53:19.554566+0000","last_fullsized":"2024-09-21T11:53:19.554566+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:49:58.702544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.702544+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:26.900715+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":765,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.539690+0000","last_change":"2024-09-21T11:51:44.478918+0000","last_active":"2024-09-21T11:53:19.539690+0000","last_peered":"2024-09-21T11:53:19.539690+0000","last_clean":"2024-09-21T11:53:19.539690+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:53:19.539690+0000","last_undegraded":"2024-09-21T11:53:19.539690+0000","last_fullsized":"2024-09-21T11:53:19.539690+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:44.478855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.478855+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.644866+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"327'3700","reported_seq":3739,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.143368+0000","last_change":"2024-09-21T11:52:45.608799+0000","last_active":"2024-09-21T11:53:21.143368+0000","last_peered":"2024-09-21T11:53:21.143368+0000","last_clean":"2024-09-21T11:53:21.143368+0000","last_became_active":"2024-09-21T11:52:45.608448+0000","last_became_peered":"2024-09-21T11:52:45.608448+0000","last_unstale":"2024-09-21T11:53:21.143368+0000","last_undegraded":"2024-09-21T11:53:21.143368+0000","last_fullsized":"2024-09-21T11:53:21.143368+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3700,"log_dups_size":0,"ondisk_log_size":3700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:51.400025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242483200,"num_objects":3700,"num_object_clones":0,"num_object_copies":11100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3700,"num_write_kb":236800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":320,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.546210+0000","last_change":"2024-09-21T11:52:39.116264+0000","last_active":"2024-09-21T11:53:19.546210+0000","last_peered":"2024-09-21T11:53:19.546210+0000","last_clean":"2024-09-21T11:53:19.546210+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:53:19.546210+0000","last_undegraded":"2024-09-21T11:53:19.546210+0000","last_fullsized":"2024-09-21T11:53:19.546210+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:52:39.116197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:39.116197+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:40.719106+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":600,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532552+0000","last_change":"2024-09-21T11:49:47.205216+0000","last_active":"2024-09-21T11:53:19.532552+0000","last_peered":"2024-09-21T11:53:19.532552+0000","last_clean":"2024-09-21T11:53:19.532552+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:53:19.532552+0000","last_undegraded":"2024-09-21T11:53:19.532552+0000","last_fullsized":"2024-09-21T11:53:19.532552+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.204889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.204889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:28.205006+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"294'9","reported_seq":126,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535750+0000","last_change":"2024-09-21T11:51:43.526079+0000","last_active":"2024-09-21T11:53:19.535750+0000","last_peered":"2024-09-21T11:53:19.535750+0000","last_clean":"2024-09-21T11:53:19.535750+0000","last_became_active":"2024-09-21T11:51:43.525566+0000","last_became_peered":"2024-09-21T11:51:43.525566+0000","last_unstale":"2024-09-21T11:53:19.535750+0000","last_undegraded":"2024-09-21T11:53:19.535750+0000","last_fullsized":"2024-09-21T11:53:19.535750+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:11.932479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:11.932479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.943835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":594,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537026+0000","last_change":"2024-09-21T11:52:38.781242+0000","last_active":"2024-09-21T11:53:19.537026+0000","last_peered":"2024-09-21T11:53:19.537026+0000","last_clean":"2024-09-21T11:53:19.537026+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:53:19.537026+0000","last_undegraded":"2024-09-21T11:53:19.537026+0000","last_fullsized":"2024-09-21T11:53:19.537026+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.781189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.781189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:50.278404+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"327'3691","reported_seq":3730,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.050435+0000","last_change":"2024-09-21T11:52:45.606555+0000","last_active":"2024-09-21T11:53:21.050435+0000","last_peered":"2024-09-21T11:53:21.050435+0000","last_clean":"2024-09-21T11:53:21.050435+0000","last_became_active":"2024-09-21T11:52:45.606226+0000","last_became_peered":"2024-09-21T11:52:45.606226+0000","last_unstale":"2024-09-21T11:53:21.050435+0000","last_undegraded":"2024-09-21T11:53:21.050435+0000","last_fullsized":"2024-09-21T11:53:21.050435+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3691,"log_dups_size":0,"ondisk_log_size":3691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.505126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241893376,"num_objects":3691,"num_object_clones":0,"num_object_copies":11073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3691,"num_write_kb":236224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":640,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532194+0000","last_change":"2024-09-21T11:52:47.279483+0000","last_active":"2024-09-21T11:53:19.532194+0000","last_peered":"2024-09-21T11:53:19.532194+0000","last_clean":"2024-09-21T11:53:19.532194+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:53:19.532194+0000","last_undegraded":"2024-09-21T11:53:19.532194+0000","last_fullsized":"2024-09-21T11:53:19.532194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:52:47.279423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:47.279423+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:10.328491+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":594,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.534155+0000","last_change":"2024-09-21T11:52:42.279545+0000","last_active":"2024-09-21T11:53:19.534155+0000","last_peered":"2024-09-21T11:53:19.534155+0000","last_clean":"2024-09-21T11:53:19.534155+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:53:19.534155+0000","last_undegraded":"2024-09-21T11:53:19.534155+0000","last_fullsized":"2024-09-21T11:53:19.534155+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:42.279496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:42.279496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.626370+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":79,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532406+0000","last_change":"2024-09-21T11:51:43.542323+0000","last_active":"2024-09-21T11:53:19.532406+0000","last_peered":"2024-09-21T11:53:19.532406+0000","last_clean":"2024-09-21T11:53:19.532406+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:53:19.532406+0000","last_undegraded":"2024-09-21T11:53:19.532406+0000","last_fullsized":"2024-09-21T11:53:19.532406+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.940335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.940335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":73,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535196+0000","last_change":"2024-09-21T11:52:01.954772+0000","last_active":"2024-09-21T11:53:19.535196+0000","last_peered":"2024-09-21T11:53:19.535196+0000","last_clean":"2024-09-21T11:53:19.535196+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:53:19.535196+0000","last_undegraded":"2024-09-21T11:53:19.535196+0000","last_fullsized":"2024-09-21T11:53:19.535196+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:08.977984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.977984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:21.812560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"327'3722","reported_seq":3761,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.045611+0000","last_change":"2024-09-21T11:52:45.635813+0000","last_active":"2024-09-21T11:53:21.045611+0000","last_peered":"2024-09-21T11:53:21.045611+0000","last_clean":"2024-09-21T11:53:21.045611+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:53:21.045611+0000","last_undegraded":"2024-09-21T11:53:21.045611+0000","last_fullsized":"2024-09-21T11:53:21.045611+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:08.048600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243924992,"num_objects":3722,"num_object_clones":0,"num_object_copies":11166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3722,"num_write_kb":238208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":591,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536248+0000","last_change":"2024-09-21T11:51:43.525478+0000","last_active":"2024-09-21T11:53:19.536248+0000","last_peered":"2024-09-21T11:53:19.536248+0000","last_clean":"2024-09-21T11:53:19.536248+0000","last_became_active":"2024-09-21T11:51:43.523852+0000","last_became_peered":"2024-09-21T11:51:43.523852+0000","last_unstale":"2024-09-21T11:53:19.536248+0000","last_undegraded":"2024-09-21T11:53:19.536248+0000","last_fullsized":"2024-09-21T11:53:19.536248+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:07.916000+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":616,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.555058+0000","last_change":"2024-09-21T11:50:28.657341+0000","last_active":"2024-09-21T11:53:19.555058+0000","last_peered":"2024-09-21T11:53:19.555058+0000","last_clean":"2024-09-21T11:53:19.555058+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:53:19.555058+0000","last_undegraded":"2024-09-21T11:53:19.555058+0000","last_fullsized":"2024-09-21T11:53:19.555058+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:28.657261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.657261+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.614169+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":770,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.531978+0000","last_change":"2024-09-21T11:51:20.675165+0000","last_active":"2024-09-21T11:53:19.531978+0000","last_peered":"2024-09-21T11:53:19.531978+0000","last_clean":"2024-09-21T11:53:19.531978+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:53:19.531978+0000","last_undegraded":"2024-09-21T11:53:19.531978+0000","last_fullsized":"2024-09-21T11:53:19.531978+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:51:20.675101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.675101+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":584,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537993+0000","last_change":"2024-09-21T11:52:14.010215+0000","last_active":"2024-09-21T11:53:19.537993+0000","last_peered":"2024-09-21T11:53:19.537993+0000","last_clean":"2024-09-21T11:53:19.537993+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:53:19.537993+0000","last_undegraded":"2024-09-21T11:53:19.537993+0000","last_fullsized":"2024-09-21T11:53:19.537993+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:14.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:14.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:42.958931+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"327'3545","reported_seq":3584,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.134364+0000","last_change":"2024-09-21T11:52:45.608507+0000","last_active":"2024-09-21T11:53:21.134364+0000","last_peered":"2024-09-21T11:53:21.134364+0000","last_clean":"2024-09-21T11:53:21.134364+0000","last_became_active":"2024-09-21T11:52:45.608219+0000","last_became_peered":"2024-09-21T11:52:45.608219+0000","last_unstale":"2024-09-21T11:53:21.134364+0000","last_undegraded":"2024-09-21T11:53:21.134364+0000","last_fullsized":"2024-09-21T11:53:21.134364+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3545,"log_dups_size":0,"ondisk_log_size":3545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.821182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232325120,"num_objects":3545,"num_object_clones":0,"num_object_copies":10635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3545,"num_write_kb":226880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":298,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.546507+0000","last_change":"2024-09-21T11:51:08.137774+0000","last_active":"2024-09-21T11:53:19.546507+0000","last_peered":"2024-09-21T11:53:19.546507+0000","last_clean":"2024-09-21T11:53:19.546507+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:53:19.546507+0000","last_undegraded":"2024-09-21T11:53:19.546507+0000","last_fullsized":"2024-09-21T11:53:19.546507+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:51:08.137700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.137700+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:30.141542+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":609,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537545+0000","last_change":"2024-09-21T11:52:39.713149+0000","last_active":"2024-09-21T11:53:19.537545+0000","last_peered":"2024-09-21T11:53:19.537545+0000","last_clean":"2024-09-21T11:53:19.537545+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:53:19.537545+0000","last_undegraded":"2024-09-21T11:53:19.537545+0000","last_fullsized":"2024-09-21T11:53:19.537545+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:39.713105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:39.713105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.384706+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":741,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551548+0000","last_change":"2024-09-21T11:52:37.704950+0000","last_active":"2024-09-21T11:53:19.551548+0000","last_peered":"2024-09-21T11:53:19.551548+0000","last_clean":"2024-09-21T11:53:19.551548+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:53:19.551548+0000","last_undegraded":"2024-09-21T11:53:19.551548+0000","last_fullsized":"2024-09-21T11:53:19.551548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:52:37.704899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:37.704899+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:53.365928+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":581,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551251+0000","last_change":"2024-09-21T11:52:43.682014+0000","last_active":"2024-09-21T11:53:19.551251+0000","last_peered":"2024-09-21T11:53:19.551251+0000","last_clean":"2024-09-21T11:53:19.551251+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:53:19.551251+0000","last_undegraded":"2024-09-21T11:53:19.551251+0000","last_fullsized":"2024-09-21T11:53:19.551251+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:43.681959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:43.681959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:47.395198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"327'3628","reported_seq":3667,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.040367+0000","last_change":"2024-09-21T11:52:45.607195+0000","last_active":"2024-09-21T11:53:21.040367+0000","last_peered":"2024-09-21T11:53:21.040367+0000","last_clean":"2024-09-21T11:53:21.040367+0000","last_became_active":"2024-09-21T11:52:45.606865+0000","last_became_peered":"2024-09-21T11:52:45.606865+0000","last_unstale":"2024-09-21T11:53:21.040367+0000","last_undegraded":"2024-09-21T11:53:21.040367+0000","last_fullsized":"2024-09-21T11:53:21.040367+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3628,"log_dups_size":0,"ondisk_log_size":3628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.533011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237764608,"num_objects":3628,"num_object_clones":0,"num_object_copies":10884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3628,"num_write_kb":232192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":632,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551698+0000","last_change":"2024-09-21T11:51:45.114831+0000","last_active":"2024-09-21T11:53:19.551698+0000","last_peered":"2024-09-21T11:53:19.551698+0000","last_clean":"2024-09-21T11:53:19.551698+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:53:19.551698+0000","last_undegraded":"2024-09-21T11:53:19.551698+0000","last_fullsized":"2024-09-21T11:53:19.551698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:51:45.114143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:45.114143+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":271,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.552305+0000","last_change":"2024-09-21T11:49:59.954903+0000","last_active":"2024-09-21T11:53:19.552305+0000","last_peered":"2024-09-21T11:53:19.552305+0000","last_clean":"2024-09-21T11:53:19.552305+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:53:19.552305+0000","last_undegraded":"2024-09-21T11:53:19.552305+0000","last_fullsized":"2024-09-21T11:53:19.552305+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:59.954835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:59.954835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:25.047222+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":569,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536814+0000","last_change":"2024-09-21T11:51:52.563725+0000","last_active":"2024-09-21T11:53:19.536814+0000","last_peered":"2024-09-21T11:53:19.536814+0000","last_clean":"2024-09-21T11:53:19.536814+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:53:19.536814+0000","last_undegraded":"2024-09-21T11:53:19.536814+0000","last_fullsized":"2024-09-21T11:53:19.536814+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:52.563656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:52.563656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:14.500261+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":279,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.545353+0000","last_change":"2024-09-21T11:52:09.412149+0000","last_active":"2024-09-21T11:53:19.545353+0000","last_peered":"2024-09-21T11:53:19.545353+0000","last_clean":"2024-09-21T11:53:19.545353+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:53:19.545353+0000","last_undegraded":"2024-09-21T11:53:19.545353+0000","last_fullsized":"2024-09-21T11:53:19.545353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:09.412091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:09.412091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:32.303837+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"327'3618","reported_seq":3656,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.051094+0000","last_change":"2024-09-21T11:52:45.607383+0000","last_active":"2024-09-21T11:53:21.051094+0000","last_peered":"2024-09-21T11:53:21.051094+0000","last_clean":"2024-09-21T11:53:21.051094+0000","last_became_active":"2024-09-21T11:52:45.607000+0000","last_became_peered":"2024-09-21T11:52:45.607000+0000","last_unstale":"2024-09-21T11:53:21.051094+0000","last_undegraded":"2024-09-21T11:53:21.051094+0000","last_fullsized":"2024-09-21T11:53:21.051094+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3618,"log_dups_size":0,"ondisk_log_size":3618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.571620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237109248,"num_objects":3618,"num_object_clones":0,"num_object_copies":10854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3618,"num_write_kb":231552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":387,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:20.530380+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:53:20.530380+0000","last_peered":"2024-09-21T11:53:20.530380+0000","last_clean":"2024-09-21T11:53:20.530380+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:53:20.530380+0000","last_undegraded":"2024-09-21T11:53:20.530380+0000","last_fullsized":"2024-09-21T11:53:20.530380+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:15.031330+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":96,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551761+0000","last_change":"2024-09-21T11:51:44.078378+0000","last_active":"2024-09-21T11:53:19.551761+0000","last_peered":"2024-09-21T11:53:19.551761+0000","last_clean":"2024-09-21T11:53:19.551761+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:53:19.551761+0000","last_undegraded":"2024-09-21T11:53:19.551761+0000","last_fullsized":"2024-09-21T11:53:19.551761+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:44.078299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.078299+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.512892+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":609,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535273+0000","last_change":"2024-09-21T11:51:12.562026+0000","last_active":"2024-09-21T11:53:19.535273+0000","last_peered":"2024-09-21T11:53:19.535273+0000","last_clean":"2024-09-21T11:53:19.535273+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:53:19.535273+0000","last_undegraded":"2024-09-21T11:53:19.535273+0000","last_fullsized":"2024-09-21T11:53:19.535273+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:51:12.561961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.561961+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.897864+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":583,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.555154+0000","last_change":"2024-09-21T11:50:37.858401+0000","last_active":"2024-09-21T11:53:19.555154+0000","last_peered":"2024-09-21T11:53:19.555154+0000","last_clean":"2024-09-21T11:53:19.555154+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:53:19.555154+0000","last_undegraded":"2024-09-21T11:53:19.555154+0000","last_fullsized":"2024-09-21T11:53:19.555154+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:04.618872+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":558,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533434+0000","last_change":"2024-09-21T11:51:49.357728+0000","last_active":"2024-09-21T11:53:19.533434+0000","last_peered":"2024-09-21T11:53:19.533434+0000","last_clean":"2024-09-21T11:53:19.533434+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:53:19.533434+0000","last_undegraded":"2024-09-21T11:53:19.533434+0000","last_fullsized":"2024-09-21T11:53:19.533434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:49.357659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:49.357659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"327'3784","reported_seq":3823,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.207623+0000","last_change":"2024-09-21T11:52:45.605195+0000","last_active":"2024-09-21T11:53:21.207623+0000","last_peered":"2024-09-21T11:53:21.207623+0000","last_clean":"2024-09-21T11:53:21.207623+0000","last_became_active":"2024-09-21T11:52:45.604600+0000","last_became_peered":"2024-09-21T11:52:45.604600+0000","last_unstale":"2024-09-21T11:53:21.207623+0000","last_undegraded":"2024-09-21T11:53:21.207623+0000","last_fullsized":"2024-09-21T11:53:21.207623+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3784,"log_dups_size":0,"ondisk_log_size":3784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:01.765652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247988224,"num_objects":3784,"num_object_clones":0,"num_object_copies":11352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3784,"num_write_kb":242176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":222,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535425+0000","last_change":"2024-09-21T11:51:41.006710+0000","last_active":"2024-09-21T11:53:19.535425+0000","last_peered":"2024-09-21T11:53:19.535425+0000","last_clean":"2024-09-21T11:53:19.535425+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:53:19.535425+0000","last_undegraded":"2024-09-21T11:53:19.535425+0000","last_fullsized":"2024-09-21T11:53:19.535425+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:51:41.006622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:41.006622+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:18.027646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":283,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.545762+0000","last_change":"2024-09-21T11:51:43.346789+0000","last_active":"2024-09-21T11:53:19.545762+0000","last_peered":"2024-09-21T11:53:19.545762+0000","last_clean":"2024-09-21T11:53:19.545762+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:53:19.545762+0000","last_undegraded":"2024-09-21T11:53:19.545762+0000","last_fullsized":"2024-09-21T11:53:19.545762+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:43.346718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.346718+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.960974+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":619,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:20.956563+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:53:20.956563+0000","last_peered":"2024-09-21T11:53:20.956563+0000","last_clean":"2024-09-21T11:53:20.956563+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:53:20.956563+0000","last_undegraded":"2024-09-21T11:53:20.956563+0000","last_fullsized":"2024-09-21T11:53:20.956563+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:42.859403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":578,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533728+0000","last_change":"2024-09-21T11:49:47.008541+0000","last_active":"2024-09-21T11:53:19.533728+0000","last_peered":"2024-09-21T11:53:19.533728+0000","last_clean":"2024-09-21T11:53:19.533728+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:53:19.533728+0000","last_undegraded":"2024-09-21T11:53:19.533728+0000","last_fullsized":"2024-09-21T11:53:19.533728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.008369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.008369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.883485+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":573,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.541773+0000","last_change":"2024-09-21T11:49:54.388131+0000","last_active":"2024-09-21T11:53:19.541773+0000","last_peered":"2024-09-21T11:53:19.541773+0000","last_clean":"2024-09-21T11:53:19.541773+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:53:19.541773+0000","last_undegraded":"2024-09-21T11:53:19.541773+0000","last_fullsized":"2024-09-21T11:53:19.541773+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.388068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.388068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:25.965811+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"327'3729","reported_seq":3768,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:22.074023+0000","last_change":"2024-09-21T11:52:45.604487+0000","last_active":"2024-09-21T11:53:22.074023+0000","last_peered":"2024-09-21T11:53:22.074023+0000","last_clean":"2024-09-21T11:53:22.074023+0000","last_became_active":"2024-09-21T11:52:45.604244+0000","last_became_peered":"2024-09-21T11:52:45.604244+0000","last_unstale":"2024-09-21T11:53:22.074023+0000","last_undegraded":"2024-09-21T11:53:22.074023+0000","last_fullsized":"2024-09-21T11:53:22.074023+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3729,"log_dups_size":0,"ondisk_log_size":3729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.006653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244383744,"num_objects":3729,"num_object_clones":0,"num_object_copies":11187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3729,"num_write_kb":238656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":659,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:20.438262+0000","last_change":"2024-09-21T11:53:20.438262+0000","last_active":"2024-09-21T11:53:20.438262+0000","last_peered":"2024-09-21T11:53:20.438262+0000","last_clean":"2024-09-21T11:53:20.438262+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:53:20.438262+0000","last_undegraded":"2024-09-21T11:53:20.438262+0000","last_fullsized":"2024-09-21T11:53:20.438262+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:53:20.438174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.438174+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:24.864136+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":593,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551895+0000","last_change":"2024-09-21T11:51:20.858177+0000","last_active":"2024-09-21T11:53:19.551895+0000","last_peered":"2024-09-21T11:53:19.551895+0000","last_clean":"2024-09-21T11:53:19.551895+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:53:19.551895+0000","last_undegraded":"2024-09-21T11:53:19.551895+0000","last_fullsized":"2024-09-21T11:53:19.551895+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.858109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.858109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.570964+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1420,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.880333+0000","last_change":"2024-09-21T11:49:51.434402+0000","last_active":"2024-09-21T11:53:19.880333+0000","last_peered":"2024-09-21T11:53:19.880333+0000","last_clean":"2024-09-21T11:53:19.880333+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:53:19.880333+0000","last_undegraded":"2024-09-21T11:53:19.880333+0000","last_fullsized":"2024-09-21T11:53:19.880333+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:51.434333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:49:51.434333+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:29.068051+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":792,"num_read_kb":787,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":591,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538855+0000","last_change":"2024-09-21T11:51:17.645920+0000","last_active":"2024-09-21T11:53:19.538855+0000","last_peered":"2024-09-21T11:53:19.538855+0000","last_clean":"2024-09-21T11:53:19.538855+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:53:19.538855+0000","last_undegraded":"2024-09-21T11:53:19.538855+0000","last_fullsized":"2024-09-21T11:53:19.538855+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.645823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.645823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:17.901222+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":287,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.552102+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:53:19.552102+0000","last_peered":"2024-09-21T11:53:19.552102+0000","last_clean":"2024-09-21T11:53:19.552102+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:53:19.552102+0000","last_undegraded":"2024-09-21T11:53:19.552102+0000","last_fullsized":"2024-09-21T11:53:19.552102+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.718884+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"327'3616","reported_seq":3655,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.044537+0000","last_change":"2024-09-21T11:52:45.607096+0000","last_active":"2024-09-21T11:53:21.044537+0000","last_peered":"2024-09-21T11:53:21.044537+0000","last_clean":"2024-09-21T11:53:21.044537+0000","last_became_active":"2024-09-21T11:52:45.606777+0000","last_became_peered":"2024-09-21T11:52:45.606777+0000","last_unstale":"2024-09-21T11:53:21.044537+0000","last_undegraded":"2024-09-21T11:53:21.044537+0000","last_fullsized":"2024-09-21T11:53:21.044537+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3616,"log_dups_size":0,"ondisk_log_size":3616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.742069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236978176,"num_objects":3616,"num_object_clones":0,"num_object_copies":10848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":231424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":782,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.544257+0000","last_change":"2024-09-21T11:50:29.389640+0000","last_active":"2024-09-21T11:53:19.544257+0000","last_peered":"2024-09-21T11:53:19.544257+0000","last_clean":"2024-09-21T11:53:19.544257+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:53:19.544257+0000","last_undegraded":"2024-09-21T11:53:19.544257+0000","last_fullsized":"2024-09-21T11:53:19.544257+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:29.389555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:29.389555+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:33.416003+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":586,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532735+0000","last_change":"2024-09-21T11:50:59.680952+0000","last_active":"2024-09-21T11:53:19.532735+0000","last_peered":"2024-09-21T11:53:19.532735+0000","last_clean":"2024-09-21T11:53:19.532735+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:53:19.532735+0000","last_undegraded":"2024-09-21T11:53:19.532735+0000","last_fullsized":"2024-09-21T11:53:19.532735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:59.680755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:59.680755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.232600+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":648,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538088+0000","last_change":"2024-09-21T11:52:53.118045+0000","last_active":"2024-09-21T11:53:19.538088+0000","last_peered":"2024-09-21T11:53:19.538088+0000","last_clean":"2024-09-21T11:53:19.538088+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:53:19.538088+0000","last_undegraded":"2024-09-21T11:53:19.538088+0000","last_fullsized":"2024-09-21T11:53:19.538088+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:52:53.117963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:52:53.117963+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:07.290308+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":585,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.541708+0000","last_change":"2024-09-21T11:51:18.318095+0000","last_active":"2024-09-21T11:53:19.541708+0000","last_peered":"2024-09-21T11:53:19.541708+0000","last_clean":"2024-09-21T11:53:19.541708+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:53:19.541708+0000","last_undegraded":"2024-09-21T11:53:19.541708+0000","last_fullsized":"2024-09-21T11:53:19.541708+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.317950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.317950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.110119+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":573,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536231+0000","last_change":"2024-09-21T11:51:46.607389+0000","last_active":"2024-09-21T11:53:19.536231+0000","last_peered":"2024-09-21T11:53:19.536231+0000","last_clean":"2024-09-21T11:53:19.536231+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:53:19.536231+0000","last_undegraded":"2024-09-21T11:53:19.536231+0000","last_fullsized":"2024-09-21T11:53:19.536231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:46.607326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:46.607326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"327'3706","reported_seq":3745,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.033998+0000","last_change":"2024-09-21T11:52:45.605911+0000","last_active":"2024-09-21T11:53:21.033998+0000","last_peered":"2024-09-21T11:53:21.033998+0000","last_clean":"2024-09-21T11:53:21.033998+0000","last_became_active":"2024-09-21T11:52:45.605575+0000","last_became_peered":"2024-09-21T11:52:45.605575+0000","last_unstale":"2024-09-21T11:53:21.033998+0000","last_undegraded":"2024-09-21T11:53:21.033998+0000","last_fullsized":"2024-09-21T11:53:21.033998+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3706,"log_dups_size":0,"ondisk_log_size":3706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:48.712206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242876416,"num_objects":3706,"num_object_clones":0,"num_object_copies":11118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3706,"num_write_kb":237184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":594,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.540270+0000","last_change":"2024-09-21T11:51:43.526788+0000","last_active":"2024-09-21T11:53:19.540270+0000","last_peered":"2024-09-21T11:53:19.540270+0000","last_clean":"2024-09-21T11:53:19.540270+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:53:19.540270+0000","last_undegraded":"2024-09-21T11:53:19.540270+0000","last_fullsized":"2024-09-21T11:53:19.540270+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:37.352834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.352834+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.596503+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":87,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.531828+0000","last_change":"2024-09-21T11:52:13.709151+0000","last_active":"2024-09-21T11:53:19.531828+0000","last_peered":"2024-09-21T11:53:19.531828+0000","last_clean":"2024-09-21T11:53:19.531828+0000","last_became_active":"2024-09-21T11:51:43.532027+0000","last_became_peered":"2024-09-21T11:51:43.532027+0000","last_unstale":"2024-09-21T11:53:19.531828+0000","last_undegraded":"2024-09-21T11:53:19.531828+0000","last_fullsized":"2024-09-21T11:53:19.531828+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:13.709074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:13.709074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:25.271458+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":568,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536087+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:53:19.536087+0000","last_peered":"2024-09-21T11:53:19.536087+0000","last_clean":"2024-09-21T11:53:19.536087+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:53:19.536087+0000","last_undegraded":"2024-09-21T11:53:19.536087+0000","last_fullsized":"2024-09-21T11:53:19.536087+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:00.863312+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":572,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532800+0000","last_change":"2024-09-21T11:51:18.664721+0000","last_active":"2024-09-21T11:53:19.532800+0000","last_peered":"2024-09-21T11:53:19.532800+0000","last_clean":"2024-09-21T11:53:19.532800+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:53:19.532800+0000","last_undegraded":"2024-09-21T11:53:19.532800+0000","last_fullsized":"2024-09-21T11:53:19.532800+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.664635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.664635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"327'3644","reported_seq":3683,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.503236+0000","last_change":"2024-09-21T11:52:45.638159+0000","last_active":"2024-09-21T11:53:21.503236+0000","last_peered":"2024-09-21T11:53:21.503236+0000","last_clean":"2024-09-21T11:53:21.503236+0000","last_became_active":"2024-09-21T11:52:45.637825+0000","last_became_peered":"2024-09-21T11:52:45.637825+0000","last_unstale":"2024-09-21T11:53:21.503236+0000","last_undegraded":"2024-09-21T11:53:21.503236+0000","last_fullsized":"2024-09-21T11:53:21.503236+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3644,"log_dups_size":0,"ondisk_log_size":3644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.377774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238813184,"num_objects":3644,"num_object_clones":0,"num_object_copies":10932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3644,"num_write_kb":233216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":665,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535089+0000","last_change":"2024-09-21T11:51:43.533280+0000","last_active":"2024-09-21T11:53:19.535089+0000","last_peered":"2024-09-21T11:53:19.535089+0000","last_clean":"2024-09-21T11:53:19.535089+0000","last_became_active":"2024-09-21T11:51:43.525941+0000","last_became_peered":"2024-09-21T11:51:43.525941+0000","last_unstale":"2024-09-21T11:53:19.535089+0000","last_undegraded":"2024-09-21T11:53:19.535089+0000","last_fullsized":"2024-09-21T11:53:19.535089+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:50:36.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.385902+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:07.872467+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":607,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538748+0000","last_change":"2024-09-21T11:51:14.312046+0000","last_active":"2024-09-21T11:53:19.538748+0000","last_peered":"2024-09-21T11:53:19.538748+0000","last_clean":"2024-09-21T11:53:19.538748+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:53:19.538748+0000","last_undegraded":"2024-09-21T11:53:19.538748+0000","last_fullsized":"2024-09-21T11:53:19.538748+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:14.311953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:14.311953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:29.454276+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":278,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551964+0000","last_change":"2024-09-21T11:52:44.976410+0000","last_active":"2024-09-21T11:53:19.551964+0000","last_peered":"2024-09-21T11:53:19.551964+0000","last_clean":"2024-09-21T11:53:19.551964+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:53:19.551964+0000","last_undegraded":"2024-09-21T11:53:19.551964+0000","last_fullsized":"2024-09-21T11:53:19.551964+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.976363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.976363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.128300+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":581,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535612+0000","last_change":"2024-09-21T11:51:47.944684+0000","last_active":"2024-09-21T11:53:19.535612+0000","last_peered":"2024-09-21T11:53:19.535612+0000","last_clean":"2024-09-21T11:53:19.535612+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:53:19.535612+0000","last_undegraded":"2024-09-21T11:53:19.535612+0000","last_fullsized":"2024-09-21T11:53:19.535612+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:47.944609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.944609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.995897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"327'3731","reported_seq":3770,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.380931+0000","last_change":"2024-09-21T11:52:45.601642+0000","last_active":"2024-09-21T11:53:21.380931+0000","last_peered":"2024-09-21T11:53:21.380931+0000","last_clean":"2024-09-21T11:53:21.380931+0000","last_became_active":"2024-09-21T11:52:45.601350+0000","last_became_peered":"2024-09-21T11:52:45.601350+0000","last_unstale":"2024-09-21T11:53:21.380931+0000","last_undegraded":"2024-09-21T11:53:21.380931+0000","last_fullsized":"2024-09-21T11:53:21.380931+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3731,"log_dups_size":0,"ondisk_log_size":3731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:47.738705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244514816,"num_objects":3731,"num_object_clones":0,"num_object_copies":11193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3731,"num_write_kb":238784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":579,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533148+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:53:19.533148+0000","last_peered":"2024-09-21T11:53:19.533148+0000","last_clean":"2024-09-21T11:53:19.533148+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:53:19.533148+0000","last_undegraded":"2024-09-21T11:53:19.533148+0000","last_fullsized":"2024-09-21T11:53:19.533148+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.147557+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":577,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:22.315543+0000","last_change":"2024-09-21T11:53:22.315543+0000","last_active":"2024-09-21T11:53:22.315543+0000","last_peered":"2024-09-21T11:53:22.315543+0000","last_clean":"2024-09-21T11:53:22.315543+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:53:22.315543+0000","last_undegraded":"2024-09-21T11:53:22.315543+0000","last_fullsized":"2024-09-21T11:53:22.315543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:22.315149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:22.315149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:31.277102+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":754,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535045+0000","last_change":"2024-09-21T11:50:02.975387+0000","last_active":"2024-09-21T11:53:19.535045+0000","last_peered":"2024-09-21T11:53:19.535045+0000","last_clean":"2024-09-21T11:53:19.535045+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:53:19.535045+0000","last_undegraded":"2024-09-21T11:53:19.535045+0000","last_fullsized":"2024-09-21T11:53:19.535045+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:50:02.975313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.975313+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:31.809007+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":398,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.553232+0000","last_change":"2024-09-21T11:52:52.981662+0000","last_active":"2024-09-21T11:53:19.553232+0000","last_peered":"2024-09-21T11:53:19.553232+0000","last_clean":"2024-09-21T11:53:19.553232+0000","last_became_active":"2024-09-21T11:51:43.543072+0000","last_became_peered":"2024-09-21T11:51:43.543072+0000","last_unstale":"2024-09-21T11:53:19.553232+0000","last_undegraded":"2024-09-21T11:53:19.553232+0000","last_fullsized":"2024-09-21T11:53:19.553232+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:52.981586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:52.981586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.522344+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"327'3695","reported_seq":3734,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.479972+0000","last_change":"2024-09-21T11:52:45.635909+0000","last_active":"2024-09-21T11:53:21.479972+0000","last_peered":"2024-09-21T11:53:21.479972+0000","last_clean":"2024-09-21T11:53:21.479972+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:53:21.479972+0000","last_undegraded":"2024-09-21T11:53:21.479972+0000","last_fullsized":"2024-09-21T11:53:21.479972+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3695,"log_dups_size":0,"ondisk_log_size":3695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.650065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242155520,"num_objects":3695,"num_object_clones":0,"num_object_copies":11085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3695,"num_write_kb":236480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":623,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551603+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:53:19.551603+0000","last_peered":"2024-09-21T11:53:19.551603+0000","last_clean":"2024-09-21T11:53:19.551603+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:53:19.551603+0000","last_undegraded":"2024-09-21T11:53:19.551603+0000","last_fullsized":"2024-09-21T11:53:19.551603+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:01.621332+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":600,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536295+0000","last_change":"2024-09-21T11:52:51.850181+0000","last_active":"2024-09-21T11:53:19.536295+0000","last_peered":"2024-09-21T11:53:19.536295+0000","last_clean":"2024-09-21T11:53:19.536295+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:53:19.536295+0000","last_undegraded":"2024-09-21T11:53:19.536295+0000","last_fullsized":"2024-09-21T11:53:19.536295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:51.850120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:51.850120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:52.704982+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":570,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535356+0000","last_change":"2024-09-21T11:48:49.993307+0000","last_active":"2024-09-21T11:53:19.535356+0000","last_peered":"2024-09-21T11:53:19.535356+0000","last_clean":"2024-09-21T11:53:19.535356+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:53:19.535356+0000","last_undegraded":"2024-09-21T11:53:19.535356+0000","last_fullsized":"2024-09-21T11:53:19.535356+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.993230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.993230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:23.798946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":557,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.550575+0000","last_change":"2024-09-21T11:51:50.032271+0000","last_active":"2024-09-21T11:53:19.550575+0000","last_peered":"2024-09-21T11:53:19.550575+0000","last_clean":"2024-09-21T11:53:19.550575+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:53:19.550575+0000","last_undegraded":"2024-09-21T11:53:19.550575+0000","last_fullsized":"2024-09-21T11:53:19.550575+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:50.032193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:50.032193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"327'3799","reported_seq":3838,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:22.110358+0000","last_change":"2024-09-21T11:52:45.636586+0000","last_active":"2024-09-21T11:53:22.110358+0000","last_peered":"2024-09-21T11:53:22.110358+0000","last_clean":"2024-09-21T11:53:22.110358+0000","last_became_active":"2024-09-21T11:52:45.636291+0000","last_became_peered":"2024-09-21T11:52:45.636291+0000","last_unstale":"2024-09-21T11:53:22.110358+0000","last_undegraded":"2024-09-21T11:53:22.110358+0000","last_fullsized":"2024-09-21T11:53:22.110358+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3799,"log_dups_size":0,"ondisk_log_size":3799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.940714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248971264,"num_objects":3799,"num_object_clones":0,"num_object_copies":11397,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3799,"num_write_kb":243136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":698,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538541+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:53:19.538541+0000","last_peered":"2024-09-21T11:53:19.538541+0000","last_clean":"2024-09-21T11:53:19.538541+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:53:19.538541+0000","last_undegraded":"2024-09-21T11:53:19.538541+0000","last_fullsized":"2024-09-21T11:53:19.538541+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.597026+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":584,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.543089+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:53:19.543089+0000","last_peered":"2024-09-21T11:53:19.543089+0000","last_clean":"2024-09-21T11:53:19.543089+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:53:19.543089+0000","last_undegraded":"2024-09-21T11:53:19.543089+0000","last_fullsized":"2024-09-21T11:53:19.543089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:30.996685+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":593,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532151+0000","last_change":"2024-09-21T11:51:48.941202+0000","last_active":"2024-09-21T11:53:19.532151+0000","last_peered":"2024-09-21T11:53:19.532151+0000","last_clean":"2024-09-21T11:53:19.532151+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:53:19.532151+0000","last_undegraded":"2024-09-21T11:53:19.532151+0000","last_fullsized":"2024-09-21T11:53:19.532151+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:48.941128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:48.941128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.021684+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":581,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538005+0000","last_change":"2024-09-21T11:52:12.100354+0000","last_active":"2024-09-21T11:53:19.538005+0000","last_peered":"2024-09-21T11:53:19.538005+0000","last_clean":"2024-09-21T11:53:19.538005+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:53:19.538005+0000","last_undegraded":"2024-09-21T11:53:19.538005+0000","last_fullsized":"2024-09-21T11:53:19.538005+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:36.518478+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"327'3688","reported_seq":3727,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.053616+0000","last_change":"2024-09-21T11:52:45.636994+0000","last_active":"2024-09-21T11:53:21.053616+0000","last_peered":"2024-09-21T11:53:21.053616+0000","last_clean":"2024-09-21T11:53:21.053616+0000","last_became_active":"2024-09-21T11:52:45.636672+0000","last_became_peered":"2024-09-21T11:52:45.636672+0000","last_unstale":"2024-09-21T11:53:21.053616+0000","last_undegraded":"2024-09-21T11:53:21.053616+0000","last_fullsized":"2024-09-21T11:53:21.053616+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3688,"log_dups_size":0,"ondisk_log_size":3688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.462991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241696768,"num_objects":3688,"num_object_clones":0,"num_object_copies":11064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3688,"num_write_kb":236032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":619,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.545522+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:53:19.545522+0000","last_peered":"2024-09-21T11:53:19.545522+0000","last_clean":"2024-09-21T11:53:19.545522+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:53:19.545522+0000","last_undegraded":"2024-09-21T11:53:19.545522+0000","last_fullsized":"2024-09-21T11:53:19.545522+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.016882+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":614,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.542539+0000","last_change":"2024-09-21T11:49:08.611218+0000","last_active":"2024-09-21T11:53:19.542539+0000","last_peered":"2024-09-21T11:53:19.542539+0000","last_clean":"2024-09-21T11:53:19.542539+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:53:19.542539+0000","last_undegraded":"2024-09-21T11:53:19.542539+0000","last_fullsized":"2024-09-21T11:53:19.542539+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.611159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.611159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:56.046273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":600,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.531714+0000","last_change":"2024-09-21T11:49:44.985594+0000","last_active":"2024-09-21T11:53:19.531714+0000","last_peered":"2024-09-21T11:53:19.531714+0000","last_clean":"2024-09-21T11:53:19.531714+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:53:19.531714+0000","last_undegraded":"2024-09-21T11:53:19.531714+0000","last_fullsized":"2024-09-21T11:53:19.531714+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:44.985234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:44.985234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.909250+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":587,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532548+0000","last_change":"2024-09-21T11:51:09.660956+0000","last_active":"2024-09-21T11:53:19.532548+0000","last_peered":"2024-09-21T11:53:19.532548+0000","last_clean":"2024-09-21T11:53:19.532548+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:53:19.532548+0000","last_undegraded":"2024-09-21T11:53:19.532548+0000","last_fullsized":"2024-09-21T11:53:19.532548+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:09.660880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.660880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"327'3603","reported_seq":3642,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.283026+0000","last_change":"2024-09-21T11:52:45.603459+0000","last_active":"2024-09-21T11:53:21.283026+0000","last_peered":"2024-09-21T11:53:21.283026+0000","last_clean":"2024-09-21T11:53:21.283026+0000","last_became_active":"2024-09-21T11:52:45.603101+0000","last_became_peered":"2024-09-21T11:52:45.603101+0000","last_unstale":"2024-09-21T11:53:21.283026+0000","last_undegraded":"2024-09-21T11:53:21.283026+0000","last_fullsized":"2024-09-21T11:53:21.283026+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3603,"log_dups_size":0,"ondisk_log_size":3603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:45.425281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236126208,"num_objects":3603,"num_object_clones":0,"num_object_copies":10809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3603,"num_write_kb":230592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":616,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.534166+0000","last_change":"2024-09-21T11:51:19.638791+0000","last_active":"2024-09-21T11:53:19.534166+0000","last_peered":"2024-09-21T11:53:19.534166+0000","last_clean":"2024-09-21T11:53:19.534166+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:53:19.534166+0000","last_undegraded":"2024-09-21T11:53:19.534166+0000","last_fullsized":"2024-09-21T11:53:19.534166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:51:19.638734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:19.638734+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":612,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.542827+0000","last_change":"2024-09-21T11:52:43.086438+0000","last_active":"2024-09-21T11:53:19.542827+0000","last_peered":"2024-09-21T11:53:19.542827+0000","last_clean":"2024-09-21T11:53:19.542827+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:53:19.542827+0000","last_undegraded":"2024-09-21T11:53:19.542827+0000","last_fullsized":"2024-09-21T11:53:19.542827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:52:43.086376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:43.086376+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.446134+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":746,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537867+0000","last_change":"2024-09-21T11:52:54.080234+0000","last_active":"2024-09-21T11:53:19.537867+0000","last_peered":"2024-09-21T11:53:19.537867+0000","last_clean":"2024-09-21T11:53:19.537867+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:53:19.537867+0000","last_undegraded":"2024-09-21T11:53:19.537867+0000","last_fullsized":"2024-09-21T11:53:19.537867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:52:54.080076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:54.080076+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:15.355476+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":565,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.534960+0000","last_change":"2024-09-21T11:49:56.669610+0000","last_active":"2024-09-21T11:53:19.534960+0000","last_peered":"2024-09-21T11:53:19.534960+0000","last_clean":"2024-09-21T11:53:19.534960+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:53:19.534960+0000","last_undegraded":"2024-09-21T11:53:19.534960+0000","last_fullsized":"2024-09-21T11:53:19.534960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:56.669547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:56.669547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:27.819090+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":600,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.544870+0000","last_change":"2024-09-21T11:52:42.454187+0000","last_active":"2024-09-21T11:53:19.544870+0000","last_peered":"2024-09-21T11:53:19.544870+0000","last_clean":"2024-09-21T11:53:19.544870+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:53:19.544870+0000","last_undegraded":"2024-09-21T11:53:19.544870+0000","last_fullsized":"2024-09-21T11:53:19.544870+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:42.454138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:42.454138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:56.161790+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":613,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.534032+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:53:19.534032+0000","last_peered":"2024-09-21T11:53:19.534032+0000","last_clean":"2024-09-21T11:53:19.534032+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:53:19.534032+0000","last_undegraded":"2024-09-21T11:53:19.534032+0000","last_fullsized":"2024-09-21T11:53:19.534032+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:01.305455+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":598,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535798+0000","last_change":"2024-09-21T11:50:58.694445+0000","last_active":"2024-09-21T11:53:19.535798+0000","last_peered":"2024-09-21T11:53:19.535798+0000","last_clean":"2024-09-21T11:53:19.535798+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:53:19.535798+0000","last_undegraded":"2024-09-21T11:53:19.535798+0000","last_fullsized":"2024-09-21T11:53:19.535798+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:58.694382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:58.694382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:20.794973+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":580,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.541761+0000","last_change":"2024-09-21T11:49:43.464860+0000","last_active":"2024-09-21T11:53:19.541761+0000","last_peered":"2024-09-21T11:53:19.541761+0000","last_clean":"2024-09-21T11:53:19.541761+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:53:19.541761+0000","last_undegraded":"2024-09-21T11:53:19.541761+0000","last_fullsized":"2024-09-21T11:53:19.541761+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:43.464792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:43.464792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:24.925506+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":614,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554970+0000","last_change":"2024-09-21T11:51:09.955823+0000","last_active":"2024-09-21T11:53:19.554970+0000","last_peered":"2024-09-21T11:53:19.554970+0000","last_clean":"2024-09-21T11:53:19.554970+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:53:19.554970+0000","last_undegraded":"2024-09-21T11:53:19.554970+0000","last_fullsized":"2024-09-21T11:53:19.554970+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:09.955744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.955744+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:22.796315+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":584,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538103+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:53:19.538103+0000","last_peered":"2024-09-21T11:53:19.538103+0000","last_clean":"2024-09-21T11:53:19.538103+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:53:19.538103+0000","last_undegraded":"2024-09-21T11:53:19.538103+0000","last_fullsized":"2024-09-21T11:53:19.538103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.143512+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":599,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533879+0000","last_change":"2024-09-21T11:52:41.245695+0000","last_active":"2024-09-21T11:53:19.533879+0000","last_peered":"2024-09-21T11:53:19.533879+0000","last_clean":"2024-09-21T11:53:19.533879+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:53:19.533879+0000","last_undegraded":"2024-09-21T11:53:19.533879+0000","last_fullsized":"2024-09-21T11:53:19.533879+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:41.245632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:41.245632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.732253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":581,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535536+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:53:19.535536+0000","last_peered":"2024-09-21T11:53:19.535536+0000","last_clean":"2024-09-21T11:53:19.535536+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:53:19.535536+0000","last_undegraded":"2024-09-21T11:53:19.535536+0000","last_fullsized":"2024-09-21T11:53:19.535536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.846816+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":597,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537351+0000","last_change":"2024-09-21T11:51:49.643640+0000","last_active":"2024-09-21T11:53:19.537351+0000","last_peered":"2024-09-21T11:53:19.537351+0000","last_clean":"2024-09-21T11:53:19.537351+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:53:19.537351+0000","last_undegraded":"2024-09-21T11:53:19.537351+0000","last_fullsized":"2024-09-21T11:53:19.537351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:49.643424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:49.643424+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":600,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.560342+0000","last_change":"2024-09-21T11:52:45.688676+0000","last_active":"2024-09-21T11:53:19.560342+0000","last_peered":"2024-09-21T11:53:19.560342+0000","last_clean":"2024-09-21T11:53:19.560342+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:53:19.560342+0000","last_undegraded":"2024-09-21T11:53:19.560342+0000","last_fullsized":"2024-09-21T11:53:19.560342+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:45.688617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:45.688617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.734406+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":730,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533051+0000","last_change":"2024-09-21T11:51:08.629172+0000","last_active":"2024-09-21T11:53:19.533051+0000","last_peered":"2024-09-21T11:53:19.533051+0000","last_clean":"2024-09-21T11:53:19.533051+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:53:19.533051+0000","last_undegraded":"2024-09-21T11:53:19.533051+0000","last_fullsized":"2024-09-21T11:53:19.533051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:08.628986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.628986+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":79,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535522+0000","last_change":"2024-09-21T11:51:43.531318+0000","last_active":"2024-09-21T11:53:19.535522+0000","last_peered":"2024-09-21T11:53:19.535522+0000","last_clean":"2024-09-21T11:53:19.535522+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:53:19.535522+0000","last_undegraded":"2024-09-21T11:53:19.535522+0000","last_fullsized":"2024-09-21T11:53:19.535522+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.826834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.826834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":633,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533239+0000","last_change":"2024-09-21T11:50:37.433622+0000","last_active":"2024-09-21T11:53:19.533239+0000","last_peered":"2024-09-21T11:53:19.533239+0000","last_clean":"2024-09-21T11:53:19.533239+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:53:19.533239+0000","last_undegraded":"2024-09-21T11:53:19.533239+0000","last_fullsized":"2024-09-21T11:53:19.533239+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:50:37.432212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.432212+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:21.303187+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":576,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538286+0000","last_change":"2024-09-21T11:49:58.400078+0000","last_active":"2024-09-21T11:53:19.538286+0000","last_peered":"2024-09-21T11:53:19.538286+0000","last_clean":"2024-09-21T11:53:19.538286+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:53:19.538286+0000","last_undegraded":"2024-09-21T11:53:19.538286+0000","last_fullsized":"2024-09-21T11:53:19.538286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.400013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.400013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:28.032196+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":768,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537211+0000","last_change":"2024-09-21T11:50:30.703282+0000","last_active":"2024-09-21T11:53:19.537211+0000","last_peered":"2024-09-21T11:53:19.537211+0000","last_clean":"2024-09-21T11:53:19.537211+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:53:19.537211+0000","last_undegraded":"2024-09-21T11:53:19.537211+0000","last_fullsized":"2024-09-21T11:53:19.537211+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:50:30.703227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:30.703227+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":580,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554374+0000","last_change":"2024-09-21T11:50:02.782565+0000","last_active":"2024-09-21T11:53:19.554374+0000","last_peered":"2024-09-21T11:53:19.554374+0000","last_clean":"2024-09-21T11:53:19.554374+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:53:19.554374+0000","last_undegraded":"2024-09-21T11:53:19.554374+0000","last_fullsized":"2024-09-21T11:53:19.554374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.782475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.782475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:32.780132+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":629,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.539781+0000","last_change":"2024-09-21T11:51:43.533694+0000","last_active":"2024-09-21T11:53:19.539781+0000","last_peered":"2024-09-21T11:53:19.539781+0000","last_clean":"2024-09-21T11:53:19.539781+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:53:19.539781+0000","last_undegraded":"2024-09-21T11:53:19.539781+0000","last_fullsized":"2024-09-21T11:53:19.539781+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:04.610554+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":87,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.542163+0000","last_change":"2024-09-21T11:51:52.159555+0000","last_active":"2024-09-21T11:53:19.542163+0000","last_peered":"2024-09-21T11:53:19.542163+0000","last_clean":"2024-09-21T11:53:19.542163+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:53:19.542163+0000","last_undegraded":"2024-09-21T11:53:19.542163+0000","last_fullsized":"2024-09-21T11:53:19.542163+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:52.159433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:52.159433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.229678+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":278,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.546355+0000","last_change":"2024-09-21T11:51:10.143792+0000","last_active":"2024-09-21T11:53:19.546355+0000","last_peered":"2024-09-21T11:53:19.546355+0000","last_clean":"2024-09-21T11:53:19.546355+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:53:19.546355+0000","last_undegraded":"2024-09-21T11:53:19.546355+0000","last_fullsized":"2024-09-21T11:53:19.546355+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.141936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.141936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:34.113373+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":565,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.531437+0000","last_change":"2024-09-21T11:52:48.256878+0000","last_active":"2024-09-21T11:53:19.531437+0000","last_peered":"2024-09-21T11:53:19.531437+0000","last_clean":"2024-09-21T11:53:19.531437+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:53:19.531437+0000","last_undegraded":"2024-09-21T11:53:19.531437+0000","last_fullsized":"2024-09-21T11:53:19.531437+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:48.256830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:48.256830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:00.348262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":289,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.546602+0000","last_change":"2024-09-21T11:51:15.200506+0000","last_active":"2024-09-21T11:53:19.546602+0000","last_peered":"2024-09-21T11:53:19.546602+0000","last_clean":"2024-09-21T11:53:19.546602+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:53:19.546602+0000","last_undegraded":"2024-09-21T11:53:19.546602+0000","last_fullsized":"2024-09-21T11:53:19.546602+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:15.200430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.200430+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:35.130532+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":584,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.542944+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:53:19.542944+0000","last_peered":"2024-09-21T11:53:19.542944+0000","last_clean":"2024-09-21T11:53:19.542944+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:53:19.542944+0000","last_undegraded":"2024-09-21T11:53:19.542944+0000","last_fullsized":"2024-09-21T11:53:19.542944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.179348+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":591,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532670+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:53:19.532670+0000","last_peered":"2024-09-21T11:53:19.532670+0000","last_clean":"2024-09-21T11:53:19.532670+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:53:19.532670+0000","last_undegraded":"2024-09-21T11:53:19.532670+0000","last_fullsized":"2024-09-21T11:53:19.532670+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.222459+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":603,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.550708+0000","last_change":"2024-09-21T11:51:16.962933+0000","last_active":"2024-09-21T11:53:19.550708+0000","last_peered":"2024-09-21T11:53:19.550708+0000","last_clean":"2024-09-21T11:53:19.550708+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:53:19.550708+0000","last_undegraded":"2024-09-21T11:53:19.550708+0000","last_fullsized":"2024-09-21T11:53:19.550708+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:51:16.962860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:16.962860+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.599324+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":612,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532002+0000","last_change":"2024-09-21T11:51:19.708885+0000","last_active":"2024-09-21T11:53:19.532002+0000","last_peered":"2024-09-21T11:53:19.532002+0000","last_clean":"2024-09-21T11:53:19.532002+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:53:19.532002+0000","last_undegraded":"2024-09-21T11:53:19.532002+0000","last_fullsized":"2024-09-21T11:53:19.532002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:51:19.708641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:19.708641+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":598,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535878+0000","last_change":"2024-09-21T11:51:01.690172+0000","last_active":"2024-09-21T11:53:19.535878+0000","last_peered":"2024-09-21T11:53:19.535878+0000","last_clean":"2024-09-21T11:53:19.535878+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:53:19.535878+0000","last_undegraded":"2024-09-21T11:53:19.535878+0000","last_fullsized":"2024-09-21T11:53:19.535878+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.686885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.686885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:23.798235+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":568,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537403+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:53:19.537403+0000","last_peered":"2024-09-21T11:53:19.537403+0000","last_clean":"2024-09-21T11:53:19.537403+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:53:19.537403+0000","last_undegraded":"2024-09-21T11:53:19.537403+0000","last_fullsized":"2024-09-21T11:53:19.537403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:00.087872+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":573,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532952+0000","last_change":"2024-09-21T11:51:05.711964+0000","last_active":"2024-09-21T11:53:19.532952+0000","last_peered":"2024-09-21T11:53:19.532952+0000","last_clean":"2024-09-21T11:53:19.532952+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:53:19.532952+0000","last_undegraded":"2024-09-21T11:53:19.532952+0000","last_fullsized":"2024-09-21T11:53:19.532952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:05.711768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.711768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.204609+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":624,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532091+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:53:19.532091+0000","last_peered":"2024-09-21T11:53:19.532091+0000","last_clean":"2024-09-21T11:53:19.532091+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:53:19.532091+0000","last_undegraded":"2024-09-21T11:53:19.532091+0000","last_fullsized":"2024-09-21T11:53:19.532091+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:56.189942+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":576,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554160+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:53:19.554160+0000","last_peered":"2024-09-21T11:53:19.554160+0000","last_clean":"2024-09-21T11:53:19.554160+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:53:19.554160+0000","last_undegraded":"2024-09-21T11:53:19.554160+0000","last_fullsized":"2024-09-21T11:53:19.554160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:35.701180+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":576,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537152+0000","last_change":"2024-09-21T11:50:38.405183+0000","last_active":"2024-09-21T11:53:19.537152+0000","last_peered":"2024-09-21T11:53:19.537152+0000","last_clean":"2024-09-21T11:53:19.537152+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:53:19.537152+0000","last_undegraded":"2024-09-21T11:53:19.537152+0000","last_fullsized":"2024-09-21T11:53:19.537152+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:38.405118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.405118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:08.616042+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":572,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537611+0000","last_change":"2024-09-21T11:52:45.065651+0000","last_active":"2024-09-21T11:53:19.537611+0000","last_peered":"2024-09-21T11:53:19.537611+0000","last_clean":"2024-09-21T11:53:19.537611+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:53:19.537611+0000","last_undegraded":"2024-09-21T11:53:19.537611+0000","last_fullsized":"2024-09-21T11:53:19.537611+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:45.065594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:45.065594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:50.236288+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":286,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.552606+0000","last_change":"2024-09-21T11:52:12.348687+0000","last_active":"2024-09-21T11:53:19.552606+0000","last_peered":"2024-09-21T11:53:19.552606+0000","last_clean":"2024-09-21T11:53:19.552606+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:53:19.552606+0000","last_undegraded":"2024-09-21T11:53:19.552606+0000","last_fullsized":"2024-09-21T11:53:19.552606+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:18.010458+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":599,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532338+0000","last_change":"2024-09-21T11:51:04.656939+0000","last_active":"2024-09-21T11:53:19.532338+0000","last_peered":"2024-09-21T11:53:19.532338+0000","last_clean":"2024-09-21T11:53:19.532338+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:53:19.532338+0000","last_undegraded":"2024-09-21T11:53:19.532338+0000","last_fullsized":"2024-09-21T11:53:19.532338+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:04.656870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:04.656870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:04.247547+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":674,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:20.555366+0000","last_change":"2024-09-21T11:53:20.555366+0000","last_active":"2024-09-21T11:53:20.555366+0000","last_peered":"2024-09-21T11:53:20.555366+0000","last_clean":"2024-09-21T11:53:20.555366+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:53:20.555366+0000","last_undegraded":"2024-09-21T11:53:20.555366+0000","last_fullsized":"2024-09-21T11:53:20.555366+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:20.554779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.554779+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:25.003905+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":585,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554482+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:53:19.554482+0000","last_peered":"2024-09-21T11:53:19.554482+0000","last_clean":"2024-09-21T11:53:19.554482+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:53:19.554482+0000","last_undegraded":"2024-09-21T11:53:19.554482+0000","last_fullsized":"2024-09-21T11:53:19.554482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:31.809614+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":3860004864,"num_objects":58899,"num_object_clones":0,"num_object_copies":176697,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58899,"num_write_kb":3769536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11639521280,"data_stored":11639521280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58899,"ondisk_log_size":58899,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1113,"num_read_kb":870,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":792,"num_read_kb":787,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234149,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1464596,"kb_used_data":914176,"kb_used_omap":1,"kb_used_meta":550398,"kb_avail":92280556,"statfs":{"total":95995035648,"available":94495289344,"internally_reserved":0,"allocated":936116224,"data_stored":935649692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":563608010},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463082,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2016328,"kb_used_data":1620020,"kb_used_omap":1,"kb_used_meta":396286,"kb_avail":91728824,"statfs":{"total":95995035648,"available":93930315776,"internally_reserved":0,"allocated":1658900480,"data_stored":1658446580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":405797318},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724719,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1686356,"kb_used_data":1163712,"kb_used_omap":1,"kb_used_meta":522622,"kb_avail":92058796,"statfs":{"total":95995035648,"available":94268207104,"internally_reserved":0,"allocated":1191641088,"data_stored":1191173364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":535165382},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920949,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2237852,"kb_used_data":1863752,"kb_used_omap":1,"kb_used_meta":374078,"kb_avail":91507300,"statfs":{"total":95995035648,"available":93703475200,"internally_reserved":0,"allocated":1908482048,"data_stored":1908007293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":383056325},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117179,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1793928,"kb_used_data":1212984,"kb_used_omap":1,"kb_used_meta":580926,"kb_avail":91951224,"statfs":{"total":95995035648,"available":94158053376,"internally_reserved":0,"allocated":1242095616,"data_stored":1241636070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":594868679},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411521,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1955340,"kb_used_data":1410492,"kb_used_omap":1,"kb_used_meta":544830,"kb_avail":91789812,"statfs":{"total":95995035648,"available":93992767488,"internally_reserved":0,"allocated":1444343808,"data_stored":1443880877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":557906373},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1226604,"kb_used_data":708124,"kb_used_omap":1,"kb_used_meta":518462,"kb_avail":92518548,"statfs":{"total":95995035648,"available":94738993152,"internally_reserved":0,"allocated":725118976,"data_stored":724623827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":530905542},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738574,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2833124,"kb_used_data":2350736,"kb_used_omap":1,"kb_used_meta":482366,"kb_avail":90912028,"statfs":{"total":95995035648,"available":93093916672,"internally_reserved":0,"allocated":2407153664,"data_stored":2406671866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":493943237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2445541376,"data_stored":2445541376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":728825856,"data_stored":728825856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1464401920,"data_stored":1464401920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1242431488,"data_stored":1242431488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1910439936,"data_stored":1910439936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1203896320,"data_stored":1203896320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1682636800,"data_stored":1682636800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":961347584,"data_stored":961347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:53:24.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:53:24.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:24.194 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:24.211 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:24.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:24.212 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:24.309 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 32 16 62639 62623 122.288 63.25 0.00295535 0.00813956 2024-09-21T11:53:24.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:24.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:24.338 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:24.439 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:24.482 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:24.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:24.483 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:24.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:24.616 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:24 smithi043 ceph-mon[36454]: from='client.16110 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:53:24.616 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:24 smithi043 ceph-mon[29710]: from='client.16110 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:53:24.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:24.620 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:24.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:24.721 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:24.758 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:24.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:24.759 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:24.827 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:24 smithi175 ceph-mon[32613]: from='client.16110 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:53:24.831 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:24.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:24.923 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:24.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:24.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:25.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:25.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:25.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:25.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:25.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:25.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:25.296 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:53:25.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:25.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 33 16 63893 63877 120.957 78.375 0.0124453 0.00826284 2024-09-21T11:53:25.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:25.385 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:25.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:25.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:53:25.486 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:53:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:53:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:53:25.520 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:25.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:25.521 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:25.590 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:53:25.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:25.682 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:25.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:25.683 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:25.705 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:25.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:25.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:25.815 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:25.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:25.840 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:25 smithi175 ceph-mon[32613]: pgmap v645: 148 pgs: 148 active+clean; 3.6 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 679 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-21T11:53:25.840 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:25 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3938210234' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:25.840 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:25 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:25.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:25 smithi043 ceph-mon[29710]: pgmap v645: 148 pgs: 148 active+clean; 3.6 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 679 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-21T11:53:25.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:25 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3938210234' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:25.878 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:25 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:25.879 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:25 smithi043 ceph-mon[36454]: pgmap v645: 148 pgs: 148 active+clean; 3.6 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 679 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-21T11:53:25.879 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:25 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3938210234' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:25.879 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:25 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:25.882 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:53:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:53:25.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:25.959 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:25.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:25.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:26.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:26.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:26.102 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:26.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:26.203 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:26.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:26.241 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:26.241 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:26.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 34 16 65165 65149 119.738 79.5 0.010067 0.00834579 2024-09-21T11:53:26.342 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:26.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:26.389 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:26.490 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:26.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:26.529 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:26.529 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:26.630 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:26.670 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:26.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:26.671 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[36454]: 5.15 scrub starts 2024-09-21T11:53:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[36454]: 5.15 scrub ok 2024-09-21T11:53:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[36454]: 4.14 scrub starts 2024-09-21T11:53:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[36454]: 4.14 scrub ok 2024-09-21T11:53:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:53:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:53:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:26.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[36454]: osdmap e329: 8 total, 8 up, 8 in 2024-09-21T11:53:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[29710]: 5.15 scrub starts 2024-09-21T11:53:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[29710]: 5.15 scrub ok 2024-09-21T11:53:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[29710]: 4.14 scrub starts 2024-09-21T11:53:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[29710]: 4.14 scrub ok 2024-09-21T11:53:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:53:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:53:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:26 smithi043 ceph-mon[29710]: osdmap e329: 8 total, 8 up, 8 in 2024-09-21T11:53:26.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:26.799 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:26.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:26.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:26.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:26.923 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:26.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:26.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:26 smithi175 ceph-mon[32613]: 5.15 scrub starts 2024-09-21T11:53:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:26 smithi175 ceph-mon[32613]: 5.15 scrub ok 2024-09-21T11:53:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:26 smithi175 ceph-mon[32613]: 4.14 scrub starts 2024-09-21T11:53:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:26 smithi175 ceph-mon[32613]: 4.14 scrub ok 2024-09-21T11:53:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:26 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:53:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:26 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:53:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:26 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:27.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:26 smithi175 ceph-mon[32613]: osdmap e329: 8 total, 8 up, 8 in 2024-09-21T11:53:27.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:27.061 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:27.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:27.062 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:27.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:27.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:27.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:27.231 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":646,"stamp":"2024-09-21T11:53:25.584197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3860601371,"num_objects":59098,"num_object_clones":0,"num_object_copies":177294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59098,"num_whiteouts":0,"num_read":2041,"num_read_kb":1898,"num_write":59734,"num_write_kb":3771209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":597267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59745,"ondisk_log_size":59745,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15214128,"kb_used_data":11243996,"kb_used_omap":12,"kb_used_meta":3969971,"kb_avail":734747088,"statfs":{"total":767960285184,"available":752381018112,"internally_reserved":0,"allocated":11513851904,"data_stored":11510089569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4065250866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[12,4,9,8,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":498597888,"num_objects":7608,"num_object_clones":0,"num_object_copies":22824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7608,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7608,"num_write_kb":486912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.549956"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":577,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554076+0000","last_change":"2024-09-21T11:52:54.792106+0000","last_active":"2024-09-21T11:53:19.554076+0000","last_peered":"2024-09-21T11:53:19.554076+0000","last_clean":"2024-09-21T11:53:19.554076+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:53:19.554076+0000","last_undegraded":"2024-09-21T11:53:19.554076+0000","last_fullsized":"2024-09-21T11:53:19.554076+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:54.792037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:54.792037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.576756+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":608,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.544585+0000","last_change":"2024-09-21T11:52:08.520781+0000","last_active":"2024-09-21T11:53:19.544585+0000","last_peered":"2024-09-21T11:53:19.544585+0000","last_clean":"2024-09-21T11:53:19.544585+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:53:19.544585+0000","last_undegraded":"2024-09-21T11:53:19.544585+0000","last_fullsized":"2024-09-21T11:53:19.544585+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:52:08.520688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:08.520688+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:17.574851+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":286,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:20.097633+0000","last_change":"2024-09-21T11:53:20.097633+0000","last_active":"2024-09-21T11:53:20.097633+0000","last_peered":"2024-09-21T11:53:20.097633+0000","last_clean":"2024-09-21T11:53:20.097633+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:53:20.097633+0000","last_undegraded":"2024-09-21T11:53:20.097633+0000","last_fullsized":"2024-09-21T11:53:20.097633+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:20.097557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.097557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:40.861158+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":389,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.534869+0000","last_change":"2024-09-21T11:51:43.528594+0000","last_active":"2024-09-21T11:53:19.534869+0000","last_peered":"2024-09-21T11:53:19.534869+0000","last_clean":"2024-09-21T11:53:19.534869+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:53:19.534869+0000","last_undegraded":"2024-09-21T11:53:19.534869+0000","last_fullsized":"2024-09-21T11:53:19.534869+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.914122+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":581,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532528+0000","last_change":"2024-09-21T11:52:11.462678+0000","last_active":"2024-09-21T11:53:19.532528+0000","last_peered":"2024-09-21T11:53:19.532528+0000","last_clean":"2024-09-21T11:53:19.532528+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:53:19.532528+0000","last_undegraded":"2024-09-21T11:53:19.532528+0000","last_fullsized":"2024-09-21T11:53:19.532528+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:11.462611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:11.462611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":570,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533616+0000","last_change":"2024-09-21T11:51:01.667426+0000","last_active":"2024-09-21T11:53:19.533616+0000","last_peered":"2024-09-21T11:53:19.533616+0000","last_clean":"2024-09-21T11:53:19.533616+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:53:19.533616+0000","last_undegraded":"2024-09-21T11:53:19.533616+0000","last_fullsized":"2024-09-21T11:53:19.533616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.667366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.667366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.200426+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":616,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532344+0000","last_change":"2024-09-21T11:51:43.409002+0000","last_active":"2024-09-21T11:53:19.532344+0000","last_peered":"2024-09-21T11:53:19.532344+0000","last_clean":"2024-09-21T11:53:19.532344+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:53:19.532344+0000","last_undegraded":"2024-09-21T11:53:19.532344+0000","last_fullsized":"2024-09-21T11:53:19.532344+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:51:43.408804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.408804+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":409,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.544740+0000","last_change":"2024-09-21T11:52:54.554185+0000","last_active":"2024-09-21T11:53:19.544740+0000","last_peered":"2024-09-21T11:53:19.544740+0000","last_clean":"2024-09-21T11:53:19.544740+0000","last_became_active":"2024-09-21T11:51:43.526945+0000","last_became_peered":"2024-09-21T11:51:43.526945+0000","last_unstale":"2024-09-21T11:53:19.544740+0000","last_undegraded":"2024-09-21T11:53:19.544740+0000","last_fullsized":"2024-09-21T11:53:19.544740+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:54.553857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:54.553857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:55.495025+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":576,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537431+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:53:19.537431+0000","last_peered":"2024-09-21T11:53:19.537431+0000","last_clean":"2024-09-21T11:53:19.537431+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:53:19.537431+0000","last_undegraded":"2024-09-21T11:53:19.537431+0000","last_fullsized":"2024-09-21T11:53:19.537431+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.870896+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":574,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532543+0000","last_change":"2024-09-21T11:52:55.214031+0000","last_active":"2024-09-21T11:53:19.532543+0000","last_peered":"2024-09-21T11:53:19.532543+0000","last_clean":"2024-09-21T11:53:19.532543+0000","last_became_active":"2024-09-21T11:51:43.545021+0000","last_became_peered":"2024-09-21T11:51:43.545021+0000","last_unstale":"2024-09-21T11:53:19.532543+0000","last_undegraded":"2024-09-21T11:53:19.532543+0000","last_fullsized":"2024-09-21T11:53:19.532543+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:52:55.213938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:55.213938+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:12.879409+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":568,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535203+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:53:19.535203+0000","last_peered":"2024-09-21T11:53:19.535203+0000","last_clean":"2024-09-21T11:53:19.535203+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:53:19.535203+0000","last_undegraded":"2024-09-21T11:53:19.535203+0000","last_fullsized":"2024-09-21T11:53:19.535203+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.824812+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":594,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551096+0000","last_change":"2024-09-21T11:52:50.731704+0000","last_active":"2024-09-21T11:53:19.551096+0000","last_peered":"2024-09-21T11:53:19.551096+0000","last_clean":"2024-09-21T11:53:19.551096+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:53:19.551096+0000","last_undegraded":"2024-09-21T11:53:19.551096+0000","last_fullsized":"2024-09-21T11:53:19.551096+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:50.731653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:50.731653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.382981+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":606,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536155+0000","last_change":"2024-09-21T11:52:11.666514+0000","last_active":"2024-09-21T11:53:19.536155+0000","last_peered":"2024-09-21T11:53:19.536155+0000","last_clean":"2024-09-21T11:53:19.536155+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:53:19.536155+0000","last_undegraded":"2024-09-21T11:53:19.536155+0000","last_fullsized":"2024-09-21T11:53:19.536155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:11.666440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:11.666440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:23.501612+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":580,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532734+0000","last_change":"2024-09-21T11:51:43.547924+0000","last_active":"2024-09-21T11:53:19.532734+0000","last_peered":"2024-09-21T11:53:19.532734+0000","last_clean":"2024-09-21T11:53:19.532734+0000","last_became_active":"2024-09-21T11:51:43.545191+0000","last_became_peered":"2024-09-21T11:51:43.545191+0000","last_unstale":"2024-09-21T11:53:19.532734+0000","last_undegraded":"2024-09-21T11:53:19.532734+0000","last_fullsized":"2024-09-21T11:53:19.532734+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.268969+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":582,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537718+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:53:19.537718+0000","last_peered":"2024-09-21T11:53:19.537718+0000","last_clean":"2024-09-21T11:53:19.537718+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:53:19.537718+0000","last_undegraded":"2024-09-21T11:53:19.537718+0000","last_fullsized":"2024-09-21T11:53:19.537718+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.104956+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":586,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536149+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:53:19.536149+0000","last_peered":"2024-09-21T11:53:19.536149+0000","last_clean":"2024-09-21T11:53:19.536149+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:53:19.536149+0000","last_undegraded":"2024-09-21T11:53:19.536149+0000","last_fullsized":"2024-09-21T11:53:19.536149+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.899330+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":79,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.539739+0000","last_change":"2024-09-21T11:51:43.524599+0000","last_active":"2024-09-21T11:53:19.539739+0000","last_peered":"2024-09-21T11:53:19.539739+0000","last_clean":"2024-09-21T11:53:19.539739+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:53:19.539739+0000","last_undegraded":"2024-09-21T11:53:19.539739+0000","last_fullsized":"2024-09-21T11:53:19.539739+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:32.302842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.302842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.577507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":568,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551860+0000","last_change":"2024-09-21T11:50:35.718093+0000","last_active":"2024-09-21T11:53:19.551860+0000","last_peered":"2024-09-21T11:53:19.551860+0000","last_clean":"2024-09-21T11:53:19.551860+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:53:19.551860+0000","last_undegraded":"2024-09-21T11:53:19.551860+0000","last_fullsized":"2024-09-21T11:53:19.551860+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:35.717991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.717991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:10.565445+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":701,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535722+0000","last_change":"2024-09-21T11:50:25.966805+0000","last_active":"2024-09-21T11:53:19.535722+0000","last_peered":"2024-09-21T11:53:19.535722+0000","last_clean":"2024-09-21T11:53:19.535722+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:53:19.535722+0000","last_undegraded":"2024-09-21T11:53:19.535722+0000","last_fullsized":"2024-09-21T11:53:19.535722+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:50:25.966740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.966740+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.907984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":278,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.552757+0000","last_change":"2024-09-21T11:50:31.012243+0000","last_active":"2024-09-21T11:53:19.552757+0000","last_peered":"2024-09-21T11:53:19.552757+0000","last_clean":"2024-09-21T11:53:19.552757+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:53:19.552757+0000","last_undegraded":"2024-09-21T11:53:19.552757+0000","last_fullsized":"2024-09-21T11:53:19.552757+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:31.012177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:31.012177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:23.104375+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":574,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533727+0000","last_change":"2024-09-21T11:52:44.264180+0000","last_active":"2024-09-21T11:53:19.533727+0000","last_peered":"2024-09-21T11:53:19.533727+0000","last_clean":"2024-09-21T11:53:19.533727+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:53:19.533727+0000","last_undegraded":"2024-09-21T11:53:19.533727+0000","last_fullsized":"2024-09-21T11:53:19.533727+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.264114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.264114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.220096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":393,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.546096+0000","last_change":"2024-09-21T11:51:43.543408+0000","last_active":"2024-09-21T11:53:19.546096+0000","last_peered":"2024-09-21T11:53:19.546096+0000","last_clean":"2024-09-21T11:53:19.546096+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:53:19.546096+0000","last_undegraded":"2024-09-21T11:53:19.546096+0000","last_fullsized":"2024-09-21T11:53:19.546096+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.122227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.122227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.038540+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":643,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533841+0000","last_change":"2024-09-21T11:51:12.652049+0000","last_active":"2024-09-21T11:53:19.533841+0000","last_peered":"2024-09-21T11:53:19.533841+0000","last_clean":"2024-09-21T11:53:19.533841+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:53:19.533841+0000","last_undegraded":"2024-09-21T11:53:19.533841+0000","last_fullsized":"2024-09-21T11:53:19.533841+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:51:12.651967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.651967+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":79,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532932+0000","last_change":"2024-09-21T11:51:43.549287+0000","last_active":"2024-09-21T11:53:19.532932+0000","last_peered":"2024-09-21T11:53:19.532932+0000","last_clean":"2024-09-21T11:53:19.532932+0000","last_became_active":"2024-09-21T11:51:43.548726+0000","last_became_peered":"2024-09-21T11:51:43.548726+0000","last_unstale":"2024-09-21T11:53:19.532932+0000","last_undegraded":"2024-09-21T11:53:19.532932+0000","last_fullsized":"2024-09-21T11:53:19.532932+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.908339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.908339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":90,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.553070+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:53:19.553070+0000","last_peered":"2024-09-21T11:53:19.553070+0000","last_clean":"2024-09-21T11:53:19.553070+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:53:19.553070+0000","last_undegraded":"2024-09-21T11:53:19.553070+0000","last_fullsized":"2024-09-21T11:53:19.553070+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.632356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":274,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.545613+0000","last_change":"2024-09-21T11:50:02.920313+0000","last_active":"2024-09-21T11:53:19.545613+0000","last_peered":"2024-09-21T11:53:19.545613+0000","last_clean":"2024-09-21T11:53:19.545613+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:53:19.545613+0000","last_undegraded":"2024-09-21T11:53:19.545613+0000","last_fullsized":"2024-09-21T11:53:19.545613+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:02.920229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.920229+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:27.127472+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":611,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554566+0000","last_change":"2024-09-21T11:49:58.702620+0000","last_active":"2024-09-21T11:53:19.554566+0000","last_peered":"2024-09-21T11:53:19.554566+0000","last_clean":"2024-09-21T11:53:19.554566+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:53:19.554566+0000","last_undegraded":"2024-09-21T11:53:19.554566+0000","last_fullsized":"2024-09-21T11:53:19.554566+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:49:58.702544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.702544+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:26.900715+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":765,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.539690+0000","last_change":"2024-09-21T11:51:44.478918+0000","last_active":"2024-09-21T11:53:19.539690+0000","last_peered":"2024-09-21T11:53:19.539690+0000","last_clean":"2024-09-21T11:53:19.539690+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:53:19.539690+0000","last_undegraded":"2024-09-21T11:53:19.539690+0000","last_fullsized":"2024-09-21T11:53:19.539690+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:44.478855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.478855+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.644866+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"327'3700","reported_seq":3739,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.143368+0000","last_change":"2024-09-21T11:52:45.608799+0000","last_active":"2024-09-21T11:53:21.143368+0000","last_peered":"2024-09-21T11:53:21.143368+0000","last_clean":"2024-09-21T11:53:21.143368+0000","last_became_active":"2024-09-21T11:52:45.608448+0000","last_became_peered":"2024-09-21T11:52:45.608448+0000","last_unstale":"2024-09-21T11:53:21.143368+0000","last_undegraded":"2024-09-21T11:53:21.143368+0000","last_fullsized":"2024-09-21T11:53:21.143368+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3700,"log_dups_size":0,"ondisk_log_size":3700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:51.400025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242483200,"num_objects":3700,"num_object_clones":0,"num_object_copies":11100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3700,"num_write_kb":236800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":320,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.546210+0000","last_change":"2024-09-21T11:52:39.116264+0000","last_active":"2024-09-21T11:53:19.546210+0000","last_peered":"2024-09-21T11:53:19.546210+0000","last_clean":"2024-09-21T11:53:19.546210+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:53:19.546210+0000","last_undegraded":"2024-09-21T11:53:19.546210+0000","last_fullsized":"2024-09-21T11:53:19.546210+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:52:39.116197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:39.116197+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:40.719106+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":600,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532552+0000","last_change":"2024-09-21T11:49:47.205216+0000","last_active":"2024-09-21T11:53:19.532552+0000","last_peered":"2024-09-21T11:53:19.532552+0000","last_clean":"2024-09-21T11:53:19.532552+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:53:19.532552+0000","last_undegraded":"2024-09-21T11:53:19.532552+0000","last_fullsized":"2024-09-21T11:53:19.532552+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.204889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.204889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:28.205006+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"294'9","reported_seq":126,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535750+0000","last_change":"2024-09-21T11:51:43.526079+0000","last_active":"2024-09-21T11:53:19.535750+0000","last_peered":"2024-09-21T11:53:19.535750+0000","last_clean":"2024-09-21T11:53:19.535750+0000","last_became_active":"2024-09-21T11:51:43.525566+0000","last_became_peered":"2024-09-21T11:51:43.525566+0000","last_unstale":"2024-09-21T11:53:19.535750+0000","last_undegraded":"2024-09-21T11:53:19.535750+0000","last_fullsized":"2024-09-21T11:53:19.535750+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:11.932479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:11.932479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.943835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":594,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537026+0000","last_change":"2024-09-21T11:52:38.781242+0000","last_active":"2024-09-21T11:53:19.537026+0000","last_peered":"2024-09-21T11:53:19.537026+0000","last_clean":"2024-09-21T11:53:19.537026+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:53:19.537026+0000","last_undegraded":"2024-09-21T11:53:19.537026+0000","last_fullsized":"2024-09-21T11:53:19.537026+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.781189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.781189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:50.278404+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"327'3691","reported_seq":3730,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.050435+0000","last_change":"2024-09-21T11:52:45.606555+0000","last_active":"2024-09-21T11:53:21.050435+0000","last_peered":"2024-09-21T11:53:21.050435+0000","last_clean":"2024-09-21T11:53:21.050435+0000","last_became_active":"2024-09-21T11:52:45.606226+0000","last_became_peered":"2024-09-21T11:52:45.606226+0000","last_unstale":"2024-09-21T11:53:21.050435+0000","last_undegraded":"2024-09-21T11:53:21.050435+0000","last_fullsized":"2024-09-21T11:53:21.050435+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3691,"log_dups_size":0,"ondisk_log_size":3691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.505126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241893376,"num_objects":3691,"num_object_clones":0,"num_object_copies":11073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3691,"num_write_kb":236224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":640,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532194+0000","last_change":"2024-09-21T11:52:47.279483+0000","last_active":"2024-09-21T11:53:19.532194+0000","last_peered":"2024-09-21T11:53:19.532194+0000","last_clean":"2024-09-21T11:53:19.532194+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:53:19.532194+0000","last_undegraded":"2024-09-21T11:53:19.532194+0000","last_fullsized":"2024-09-21T11:53:19.532194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:52:47.279423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:47.279423+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:10.328491+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":594,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.534155+0000","last_change":"2024-09-21T11:52:42.279545+0000","last_active":"2024-09-21T11:53:19.534155+0000","last_peered":"2024-09-21T11:53:19.534155+0000","last_clean":"2024-09-21T11:53:19.534155+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:53:19.534155+0000","last_undegraded":"2024-09-21T11:53:19.534155+0000","last_fullsized":"2024-09-21T11:53:19.534155+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:42.279496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:42.279496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.626370+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":79,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532406+0000","last_change":"2024-09-21T11:51:43.542323+0000","last_active":"2024-09-21T11:53:19.532406+0000","last_peered":"2024-09-21T11:53:19.532406+0000","last_clean":"2024-09-21T11:53:19.532406+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:53:19.532406+0000","last_undegraded":"2024-09-21T11:53:19.532406+0000","last_fullsized":"2024-09-21T11:53:19.532406+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.940335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.940335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":73,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535196+0000","last_change":"2024-09-21T11:52:01.954772+0000","last_active":"2024-09-21T11:53:19.535196+0000","last_peered":"2024-09-21T11:53:19.535196+0000","last_clean":"2024-09-21T11:53:19.535196+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:53:19.535196+0000","last_undegraded":"2024-09-21T11:53:19.535196+0000","last_fullsized":"2024-09-21T11:53:19.535196+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:08.977984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.977984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:21.812560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"327'3722","reported_seq":3761,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.045611+0000","last_change":"2024-09-21T11:52:45.635813+0000","last_active":"2024-09-21T11:53:21.045611+0000","last_peered":"2024-09-21T11:53:21.045611+0000","last_clean":"2024-09-21T11:53:21.045611+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:53:21.045611+0000","last_undegraded":"2024-09-21T11:53:21.045611+0000","last_fullsized":"2024-09-21T11:53:21.045611+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3722,"log_dups_size":0,"ondisk_log_size":3722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:08.048600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243924992,"num_objects":3722,"num_object_clones":0,"num_object_copies":11166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3722,"num_write_kb":238208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":591,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536248+0000","last_change":"2024-09-21T11:51:43.525478+0000","last_active":"2024-09-21T11:53:19.536248+0000","last_peered":"2024-09-21T11:53:19.536248+0000","last_clean":"2024-09-21T11:53:19.536248+0000","last_became_active":"2024-09-21T11:51:43.523852+0000","last_became_peered":"2024-09-21T11:51:43.523852+0000","last_unstale":"2024-09-21T11:53:19.536248+0000","last_undegraded":"2024-09-21T11:53:19.536248+0000","last_fullsized":"2024-09-21T11:53:19.536248+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:07.916000+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":616,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.555058+0000","last_change":"2024-09-21T11:50:28.657341+0000","last_active":"2024-09-21T11:53:19.555058+0000","last_peered":"2024-09-21T11:53:19.555058+0000","last_clean":"2024-09-21T11:53:19.555058+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:53:19.555058+0000","last_undegraded":"2024-09-21T11:53:19.555058+0000","last_fullsized":"2024-09-21T11:53:19.555058+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:28.657261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.657261+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.614169+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":770,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.531978+0000","last_change":"2024-09-21T11:51:20.675165+0000","last_active":"2024-09-21T11:53:19.531978+0000","last_peered":"2024-09-21T11:53:19.531978+0000","last_clean":"2024-09-21T11:53:19.531978+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:53:19.531978+0000","last_undegraded":"2024-09-21T11:53:19.531978+0000","last_fullsized":"2024-09-21T11:53:19.531978+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:51:20.675101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.675101+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":584,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537993+0000","last_change":"2024-09-21T11:52:14.010215+0000","last_active":"2024-09-21T11:53:19.537993+0000","last_peered":"2024-09-21T11:53:19.537993+0000","last_clean":"2024-09-21T11:53:19.537993+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:53:19.537993+0000","last_undegraded":"2024-09-21T11:53:19.537993+0000","last_fullsized":"2024-09-21T11:53:19.537993+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:14.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:14.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:42.958931+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"327'3545","reported_seq":3584,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.134364+0000","last_change":"2024-09-21T11:52:45.608507+0000","last_active":"2024-09-21T11:53:21.134364+0000","last_peered":"2024-09-21T11:53:21.134364+0000","last_clean":"2024-09-21T11:53:21.134364+0000","last_became_active":"2024-09-21T11:52:45.608219+0000","last_became_peered":"2024-09-21T11:52:45.608219+0000","last_unstale":"2024-09-21T11:53:21.134364+0000","last_undegraded":"2024-09-21T11:53:21.134364+0000","last_fullsized":"2024-09-21T11:53:21.134364+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3545,"log_dups_size":0,"ondisk_log_size":3545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.821182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232325120,"num_objects":3545,"num_object_clones":0,"num_object_copies":10635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3545,"num_write_kb":226880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":298,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.546507+0000","last_change":"2024-09-21T11:51:08.137774+0000","last_active":"2024-09-21T11:53:19.546507+0000","last_peered":"2024-09-21T11:53:19.546507+0000","last_clean":"2024-09-21T11:53:19.546507+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:53:19.546507+0000","last_undegraded":"2024-09-21T11:53:19.546507+0000","last_fullsized":"2024-09-21T11:53:19.546507+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:51:08.137700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.137700+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:30.141542+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":609,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537545+0000","last_change":"2024-09-21T11:52:39.713149+0000","last_active":"2024-09-21T11:53:19.537545+0000","last_peered":"2024-09-21T11:53:19.537545+0000","last_clean":"2024-09-21T11:53:19.537545+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:53:19.537545+0000","last_undegraded":"2024-09-21T11:53:19.537545+0000","last_fullsized":"2024-09-21T11:53:19.537545+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:39.713105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:39.713105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.384706+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":741,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551548+0000","last_change":"2024-09-21T11:52:37.704950+0000","last_active":"2024-09-21T11:53:19.551548+0000","last_peered":"2024-09-21T11:53:19.551548+0000","last_clean":"2024-09-21T11:53:19.551548+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:53:19.551548+0000","last_undegraded":"2024-09-21T11:53:19.551548+0000","last_fullsized":"2024-09-21T11:53:19.551548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:52:37.704899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:37.704899+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:53.365928+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":581,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551251+0000","last_change":"2024-09-21T11:52:43.682014+0000","last_active":"2024-09-21T11:53:19.551251+0000","last_peered":"2024-09-21T11:53:19.551251+0000","last_clean":"2024-09-21T11:53:19.551251+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:53:19.551251+0000","last_undegraded":"2024-09-21T11:53:19.551251+0000","last_fullsized":"2024-09-21T11:53:19.551251+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:43.681959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:43.681959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:47.395198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"327'3628","reported_seq":3667,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.040367+0000","last_change":"2024-09-21T11:52:45.607195+0000","last_active":"2024-09-21T11:53:21.040367+0000","last_peered":"2024-09-21T11:53:21.040367+0000","last_clean":"2024-09-21T11:53:21.040367+0000","last_became_active":"2024-09-21T11:52:45.606865+0000","last_became_peered":"2024-09-21T11:52:45.606865+0000","last_unstale":"2024-09-21T11:53:21.040367+0000","last_undegraded":"2024-09-21T11:53:21.040367+0000","last_fullsized":"2024-09-21T11:53:21.040367+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3628,"log_dups_size":0,"ondisk_log_size":3628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.533011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237764608,"num_objects":3628,"num_object_clones":0,"num_object_copies":10884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3628,"num_write_kb":232192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":632,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551698+0000","last_change":"2024-09-21T11:51:45.114831+0000","last_active":"2024-09-21T11:53:19.551698+0000","last_peered":"2024-09-21T11:53:19.551698+0000","last_clean":"2024-09-21T11:53:19.551698+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:53:19.551698+0000","last_undegraded":"2024-09-21T11:53:19.551698+0000","last_fullsized":"2024-09-21T11:53:19.551698+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:51:45.114143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:45.114143+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":271,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.552305+0000","last_change":"2024-09-21T11:49:59.954903+0000","last_active":"2024-09-21T11:53:19.552305+0000","last_peered":"2024-09-21T11:53:19.552305+0000","last_clean":"2024-09-21T11:53:19.552305+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:53:19.552305+0000","last_undegraded":"2024-09-21T11:53:19.552305+0000","last_fullsized":"2024-09-21T11:53:19.552305+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:59.954835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:59.954835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:25.047222+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":569,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536814+0000","last_change":"2024-09-21T11:51:52.563725+0000","last_active":"2024-09-21T11:53:19.536814+0000","last_peered":"2024-09-21T11:53:19.536814+0000","last_clean":"2024-09-21T11:53:19.536814+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:53:19.536814+0000","last_undegraded":"2024-09-21T11:53:19.536814+0000","last_fullsized":"2024-09-21T11:53:19.536814+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:52.563656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:52.563656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:14.500261+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":279,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.545353+0000","last_change":"2024-09-21T11:52:09.412149+0000","last_active":"2024-09-21T11:53:19.545353+0000","last_peered":"2024-09-21T11:53:19.545353+0000","last_clean":"2024-09-21T11:53:19.545353+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:53:19.545353+0000","last_undegraded":"2024-09-21T11:53:19.545353+0000","last_fullsized":"2024-09-21T11:53:19.545353+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:09.412091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:09.412091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:32.303837+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"327'3618","reported_seq":3656,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.051094+0000","last_change":"2024-09-21T11:52:45.607383+0000","last_active":"2024-09-21T11:53:21.051094+0000","last_peered":"2024-09-21T11:53:21.051094+0000","last_clean":"2024-09-21T11:53:21.051094+0000","last_became_active":"2024-09-21T11:52:45.607000+0000","last_became_peered":"2024-09-21T11:52:45.607000+0000","last_unstale":"2024-09-21T11:53:21.051094+0000","last_undegraded":"2024-09-21T11:53:21.051094+0000","last_fullsized":"2024-09-21T11:53:21.051094+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3618,"log_dups_size":0,"ondisk_log_size":3618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.571620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237109248,"num_objects":3618,"num_object_clones":0,"num_object_copies":10854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3618,"num_write_kb":231552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":387,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:20.530380+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:53:20.530380+0000","last_peered":"2024-09-21T11:53:20.530380+0000","last_clean":"2024-09-21T11:53:20.530380+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:53:20.530380+0000","last_undegraded":"2024-09-21T11:53:20.530380+0000","last_fullsized":"2024-09-21T11:53:20.530380+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:15.031330+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":96,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551761+0000","last_change":"2024-09-21T11:51:44.078378+0000","last_active":"2024-09-21T11:53:19.551761+0000","last_peered":"2024-09-21T11:53:19.551761+0000","last_clean":"2024-09-21T11:53:19.551761+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:53:19.551761+0000","last_undegraded":"2024-09-21T11:53:19.551761+0000","last_fullsized":"2024-09-21T11:53:19.551761+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:44.078299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.078299+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.512892+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":609,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535273+0000","last_change":"2024-09-21T11:51:12.562026+0000","last_active":"2024-09-21T11:53:19.535273+0000","last_peered":"2024-09-21T11:53:19.535273+0000","last_clean":"2024-09-21T11:53:19.535273+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:53:19.535273+0000","last_undegraded":"2024-09-21T11:53:19.535273+0000","last_fullsized":"2024-09-21T11:53:19.535273+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:51:12.561961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.561961+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.897864+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":583,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.555154+0000","last_change":"2024-09-21T11:50:37.858401+0000","last_active":"2024-09-21T11:53:19.555154+0000","last_peered":"2024-09-21T11:53:19.555154+0000","last_clean":"2024-09-21T11:53:19.555154+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:53:19.555154+0000","last_undegraded":"2024-09-21T11:53:19.555154+0000","last_fullsized":"2024-09-21T11:53:19.555154+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:04.618872+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":558,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533434+0000","last_change":"2024-09-21T11:51:49.357728+0000","last_active":"2024-09-21T11:53:19.533434+0000","last_peered":"2024-09-21T11:53:19.533434+0000","last_clean":"2024-09-21T11:53:19.533434+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:53:19.533434+0000","last_undegraded":"2024-09-21T11:53:19.533434+0000","last_fullsized":"2024-09-21T11:53:19.533434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:49.357659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:49.357659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"327'3784","reported_seq":3823,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.207623+0000","last_change":"2024-09-21T11:52:45.605195+0000","last_active":"2024-09-21T11:53:21.207623+0000","last_peered":"2024-09-21T11:53:21.207623+0000","last_clean":"2024-09-21T11:53:21.207623+0000","last_became_active":"2024-09-21T11:52:45.604600+0000","last_became_peered":"2024-09-21T11:52:45.604600+0000","last_unstale":"2024-09-21T11:53:21.207623+0000","last_undegraded":"2024-09-21T11:53:21.207623+0000","last_fullsized":"2024-09-21T11:53:21.207623+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3784,"log_dups_size":0,"ondisk_log_size":3784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:01.765652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247988224,"num_objects":3784,"num_object_clones":0,"num_object_copies":11352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3784,"num_write_kb":242176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":222,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535425+0000","last_change":"2024-09-21T11:51:41.006710+0000","last_active":"2024-09-21T11:53:19.535425+0000","last_peered":"2024-09-21T11:53:19.535425+0000","last_clean":"2024-09-21T11:53:19.535425+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:53:19.535425+0000","last_undegraded":"2024-09-21T11:53:19.535425+0000","last_fullsized":"2024-09-21T11:53:19.535425+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:51:41.006622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:41.006622+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:18.027646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":283,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.545762+0000","last_change":"2024-09-21T11:51:43.346789+0000","last_active":"2024-09-21T11:53:19.545762+0000","last_peered":"2024-09-21T11:53:19.545762+0000","last_clean":"2024-09-21T11:53:19.545762+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:53:19.545762+0000","last_undegraded":"2024-09-21T11:53:19.545762+0000","last_fullsized":"2024-09-21T11:53:19.545762+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:43.346718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.346718+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.960974+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":619,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:20.956563+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:53:20.956563+0000","last_peered":"2024-09-21T11:53:20.956563+0000","last_clean":"2024-09-21T11:53:20.956563+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:53:20.956563+0000","last_undegraded":"2024-09-21T11:53:20.956563+0000","last_fullsized":"2024-09-21T11:53:20.956563+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:42.859403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":578,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533728+0000","last_change":"2024-09-21T11:49:47.008541+0000","last_active":"2024-09-21T11:53:19.533728+0000","last_peered":"2024-09-21T11:53:19.533728+0000","last_clean":"2024-09-21T11:53:19.533728+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:53:19.533728+0000","last_undegraded":"2024-09-21T11:53:19.533728+0000","last_fullsized":"2024-09-21T11:53:19.533728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.008369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.008369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.883485+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":573,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.541773+0000","last_change":"2024-09-21T11:49:54.388131+0000","last_active":"2024-09-21T11:53:19.541773+0000","last_peered":"2024-09-21T11:53:19.541773+0000","last_clean":"2024-09-21T11:53:19.541773+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:53:19.541773+0000","last_undegraded":"2024-09-21T11:53:19.541773+0000","last_fullsized":"2024-09-21T11:53:19.541773+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.388068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.388068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:25.965811+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"327'3729","reported_seq":3768,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:22.074023+0000","last_change":"2024-09-21T11:52:45.604487+0000","last_active":"2024-09-21T11:53:22.074023+0000","last_peered":"2024-09-21T11:53:22.074023+0000","last_clean":"2024-09-21T11:53:22.074023+0000","last_became_active":"2024-09-21T11:52:45.604244+0000","last_became_peered":"2024-09-21T11:52:45.604244+0000","last_unstale":"2024-09-21T11:53:22.074023+0000","last_undegraded":"2024-09-21T11:53:22.074023+0000","last_fullsized":"2024-09-21T11:53:22.074023+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3729,"log_dups_size":0,"ondisk_log_size":3729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.006653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244383744,"num_objects":3729,"num_object_clones":0,"num_object_copies":11187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3729,"num_write_kb":238656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":659,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:20.438262+0000","last_change":"2024-09-21T11:53:20.438262+0000","last_active":"2024-09-21T11:53:20.438262+0000","last_peered":"2024-09-21T11:53:20.438262+0000","last_clean":"2024-09-21T11:53:20.438262+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:53:20.438262+0000","last_undegraded":"2024-09-21T11:53:20.438262+0000","last_fullsized":"2024-09-21T11:53:20.438262+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:53:20.438174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.438174+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:24.864136+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":593,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551895+0000","last_change":"2024-09-21T11:51:20.858177+0000","last_active":"2024-09-21T11:53:19.551895+0000","last_peered":"2024-09-21T11:53:19.551895+0000","last_clean":"2024-09-21T11:53:19.551895+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:53:19.551895+0000","last_undegraded":"2024-09-21T11:53:19.551895+0000","last_fullsized":"2024-09-21T11:53:19.551895+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.858109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.858109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.570964+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1420,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.880333+0000","last_change":"2024-09-21T11:49:51.434402+0000","last_active":"2024-09-21T11:53:19.880333+0000","last_peered":"2024-09-21T11:53:19.880333+0000","last_clean":"2024-09-21T11:53:19.880333+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:53:19.880333+0000","last_undegraded":"2024-09-21T11:53:19.880333+0000","last_fullsized":"2024-09-21T11:53:19.880333+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:51.434333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:49:51.434333+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:29.068051+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":792,"num_read_kb":787,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":591,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538855+0000","last_change":"2024-09-21T11:51:17.645920+0000","last_active":"2024-09-21T11:53:19.538855+0000","last_peered":"2024-09-21T11:53:19.538855+0000","last_clean":"2024-09-21T11:53:19.538855+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:53:19.538855+0000","last_undegraded":"2024-09-21T11:53:19.538855+0000","last_fullsized":"2024-09-21T11:53:19.538855+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.645823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.645823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:17.901222+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":287,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.552102+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:53:19.552102+0000","last_peered":"2024-09-21T11:53:19.552102+0000","last_clean":"2024-09-21T11:53:19.552102+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:53:19.552102+0000","last_undegraded":"2024-09-21T11:53:19.552102+0000","last_fullsized":"2024-09-21T11:53:19.552102+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.718884+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"327'3616","reported_seq":3655,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.044537+0000","last_change":"2024-09-21T11:52:45.607096+0000","last_active":"2024-09-21T11:53:21.044537+0000","last_peered":"2024-09-21T11:53:21.044537+0000","last_clean":"2024-09-21T11:53:21.044537+0000","last_became_active":"2024-09-21T11:52:45.606777+0000","last_became_peered":"2024-09-21T11:52:45.606777+0000","last_unstale":"2024-09-21T11:53:21.044537+0000","last_undegraded":"2024-09-21T11:53:21.044537+0000","last_fullsized":"2024-09-21T11:53:21.044537+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3616,"log_dups_size":0,"ondisk_log_size":3616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.742069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236978176,"num_objects":3616,"num_object_clones":0,"num_object_copies":10848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":231424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":782,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.544257+0000","last_change":"2024-09-21T11:50:29.389640+0000","last_active":"2024-09-21T11:53:19.544257+0000","last_peered":"2024-09-21T11:53:19.544257+0000","last_clean":"2024-09-21T11:53:19.544257+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:53:19.544257+0000","last_undegraded":"2024-09-21T11:53:19.544257+0000","last_fullsized":"2024-09-21T11:53:19.544257+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:29.389555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:29.389555+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:33.416003+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":586,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532735+0000","last_change":"2024-09-21T11:50:59.680952+0000","last_active":"2024-09-21T11:53:19.532735+0000","last_peered":"2024-09-21T11:53:19.532735+0000","last_clean":"2024-09-21T11:53:19.532735+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:53:19.532735+0000","last_undegraded":"2024-09-21T11:53:19.532735+0000","last_fullsized":"2024-09-21T11:53:19.532735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:59.680755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:59.680755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.232600+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":648,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538088+0000","last_change":"2024-09-21T11:52:53.118045+0000","last_active":"2024-09-21T11:53:19.538088+0000","last_peered":"2024-09-21T11:53:19.538088+0000","last_clean":"2024-09-21T11:53:19.538088+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:53:19.538088+0000","last_undegraded":"2024-09-21T11:53:19.538088+0000","last_fullsized":"2024-09-21T11:53:19.538088+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:52:53.117963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:52:53.117963+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:07.290308+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":585,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.541708+0000","last_change":"2024-09-21T11:51:18.318095+0000","last_active":"2024-09-21T11:53:19.541708+0000","last_peered":"2024-09-21T11:53:19.541708+0000","last_clean":"2024-09-21T11:53:19.541708+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:53:19.541708+0000","last_undegraded":"2024-09-21T11:53:19.541708+0000","last_fullsized":"2024-09-21T11:53:19.541708+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.317950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.317950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.110119+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":573,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536231+0000","last_change":"2024-09-21T11:51:46.607389+0000","last_active":"2024-09-21T11:53:19.536231+0000","last_peered":"2024-09-21T11:53:19.536231+0000","last_clean":"2024-09-21T11:53:19.536231+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:53:19.536231+0000","last_undegraded":"2024-09-21T11:53:19.536231+0000","last_fullsized":"2024-09-21T11:53:19.536231+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:46.607326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:46.607326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"327'3706","reported_seq":3745,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.033998+0000","last_change":"2024-09-21T11:52:45.605911+0000","last_active":"2024-09-21T11:53:21.033998+0000","last_peered":"2024-09-21T11:53:21.033998+0000","last_clean":"2024-09-21T11:53:21.033998+0000","last_became_active":"2024-09-21T11:52:45.605575+0000","last_became_peered":"2024-09-21T11:52:45.605575+0000","last_unstale":"2024-09-21T11:53:21.033998+0000","last_undegraded":"2024-09-21T11:53:21.033998+0000","last_fullsized":"2024-09-21T11:53:21.033998+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3706,"log_dups_size":0,"ondisk_log_size":3706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:48.712206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242876416,"num_objects":3706,"num_object_clones":0,"num_object_copies":11118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3706,"num_write_kb":237184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":594,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.540270+0000","last_change":"2024-09-21T11:51:43.526788+0000","last_active":"2024-09-21T11:53:19.540270+0000","last_peered":"2024-09-21T11:53:19.540270+0000","last_clean":"2024-09-21T11:53:19.540270+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:53:19.540270+0000","last_undegraded":"2024-09-21T11:53:19.540270+0000","last_fullsized":"2024-09-21T11:53:19.540270+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:37.352834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.352834+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.596503+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":87,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.531828+0000","last_change":"2024-09-21T11:52:13.709151+0000","last_active":"2024-09-21T11:53:19.531828+0000","last_peered":"2024-09-21T11:53:19.531828+0000","last_clean":"2024-09-21T11:53:19.531828+0000","last_became_active":"2024-09-21T11:51:43.532027+0000","last_became_peered":"2024-09-21T11:51:43.532027+0000","last_unstale":"2024-09-21T11:53:19.531828+0000","last_undegraded":"2024-09-21T11:53:19.531828+0000","last_fullsized":"2024-09-21T11:53:19.531828+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:13.709074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:13.709074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:25.271458+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":568,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536087+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:53:19.536087+0000","last_peered":"2024-09-21T11:53:19.536087+0000","last_clean":"2024-09-21T11:53:19.536087+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:53:19.536087+0000","last_undegraded":"2024-09-21T11:53:19.536087+0000","last_fullsized":"2024-09-21T11:53:19.536087+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:00.863312+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":572,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532800+0000","last_change":"2024-09-21T11:51:18.664721+0000","last_active":"2024-09-21T11:53:19.532800+0000","last_peered":"2024-09-21T11:53:19.532800+0000","last_clean":"2024-09-21T11:53:19.532800+0000","last_became_active":"2024-09-21T11:47:03.378594+0000","last_became_peered":"2024-09-21T11:47:03.378594+0000","last_unstale":"2024-09-21T11:53:19.532800+0000","last_undegraded":"2024-09-21T11:53:19.532800+0000","last_fullsized":"2024-09-21T11:53:19.532800+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.664635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.664635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"327'3644","reported_seq":3683,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.503236+0000","last_change":"2024-09-21T11:52:45.638159+0000","last_active":"2024-09-21T11:53:21.503236+0000","last_peered":"2024-09-21T11:53:21.503236+0000","last_clean":"2024-09-21T11:53:21.503236+0000","last_became_active":"2024-09-21T11:52:45.637825+0000","last_became_peered":"2024-09-21T11:52:45.637825+0000","last_unstale":"2024-09-21T11:53:21.503236+0000","last_undegraded":"2024-09-21T11:53:21.503236+0000","last_fullsized":"2024-09-21T11:53:21.503236+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3644,"log_dups_size":0,"ondisk_log_size":3644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.377774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238813184,"num_objects":3644,"num_object_clones":0,"num_object_copies":10932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3644,"num_write_kb":233216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":665,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535089+0000","last_change":"2024-09-21T11:51:43.533280+0000","last_active":"2024-09-21T11:53:19.535089+0000","last_peered":"2024-09-21T11:53:19.535089+0000","last_clean":"2024-09-21T11:53:19.535089+0000","last_became_active":"2024-09-21T11:51:43.525941+0000","last_became_peered":"2024-09-21T11:51:43.525941+0000","last_unstale":"2024-09-21T11:53:19.535089+0000","last_undegraded":"2024-09-21T11:53:19.535089+0000","last_fullsized":"2024-09-21T11:53:19.535089+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:50:36.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.385902+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:07.872467+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":607,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538748+0000","last_change":"2024-09-21T11:51:14.312046+0000","last_active":"2024-09-21T11:53:19.538748+0000","last_peered":"2024-09-21T11:53:19.538748+0000","last_clean":"2024-09-21T11:53:19.538748+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:53:19.538748+0000","last_undegraded":"2024-09-21T11:53:19.538748+0000","last_fullsized":"2024-09-21T11:53:19.538748+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:14.311953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:14.311953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:29.454276+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":278,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551964+0000","last_change":"2024-09-21T11:52:44.976410+0000","last_active":"2024-09-21T11:53:19.551964+0000","last_peered":"2024-09-21T11:53:19.551964+0000","last_clean":"2024-09-21T11:53:19.551964+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:53:19.551964+0000","last_undegraded":"2024-09-21T11:53:19.551964+0000","last_fullsized":"2024-09-21T11:53:19.551964+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.976363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.976363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.128300+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":581,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535612+0000","last_change":"2024-09-21T11:51:47.944684+0000","last_active":"2024-09-21T11:53:19.535612+0000","last_peered":"2024-09-21T11:53:19.535612+0000","last_clean":"2024-09-21T11:53:19.535612+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:53:19.535612+0000","last_undegraded":"2024-09-21T11:53:19.535612+0000","last_fullsized":"2024-09-21T11:53:19.535612+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:47.944609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.944609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.995897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"327'3731","reported_seq":3770,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.380931+0000","last_change":"2024-09-21T11:52:45.601642+0000","last_active":"2024-09-21T11:53:21.380931+0000","last_peered":"2024-09-21T11:53:21.380931+0000","last_clean":"2024-09-21T11:53:21.380931+0000","last_became_active":"2024-09-21T11:52:45.601350+0000","last_became_peered":"2024-09-21T11:52:45.601350+0000","last_unstale":"2024-09-21T11:53:21.380931+0000","last_undegraded":"2024-09-21T11:53:21.380931+0000","last_fullsized":"2024-09-21T11:53:21.380931+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3731,"log_dups_size":0,"ondisk_log_size":3731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:47.738705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244514816,"num_objects":3731,"num_object_clones":0,"num_object_copies":11193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3731,"num_write_kb":238784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":579,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533148+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:53:19.533148+0000","last_peered":"2024-09-21T11:53:19.533148+0000","last_clean":"2024-09-21T11:53:19.533148+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:53:19.533148+0000","last_undegraded":"2024-09-21T11:53:19.533148+0000","last_fullsized":"2024-09-21T11:53:19.533148+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.147557+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":577,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:22.315543+0000","last_change":"2024-09-21T11:53:22.315543+0000","last_active":"2024-09-21T11:53:22.315543+0000","last_peered":"2024-09-21T11:53:22.315543+0000","last_clean":"2024-09-21T11:53:22.315543+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:53:22.315543+0000","last_undegraded":"2024-09-21T11:53:22.315543+0000","last_fullsized":"2024-09-21T11:53:22.315543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:22.315149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:22.315149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:31.277102+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":754,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535045+0000","last_change":"2024-09-21T11:50:02.975387+0000","last_active":"2024-09-21T11:53:19.535045+0000","last_peered":"2024-09-21T11:53:19.535045+0000","last_clean":"2024-09-21T11:53:19.535045+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:53:19.535045+0000","last_undegraded":"2024-09-21T11:53:19.535045+0000","last_fullsized":"2024-09-21T11:53:19.535045+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:50:02.975313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.975313+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:31.809007+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":398,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.553232+0000","last_change":"2024-09-21T11:52:52.981662+0000","last_active":"2024-09-21T11:53:19.553232+0000","last_peered":"2024-09-21T11:53:19.553232+0000","last_clean":"2024-09-21T11:53:19.553232+0000","last_became_active":"2024-09-21T11:51:43.543072+0000","last_became_peered":"2024-09-21T11:51:43.543072+0000","last_unstale":"2024-09-21T11:53:19.553232+0000","last_undegraded":"2024-09-21T11:53:19.553232+0000","last_fullsized":"2024-09-21T11:53:19.553232+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:52.981586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:52.981586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.522344+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"327'3695","reported_seq":3734,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.479972+0000","last_change":"2024-09-21T11:52:45.635909+0000","last_active":"2024-09-21T11:53:21.479972+0000","last_peered":"2024-09-21T11:53:21.479972+0000","last_clean":"2024-09-21T11:53:21.479972+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:53:21.479972+0000","last_undegraded":"2024-09-21T11:53:21.479972+0000","last_fullsized":"2024-09-21T11:53:21.479972+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3695,"log_dups_size":0,"ondisk_log_size":3695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.650065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242155520,"num_objects":3695,"num_object_clones":0,"num_object_copies":11085,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3695,"num_write_kb":236480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":623,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.551603+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:53:19.551603+0000","last_peered":"2024-09-21T11:53:19.551603+0000","last_clean":"2024-09-21T11:53:19.551603+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:53:19.551603+0000","last_undegraded":"2024-09-21T11:53:19.551603+0000","last_fullsized":"2024-09-21T11:53:19.551603+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:01.621332+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":600,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.536295+0000","last_change":"2024-09-21T11:52:51.850181+0000","last_active":"2024-09-21T11:53:19.536295+0000","last_peered":"2024-09-21T11:53:19.536295+0000","last_clean":"2024-09-21T11:53:19.536295+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:53:19.536295+0000","last_undegraded":"2024-09-21T11:53:19.536295+0000","last_fullsized":"2024-09-21T11:53:19.536295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:51.850120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:51.850120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:52.704982+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":570,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535356+0000","last_change":"2024-09-21T11:48:49.993307+0000","last_active":"2024-09-21T11:53:19.535356+0000","last_peered":"2024-09-21T11:53:19.535356+0000","last_clean":"2024-09-21T11:53:19.535356+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:53:19.535356+0000","last_undegraded":"2024-09-21T11:53:19.535356+0000","last_fullsized":"2024-09-21T11:53:19.535356+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:49.993230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:49.993230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:23.798946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":557,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.550575+0000","last_change":"2024-09-21T11:51:50.032271+0000","last_active":"2024-09-21T11:53:19.550575+0000","last_peered":"2024-09-21T11:53:19.550575+0000","last_clean":"2024-09-21T11:53:19.550575+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:53:19.550575+0000","last_undegraded":"2024-09-21T11:53:19.550575+0000","last_fullsized":"2024-09-21T11:53:19.550575+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:50.032193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:50.032193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"327'3799","reported_seq":3838,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:22.110358+0000","last_change":"2024-09-21T11:52:45.636586+0000","last_active":"2024-09-21T11:53:22.110358+0000","last_peered":"2024-09-21T11:53:22.110358+0000","last_clean":"2024-09-21T11:53:22.110358+0000","last_became_active":"2024-09-21T11:52:45.636291+0000","last_became_peered":"2024-09-21T11:52:45.636291+0000","last_unstale":"2024-09-21T11:53:22.110358+0000","last_undegraded":"2024-09-21T11:53:22.110358+0000","last_fullsized":"2024-09-21T11:53:22.110358+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3799,"log_dups_size":0,"ondisk_log_size":3799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.940714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248971264,"num_objects":3799,"num_object_clones":0,"num_object_copies":11397,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3799,"num_write_kb":243136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":698,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538541+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:53:19.538541+0000","last_peered":"2024-09-21T11:53:19.538541+0000","last_clean":"2024-09-21T11:53:19.538541+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:53:19.538541+0000","last_undegraded":"2024-09-21T11:53:19.538541+0000","last_fullsized":"2024-09-21T11:53:19.538541+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.597026+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":584,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.543089+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:53:19.543089+0000","last_peered":"2024-09-21T11:53:19.543089+0000","last_clean":"2024-09-21T11:53:19.543089+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:53:19.543089+0000","last_undegraded":"2024-09-21T11:53:19.543089+0000","last_fullsized":"2024-09-21T11:53:19.543089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:30.996685+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":593,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532151+0000","last_change":"2024-09-21T11:51:48.941202+0000","last_active":"2024-09-21T11:53:19.532151+0000","last_peered":"2024-09-21T11:53:19.532151+0000","last_clean":"2024-09-21T11:53:19.532151+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:53:19.532151+0000","last_undegraded":"2024-09-21T11:53:19.532151+0000","last_fullsized":"2024-09-21T11:53:19.532151+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:48.941128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:48.941128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.021684+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":581,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538005+0000","last_change":"2024-09-21T11:52:12.100354+0000","last_active":"2024-09-21T11:53:19.538005+0000","last_peered":"2024-09-21T11:53:19.538005+0000","last_clean":"2024-09-21T11:53:19.538005+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:53:19.538005+0000","last_undegraded":"2024-09-21T11:53:19.538005+0000","last_fullsized":"2024-09-21T11:53:19.538005+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:36.518478+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"327'3688","reported_seq":3727,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.053616+0000","last_change":"2024-09-21T11:52:45.636994+0000","last_active":"2024-09-21T11:53:21.053616+0000","last_peered":"2024-09-21T11:53:21.053616+0000","last_clean":"2024-09-21T11:53:21.053616+0000","last_became_active":"2024-09-21T11:52:45.636672+0000","last_became_peered":"2024-09-21T11:52:45.636672+0000","last_unstale":"2024-09-21T11:53:21.053616+0000","last_undegraded":"2024-09-21T11:53:21.053616+0000","last_fullsized":"2024-09-21T11:53:21.053616+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3688,"log_dups_size":0,"ondisk_log_size":3688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.462991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241696768,"num_objects":3688,"num_object_clones":0,"num_object_copies":11064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3688,"num_write_kb":236032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":619,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.545522+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:53:19.545522+0000","last_peered":"2024-09-21T11:53:19.545522+0000","last_clean":"2024-09-21T11:53:19.545522+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:53:19.545522+0000","last_undegraded":"2024-09-21T11:53:19.545522+0000","last_fullsized":"2024-09-21T11:53:19.545522+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.016882+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":614,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.542539+0000","last_change":"2024-09-21T11:49:08.611218+0000","last_active":"2024-09-21T11:53:19.542539+0000","last_peered":"2024-09-21T11:53:19.542539+0000","last_clean":"2024-09-21T11:53:19.542539+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:53:19.542539+0000","last_undegraded":"2024-09-21T11:53:19.542539+0000","last_fullsized":"2024-09-21T11:53:19.542539+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.611159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.611159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:56.046273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":600,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.531714+0000","last_change":"2024-09-21T11:49:44.985594+0000","last_active":"2024-09-21T11:53:19.531714+0000","last_peered":"2024-09-21T11:53:19.531714+0000","last_clean":"2024-09-21T11:53:19.531714+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:53:19.531714+0000","last_undegraded":"2024-09-21T11:53:19.531714+0000","last_fullsized":"2024-09-21T11:53:19.531714+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:44.985234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:44.985234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.909250+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":587,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532548+0000","last_change":"2024-09-21T11:51:09.660956+0000","last_active":"2024-09-21T11:53:19.532548+0000","last_peered":"2024-09-21T11:53:19.532548+0000","last_clean":"2024-09-21T11:53:19.532548+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:53:19.532548+0000","last_undegraded":"2024-09-21T11:53:19.532548+0000","last_fullsized":"2024-09-21T11:53:19.532548+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:09.660880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.660880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"327'3603","reported_seq":3642,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:21.283026+0000","last_change":"2024-09-21T11:52:45.603459+0000","last_active":"2024-09-21T11:53:21.283026+0000","last_peered":"2024-09-21T11:53:21.283026+0000","last_clean":"2024-09-21T11:53:21.283026+0000","last_became_active":"2024-09-21T11:52:45.603101+0000","last_became_peered":"2024-09-21T11:52:45.603101+0000","last_unstale":"2024-09-21T11:53:21.283026+0000","last_undegraded":"2024-09-21T11:53:21.283026+0000","last_fullsized":"2024-09-21T11:53:21.283026+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":3603,"log_dups_size":0,"ondisk_log_size":3603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:45.425281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236126208,"num_objects":3603,"num_object_clones":0,"num_object_copies":10809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3603,"num_write_kb":230592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":616,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.534166+0000","last_change":"2024-09-21T11:51:19.638791+0000","last_active":"2024-09-21T11:53:19.534166+0000","last_peered":"2024-09-21T11:53:19.534166+0000","last_clean":"2024-09-21T11:53:19.534166+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:53:19.534166+0000","last_undegraded":"2024-09-21T11:53:19.534166+0000","last_fullsized":"2024-09-21T11:53:19.534166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:51:19.638734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:19.638734+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":612,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.542827+0000","last_change":"2024-09-21T11:52:43.086438+0000","last_active":"2024-09-21T11:53:19.542827+0000","last_peered":"2024-09-21T11:53:19.542827+0000","last_clean":"2024-09-21T11:53:19.542827+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:53:19.542827+0000","last_undegraded":"2024-09-21T11:53:19.542827+0000","last_fullsized":"2024-09-21T11:53:19.542827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:52:43.086376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:43.086376+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.446134+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":746,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537867+0000","last_change":"2024-09-21T11:52:54.080234+0000","last_active":"2024-09-21T11:53:19.537867+0000","last_peered":"2024-09-21T11:53:19.537867+0000","last_clean":"2024-09-21T11:53:19.537867+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:53:19.537867+0000","last_undegraded":"2024-09-21T11:53:19.537867+0000","last_fullsized":"2024-09-21T11:53:19.537867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:52:54.080076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:54.080076+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:15.355476+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":565,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.534960+0000","last_change":"2024-09-21T11:49:56.669610+0000","last_active":"2024-09-21T11:53:19.534960+0000","last_peered":"2024-09-21T11:53:19.534960+0000","last_clean":"2024-09-21T11:53:19.534960+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:53:19.534960+0000","last_undegraded":"2024-09-21T11:53:19.534960+0000","last_fullsized":"2024-09-21T11:53:19.534960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:56.669547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:56.669547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:27.819090+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":600,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.544870+0000","last_change":"2024-09-21T11:52:42.454187+0000","last_active":"2024-09-21T11:53:19.544870+0000","last_peered":"2024-09-21T11:53:19.544870+0000","last_clean":"2024-09-21T11:53:19.544870+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:53:19.544870+0000","last_undegraded":"2024-09-21T11:53:19.544870+0000","last_fullsized":"2024-09-21T11:53:19.544870+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:42.454138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:42.454138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:56.161790+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":613,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.534032+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:53:19.534032+0000","last_peered":"2024-09-21T11:53:19.534032+0000","last_clean":"2024-09-21T11:53:19.534032+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:53:19.534032+0000","last_undegraded":"2024-09-21T11:53:19.534032+0000","last_fullsized":"2024-09-21T11:53:19.534032+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:01.305455+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":598,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535798+0000","last_change":"2024-09-21T11:50:58.694445+0000","last_active":"2024-09-21T11:53:19.535798+0000","last_peered":"2024-09-21T11:53:19.535798+0000","last_clean":"2024-09-21T11:53:19.535798+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:53:19.535798+0000","last_undegraded":"2024-09-21T11:53:19.535798+0000","last_fullsized":"2024-09-21T11:53:19.535798+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:58.694382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:58.694382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:20.794973+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":580,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.541761+0000","last_change":"2024-09-21T11:49:43.464860+0000","last_active":"2024-09-21T11:53:19.541761+0000","last_peered":"2024-09-21T11:53:19.541761+0000","last_clean":"2024-09-21T11:53:19.541761+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:53:19.541761+0000","last_undegraded":"2024-09-21T11:53:19.541761+0000","last_fullsized":"2024-09-21T11:53:19.541761+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:43.464792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:43.464792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:24.925506+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":614,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554970+0000","last_change":"2024-09-21T11:51:09.955823+0000","last_active":"2024-09-21T11:53:19.554970+0000","last_peered":"2024-09-21T11:53:19.554970+0000","last_clean":"2024-09-21T11:53:19.554970+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:53:19.554970+0000","last_undegraded":"2024-09-21T11:53:19.554970+0000","last_fullsized":"2024-09-21T11:53:19.554970+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:09.955744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.955744+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:22.796315+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":584,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538103+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:53:19.538103+0000","last_peered":"2024-09-21T11:53:19.538103+0000","last_clean":"2024-09-21T11:53:19.538103+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:53:19.538103+0000","last_undegraded":"2024-09-21T11:53:19.538103+0000","last_fullsized":"2024-09-21T11:53:19.538103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.143512+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":599,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533879+0000","last_change":"2024-09-21T11:52:41.245695+0000","last_active":"2024-09-21T11:53:19.533879+0000","last_peered":"2024-09-21T11:53:19.533879+0000","last_clean":"2024-09-21T11:53:19.533879+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:53:19.533879+0000","last_undegraded":"2024-09-21T11:53:19.533879+0000","last_fullsized":"2024-09-21T11:53:19.533879+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:41.245632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:41.245632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.732253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":581,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535536+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:53:19.535536+0000","last_peered":"2024-09-21T11:53:19.535536+0000","last_clean":"2024-09-21T11:53:19.535536+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:53:19.535536+0000","last_undegraded":"2024-09-21T11:53:19.535536+0000","last_fullsized":"2024-09-21T11:53:19.535536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.846816+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":597,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537351+0000","last_change":"2024-09-21T11:51:49.643640+0000","last_active":"2024-09-21T11:53:19.537351+0000","last_peered":"2024-09-21T11:53:19.537351+0000","last_clean":"2024-09-21T11:53:19.537351+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:53:19.537351+0000","last_undegraded":"2024-09-21T11:53:19.537351+0000","last_fullsized":"2024-09-21T11:53:19.537351+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:49.643424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:49.643424+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":600,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.560342+0000","last_change":"2024-09-21T11:52:45.688676+0000","last_active":"2024-09-21T11:53:19.560342+0000","last_peered":"2024-09-21T11:53:19.560342+0000","last_clean":"2024-09-21T11:53:19.560342+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:53:19.560342+0000","last_undegraded":"2024-09-21T11:53:19.560342+0000","last_fullsized":"2024-09-21T11:53:19.560342+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:45.688617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:45.688617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.734406+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":730,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533051+0000","last_change":"2024-09-21T11:51:08.629172+0000","last_active":"2024-09-21T11:53:19.533051+0000","last_peered":"2024-09-21T11:53:19.533051+0000","last_clean":"2024-09-21T11:53:19.533051+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:53:19.533051+0000","last_undegraded":"2024-09-21T11:53:19.533051+0000","last_fullsized":"2024-09-21T11:53:19.533051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:08.628986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.628986+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":79,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535522+0000","last_change":"2024-09-21T11:51:43.531318+0000","last_active":"2024-09-21T11:53:19.535522+0000","last_peered":"2024-09-21T11:53:19.535522+0000","last_clean":"2024-09-21T11:53:19.535522+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:53:19.535522+0000","last_undegraded":"2024-09-21T11:53:19.535522+0000","last_fullsized":"2024-09-21T11:53:19.535522+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.826834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.826834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":633,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.533239+0000","last_change":"2024-09-21T11:50:37.433622+0000","last_active":"2024-09-21T11:53:19.533239+0000","last_peered":"2024-09-21T11:53:19.533239+0000","last_clean":"2024-09-21T11:53:19.533239+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:53:19.533239+0000","last_undegraded":"2024-09-21T11:53:19.533239+0000","last_fullsized":"2024-09-21T11:53:19.533239+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:50:37.432212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.432212+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:21.303187+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":576,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.538286+0000","last_change":"2024-09-21T11:49:58.400078+0000","last_active":"2024-09-21T11:53:19.538286+0000","last_peered":"2024-09-21T11:53:19.538286+0000","last_clean":"2024-09-21T11:53:19.538286+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:53:19.538286+0000","last_undegraded":"2024-09-21T11:53:19.538286+0000","last_fullsized":"2024-09-21T11:53:19.538286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.400013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.400013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:28.032196+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":768,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537211+0000","last_change":"2024-09-21T11:50:30.703282+0000","last_active":"2024-09-21T11:53:19.537211+0000","last_peered":"2024-09-21T11:53:19.537211+0000","last_clean":"2024-09-21T11:53:19.537211+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:53:19.537211+0000","last_undegraded":"2024-09-21T11:53:19.537211+0000","last_fullsized":"2024-09-21T11:53:19.537211+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:50:30.703227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:30.703227+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":580,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554374+0000","last_change":"2024-09-21T11:50:02.782565+0000","last_active":"2024-09-21T11:53:19.554374+0000","last_peered":"2024-09-21T11:53:19.554374+0000","last_clean":"2024-09-21T11:53:19.554374+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:53:19.554374+0000","last_undegraded":"2024-09-21T11:53:19.554374+0000","last_fullsized":"2024-09-21T11:53:19.554374+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.782475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.782475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:32.780132+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":629,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.539781+0000","last_change":"2024-09-21T11:51:43.533694+0000","last_active":"2024-09-21T11:53:19.539781+0000","last_peered":"2024-09-21T11:53:19.539781+0000","last_clean":"2024-09-21T11:53:19.539781+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:53:19.539781+0000","last_undegraded":"2024-09-21T11:53:19.539781+0000","last_fullsized":"2024-09-21T11:53:19.539781+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:04.610554+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":87,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.542163+0000","last_change":"2024-09-21T11:51:52.159555+0000","last_active":"2024-09-21T11:53:19.542163+0000","last_peered":"2024-09-21T11:53:19.542163+0000","last_clean":"2024-09-21T11:53:19.542163+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:53:19.542163+0000","last_undegraded":"2024-09-21T11:53:19.542163+0000","last_fullsized":"2024-09-21T11:53:19.542163+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:52.159433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:52.159433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.229678+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":278,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.546355+0000","last_change":"2024-09-21T11:51:10.143792+0000","last_active":"2024-09-21T11:53:19.546355+0000","last_peered":"2024-09-21T11:53:19.546355+0000","last_clean":"2024-09-21T11:53:19.546355+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:53:19.546355+0000","last_undegraded":"2024-09-21T11:53:19.546355+0000","last_fullsized":"2024-09-21T11:53:19.546355+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.141936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.141936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:34.113373+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":565,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.531437+0000","last_change":"2024-09-21T11:52:48.256878+0000","last_active":"2024-09-21T11:53:19.531437+0000","last_peered":"2024-09-21T11:53:19.531437+0000","last_clean":"2024-09-21T11:53:19.531437+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:53:19.531437+0000","last_undegraded":"2024-09-21T11:53:19.531437+0000","last_fullsized":"2024-09-21T11:53:19.531437+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:48.256830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:48.256830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:00.348262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":289,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.546602+0000","last_change":"2024-09-21T11:51:15.200506+0000","last_active":"2024-09-21T11:53:19.546602+0000","last_peered":"2024-09-21T11:53:19.546602+0000","last_clean":"2024-09-21T11:53:19.546602+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:53:19.546602+0000","last_undegraded":"2024-09-21T11:53:19.546602+0000","last_fullsized":"2024-09-21T11:53:19.546602+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:15.200430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.200430+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:35.130532+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":584,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.542944+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:53:19.542944+0000","last_peered":"2024-09-21T11:53:19.542944+0000","last_clean":"2024-09-21T11:53:19.542944+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:53:19.542944+0000","last_undegraded":"2024-09-21T11:53:19.542944+0000","last_fullsized":"2024-09-21T11:53:19.542944+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.179348+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":591,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532670+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:53:19.532670+0000","last_peered":"2024-09-21T11:53:19.532670+0000","last_clean":"2024-09-21T11:53:19.532670+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:53:19.532670+0000","last_undegraded":"2024-09-21T11:53:19.532670+0000","last_fullsized":"2024-09-21T11:53:19.532670+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.222459+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":603,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.550708+0000","last_change":"2024-09-21T11:51:16.962933+0000","last_active":"2024-09-21T11:53:19.550708+0000","last_peered":"2024-09-21T11:53:19.550708+0000","last_clean":"2024-09-21T11:53:19.550708+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:53:19.550708+0000","last_undegraded":"2024-09-21T11:53:19.550708+0000","last_fullsized":"2024-09-21T11:53:19.550708+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:51:16.962860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:16.962860+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.599324+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":612,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532002+0000","last_change":"2024-09-21T11:51:19.708885+0000","last_active":"2024-09-21T11:53:19.532002+0000","last_peered":"2024-09-21T11:53:19.532002+0000","last_clean":"2024-09-21T11:53:19.532002+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:53:19.532002+0000","last_undegraded":"2024-09-21T11:53:19.532002+0000","last_fullsized":"2024-09-21T11:53:19.532002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:51:19.708641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:19.708641+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":598,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.535878+0000","last_change":"2024-09-21T11:51:01.690172+0000","last_active":"2024-09-21T11:53:19.535878+0000","last_peered":"2024-09-21T11:53:19.535878+0000","last_clean":"2024-09-21T11:53:19.535878+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:53:19.535878+0000","last_undegraded":"2024-09-21T11:53:19.535878+0000","last_fullsized":"2024-09-21T11:53:19.535878+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.686885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.686885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:23.798235+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":568,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537403+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:53:19.537403+0000","last_peered":"2024-09-21T11:53:19.537403+0000","last_clean":"2024-09-21T11:53:19.537403+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:53:19.537403+0000","last_undegraded":"2024-09-21T11:53:19.537403+0000","last_fullsized":"2024-09-21T11:53:19.537403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:00.087872+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":573,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532952+0000","last_change":"2024-09-21T11:51:05.711964+0000","last_active":"2024-09-21T11:53:19.532952+0000","last_peered":"2024-09-21T11:53:19.532952+0000","last_clean":"2024-09-21T11:53:19.532952+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:53:19.532952+0000","last_undegraded":"2024-09-21T11:53:19.532952+0000","last_fullsized":"2024-09-21T11:53:19.532952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:05.711768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.711768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.204609+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":624,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532091+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:53:19.532091+0000","last_peered":"2024-09-21T11:53:19.532091+0000","last_clean":"2024-09-21T11:53:19.532091+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:53:19.532091+0000","last_undegraded":"2024-09-21T11:53:19.532091+0000","last_fullsized":"2024-09-21T11:53:19.532091+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:56.189942+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":576,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554160+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:53:19.554160+0000","last_peered":"2024-09-21T11:53:19.554160+0000","last_clean":"2024-09-21T11:53:19.554160+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:53:19.554160+0000","last_undegraded":"2024-09-21T11:53:19.554160+0000","last_fullsized":"2024-09-21T11:53:19.554160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:35.701180+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":576,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537152+0000","last_change":"2024-09-21T11:50:38.405183+0000","last_active":"2024-09-21T11:53:19.537152+0000","last_peered":"2024-09-21T11:53:19.537152+0000","last_clean":"2024-09-21T11:53:19.537152+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:53:19.537152+0000","last_undegraded":"2024-09-21T11:53:19.537152+0000","last_fullsized":"2024-09-21T11:53:19.537152+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:38.405118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.405118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:08.616042+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":572,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.537611+0000","last_change":"2024-09-21T11:52:45.065651+0000","last_active":"2024-09-21T11:53:19.537611+0000","last_peered":"2024-09-21T11:53:19.537611+0000","last_clean":"2024-09-21T11:53:19.537611+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:53:19.537611+0000","last_undegraded":"2024-09-21T11:53:19.537611+0000","last_fullsized":"2024-09-21T11:53:19.537611+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:45.065594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:45.065594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:50.236288+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":286,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.552606+0000","last_change":"2024-09-21T11:52:12.348687+0000","last_active":"2024-09-21T11:53:19.552606+0000","last_peered":"2024-09-21T11:53:19.552606+0000","last_clean":"2024-09-21T11:53:19.552606+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:53:19.552606+0000","last_undegraded":"2024-09-21T11:53:19.552606+0000","last_fullsized":"2024-09-21T11:53:19.552606+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:18.010458+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":599,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.532338+0000","last_change":"2024-09-21T11:51:04.656939+0000","last_active":"2024-09-21T11:53:19.532338+0000","last_peered":"2024-09-21T11:53:19.532338+0000","last_clean":"2024-09-21T11:53:19.532338+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:53:19.532338+0000","last_undegraded":"2024-09-21T11:53:19.532338+0000","last_fullsized":"2024-09-21T11:53:19.532338+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:04.656870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:04.656870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:04.247547+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":674,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:20.555366+0000","last_change":"2024-09-21T11:53:20.555366+0000","last_active":"2024-09-21T11:53:20.555366+0000","last_peered":"2024-09-21T11:53:20.555366+0000","last_clean":"2024-09-21T11:53:20.555366+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:53:20.555366+0000","last_undegraded":"2024-09-21T11:53:20.555366+0000","last_fullsized":"2024-09-21T11:53:20.555366+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:20.554779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.554779+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:25.003905+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":585,"reported_epoch":327,"state":"active+clean","last_fresh":"2024-09-21T11:53:19.554482+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:53:19.554482+0000","last_peered":"2024-09-21T11:53:19.554482+0000","last_clean":"2024-09-21T11:53:19.554482+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:53:19.554482+0000","last_undegraded":"2024-09-21T11:53:19.554482+0000","last_fullsized":"2024-09-21T11:53:19.554482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:31.809614+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":3860004864,"num_objects":58899,"num_object_clones":0,"num_object_copies":176697,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58899,"num_write_kb":3769536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11639521280,"data_stored":11639521280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58899,"ondisk_log_size":58899,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1113,"num_read_kb":870,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":792,"num_read_kb":787,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234149,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1464596,"kb_used_data":914176,"kb_used_omap":1,"kb_used_meta":550398,"kb_avail":92280556,"statfs":{"total":95995035648,"available":94495289344,"internally_reserved":0,"allocated":936116224,"data_stored":935649692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":563608010},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463082,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2016328,"kb_used_data":1620020,"kb_used_omap":1,"kb_used_meta":396286,"kb_avail":91728824,"statfs":{"total":95995035648,"available":93930315776,"internally_reserved":0,"allocated":1658900480,"data_stored":1658446580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":405797318},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724719,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1686356,"kb_used_data":1163712,"kb_used_omap":1,"kb_used_meta":522622,"kb_avail":92058796,"statfs":{"total":95995035648,"available":94268207104,"internally_reserved":0,"allocated":1191641088,"data_stored":1191173364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":535165382},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920949,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2237852,"kb_used_data":1863752,"kb_used_omap":1,"kb_used_meta":374078,"kb_avail":91507300,"statfs":{"total":95995035648,"available":93703475200,"internally_reserved":0,"allocated":1908482048,"data_stored":1908007293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":383056325},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117179,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1793928,"kb_used_data":1212984,"kb_used_omap":1,"kb_used_meta":580926,"kb_avail":91951224,"statfs":{"total":95995035648,"available":94158053376,"internally_reserved":0,"allocated":1242095616,"data_stored":1241636070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":594868679},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411521,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1955340,"kb_used_data":1410492,"kb_used_omap":1,"kb_used_meta":544830,"kb_avail":91789812,"statfs":{"total":95995035648,"available":93992767488,"internally_reserved":0,"allocated":1444343808,"data_stored":1443880877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":557906373},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1226604,"kb_used_data":708124,"kb_used_omap":1,"kb_used_meta":518462,"kb_avail":92518548,"statfs":{"total":95995035648,"available":94738993152,"internally_reserved":0,"allocated":725118976,"data_stored":724623827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":530905542},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738574,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2833124,"kb_used_data":2350736,"kb_used_omap":1,"kb_used_meta":482366,"kb_avail":90912028,"statfs":{"total":95995035648,"available":93093916672,"internally_reserved":0,"allocated":2407153664,"data_stored":2406671866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":493943237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2445541376,"data_stored":2445541376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":728825856,"data_stored":728825856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1464401920,"data_stored":1464401920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1242431488,"data_stored":1242431488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1910439936,"data_stored":1910439936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1203896320,"data_stored":1203896320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1682636800,"data_stored":1682636800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":961347584,"data_stored":961347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:53:27.238 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-21T11:53:27.238 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:53:27.238 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T11:53:27.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-21T11:53:27.257 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:27.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:27.314 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 35 16 66166 66150 118.104 62.5625 0.00936579 0.00846258 2024-09-21T11:53:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:27.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:27.350 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:27.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:27.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:27.501 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:27.502 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:27.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:27.641 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:27.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:27.642 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:27.743 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:27.764 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:27.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:27.765 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:27.832 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[36454]: from='client.25964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:53:27.832 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[36454]: pgmap v647: 148 pgs: 148 active+clean; 3.6 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 786 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:53:27.832 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[36454]: 6.15 scrub starts 2024-09-21T11:53:27.832 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[36454]: 6.15 scrub ok 2024-09-21T11:53:27.832 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[36454]: 5.a scrub starts 2024-09-21T11:53:27.832 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[36454]: 5.a scrub ok 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[36454]: 3.4 scrub starts 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[36454]: 3.4 scrub ok 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[36454]: osdmap e330: 8 total, 8 up, 8 in 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[29710]: from='client.25964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[29710]: pgmap v647: 148 pgs: 148 active+clean; 3.6 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 786 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[29710]: 6.15 scrub starts 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[29710]: 6.15 scrub ok 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[29710]: 5.a scrub starts 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[29710]: 5.a scrub ok 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[29710]: 3.4 scrub starts 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[29710]: 3.4 scrub ok 2024-09-21T11:53:27.833 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:27 smithi043 ceph-mon[29710]: osdmap e330: 8 total, 8 up, 8 in 2024-09-21T11:53:27.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:27.893 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:53:27.893 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:53:27.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:27.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:27.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:53:27.930 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:53:27.955 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:27.966 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:53:27.967 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:53:28.006 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:53:28.006 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:53:28.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:27 smithi175 ceph-mon[32613]: from='client.25964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:53:28.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:27 smithi175 ceph-mon[32613]: pgmap v647: 148 pgs: 148 active+clean; 3.6 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 786 B/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:53:28.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:27 smithi175 ceph-mon[32613]: 6.15 scrub starts 2024-09-21T11:53:28.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:27 smithi175 ceph-mon[32613]: 6.15 scrub ok 2024-09-21T11:53:28.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:27 smithi175 ceph-mon[32613]: 5.a scrub starts 2024-09-21T11:53:28.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:27 smithi175 ceph-mon[32613]: 5.a scrub ok 2024-09-21T11:53:28.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:27 smithi175 ceph-mon[32613]: 3.4 scrub starts 2024-09-21T11:53:28.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:27 smithi175 ceph-mon[32613]: 3.4 scrub ok 2024-09-21T11:53:28.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:27 smithi175 ceph-mon[32613]: osdmap e330: 8 total, 8 up, 8 in 2024-09-21T11:53:28.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:28.027 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:53:28.027 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:53:28.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:28.036 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:28.036 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:28.045 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:53:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:53:28.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:28.159 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:53:28.159 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:53:28.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:28.187 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:28.188 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:28.197 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:53:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:53:28.288 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:28.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 36 16 67144 67128 116.521 61.125 0.0100306 0.00857684 2024-09-21T11:53:28.322 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:28.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:28.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:28.424 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:28.445 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:28.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:28.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:28.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:53:28.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:28.577 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:28.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:28.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:28.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:28.712 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:28.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:28.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:28.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T11:53:28.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T11:53:28.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[36454]: 4.1e scrub starts 2024-09-21T11:53:28.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[36454]: 4.1e scrub ok 2024-09-21T11:53:28.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:53:28.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:53:28.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/583181881' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:28.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[36454]: 6.2 scrub starts 2024-09-21T11:53:28.795 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[36454]: 6.2 scrub ok 2024-09-21T11:53:28.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T11:53:28.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T11:53:28.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[29710]: 4.1e scrub starts 2024-09-21T11:53:28.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[29710]: 4.1e scrub ok 2024-09-21T11:53:28.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:53:28.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:53:28.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/583181881' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:28.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[29710]: 6.2 scrub starts 2024-09-21T11:53:28.795 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:28 smithi043 ceph-mon[29710]: 6.2 scrub ok 2024-09-21T11:53:28.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:28.853 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:28.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:28.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:28.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:28.995 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:28.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:28.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:28 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T11:53:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:28 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T11:53:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:28 smithi175 ceph-mon[32613]: 4.1e scrub starts 2024-09-21T11:53:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:28 smithi175 ceph-mon[32613]: 4.1e scrub ok 2024-09-21T11:53:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:28 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:53:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:28 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:53:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:28 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/583181881' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:28 smithi175 ceph-mon[32613]: 6.2 scrub starts 2024-09-21T11:53:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:28 smithi175 ceph-mon[32613]: 6.2 scrub ok 2024-09-21T11:53:29.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:29.121 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:29.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:29.122 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:29.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:29.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:29.249 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:29.266 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:29.276 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:29.277 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:29.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 37 16 67892 67876 114.636 46.75 0.0028764 0.00866156 2024-09-21T11:53:29.350 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:29.387 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:29.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:29.388 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:29.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:29.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:29.529 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:29.535 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:53:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:53:29.536 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 331, 2024-09-21T11:53:29.536 INFO:teuthology.orchestra.run.smithi043.stdout: "fsid": "4f2c5252-780d-11ef-baf6-efdc52797490", 2024-09-21T11:53:29.536 INFO:teuthology.orchestra.run.smithi043.stdout: "created": "2024-09-21T11:34:07.249195+0000", 2024-09-21T11:53:29.536 INFO:teuthology.orchestra.run.smithi043.stdout: "modified": "2024-09-21T11:53:28.572972+0000", 2024-09-21T11:53:29.536 INFO:teuthology.orchestra.run.smithi043.stdout: "last_up_change": "2024-09-21T11:39:49.206441+0000", 2024-09-21T11:53:29.536 INFO:teuthology.orchestra.run.smithi043.stdout: "last_in_change": "2024-09-21T11:47:02.337266+0000", 2024-09-21T11:53:29.536 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T11:53:29.536 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_num": 5799936, 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_set": [ 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "pglog_hardlimit", 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "purged_snapdirs", 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_deletes", 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "sortbitwise" 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_version": 18, 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_snapshot": "", 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_max": 10, 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd": 8, 2024-09-21T11:53:29.537 INFO:teuthology.orchestra.run.smithi043.stdout: "require_min_compat_client": "luminous", 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "min_compat_client": "luminous", 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "require_osd_release": "squid", 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_crimson": false, 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "pools": [ 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 1, 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".mgr", 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:37:26.023430+0000", 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:29.538 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 1, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 1, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 1, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 1, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 1, 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:29.539 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "253", 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:29.540 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:29.541 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:29.542 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_max": 32, 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_min": 1 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": {} 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 6.9800000190734863, 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 6.9800000190734863, 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.43000000715255737, 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 3, 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 3, 2024-09-21T11:53:29.543 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 2, 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "datapool", 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:58.262982+0000", 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 8193, 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:29.544 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 3, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 3, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 3, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 3, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 3, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:29.545 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "61", 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 2, 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 61, 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:29.546 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:29.547 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:29.548 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd": {} 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.3299999237060547, 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.3299999237060547, 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1, 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 3, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".rgw.root", 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:59.229530+0000", 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:29.550 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:29.551 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:29.552 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:29.552 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:29.552 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:29.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.552 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "60", 2024-09-21T11:53:29.552 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:29.552 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:29.552 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:29.552 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:29.553 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:29.554 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.5399999618530273, 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.5399999618530273, 2024-09-21T11:53:29.555 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.75, 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.75, 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 4, 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.log", 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:00.380059+0000", 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:29.556 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:53:29.557 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "62", 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:29.558 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:29.559 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:29.560 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.190000057220459, 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.190000057220459, 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2.5, 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2.5, 2024-09-21T11:53:29.561 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 5, 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.control", 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:02.330041+0000", 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:29.562 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:29.563 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "64", 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:29.564 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:29.565 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:53:29.566 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.3200000524520874, 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.3200000524520874, 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.5, 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.5, 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 6, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.meta", 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:04.395413+0000", 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:29.568 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:29.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "66", 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:29.570 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.571 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_bias": 4 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:29.572 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.5399999618530273, 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.5399999618530273, 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.75, 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.75, 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 10, 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "unique_pool_3", 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:52:44.466175+0000", 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:29.574 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 16, 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 16, 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 16, 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 16, 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 16, 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:29.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "317", 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:29.576 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:29.577 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "rados": {} 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.578 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 3.0699999332427979, 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 3.0699999332427979, 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 3.5, 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 3.5, 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.579 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4", 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 8, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6802", 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.581 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6804", 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6803", 2024-09-21T11:53:29.582 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:53:29.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.583 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6801/817279193", 2024-09-21T11:53:29.583 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6802/817279193", 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6804/817279193", 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6803/817279193", 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607", 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:29.584 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 13, 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6805", 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.585 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6806", 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.586 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6808", 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6807", 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.587 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6805/263756171", 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6806/263756171", 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6808/263756171", 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6807/263756171", 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce", 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:29.588 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 18, 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6809", 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6810", 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.590 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6812", 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6811", 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6809/687577863", 2024-09-21T11:53:29.591 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6810/687577863", 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6812/687577863", 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6811/687577863", 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d", 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:29.592 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 27, 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 293, 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6813", 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6814", 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.594 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6816", 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6815", 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.595 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6813/129493158", 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6814/129493158", 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6816/129493158", 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6815/129493158", 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c", 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:29.596 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 0, 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 33, 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 235, 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6800", 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6801", 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6803", 2024-09-21T11:53:29.598 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6802", 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6800/1323992640", 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6801/1323992640", 2024-09-21T11:53:29.599 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6803/1323992640", 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6802/1323992640", 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "4d647321-71a1-412b-bbf5-921441923a65", 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:29.600 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 39, 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6804", 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.601 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6805", 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6807", 2024-09-21T11:53:29.602 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6806", 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6804/1163208458", 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6805/1163208458", 2024-09-21T11:53:29.603 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6807/1163208458", 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6806/1163208458", 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354", 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 0.9737701416015625, 2024-09-21T11:53:29.604 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 47, 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6808", 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.605 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6809", 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6811", 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:53:29.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6810", 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6808/932395625", 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6809/932395625", 2024-09-21T11:53:29.607 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6811/932395625", 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6810/932395625", 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22", 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:29.608 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 54, 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6812", 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:29.609 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6813", 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6815", 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:53:29.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6814", 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6812/3676713275", 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6813/3676713275", 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6815/3676713275", 2024-09-21T11:53:29.611 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6814/3676713275", 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_xinfo": [ 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:29.612 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:53:16.468321+0000", 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:52:08.907218+0000", 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:53:27.671975+0000", 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:29.614 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:53:06.915356+0000", 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:52:28.472180+0000", 2024-09-21T11:53:29.615 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:52:58.679157+0000", 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:29.616 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:52:46.310580+0000", 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:29.617 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:52:28.003963+0000", 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap": [ 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.2", 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: 2, 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: 4, 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: 5 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.c", 2024-09-21T11:53:29.618 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: 3, 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: 7, 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: 1 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.1c", 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: 7, 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: 2 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:29.619 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_items": [ 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.4", 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.7", 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:53:29.620 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.d", 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.621 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.13", 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.16", 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.622 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_primaries": [], 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_temp": [], 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_temp": [], 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklist": { 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2419845767": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3249046048": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2518306973": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3115176321": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/2583179759": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/4225358170": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/347827280": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/1513342881": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:53:29.623 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/1113684867": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3345448895": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2454875216": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/911630798": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/983028036": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/4145025121": "2024-09-22T11:40:29.670859+0000" 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "range_blocklist": {}, 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profiles": { 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "default": { 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-failure-domain": "osd", 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "k": "2", 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "m": "1", 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "plugin": "jerasure", 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: "technique": "reed_sol_van" 2024-09-21T11:53:29.624 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps_queue": [], 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "new_removed_snaps": [], 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "new_purged_snaps": [], 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_node_flags": {}, 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "device_class_flags": {}, 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode": { 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_enabled": false, 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_bucket_count": 0, 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "degraded_stretch_mode": 0, 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "recovering_stretch_mode": 0, 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_bucket": 0 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:29.625 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:53:29.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:29.665 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:29.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:29.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:29.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[29710]: 6.a scrub starts 2024-09-21T11:53:29.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[29710]: 6.a scrub ok 2024-09-21T11:53:29.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[29710]: pgmap v649: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-21T11:53:29.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[29710]: osdmap e331: 8 total, 8 up, 8 in 2024-09-21T11:53:29.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[29710]: 3.16 scrub starts 2024-09-21T11:53:29.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[29710]: 3.16 scrub ok 2024-09-21T11:53:29.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[29710]: 3.19 scrub starts 2024-09-21T11:53:29.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[29710]: 3.19 scrub ok 2024-09-21T11:53:29.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3515060324' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:53:29.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[36454]: 6.a scrub starts 2024-09-21T11:53:29.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[36454]: 6.a scrub ok 2024-09-21T11:53:29.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[36454]: pgmap v649: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-21T11:53:29.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[36454]: osdmap e331: 8 total, 8 up, 8 in 2024-09-21T11:53:29.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[36454]: 3.16 scrub starts 2024-09-21T11:53:29.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[36454]: 3.16 scrub ok 2024-09-21T11:53:29.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[36454]: 3.19 scrub starts 2024-09-21T11:53:29.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[36454]: 3.19 scrub ok 2024-09-21T11:53:29.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3515060324' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:53:29.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:29.804 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:29.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:29.805 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:29.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:29.940 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:29.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:29.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:29 smithi175 ceph-mon[32613]: 6.a scrub starts 2024-09-21T11:53:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:29 smithi175 ceph-mon[32613]: 6.a scrub ok 2024-09-21T11:53:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:29 smithi175 ceph-mon[32613]: pgmap v649: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-21T11:53:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:29 smithi175 ceph-mon[32613]: osdmap e331: 8 total, 8 up, 8 in 2024-09-21T11:53:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:29 smithi175 ceph-mon[32613]: 3.16 scrub starts 2024-09-21T11:53:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:29 smithi175 ceph-mon[32613]: 3.16 scrub ok 2024-09-21T11:53:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:29 smithi175 ceph-mon[32613]: 3.19 scrub starts 2024-09-21T11:53:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:29 smithi175 ceph-mon[32613]: 3.19 scrub ok 2024-09-21T11:53:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3515060324' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:53:30.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:30.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:30.085 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:30.186 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:30.218 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:30.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:30.219 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:30.309 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 38 16 69151 69135 113.69 78.6875 0.00800907 0.00879068 2024-09-21T11:53:30.319 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:30.355 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:30.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:30.356 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:30.457 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:30.482 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:30.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:30.483 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:30.584 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:30.615 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:30.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:30.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:30.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:30.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:30.755 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:30.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:30.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:30.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[36454]: 4.16 scrub starts 2024-09-21T11:53:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[36454]: 4.16 scrub ok 2024-09-21T11:53:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[36454]: 4.8 scrub starts 2024-09-21T11:53:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[36454]: 4.8 scrub ok 2024-09-21T11:53:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[36454]: 4.2 scrub starts 2024-09-21T11:53:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[36454]: 4.2 scrub ok 2024-09-21T11:53:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[29710]: 4.16 scrub starts 2024-09-21T11:53:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[29710]: 4.16 scrub ok 2024-09-21T11:53:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[29710]: 4.8 scrub starts 2024-09-21T11:53:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[29710]: 4.8 scrub ok 2024-09-21T11:53:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[29710]: 4.2 scrub starts 2024-09-21T11:53:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:30 smithi043 ceph-mon[29710]: 4.2 scrub ok 2024-09-21T11:53:30.993 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:31.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:53:31.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:30 smithi175 ceph-mon[32613]: 4.16 scrub starts 2024-09-21T11:53:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:30 smithi175 ceph-mon[32613]: 4.16 scrub ok 2024-09-21T11:53:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:30 smithi175 ceph-mon[32613]: 4.8 scrub starts 2024-09-21T11:53:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:30 smithi175 ceph-mon[32613]: 4.8 scrub ok 2024-09-21T11:53:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:30 smithi175 ceph-mon[32613]: 4.2 scrub starts 2024-09-21T11:53:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:30 smithi175 ceph-mon[32613]: 4.2 scrub ok 2024-09-21T11:53:31.015 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:31.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:31.015 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:31.118 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:31.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:31.173 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:31.173 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:31.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:31.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:31.301 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:31.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:31.308 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 39 16 70288 70272 112.597 71.0625 0.00900001 0.00887098 2024-09-21T11:53:31.402 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:31.435 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:31.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:31.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:31.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:31.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:31.575 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:31.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:31.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:31.714 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:31.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:31.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:31.838 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:31.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:31.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:31.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:31.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:31.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:31.976 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[36454]: pgmap v651: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-21T11:53:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[36454]: 3.8 scrub starts 2024-09-21T11:53:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[36454]: 3.8 scrub ok 2024-09-21T11:53:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[36454]: 4.15 scrub starts 2024-09-21T11:53:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[36454]: 4.15 scrub ok 2024-09-21T11:53:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:31.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[29710]: pgmap v651: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-21T11:53:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[29710]: 3.8 scrub starts 2024-09-21T11:53:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[29710]: 3.8 scrub ok 2024-09-21T11:53:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[29710]: 4.15 scrub starts 2024-09-21T11:53:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[29710]: 4.15 scrub ok 2024-09-21T11:53:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:31.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:31 smithi175 ceph-mon[32613]: pgmap v651: 148 pgs: 148 active+clean; 4.0 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-21T11:53:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:31 smithi175 ceph-mon[32613]: 3.8 scrub starts 2024-09-21T11:53:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:31 smithi175 ceph-mon[32613]: 3.8 scrub ok 2024-09-21T11:53:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:31 smithi175 ceph-mon[32613]: 4.15 scrub starts 2024-09-21T11:53:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:31 smithi175 ceph-mon[32613]: 4.15 scrub ok 2024-09-21T11:53:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:31 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:32.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:53:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:53:32.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:32.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:32.121 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:32.121 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:32.200 INFO:tasks.thrashosds.thrasher:j is {'epoch': 331, 'fsid': '4f2c5252-780d-11ef-baf6-efdc52797490', 'created': '2024-09-21T11:34:07.249195+0000', 'modified': '2024-09-21T11:53:28.572972+0000', 'last_up_change': '2024-09-21T11:39:49.206441+0000', 'last_in_change': '2024-09-21T11:47:02.337266+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T11:37:26.023430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.980000019073486, 'score_stable': 6.980000019073486, 'optimal_score': 0.4300000071525574, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-21T11:39:58.262982+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 1, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-21T11:39:59.229530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-21T11:40:00.380059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.190000057220459, 'score_stable': 2.190000057220459, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-21T11:40:02.330041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-21T11:40:04.395413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-21T11:52:44.466175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '317', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.069999933242798, 'score_stable': 3.069999933242798, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3.5, 'raw_score_stable': 3.5, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': '5229029d-4fd6-46b6-8325-a3e004df09b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6801', 'nonce': 817279193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 817279193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 817279193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6803', 'nonce': 817279193}]}, 'public_addr': '172.21.15.43:6801/817279193', 'cluster_addr': '172.21.15.43:6802/817279193', 'heartbeat_back_addr': '172.21.15.43:6804/817279193', 'heartbeat_front_addr': '172.21.15.43:6803/817279193', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a2a7b58b-ba99-4383-a6d3-3b19388b2607', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6805', 'nonce': 263756171}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 263756171}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 263756171}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6807', 'nonce': 263756171}]}, 'public_addr': '172.21.15.43:6805/263756171', 'cluster_addr': '172.21.15.43:6806/263756171', 'heartbeat_back_addr': '172.21.15.43:6808/263756171', 'heartbeat_front_addr': '172.21.15.43:6807/263756171', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c24a60b5-5713-492b-82ee-576ea7a743ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6809', 'nonce': 687577863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 687577863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 687577863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6811', 'nonce': 687577863}]}, 'public_addr': '172.21.15.43:6809/687577863', 'cluster_addr': '172.21.15.43:6810/687577863', 'heartbeat_back_addr': '172.21.15.43:6812/687577863', 'heartbeat_front_addr': '172.21.15.43:6811/687577863', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c5d43cf8-1c72-4d7d-ae41-e8136b47177d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 293, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6813', 'nonce': 129493158}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 129493158}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 129493158}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6815', 'nonce': 129493158}]}, 'public_addr': '172.21.15.43:6813/129493158', 'cluster_addr': '172.21.15.43:6814/129493158', 'heartbeat_back_addr': '172.21.15.43:6816/129493158', 'heartbeat_front_addr': '172.21.15.43:6815/129493158', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '79ee85c5-afdc-4d68-a253-89b7b0eb652c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 33, 'up_thru': 235, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 1323992640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6801', 'nonce': 1323992640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6803', 'nonce': 1323992640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 1323992640}]}, 'public_addr': '172.21.15.175:6800/1323992640', 'cluster_addr': '172.21.15.175:6801/1323992640', 'heartbeat_back_addr': '172.21.15.175:6803/1323992640', 'heartbeat_front_addr': '172.21.15.175:6802/1323992640', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4d647321-71a1-412b-bbf5-921441923a65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 1163208458}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6805', 'nonce': 1163208458}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6807', 'nonce': 1163208458}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 1163208458}]}, 'public_addr': '172.21.15.175:6804/1163208458', 'cluster_addr': '172.21.15.175:6805/1163208458', 'heartbeat_back_addr': '172.21.15.175:6807/1163208458', 'heartbeat_front_addr': '172.21.15.175:6806/1163208458', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3c33bea7-94b5-46c1-9905-81b4612a2354', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9737701416015625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 47, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 932395625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6809', 'nonce': 932395625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6811', 'nonce': 932395625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 932395625}]}, 'public_addr': '172.21.15.175:6808/932395625', 'cluster_addr': '172.21.15.175:6809/932395625', 'heartbeat_back_addr': '172.21.15.175:6811/932395625', 'heartbeat_front_addr': '172.21.15.175:6810/932395625', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '41bc9817-d9d1-4c11-94ff-7de332af6d22', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 54, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 3676713275}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6813', 'nonce': 3676713275}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6815', 'nonce': 3676713275}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 3676713275}]}, 'public_addr': '172.21.15.175:6812/3676713275', 'cluster_addr': '172.21.15.175:6813/3676713275', 'heartbeat_back_addr': '172.21.15.175:6815/3676713275', 'heartbeat_front_addr': '172.21.15.175:6814/3676713275', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:53:16.468321+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:52:08.907218+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:53:27.671975+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:53:06.915356+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:52:28.472180+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:52:58.679157+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:52:46.310580+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:52:28.003963+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '6.2', 'osds': [2, 4, 5]}, {'pgid': '6.c', 'osds': [3, 7, 1]}, {'pgid': '6.1c', 'osds': [0, 7, 2]}], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.7', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.16', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.43:0/2419845767': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3249046048': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/2518306973': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/3115176321': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:6800/2583179759': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/4225358170': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/347827280': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/1513342881': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/1113684867': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3345448895': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/2454875216': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/911630798': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/983028036': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/4145025121': '2024-09-22T11:40:29.670859+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T11:53:32.201 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 6.c 2024-09-21T11:53:32.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd rm-pg-upmap 6.c 2024-09-21T11:53:32.222 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:32.312 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:53:32.309538+0000 min lat: 0.00131482 max lat: 0.302431 avg lat: 0.0089472 2024-09-21T11:53:32.313 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:53:32.313 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 40 16 71510 71494 111.687 76.375 0.0143695 0.0089472 2024-09-21T11:53:32.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:32.318 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:32.359 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:32.390 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:53:32.419 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:32.483 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:32.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:32.486 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:32.587 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:32.626 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:32.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:32.627 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:32.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: 6.9 scrub starts 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: 6.9 scrub ok 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: 6.7 scrub starts 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: 6.7 scrub ok 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: 5.d scrub starts 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: 5.d scrub ok 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/37241920' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/37241920' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[29710]: osdmap e332: 8 total, 8 up, 8 in 2024-09-21T11:53:32.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: 6.9 scrub starts 2024-09-21T11:53:32.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: 6.9 scrub ok 2024-09-21T11:53:32.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:32.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: 6.7 scrub starts 2024-09-21T11:53:32.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: 6.7 scrub ok 2024-09-21T11:53:32.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: 5.d scrub starts 2024-09-21T11:53:32.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: 5.d scrub ok 2024-09-21T11:53:32.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/37241920' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:53:32.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:32.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/37241920' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:53:32.684 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:32 smithi043 ceph-mon[36454]: osdmap e332: 8 total, 8 up, 8 in 2024-09-21T11:53:32.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:32.757 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:32.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:32.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:32.859 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:32.881 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:32.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:32.882 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:32.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:32.990 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:53:32.990 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:53:33.007 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:33.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:33.008 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:33.008 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:53:33.008 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:53:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: 6.9 scrub starts 2024-09-21T11:53:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: 6.9 scrub ok 2024-09-21T11:53:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: 6.7 scrub starts 2024-09-21T11:53:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: 6.7 scrub ok 2024-09-21T11:53:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: 5.d scrub starts 2024-09-21T11:53:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: 5.d scrub ok 2024-09-21T11:53:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/37241920' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:53:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/37241920' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:53:33.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:32 smithi175 ceph-mon[32613]: osdmap e332: 8 total, 8 up, 8 in 2024-09-21T11:53:33.025 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:53:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:53:33.041 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:53:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:53:33.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:33.131 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:33.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:33.132 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:33.232 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:53:33.232 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:53:33.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:33.265 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:53:33.265 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:53:33.270 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:33.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:33.271 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:33.294 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:53:33.296 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:53:33.296 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:53:33.310 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 41 16 72346 72330 110.238 52.25 0.00764047 0.00906578 2024-09-21T11:53:33.330 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:53:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:53:33.372 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:33.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:33.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:33.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:33.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:33.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:33.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:33.671 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:33.712 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:33.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:33.713 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:33.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:33.863 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:33.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:33.864 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:33.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:33.965 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:33.975 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:33 smithi175 ceph-mon[32613]: pgmap v652: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.5 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-21T11:53:33.975 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:33 smithi175 ceph-mon[32613]: 4.11 scrub starts 2024-09-21T11:53:33.975 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:33 smithi175 ceph-mon[32613]: 4.11 scrub ok 2024-09-21T11:53:33.975 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:33 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1655778680' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:33.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:33 smithi043 ceph-mon[36454]: pgmap v652: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.5 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-21T11:53:33.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:33 smithi043 ceph-mon[36454]: 4.11 scrub starts 2024-09-21T11:53:33.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:33 smithi043 ceph-mon[36454]: 4.11 scrub ok 2024-09-21T11:53:33.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:33 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1655778680' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:33 smithi043 ceph-mon[29710]: pgmap v652: 148 pgs: 148 active+clean; 4.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.5 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-21T11:53:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:33 smithi043 ceph-mon[29710]: 4.11 scrub starts 2024-09-21T11:53:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:33 smithi043 ceph-mon[29710]: 4.11 scrub ok 2024-09-21T11:53:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:33 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1655778680' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:34.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:34.009 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:34.009 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:34.026 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:34.110 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:34.144 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:34.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:34.145 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:34.246 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:34.278 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:34.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:34.279 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:34.310 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 42 16 73692 73676 109.616 84.125 0.0070809 0.00911759 2024-09-21T11:53:34.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:34.416 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:34.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:34.417 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:34.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:34.519 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:34.519 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:34.554 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:34.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:34.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:34.656 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:34.691 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:34.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:34.692 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:34.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:34.835 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:34.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:34.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:34.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:34.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:34.978 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:35.079 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:35.116 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:35.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:35.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:35.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:35.230 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:53:35.260 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:35.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:35.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:35.310 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 43 16 74975 74959 108.931 80.1875 0.0111484 0.00917508 2024-09-21T11:53:35.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:35.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:35.414 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:35.415 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:53:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:53:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:53:35.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:35.553 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:35.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:35.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:35.633 INFO:teuthology.orchestra.run.smithi043.stderr:clear 6.c pg_upmap mapping 2024-09-21T11:53:35.647 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:35.656 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:35.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:35.711 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:35.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:35.735 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:53:35.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:35.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:35.860 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:35.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:35.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:35 smithi175 ceph-mon[32613]: pgmap v654: 148 pgs: 148 active+clean; 4.3 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 769 B/s rd, 49 MiB/s wr, 779 op/s 2024-09-21T11:53:35.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:35 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3245199241' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap", "pgid": "6.c"} : dispatch 2024-09-21T11:53:35.939 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:35 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap", "pgid": "6.c"} : dispatch 2024-09-21T11:53:35.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:35 smithi043 ceph-mon[36454]: pgmap v654: 148 pgs: 148 active+clean; 4.3 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 769 B/s rd, 49 MiB/s wr, 779 op/s 2024-09-21T11:53:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:35 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3245199241' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap", "pgid": "6.c"} : dispatch 2024-09-21T11:53:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:35 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap", "pgid": "6.c"} : dispatch 2024-09-21T11:53:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:35 smithi043 ceph-mon[29710]: pgmap v654: 148 pgs: 148 active+clean; 4.3 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 769 B/s rd, 49 MiB/s wr, 779 op/s 2024-09-21T11:53:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:35 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3245199241' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap", "pgid": "6.c"} : dispatch 2024-09-21T11:53:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:35 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap", "pgid": "6.c"} : dispatch 2024-09-21T11:53:36.013 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:36.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:36.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:36.114 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:36.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:36.156 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:36.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:36.257 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:36.267 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:36.268 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:36.290 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:36.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:36.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:36.312 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 44 16 76197 76181 108.188 76.375 0.00979398 0.00923686 2024-09-21T11:53:36.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:36.426 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:36.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:36.427 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:36.427 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:36.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:36.563 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:36.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:36.564 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:36.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:53:36.665 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:36.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:36.701 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:36.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:36.803 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:36.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:36 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap", "pgid": "6.c"}]': finished 2024-09-21T11:53:36.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:36 smithi043 ceph-mon[29710]: osdmap e333: 8 total, 8 up, 8 in 2024-09-21T11:53:36.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:36 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:53:36.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:36 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:53:36.824 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:36 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap", "pgid": "6.c"}]': finished 2024-09-21T11:53:36.824 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:36 smithi043 ceph-mon[36454]: osdmap e333: 8 total, 8 up, 8 in 2024-09-21T11:53:36.824 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:36 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:53:36.824 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:36 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:53:36.851 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:36.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:36.852 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:36.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:36 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap", "pgid": "6.c"}]': finished 2024-09-21T11:53:36.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:36 smithi175 ceph-mon[32613]: osdmap e333: 8 total, 8 up, 8 in 2024-09-21T11:53:36.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:36 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:53:36.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:36 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:53:36.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:53:36.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:36.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:36.974 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:36.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:37.075 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:37.119 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:37.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:37.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:37.221 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:37.267 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:37.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:37.268 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:37.312 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 45 16 77344 77328 107.377 71.6875 0.0146718 0.00930774 2024-09-21T11:53:37.368 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:37.369 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:37.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:37.408 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:37.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:37.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:37.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:37.557 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:37.558 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:37.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:37.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:37.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:37 smithi175 ceph-mon[32613]: pgmap v656: 148 pgs: 148 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 687 B/s rd, 43 MiB/s wr, 695 op/s 2024-09-21T11:53:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:37 smithi175 ceph-mon[32613]: osdmap e334: 8 total, 8 up, 8 in 2024-09-21T11:53:37.795 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:37.826 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:37.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:37.827 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:37.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:37.950 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:37.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:37.951 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:37.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:37 smithi043 ceph-mon[36454]: pgmap v656: 148 pgs: 148 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 687 B/s rd, 43 MiB/s wr, 695 op/s 2024-09-21T11:53:37.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:37 smithi043 ceph-mon[36454]: osdmap e334: 8 total, 8 up, 8 in 2024-09-21T11:53:37.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:37 smithi043 ceph-mon[29710]: pgmap v656: 148 pgs: 148 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 687 B/s rd, 43 MiB/s wr, 695 op/s 2024-09-21T11:53:37.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:37 smithi043 ceph-mon[29710]: osdmap e334: 8 total, 8 up, 8 in 2024-09-21T11:53:38.052 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:38.091 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:38.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:38.092 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:38.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:38.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:38.226 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:38.312 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 46 16 78351 78335 106.411 62.9375 0.00387429 0.00938816 2024-09-21T11:53:38.327 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:38.348 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:38.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:38.349 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:38.450 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:38.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:38.482 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:38.482 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:38.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:38.616 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:38.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:38.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:38.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:38.745 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:38.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:38.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:38.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:38.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:38.888 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:38.889 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:38.989 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:39.012 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:39.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:39.012 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:39.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:39.158 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:39.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:39.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:39.164 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:39.164 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:39.261 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:39.292 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:39.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:39.293 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:39.312 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 47 16 79358 79342 105.486 62.9375 0.00958215 0.00947546 2024-09-21T11:53:39.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:39.422 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:39.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:39.423 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:39.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:39.549 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:39.561 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:39.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:39.562 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:39.643 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:53:39.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:39.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:39 smithi043 ceph-mon[36454]: pgmap v658: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:53:39.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3710875353' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:39.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:39 smithi043 ceph-mon[29710]: pgmap v658: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:53:39.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3710875353' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:39.715 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:39.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:39.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:39.817 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:39.850 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:39.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:39.851 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:39.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:39.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:39.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:39 smithi175 ceph-mon[32613]: pgmap v658: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:53:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3710875353' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:40.089 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:40.123 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:40.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:40.124 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:40.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:40.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:53:40.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:40.267 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:40.267 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:40.312 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 48 16 80514 80498 104.793 72.25 0.00936336 0.00953654 2024-09-21T11:53:40.368 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:40.402 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:40.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:40.403 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:40.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:40.539 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:40.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:40.541 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:40.642 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:40.656 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:40.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:40.683 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:40.684 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:40.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:40.821 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:40.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:40.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:40.923 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:40.959 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:40 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:40.959 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:40 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3710875353' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:40.959 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:40 smithi175 ceph-mon[32613]: osdmap e335: 8 total, 8 up, 8 in 2024-09-21T11:53:40.960 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:40.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:40.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:40.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:40 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:40.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3710875353' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:40.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:40 smithi043 ceph-mon[29710]: osdmap e335: 8 total, 8 up, 8 in 2024-09-21T11:53:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:40 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:40 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3710875353' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:40 smithi043 ceph-mon[36454]: osdmap e335: 8 total, 8 up, 8 in 2024-09-21T11:53:41.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:41.104 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:41.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:41.105 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:41.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:41.243 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:41.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:41.245 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:41.279 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:41.279 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:41.312 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 49 16 81622 81606 104.068 69.25 0.0105946 0.00960369 2024-09-21T11:53:41.346 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:41.362 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:53:41.362 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:53:41.362 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-21T11:53:41.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-21T11:53:41.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:41.382 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:41.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:41.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:41.528 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:41.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:41.529 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:41.630 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:41.659 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:41.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:41.660 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:41.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:41 smithi043 ceph-mon[29710]: pgmap v660: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 975 op/s 2024-09-21T11:53:41.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:41 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:53:41.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:41 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:53:41.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:41.801 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:41.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:41.803 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:41.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:41.926 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:41.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:41.927 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:41.934 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:41 smithi043 ceph-mon[36454]: pgmap v660: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 975 op/s 2024-09-21T11:53:41.934 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:41 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:53:41.934 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:41 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:53:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:41 smithi175 ceph-mon[32613]: pgmap v660: 148 pgs: 148 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 975 op/s 2024-09-21T11:53:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:41 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:53:42.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:41 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:53:42.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:53:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:53:42.028 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:42.061 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:42.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:42.062 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:42.093 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:53:42.094 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:53:42.135 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:53:42.136 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:53:42.163 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:42.171 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:53:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:53:42.190 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:42.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:42.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:42.208 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:53:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:53:42.251 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:53:42.251 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:53:42.285 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:53:42.286 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:53:42.292 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:42.312 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 50 16 82686 82670 103.317 66.5 0.00798095 0.00967356 2024-09-21T11:53:42.325 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:53:42.325 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:53:42.326 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:42.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:42.327 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:42.358 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:53:42.358 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:53:42.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:42.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:42.452 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:42.452 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:42.553 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:42.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:42.592 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:42.592 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:42.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:42.738 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:42.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:42.739 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:42.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:42.876 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:42.876 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:42 smithi175 ceph-mon[32613]: osdmap e336: 8 total, 8 up, 8 in 2024-09-21T11:53:42.876 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:42 smithi175 ceph-mon[32613]: pgmap v662: 148 pgs: 148 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 111 MiB/s wr, 1.77k op/s 2024-09-21T11:53:42.876 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4287520387' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:42.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:42.877 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:42.877 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:42.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:53:42.978 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:42.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:42.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:42 smithi043 ceph-mon[29710]: osdmap e336: 8 total, 8 up, 8 in 2024-09-21T11:53:42.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:42 smithi043 ceph-mon[29710]: pgmap v662: 148 pgs: 148 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 111 MiB/s wr, 1.77k op/s 2024-09-21T11:53:42.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4287520387' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:42.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:42.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:42 smithi043 ceph-mon[36454]: osdmap e336: 8 total, 8 up, 8 in 2024-09-21T11:53:42.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:42 smithi043 ceph-mon[36454]: pgmap v662: 148 pgs: 148 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 111 MiB/s wr, 1.77k op/s 2024-09-21T11:53:42.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4287520387' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:43.029 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:43.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:43.030 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:43.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:43.173 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:43.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:43.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:43.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:43.312 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 51 16 83868 83852 102.739 73.875 0.017867 0.0097282 2024-09-21T11:53:43.318 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:43.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:43.319 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:43.350 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:43.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:43.458 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:43.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:43.460 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:43.561 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:43.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:43.595 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:43.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:43.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:43.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:43.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:43.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:43.854 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:53:43.855 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 336, 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "fsid": "4f2c5252-780d-11ef-baf6-efdc52797490", 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "created": "2024-09-21T11:34:07.249195+0000", 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "modified": "2024-09-21T11:53:41.660700+0000", 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "last_up_change": "2024-09-21T11:39:49.206441+0000", 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "last_in_change": "2024-09-21T11:47:02.337266+0000", 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_num": 5806080, 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_set": [ 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "nodeep-scrub", 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "noscrub", 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "pglog_hardlimit", 2024-09-21T11:53:43.856 INFO:teuthology.orchestra.run.smithi043.stdout: "purged_snapdirs", 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_deletes", 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "sortbitwise" 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_version": 18, 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_snapshot": "", 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_max": 10, 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd": 8, 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "require_min_compat_client": "luminous", 2024-09-21T11:53:43.857 INFO:teuthology.orchestra.run.smithi043.stdout: "min_compat_client": "luminous", 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "require_osd_release": "squid", 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_crimson": false, 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "pools": [ 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 1, 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".mgr", 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:37:26.023430+0000", 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:43.858 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:43.859 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:43.859 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:43.859 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:43.859 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:43.859 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:43.859 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:43.859 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 1, 2024-09-21T11:53:43.859 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 1, 2024-09-21T11:53:43.859 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 1, 2024-09-21T11:53:43.859 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 1, 2024-09-21T11:53:43.860 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 1, 2024-09-21T11:53:43.860 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:43.860 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:43.860 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:43.860 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:43.860 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:43.860 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:43.860 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:43.860 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.860 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "253", 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:43.861 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.862 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:43.863 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:43.863 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:43.863 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:43.863 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:43.863 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:43.863 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:43.863 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:43.863 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:43.863 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:43.863 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:43.864 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:53:43.864 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_max": 32, 2024-09-21T11:53:43.864 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_min": 1 2024-09-21T11:53:43.864 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.864 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:43.864 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": {} 2024-09-21T11:53:43.865 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.865 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:43.865 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:43.865 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 6.9800000190734863, 2024-09-21T11:53:43.865 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 6.9800000190734863, 2024-09-21T11:53:43.865 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.43000000715255737, 2024-09-21T11:53:43.866 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 3, 2024-09-21T11:53:43.866 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 3, 2024-09-21T11:53:43.866 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:43.866 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:43.866 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:43.866 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.866 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.867 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.867 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 2, 2024-09-21T11:53:43.867 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "datapool", 2024-09-21T11:53:43.867 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:58.262982+0000", 2024-09-21T11:53:43.867 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 8193, 2024-09-21T11:53:43.867 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T11:53:43.868 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:43.868 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:43.868 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:43.868 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:43.869 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:43.869 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:43.869 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:43.869 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:43.870 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:43.870 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:43.870 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:43.870 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 3, 2024-09-21T11:53:43.870 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 3, 2024-09-21T11:53:43.870 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 3, 2024-09-21T11:53:43.870 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 3, 2024-09-21T11:53:43.871 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 3, 2024-09-21T11:53:43.871 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:43.871 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:43.871 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:43.871 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:43.871 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:43.872 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:43.872 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:43.872 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.872 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "61", 2024-09-21T11:53:43.872 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:43.872 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:43.873 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:43.873 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:43.873 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:43.873 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 2, 2024-09-21T11:53:43.873 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 61, 2024-09-21T11:53:43.873 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:43.873 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:43.874 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:43.874 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:43.874 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:43.874 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:43.874 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:43.874 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:43.875 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:43.875 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:43.875 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:43.875 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:43.875 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:43.875 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:43.876 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:43.876 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:43.876 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:43.876 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:43.876 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:43.876 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.876 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:43.877 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:53:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd": {} 2024-09-21T11:53:43.878 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.3299999237060547, 2024-09-21T11:53:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.3299999237060547, 2024-09-21T11:53:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1, 2024-09-21T11:53:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-21T11:53:43.879 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-21T11:53:43.880 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:43.880 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:43.880 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:43.880 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.880 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.880 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.880 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 3, 2024-09-21T11:53:43.880 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".rgw.root", 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:59.229530+0000", 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:43.881 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.882 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "60", 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:43.883 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:43.884 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:43.885 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.5399999618530273, 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.5399999618530273, 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.75, 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.75, 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 4, 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.log", 2024-09-21T11:53:43.886 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:00.380059+0000", 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:43.887 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "62", 2024-09-21T11:53:43.888 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:43.889 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:43.890 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:43.891 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.190000057220459, 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.190000057220459, 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2.5, 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2.5, 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 5, 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.control", 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:02.330041+0000", 2024-09-21T11:53:43.892 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:53:43.893 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "64", 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:43.894 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:43.895 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:43.896 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:43.896 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:43.896 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:43.896 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:43.896 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:43.896 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:43.896 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:43.896 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:43.896 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.896 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:43.897 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:53:43.898 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.898 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:43.898 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:43.898 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.3200000524520874, 2024-09-21T11:53:43.898 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.3200000524520874, 2024-09-21T11:53:43.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:53:43.899 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:53:43.899 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.5, 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.5, 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 6, 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.meta", 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:04.395413+0000", 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:43.900 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:53:43.901 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "66", 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:43.902 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:43.903 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:43.904 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_bias": 4 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:53:43.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.5399999618530273, 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.5399999618530273, 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.75, 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.75, 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 10, 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "unique_pool_3", 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:52:44.466175+0000", 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:53:43.907 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:53:43.908 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:53:43.908 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:53:43.908 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 16, 2024-09-21T11:53:43.908 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 16, 2024-09-21T11:53:43.908 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 16, 2024-09-21T11:53:43.908 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 16, 2024-09-21T11:53:43.908 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 16, 2024-09-21T11:53:43.908 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:53:43.908 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "317", 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:53:43.909 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:53:43.910 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:53:43.911 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:53:43.911 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:53:43.911 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:53:43.911 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:53:43.911 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:53:43.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:53:43.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.911 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:53:43.911 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:53:43.911 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:53:43.912 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:53:43.912 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:53:43.912 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:53:43.912 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:53:43.912 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:53:43.912 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:53:43.912 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "rados": {} 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 3.0699999332427979, 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 3.0699999332427979, 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 3.5, 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 3.5, 2024-09-21T11:53:43.913 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4", 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:43.914 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 8, 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6802", 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.916 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6804", 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6803", 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6801/817279193", 2024-09-21T11:53:43.917 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6802/817279193", 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6804/817279193", 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6803/817279193", 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607", 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:43.918 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 13, 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.919 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6805", 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6806", 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:43.920 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6808", 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6807", 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:53:43.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6805/263756171", 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6806/263756171", 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6808/263756171", 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6807/263756171", 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:53:43.922 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce", 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 18, 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.923 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6809", 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6810", 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.924 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6812", 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6811", 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:53:43.925 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6809/687577863", 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6810/687577863", 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6812/687577863", 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6811/687577863", 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.926 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d", 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 27, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 333, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.927 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6813", 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6814", 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.928 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6816", 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6815", 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:53:43.929 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6813/129493158", 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6814/129493158", 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6816/129493158", 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6815/129493158", 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.930 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c", 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 0, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 33, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 235, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6800", 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6801", 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.933 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:43.933 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.933 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.933 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6803", 2024-09-21T11:53:43.933 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:53:43.933 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.933 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.933 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6802", 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6800/1323992640", 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6801/1323992640", 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6803/1323992640", 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6802/1323992640", 2024-09-21T11:53:43.934 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "4d647321-71a1-412b-bbf5-921441923a65", 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 39, 2024-09-21T11:53:43.935 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:43.936 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:43.936 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:43.936 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:43.936 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.936 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.936 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.936 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6804", 2024-09-21T11:53:43.936 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:53:43.936 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.936 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:43.937 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:43.938 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:43.938 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.938 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:43.938 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6805", 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6807", 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:53:43.939 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6806", 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6804/1163208458", 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6805/1163208458", 2024-09-21T11:53:43.940 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6807/1163208458", 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6806/1163208458", 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354", 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 0.9737701416015625, 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:43.941 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 47, 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6808", 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6809", 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:43.943 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6811", 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6810", 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:53:43.944 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6808/932395625", 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6809/932395625", 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6811/932395625", 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6810/932395625", 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22", 2024-09-21T11:53:43.945 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 54, 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 313, 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.946 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6812", 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6813", 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6815", 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6814", 2024-09-21T11:53:43.948 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6812/3676713275", 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6813/3676713275", 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6815/3676713275", 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6814/3676713275", 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:43.949 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_xinfo": [ 2024-09-21T11:53:43.950 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.950 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:53:43.950 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:43.950 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:43.950 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:43.950 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:43.950 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:43.950 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:53:16.468321+0000", 2024-09-21T11:53:43.950 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:43.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.951 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.951 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:53:43.951 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:43.951 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:43.951 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:43.951 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:43.951 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:43.951 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:53:35.706107+0000", 2024-09-21T11:53:43.951 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:43.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:53:27.671975+0000", 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.952 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:53:06.915356+0000", 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:43.953 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:52:28.472180+0000", 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:43.954 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:52:58.679157+0000", 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:52:46.310580+0000", 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.955 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:53:41.021095+0000", 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap": [ 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.2", 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: 2, 2024-09-21T11:53:43.956 INFO:teuthology.orchestra.run.smithi043.stdout: 4, 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: 5 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.1c", 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: 7, 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: 2 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:43.957 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_items": [ 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.4", 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.7", 2024-09-21T11:53:43.958 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.d", 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.959 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.13", 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.16", 2024-09-21T11:53:43.960 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_primaries": [], 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_temp": [], 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_temp": [], 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklist": { 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2419845767": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3249046048": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:53:43.961 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2518306973": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3115176321": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/2583179759": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/4225358170": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/347827280": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/1513342881": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/1113684867": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3345448895": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2454875216": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/911630798": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/983028036": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/4145025121": "2024-09-22T11:40:29.670859+0000" 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "range_blocklist": {}, 2024-09-21T11:53:43.962 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profiles": { 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "default": { 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-failure-domain": "osd", 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "k": "2", 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "m": "1", 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "plugin": "jerasure", 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "technique": "reed_sol_van" 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps_queue": [], 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "new_removed_snaps": [], 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "new_purged_snaps": [], 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_node_flags": {}, 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "device_class_flags": {}, 2024-09-21T11:53:43.963 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode": { 2024-09-21T11:53:43.964 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_enabled": false, 2024-09-21T11:53:43.964 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_bucket_count": 0, 2024-09-21T11:53:43.964 INFO:teuthology.orchestra.run.smithi043.stdout: "degraded_stretch_mode": 0, 2024-09-21T11:53:43.964 INFO:teuthology.orchestra.run.smithi043.stdout: "recovering_stretch_mode": 0, 2024-09-21T11:53:43.964 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_bucket": 0 2024-09-21T11:53:43.964 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:53:43.964 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:53:43.988 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:53:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:53:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.988 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:84635 10.a 10:5edaa3c1:::benchmark_data_smithi043_150678_object84634:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.984012+0000", 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.001669282, 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001696957, 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 84635 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:43.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984012+0000", 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984012+0000", 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984013+0000", 2024-09-21T11:53:43.990 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6400000000000001e-07 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984114+0000", 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100646 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984123+0000", 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.064e-06 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984129+0000", 2024-09-21T11:53:43.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3459999999999999e-06 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984240+0000", 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111267 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984359+0000", 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011849999999999999 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984491+0000", 2024-09-21T11:53:43.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013254799999999999 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:84627 10.d e336/313)", 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.984209+0000", 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.001472246, 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015663859999999999, 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "queued for pg", 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:43.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984209+0000", 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984209+0000", 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984210+0000", 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2200000000000003e-07 2024-09-21T11:53:43.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984214+0000", 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2729999999999996e-06 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984224+0000", 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5619999999999993e-06 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984230+0000", 2024-09-21T11:53:43.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3239999999999999e-06 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:84626 10.e e336/313)", 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.984253+0000", 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.001428557, 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015611399999999999, 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "queued for pg", 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:43.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984253+0000", 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984253+0000", 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984253+0000", 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.72e-07 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984256+0000", 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.154e-06 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984262+0000", 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2489999999999998e-06 2024-09-21T11:53:43.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984267+0000", 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2220000000000001e-06 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:84637 10.7 10:eb3d7c57:::benchmark_data_smithi043_150678_object84636:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.984697+0000", 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0009845679999999999, 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00115726, 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:43.999 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 84637 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984697+0000", 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:44.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984697+0000", 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984698+0000", 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.02e-07 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984761+0000", 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2774000000000006e-05 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:44.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984773+0000", 2024-09-21T11:53:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1958e-05 2024-09-21T11:53:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984779+0000", 2024-09-21T11:53:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.635e-06 2024-09-21T11:53:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:53:44.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984955+0000", 2024-09-21T11:53:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017569300000000001 2024-09-21T11:53:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:53:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.985071+0000", 2024-09-21T11:53:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116322 2024-09-21T11:53:44.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.985239+0000", 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016751099999999999 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:84619 10.0 10:0db571ea:::benchmark_data_smithi043_150678_object84618:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.972227+0000", 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.013454433, 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013690847000000001, 2024-09-21T11:53:44.004 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 84619 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972227+0000", 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.005 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972227+0000", 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972228+0000", 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.64e-07 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972291+0000", 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2525000000000002e-05 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972298+0000", 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8720000000000002e-06 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972306+0000", 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4070000000000004e-06 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972358+0000", 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2132999999999997e-05 2024-09-21T11:53:44.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972457+0000", 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9380000000000001e-05 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972533+0000", 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5516e-05 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:53:44.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.976362+0000", 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0038295859999999998 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.982133+0000", 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057703049999999999 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.985907+0000", 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0037743910000000002 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:84620 10.f 10:f1459a74:::benchmark_data_smithi043_150678_object84619:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.972566+0000", 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.013115597999999999, 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.013427421, 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 84620 2024-09-21T11:53:44.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972566+0000", 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972566+0000", 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:44.011 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972567+0000", 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5600000000000004e-07 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972638+0000", 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0945000000000006e-05 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972646+0000", 2024-09-21T11:53:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2539999999999992e-06 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.972652+0000", 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1569999999999997e-06 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.974737+0000", 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020844399999999999 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:53:44.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.974852+0000", 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115133 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978433+0000", 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003581051 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.984913+0000", 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0064802690000000003 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.014 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:84624 10.f 10:ff739ac7:::benchmark_data_smithi043_150678_object84623:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.978432+0000", 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0072494810000000003, 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0076229139999999997, 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:53:44.015 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 84624 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978432+0000", 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978432+0000", 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:44.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978433+0000", 2024-09-21T11:53:44.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.198e-06 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978538+0000", 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104505 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978547+0000", 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0920000000000004e-06 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978554+0000", 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5979999999999999e-06 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.981545+0000", 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0029904929999999999 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.981650+0000", 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010534 2024-09-21T11:53:44.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.981763+0000", 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000112309 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:84626 10.e 10:71833efd:::benchmark_data_smithi043_150678_object84625:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.978579+0000", 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.007102209, 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0075351200000000002, 2024-09-21T11:53:44.021 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 84626 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978579+0000", 2024-09-21T11:53:44.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.023 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:44.023 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978579+0000", 2024-09-21T11:53:44.023 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.023 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:44.023 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978580+0000", 2024-09-21T11:53:44.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5499999999999998e-07 2024-09-21T11:53:44.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:44.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978628+0000", 2024-09-21T11:53:44.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7874000000000003e-05 2024-09-21T11:53:44.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:44.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978634+0000", 2024-09-21T11:53:44.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.635e-06 2024-09-21T11:53:44.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:44.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978640+0000", 2024-09-21T11:53:44.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2959999999999998e-06 2024-09-21T11:53:44.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:53:44.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978706+0000", 2024-09-21T11:53:44.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6079000000000002e-05 2024-09-21T11:53:44.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:53:44.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978841+0000", 2024-09-21T11:53:44.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013468900000000001 2024-09-21T11:53:44.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,4", 2024-09-21T11:53:44.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.979161+0000", 2024-09-21T11:53:44.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032062900000000001 2024-09-21T11:53:44.027 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.027 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:44.027 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.028 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:84627 10.d 10:be6639d7:::benchmark_data_smithi043_150678_object84626:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:53:44.028 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.978893+0000", 2024-09-21T11:53:44.028 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0067888030000000004, 2024-09-21T11:53:44.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0072781410000000001, 2024-09-21T11:53:44.028 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:44.028 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:44.028 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:53:44.028 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:53:44.029 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:53:44.029 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:53:44.029 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 84627 2024-09-21T11:53:44.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.029 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:44.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:44.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978893+0000", 2024-09-21T11:53:44.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:44.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978893+0000", 2024-09-21T11:53:44.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:44.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978893+0000", 2024-09-21T11:53:44.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3e-07 2024-09-21T11:53:44.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:44.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978962+0000", 2024-09-21T11:53:44.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8448000000000006e-05 2024-09-21T11:53:44.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:44.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978972+0000", 2024-09-21T11:53:44.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9830000000000008e-06 2024-09-21T11:53:44.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:44.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.978978+0000", 2024-09-21T11:53:44.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4099999999999996e-06 2024-09-21T11:53:44.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:53:44.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.979397+0000", 2024-09-21T11:53:44.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00041874499999999999 2024-09-21T11:53:44.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:53:44.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.979536+0000", 2024-09-21T11:53:44.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000139099 2024-09-21T11:53:44.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:53:44.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.979645+0000", 2024-09-21T11:53:44.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108811 2024-09-21T11:53:44.034 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.034 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:44.034 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.035 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:84629 10.b 10:dd44051a:::benchmark_data_smithi043_150678_object84628:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:53:44.035 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.980275+0000", 2024-09-21T11:53:44.035 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0054069610000000001, 2024-09-21T11:53:44.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0059535810000000003, 2024-09-21T11:53:44.035 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:44.035 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:44.035 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:53:44.035 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:53:44.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:53:44.036 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:53:44.036 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 84629 2024-09-21T11:53:44.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.036 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:44.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.036 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:44.036 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.980275+0000", 2024-09-21T11:53:44.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.980275+0000", 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.980276+0000", 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9800000000000002e-07 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.980360+0000", 2024-09-21T11:53:44.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4259999999999996e-05 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.980370+0000", 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0023999999999999e-05 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.980377+0000", 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8199999999999999e-06 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:53:44.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.980420+0000", 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3785999999999997e-05 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.980523+0000", 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010225000000000001 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1", 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.980634+0000", 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111079 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:84626 10.e e336/313)", 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.983383+0000", 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0022988420000000002, 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028959910000000001, 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "queued for pg", 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983383+0000", 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983383+0000", 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983383+0000", 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2300000000000001e-07 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983391+0000", 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8569999999999995e-06 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983400+0000", 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3910000000000004e-06 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983407+0000", 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0769999999999996e-06 2024-09-21T11:53:44.042 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:84630 10.0 10:014ae763:::benchmark_data_smithi043_150678_object84629:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.983550+0000", 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0021313529999999999, 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027683809999999999, 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:53:44.043 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 84630 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983550+0000", 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983550+0000", 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983551+0000", 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.189e-06 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983596+0000", 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4351000000000003e-05 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983605+0000", 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3889999999999994e-06 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:44.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983611+0000", 2024-09-21T11:53:44.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1190000000000002e-06 2024-09-21T11:53:44.046 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.046 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:53:44.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.986274+0000", 2024-09-21T11:53:44.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026625949999999998 2024-09-21T11:53:44.046 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.046 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:44.046 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.047 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.047 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.047 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:84631 10.f 10:f85b8c0e:::benchmark_data_smithi043_150678_object84630:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:53:44.047 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:43.983630+0000", 2024-09-21T11:53:44.047 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0020510559999999999, 2024-09-21T11:53:44.047 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027420019999999999, 2024-09-21T11:53:44.047 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:53:44.047 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:53:44.047 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:53:44.047 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 84631 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983630+0000", 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.048 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:53:44.049 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983630+0000", 2024-09-21T11:53:44.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:53:44.049 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.049 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.049 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:53:44.049 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983631+0000", 2024-09-21T11:53:44.049 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8400000000000005e-07 2024-09-21T11:53:44.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:53:44.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983666+0000", 2024-09-21T11:53:44.050 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5116000000000001e-05 2024-09-21T11:53:44.050 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.050 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.050 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:53:44.050 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983672+0000", 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1430000000000003e-06 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.983677+0000", 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2190000000000003e-06 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.985474+0000", 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001796513 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.051 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.985577+0000", 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010273299999999999 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:43.985693+0000", 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116149 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.052 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:53:44.053 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:53:44.053 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 13 2024-09-21T11:53:44.053 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:53:44.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:44.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:44.096 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:44.139 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:43 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3609711221' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:53:44.139 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:43 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3609711221' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:53:44.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:44.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:44.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:44.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:43 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3609711221' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:53:44.314 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 52 15 85001 84986 102.123 70.875 0.00922423 0.00978673 2024-09-21T11:53:44.346 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:44.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:44.369 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:44.369 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:44.470 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:44.504 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:44.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:44.505 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:44.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:44.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:44.643 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:44.644 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:44.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:44.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:44.784 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:44.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:44.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:44.907 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:44.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:44.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:45.009 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:45.021 INFO:tasks.thrashosds.thrasher:j is {'epoch': 336, 'fsid': '4f2c5252-780d-11ef-baf6-efdc52797490', 'created': '2024-09-21T11:34:07.249195+0000', 'modified': '2024-09-21T11:53:41.660700+0000', 'last_up_change': '2024-09-21T11:39:49.206441+0000', 'last_in_change': '2024-09-21T11:47:02.337266+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T11:37:26.023430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.980000019073486, 'score_stable': 6.980000019073486, 'optimal_score': 0.4300000071525574, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-21T11:39:58.262982+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 1, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-21T11:39:59.229530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-21T11:40:00.380059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.190000057220459, 'score_stable': 2.190000057220459, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-21T11:40:02.330041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-21T11:40:04.395413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-21T11:52:44.466175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '317', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.069999933242798, 'score_stable': 3.069999933242798, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3.5, 'raw_score_stable': 3.5, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': '5229029d-4fd6-46b6-8325-a3e004df09b4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6801', 'nonce': 817279193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 817279193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 817279193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6803', 'nonce': 817279193}]}, 'public_addr': '172.21.15.43:6801/817279193', 'cluster_addr': '172.21.15.43:6802/817279193', 'heartbeat_back_addr': '172.21.15.43:6804/817279193', 'heartbeat_front_addr': '172.21.15.43:6803/817279193', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a2a7b58b-ba99-4383-a6d3-3b19388b2607', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6805', 'nonce': 263756171}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 263756171}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 263756171}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6807', 'nonce': 263756171}]}, 'public_addr': '172.21.15.43:6805/263756171', 'cluster_addr': '172.21.15.43:6806/263756171', 'heartbeat_back_addr': '172.21.15.43:6808/263756171', 'heartbeat_front_addr': '172.21.15.43:6807/263756171', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c24a60b5-5713-492b-82ee-576ea7a743ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6809', 'nonce': 687577863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 687577863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 687577863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6811', 'nonce': 687577863}]}, 'public_addr': '172.21.15.43:6809/687577863', 'cluster_addr': '172.21.15.43:6810/687577863', 'heartbeat_back_addr': '172.21.15.43:6812/687577863', 'heartbeat_front_addr': '172.21.15.43:6811/687577863', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c5d43cf8-1c72-4d7d-ae41-e8136b47177d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 333, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6813', 'nonce': 129493158}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 129493158}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 129493158}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6815', 'nonce': 129493158}]}, 'public_addr': '172.21.15.43:6813/129493158', 'cluster_addr': '172.21.15.43:6814/129493158', 'heartbeat_back_addr': '172.21.15.43:6816/129493158', 'heartbeat_front_addr': '172.21.15.43:6815/129493158', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '79ee85c5-afdc-4d68-a253-89b7b0eb652c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 33, 'up_thru': 235, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 1323992640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6801', 'nonce': 1323992640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6803', 'nonce': 1323992640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 1323992640}]}, 'public_addr': '172.21.15.175:6800/1323992640', 'cluster_addr': '172.21.15.175:6801/1323992640', 'heartbeat_back_addr': '172.21.15.175:6803/1323992640', 'heartbeat_front_addr': '172.21.15.175:6802/1323992640', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4d647321-71a1-412b-bbf5-921441923a65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 1163208458}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6805', 'nonce': 1163208458}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6807', 'nonce': 1163208458}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 1163208458}]}, 'public_addr': '172.21.15.175:6804/1163208458', 'cluster_addr': '172.21.15.175:6805/1163208458', 'heartbeat_back_addr': '172.21.15.175:6807/1163208458', 'heartbeat_front_addr': '172.21.15.175:6806/1163208458', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3c33bea7-94b5-46c1-9905-81b4612a2354', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.9737701416015625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 47, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 932395625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6809', 'nonce': 932395625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6811', 'nonce': 932395625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 932395625}]}, 'public_addr': '172.21.15.175:6808/932395625', 'cluster_addr': '172.21.15.175:6809/932395625', 'heartbeat_back_addr': '172.21.15.175:6811/932395625', 'heartbeat_front_addr': '172.21.15.175:6810/932395625', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '41bc9817-d9d1-4c11-94ff-7de332af6d22', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 54, 'up_thru': 313, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 3676713275}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6813', 'nonce': 3676713275}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6815', 'nonce': 3676713275}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 3676713275}]}, 'public_addr': '172.21.15.175:6812/3676713275', 'cluster_addr': '172.21.15.175:6813/3676713275', 'heartbeat_back_addr': '172.21.15.175:6815/3676713275', 'heartbeat_front_addr': '172.21.15.175:6814/3676713275', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:53:16.468321+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:53:35.706107+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:53:27.671975+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:53:06.915356+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:52:28.472180+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:52:58.679157+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:52:46.310580+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:53:41.021095+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '6.2', 'osds': [2, 4, 5]}, {'pgid': '6.1c', 'osds': [0, 7, 2]}], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.7', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.16', 'mappings': [{'from': 3, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.43:0/2419845767': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3249046048': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/2518306973': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/3115176321': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:6800/2583179759': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/4225358170': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/347827280': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/1513342881': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/1113684867': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3345448895': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/2454875216': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/911630798': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/983028036': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/4145025121': '2024-09-22T11:40:29.670859+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T11:53:45.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:53:45.030 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:45.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:45.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:45.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:45.154 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:45.219 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:45.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:45.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:45.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:53:45.228 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:44 smithi175 ceph-mon[32613]: pgmap v663: 148 pgs: 148 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 691 B/s rd, 52 MiB/s wr, 832 op/s 2024-09-21T11:53:45.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:44 smithi043 ceph-mon[29710]: pgmap v663: 148 pgs: 148 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 691 B/s rd, 52 MiB/s wr, 832 op/s 2024-09-21T11:53:45.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:44 smithi043 ceph-mon[36454]: pgmap v663: 148 pgs: 148 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 691 B/s rd, 52 MiB/s wr, 832 op/s 2024-09-21T11:53:45.314 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 53 16 85768 85752 101.099 47.875 0.0680826 0.0098779 2024-09-21T11:53:45.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:45.348 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:45.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:45.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:45.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:45.493 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:53:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:53:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:53:45.507 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:45.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:45.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:45.609 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:45.645 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:45.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:45.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:45.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:45.786 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:45.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:45.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:45.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:45.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:45.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:46.022 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:46.055 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:46.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:46.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:46.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:53:46.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:46.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:46.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:46.283 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:46.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 54 16 86793 86777 100.413 64.0625 0.010037 0.00995358 2024-09-21T11:53:46.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:46.333 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:46.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:46.434 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:46.457 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:46.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:46.478 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:46.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:46.535 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:46.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:46.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:46.631 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:46.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:46.731 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:46.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:46.772 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:46.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:46.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:46.923 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:46.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:46.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:47.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:47.056 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:47.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:47.058 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:47.117 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:53:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:53:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:53:47.117 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:53:47.117 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:53:47.155 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:47.158 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:47.193 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:47.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:47.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:47.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:47.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 55 16 87860 87844 99.8 66.6875 0.00857645 0.0100154 2024-09-21T11:53:47.326 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:47.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:47.327 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:47.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:47.462 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:47.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:47.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:47.507 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:53:47.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:47 smithi043 ceph-mon[29710]: pgmap v664: 148 pgs: 148 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 682 op/s 2024-09-21T11:53:47.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:47 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:47.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:47 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:47.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/874424542' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:53:47.547 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:47 smithi043 ceph-mon[36454]: pgmap v664: 148 pgs: 148 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 682 op/s 2024-09-21T11:53:47.547 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:47 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:47.547 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:47 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:47.547 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/874424542' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:53:47.559 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:53:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:53:47.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:47.612 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:47.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:47.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:47.714 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:47.753 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:47.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:47.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:47.783 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:47 smithi175 ceph-mon[32613]: pgmap v664: 148 pgs: 148 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 682 op/s 2024-09-21T11:53:47.783 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:47 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:53:47.783 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:47 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:53:47.783 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/874424542' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:53:47.819 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:53:47.819 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:53:47.835 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:53:47.853 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:53:47.853 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:53:47.855 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:47.885 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:53:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:53:47.915 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:47.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:47.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:47.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:53:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:53:47.959 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:53:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:53:47.995 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:53:47.996 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:53:48.016 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:48.035 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:53:48.036 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:53:48.059 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:48.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:48.060 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:48.069 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:53:48.069 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:53:48.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:48.197 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:48.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:48.198 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:48.299 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:48.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 56 16 88947 88931 99.231 67.9375 0.0197645 0.0100726 2024-09-21T11:53:48.324 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:48.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:48.346 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:48.346 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:48.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:48.485 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:48.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:48.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:48.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:48.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:48.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:48.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:48 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:48.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:48 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/874424542' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:53:48.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:48 smithi043 ceph-mon[36454]: osdmap e337: 8 total, 8 up, 8 in 2024-09-21T11:53:48.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:48 smithi043 ceph-mon[36454]: from='client.16170 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:53:48.655 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:48 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2008291841' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:48.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:48 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:48.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:48 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/874424542' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:53:48.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:48 smithi043 ceph-mon[29710]: osdmap e337: 8 total, 8 up, 8 in 2024-09-21T11:53:48.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:48 smithi043 ceph-mon[29710]: from='client.16170 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:53:48.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:48 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2008291841' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:48.657 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":665,"stamp":"2024-09-21T11:53:47.508121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5691611675,"num_objects":87037,"num_object_clones":0,"num_object_copies":261111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87037,"num_whiteouts":0,"num_read":2066,"num_read_kb":1923,"num_write":87673,"num_write_kb":5559305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":597267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87084,"ondisk_log_size":87084,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20726716,"kb_used_data":16630268,"kb_used_omap":12,"kb_used_meta":4096307,"kb_avail":729234500,"statfs":{"total":767960285184,"available":746736128000,"internally_reserved":0,"allocated":17029394432,"data_stored":17025252641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4194618930},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,3,8,3,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":697827328,"num_objects":10648,"num_object_clones":0,"num_object_copies":31944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10648,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10648,"num_write_kb":681472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.865017"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":595,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680632+0000","last_change":"2024-09-21T11:52:54.792106+0000","last_active":"2024-09-21T11:53:41.680632+0000","last_peered":"2024-09-21T11:53:41.680632+0000","last_clean":"2024-09-21T11:53:41.680632+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:53:41.680632+0000","last_undegraded":"2024-09-21T11:53:41.680632+0000","last_fullsized":"2024-09-21T11:53:41.680632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:54.792037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:54.792037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.576756+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":626,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683087+0000","last_change":"2024-09-21T11:52:08.520781+0000","last_active":"2024-09-21T11:53:41.683087+0000","last_peered":"2024-09-21T11:53:41.683087+0000","last_clean":"2024-09-21T11:53:41.683087+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:53:41.683087+0000","last_undegraded":"2024-09-21T11:53:41.683087+0000","last_fullsized":"2024-09-21T11:53:41.683087+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:52:08.520688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:08.520688+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:17.574851+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":304,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.677396+0000","last_change":"2024-09-21T11:53:20.097633+0000","last_active":"2024-09-21T11:53:41.677396+0000","last_peered":"2024-09-21T11:53:41.677396+0000","last_clean":"2024-09-21T11:53:41.677396+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:53:41.677396+0000","last_undegraded":"2024-09-21T11:53:41.677396+0000","last_fullsized":"2024-09-21T11:53:41.677396+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:20.097557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.097557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:40.861158+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":407,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680172+0000","last_change":"2024-09-21T11:51:43.528594+0000","last_active":"2024-09-21T11:53:41.680172+0000","last_peered":"2024-09-21T11:53:41.680172+0000","last_clean":"2024-09-21T11:53:41.680172+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:53:41.680172+0000","last_undegraded":"2024-09-21T11:53:41.680172+0000","last_fullsized":"2024-09-21T11:53:41.680172+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.914122+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":599,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.688970+0000","last_change":"2024-09-21T11:52:11.462678+0000","last_active":"2024-09-21T11:53:41.688970+0000","last_peered":"2024-09-21T11:53:41.688970+0000","last_clean":"2024-09-21T11:53:41.688970+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:53:41.688970+0000","last_undegraded":"2024-09-21T11:53:41.688970+0000","last_fullsized":"2024-09-21T11:53:41.688970+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:11.462611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:11.462611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":588,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.688558+0000","last_change":"2024-09-21T11:51:01.667426+0000","last_active":"2024-09-21T11:53:41.688558+0000","last_peered":"2024-09-21T11:53:41.688558+0000","last_clean":"2024-09-21T11:53:41.688558+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:53:41.688558+0000","last_undegraded":"2024-09-21T11:53:41.688558+0000","last_fullsized":"2024-09-21T11:53:41.688558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.667366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.667366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.200426+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":634,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.688780+0000","last_change":"2024-09-21T11:51:43.409002+0000","last_active":"2024-09-21T11:53:41.688780+0000","last_peered":"2024-09-21T11:53:41.688780+0000","last_clean":"2024-09-21T11:53:41.688780+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:53:41.688780+0000","last_undegraded":"2024-09-21T11:53:41.688780+0000","last_fullsized":"2024-09-21T11:53:41.688780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:51:43.408804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.408804+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":427,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683172+0000","last_change":"2024-09-21T11:52:54.554185+0000","last_active":"2024-09-21T11:53:41.683172+0000","last_peered":"2024-09-21T11:53:41.683172+0000","last_clean":"2024-09-21T11:53:41.683172+0000","last_became_active":"2024-09-21T11:51:43.526945+0000","last_became_peered":"2024-09-21T11:51:43.526945+0000","last_unstale":"2024-09-21T11:53:41.683172+0000","last_undegraded":"2024-09-21T11:53:41.683172+0000","last_fullsized":"2024-09-21T11:53:41.683172+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:54.553857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:54.553857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:55.495025+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":594,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680890+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:53:41.680890+0000","last_peered":"2024-09-21T11:53:41.680890+0000","last_clean":"2024-09-21T11:53:41.680890+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:53:41.680890+0000","last_undegraded":"2024-09-21T11:53:41.680890+0000","last_fullsized":"2024-09-21T11:53:41.680890+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.870896+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":592,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.688304+0000","last_change":"2024-09-21T11:52:55.214031+0000","last_active":"2024-09-21T11:53:41.688304+0000","last_peered":"2024-09-21T11:53:41.688304+0000","last_clean":"2024-09-21T11:53:41.688304+0000","last_became_active":"2024-09-21T11:51:43.545021+0000","last_became_peered":"2024-09-21T11:51:43.545021+0000","last_unstale":"2024-09-21T11:53:41.688304+0000","last_undegraded":"2024-09-21T11:53:41.688304+0000","last_fullsized":"2024-09-21T11:53:41.688304+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:52:55.213938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:55.213938+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:12.879409+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":586,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680414+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:53:41.680414+0000","last_peered":"2024-09-21T11:53:41.680414+0000","last_clean":"2024-09-21T11:53:41.680414+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:53:41.680414+0000","last_undegraded":"2024-09-21T11:53:41.680414+0000","last_fullsized":"2024-09-21T11:53:41.680414+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.824812+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":612,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.684808+0000","last_change":"2024-09-21T11:52:50.731704+0000","last_active":"2024-09-21T11:53:41.684808+0000","last_peered":"2024-09-21T11:53:41.684808+0000","last_clean":"2024-09-21T11:53:41.684808+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:53:41.684808+0000","last_undegraded":"2024-09-21T11:53:41.684808+0000","last_fullsized":"2024-09-21T11:53:41.684808+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:50.731653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:50.731653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.382981+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":624,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.681283+0000","last_change":"2024-09-21T11:52:11.666514+0000","last_active":"2024-09-21T11:53:41.681283+0000","last_peered":"2024-09-21T11:53:41.681283+0000","last_clean":"2024-09-21T11:53:41.681283+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:53:41.681283+0000","last_undegraded":"2024-09-21T11:53:41.681283+0000","last_fullsized":"2024-09-21T11:53:41.681283+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:11.666440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:11.666440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:32.994631+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":598,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.689252+0000","last_change":"2024-09-21T11:51:43.547924+0000","last_active":"2024-09-21T11:53:41.689252+0000","last_peered":"2024-09-21T11:53:41.689252+0000","last_clean":"2024-09-21T11:53:41.689252+0000","last_became_active":"2024-09-21T11:51:43.545191+0000","last_became_peered":"2024-09-21T11:51:43.545191+0000","last_unstale":"2024-09-21T11:53:41.689252+0000","last_undegraded":"2024-09-21T11:53:41.689252+0000","last_fullsized":"2024-09-21T11:53:41.689252+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.268969+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":600,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.685566+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:53:41.685566+0000","last_peered":"2024-09-21T11:53:41.685566+0000","last_clean":"2024-09-21T11:53:41.685566+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:53:41.685566+0000","last_undegraded":"2024-09-21T11:53:41.685566+0000","last_fullsized":"2024-09-21T11:53:41.685566+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.104956+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":604,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680288+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:53:41.680288+0000","last_peered":"2024-09-21T11:53:41.680288+0000","last_clean":"2024-09-21T11:53:41.680288+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:53:41.680288+0000","last_undegraded":"2024-09-21T11:53:41.680288+0000","last_fullsized":"2024-09-21T11:53:41.680288+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.899330+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":98,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683033+0000","last_change":"2024-09-21T11:51:43.524599+0000","last_active":"2024-09-21T11:53:41.683033+0000","last_peered":"2024-09-21T11:53:41.683033+0000","last_clean":"2024-09-21T11:53:41.683033+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:53:41.683033+0000","last_undegraded":"2024-09-21T11:53:41.683033+0000","last_fullsized":"2024-09-21T11:53:41.683033+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:32.302842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.302842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.577507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":586,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.681575+0000","last_change":"2024-09-21T11:50:35.718093+0000","last_active":"2024-09-21T11:53:41.681575+0000","last_peered":"2024-09-21T11:53:41.681575+0000","last_clean":"2024-09-21T11:53:41.681575+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:53:41.681575+0000","last_undegraded":"2024-09-21T11:53:41.681575+0000","last_fullsized":"2024-09-21T11:53:41.681575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:35.717991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.717991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:10.565445+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":719,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.695124+0000","last_change":"2024-09-21T11:50:25.966805+0000","last_active":"2024-09-21T11:53:41.695124+0000","last_peered":"2024-09-21T11:53:41.695124+0000","last_clean":"2024-09-21T11:53:41.695124+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:53:41.695124+0000","last_undegraded":"2024-09-21T11:53:41.695124+0000","last_fullsized":"2024-09-21T11:53:41.695124+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:50:25.966740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.966740+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.907984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":304,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.678170+0000","last_change":"2024-09-21T11:53:23.118402+0000","last_active":"2024-09-21T11:53:41.678170+0000","last_peered":"2024-09-21T11:53:41.678170+0000","last_clean":"2024-09-21T11:53:41.678170+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:53:41.678170+0000","last_undegraded":"2024-09-21T11:53:41.678170+0000","last_fullsized":"2024-09-21T11:53:41.678170+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:23.118315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:23.118315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:39.620390+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":592,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.688449+0000","last_change":"2024-09-21T11:52:44.264180+0000","last_active":"2024-09-21T11:53:41.688449+0000","last_peered":"2024-09-21T11:53:41.688449+0000","last_clean":"2024-09-21T11:53:41.688449+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:53:41.688449+0000","last_undegraded":"2024-09-21T11:53:41.688449+0000","last_fullsized":"2024-09-21T11:53:41.688449+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.264114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.264114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.220096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":411,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.677953+0000","last_change":"2024-09-21T11:51:43.543408+0000","last_active":"2024-09-21T11:53:41.677953+0000","last_peered":"2024-09-21T11:53:41.677953+0000","last_clean":"2024-09-21T11:53:41.677953+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:53:41.677953+0000","last_undegraded":"2024-09-21T11:53:41.677953+0000","last_fullsized":"2024-09-21T11:53:41.677953+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:50.122227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:50.122227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.038540+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":661,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.690867+0000","last_change":"2024-09-21T11:51:12.652049+0000","last_active":"2024-09-21T11:53:41.690867+0000","last_peered":"2024-09-21T11:53:41.690867+0000","last_clean":"2024-09-21T11:53:41.690867+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:53:41.690867+0000","last_undegraded":"2024-09-21T11:53:41.690867+0000","last_fullsized":"2024-09-21T11:53:41.690867+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:51:12.651967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.651967+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:35.382332+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":97,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.691696+0000","last_change":"2024-09-21T11:51:43.549287+0000","last_active":"2024-09-21T11:53:41.691696+0000","last_peered":"2024-09-21T11:53:41.691696+0000","last_clean":"2024-09-21T11:53:41.691696+0000","last_became_active":"2024-09-21T11:51:43.548726+0000","last_became_peered":"2024-09-21T11:51:43.548726+0000","last_unstale":"2024-09-21T11:53:41.691696+0000","last_undegraded":"2024-09-21T11:53:41.691696+0000","last_fullsized":"2024-09-21T11:53:41.691696+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.908339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.908339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":108,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.684267+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:53:41.684267+0000","last_peered":"2024-09-21T11:53:41.684267+0000","last_clean":"2024-09-21T11:53:41.684267+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:53:41.684267+0000","last_undegraded":"2024-09-21T11:53:41.684267+0000","last_fullsized":"2024-09-21T11:53:41.684267+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.632356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":300,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.677616+0000","last_change":"2024-09-21T11:53:29.154739+0000","last_active":"2024-09-21T11:53:41.677616+0000","last_peered":"2024-09-21T11:53:41.677616+0000","last_clean":"2024-09-21T11:53:41.677616+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:53:41.677616+0000","last_undegraded":"2024-09-21T11:53:41.677616+0000","last_fullsized":"2024-09-21T11:53:41.677616+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:29.154688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.154688+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:32.804161+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":637,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680668+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:53:41.680668+0000","last_peered":"2024-09-21T11:53:41.680668+0000","last_clean":"2024-09-21T11:53:41.680668+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:53:41.680668+0000","last_undegraded":"2024-09-21T11:53:41.680668+0000","last_fullsized":"2024-09-21T11:53:41.680668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:32.227739+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":788,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683465+0000","last_change":"2024-09-21T11:51:44.478918+0000","last_active":"2024-09-21T11:53:41.683465+0000","last_peered":"2024-09-21T11:53:41.683465+0000","last_clean":"2024-09-21T11:53:41.683465+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:53:41.683465+0000","last_undegraded":"2024-09-21T11:53:41.683465+0000","last_fullsized":"2024-09-21T11:53:41.683465+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:44.478855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.478855+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.644866+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"336'5382","reported_seq":5439,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.129614+0000","last_change":"2024-09-21T11:52:45.608799+0000","last_active":"2024-09-21T11:53:46.129614+0000","last_peered":"2024-09-21T11:53:46.129614+0000","last_clean":"2024-09-21T11:53:46.129614+0000","last_became_active":"2024-09-21T11:52:45.608448+0000","last_became_peered":"2024-09-21T11:52:45.608448+0000","last_unstale":"2024-09-21T11:53:46.129614+0000","last_undegraded":"2024-09-21T11:53:46.129614+0000","last_fullsized":"2024-09-21T11:53:46.129614+0000","mapping_epoch":313,"log_start":"318'100","ondisk_log_start":"318'100","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5282,"log_dups_size":0,"ondisk_log_size":5282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:51.400025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":352714752,"num_objects":5382,"num_object_clones":0,"num_object_copies":16146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5382,"num_write_kb":344448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":338,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.679009+0000","last_change":"2024-09-21T11:52:39.116264+0000","last_active":"2024-09-21T11:53:41.679009+0000","last_peered":"2024-09-21T11:53:41.679009+0000","last_clean":"2024-09-21T11:53:41.679009+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:53:41.679009+0000","last_undegraded":"2024-09-21T11:53:41.679009+0000","last_fullsized":"2024-09-21T11:53:41.679009+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:52:39.116197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:39.116197+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":626,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.687871+0000","last_change":"2024-09-21T11:53:30.353707+0000","last_active":"2024-09-21T11:53:41.687871+0000","last_peered":"2024-09-21T11:53:41.687871+0000","last_clean":"2024-09-21T11:53:41.687871+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:53:41.687871+0000","last_undegraded":"2024-09-21T11:53:41.687871+0000","last_fullsized":"2024-09-21T11:53:41.687871+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:30.353632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:30.353632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:48.152525+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"294'9","reported_seq":149,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.681679+0000","last_change":"2024-09-21T11:51:43.526079+0000","last_active":"2024-09-21T11:53:41.681679+0000","last_peered":"2024-09-21T11:53:41.681679+0000","last_clean":"2024-09-21T11:53:41.681679+0000","last_became_active":"2024-09-21T11:51:43.525566+0000","last_became_peered":"2024-09-21T11:51:43.525566+0000","last_unstale":"2024-09-21T11:53:41.681679+0000","last_undegraded":"2024-09-21T11:53:41.681679+0000","last_fullsized":"2024-09-21T11:53:41.681679+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:11.932479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:11.932479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.943835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":613,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680467+0000","last_change":"2024-09-21T11:52:38.781242+0000","last_active":"2024-09-21T11:53:41.680467+0000","last_peered":"2024-09-21T11:53:41.680467+0000","last_clean":"2024-09-21T11:53:41.680467+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:53:41.680467+0000","last_undegraded":"2024-09-21T11:53:41.680467+0000","last_fullsized":"2024-09-21T11:53:41.680467+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.781189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.781189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:50.278404+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"336'5455","reported_seq":5512,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.086656+0000","last_change":"2024-09-21T11:52:45.606555+0000","last_active":"2024-09-21T11:53:46.086656+0000","last_peered":"2024-09-21T11:53:46.086656+0000","last_clean":"2024-09-21T11:53:46.086656+0000","last_became_active":"2024-09-21T11:52:45.606226+0000","last_became_peered":"2024-09-21T11:52:45.606226+0000","last_unstale":"2024-09-21T11:53:46.086656+0000","last_undegraded":"2024-09-21T11:53:46.086656+0000","last_fullsized":"2024-09-21T11:53:46.086656+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5455,"log_dups_size":0,"ondisk_log_size":5455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.505126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357498880,"num_objects":5455,"num_object_clones":0,"num_object_copies":16365,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5455,"num_write_kb":349120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":658,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.687664+0000","last_change":"2024-09-21T11:52:47.279483+0000","last_active":"2024-09-21T11:53:41.687664+0000","last_peered":"2024-09-21T11:53:41.687664+0000","last_clean":"2024-09-21T11:53:41.687664+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:53:41.687664+0000","last_undegraded":"2024-09-21T11:53:41.687664+0000","last_fullsized":"2024-09-21T11:53:41.687664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:52:47.279423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:47.279423+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:10.328491+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":612,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.691988+0000","last_change":"2024-09-21T11:52:42.279545+0000","last_active":"2024-09-21T11:53:41.691988+0000","last_peered":"2024-09-21T11:53:41.691988+0000","last_clean":"2024-09-21T11:53:41.691988+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:53:41.691988+0000","last_undegraded":"2024-09-21T11:53:41.691988+0000","last_fullsized":"2024-09-21T11:53:41.691988+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:42.279496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:42.279496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.626370+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":97,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.688441+0000","last_change":"2024-09-21T11:51:43.542323+0000","last_active":"2024-09-21T11:53:41.688441+0000","last_peered":"2024-09-21T11:53:41.688441+0000","last_clean":"2024-09-21T11:53:41.688441+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:53:41.688441+0000","last_undegraded":"2024-09-21T11:53:41.688441+0000","last_fullsized":"2024-09-21T11:53:41.688441+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.940335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.940335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":99,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.694045+0000","last_change":"2024-09-21T11:53:27.999159+0000","last_active":"2024-09-21T11:53:41.694045+0000","last_peered":"2024-09-21T11:53:41.694045+0000","last_clean":"2024-09-21T11:53:41.694045+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:53:41.694045+0000","last_undegraded":"2024-09-21T11:53:41.694045+0000","last_fullsized":"2024-09-21T11:53:41.694045+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.999084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.999084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:45.599646+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"336'5522","reported_seq":5578,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.094991+0000","last_change":"2024-09-21T11:52:45.635813+0000","last_active":"2024-09-21T11:53:46.094991+0000","last_peered":"2024-09-21T11:53:46.094991+0000","last_clean":"2024-09-21T11:53:46.094991+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:53:46.094991+0000","last_undegraded":"2024-09-21T11:53:46.094991+0000","last_fullsized":"2024-09-21T11:53:46.094991+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5522,"log_dups_size":0,"ondisk_log_size":5522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:08.048600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361889792,"num_objects":5522,"num_object_clones":0,"num_object_copies":16566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5522,"num_write_kb":353408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":609,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680321+0000","last_change":"2024-09-21T11:51:43.525478+0000","last_active":"2024-09-21T11:53:41.680321+0000","last_peered":"2024-09-21T11:53:41.680321+0000","last_clean":"2024-09-21T11:53:41.680321+0000","last_became_active":"2024-09-21T11:51:43.523852+0000","last_became_peered":"2024-09-21T11:51:43.523852+0000","last_unstale":"2024-09-21T11:53:41.680321+0000","last_undegraded":"2024-09-21T11:53:41.680321+0000","last_fullsized":"2024-09-21T11:53:41.680321+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:07.916000+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":634,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.681060+0000","last_change":"2024-09-21T11:50:28.657341+0000","last_active":"2024-09-21T11:53:41.681060+0000","last_peered":"2024-09-21T11:53:41.681060+0000","last_clean":"2024-09-21T11:53:41.681060+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:53:41.681060+0000","last_undegraded":"2024-09-21T11:53:41.681060+0000","last_fullsized":"2024-09-21T11:53:41.681060+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:28.657261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.657261+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.614169+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":793,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.687533+0000","last_change":"2024-09-21T11:51:20.675165+0000","last_active":"2024-09-21T11:53:41.687533+0000","last_peered":"2024-09-21T11:53:41.687533+0000","last_clean":"2024-09-21T11:53:41.687533+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:53:41.687533+0000","last_undegraded":"2024-09-21T11:53:41.687533+0000","last_fullsized":"2024-09-21T11:53:41.687533+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:51:20.675101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.675101+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":602,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.685773+0000","last_change":"2024-09-21T11:52:14.010215+0000","last_active":"2024-09-21T11:53:41.685773+0000","last_peered":"2024-09-21T11:53:41.685773+0000","last_clean":"2024-09-21T11:53:41.685773+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:53:41.685773+0000","last_undegraded":"2024-09-21T11:53:41.685773+0000","last_fullsized":"2024-09-21T11:53:41.685773+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:14.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:14.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:42.958931+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"336'5387","reported_seq":5444,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.113575+0000","last_change":"2024-09-21T11:52:45.608507+0000","last_active":"2024-09-21T11:53:46.113575+0000","last_peered":"2024-09-21T11:53:46.113575+0000","last_clean":"2024-09-21T11:53:46.113575+0000","last_became_active":"2024-09-21T11:52:45.608219+0000","last_became_peered":"2024-09-21T11:52:45.608219+0000","last_unstale":"2024-09-21T11:53:46.113575+0000","last_undegraded":"2024-09-21T11:53:46.113575+0000","last_fullsized":"2024-09-21T11:53:46.113575+0000","mapping_epoch":313,"log_start":"318'100","ondisk_log_start":"318'100","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5287,"log_dups_size":0,"ondisk_log_size":5287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.821182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353042432,"num_objects":5387,"num_object_clones":0,"num_object_copies":16161,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5387,"num_write_kb":344768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":324,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.678248+0000","last_change":"2024-09-21T11:53:30.179784+0000","last_active":"2024-09-21T11:53:41.678248+0000","last_peered":"2024-09-21T11:53:41.678248+0000","last_clean":"2024-09-21T11:53:41.678248+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:53:41.678248+0000","last_undegraded":"2024-09-21T11:53:41.678248+0000","last_fullsized":"2024-09-21T11:53:41.678248+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:30.179690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:30.179690+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:59.902366+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":627,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680967+0000","last_change":"2024-09-21T11:52:39.713149+0000","last_active":"2024-09-21T11:53:41.680967+0000","last_peered":"2024-09-21T11:53:41.680967+0000","last_clean":"2024-09-21T11:53:41.680967+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:53:41.680967+0000","last_undegraded":"2024-09-21T11:53:41.680967+0000","last_fullsized":"2024-09-21T11:53:41.680967+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:39.713105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:39.713105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.384706+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":764,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.681144+0000","last_change":"2024-09-21T11:52:37.704950+0000","last_active":"2024-09-21T11:53:41.681144+0000","last_peered":"2024-09-21T11:53:41.681144+0000","last_clean":"2024-09-21T11:53:41.681144+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:53:41.681144+0000","last_undegraded":"2024-09-21T11:53:41.681144+0000","last_fullsized":"2024-09-21T11:53:41.681144+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:52:37.704899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:37.704899+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:53.365928+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":599,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.684939+0000","last_change":"2024-09-21T11:52:43.682014+0000","last_active":"2024-09-21T11:53:41.684939+0000","last_peered":"2024-09-21T11:53:41.684939+0000","last_clean":"2024-09-21T11:53:41.684939+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:53:41.684939+0000","last_undegraded":"2024-09-21T11:53:41.684939+0000","last_fullsized":"2024-09-21T11:53:41.684939+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:43.681959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:43.681959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:47.395198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"336'5313","reported_seq":5370,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.019732+0000","last_change":"2024-09-21T11:52:45.607195+0000","last_active":"2024-09-21T11:53:46.019732+0000","last_peered":"2024-09-21T11:53:46.019732+0000","last_clean":"2024-09-21T11:53:46.019732+0000","last_became_active":"2024-09-21T11:52:45.606865+0000","last_became_peered":"2024-09-21T11:52:45.606865+0000","last_unstale":"2024-09-21T11:53:46.019732+0000","last_undegraded":"2024-09-21T11:53:46.019732+0000","last_fullsized":"2024-09-21T11:53:46.019732+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5313,"log_dups_size":0,"ondisk_log_size":5313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.533011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348192768,"num_objects":5313,"num_object_clones":0,"num_object_copies":15939,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5313,"num_write_kb":340032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":658,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.681284+0000","last_change":"2024-09-21T11:53:22.488389+0000","last_active":"2024-09-21T11:53:41.681284+0000","last_peered":"2024-09-21T11:53:41.681284+0000","last_clean":"2024-09-21T11:53:41.681284+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:53:41.681284+0000","last_undegraded":"2024-09-21T11:53:41.681284+0000","last_fullsized":"2024-09-21T11:53:41.681284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:53:22.488238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:22.488238+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:24.844477+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":297,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682379+0000","last_change":"2024-09-21T11:53:27.160558+0000","last_active":"2024-09-21T11:53:41.682379+0000","last_peered":"2024-09-21T11:53:41.682379+0000","last_clean":"2024-09-21T11:53:41.682379+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:53:41.682379+0000","last_undegraded":"2024-09-21T11:53:41.682379+0000","last_fullsized":"2024-09-21T11:53:41.682379+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.160486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.160486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:38.227144+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":587,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683441+0000","last_change":"2024-09-21T11:51:52.563725+0000","last_active":"2024-09-21T11:53:41.683441+0000","last_peered":"2024-09-21T11:53:41.683441+0000","last_clean":"2024-09-21T11:53:41.683441+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:53:41.683441+0000","last_undegraded":"2024-09-21T11:53:41.683441+0000","last_fullsized":"2024-09-21T11:53:41.683441+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:52.563656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:52.563656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:14.500261+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":297,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.677302+0000","last_change":"2024-09-21T11:52:09.412149+0000","last_active":"2024-09-21T11:53:41.677302+0000","last_peered":"2024-09-21T11:53:41.677302+0000","last_clean":"2024-09-21T11:53:41.677302+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:53:41.677302+0000","last_undegraded":"2024-09-21T11:53:41.677302+0000","last_fullsized":"2024-09-21T11:53:41.677302+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:09.412091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:09.412091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:41.280027+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"336'5355","reported_seq":5412,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.096674+0000","last_change":"2024-09-21T11:52:45.607383+0000","last_active":"2024-09-21T11:53:46.096674+0000","last_peered":"2024-09-21T11:53:46.096674+0000","last_clean":"2024-09-21T11:53:46.096674+0000","last_became_active":"2024-09-21T11:52:45.607000+0000","last_became_peered":"2024-09-21T11:52:45.607000+0000","last_unstale":"2024-09-21T11:53:46.096674+0000","last_undegraded":"2024-09-21T11:53:46.096674+0000","last_fullsized":"2024-09-21T11:53:46.096674+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5355,"log_dups_size":0,"ondisk_log_size":5355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.571620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":350945280,"num_objects":5355,"num_object_clones":0,"num_object_copies":16065,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5355,"num_write_kb":342720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":410,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682269+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:53:41.682269+0000","last_peered":"2024-09-21T11:53:41.682269+0000","last_clean":"2024-09-21T11:53:41.682269+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:53:41.682269+0000","last_undegraded":"2024-09-21T11:53:41.682269+0000","last_fullsized":"2024-09-21T11:53:41.682269+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:15.031330+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":114,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.681655+0000","last_change":"2024-09-21T11:51:44.078378+0000","last_active":"2024-09-21T11:53:41.681655+0000","last_peered":"2024-09-21T11:53:41.681655+0000","last_clean":"2024-09-21T11:53:41.681655+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:53:41.681655+0000","last_undegraded":"2024-09-21T11:53:41.681655+0000","last_fullsized":"2024-09-21T11:53:41.681655+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:44.078299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.078299+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.512892+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":627,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680958+0000","last_change":"2024-09-21T11:51:12.562026+0000","last_active":"2024-09-21T11:53:41.680958+0000","last_peered":"2024-09-21T11:53:41.680958+0000","last_clean":"2024-09-21T11:53:41.680958+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:53:41.680958+0000","last_undegraded":"2024-09-21T11:53:41.680958+0000","last_fullsized":"2024-09-21T11:53:41.680958+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:51:12.561961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.561961+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.897864+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":601,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682972+0000","last_change":"2024-09-21T11:50:37.858401+0000","last_active":"2024-09-21T11:53:41.682972+0000","last_peered":"2024-09-21T11:53:41.682972+0000","last_clean":"2024-09-21T11:53:41.682972+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:53:41.682972+0000","last_undegraded":"2024-09-21T11:53:41.682972+0000","last_fullsized":"2024-09-21T11:53:41.682972+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:04.618872+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":576,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.688303+0000","last_change":"2024-09-21T11:51:49.357728+0000","last_active":"2024-09-21T11:53:41.688303+0000","last_peered":"2024-09-21T11:53:41.688303+0000","last_clean":"2024-09-21T11:53:41.688303+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:53:41.688303+0000","last_undegraded":"2024-09-21T11:53:41.688303+0000","last_fullsized":"2024-09-21T11:53:41.688303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:49.357659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:49.357659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"336'5524","reported_seq":5581,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.303491+0000","last_change":"2024-09-21T11:52:45.605195+0000","last_active":"2024-09-21T11:53:46.303491+0000","last_peered":"2024-09-21T11:53:46.303491+0000","last_clean":"2024-09-21T11:53:46.303491+0000","last_became_active":"2024-09-21T11:52:45.604600+0000","last_became_peered":"2024-09-21T11:52:45.604600+0000","last_unstale":"2024-09-21T11:53:46.303491+0000","last_undegraded":"2024-09-21T11:53:46.303491+0000","last_fullsized":"2024-09-21T11:53:46.303491+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5524,"log_dups_size":0,"ondisk_log_size":5524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:01.765652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362020864,"num_objects":5524,"num_object_clones":0,"num_object_copies":16572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5524,"num_write_kb":353536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":240,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.696269+0000","last_change":"2024-09-21T11:51:41.006710+0000","last_active":"2024-09-21T11:53:41.696269+0000","last_peered":"2024-09-21T11:53:41.696269+0000","last_clean":"2024-09-21T11:53:41.696269+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:53:41.696269+0000","last_undegraded":"2024-09-21T11:53:41.696269+0000","last_fullsized":"2024-09-21T11:53:41.696269+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:51:41.006622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:41.006622+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:18.027646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":301,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.679952+0000","last_change":"2024-09-21T11:51:43.346789+0000","last_active":"2024-09-21T11:53:41.679952+0000","last_peered":"2024-09-21T11:53:41.679952+0000","last_clean":"2024-09-21T11:53:41.679952+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:53:41.679952+0000","last_undegraded":"2024-09-21T11:53:41.679952+0000","last_fullsized":"2024-09-21T11:53:41.679952+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:43.346718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.346718+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.960974+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":637,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.694307+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:53:41.694307+0000","last_peered":"2024-09-21T11:53:41.694307+0000","last_clean":"2024-09-21T11:53:41.694307+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:53:41.694307+0000","last_undegraded":"2024-09-21T11:53:41.694307+0000","last_fullsized":"2024-09-21T11:53:41.694307+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:42.859403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":596,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.695899+0000","last_change":"2024-09-21T11:49:47.008541+0000","last_active":"2024-09-21T11:53:41.695899+0000","last_peered":"2024-09-21T11:53:41.695899+0000","last_clean":"2024-09-21T11:53:41.695899+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:53:41.695899+0000","last_undegraded":"2024-09-21T11:53:41.695899+0000","last_fullsized":"2024-09-21T11:53:41.695899+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.008369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.008369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:59.883485+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":591,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.685872+0000","last_change":"2024-09-21T11:49:54.388131+0000","last_active":"2024-09-21T11:53:41.685872+0000","last_peered":"2024-09-21T11:53:41.685872+0000","last_clean":"2024-09-21T11:53:41.685872+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:53:41.685872+0000","last_undegraded":"2024-09-21T11:53:41.685872+0000","last_fullsized":"2024-09-21T11:53:41.685872+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:54.388068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:54.388068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:33.162173+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"336'5441","reported_seq":5498,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:47.140055+0000","last_change":"2024-09-21T11:52:45.604487+0000","last_active":"2024-09-21T11:53:47.140055+0000","last_peered":"2024-09-21T11:53:47.140055+0000","last_clean":"2024-09-21T11:53:47.140055+0000","last_became_active":"2024-09-21T11:52:45.604244+0000","last_became_peered":"2024-09-21T11:52:45.604244+0000","last_unstale":"2024-09-21T11:53:47.140055+0000","last_undegraded":"2024-09-21T11:53:47.140055+0000","last_fullsized":"2024-09-21T11:53:47.140055+0000","mapping_epoch":313,"log_start":"318'100","ondisk_log_start":"318'100","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.006653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356581376,"num_objects":5441,"num_object_clones":0,"num_object_copies":16323,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5441,"num_write_kb":348224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":677,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680770+0000","last_change":"2024-09-21T11:53:20.438262+0000","last_active":"2024-09-21T11:53:41.680770+0000","last_peered":"2024-09-21T11:53:41.680770+0000","last_clean":"2024-09-21T11:53:41.680770+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:53:41.680770+0000","last_undegraded":"2024-09-21T11:53:41.680770+0000","last_fullsized":"2024-09-21T11:53:41.680770+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:53:20.438174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.438174+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:24.864136+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":611,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682460+0000","last_change":"2024-09-21T11:51:20.858177+0000","last_active":"2024-09-21T11:53:41.682460+0000","last_peered":"2024-09-21T11:53:41.682460+0000","last_clean":"2024-09-21T11:53:41.682460+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:53:41.682460+0000","last_undegraded":"2024-09-21T11:53:41.682460+0000","last_fullsized":"2024-09-21T11:53:41.682460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.858109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.858109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.570964+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1463,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:45.071734+0000","last_change":"2024-09-21T11:49:51.434402+0000","last_active":"2024-09-21T11:53:45.071734+0000","last_peered":"2024-09-21T11:53:45.071734+0000","last_clean":"2024-09-21T11:53:45.071734+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:53:45.071734+0000","last_undegraded":"2024-09-21T11:53:45.071734+0000","last_fullsized":"2024-09-21T11:53:45.071734+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:51.434333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:49:51.434333+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:38.009881+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":817,"num_read_kb":812,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":609,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682586+0000","last_change":"2024-09-21T11:51:17.645920+0000","last_active":"2024-09-21T11:53:41.682586+0000","last_peered":"2024-09-21T11:53:41.682586+0000","last_clean":"2024-09-21T11:53:41.682586+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:53:41.682586+0000","last_undegraded":"2024-09-21T11:53:41.682586+0000","last_fullsized":"2024-09-21T11:53:41.682586+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.645823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.645823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:17.901222+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":305,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682140+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:53:41.682140+0000","last_peered":"2024-09-21T11:53:41.682140+0000","last_clean":"2024-09-21T11:53:41.682140+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:53:41.682140+0000","last_undegraded":"2024-09-21T11:53:41.682140+0000","last_fullsized":"2024-09-21T11:53:41.682140+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.718884+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"336'5337","reported_seq":5394,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.088711+0000","last_change":"2024-09-21T11:52:45.607096+0000","last_active":"2024-09-21T11:53:46.088711+0000","last_peered":"2024-09-21T11:53:46.088711+0000","last_clean":"2024-09-21T11:53:46.088711+0000","last_became_active":"2024-09-21T11:52:45.606777+0000","last_became_peered":"2024-09-21T11:52:45.606777+0000","last_unstale":"2024-09-21T11:53:46.088711+0000","last_undegraded":"2024-09-21T11:53:46.088711+0000","last_fullsized":"2024-09-21T11:53:46.088711+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.742069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":349765632,"num_objects":5337,"num_object_clones":0,"num_object_copies":16011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5337,"num_write_kb":341568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":805,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682885+0000","last_change":"2024-09-21T11:50:29.389640+0000","last_active":"2024-09-21T11:53:41.682885+0000","last_peered":"2024-09-21T11:53:41.682885+0000","last_clean":"2024-09-21T11:53:41.682885+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:53:41.682885+0000","last_undegraded":"2024-09-21T11:53:41.682885+0000","last_fullsized":"2024-09-21T11:53:41.682885+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:29.389555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:29.389555+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:33.540476+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":604,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.688704+0000","last_change":"2024-09-21T11:50:59.680952+0000","last_active":"2024-09-21T11:53:41.688704+0000","last_peered":"2024-09-21T11:53:41.688704+0000","last_clean":"2024-09-21T11:53:41.688704+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:53:41.688704+0000","last_undegraded":"2024-09-21T11:53:41.688704+0000","last_fullsized":"2024-09-21T11:53:41.688704+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:59.680755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:59.680755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:03.232600+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":666,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.686385+0000","last_change":"2024-09-21T11:52:53.118045+0000","last_active":"2024-09-21T11:53:41.686385+0000","last_peered":"2024-09-21T11:53:41.686385+0000","last_clean":"2024-09-21T11:53:41.686385+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:53:41.686385+0000","last_undegraded":"2024-09-21T11:53:41.686385+0000","last_fullsized":"2024-09-21T11:53:41.686385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:52:53.117963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:52:53.117963+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:07.290308+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":603,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.686381+0000","last_change":"2024-09-21T11:51:18.318095+0000","last_active":"2024-09-21T11:53:41.686381+0000","last_peered":"2024-09-21T11:53:41.686381+0000","last_clean":"2024-09-21T11:53:41.686381+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:53:41.686381+0000","last_undegraded":"2024-09-21T11:53:41.686381+0000","last_fullsized":"2024-09-21T11:53:41.686381+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.317950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.317950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.110119+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":599,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682449+0000","last_change":"2024-09-21T11:53:31.966692+0000","last_active":"2024-09-21T11:53:41.682449+0000","last_peered":"2024-09-21T11:53:41.682449+0000","last_clean":"2024-09-21T11:53:41.682449+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:53:41.682449+0000","last_undegraded":"2024-09-21T11:53:41.682449+0000","last_fullsized":"2024-09-21T11:53:41.682449+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:31.966508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:31.966508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:43.384454+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"336'5495","reported_seq":5552,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.088116+0000","last_change":"2024-09-21T11:52:45.605911+0000","last_active":"2024-09-21T11:53:46.088116+0000","last_peered":"2024-09-21T11:53:46.088116+0000","last_clean":"2024-09-21T11:53:46.088116+0000","last_became_active":"2024-09-21T11:52:45.605575+0000","last_became_peered":"2024-09-21T11:52:45.605575+0000","last_unstale":"2024-09-21T11:53:46.088116+0000","last_undegraded":"2024-09-21T11:53:46.088116+0000","last_fullsized":"2024-09-21T11:53:46.088116+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5495,"log_dups_size":0,"ondisk_log_size":5495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:48.712206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360120320,"num_objects":5495,"num_object_clones":0,"num_object_copies":16485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5495,"num_write_kb":351680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":612,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683084+0000","last_change":"2024-09-21T11:51:43.526788+0000","last_active":"2024-09-21T11:53:41.683084+0000","last_peered":"2024-09-21T11:53:41.683084+0000","last_clean":"2024-09-21T11:53:41.683084+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:53:41.683084+0000","last_undegraded":"2024-09-21T11:53:41.683084+0000","last_fullsized":"2024-09-21T11:53:41.683084+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:37.352834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.352834+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.596503+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":105,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.694086+0000","last_change":"2024-09-21T11:52:13.709151+0000","last_active":"2024-09-21T11:53:41.694086+0000","last_peered":"2024-09-21T11:53:41.694086+0000","last_clean":"2024-09-21T11:53:41.694086+0000","last_became_active":"2024-09-21T11:51:43.532027+0000","last_became_peered":"2024-09-21T11:51:43.532027+0000","last_unstale":"2024-09-21T11:53:41.694086+0000","last_undegraded":"2024-09-21T11:53:41.694086+0000","last_fullsized":"2024-09-21T11:53:41.694086+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:13.709074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:13.709074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:38.032513+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":587,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682310+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:53:41.682310+0000","last_peered":"2024-09-21T11:53:41.682310+0000","last_clean":"2024-09-21T11:53:41.682310+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:53:41.682310+0000","last_undegraded":"2024-09-21T11:53:41.682310+0000","last_fullsized":"2024-09-21T11:53:41.682310+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:00.863312+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":598,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.691604+0000","last_change":"2024-09-21T11:53:36.649688+0000","last_active":"2024-09-21T11:53:41.691604+0000","last_peered":"2024-09-21T11:53:41.691604+0000","last_clean":"2024-09-21T11:53:41.691604+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:53:41.691604+0000","last_undegraded":"2024-09-21T11:53:41.691604+0000","last_fullsized":"2024-09-21T11:53:41.691604+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.664635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.664635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"336'5438","reported_seq":5494,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.505514+0000","last_change":"2024-09-21T11:52:45.638159+0000","last_active":"2024-09-21T11:53:46.505514+0000","last_peered":"2024-09-21T11:53:46.505514+0000","last_clean":"2024-09-21T11:53:46.505514+0000","last_became_active":"2024-09-21T11:52:45.637825+0000","last_became_peered":"2024-09-21T11:52:45.637825+0000","last_unstale":"2024-09-21T11:53:46.505514+0000","last_undegraded":"2024-09-21T11:53:46.505514+0000","last_fullsized":"2024-09-21T11:53:46.505514+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5438,"log_dups_size":0,"ondisk_log_size":5438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:57.377774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356384768,"num_objects":5438,"num_object_clones":0,"num_object_copies":16314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5438,"num_write_kb":348032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":683,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.695547+0000","last_change":"2024-09-21T11:51:43.533280+0000","last_active":"2024-09-21T11:53:41.695547+0000","last_peered":"2024-09-21T11:53:41.695547+0000","last_clean":"2024-09-21T11:53:41.695547+0000","last_became_active":"2024-09-21T11:51:43.525941+0000","last_became_peered":"2024-09-21T11:51:43.525941+0000","last_unstale":"2024-09-21T11:53:41.695547+0000","last_undegraded":"2024-09-21T11:53:41.695547+0000","last_fullsized":"2024-09-21T11:53:41.695547+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:50:36.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.385902+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:07.872467+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":625,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683888+0000","last_change":"2024-09-21T11:51:14.312046+0000","last_active":"2024-09-21T11:53:41.683888+0000","last_peered":"2024-09-21T11:53:41.683888+0000","last_clean":"2024-09-21T11:53:41.683888+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:53:41.683888+0000","last_undegraded":"2024-09-21T11:53:41.683888+0000","last_fullsized":"2024-09-21T11:53:41.683888+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:14.311953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:14.311953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:34.537699+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":296,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682059+0000","last_change":"2024-09-21T11:52:44.976410+0000","last_active":"2024-09-21T11:53:41.682059+0000","last_peered":"2024-09-21T11:53:41.682059+0000","last_clean":"2024-09-21T11:53:41.682059+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:53:41.682059+0000","last_undegraded":"2024-09-21T11:53:41.682059+0000","last_fullsized":"2024-09-21T11:53:41.682059+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.976363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.976363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.128300+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":600,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.696400+0000","last_change":"2024-09-21T11:51:47.944684+0000","last_active":"2024-09-21T11:53:41.696400+0000","last_peered":"2024-09-21T11:53:41.696400+0000","last_clean":"2024-09-21T11:53:41.696400+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:53:41.696400+0000","last_undegraded":"2024-09-21T11:53:41.696400+0000","last_fullsized":"2024-09-21T11:53:41.696400+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:47.944609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.944609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.995897+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"336'5452","reported_seq":5509,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.399174+0000","last_change":"2024-09-21T11:52:45.601642+0000","last_active":"2024-09-21T11:53:46.399174+0000","last_peered":"2024-09-21T11:53:46.399174+0000","last_clean":"2024-09-21T11:53:46.399174+0000","last_became_active":"2024-09-21T11:52:45.601350+0000","last_became_peered":"2024-09-21T11:52:45.601350+0000","last_unstale":"2024-09-21T11:53:46.399174+0000","last_undegraded":"2024-09-21T11:53:46.399174+0000","last_fullsized":"2024-09-21T11:53:46.399174+0000","mapping_epoch":313,"log_start":"318'100","ondisk_log_start":"318'100","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5352,"log_dups_size":0,"ondisk_log_size":5352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:47.738705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357302272,"num_objects":5452,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5452,"num_write_kb":348928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":597,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.691920+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:53:41.691920+0000","last_peered":"2024-09-21T11:53:41.691920+0000","last_clean":"2024-09-21T11:53:41.691920+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:53:41.691920+0000","last_undegraded":"2024-09-21T11:53:41.691920+0000","last_fullsized":"2024-09-21T11:53:41.691920+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.147557+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":596,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.689375+0000","last_change":"2024-09-21T11:53:22.315543+0000","last_active":"2024-09-21T11:53:41.689375+0000","last_peered":"2024-09-21T11:53:41.689375+0000","last_clean":"2024-09-21T11:53:41.689375+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:53:41.689375+0000","last_undegraded":"2024-09-21T11:53:41.689375+0000","last_fullsized":"2024-09-21T11:53:41.689375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:22.315149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:22.315149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:31.277102+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":786,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.695493+0000","last_change":"2024-09-21T11:53:31.991392+0000","last_active":"2024-09-21T11:53:41.695493+0000","last_peered":"2024-09-21T11:53:41.695493+0000","last_clean":"2024-09-21T11:53:41.695493+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:53:41.695493+0000","last_undegraded":"2024-09-21T11:53:41.695493+0000","last_fullsized":"2024-09-21T11:53:41.695493+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:53:31.991316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:31.991316+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:39.513662+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":416,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680556+0000","last_change":"2024-09-21T11:52:52.981662+0000","last_active":"2024-09-21T11:53:41.680556+0000","last_peered":"2024-09-21T11:53:41.680556+0000","last_clean":"2024-09-21T11:53:41.680556+0000","last_became_active":"2024-09-21T11:51:43.543072+0000","last_became_peered":"2024-09-21T11:51:43.543072+0000","last_unstale":"2024-09-21T11:53:41.680556+0000","last_undegraded":"2024-09-21T11:53:41.680556+0000","last_fullsized":"2024-09-21T11:53:41.680556+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:52.981586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:52.981586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.522344+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"336'5429","reported_seq":5486,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.520408+0000","last_change":"2024-09-21T11:52:45.635909+0000","last_active":"2024-09-21T11:53:46.520408+0000","last_peered":"2024-09-21T11:53:46.520408+0000","last_clean":"2024-09-21T11:53:46.520408+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:53:46.520408+0000","last_undegraded":"2024-09-21T11:53:46.520408+0000","last_fullsized":"2024-09-21T11:53:46.520408+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5429,"log_dups_size":0,"ondisk_log_size":5429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.650065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":355794944,"num_objects":5429,"num_object_clones":0,"num_object_copies":16287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5429,"num_write_kb":347456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":642,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.681193+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:53:41.681193+0000","last_peered":"2024-09-21T11:53:41.681193+0000","last_clean":"2024-09-21T11:53:41.681193+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:53:41.681193+0000","last_undegraded":"2024-09-21T11:53:41.681193+0000","last_fullsized":"2024-09-21T11:53:41.681193+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:01.621332+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":619,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.681418+0000","last_change":"2024-09-21T11:52:51.850181+0000","last_active":"2024-09-21T11:53:41.681418+0000","last_peered":"2024-09-21T11:53:41.681418+0000","last_clean":"2024-09-21T11:53:41.681418+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:53:41.681418+0000","last_undegraded":"2024-09-21T11:53:41.681418+0000","last_fullsized":"2024-09-21T11:53:41.681418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:51.850120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:51.850120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:52.704982+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":596,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.694269+0000","last_change":"2024-09-21T11:53:26.990691+0000","last_active":"2024-09-21T11:53:41.694269+0000","last_peered":"2024-09-21T11:53:41.694269+0000","last_clean":"2024-09-21T11:53:41.694269+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:53:41.694269+0000","last_undegraded":"2024-09-21T11:53:41.694269+0000","last_fullsized":"2024-09-21T11:53:41.694269+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.990626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.990626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:44.133530+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":583,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682379+0000","last_change":"2024-09-21T11:53:31.442098+0000","last_active":"2024-09-21T11:53:41.682379+0000","last_peered":"2024-09-21T11:53:41.682379+0000","last_clean":"2024-09-21T11:53:41.682379+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:53:41.682379+0000","last_undegraded":"2024-09-21T11:53:41.682379+0000","last_fullsized":"2024-09-21T11:53:41.682379+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:31.442027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:31.442027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:44.000546+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"336'5495","reported_seq":5552,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:47.050750+0000","last_change":"2024-09-21T11:52:45.636586+0000","last_active":"2024-09-21T11:53:47.050750+0000","last_peered":"2024-09-21T11:53:47.050750+0000","last_clean":"2024-09-21T11:53:47.050750+0000","last_became_active":"2024-09-21T11:52:45.636291+0000","last_became_peered":"2024-09-21T11:52:45.636291+0000","last_unstale":"2024-09-21T11:53:47.050750+0000","last_undegraded":"2024-09-21T11:53:47.050750+0000","last_fullsized":"2024-09-21T11:53:47.050750+0000","mapping_epoch":313,"log_start":"318'200","ondisk_log_start":"318'200","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5295,"log_dups_size":0,"ondisk_log_size":5295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.940714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":360120320,"num_objects":5495,"num_object_clones":0,"num_object_copies":16485,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5495,"num_write_kb":351680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":716,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683771+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:53:41.683771+0000","last_peered":"2024-09-21T11:53:41.683771+0000","last_clean":"2024-09-21T11:53:41.683771+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:53:41.683771+0000","last_undegraded":"2024-09-21T11:53:41.683771+0000","last_fullsized":"2024-09-21T11:53:41.683771+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.597026+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":603,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.686273+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:53:41.686273+0000","last_peered":"2024-09-21T11:53:41.686273+0000","last_clean":"2024-09-21T11:53:41.686273+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:53:41.686273+0000","last_undegraded":"2024-09-21T11:53:41.686273+0000","last_fullsized":"2024-09-21T11:53:41.686273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:38.964046+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":612,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.696254+0000","last_change":"2024-09-21T11:51:48.941202+0000","last_active":"2024-09-21T11:53:41.696254+0000","last_peered":"2024-09-21T11:53:41.696254+0000","last_clean":"2024-09-21T11:53:41.696254+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:53:41.696254+0000","last_undegraded":"2024-09-21T11:53:41.696254+0000","last_fullsized":"2024-09-21T11:53:41.696254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:48.941128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:48.941128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.021684+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":599,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.686294+0000","last_change":"2024-09-21T11:52:12.100354+0000","last_active":"2024-09-21T11:53:41.686294+0000","last_peered":"2024-09-21T11:53:41.686294+0000","last_clean":"2024-09-21T11:53:41.686294+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:53:41.686294+0000","last_undegraded":"2024-09-21T11:53:41.686294+0000","last_fullsized":"2024-09-21T11:53:41.686294+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:39.968435+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"336'5466","reported_seq":5523,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.095025+0000","last_change":"2024-09-21T11:52:45.636994+0000","last_active":"2024-09-21T11:53:46.095025+0000","last_peered":"2024-09-21T11:53:46.095025+0000","last_clean":"2024-09-21T11:53:46.095025+0000","last_became_active":"2024-09-21T11:52:45.636672+0000","last_became_peered":"2024-09-21T11:52:45.636672+0000","last_unstale":"2024-09-21T11:53:46.095025+0000","last_undegraded":"2024-09-21T11:53:46.095025+0000","last_fullsized":"2024-09-21T11:53:46.095025+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5466,"log_dups_size":0,"ondisk_log_size":5466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.462991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358219776,"num_objects":5466,"num_object_clones":0,"num_object_copies":16398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5466,"num_write_kb":349824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":637,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.677473+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:53:41.677473+0000","last_peered":"2024-09-21T11:53:41.677473+0000","last_clean":"2024-09-21T11:53:41.677473+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:53:41.677473+0000","last_undegraded":"2024-09-21T11:53:41.677473+0000","last_fullsized":"2024-09-21T11:53:41.677473+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:11.016882+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":633,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.686457+0000","last_change":"2024-09-21T11:49:08.611218+0000","last_active":"2024-09-21T11:53:41.686457+0000","last_peered":"2024-09-21T11:53:41.686457+0000","last_clean":"2024-09-21T11:53:41.686457+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:53:41.686457+0000","last_undegraded":"2024-09-21T11:53:41.686457+0000","last_fullsized":"2024-09-21T11:53:41.686457+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.611159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.611159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:56.046273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":618,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.694052+0000","last_change":"2024-09-21T11:49:44.985594+0000","last_active":"2024-09-21T11:53:41.694052+0000","last_peered":"2024-09-21T11:53:41.694052+0000","last_clean":"2024-09-21T11:53:41.694052+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:53:41.694052+0000","last_undegraded":"2024-09-21T11:53:41.694052+0000","last_fullsized":"2024-09-21T11:53:41.694052+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:44.985234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:44.985234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:58.909250+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":606,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.689067+0000","last_change":"2024-09-21T11:51:09.660956+0000","last_active":"2024-09-21T11:53:41.689067+0000","last_peered":"2024-09-21T11:53:41.689067+0000","last_clean":"2024-09-21T11:53:41.689067+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:53:41.689067+0000","last_undegraded":"2024-09-21T11:53:41.689067+0000","last_fullsized":"2024-09-21T11:53:41.689067+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:09.660880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.660880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:41.450757+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"336'5347","reported_seq":5403,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:46.327286+0000","last_change":"2024-09-21T11:52:45.603459+0000","last_active":"2024-09-21T11:53:46.327286+0000","last_peered":"2024-09-21T11:53:46.327286+0000","last_clean":"2024-09-21T11:53:46.327286+0000","last_became_active":"2024-09-21T11:52:45.603101+0000","last_became_peered":"2024-09-21T11:52:45.603101+0000","last_unstale":"2024-09-21T11:53:46.327286+0000","last_undegraded":"2024-09-21T11:53:46.327286+0000","last_fullsized":"2024-09-21T11:53:46.327286+0000","mapping_epoch":313,"log_start":"0'0","ondisk_log_start":"0'0","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5347,"log_dups_size":0,"ondisk_log_size":5347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":350420992,"num_objects":5347,"num_object_clones":0,"num_object_copies":16041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5347,"num_write_kb":342208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":642,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680030+0000","last_change":"2024-09-21T11:53:29.958829+0000","last_active":"2024-09-21T11:53:41.680030+0000","last_peered":"2024-09-21T11:53:41.680030+0000","last_clean":"2024-09-21T11:53:41.680030+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:53:41.680030+0000","last_undegraded":"2024-09-21T11:53:41.680030+0000","last_fullsized":"2024-09-21T11:53:41.680030+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:53:29.958746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.958746+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:48.473922+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":631,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.686351+0000","last_change":"2024-09-21T11:52:43.086438+0000","last_active":"2024-09-21T11:53:41.686351+0000","last_peered":"2024-09-21T11:53:41.686351+0000","last_clean":"2024-09-21T11:53:41.686351+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:53:41.686351+0000","last_undegraded":"2024-09-21T11:53:41.686351+0000","last_fullsized":"2024-09-21T11:53:41.686351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:52:43.086376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:43.086376+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:54.446134+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":769,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.685803+0000","last_change":"2024-09-21T11:52:54.080234+0000","last_active":"2024-09-21T11:53:41.685803+0000","last_peered":"2024-09-21T11:53:41.685803+0000","last_clean":"2024-09-21T11:53:41.685803+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:53:41.685803+0000","last_undegraded":"2024-09-21T11:53:41.685803+0000","last_fullsized":"2024-09-21T11:53:41.685803+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:52:54.080076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:54.080076+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:15.355476+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":591,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680408+0000","last_change":"2024-09-21T11:53:27.992553+0000","last_active":"2024-09-21T11:53:41.680408+0000","last_peered":"2024-09-21T11:53:41.680408+0000","last_clean":"2024-09-21T11:53:41.680408+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:53:41.680408+0000","last_undegraded":"2024-09-21T11:53:41.680408+0000","last_fullsized":"2024-09-21T11:53:41.680408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.992489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.992489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:29.913346+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":618,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683888+0000","last_change":"2024-09-21T11:52:42.454187+0000","last_active":"2024-09-21T11:53:41.683888+0000","last_peered":"2024-09-21T11:53:41.683888+0000","last_clean":"2024-09-21T11:53:41.683888+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:53:41.683888+0000","last_undegraded":"2024-09-21T11:53:41.683888+0000","last_fullsized":"2024-09-21T11:53:41.683888+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:42.454138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:42.454138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:56.161790+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":631,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.691848+0000","last_change":"2024-09-21T11:47:03.428872+0000","last_active":"2024-09-21T11:53:41.691848+0000","last_peered":"2024-09-21T11:53:41.691848+0000","last_clean":"2024-09-21T11:53:41.691848+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:53:41.691848+0000","last_undegraded":"2024-09-21T11:53:41.691848+0000","last_fullsized":"2024-09-21T11:53:41.691848+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-21T11:46:28.677662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:28.677662+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:01.305455+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":624,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.681723+0000","last_change":"2024-09-21T11:53:26.982285+0000","last_active":"2024-09-21T11:53:41.681723+0000","last_peered":"2024-09-21T11:53:41.681723+0000","last_clean":"2024-09-21T11:53:41.681723+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:53:41.681723+0000","last_undegraded":"2024-09-21T11:53:41.681723+0000","last_fullsized":"2024-09-21T11:53:41.681723+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.982143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.982143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:29.099063+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":606,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.685775+0000","last_change":"2024-09-21T11:53:26.139062+0000","last_active":"2024-09-21T11:53:41.685775+0000","last_peered":"2024-09-21T11:53:41.685775+0000","last_clean":"2024-09-21T11:53:41.685775+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:53:41.685775+0000","last_undegraded":"2024-09-21T11:53:41.685775+0000","last_fullsized":"2024-09-21T11:53:41.685775+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.138988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.138988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:31.427930+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":640,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680955+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:53:41.680955+0000","last_peered":"2024-09-21T11:53:41.680955+0000","last_clean":"2024-09-21T11:53:41.680955+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:53:41.680955+0000","last_undegraded":"2024-09-21T11:53:41.680955+0000","last_fullsized":"2024-09-21T11:53:41.680955+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:46.987763+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":602,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.685982+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:53:41.685982+0000","last_peered":"2024-09-21T11:53:41.685982+0000","last_clean":"2024-09-21T11:53:41.685982+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:53:41.685982+0000","last_undegraded":"2024-09-21T11:53:41.685982+0000","last_fullsized":"2024-09-21T11:53:41.685982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.143512+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":617,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.691614+0000","last_change":"2024-09-21T11:52:41.245695+0000","last_active":"2024-09-21T11:53:41.691614+0000","last_peered":"2024-09-21T11:53:41.691614+0000","last_clean":"2024-09-21T11:53:41.691614+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:53:41.691614+0000","last_undegraded":"2024-09-21T11:53:41.691614+0000","last_fullsized":"2024-09-21T11:53:41.691614+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:41.245632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:41.245632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.732253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":599,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.694648+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:53:41.694648+0000","last_peered":"2024-09-21T11:53:41.694648+0000","last_clean":"2024-09-21T11:53:41.694648+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:53:41.694648+0000","last_undegraded":"2024-09-21T11:53:41.694648+0000","last_fullsized":"2024-09-21T11:53:41.694648+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:02.846816+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":623,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680807+0000","last_change":"2024-09-21T11:53:30.947056+0000","last_active":"2024-09-21T11:53:41.680807+0000","last_peered":"2024-09-21T11:53:41.680807+0000","last_clean":"2024-09-21T11:53:41.680807+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:53:41.680807+0000","last_undegraded":"2024-09-21T11:53:41.680807+0000","last_fullsized":"2024-09-21T11:53:41.680807+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:30.946969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:30.946969+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:38.840876+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":618,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.685254+0000","last_change":"2024-09-21T11:52:45.688676+0000","last_active":"2024-09-21T11:53:41.685254+0000","last_peered":"2024-09-21T11:53:41.685254+0000","last_clean":"2024-09-21T11:53:41.685254+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:53:41.685254+0000","last_undegraded":"2024-09-21T11:53:41.685254+0000","last_fullsized":"2024-09-21T11:53:41.685254+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:45.688617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:45.688617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.734406+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":753,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.691816+0000","last_change":"2024-09-21T11:51:08.629172+0000","last_active":"2024-09-21T11:53:41.691816+0000","last_peered":"2024-09-21T11:53:41.691816+0000","last_clean":"2024-09-21T11:53:41.691816+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:53:41.691816+0000","last_undegraded":"2024-09-21T11:53:41.691816+0000","last_fullsized":"2024-09-21T11:53:41.691816+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:08.628986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.628986+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:36.430571+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":97,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.696916+0000","last_change":"2024-09-21T11:51:43.531318+0000","last_active":"2024-09-21T11:53:41.696916+0000","last_peered":"2024-09-21T11:53:41.696916+0000","last_clean":"2024-09-21T11:53:41.696916+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:53:41.696916+0000","last_undegraded":"2024-09-21T11:53:41.696916+0000","last_fullsized":"2024-09-21T11:53:41.696916+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.826834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.826834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:09.899928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":659,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.692004+0000","last_change":"2024-09-21T11:53:25.264037+0000","last_active":"2024-09-21T11:53:41.692004+0000","last_peered":"2024-09-21T11:53:41.692004+0000","last_clean":"2024-09-21T11:53:41.692004+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:53:41.692004+0000","last_undegraded":"2024-09-21T11:53:41.692004+0000","last_fullsized":"2024-09-21T11:53:41.692004+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:25.263955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:25.263955+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:48.576226+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":594,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.686240+0000","last_change":"2024-09-21T11:49:58.400078+0000","last_active":"2024-09-21T11:53:41.686240+0000","last_peered":"2024-09-21T11:53:41.686240+0000","last_clean":"2024-09-21T11:53:41.686240+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:53:41.686240+0000","last_undegraded":"2024-09-21T11:53:41.686240+0000","last_fullsized":"2024-09-21T11:53:41.686240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:58.400013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:58.400013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:35.138876+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":799,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680662+0000","last_change":"2024-09-21T11:53:24.947837+0000","last_active":"2024-09-21T11:53:41.680662+0000","last_peered":"2024-09-21T11:53:41.680662+0000","last_clean":"2024-09-21T11:53:41.680662+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:53:41.680662+0000","last_undegraded":"2024-09-21T11:53:41.680662+0000","last_fullsized":"2024-09-21T11:53:41.680662+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:53:24.947744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:24.947744+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:54.797043+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":598,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680412+0000","last_change":"2024-09-21T11:50:02.782565+0000","last_active":"2024-09-21T11:53:41.680412+0000","last_peered":"2024-09-21T11:53:41.680412+0000","last_clean":"2024-09-21T11:53:41.680412+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:53:41.680412+0000","last_undegraded":"2024-09-21T11:53:41.680412+0000","last_fullsized":"2024-09-21T11:53:41.680412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.782475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.782475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:33.407536+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":647,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683152+0000","last_change":"2024-09-21T11:51:43.533694+0000","last_active":"2024-09-21T11:53:41.683152+0000","last_peered":"2024-09-21T11:53:41.683152+0000","last_clean":"2024-09-21T11:53:41.683152+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:53:41.683152+0000","last_undegraded":"2024-09-21T11:53:41.683152+0000","last_fullsized":"2024-09-21T11:53:41.683152+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:04.610554+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":105,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.686009+0000","last_change":"2024-09-21T11:51:52.159555+0000","last_active":"2024-09-21T11:53:41.686009+0000","last_peered":"2024-09-21T11:53:41.686009+0000","last_clean":"2024-09-21T11:53:41.686009+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:53:41.686009+0000","last_undegraded":"2024-09-21T11:53:41.686009+0000","last_fullsized":"2024-09-21T11:53:41.686009+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:52.159433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:52.159433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.229678+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":296,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.678162+0000","last_change":"2024-09-21T11:51:10.143792+0000","last_active":"2024-09-21T11:53:41.678162+0000","last_peered":"2024-09-21T11:53:41.678162+0000","last_clean":"2024-09-21T11:53:41.678162+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:53:41.678162+0000","last_undegraded":"2024-09-21T11:53:41.678162+0000","last_fullsized":"2024-09-21T11:53:41.678162+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.141936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.141936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:35.277541+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":583,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.688348+0000","last_change":"2024-09-21T11:52:48.256878+0000","last_active":"2024-09-21T11:53:41.688348+0000","last_peered":"2024-09-21T11:53:41.688348+0000","last_clean":"2024-09-21T11:53:41.688348+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:53:41.688348+0000","last_undegraded":"2024-09-21T11:53:41.688348+0000","last_fullsized":"2024-09-21T11:53:41.688348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:48.256830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:48.256830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:00.348262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":307,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.678420+0000","last_change":"2024-09-21T11:51:15.200506+0000","last_active":"2024-09-21T11:53:41.678420+0000","last_peered":"2024-09-21T11:53:41.678420+0000","last_clean":"2024-09-21T11:53:41.678420+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:53:41.678420+0000","last_undegraded":"2024-09-21T11:53:41.678420+0000","last_fullsized":"2024-09-21T11:53:41.678420+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:51:15.200430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.200430+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:36.276193+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":602,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.686130+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:53:41.686130+0000","last_peered":"2024-09-21T11:53:41.686130+0000","last_clean":"2024-09-21T11:53:41.686130+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:53:41.686130+0000","last_undegraded":"2024-09-21T11:53:41.686130+0000","last_fullsized":"2024-09-21T11:53:41.686130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:13.179348+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":609,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.689216+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:53:41.689216+0000","last_peered":"2024-09-21T11:53:41.689216+0000","last_clean":"2024-09-21T11:53:41.689216+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:53:41.689216+0000","last_undegraded":"2024-09-21T11:53:41.689216+0000","last_fullsized":"2024-09-21T11:53:41.689216+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:05.222459+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":621,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.684668+0000","last_change":"2024-09-21T11:51:16.962933+0000","last_active":"2024-09-21T11:53:41.684668+0000","last_peered":"2024-09-21T11:53:41.684668+0000","last_clean":"2024-09-21T11:53:41.684668+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:53:41.684668+0000","last_undegraded":"2024-09-21T11:53:41.684668+0000","last_fullsized":"2024-09-21T11:53:41.684668+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:51:16.962860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:16.962860+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:06.599324+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":638,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.688528+0000","last_change":"2024-09-21T11:53:32.308905+0000","last_active":"2024-09-21T11:53:41.688528+0000","last_peered":"2024-09-21T11:53:41.688528+0000","last_clean":"2024-09-21T11:53:41.688528+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:53:41.688528+0000","last_undegraded":"2024-09-21T11:53:41.688528+0000","last_fullsized":"2024-09-21T11:53:41.688528+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:32.308827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:32.308827+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:50.813704+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":624,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683803+0000","last_change":"2024-09-21T11:53:28.945615+0000","last_active":"2024-09-21T11:53:41.683803+0000","last_peered":"2024-09-21T11:53:41.683803+0000","last_clean":"2024-09-21T11:53:41.683803+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:53:41.683803+0000","last_undegraded":"2024-09-21T11:53:41.683803+0000","last_fullsized":"2024-09-21T11:53:41.683803+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:28.945564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:28.945564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:36.503501+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":586,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.684289+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:53:41.684289+0000","last_peered":"2024-09-21T11:53:41.684289+0000","last_clean":"2024-09-21T11:53:41.684289+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:53:41.684289+0000","last_undegraded":"2024-09-21T11:53:41.684289+0000","last_fullsized":"2024-09-21T11:53:41.684289+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:00.087872+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":591,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.689309+0000","last_change":"2024-09-21T11:51:05.711964+0000","last_active":"2024-09-21T11:53:41.689309+0000","last_peered":"2024-09-21T11:53:41.689309+0000","last_clean":"2024-09-21T11:53:41.689309+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:53:41.689309+0000","last_undegraded":"2024-09-21T11:53:41.689309+0000","last_fullsized":"2024-09-21T11:53:41.689309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:05.711768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:05.711768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:16.204609+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":642,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.687581+0000","last_change":"2024-09-21T11:47:03.406606+0000","last_active":"2024-09-21T11:53:41.687581+0000","last_peered":"2024-09-21T11:53:41.687581+0000","last_clean":"2024-09-21T11:53:41.687581+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:53:41.687581+0000","last_undegraded":"2024-09-21T11:53:41.687581+0000","last_fullsized":"2024-09-21T11:53:41.687581+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-21T11:46:51.935389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:46:51.935389+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:56.189942+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":594,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680741+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:53:41.680741+0000","last_peered":"2024-09-21T11:53:41.680741+0000","last_clean":"2024-09-21T11:53:41.680741+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:53:41.680741+0000","last_undegraded":"2024-09-21T11:53:41.680741+0000","last_fullsized":"2024-09-21T11:53:41.680741+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:37.370731+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":594,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.682413+0000","last_change":"2024-09-21T11:50:38.405183+0000","last_active":"2024-09-21T11:53:41.682413+0000","last_peered":"2024-09-21T11:53:41.682413+0000","last_clean":"2024-09-21T11:53:41.682413+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:53:41.682413+0000","last_undegraded":"2024-09-21T11:53:41.682413+0000","last_fullsized":"2024-09-21T11:53:41.682413+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:38.405118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.405118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:08.616042+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":590,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.685481+0000","last_change":"2024-09-21T11:52:45.065651+0000","last_active":"2024-09-21T11:53:41.685481+0000","last_peered":"2024-09-21T11:53:41.685481+0000","last_clean":"2024-09-21T11:53:41.685481+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:53:41.685481+0000","last_undegraded":"2024-09-21T11:53:41.685481+0000","last_fullsized":"2024-09-21T11:53:41.685481+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:45.065594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:45.065594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:53:50.236288+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":304,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.678037+0000","last_change":"2024-09-21T11:52:12.348687+0000","last_active":"2024-09-21T11:53:41.678037+0000","last_peered":"2024-09-21T11:53:41.678037+0000","last_clean":"2024-09-21T11:53:41.678037+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:53:41.678037+0000","last_undegraded":"2024-09-21T11:53:41.678037+0000","last_fullsized":"2024-09-21T11:53:41.678037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:18.010458+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":617,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.687797+0000","last_change":"2024-09-21T11:51:04.656939+0000","last_active":"2024-09-21T11:53:41.687797+0000","last_peered":"2024-09-21T11:53:41.687797+0000","last_clean":"2024-09-21T11:53:41.687797+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:53:41.687797+0000","last_undegraded":"2024-09-21T11:53:41.687797+0000","last_fullsized":"2024-09-21T11:53:41.687797+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:04.656870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:04.656870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:04.247547+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":692,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.683511+0000","last_change":"2024-09-21T11:53:20.555366+0000","last_active":"2024-09-21T11:53:41.683511+0000","last_peered":"2024-09-21T11:53:41.683511+0000","last_clean":"2024-09-21T11:53:41.683511+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:53:41.683511+0000","last_undegraded":"2024-09-21T11:53:41.683511+0000","last_fullsized":"2024-09-21T11:53:41.683511+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:20.554779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.554779+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:25.003905+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":603,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-21T11:53:41.680577+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:53:41.680577+0000","last_peered":"2024-09-21T11:53:41.680577+0000","last_clean":"2024-09-21T11:53:41.680577+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:53:41.680577+0000","last_undegraded":"2024-09-21T11:53:41.680577+0000","last_fullsized":"2024-09-21T11:53:41.680577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:32.426448+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":5691015168,"num_objects":86838,"num_object_clones":0,"num_object_copies":260514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86838,"num_write_kb":5557632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17113088000,"data_stored":17113088000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86238,"ondisk_log_size":86238,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1113,"num_read_kb":870,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":817,"num_read_kb":812,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234154,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1950332,"kb_used_data":1387436,"kb_used_omap":1,"kb_used_meta":562878,"kb_avail":91794820,"statfs":{"total":95995035648,"available":93997895680,"internally_reserved":0,"allocated":1420734464,"data_stored":1420220500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":576387530},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463087,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2820016,"kb_used_data":2413984,"kb_used_omap":1,"kb_used_meta":406014,"kb_avail":90925136,"statfs":{"total":95995035648,"available":93107339264,"internally_reserved":0,"allocated":2471919616,"data_stored":2471418284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":415758790},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724724,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2257532,"kb_used_data":1734892,"kb_used_omap":1,"kb_used_meta":522622,"kb_avail":91487620,"statfs":{"total":95995035648,"available":93683322880,"internally_reserved":0,"allocated":1776529408,"data_stored":1776014252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":535165382},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920954,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3183876,"kb_used_data":2755636,"kb_used_omap":1,"kb_used_meta":428222,"kb_avail":90561276,"statfs":{"total":95995035648,"available":92734746624,"internally_reserved":0,"allocated":2821771264,"data_stored":2821249077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":438499781},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117184,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2345076,"kb_used_data":1750244,"kb_used_omap":1,"kb_used_meta":594814,"kb_avail":91400076,"statfs":{"total":95995035648,"available":93593677824,"internally_reserved":0,"allocated":1792249856,"data_stored":1791742878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":609089991},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411526,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2643960,"kb_used_data":2093544,"kb_used_omap":1,"kb_used_meta":550398,"kb_avail":91101192,"statfs":{"total":95995035648,"available":93287620608,"internally_reserved":0,"allocated":2143789056,"data_stored":2143278693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":563608005},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575053,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1567000,"kb_used_data":1044360,"kb_used_omap":1,"kb_used_meta":522622,"kb_avail":92178152,"statfs":{"total":95995035648,"available":94390427648,"internally_reserved":0,"allocated":1069424640,"data_stored":1068882059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":535165382},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738579,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3958924,"kb_used_data":3450172,"kb_used_omap":1,"kb_used_meta":508734,"kb_avail":89786228,"statfs":{"total":95995035648,"available":91941097472,"internally_reserved":0,"allocated":3532976128,"data_stored":3532446898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":520944069},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3568107520,"data_stored":3568107520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1068826624,"data_stored":1068826624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2150825984,"data_stored":2150825984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1809907712,"data_stored":1809907712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2829516800,"data_stored":2829516800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1778843648,"data_stored":1778843648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2486829056,"data_stored":2486829056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1420230656,"data_stored":1420230656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:53:48.663 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [4, 0, 6] 2024-09-21T11:53:48.663 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.3', '4', '0', '6'] 2024-09-21T11:53:48.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pg-upmap 3.3 4 0 6 2024-09-21T11:53:48.726 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:48 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:53:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:48 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/874424542' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:53:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:48 smithi175 ceph-mon[32613]: osdmap e337: 8 total, 8 up, 8 in 2024-09-21T11:53:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:48 smithi175 ceph-mon[32613]: from='client.16170 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:53:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:48 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2008291841' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:48.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:48.807 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:48.907 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:48.937 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:48.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:48.938 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:49.039 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:49.077 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:49.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:49.078 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:49.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:49.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:49.202 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:49.202 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:49.302 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:49.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 57 16 90048 90032 98.6973 68.8125 0.00969234 0.0101271 2024-09-21T11:53:49.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:49.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:49.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:49.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:49.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:49.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:49.622 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:49 smithi175 ceph-mon[32613]: pgmap v666: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-21T11:53:49.622 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:49 smithi175 ceph-mon[32613]: 6.0 scrub starts 2024-09-21T11:53:49.622 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:49 smithi175 ceph-mon[32613]: 6.0 scrub ok 2024-09-21T11:53:49.630 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:49.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:49.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:49.679 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:49.679 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:49.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:49.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:49 smithi043 ceph-mon[36454]: pgmap v666: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-21T11:53:49.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:49 smithi043 ceph-mon[36454]: 6.0 scrub starts 2024-09-21T11:53:49.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:49 smithi043 ceph-mon[36454]: 6.0 scrub ok 2024-09-21T11:53:49.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:49 smithi043 ceph-mon[29710]: pgmap v666: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-21T11:53:49.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:49 smithi043 ceph-mon[29710]: 6.0 scrub starts 2024-09-21T11:53:49.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:49 smithi043 ceph-mon[29710]: 6.0 scrub ok 2024-09-21T11:53:49.771 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:49.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:49.772 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:49.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:49.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:49.905 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:50.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:50.036 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:50.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:50.037 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:50.138 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:50.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:50.183 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:50.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:50.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:50.314 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:50.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:50.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:50.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 58 16 91057 91041 98.0829 63.0625 0.0509498 0.01019 2024-09-21T11:53:50.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:50.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:50.450 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:50.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:50.586 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:50.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:50.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:50.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:50.693 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:50 smithi043 ceph-mon[29710]: 10.6 scrub starts 2024-09-21T11:53:50.693 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:50 smithi043 ceph-mon[36454]: 10.6 scrub starts 2024-09-21T11:53:50.709 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:50.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:50.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:50.810 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:50.831 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:50.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:50.832 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:50.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:50.975 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:50.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:50.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:50 smithi175 ceph-mon[32613]: 10.6 scrub starts 2024-09-21T11:53:51.077 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:51.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:51.101 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:51.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:51.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:51.239 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:51.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:51.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:51.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 59 16 91828 91812 97.2372 48.1875 0.0213827 0.0102782 2024-09-21T11:53:51.341 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:51.384 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:51.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:51.385 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:51.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:53:51.486 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:51.521 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:51.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:51.522 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:set 3.3 pg_upmap mapping to [4,0,6] 2024-09-21T11:53:51.623 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:51.650 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:51.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:51.651 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:51.751 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:51.795 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:51.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:51.796 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:51.821 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:51 smithi175 ceph-mon[32613]: pgmap v667: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 611 B/s rd, 44 MiB/s wr, 702 op/s 2024-09-21T11:53:51.821 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:51 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:53:51.821 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:51 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:53:51.821 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:51 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/719343941' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.3", "id": [4, 0, 6]} : dispatch 2024-09-21T11:53:51.821 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:53:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:53:51.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:53:51.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:51.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:51.962 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:51.963 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:51.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:51 smithi043 ceph-mon[36454]: pgmap v667: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 611 B/s rd, 44 MiB/s wr, 702 op/s 2024-09-21T11:53:51.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:51 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:53:51.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:51 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:53:51.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:51 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/719343941' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.3", "id": [4, 0, 6]} : dispatch 2024-09-21T11:53:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:51 smithi043 ceph-mon[29710]: pgmap v667: 148 pgs: 148 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 611 B/s rd, 44 MiB/s wr, 702 op/s 2024-09-21T11:53:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:51 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:53:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:51 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:53:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:51 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/719343941' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.3", "id": [4, 0, 6]} : dispatch 2024-09-21T11:53:52.063 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:52.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:52.104 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:52.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:52.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:53:52.205 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:52.262 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:52.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:52.263 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:52.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:53:52.314611+0000 min lat: 0.00131482 max lat: 0.315011 avg lat: 0.0103551 2024-09-21T11:53:52.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:53:52.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 60 16 92692 92676 96.5166 54 0.011088 0.0103551 2024-09-21T11:53:52.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:52.394 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:52.406 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:52.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:52.407 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:52.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:52.551 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:52.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:52.552 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:52.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:52.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:52.711 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:52.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:52.841 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:52.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:52.875 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:52.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:52.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:52 smithi175 ceph-mon[32613]: 10.2 scrub starts 2024-09-21T11:53:52.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:52 smithi175 ceph-mon[32613]: 10.6 scrub ok 2024-09-21T11:53:52.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:52 smithi175 ceph-mon[32613]: 10.0 scrub starts 2024-09-21T11:53:52.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:52 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/719343941' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.3", "id": [4, 0, 6]}]': finished 2024-09-21T11:53:52.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:52 smithi175 ceph-mon[32613]: osdmap e338: 8 total, 8 up, 8 in 2024-09-21T11:53:52.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:52.976 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:53.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[29710]: 10.2 scrub starts 2024-09-21T11:53:53.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[29710]: 10.6 scrub ok 2024-09-21T11:53:53.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[29710]: 10.0 scrub starts 2024-09-21T11:53:53.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/719343941' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.3", "id": [4, 0, 6]}]': finished 2024-09-21T11:53:53.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[29710]: osdmap e338: 8 total, 8 up, 8 in 2024-09-21T11:53:53.144 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:53.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[36454]: 10.2 scrub starts 2024-09-21T11:53:53.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[36454]: 10.6 scrub ok 2024-09-21T11:53:53.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[36454]: 10.0 scrub starts 2024-09-21T11:53:53.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/719343941' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.3", "id": [4, 0, 6]}]': finished 2024-09-21T11:53:53.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[36454]: osdmap e338: 8 total, 8 up, 8 in 2024-09-21T11:53:53.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:53:53.181 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:53.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:53.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:53.283 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:53.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 61 16 93631 93615 95.8964 58.6875 0.0321466 0.0104207 2024-09-21T11:53:53.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:53.324 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:53.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:53.404 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:53.404 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:53.425 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:53.450 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:53.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:53.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:53.470 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:53.570 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:53.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:53.602 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:53.602 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:53.703 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:53.736 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:53.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:53.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:53.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:53.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[29710]: pgmap v669: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[29710]: 6.c scrub starts 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[29710]: 6.c scrub ok 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[29710]: 10.2 scrub ok 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[29710]: 6.12 scrub starts 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[29710]: 6.12 scrub ok 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[36454]: pgmap v669: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[36454]: 6.c scrub starts 2024-09-21T11:53:53.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[36454]: 6.c scrub ok 2024-09-21T11:53:53.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[36454]: 10.2 scrub ok 2024-09-21T11:53:53.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[36454]: 6.12 scrub starts 2024-09-21T11:53:53.878 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:53 smithi043 ceph-mon[36454]: 6.12 scrub ok 2024-09-21T11:53:53.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:53.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:53.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:53 smithi175 ceph-mon[32613]: pgmap v669: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:53:53.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:53 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T11:53:53.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:53 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T11:53:53.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:53 smithi175 ceph-mon[32613]: 6.c scrub starts 2024-09-21T11:53:53.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:53 smithi175 ceph-mon[32613]: 6.c scrub ok 2024-09-21T11:53:53.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:53 smithi175 ceph-mon[32613]: 10.2 scrub ok 2024-09-21T11:53:53.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:53 smithi175 ceph-mon[32613]: 6.12 scrub starts 2024-09-21T11:53:53.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:53 smithi175 ceph-mon[32613]: 6.12 scrub ok 2024-09-21T11:53:53.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:54.019 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:54.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:54.020 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:54.065 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:53:54.065 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:53:54.100 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:53:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:53:54.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:54.134 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:53:54.134 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:53:54.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:54.165 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:54.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:54.169 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:53:54.170 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:53:54.220 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:53:54.220 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:53:54.223 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:53:54.255 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:53:54.255 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:53:54.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:54.294 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:53:54.294 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:53:54.307 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:54.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:54.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:54.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 62 16 94715 94699 95.4424 67.75 0.0169827 0.0104722 2024-09-21T11:53:54.328 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:53:54.328 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:53:54.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:54.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:54.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:54.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:54.586 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:54.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:54.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:54.611 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:53:54.657 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:54.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:54.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:54 smithi043 ceph-mon[29710]: 10.0 scrub ok 2024-09-21T11:53:54.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:54 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2143374851' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:54.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:54 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1153574527' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:54.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:54 smithi043 ceph-mon[36454]: 10.0 scrub ok 2024-09-21T11:53:54.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:54 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2143374851' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:54.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:54 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1153574527' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:54.739 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:54.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:54.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:54.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:54.868 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:54.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:54.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:54.948 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:54 smithi175 ceph-mon[32613]: 10.0 scrub ok 2024-09-21T11:53:54.948 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:54 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2143374851' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:53:54.948 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:54 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1153574527' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:54.970 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:55.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:55.008 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:55.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:55.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:55.149 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:55.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:55.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:55.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:55.283 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:53:55.283 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:53:55.283 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:53:55.283 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:53:55.283 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:53:55.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:55.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:55.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 63 16 95816 95800 95.0197 68.8125 0.00768228 0.0105192 2024-09-21T11:53:55.393 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:55.428 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:55.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:55.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:53:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:53:55.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:55.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:55.564 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:55.665 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:55.699 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:55.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:55.700 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:55.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:55.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:55.839 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:55.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:55.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[29710]: pgmap v670: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-21T11:53:55.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[29710]: 4.19 scrub starts 2024-09-21T11:53:55.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[29710]: 4.19 scrub ok 2024-09-21T11:53:55.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:53:55.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:53:55.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2143374851' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:55.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[29710]: osdmap e339: 8 total, 8 up, 8 in 2024-09-21T11:53:55.893 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[36454]: pgmap v670: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-21T11:53:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[36454]: 4.19 scrub starts 2024-09-21T11:53:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[36454]: 4.19 scrub ok 2024-09-21T11:53:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:53:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:53:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2143374851' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:55.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:55 smithi043 ceph-mon[36454]: osdmap e339: 8 total, 8 up, 8 in 2024-09-21T11:53:55.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:55.953 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:55 smithi175 ceph-mon[32613]: pgmap v670: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-21T11:53:55.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:55 smithi175 ceph-mon[32613]: 4.19 scrub starts 2024-09-21T11:53:55.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:55 smithi175 ceph-mon[32613]: 4.19 scrub ok 2024-09-21T11:53:55.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:55 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:53:55.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:55 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:53:55.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:55 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2143374851' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:53:55.954 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:55 smithi175 ceph-mon[32613]: osdmap e339: 8 total, 8 up, 8 in 2024-09-21T11:53:55.988 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:55.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:55.989 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:56.089 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:56.111 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:56.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:56.112 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:56.212 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:56.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:56.253 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:56.253 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:56.315 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 64 16 96797 96781 94.493 61.3125 0.0210468 0.0105783 2024-09-21T11:53:56.354 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:56.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:56.392 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:56.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:56.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:56.523 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:56.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:56.524 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:56.625 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:56.667 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:56.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:56.668 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:56.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:56.808 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:56.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:56.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:56.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:56.932 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:56.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:56.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:56.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:56 smithi043 ceph-mon[36454]: 10.1 scrub starts 2024-09-21T11:53:56.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:56 smithi043 ceph-mon[36454]: 5.1c scrub starts 2024-09-21T11:53:56.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:56 smithi043 ceph-mon[36454]: 5.1c scrub ok 2024-09-21T11:53:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:56 smithi043 ceph-mon[29710]: 10.1 scrub starts 2024-09-21T11:53:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:56 smithi043 ceph-mon[29710]: 5.1c scrub starts 2024-09-21T11:53:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:56 smithi043 ceph-mon[29710]: 5.1c scrub ok 2024-09-21T11:53:57.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:56 smithi175 ceph-mon[32613]: 10.1 scrub starts 2024-09-21T11:53:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:56 smithi175 ceph-mon[32613]: 5.1c scrub starts 2024-09-21T11:53:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:56 smithi175 ceph-mon[32613]: 5.1c scrub ok 2024-09-21T11:53:57.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:53:57.056 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:57.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:57.057 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:53:57.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:57.187 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:57.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:57.188 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:57.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:53:57.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:53:57.289 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:57.316 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 65 16 97540 97524 93.7537 46.4375 0.0334981 0.010661 2024-09-21T11:53:57.339 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:57.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:57.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:57.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:53:57.476 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:57.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:57.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:53:57.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:57.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:57.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:57.708 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:53:57.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:57.774 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:57.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:57.776 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:57.801 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:53:57.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 0 1 2024-09-21T11:53:57.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:57.933 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:57.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:57.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:57.947 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:53:57.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[29710]: pgmap v672: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 624 op/s 2024-09-21T11:53:57.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T11:53:57.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T11:53:57.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[29710]: 5.3 scrub starts 2024-09-21T11:53:57.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[29710]: 5.3 scrub ok 2024-09-21T11:53:57.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[29710]: 10.1 scrub ok 2024-09-21T11:53:57.968 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[36454]: pgmap v672: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 624 op/s 2024-09-21T11:53:57.968 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T11:53:57.968 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T11:53:57.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[36454]: 5.3 scrub starts 2024-09-21T11:53:57.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[36454]: 5.3 scrub ok 2024-09-21T11:53:57.969 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:57 smithi043 ceph-mon[36454]: 10.1 scrub ok 2024-09-21T11:53:58.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:57 smithi175 ceph-mon[32613]: pgmap v672: 148 pgs: 2 active+clean+scrubbing, 146 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 624 op/s 2024-09-21T11:53:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:57 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T11:53:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:57 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T11:53:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:57 smithi175 ceph-mon[32613]: 5.3 scrub starts 2024-09-21T11:53:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:57 smithi175 ceph-mon[32613]: 5.3 scrub ok 2024-09-21T11:53:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:57 smithi175 ceph-mon[32613]: 10.1 scrub ok 2024-09-21T11:53:58.034 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:58.068 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:58.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:58.069 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:58.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:58.199 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:58.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:58.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:58.302 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:58.316 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 66 16 98498 98482 93.2403 59.875 0.0301311 0.0107199 2024-09-21T11:53:58.327 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:53:58.328 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:53:58.344 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:58.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:58.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:58.370 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:53:58.371 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:53:58.407 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:53:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:53:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:58.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:58.451 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:53:58.452 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:53:58.483 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:53:58.484 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:53:58.507 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:53:58.507 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:58.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:58.509 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:58.517 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:53:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:53:58.560 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:53:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:53:58.600 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:53:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:53:58.610 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:53:58.645 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:58.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:58.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:53:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:58 smithi043 ceph-mon[29710]: 10.3 scrub starts 2024-09-21T11:53:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:58 smithi043 ceph-mon[29710]: 3.e scrub starts 2024-09-21T11:53:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:58 smithi043 ceph-mon[29710]: 3.e scrub ok 2024-09-21T11:53:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:58 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1913634640' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:58 smithi043 ceph-mon[36454]: 10.3 scrub starts 2024-09-21T11:53:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:58 smithi043 ceph-mon[36454]: 3.e scrub starts 2024-09-21T11:53:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:58 smithi043 ceph-mon[36454]: 3.e scrub ok 2024-09-21T11:53:58.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:58 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1913634640' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:58.747 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:58.782 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:58.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:58.783 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:58.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:58.921 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:58.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:58.922 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:58.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:58 smithi175 ceph-mon[32613]: 10.3 scrub starts 2024-09-21T11:53:58.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:58 smithi175 ceph-mon[32613]: 3.e scrub starts 2024-09-21T11:53:58.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:58 smithi175 ceph-mon[32613]: 3.e scrub ok 2024-09-21T11:53:58.994 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:58 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1913634640' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:53:59.023 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:59.062 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:59.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:59.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:59.156 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:53:59.156 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:53:59.163 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:53:59.203 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:59.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:59.204 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:53:59.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:59.317 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 67 16 99395 99379 92.6854 56.0625 0.0925481 0.0107745 2024-09-21T11:53:59.338 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:59.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:59.339 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:59.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:53:59.481 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:59.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:59.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:53:59.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:53:59.616 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:53:59.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:59.617 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:53:59.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:53:59.768 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:59.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:59.770 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:53:59.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:53:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:53:59.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:53:59.910 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:53:59.947 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:59 smithi175 ceph-mon[32613]: pgmap v673: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-21T11:53:59.947 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:59 smithi175 ceph-mon[32613]: 4.e scrub starts 2024-09-21T11:53:59.947 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:53:59 smithi175 ceph-mon[32613]: 4.e scrub ok 2024-09-21T11:53:59.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:59 smithi043 ceph-mon[36454]: pgmap v673: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-21T11:53:59.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:59 smithi043 ceph-mon[36454]: 4.e scrub starts 2024-09-21T11:53:59.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:53:59 smithi043 ceph-mon[36454]: 4.e scrub ok 2024-09-21T11:53:59.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:59 smithi043 ceph-mon[29710]: pgmap v673: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-21T11:53:59.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:59 smithi043 ceph-mon[29710]: 4.e scrub starts 2024-09-21T11:53:59.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:53:59 smithi043 ceph-mon[29710]: 4.e scrub ok 2024-09-21T11:54:00.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:00.038 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:00.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:00.039 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:00.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:00.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:00.164 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:00.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:00.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:00.311 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:00.316 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 68 16 100199 100183 92.0614 50.25 0.00908026 0.0108567 2024-09-21T11:54:00.412 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:00.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:00.450 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:00.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:00.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:54:00.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:00.604 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:00.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:00.605 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T11:54:00.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:00.736 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:00.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:00.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:00.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:00 smithi175 ceph-mon[32613]: 10.4 scrub starts 2024-09-21T11:54:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2452226422' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:54:00.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:00 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:54:00.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:00.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:00.885 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:00.885 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:00.986 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:00 smithi043 ceph-mon[36454]: 10.4 scrub starts 2024-09-21T11:54:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2452226422' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:54:00.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:00 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:54:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:00 smithi043 ceph-mon[29710]: 10.4 scrub starts 2024-09-21T11:54:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2452226422' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:54:00.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:00 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:54:01.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:01.026 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:01.026 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:01.127 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:01.165 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:01.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:01.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:01.267 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:01.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:01.305 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:01.305 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:01.317 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 69 16 101130 101114 91.5692 58.1875 0.0204209 0.010915 2024-09-21T11:54:01.406 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:01.444 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:01.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:01.445 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:01.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:01.592 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:01.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:01.593 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:01.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:54:01.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:01.766 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:01.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:01.767 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:01.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:01.909 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: pgmap v674: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 602 B/s rd, 41 MiB/s wr, 655 op/s 2024-09-21T11:54:01.909 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: 4.10 scrub starts 2024-09-21T11:54:01.909 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: 4.10 scrub ok 2024-09-21T11:54:01.909 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: 10.e scrub starts 2024-09-21T11:54:01.909 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: 10.3 scrub ok 2024-09-21T11:54:01.909 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:54:01.909 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: osdmap e340: 8 total, 8 up, 8 in 2024-09-21T11:54:01.909 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: 5.8 scrub starts 2024-09-21T11:54:01.909 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: 5.8 scrub ok 2024-09-21T11:54:01.910 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:54:01.910 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:01.910 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:01.910 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2497243129' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:54:01.910 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: pgmap v674: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 602 B/s rd, 41 MiB/s wr, 655 op/s 2024-09-21T11:54:01.910 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: 4.10 scrub starts 2024-09-21T11:54:01.910 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: 4.10 scrub ok 2024-09-21T11:54:01.910 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: 10.e scrub starts 2024-09-21T11:54:01.910 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: 10.3 scrub ok 2024-09-21T11:54:01.911 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:54:01.911 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: osdmap e340: 8 total, 8 up, 8 in 2024-09-21T11:54:01.911 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: 5.8 scrub starts 2024-09-21T11:54:01.911 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: 5.8 scrub ok 2024-09-21T11:54:01.911 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:54:01.911 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:01.911 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:01.911 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2497243129' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:54:01.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:01.913 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:01.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:02.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:54:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: pgmap v674: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 5.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 602 B/s rd, 41 MiB/s wr, 655 op/s 2024-09-21T11:54:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: 4.10 scrub starts 2024-09-21T11:54:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: 4.10 scrub ok 2024-09-21T11:54:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: 10.e scrub starts 2024-09-21T11:54:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: 10.3 scrub ok 2024-09-21T11:54:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:54:02.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: osdmap e340: 8 total, 8 up, 8 in 2024-09-21T11:54:02.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: 5.8 scrub starts 2024-09-21T11:54:02.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: 5.8 scrub ok 2024-09-21T11:54:02.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:54:02.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:02.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:02.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2497243129' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:54:02.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:02.036 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:02.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:02.037 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:02.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:02.177 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:02.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:02.178 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:02.279 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:02.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:02.300 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:02.300 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:02.317 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 70 16 101867 101851 90.9192 46.0625 0.00568343 0.0109904 2024-09-21T11:54:02.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:02.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:02.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:02.530 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:54:02.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:02.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:02.580 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:02.681 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:02.705 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:02.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:02.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:02.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:02.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:02.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:02.947 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:02.984 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:02.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:02.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[36454]: 10.4 scrub ok 2024-09-21T11:54:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[36454]: 10.e scrub ok 2024-09-21T11:54:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2497243129' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:54:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[36454]: osdmap e341: 8 total, 8 up, 8 in 2024-09-21T11:54:02.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[29710]: 10.4 scrub ok 2024-09-21T11:54:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[29710]: 10.e scrub ok 2024-09-21T11:54:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2497243129' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:54:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:02 smithi043 ceph-mon[29710]: osdmap e341: 8 total, 8 up, 8 in 2024-09-21T11:54:03.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:03.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:02 smithi175 ceph-mon[32613]: 10.4 scrub ok 2024-09-21T11:54:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:02 smithi175 ceph-mon[32613]: 10.e scrub ok 2024-09-21T11:54:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:02 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2497243129' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:54:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:02 smithi175 ceph-mon[32613]: osdmap e341: 8 total, 8 up, 8 in 2024-09-21T11:54:03.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:03.125 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:03.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:03.126 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:03.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:03.263 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:03.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:03.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:03.317 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 71 16 102994 102978 90.6307 70.4375 0.0210611 0.0110289 2024-09-21T11:54:03.365 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:03.408 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:03.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:03.409 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:03.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:03.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:03.546 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:03.546 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:03.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:03.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:03.686 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:03.686 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:03.787 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:03.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 2 1 2024-09-21T11:54:03.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:03.827 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:03.827 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:03.927 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:03.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:03.977 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:03.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:03.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:03 smithi043 ceph-mon[36454]: pgmap v676: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.10k op/s 2024-09-21T11:54:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:03 smithi043 ceph-mon[36454]: 4.17 scrub starts 2024-09-21T11:54:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:03 smithi043 ceph-mon[36454]: 4.17 scrub ok 2024-09-21T11:54:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:03 smithi043 ceph-mon[36454]: 3.1a scrub starts 2024-09-21T11:54:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:03 smithi043 ceph-mon[36454]: 3.1a scrub ok 2024-09-21T11:54:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:03 smithi043 ceph-mon[29710]: pgmap v676: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.10k op/s 2024-09-21T11:54:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:03 smithi043 ceph-mon[29710]: 4.17 scrub starts 2024-09-21T11:54:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:03 smithi043 ceph-mon[29710]: 4.17 scrub ok 2024-09-21T11:54:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:03 smithi043 ceph-mon[29710]: 3.1a scrub starts 2024-09-21T11:54:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:03 smithi043 ceph-mon[29710]: 3.1a scrub ok 2024-09-21T11:54:04.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:03 smithi175 ceph-mon[32613]: pgmap v676: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.10k op/s 2024-09-21T11:54:04.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:03 smithi175 ceph-mon[32613]: 4.17 scrub starts 2024-09-21T11:54:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:03 smithi175 ceph-mon[32613]: 4.17 scrub ok 2024-09-21T11:54:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:03 smithi175 ceph-mon[32613]: 3.1a scrub starts 2024-09-21T11:54:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:03 smithi175 ceph-mon[32613]: 3.1a scrub ok 2024-09-21T11:54:04.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:04.110 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:04.111 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:04.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:04.112 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:04.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:04.242 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:04.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:04.243 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:04.317 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 72 16 104002 103986 90.2469 63 0.00621415 0.0110756 2024-09-21T11:54:04.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:04.384 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:04.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:04.385 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:04.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:04.527 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:04.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:04.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:04.628 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:04.668 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:04.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:04.669 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:04.770 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:04.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:04.828 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:04.829 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:04.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:04.971 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:04.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:04.972 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:05.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:05.111 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:05.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:05.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:05.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:05.248 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:05.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:05.249 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:05.317 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 73 16 105105 105089 89.9549 68.9375 0.0106381 0.0111113 2024-09-21T11:54:05.350 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:05.375 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:05.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:05.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:05.477 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:54:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:54:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:05.512 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:05.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:05.513 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:05.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:05.650 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:05.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:05.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:05.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:05.788 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:05.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:05.789 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:05.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:05.931 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:05.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:05.935 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:05.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:05 smithi043 ceph-mon[29710]: pgmap v678: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:54:05.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:05 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:54:05.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:05 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:54:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:05 smithi043 ceph-mon[36454]: pgmap v678: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:54:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:05 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:54:05.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:05 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:54:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:05 smithi175 ceph-mon[32613]: pgmap v678: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s 2024-09-21T11:54:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:05 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:54:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:05 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:54:06.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:06.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:06.072 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:06.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:06.173 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:06.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:06.217 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:06.217 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:06.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:06.319 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 74 16 106237 106221 89.6947 70.75 0.0252065 0.0111439 2024-09-21T11:54:06.348 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:06.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:06.349 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:06.409 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:06.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:06.503 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:06.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:06.504 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:06.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:06.644 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:06.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:06.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:06.745 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:06.780 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:06.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:06.781 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:06.881 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:06.918 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:06.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:06.919 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:06.931 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:06 smithi175 ceph-mon[32613]: osdmap e342: 8 total, 8 up, 8 in 2024-09-21T11:54:06.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:06 smithi043 ceph-mon[36454]: osdmap e342: 8 total, 8 up, 8 in 2024-09-21T11:54:06.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:06 smithi043 ceph-mon[29710]: osdmap e342: 8 total, 8 up, 8 in 2024-09-21T11:54:07.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:07.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:07.068 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:07.068 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:07.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:07.193 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:07.196 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:07.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:07.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:07.243 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:54:07.244 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:54:07.298 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:07.318 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 75 16 107282 107266 89.3688 65.3125 0.00555595 0.0111846 2024-09-21T11:54:07.346 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:07.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:07.347 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:07.448 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:07.471 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:07.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:07.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:07.473 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:54:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:54:07.507 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:54:07.508 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:54:07.542 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:54:07.542 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:54:07.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:07.576 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:54:07.576 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:54:07.611 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:07.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:07.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:07.613 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:54:07.614 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:54:07.648 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:54:07.649 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:54:07.689 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:54:07.689 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:54:07.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:07.719 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:54:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:54:07.755 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:07.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:07.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:07.764 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:07 smithi043 ceph-mon[36454]: pgmap v680: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 45 MiB/s wr, 728 op/s 2024-09-21T11:54:07.764 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3410211124' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:07.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:07 smithi043 ceph-mon[29710]: pgmap v680: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 45 MiB/s wr, 728 op/s 2024-09-21T11:54:07.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3410211124' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:07.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:07.900 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:07.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:07.901 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:08.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:08.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:07 smithi175 ceph-mon[32613]: pgmap v680: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 45 MiB/s wr, 728 op/s 2024-09-21T11:54:08.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3410211124' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:08.040 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:08.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:08.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:08.142 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:08.163 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:08.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:08.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:08.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:08.307 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:08.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:08.308 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:08.318 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 76 16 108236 108220 88.9774 59.625 0.0140368 0.0112343 2024-09-21T11:54:08.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:54:08.409 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:08.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:08.448 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:08.449 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:08.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:08.576 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:08.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:08.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:08.678 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:08.707 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:08.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:08.708 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:08.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:08.854 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:08.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:08.855 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:08.956 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:09.004 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:09.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:09.005 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:09.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:09.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:09.157 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:09.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:09.258 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:09.309 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:09.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:09.311 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:09.320 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 77 16 109306 109290 88.6885 66.875 0.0149431 0.01127 2024-09-21T11:54:09.412 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:09.458 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:09.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:09.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:09.559 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:09.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:09.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:09.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:09.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:09.762 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:09.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:09.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:09.765 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:09.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:09.899 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:09.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:09.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:09.962 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T11:54:09.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[36454]: pgmap v681: 148 pgs: 148 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-21T11:54:09.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:09.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:09.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:54:09.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:54:09.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1719586290' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:54:09.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:54:09.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[29710]: pgmap v681: 148 pgs: 148 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-21T11:54:09.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:09.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:09.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:54:09.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:54:09.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1719586290' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:54:09.967 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:09 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:54:10.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:10.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:09 smithi175 ceph-mon[32613]: pgmap v681: 148 pgs: 148 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-21T11:54:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:09 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:09 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:09 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:54:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:09 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:54:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1719586290' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:54:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:09 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:54:10.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:10.045 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:10.045 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:10.146 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:10.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:10.184 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:10.285 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:10.320 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 78 16 110439 110423 88.4593 70.8125 0.0174496 0.0113001 2024-09-21T11:54:10.323 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:10.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:10.324 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:10.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:10.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:10.459 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:10.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:54:10.561 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:10.598 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:10.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:10.599 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:10.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:10.733 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:10.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:10.734 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:10.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:10.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:10.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:10.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:11.021 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:11.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:11.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:11.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 3 1 2024-09-21T11:54:11.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:11.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:11.180 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:11.180 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:11.219 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:10 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:54:11.220 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:10 smithi043 ceph-mon[36454]: osdmap e343: 8 total, 8 up, 8 in 2024-09-21T11:54:11.220 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:10 smithi043 ceph-mon[36454]: pgmap v683: 148 pgs: 148 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 544 B/s rd, 43 MiB/s wr, 682 op/s 2024-09-21T11:54:11.220 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:10 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:54:11.220 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:10 smithi043 ceph-mon[29710]: osdmap e343: 8 total, 8 up, 8 in 2024-09-21T11:54:11.220 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:10 smithi043 ceph-mon[29710]: pgmap v683: 148 pgs: 148 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 544 B/s rd, 43 MiB/s wr, 682 op/s 2024-09-21T11:54:11.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:10 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:54:11.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:10 smithi175 ceph-mon[32613]: osdmap e343: 8 total, 8 up, 8 in 2024-09-21T11:54:11.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:10 smithi175 ceph-mon[32613]: pgmap v683: 148 pgs: 148 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 544 B/s rd, 43 MiB/s wr, 682 op/s 2024-09-21T11:54:11.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:11.320 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 79 16 111523 111507 88.1971 67.75 0.0125062 0.0113334 2024-09-21T11:54:11.336 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:11.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:11.337 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:11.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:11.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:11.489 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:11.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:11.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:11.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:11.637 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:11.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:54:11.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:11.803 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:11.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:11.804 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:11.905 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:11.939 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:11.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:11.941 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:11.967 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:54:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:54:12.042 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:12.069 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:12.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:12.070 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:12.155 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:11 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:12.155 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:11 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:12.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:12.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:12.235 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:12.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:12.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:11 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:12.316 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:12.320 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:54:12.320050+0000 min lat: 0.00131482 max lat: 0.315011 avg lat: 0.0113866 2024-09-21T11:54:12.321 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:54:12.321 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 80 16 112408 112392 87.786 55.3125 0.0126821 0.0113866 2024-09-21T11:54:12.334 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:54:12.334 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:54:12.334 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:54:12.334 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:54:12.334 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:54:12.337 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:12.367 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:12.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:12.368 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:12.469 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:12.509 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:12.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:12.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:12.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:12.614 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:12.680 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:12.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:12.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:12.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:12.819 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:12.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:12.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:12.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:12.964 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:12.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:12.965 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:12.977 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:54:13.065 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:13.101 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:13.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:13.102 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:13.203 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:13.233 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:13.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:13.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:13.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:12 smithi043 ceph-mon[29710]: pgmap v684: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-21T11:54:13.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:12 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2015802174' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:13.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:12 smithi043 ceph-mon[36454]: pgmap v684: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-21T11:54:13.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:12 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2015802174' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:13.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:12 smithi175 ceph-mon[32613]: pgmap v684: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 76 MiB/s wr, 1.22k op/s 2024-09-21T11:54:13.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:12 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2015802174' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:13.321 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 81 16 113517 113501 87.5579 69.3125 0.00938267 0.0114165 2024-09-21T11:54:13.335 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:13.362 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:13.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:13.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:13.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:54:13.463 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:13.496 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:13.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:13.497 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:13.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:13.627 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:13.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:13.628 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:13.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:13.768 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:13.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:13.769 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:13.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:13.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:13.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:13.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:14.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:14.054 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:14.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:14.055 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:14.155 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:14.187 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:14.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:14.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:13 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2015802174' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:54:14.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:13 smithi043 ceph-mon[36454]: osdmap e344: 8 total, 8 up, 8 in 2024-09-21T11:54:14.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:13 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:14.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2015802174' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:54:14.246 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:13 smithi043 ceph-mon[29710]: osdmap e344: 8 total, 8 up, 8 in 2024-09-21T11:54:14.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:13 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:14.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2015802174' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:54:14.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:13 smithi175 ceph-mon[32613]: osdmap e344: 8 total, 8 up, 8 in 2024-09-21T11:54:14.289 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:14.322 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 82 16 114617 114601 87.3274 68.75 0.0135236 0.0114465 2024-09-21T11:54:14.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:14.337 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:14.337 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:14.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:14.479 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:14.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:14.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:14.581 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:14.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:14.614 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:14.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:14.715 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:14.739 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:14.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:14.740 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:14.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:14.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:14.893 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:14.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:14.986 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T11:54:14.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:15.019 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:15.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:15.020 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:15.121 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:15.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:15.156 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:15.156 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:15.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:14 smithi043 ceph-mon[29710]: pgmap v686: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-21T11:54:15.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2638579436' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:54:15.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:14 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:54:15.213 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:14 smithi043 ceph-mon[36454]: pgmap v686: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-21T11:54:15.213 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2638579436' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:54:15.213 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:14 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:54:15.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:15.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:14 smithi175 ceph-mon[32613]: pgmap v686: 148 pgs: 148 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-21T11:54:15.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2638579436' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:54:15.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:14 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:54:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:15.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:15.308 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:15.322 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 83 16 115650 115634 87.0531 64.5625 0.0125453 0.0114825 2024-09-21T11:54:15.373 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:54:15.373 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:54:15.373 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:54:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:54:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:54:15.373 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:54:15.373 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:54:15.374 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:54:15.393 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:54:15.393 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:54:15.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:15.434 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:54:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:54:15.456 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:15.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:15.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:15.469 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:54:15.469 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:54:15.469 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:54:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:54:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:15.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:15.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:15.590 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:15.591 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:15.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:15.730 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:15.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:15.731 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:15.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:15.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:15.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:15.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:16.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:16.016 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:16.016 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:16.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:16.153 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:16.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:16.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:16.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:16.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:15 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:54:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:15 smithi175 ceph-mon[32613]: osdmap e345: 8 total, 8 up, 8 in 2024-09-21T11:54:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2563084024' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:16.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:54:16.290 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:16.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:16.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:16.303 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:15 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:54:16.303 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:15 smithi043 ceph-mon[36454]: osdmap e345: 8 total, 8 up, 8 in 2024-09-21T11:54:16.303 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2563084024' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:16.303 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:15 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:54:16.303 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:15 smithi043 ceph-mon[29710]: osdmap e345: 8 total, 8 up, 8 in 2024-09-21T11:54:16.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2563084024' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:16.322 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 84 16 116804 116788 86.8747 72.125 0.015219 0.0115059 2024-09-21T11:54:16.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 4 1 2024-09-21T11:54:16.391 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:16.418 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:16.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:16.420 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:16.520 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:16.549 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:16.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:16.550 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:16.651 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:16.688 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:16.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:16.689 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:16.789 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:16.823 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:16.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:16.824 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:16.925 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:16.955 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:16.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:16.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:17.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:17.083 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:17.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:17.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:17.117 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:17.138 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:17.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:17.223 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:17.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:17.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:17.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 85 16 117819 117803 86.5989 63.4375 0.0104993 0.0115428 2024-09-21T11:54:17.324 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:17.356 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:17.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:17.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:17.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:17.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:17.505 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:17.505 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:17.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:17.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:17.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:17.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:54:17.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:54:17.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[36454]: pgmap v688: 148 pgs: 148 active+clean; 6.8 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 839 B/s rd, 54 MiB/s wr, 865 op/s 2024-09-21T11:54:17.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:17.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:17.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[36454]: osdmap e346: 8 total, 8 up, 8 in 2024-09-21T11:54:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:54:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:54:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[29710]: pgmap v688: 148 pgs: 148 active+clean; 6.8 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 839 B/s rd, 54 MiB/s wr, 865 op/s 2024-09-21T11:54:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:17 smithi043 ceph-mon[29710]: osdmap e346: 8 total, 8 up, 8 in 2024-09-21T11:54:17.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:17.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:17.799 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:17.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:17.939 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:17.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:17.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:18.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:17 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:54:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:17 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:54:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:17 smithi175 ceph-mon[32613]: pgmap v688: 148 pgs: 148 active+clean; 6.8 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 839 B/s rd, 54 MiB/s wr, 865 op/s 2024-09-21T11:54:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:17 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:17 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:17 smithi175 ceph-mon[32613]: osdmap e346: 8 total, 8 up, 8 in 2024-09-21T11:54:18.041 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:18.091 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:18.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:18.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:18.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:18.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:18.225 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:18.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:54:18.303 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:54:18.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 86 16 118899 118883 86.3768 67.5 0.0151851 0.0115721 2024-09-21T11:54:18.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:18.370 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:18.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:18.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:18.472 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:18.493 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:18.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:18.494 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:18.595 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:18.626 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:18.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:18.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:18.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:18.760 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:18.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:18.761 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:18.862 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:18.900 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:18.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:18.901 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:19.002 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:19.034 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:19.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:19.035 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:19.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:54:19.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:19.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:19.186 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:19.186 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:19.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:19.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 87 16 119961 119945 86.1467 66.375 0.0252671 0.0116032 2024-09-21T11:54:19.332 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:19.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:19.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:19.434 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:19.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:19.477 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:19.477 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:19.578 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:19.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:19.608 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:19.608 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:19.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:19.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:19.747 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:19.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:19.827 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:19 smithi175 ceph-mon[32613]: pgmap v690: 148 pgs: 148 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-21T11:54:19.848 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:19.866 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:19 smithi043 ceph-mon[29710]: pgmap v690: 148 pgs: 148 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-21T11:54:19.866 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:19 smithi043 ceph-mon[36454]: pgmap v690: 148 pgs: 148 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 62 MiB/s wr, 996 op/s 2024-09-21T11:54:19.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:54:19.883 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:19.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:19.883 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:19.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:20.012 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:20.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:20.013 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:20.113 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:20.139 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:20.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:20.142 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:20.242 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:20.275 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:20.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:20.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:20.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 88 16 120939 120923 85.8624 61.125 0.012486 0.011641 2024-09-21T11:54:20.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:20.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:20.475 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:20.475 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:20.575 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:20.620 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:20.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:20.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:20.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:20.755 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:20.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:20.760 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:20.825 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:20.862 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:20.906 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T11:54:20.909 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:20.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:20.910 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:21.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:21.048 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:21.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:21.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:21.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:21.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:21.192 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[29710]: pgmap v691: 148 pgs: 148 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 724 B/s rd, 47 MiB/s wr, 750 op/s 2024-09-21T11:54:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3381858679' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:54:21.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:54:21.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:54:21.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[36454]: pgmap v691: 148 pgs: 148 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 724 B/s rd, 47 MiB/s wr, 750 op/s 2024-09-21T11:54:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3381858679' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:54:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:54:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:54:21.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:20 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:20 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:20 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:20 smithi175 ceph-mon[32613]: pgmap v691: 148 pgs: 148 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 724 B/s rd, 47 MiB/s wr, 750 op/s 2024-09-21T11:54:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3381858679' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:54:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:20 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:54:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:20 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:54:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:20 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:21.293 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:21.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 89 16 121771 121755 85.4819 52 0.0248315 0.0116933 2024-09-21T11:54:21.334 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:21.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:21.335 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:21.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:21.470 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:21.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:21.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:21.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:21.603 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:21.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:21.605 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:21.706 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:21.742 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:21.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:21.743 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:21.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:21.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:21.885 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:21.896 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:54:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:54:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:54:21.919 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:54:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:54:21.934 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:54:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:54:21.982 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:54:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:54:21.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:22.019 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:54:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:54:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:22.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:22.040 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:22.064 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:54:22.064 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:54:22.108 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:54:22.108 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:54:22.142 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3381858679' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[29710]: osdmap e347: 8 total, 8 up, 8 in 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/721230976' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4106731377' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3381858679' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[36454]: osdmap e347: 8 total, 8 up, 8 in 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/721230976' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:22.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:21 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4106731377' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:22.150 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:54:22.150 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:54:22.170 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:22.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:22.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:22.197 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:54:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:54:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:21 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3381858679' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:54:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:21 smithi175 ceph-mon[32613]: osdmap e347: 8 total, 8 up, 8 in 2024-09-21T11:54:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:21 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/721230976' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:21 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:21 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:21 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4106731377' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:22.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:22.302 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:54:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:54:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:122583 10.2 10:46d79133:::benchmark_data_smithi043_150678_object122582:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:54:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:22.143427+0000", 2024-09-21T11:54:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00057209700000000004, 2024-09-21T11:54:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00060449199999999999, 2024-09-21T11:54:22.302 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 122583 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:22.143427+0000", 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:54:22.303 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:22.143427+0000", 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:22.143429+0000", 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.463e-06 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:22.143452+0000", 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3487999999999999e-05 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:22.304 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:22.143468+0000", 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6147999999999999e-05 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:22.143476+0000", 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8390000000000007e-06 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:22.143645+0000", 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016923099999999999 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:22.305 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:22.144011+0000", 2024-09-21T11:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00036610400000000002 2024-09-21T11:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 1 2024-09-21T11:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:54:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:22.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:22.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:22.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Total time run: 90.0063 2024-09-21T11:54:22.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Total writes made: 122748 2024-09-21T11:54:22.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Write size: 65536 2024-09-21T11:54:22.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Object size: 65536 2024-09-21T11:54:22.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Bandwidth (MB/sec): 85.2357 2024-09-21T11:54:22.323 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev Bandwidth: 36.2567 2024-09-21T11:54:22.324 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max bandwidth (MB/sec): 222.312 2024-09-21T11:54:22.324 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min bandwidth (MB/sec): 46.0625 2024-09-21T11:54:22.324 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Average IOPS: 1363 2024-09-21T11:54:22.324 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev IOPS: 580.107 2024-09-21T11:54:22.324 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max IOPS: 3557 2024-09-21T11:54:22.324 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min IOPS: 737 2024-09-21T11:54:22.324 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Average Latency(s): 0.0117282 2024-09-21T11:54:22.324 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev Latency(s): 0.0125968 2024-09-21T11:54:22.324 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max latency(s): 0.315011 2024-09-21T11:54:22.324 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min latency(s): 0.00131482 2024-09-21T11:54:22.328 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Cleaning up (deleting benchmark objects) 2024-09-21T11:54:22.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:22.441 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:22.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:22.442 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:22.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:22.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:22.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:22.682 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:22.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:22.717 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:22.717 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:22.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:22.851 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:22.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:22.852 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:22.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:22.986 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:22.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:22.987 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:23.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:23.124 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:23.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:23.125 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:23.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:54:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[29710]: osdmap e348: 8 total, 8 up, 8 in 2024-09-21T11:54:23.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[29710]: pgmap v694: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-21T11:54:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:54:23.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:54:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:54:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[36454]: osdmap e348: 8 total, 8 up, 8 in 2024-09-21T11:54:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[36454]: pgmap v694: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-21T11:54:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:54:23.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:22 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:54:23.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:22 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:22 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:54:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:22 smithi175 ceph-mon[32613]: osdmap e348: 8 total, 8 up, 8 in 2024-09-21T11:54:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:22 smithi175 ceph-mon[32613]: pgmap v694: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-21T11:54:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:22 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:54:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:22 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:54:23.268 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:23.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:23.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:23.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:23.405 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:23.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:23.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:23.506 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:23.544 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:23.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:23.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:23.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:23.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:23.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:23.786 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:23.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:23.827 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:23.827 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:23.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:23.961 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:23.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:23.962 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:24.063 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:24.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:24.095 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:24.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:24.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:24.232 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:24.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:24.233 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:24.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T11:54:24.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T11:54:24.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[36454]: 5.9 scrub starts 2024-09-21T11:54:24.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[36454]: 5.9 scrub ok 2024-09-21T11:54:24.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[36454]: 3.18 scrub starts 2024-09-21T11:54:24.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[36454]: 3.18 scrub ok 2024-09-21T11:54:24.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T11:54:24.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T11:54:24.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[29710]: 5.9 scrub starts 2024-09-21T11:54:24.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[29710]: 5.9 scrub ok 2024-09-21T11:54:24.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[29710]: 3.18 scrub starts 2024-09-21T11:54:24.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:23 smithi043 ceph-mon[29710]: 3.18 scrub ok 2024-09-21T11:54:24.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:23 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T11:54:24.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:23 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T11:54:24.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:23 smithi175 ceph-mon[32613]: 5.9 scrub starts 2024-09-21T11:54:24.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:23 smithi175 ceph-mon[32613]: 5.9 scrub ok 2024-09-21T11:54:24.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:23 smithi175 ceph-mon[32613]: 3.18 scrub starts 2024-09-21T11:54:24.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:23 smithi175 ceph-mon[32613]: 3.18 scrub ok 2024-09-21T11:54:24.334 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:24.367 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:24.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:24.368 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:24.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:24.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:24.510 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:24.510 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:24.611 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:24.631 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:24.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:24.633 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:24.733 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:24.767 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:24.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:24.768 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:24.869 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:24.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:24.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:25.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:25.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:25.048 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:25.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:25.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:25.175 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:25.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:25.176 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:24 smithi043 ceph-mon[36454]: 3.1b scrub starts 2024-09-21T11:54:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:24 smithi043 ceph-mon[36454]: 3.1b scrub ok 2024-09-21T11:54:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:24 smithi043 ceph-mon[36454]: pgmap v695: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 725 B/s rd, 50 MiB/s wr, 802 op/s 2024-09-21T11:54:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:24 smithi043 ceph-mon[29710]: 3.1b scrub starts 2024-09-21T11:54:25.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:24 smithi043 ceph-mon[29710]: 3.1b scrub ok 2024-09-21T11:54:25.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:24 smithi043 ceph-mon[29710]: pgmap v695: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 725 B/s rd, 50 MiB/s wr, 802 op/s 2024-09-21T11:54:25.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:24 smithi175 ceph-mon[32613]: 3.1b scrub starts 2024-09-21T11:54:25.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:24 smithi175 ceph-mon[32613]: 3.1b scrub ok 2024-09-21T11:54:25.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:24 smithi175 ceph-mon[32613]: pgmap v695: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 725 B/s rd, 50 MiB/s wr, 802 op/s 2024-09-21T11:54:25.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:25.306 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:25.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:25.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:25.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:54:25.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:25.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:25.431 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:54:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:54:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:25.531 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:25.566 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:25.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:25.567 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:25.668 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:25.704 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:25.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:25.705 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:25.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:25.828 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:25.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:25.828 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:25.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:25.950 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:25.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:25.951 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:26.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:26.074 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:26.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:26.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:26.176 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:26.208 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:26.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:26.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:26.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:25 smithi043 ceph-mon[36454]: 6.13 scrub starts 2024-09-21T11:54:26.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:25 smithi043 ceph-mon[36454]: 6.13 scrub ok 2024-09-21T11:54:26.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:25 smithi043 ceph-mon[29710]: 6.13 scrub starts 2024-09-21T11:54:26.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:25 smithi043 ceph-mon[29710]: 6.13 scrub ok 2024-09-21T11:54:26.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:25 smithi175 ceph-mon[32613]: 6.13 scrub starts 2024-09-21T11:54:26.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:25 smithi175 ceph-mon[32613]: 6.13 scrub ok 2024-09-21T11:54:26.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:26.343 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:26.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:26.344 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:26.445 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:26.484 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:26.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:26.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:26.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:26.621 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:26.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:26.623 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:26.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:26.748 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:26.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:26.750 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:26.850 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:26.889 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:26.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:26.890 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:26.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:27.033 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:27.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:27.034 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:27.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:54:27.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:27.161 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:27.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:27.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:27.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:26 smithi043 ceph-mon[29710]: 5.2 scrub starts 2024-09-21T11:54:27.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:26 smithi043 ceph-mon[29710]: 5.2 scrub ok 2024-09-21T11:54:27.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:26 smithi043 ceph-mon[29710]: pgmap v696: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 620 op/s 2024-09-21T11:54:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:26 smithi043 ceph-mon[36454]: 5.2 scrub starts 2024-09-21T11:54:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:26 smithi043 ceph-mon[36454]: 5.2 scrub ok 2024-09-21T11:54:27.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:26 smithi043 ceph-mon[36454]: pgmap v696: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 620 op/s 2024-09-21T11:54:27.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:26 smithi175 ceph-mon[32613]: 5.2 scrub starts 2024-09-21T11:54:27.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:26 smithi175 ceph-mon[32613]: 5.2 scrub ok 2024-09-21T11:54:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:26 smithi175 ceph-mon[32613]: pgmap v696: 148 pgs: 148 active+clean; 7.4 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 639 B/s rd, 39 MiB/s wr, 620 op/s 2024-09-21T11:54:27.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:27.300 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:27.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:27.301 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:27.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:27.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:27.443 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:27.544 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:27.567 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:27.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:27.568 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:27.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:27.711 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:27.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:27.712 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:27.813 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:27.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:27.853 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:27.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:27.954 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:27.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 5 1 2024-09-21T11:54:27.985 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:27.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:27.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:28.087 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:28.119 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:28.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:28.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:28.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:28.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:28.277 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:28.378 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:28.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:28.419 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:28.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:28.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:54:28.520 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:28.535 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:28.543 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:28.545 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:28.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:28.546 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:28.647 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:28.677 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:28 smithi043 ceph-mon[29710]: 4.18 scrub starts 2024-09-21T11:54:28.677 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:28 smithi043 ceph-mon[29710]: 4.18 scrub ok 2024-09-21T11:54:28.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:28 smithi043 ceph-mon[36454]: 4.18 scrub starts 2024-09-21T11:54:28.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:28 smithi043 ceph-mon[36454]: 4.18 scrub ok 2024-09-21T11:54:28.681 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:28.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:28.682 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:28.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:28 smithi175 ceph-mon[32613]: 4.18 scrub starts 2024-09-21T11:54:28.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:28 smithi175 ceph-mon[32613]: 4.18 scrub ok 2024-09-21T11:54:28.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:28.818 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:28.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:28.819 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:28.920 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:28.970 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:28.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:28.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:29.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:29.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:29.115 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:29.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:29.151 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:29.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:29.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:29.247 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:29.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:29.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:29.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:29.383 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:29.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:29.485 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:29.516 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:29.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:29.517 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:29.617 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:29.638 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:29.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:29.639 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:29.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:29 smithi043 ceph-mon[36454]: pgmap v697: 148 pgs: 148 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.4 KiB/s rd, 46 MiB/s wr, 1.30k op/s 2024-09-21T11:54:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:29 smithi043 ceph-mon[36454]: 5.18 scrub starts 2024-09-21T11:54:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:29 smithi043 ceph-mon[36454]: 5.18 scrub ok 2024-09-21T11:54:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:29 smithi043 ceph-mon[29710]: pgmap v697: 148 pgs: 148 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.4 KiB/s rd, 46 MiB/s wr, 1.30k op/s 2024-09-21T11:54:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:29 smithi043 ceph-mon[29710]: 5.18 scrub starts 2024-09-21T11:54:29.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:29 smithi043 ceph-mon[29710]: 5.18 scrub ok 2024-09-21T11:54:29.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:29 smithi175 ceph-mon[32613]: pgmap v697: 148 pgs: 148 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.4 KiB/s rd, 46 MiB/s wr, 1.30k op/s 2024-09-21T11:54:29.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:29 smithi175 ceph-mon[32613]: 5.18 scrub starts 2024-09-21T11:54:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:29 smithi175 ceph-mon[32613]: 5.18 scrub ok 2024-09-21T11:54:29.762 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:29.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:29.763 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:29.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:29.899 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:29.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:29.900 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:30.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:30.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:30.046 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:30.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:30.147 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:30.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:30.192 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:30.192 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:30.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:30.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:30.331 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:30.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:30.370 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:54:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:54:30.405 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:30.411 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:54:30.411 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:54:30.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:54:30.447 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:54:30.447 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:54:30.465 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:30.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:30.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:30.489 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:54:30.490 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:54:30.526 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:54:30.527 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:54:30.567 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:30.568 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:54:30.568 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:54:30.602 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:30 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1793013865' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:30.602 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:30 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2247866394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:30.602 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:30 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1793013865' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:30.602 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:30 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2247866394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:30.603 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:30.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:30.604 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:30.610 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:54:30.610 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:54:30.652 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:54:30.652 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:54:30.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:30.741 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:30.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:30.742 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:30.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:30 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1793013865' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:30.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:30 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2247866394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:30.844 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:30.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:30.882 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:30.883 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:30.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:31.021 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:31.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:31.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:31.123 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:31.162 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:31.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:31.163 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:31.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:31.297 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:31.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:31.298 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:31.399 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:31.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:31.438 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:31.438 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:31.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[29710]: pgmap v698: 148 pgs: 148 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 40 MiB/s wr, 1.14k op/s 2024-09-21T11:54:31.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[29710]: 10.9 scrub starts 2024-09-21T11:54:31.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:54:31.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:54:31.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1793013865' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:54:31.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[29710]: osdmap e349: 8 total, 8 up, 8 in 2024-09-21T11:54:31.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[29710]: 6.10 scrub starts 2024-09-21T11:54:31.463 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[29710]: 6.10 scrub ok 2024-09-21T11:54:31.463 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[36454]: pgmap v698: 148 pgs: 148 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 40 MiB/s wr, 1.14k op/s 2024-09-21T11:54:31.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[36454]: 10.9 scrub starts 2024-09-21T11:54:31.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:54:31.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:54:31.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1793013865' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:54:31.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[36454]: osdmap e349: 8 total, 8 up, 8 in 2024-09-21T11:54:31.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[36454]: 6.10 scrub starts 2024-09-21T11:54:31.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:31 smithi043 ceph-mon[36454]: 6.10 scrub ok 2024-09-21T11:54:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:31 smithi175 ceph-mon[32613]: pgmap v698: 148 pgs: 148 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 40 MiB/s wr, 1.14k op/s 2024-09-21T11:54:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:31 smithi175 ceph-mon[32613]: 10.9 scrub starts 2024-09-21T11:54:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:31 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:54:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:31 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:54:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1793013865' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:54:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:31 smithi175 ceph-mon[32613]: osdmap e349: 8 total, 8 up, 8 in 2024-09-21T11:54:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:31 smithi175 ceph-mon[32613]: 6.10 scrub starts 2024-09-21T11:54:31.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:31 smithi175 ceph-mon[32613]: 6.10 scrub ok 2024-09-21T11:54:31.540 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:31.578 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:31.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:31.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:31.679 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:31.712 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:31.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:31.713 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:31.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:31.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:31.852 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:31.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:31.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:31.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:32.000 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:32.000 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:32.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:54:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:54:32.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:32.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:32.148 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:32.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:32.278 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:32.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:32.279 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:32.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:32.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:32.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:32.523 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:32.556 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:32.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:32.557 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:32.599 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T11:54:32.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:32.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:32.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:32.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[29710]: 10.9 scrub ok 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[29710]: 3.1 scrub starts 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[29710]: 3.1 scrub ok 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/727420678' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[36454]: 10.9 scrub ok 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[36454]: 3.1 scrub starts 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[36454]: 3.1 scrub ok 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/727420678' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:54:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:32 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:54:32.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:32.825 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:54:32.826 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:54:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:32.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:32.831 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:32.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:32.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:32.967 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:32.968 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:32 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:32 smithi175 ceph-mon[32613]: 10.9 scrub ok 2024-09-21T11:54:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:32 smithi175 ceph-mon[32613]: 3.1 scrub starts 2024-09-21T11:54:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:32 smithi175 ceph-mon[32613]: 3.1 scrub ok 2024-09-21T11:54:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:32 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/727420678' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:54:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:32 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:54:33.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:33.106 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:33.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:33.107 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:33.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:33.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:33.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:33.353 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:33.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:54:33.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:33.388 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:33.388 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:33.489 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:33.519 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:33.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:33.520 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:33.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:33.660 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:33.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:33.661 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:33.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T11:54:33.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[29710]: pgmap v700: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 9.7 MiB/s wr, 1.07k op/s 2024-09-21T11:54:33.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T11:54:33.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:54:33.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[29710]: osdmap e350: 8 total, 8 up, 8 in 2024-09-21T11:54:33.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[29710]: 3.19 scrub starts 2024-09-21T11:54:33.676 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[29710]: 3.19 scrub ok 2024-09-21T11:54:33.676 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T11:54:33.676 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[36454]: pgmap v700: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 9.7 MiB/s wr, 1.07k op/s 2024-09-21T11:54:33.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T11:54:33.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:54:33.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[36454]: osdmap e350: 8 total, 8 up, 8 in 2024-09-21T11:54:33.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[36454]: 3.19 scrub starts 2024-09-21T11:54:33.677 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:33 smithi043 ceph-mon[36454]: 3.19 scrub ok 2024-09-21T11:54:33.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:54:33.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:33.825 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:33.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:33.826 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:33.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:33.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:33.981 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:33.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:34.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:33 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T11:54:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:33 smithi175 ceph-mon[32613]: pgmap v700: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 9.7 MiB/s wr, 1.07k op/s 2024-09-21T11:54:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:33 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T11:54:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:33 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:54:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:33 smithi175 ceph-mon[32613]: osdmap e350: 8 total, 8 up, 8 in 2024-09-21T11:54:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:33 smithi175 ceph-mon[32613]: 3.19 scrub starts 2024-09-21T11:54:34.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:33 smithi175 ceph-mon[32613]: 3.19 scrub ok 2024-09-21T11:54:34.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:34.134 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:34.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:34.135 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:34.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:34.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:34.292 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:34.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 6 1 2024-09-21T11:54:34.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:34.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:54:34.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:34.471 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:34.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:34.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:34.599 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:34.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:34.600 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:34.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:34.731 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:34.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:34.732 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:34.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:34 smithi043 ceph-mon[29710]: 3.1c scrub starts 2024-09-21T11:54:34.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:34 smithi043 ceph-mon[29710]: 3.1c scrub ok 2024-09-21T11:54:34.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:34 smithi043 ceph-mon[29710]: 6.5 scrub starts 2024-09-21T11:54:34.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:34 smithi043 ceph-mon[29710]: 6.5 scrub ok 2024-09-21T11:54:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:34 smithi043 ceph-mon[36454]: 3.1c scrub starts 2024-09-21T11:54:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:34 smithi043 ceph-mon[36454]: 3.1c scrub ok 2024-09-21T11:54:34.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:34 smithi043 ceph-mon[36454]: 6.5 scrub starts 2024-09-21T11:54:34.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:34 smithi043 ceph-mon[36454]: 6.5 scrub ok 2024-09-21T11:54:34.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:34.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:34.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:34.913 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:35.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:34 smithi175 ceph-mon[32613]: 3.1c scrub starts 2024-09-21T11:54:35.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:34 smithi175 ceph-mon[32613]: 3.1c scrub ok 2024-09-21T11:54:35.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:34 smithi175 ceph-mon[32613]: 6.5 scrub starts 2024-09-21T11:54:35.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:34 smithi175 ceph-mon[32613]: 6.5 scrub ok 2024-09-21T11:54:35.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:35.041 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:35.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:35.042 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:35.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:35.185 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:35.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:35.186 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:35.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:35.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:35.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:35.423 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:35.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:35.465 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:35.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:54:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:35.566 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:35.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:35.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:35.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:35.696 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:35.726 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:35.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:35.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:35.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:35.861 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:35.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:35.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:35.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:35.984 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:35.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:35.985 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[36454]: pgmap v702: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.4 KiB/s rd, 7.0 MiB/s wr, 1.34k op/s 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[36454]: 6.6 scrub starts 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[36454]: 6.6 scrub ok 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[36454]: 3.13 scrub starts 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[36454]: 3.13 scrub ok 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[36454]: 5.12 scrub starts 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[36454]: 5.12 scrub ok 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[29710]: pgmap v702: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.4 KiB/s rd, 7.0 MiB/s wr, 1.34k op/s 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[29710]: 6.6 scrub starts 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[29710]: 6.6 scrub ok 2024-09-21T11:54:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[29710]: 3.13 scrub starts 2024-09-21T11:54:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[29710]: 3.13 scrub ok 2024-09-21T11:54:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[29710]: 5.12 scrub starts 2024-09-21T11:54:35.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:35 smithi043 ceph-mon[29710]: 5.12 scrub ok 2024-09-21T11:54:36.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:35 smithi175 ceph-mon[32613]: pgmap v702: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.4 KiB/s rd, 7.0 MiB/s wr, 1.34k op/s 2024-09-21T11:54:36.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:35 smithi175 ceph-mon[32613]: 6.6 scrub starts 2024-09-21T11:54:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:35 smithi175 ceph-mon[32613]: 6.6 scrub ok 2024-09-21T11:54:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:35 smithi175 ceph-mon[32613]: 3.13 scrub starts 2024-09-21T11:54:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:35 smithi175 ceph-mon[32613]: 3.13 scrub ok 2024-09-21T11:54:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:35 smithi175 ceph-mon[32613]: 5.12 scrub starts 2024-09-21T11:54:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:35 smithi175 ceph-mon[32613]: 5.12 scrub ok 2024-09-21T11:54:36.059 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:54:36.059 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:54:36.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:36.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:36.128 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:36.128 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:36.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:36.271 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:36.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:36.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:36.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:36.408 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:36.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:36.409 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:36.509 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:36.547 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:36.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:36.548 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:36.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:36.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:36.687 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:36.687 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:36.788 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:36.821 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:36.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:36.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:36.923 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:36.937 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:36 smithi043 ceph-mon[36454]: 4.5 scrub starts 2024-09-21T11:54:36.937 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:36 smithi043 ceph-mon[36454]: 4.5 scrub ok 2024-09-21T11:54:36.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:36 smithi043 ceph-mon[29710]: 4.5 scrub starts 2024-09-21T11:54:36.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:36 smithi043 ceph-mon[29710]: 4.5 scrub ok 2024-09-21T11:54:36.955 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:36.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:36.956 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:37.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:36 smithi175 ceph-mon[32613]: 4.5 scrub starts 2024-09-21T11:54:37.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:36 smithi175 ceph-mon[32613]: 4.5 scrub ok 2024-09-21T11:54:37.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:37.104 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:37.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:37.105 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:37.205 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:37.238 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:37.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:37.239 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:37.340 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:37.369 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:37.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:37.370 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:37.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:37.509 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:37.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:37.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:37.611 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:37.644 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:37.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:37.646 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:37.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:37.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:54:37.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:37.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:37.801 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:37.801 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:37.888 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:37 smithi043 ceph-mon[36454]: pgmap v703: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 661 op/s 2024-09-21T11:54:37.888 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:37 smithi043 ceph-mon[36454]: 3.a scrub starts 2024-09-21T11:54:37.888 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:37 smithi043 ceph-mon[36454]: 3.a scrub ok 2024-09-21T11:54:37.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:37 smithi043 ceph-mon[29710]: pgmap v703: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 661 op/s 2024-09-21T11:54:37.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:37 smithi043 ceph-mon[29710]: 3.a scrub starts 2024-09-21T11:54:37.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:37 smithi043 ceph-mon[29710]: 3.a scrub ok 2024-09-21T11:54:37.903 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:37.956 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:37.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:37.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:38.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:37 smithi175 ceph-mon[32613]: pgmap v703: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 6.9 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 661 op/s 2024-09-21T11:54:38.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:37 smithi175 ceph-mon[32613]: 3.a scrub starts 2024-09-21T11:54:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:37 smithi175 ceph-mon[32613]: 3.a scrub ok 2024-09-21T11:54:38.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:38.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:38.090 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:38.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:38.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:38.243 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:38.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:38.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:38.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:38.383 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:38.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:38.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:38.484 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:38.512 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:38.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:38.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:38.614 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:38.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:38.646 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:38.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:38.666 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:54:38.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[29710]: 4.1d scrub starts 2024-09-21T11:54:38.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[29710]: 4.1d scrub ok 2024-09-21T11:54:38.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[29710]: 2.0 scrub starts 2024-09-21T11:54:38.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[29710]: 2.0 scrub ok 2024-09-21T11:54:38.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1494817987' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:54:38.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[29710]: 4.12 scrub starts 2024-09-21T11:54:38.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[29710]: 4.12 scrub ok 2024-09-21T11:54:38.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[36454]: 4.1d scrub starts 2024-09-21T11:54:38.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[36454]: 4.1d scrub ok 2024-09-21T11:54:38.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[36454]: 2.0 scrub starts 2024-09-21T11:54:38.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[36454]: 2.0 scrub ok 2024-09-21T11:54:38.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1494817987' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:54:38.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[36454]: 4.12 scrub starts 2024-09-21T11:54:38.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:38 smithi043 ceph-mon[36454]: 4.12 scrub ok 2024-09-21T11:54:38.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:38.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:38.798 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:38.799 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:38.899 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:38.932 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:38.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:38.933 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:39.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:38 smithi175 ceph-mon[32613]: 4.1d scrub starts 2024-09-21T11:54:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:38 smithi175 ceph-mon[32613]: 4.1d scrub ok 2024-09-21T11:54:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:38 smithi175 ceph-mon[32613]: 2.0 scrub starts 2024-09-21T11:54:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:38 smithi175 ceph-mon[32613]: 2.0 scrub ok 2024-09-21T11:54:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:38 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1494817987' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:54:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:38 smithi175 ceph-mon[32613]: 4.12 scrub starts 2024-09-21T11:54:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:38 smithi175 ceph-mon[32613]: 4.12 scrub ok 2024-09-21T11:54:39.033 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:39.063 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:39.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:39.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:39.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:39.206 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:39.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:39.207 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:39.230 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:54:39.230 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:54:39.269 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:54:39.272 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:54:39.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:39.310 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:54:39.323 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:54:39.343 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:54:39.344 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:54:39.350 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:39.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:39.351 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:39.374 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:54:39.374 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:54:39.410 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:54:39.410 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:54:39.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:39.456 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:54:39.456 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:54:39.494 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:54:39.494 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:54:39.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:39.495 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:39.496 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:39.597 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:39.634 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:39.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:39.635 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:39.669 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T11:54:39.735 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:39.767 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:39.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:39.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:39.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:39.910 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:39.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:39.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:39.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[36454]: pgmap v704: 148 pgs: 148 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-21T11:54:39.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[36454]: 3.10 scrub starts 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[36454]: 3.10 scrub ok 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1494817987' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[36454]: osdmap e351: 8 total, 8 up, 8 in 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/724869839' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3038662933' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[29710]: pgmap v704: 148 pgs: 148 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[29710]: 3.10 scrub starts 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[29710]: 3.10 scrub ok 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1494817987' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[29710]: osdmap e351: 8 total, 8 up, 8 in 2024-09-21T11:54:39.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/724869839' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:54:39.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3038662933' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:40.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:39 smithi175 ceph-mon[32613]: pgmap v704: 148 pgs: 148 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-21T11:54:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:39 smithi175 ceph-mon[32613]: 3.10 scrub starts 2024-09-21T11:54:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:39 smithi175 ceph-mon[32613]: 3.10 scrub ok 2024-09-21T11:54:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:39 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1494817987' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:54:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:39 smithi175 ceph-mon[32613]: osdmap e351: 8 total, 8 up, 8 in 2024-09-21T11:54:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/724869839' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:54:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3038662933' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:40.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:40.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:40.050 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:40.050 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:40.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:40.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:40.189 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:40.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:40.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:40.327 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:40.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:40.328 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:40.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:40.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:40.475 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:40.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:40.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 7 1 2024-09-21T11:54:40.612 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:40.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:40.613 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:40.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:40.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:40.766 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:40.773 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:40.867 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:40.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:40.906 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:40.906 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:40.954 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:40 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/724869839' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:54:40.955 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:40 smithi043 ceph-mon[36454]: osdmap e352: 8 total, 8 up, 8 in 2024-09-21T11:54:40.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/724869839' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:54:40.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:40 smithi043 ceph-mon[29710]: osdmap e352: 8 total, 8 up, 8 in 2024-09-21T11:54:41.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:41.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:40 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/724869839' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:54:41.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:40 smithi175 ceph-mon[32613]: osdmap e352: 8 total, 8 up, 8 in 2024-09-21T11:54:41.047 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:41.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:41.048 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:41.149 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:41.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:41.185 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:41.185 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:41.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:41.322 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:41.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:41.323 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:41.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:41.488 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:41.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:41.490 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:41.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:41.631 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:41.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:41.632 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:41.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:41 smithi043 ceph-mon[36454]: pgmap v707: 148 pgs: 148 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 685 B/s rd, 0 B/s wr, 728 op/s 2024-09-21T11:54:41.733 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:41.765 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:41.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:41.766 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:41.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:54:41.866 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:41.895 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:41.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:41.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:41.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:41 smithi043 ceph-mon[29710]: pgmap v707: 148 pgs: 148 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 685 B/s rd, 0 B/s wr, 728 op/s 2024-09-21T11:54:41.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:42.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:41 smithi175 ceph-mon[32613]: pgmap v707: 148 pgs: 148 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 685 B/s rd, 0 B/s wr, 728 op/s 2024-09-21T11:54:42.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:54:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:54:42.014 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:42.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:42.049 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:42.056 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:54:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:54:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.056 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:144361 10.3 10:c069dcf6:::benchmark_data_smithi043_150678_object21610:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:42.027191+0000", 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.025061132999999999, 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.025107773999999999, 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 144361 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.027191+0000", 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.027191+0000", 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.027193+0000", 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.846e-06 2024-09-21T11:54:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.027201+0000", 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8050000000000007e-06 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.027221+0000", 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9911999999999998e-05 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.027230+0000", 2024-09-21T11:54:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0599999999999997e-06 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.027494+0000", 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00026370400000000002 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.027643+0000", 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014946 2024-09-21T11:54:42.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.027765+0000", 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121876 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.035483+0000", 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077180540000000002 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:54:42.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.036006+0000", 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000522943 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:144365 10.3 10:cbc64b09:::benchmark_data_smithi043_150678_object21614:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:42.029913+0000", 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.022339106000000001, 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022464877000000001, 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:54:42.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 144365 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.029913+0000", 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:54:42.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.029913+0000", 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.029915+0000", 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3799999999999999e-06 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.029920+0000", 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4859999999999997e-06 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.029935+0000", 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4448e-05 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.029942+0000", 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4909999999999999e-06 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.031881+0000", 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001938412 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.032011+0000", 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013023900000000001 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.032130+0000", 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119447 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.036017+0000", 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0038865589999999999 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.036531+0000", 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00051447400000000001 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:144368 10.5 10:ae3fbbbf:::benchmark_data_smithi043_150678_object21617:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:42.038305+0000", 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.013946991000000001, 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014143365, 2024-09-21T11:54:42.067 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 144368 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038305+0000", 2024-09-21T11:54:42.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038305+0000", 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038307+0000", 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2440000000000001e-06 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:54:42.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038311+0000", 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4499999999999997e-06 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038321+0000", 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0339e-05 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038329+0000", 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1010000000000003e-06 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.039001+0000", 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00067291499999999995 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.039491+0000", 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00048971599999999996 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.041399+0000", 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019078470000000001 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:54:42.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:144371 10.5 10:a1e1e68e:::benchmark_data_smithi043_150678_object21620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:42.038373+0000", 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.013879493999999999, 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014136424999999999, 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:54:42.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 144371 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038373+0000", 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038373+0000", 2024-09-21T11:54:42.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038374+0000", 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1200000000000003e-07 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038377+0000", 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6770000000000001e-06 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:54:42.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038385+0000", 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6960000000000005e-06 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038391+0000", 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4509999999999997e-06 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.045162+0000", 2024-09-21T11:54:42.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0067702750000000001 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.045293+0000", 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013172999999999999 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.045411+0000", 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011734399999999999 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25910.0:144350 10.5 e352/313)", 2024-09-21T11:54:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:42.039636+0000", 2024-09-21T11:54:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.01261672, 2024-09-21T11:54:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012928686, 2024-09-21T11:54:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:54:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:54:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:54:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:54:42.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.039636+0000", 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.039636+0000", 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.039637+0000", 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4799999999999997e-07 2024-09-21T11:54:42.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.039641+0000", 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.245e-06 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.039648+0000", 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5660000000000001e-06 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.039655+0000", 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0929999999999999e-06 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:144350 10.5 10:a26d76d2:::benchmark_data_smithi043_150678_object21599:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:42.006681+0000", 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.045571388999999997, 2024-09-21T11:54:42.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045926775000000003, 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 144350 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.081 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:54:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.006681+0000", 2024-09-21T11:54:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:54:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.006681+0000", 2024-09-21T11:54:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:54:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:54:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.006683+0000", 2024-09-21T11:54:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7260000000000001e-06 2024-09-21T11:54:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:54:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.006689+0000", 2024-09-21T11:54:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8030000000000002e-06 2024-09-21T11:54:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.006697+0000", 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6859999999999995e-06 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.006705+0000", 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6709999999999999e-06 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:54:42.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.015270+0000", 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0085655060000000005 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.015412+0000", 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014205000000000001 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.015536+0000", 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012378000000000001 2024-09-21T11:54:42.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.023593+0000", 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080572349999999994 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:42.038851+0000", 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015257270999999999 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:54:42.086 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:54:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:54:42.087 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 6 2024-09-21T11:54:42.087 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:54:42.150 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:42.184 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:42.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:42.185 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:42.286 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:42.322 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:42.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:42.324 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:42.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:42.461 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:42.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:42.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:42.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:42.590 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:42.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:42.591 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:42.692 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:42.728 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:42.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:42.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:42.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:42.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:42.874 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:42.874 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:42.975 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:42.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:42.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:43.010 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:43.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:43.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:43.011 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:43.112 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:43.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:43.144 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:43.144 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:43.245 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:43.280 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:43.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:43.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:43.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:54:43.381 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:43.405 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:43.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:43.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:43.426 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:43.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:54:43.507 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:43.534 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:43.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:43.535 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:43.636 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:43.664 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:43.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:43.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:43.725 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T11:54:43.767 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:43.793 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:43.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:43.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:43.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:43.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:43.924 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:43.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:43 smithi043 ceph-mon[29710]: pgmap v708: 148 pgs: 148 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-21T11:54:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:43 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:54:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:43 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:54:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:43 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1671143720' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:54:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:43 smithi043 ceph-mon[36454]: pgmap v708: 148 pgs: 148 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-21T11:54:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:43 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:54:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:43 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:54:43.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:43 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1671143720' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:54:44.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:43 smithi175 ceph-mon[32613]: pgmap v708: 148 pgs: 148 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-21T11:54:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:43 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:54:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:43 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:54:44.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:43 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1671143720' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:54:44.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:44.060 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:44.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:44.060 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:44.161 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:44.192 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:44.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:44.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:44.294 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:44.330 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:44.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:44.331 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:44.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:44.474 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:44.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:44.475 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:44.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:44.608 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:44.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:44.608 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:44.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:44.749 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:54:44.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:44.755 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:44.755 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:44.856 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:44.892 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:44.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:44.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:44.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1671143720' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:54:44.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:44 smithi043 ceph-mon[29710]: osdmap e353: 8 total, 8 up, 8 in 2024-09-21T11:54:44.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:44 smithi043 ceph-mon[29710]: pgmap v710: 148 pgs: 148 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 948 op/s 2024-09-21T11:54:44.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2847911041' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:44.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:44 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:44.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1671143720' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:54:44.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:44 smithi043 ceph-mon[36454]: osdmap e353: 8 total, 8 up, 8 in 2024-09-21T11:54:44.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:44 smithi043 ceph-mon[36454]: pgmap v710: 148 pgs: 148 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 948 op/s 2024-09-21T11:54:44.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2847911041' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:44.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:44 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:44.993 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:45.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 1 1 2024-09-21T11:54:45.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1671143720' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:54:45.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:44 smithi175 ceph-mon[32613]: osdmap e353: 8 total, 8 up, 8 in 2024-09-21T11:54:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:44 smithi175 ceph-mon[32613]: pgmap v710: 148 pgs: 148 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 948 op/s 2024-09-21T11:54:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2847911041' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:44 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:54:45.023 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:45.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:45.024 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:45.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:45.159 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:45.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:45.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:45.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:45.290 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:45.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:45.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:45.343 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:54:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:54:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:45.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:45.429 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:45.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:45.430 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:45.531 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:45.564 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:45.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:45.565 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:45.665 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:45.695 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:45.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:45.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:45.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:45.805 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:45.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:45.842 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:45.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:45.943 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:45.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:45.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:45.976 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:45 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:45 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:54:45.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:45 smithi043 ceph-mon[36454]: osdmap e354: 8 total, 8 up, 8 in 2024-09-21T11:54:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:45 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:45 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:54:45.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:45 smithi043 ceph-mon[29710]: osdmap e354: 8 total, 8 up, 8 in 2024-09-21T11:54:46.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:45 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:45 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:54:46.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:45 smithi175 ceph-mon[32613]: osdmap e354: 8 total, 8 up, 8 in 2024-09-21T11:54:46.077 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:46.104 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:46.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:46.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:46.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:46.243 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:46.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:46.247 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:46.348 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:46.377 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:46.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:46.378 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:46.479 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:46.506 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:46.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:46.507 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:46.608 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:46.621 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:54:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:54:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:54:46.621 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:54:46.622 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:54:46.639 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:46.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:46.640 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:46.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:46.784 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:46.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:46.785 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:46.886 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:46.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:46.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:47.029 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:47.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:47.059 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:47.059 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:47.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:47.191 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:47.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:47.192 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:47.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:47.330 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:47.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:47.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:47.432 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:47.466 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:47.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:47.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:47.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:47.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:54:47.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:47.600 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:47.600 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:47.701 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:47.718 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:54:47.718 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:54:47.718 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:54:47.718 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:54:47.718 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:54:47.718 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:54:47.719 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:54:47.719 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:54:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:54:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:54:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:54:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:54:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:54:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:54:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:54:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:54:47.741 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:47.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:47.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:47.803 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:54:47.842 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:47.848 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:47 smithi043 ceph-mon[29710]: pgmap v712: 148 pgs: 148 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 799 B/s rd, 0 B/s wr, 890 op/s 2024-09-21T11:54:47.848 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:47 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:47.848 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:47 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:47.848 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1593247695' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:47.848 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:47 smithi043 ceph-mon[36454]: pgmap v712: 148 pgs: 148 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 799 B/s rd, 0 B/s wr, 890 op/s 2024-09-21T11:54:47.848 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:47 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:47.848 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:47 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:47.849 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1593247695' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:47.871 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:47.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:47.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:47.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:48.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:47 smithi175 ceph-mon[32613]: pgmap v712: 148 pgs: 148 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 799 B/s rd, 0 B/s wr, 890 op/s 2024-09-21T11:54:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:47 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:54:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:47 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:54:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1593247695' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:48.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:48.020 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:48.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:48.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:48.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:48.174 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:48.174 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:48.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:48.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:48.315 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:48.315 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:48.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:48.456 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:48.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:48.458 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:48.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:48.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:48.597 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:48.597 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:48.597 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:48.685 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:48.686 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:48.697 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:48.732 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:48.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:48.733 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:48.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:48.875 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:48.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:48.877 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:48.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:49.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:49.013 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:49.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:49.114 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:49.141 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:49.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:49.142 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:49.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:49.284 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:49.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:49.285 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:49.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:49.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:49.454 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:49.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:49.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:49.606 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:49.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:49.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:49.708 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:49.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:49.743 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:49.743 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:49.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:49.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:49.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:49.932 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:49 smithi043 ceph-mon[36454]: pgmap v713: 148 pgs: 148 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-21T11:54:49.932 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:49 smithi043 ceph-mon[29710]: pgmap v713: 148 pgs: 148 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-21T11:54:49.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:50.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:49 smithi175 ceph-mon[32613]: pgmap v713: 148 pgs: 148 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-21T11:54:50.020 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:50.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:50.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:50.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:50.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:50.153 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:50.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:50.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:50.270 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:54:50.270 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:54:50.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:50.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:50.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:50.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:50.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:50.535 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:50.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:54:50.567 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:50.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:50.568 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T11:54:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:54:50.668 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:50.692 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:50.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:50.693 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:50.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:50.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:50.840 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:50.840 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:50.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:50.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:50.980 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:50.980 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:50.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2949377787' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:54:50.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:50 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:54:50.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2099909206' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:50.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:50 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2949377787' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:54:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:50 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:54:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2099909206' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:50 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:51.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2949377787' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:54:51.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:50 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:54:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2099909206' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:50 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:54:51.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:51.116 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:51.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:51.117 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:51.217 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:51.253 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:51.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:51.254 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:51.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:54:51.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:51.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:51.384 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:51.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:51.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:51.530 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:51.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:51.531 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:51.632 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:51.671 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:51.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:51.672 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:51.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:51.812 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:51.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:51.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:51.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:51.947 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:51.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:51.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:51.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:51 smithi043 ceph-mon[36454]: pgmap v714: 148 pgs: 148 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 800 op/s 2024-09-21T11:54:51.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:51 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:51.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:51 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:54:51.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:51 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:54:51.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:51 smithi043 ceph-mon[36454]: osdmap e355: 8 total, 8 up, 8 in 2024-09-21T11:54:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:51 smithi043 ceph-mon[29710]: pgmap v714: 148 pgs: 148 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 800 op/s 2024-09-21T11:54:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:51 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:51 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:54:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:51 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:54:51.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:51 smithi043 ceph-mon[29710]: osdmap e355: 8 total, 8 up, 8 in 2024-09-21T11:54:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:51 smithi175 ceph-mon[32613]: pgmap v714: 148 pgs: 148 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 800 op/s 2024-09-21T11:54:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:51 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:54:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:51 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:54:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:51 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:54:52.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:51 smithi175 ceph-mon[32613]: osdmap e355: 8 total, 8 up, 8 in 2024-09-21T11:54:52.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:54:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:54:52.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:52.089 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:52.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:52.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:52.121 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T11:54:52.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:54:52.191 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:52.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:52.223 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:52.223 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:52.324 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:52.353 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:52.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:52.354 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:52.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:52.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:52.497 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:52.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:52.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:52.625 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:52.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:52.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:52.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:52.749 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:52.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:52.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:52.851 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:52.888 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:52.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:52.889 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:52.989 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:52.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:52.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:52 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:54:52.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:52 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:54:52.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:52.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:52 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:54:52.995 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:52 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:54:53.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:54:53.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:52 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:54:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:52 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:54:53.017 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:53.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:53.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:53.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:53.175 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:53.199 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:53.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:53.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:53.256 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:54:53.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:53.364 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:53.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:53.365 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:53.465 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:53.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:53.501 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:53.501 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:53.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:53.637 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:53.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:53.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:53.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:53.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:53.772 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:53.773 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:53.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:53.916 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[36454]: pgmap v716: 148 pgs: 148 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-21T11:54:53.916 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[36454]: 3.3 scrub starts 2024-09-21T11:54:53.916 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[36454]: 3.3 scrub ok 2024-09-21T11:54:53.916 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[36454]: 10.f scrub starts 2024-09-21T11:54:53.916 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[36454]: osdmap e356: 8 total, 8 up, 8 in 2024-09-21T11:54:53.917 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[36454]: 5.19 scrub starts 2024-09-21T11:54:53.917 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[36454]: 5.19 scrub ok 2024-09-21T11:54:53.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[29710]: pgmap v716: 148 pgs: 148 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-21T11:54:53.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[29710]: 3.3 scrub starts 2024-09-21T11:54:53.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[29710]: 3.3 scrub ok 2024-09-21T11:54:53.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[29710]: 10.f scrub starts 2024-09-21T11:54:53.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[29710]: osdmap e356: 8 total, 8 up, 8 in 2024-09-21T11:54:53.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[29710]: 5.19 scrub starts 2024-09-21T11:54:53.917 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:53 smithi043 ceph-mon[29710]: 5.19 scrub ok 2024-09-21T11:54:53.918 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:53.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:53.919 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:53 smithi175 ceph-mon[32613]: pgmap v716: 148 pgs: 148 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-21T11:54:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:53 smithi175 ceph-mon[32613]: 3.3 scrub starts 2024-09-21T11:54:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:53 smithi175 ceph-mon[32613]: 3.3 scrub ok 2024-09-21T11:54:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:53 smithi175 ceph-mon[32613]: 10.f scrub starts 2024-09-21T11:54:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:53 smithi175 ceph-mon[32613]: osdmap e356: 8 total, 8 up, 8 in 2024-09-21T11:54:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:53 smithi175 ceph-mon[32613]: 5.19 scrub starts 2024-09-21T11:54:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:53 smithi175 ceph-mon[32613]: 5.19 scrub ok 2024-09-21T11:54:54.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:54.055 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:54.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:54.056 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:54.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:54.197 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:54.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:54.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:54.229 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:54:54.229 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:54:54.268 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:54:54.268 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:54:54.269 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:54:54.269 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:54:54.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:54.302 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:54:54.303 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:54:54.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:54.338 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:54.338 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:54.339 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:54:54.339 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:54:54.375 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:54:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:54:54.409 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:54:54.410 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:54:54.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:54.451 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:54:54.451 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:54:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:54.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:54.475 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:54.492 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:54:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:54:54.576 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:54.613 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:54.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:54.614 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:54.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:54.754 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:54.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:54.755 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:54.850 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: 3.1f scrub starts 2024-09-21T11:54:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: 3.1f scrub ok 2024-09-21T11:54:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: 6.1b scrub starts 2024-09-21T11:54:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: 6.1b scrub ok 2024-09-21T11:54:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: 10.7 scrub starts 2024-09-21T11:54:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:54:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:54:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: 6.2 scrub starts 2024-09-21T11:54:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: 6.2 scrub ok 2024-09-21T11:54:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: 10.f scrub ok 2024-09-21T11:54:54.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2217710375' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: 3.1f scrub starts 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: 3.1f scrub ok 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: 6.1b scrub starts 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: 6.1b scrub ok 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: 10.7 scrub starts 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: 6.2 scrub starts 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: 6.2 scrub ok 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: 10.f scrub ok 2024-09-21T11:54:54.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:54 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2217710375' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:54.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:54.935 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:54.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:54.937 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: 3.1f scrub starts 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: 3.1f scrub ok 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: 6.1b scrub starts 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: 6.1b scrub ok 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: 10.7 scrub starts 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: 6.2 scrub starts 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: 6.2 scrub ok 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: 10.f scrub ok 2024-09-21T11:54:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:54 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2217710375' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:54:55.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:55.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:54:55.073 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:55.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:55.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:55.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:55.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:55.226 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:55.226 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:55.326 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:55.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:55.355 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:55.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:55.455 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:55.490 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:54:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:54:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:54:55.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:55.492 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:55.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:55.638 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:55.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:55.639 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:55.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:55.774 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:54:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:54:55.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:55.779 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:55.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:55.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:55.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:55.943 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:55.944 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:55.948 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:55 smithi043 ceph-mon[29710]: pgmap v718: 148 pgs: 148 active+clean; 5.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-21T11:54:55.948 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:55 smithi043 ceph-mon[29710]: osdmap e357: 8 total, 8 up, 8 in 2024-09-21T11:54:55.948 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:55 smithi043 ceph-mon[29710]: 4.15 scrub starts 2024-09-21T11:54:55.948 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:55 smithi043 ceph-mon[29710]: 4.15 scrub ok 2024-09-21T11:54:55.948 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:55 smithi043 ceph-mon[36454]: pgmap v718: 148 pgs: 148 active+clean; 5.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-21T11:54:55.949 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:55 smithi043 ceph-mon[36454]: osdmap e357: 8 total, 8 up, 8 in 2024-09-21T11:54:55.949 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:55 smithi043 ceph-mon[36454]: 4.15 scrub starts 2024-09-21T11:54:55.949 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:55 smithi043 ceph-mon[36454]: 4.15 scrub ok 2024-09-21T11:54:55.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:54:56.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:55 smithi175 ceph-mon[32613]: pgmap v718: 148 pgs: 148 active+clean; 5.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-21T11:54:56.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:55 smithi175 ceph-mon[32613]: osdmap e357: 8 total, 8 up, 8 in 2024-09-21T11:54:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:55 smithi175 ceph-mon[32613]: 4.15 scrub starts 2024-09-21T11:54:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:55 smithi175 ceph-mon[32613]: 4.15 scrub ok 2024-09-21T11:54:56.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:56.094 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:56.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:56.096 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:56.196 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:56.225 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:56.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:56.226 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:56.327 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:56.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:56.374 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:56.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:56.514 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:56.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:56.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:56.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:56.646 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:56.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:56.647 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:56.748 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:56.783 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:56.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:56.784 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:56.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:56.917 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:56.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:56.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[29710]: 10.8 scrub starts 2024-09-21T11:54:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[29710]: 10.7 scrub ok 2024-09-21T11:54:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[29710]: 3.7 scrub starts 2024-09-21T11:54:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:54:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:54:56.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[29710]: 3.7 scrub ok 2024-09-21T11:54:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[36454]: 10.8 scrub starts 2024-09-21T11:54:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[36454]: 10.7 scrub ok 2024-09-21T11:54:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[36454]: 3.7 scrub starts 2024-09-21T11:54:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:54:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:54:56.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:56 smithi043 ceph-mon[36454]: 3.7 scrub ok 2024-09-21T11:54:57.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:56 smithi175 ceph-mon[32613]: 10.8 scrub starts 2024-09-21T11:54:57.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:56 smithi175 ceph-mon[32613]: 10.7 scrub ok 2024-09-21T11:54:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:56 smithi175 ceph-mon[32613]: 3.7 scrub starts 2024-09-21T11:54:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:56 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:54:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:56 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:54:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:56 smithi175 ceph-mon[32613]: 3.7 scrub ok 2024-09-21T11:54:57.019 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:57.056 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:57.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:57.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:57.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:57.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:57.210 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:57.210 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:57.311 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:57.342 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:57.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:57.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:57.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:57.475 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:57.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:57.476 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:57.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:57.612 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:57.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:57.613 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:57.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:54:57.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:57.741 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:57.741 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:54:57.842 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:57.876 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:57.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:57.877 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:57.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:57.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:57 smithi043 ceph-mon[36454]: from='client.16311 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:54:57.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:57 smithi043 ceph-mon[36454]: pgmap v720: 148 pgs: 148 active+clean; 5.5 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 946 op/s 2024-09-21T11:54:57.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:57 smithi043 ceph-mon[29710]: from='client.16311 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:54:57.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:57 smithi043 ceph-mon[29710]: pgmap v720: 148 pgs: 148 active+clean; 5.5 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 946 op/s 2024-09-21T11:54:58.007 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:58.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:58.008 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:58.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:57 smithi175 ceph-mon[32613]: from='client.16311 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:54:58.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:57 smithi175 ceph-mon[32613]: pgmap v720: 148 pgs: 148 active+clean; 5.5 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 946 op/s 2024-09-21T11:54:58.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:54:58.131 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:58.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:58.132 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:54:58.233 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:58.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:58.272 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:58.272 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:58.373 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:54:58.397 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:58.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:58.400 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:54:58.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:58.537 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:58.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:58.538 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:58.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:58.678 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:58.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:58.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:54:58.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:58.814 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:58.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:58.815 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:58.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:58.945 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:58.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:58.946 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:58 smithi043 ceph-mon[36454]: 10.8 scrub ok 2024-09-21T11:54:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:58 smithi043 ceph-mon[36454]: 6.12 scrub starts 2024-09-21T11:54:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:58 smithi043 ceph-mon[36454]: 6.12 scrub ok 2024-09-21T11:54:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:58 smithi043 ceph-mon[36454]: 6.7 scrub starts 2024-09-21T11:54:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:58 smithi043 ceph-mon[36454]: 6.7 scrub ok 2024-09-21T11:54:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:58 smithi043 ceph-mon[29710]: 10.8 scrub ok 2024-09-21T11:54:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:58 smithi043 ceph-mon[29710]: 6.12 scrub starts 2024-09-21T11:54:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:58 smithi043 ceph-mon[29710]: 6.12 scrub ok 2024-09-21T11:54:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:58 smithi043 ceph-mon[29710]: 6.7 scrub starts 2024-09-21T11:54:58.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:58 smithi043 ceph-mon[29710]: 6.7 scrub ok 2024-09-21T11:54:59.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:58 smithi175 ceph-mon[32613]: 10.8 scrub ok 2024-09-21T11:54:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:58 smithi175 ceph-mon[32613]: 6.12 scrub starts 2024-09-21T11:54:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:58 smithi175 ceph-mon[32613]: 6.12 scrub ok 2024-09-21T11:54:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:58 smithi175 ceph-mon[32613]: 6.7 scrub starts 2024-09-21T11:54:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:58 smithi175 ceph-mon[32613]: 6.7 scrub ok 2024-09-21T11:54:59.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:59.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:59.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:59.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:54:59.186 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:59.207 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:59.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:59.208 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:59.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:59.357 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:59.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:59.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:59.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:54:59.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:59.492 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:59.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:54:59.593 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:54:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:59.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:59.616 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:54:59.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:54:59.741 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:54:59.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:59.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:54:59.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:54:59.880 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:54:59.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:54:59.881 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:54:59.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:54:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:54:59 smithi043 ceph-mon[29710]: pgmap v721: 148 pgs: 148 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-21T11:54:59.995 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:54:59 smithi043 ceph-mon[36454]: pgmap v721: 148 pgs: 148 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-21T11:55:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:54:59 smithi175 ceph-mon[32613]: pgmap v721: 148 pgs: 148 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-21T11:55:00.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:00.018 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:00.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:00.119 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:00.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:00.152 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:00.153 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:00.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:00.291 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:00.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:00.292 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:00.393 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:00.414 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:00.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:00.415 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:00.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:00.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:00.552 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:00.653 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:00.686 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:00.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:00.687 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:00.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:00.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:00.830 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:00.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:00.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:00.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:00.966 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:00.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:01.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:01.094 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:01.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:01.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:01.195 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:01.233 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:01.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:01.234 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:01.335 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:01.374 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:01.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:01.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:01.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:01.481 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":719,"stamp":"2024-09-21T11:54:54.668574+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5926689331,"num_objects":90625,"num_object_clones":0,"num_object_copies":271875,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90625,"num_whiteouts":0,"num_read":2131,"num_read_kb":1988,"num_write":155907,"num_write_kb":7857546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":597267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94118,"ondisk_log_size":94118,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":21874936,"kb_used_data":17418724,"kb_used_omap":12,"kb_used_meta":4455859,"kb_avail":728086280,"statfs":{"total":767960285184,"available":745560350720,"internally_reserved":0,"allocated":17836773376,"data_stored":17833571029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4562800178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-371916800,"num_objects":-5675,"num_object_clones":0,"num_object_copies":-17025,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5675,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5675,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.593270"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":634,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.667290+0000","last_change":"2024-09-21T11:52:54.792106+0000","last_active":"2024-09-21T11:54:50.667290+0000","last_peered":"2024-09-21T11:54:50.667290+0000","last_clean":"2024-09-21T11:54:50.667290+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:54:50.667290+0000","last_undegraded":"2024-09-21T11:54:50.667290+0000","last_fullsized":"2024-09-21T11:54:50.667290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:54.792037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:54.792037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":673,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.653479+0000","last_change":"2024-09-21T11:54:27.283019+0000","last_active":"2024-09-21T11:54:50.653479+0000","last_peered":"2024-09-21T11:54:50.653479+0000","last_clean":"2024-09-21T11:54:50.653479+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:54:50.653479+0000","last_undegraded":"2024-09-21T11:54:50.653479+0000","last_fullsized":"2024-09-21T11:54:50.653479+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:27.282826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:27.282826+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:43.536640+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":343,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.662618+0000","last_change":"2024-09-21T11:53:20.097633+0000","last_active":"2024-09-21T11:54:50.662618+0000","last_peered":"2024-09-21T11:54:50.662618+0000","last_clean":"2024-09-21T11:54:50.662618+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:54:50.662618+0000","last_undegraded":"2024-09-21T11:54:50.662618+0000","last_fullsized":"2024-09-21T11:54:50.662618+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:20.097557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.097557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":446,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.649960+0000","last_change":"2024-09-21T11:51:43.528594+0000","last_active":"2024-09-21T11:54:50.649960+0000","last_peered":"2024-09-21T11:54:50.649960+0000","last_clean":"2024-09-21T11:54:50.649960+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:54:50.649960+0000","last_undegraded":"2024-09-21T11:54:50.649960+0000","last_fullsized":"2024-09-21T11:54:50.649960+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:06.268427+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":637,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.669432+0000","last_change":"2024-09-21T11:52:11.462678+0000","last_active":"2024-09-21T11:54:50.669432+0000","last_peered":"2024-09-21T11:54:50.669432+0000","last_clean":"2024-09-21T11:54:50.669432+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:54:50.669432+0000","last_undegraded":"2024-09-21T11:54:50.669432+0000","last_fullsized":"2024-09-21T11:54:50.669432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:11.462611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:11.462611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":627,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.655833+0000","last_change":"2024-09-21T11:51:01.667426+0000","last_active":"2024-09-21T11:54:50.655833+0000","last_peered":"2024-09-21T11:54:50.655833+0000","last_clean":"2024-09-21T11:54:50.655833+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:54:50.655833+0000","last_undegraded":"2024-09-21T11:54:50.655833+0000","last_fullsized":"2024-09-21T11:54:50.655833+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.667366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.667366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:20.606273+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":681,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.657589+0000","last_change":"2024-09-21T11:53:54.438887+0000","last_active":"2024-09-21T11:54:50.657589+0000","last_peered":"2024-09-21T11:54:50.657589+0000","last_clean":"2024-09-21T11:54:50.657589+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:54:50.657589+0000","last_undegraded":"2024-09-21T11:54:50.657589+0000","last_fullsized":"2024-09-21T11:54:50.657589+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:53:54.438798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:54.438798+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:20.197167+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":474,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.653494+0000","last_change":"2024-09-21T11:54:28.252705+0000","last_active":"2024-09-21T11:54:50.653494+0000","last_peered":"2024-09-21T11:54:50.653494+0000","last_clean":"2024-09-21T11:54:50.653494+0000","last_became_active":"2024-09-21T11:51:43.526945+0000","last_became_peered":"2024-09-21T11:51:43.526945+0000","last_unstale":"2024-09-21T11:54:50.653494+0000","last_undegraded":"2024-09-21T11:54:50.653494+0000","last_fullsized":"2024-09-21T11:54:50.653494+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:28.252642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:28.252642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:41.593068+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":632,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650637+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:54:50.650637+0000","last_peered":"2024-09-21T11:54:50.650637+0000","last_clean":"2024-09-21T11:54:50.650637+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:54:50.650637+0000","last_undegraded":"2024-09-21T11:54:50.650637+0000","last_fullsized":"2024-09-21T11:54:50.650637+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:15.115091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":631,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.654599+0000","last_change":"2024-09-21T11:52:55.214031+0000","last_active":"2024-09-21T11:54:50.654599+0000","last_peered":"2024-09-21T11:54:50.654599+0000","last_clean":"2024-09-21T11:54:50.654599+0000","last_became_active":"2024-09-21T11:51:43.545021+0000","last_became_peered":"2024-09-21T11:51:43.545021+0000","last_unstale":"2024-09-21T11:54:50.654599+0000","last_undegraded":"2024-09-21T11:54:50.654599+0000","last_fullsized":"2024-09-21T11:54:50.654599+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:52:55.213938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:55.213938+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":624,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.652141+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:54:50.652141+0000","last_peered":"2024-09-21T11:54:50.652141+0000","last_clean":"2024-09-21T11:54:50.652141+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:54:50.652141+0000","last_undegraded":"2024-09-21T11:54:50.652141+0000","last_fullsized":"2024-09-21T11:54:50.652141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.252990+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":651,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.667719+0000","last_change":"2024-09-21T11:52:50.731704+0000","last_active":"2024-09-21T11:54:50.667719+0000","last_peered":"2024-09-21T11:54:50.667719+0000","last_clean":"2024-09-21T11:54:50.667719+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:54:50.667719+0000","last_undegraded":"2024-09-21T11:54:50.667719+0000","last_fullsized":"2024-09-21T11:54:50.667719+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:50.731653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:50.731653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":670,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.649993+0000","last_change":"2024-09-21T11:54:33.055621+0000","last_active":"2024-09-21T11:54:50.649993+0000","last_peered":"2024-09-21T11:54:50.649993+0000","last_clean":"2024-09-21T11:54:50.649993+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:54:50.649993+0000","last_undegraded":"2024-09-21T11:54:50.649993+0000","last_fullsized":"2024-09-21T11:54:50.649993+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:33.055554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:33.055554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:02.654956+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":637,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.673875+0000","last_change":"2024-09-21T11:51:43.547924+0000","last_active":"2024-09-21T11:54:50.673875+0000","last_peered":"2024-09-21T11:54:50.673875+0000","last_clean":"2024-09-21T11:54:50.673875+0000","last_became_active":"2024-09-21T11:51:43.545191+0000","last_became_peered":"2024-09-21T11:51:43.545191+0000","last_unstale":"2024-09-21T11:54:50.673875+0000","last_undegraded":"2024-09-21T11:54:50.673875+0000","last_fullsized":"2024-09-21T11:54:50.673875+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:03.470173+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":638,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.636872+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:54:50.636872+0000","last_peered":"2024-09-21T11:54:50.636872+0000","last_clean":"2024-09-21T11:54:50.636872+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:54:50.636872+0000","last_undegraded":"2024-09-21T11:54:50.636872+0000","last_fullsized":"2024-09-21T11:54:50.636872+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.290553+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":643,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650059+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:54:50.650059+0000","last_peered":"2024-09-21T11:54:50.650059+0000","last_clean":"2024-09-21T11:54:50.650059+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:54:50.650059+0000","last_undegraded":"2024-09-21T11:54:50.650059+0000","last_fullsized":"2024-09-21T11:54:50.650059+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.222133+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":136,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.654068+0000","last_change":"2024-09-21T11:51:43.524599+0000","last_active":"2024-09-21T11:54:50.654068+0000","last_peered":"2024-09-21T11:54:50.654068+0000","last_clean":"2024-09-21T11:54:50.654068+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:54:50.654068+0000","last_undegraded":"2024-09-21T11:54:50.654068+0000","last_fullsized":"2024-09-21T11:54:50.654068+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:32.302842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.302842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.350209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":632,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.675866+0000","last_change":"2024-09-21T11:54:23.070082+0000","last_active":"2024-09-21T11:54:50.675866+0000","last_peered":"2024-09-21T11:54:50.675866+0000","last_clean":"2024-09-21T11:54:50.675866+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:54:50.675866+0000","last_undegraded":"2024-09-21T11:54:50.675866+0000","last_fullsized":"2024-09-21T11:54:50.675866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:23.069995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.069995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:24.259600+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":757,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.647741+0000","last_change":"2024-09-21T11:50:25.966805+0000","last_active":"2024-09-21T11:54:50.647741+0000","last_peered":"2024-09-21T11:54:50.647741+0000","last_clean":"2024-09-21T11:54:50.647741+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:54:50.647741+0000","last_undegraded":"2024-09-21T11:54:50.647741+0000","last_fullsized":"2024-09-21T11:54:50.647741+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:50:25.966740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.966740+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:06.137315+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":343,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.661665+0000","last_change":"2024-09-21T11:53:23.118402+0000","last_active":"2024-09-21T11:54:50.661665+0000","last_peered":"2024-09-21T11:54:50.661665+0000","last_clean":"2024-09-21T11:54:50.661665+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:54:50.661665+0000","last_undegraded":"2024-09-21T11:54:50.661665+0000","last_fullsized":"2024-09-21T11:54:50.661665+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:23.118315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:23.118315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:48.518742+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":631,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.659805+0000","last_change":"2024-09-21T11:52:44.264180+0000","last_active":"2024-09-21T11:54:50.659805+0000","last_peered":"2024-09-21T11:54:50.659805+0000","last_clean":"2024-09-21T11:54:50.659805+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:54:50.659805+0000","last_undegraded":"2024-09-21T11:54:50.659805+0000","last_fullsized":"2024-09-21T11:54:50.659805+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.264114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.264114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":458,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.662816+0000","last_change":"2024-09-21T11:54:02.517182+0000","last_active":"2024-09-21T11:54:50.662816+0000","last_peered":"2024-09-21T11:54:50.662816+0000","last_clean":"2024-09-21T11:54:50.662816+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:54:50.662816+0000","last_undegraded":"2024-09-21T11:54:50.662816+0000","last_fullsized":"2024-09-21T11:54:50.662816+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:02.517119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:02.517119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:26.878233+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":707,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.654671+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:54:50.654671+0000","last_peered":"2024-09-21T11:54:50.654671+0000","last_clean":"2024-09-21T11:54:50.654671+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:54:50.654671+0000","last_undegraded":"2024-09-21T11:54:50.654671+0000","last_fullsized":"2024-09-21T11:54:50.654671+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:04.849420+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":144,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.656104+0000","last_change":"2024-09-21T11:53:55.486849+0000","last_active":"2024-09-21T11:54:50.656104+0000","last_peered":"2024-09-21T11:54:50.656104+0000","last_clean":"2024-09-21T11:54:50.656104+0000","last_became_active":"2024-09-21T11:51:43.548726+0000","last_became_peered":"2024-09-21T11:51:43.548726+0000","last_unstale":"2024-09-21T11:54:50.656104+0000","last_undegraded":"2024-09-21T11:54:50.656104+0000","last_fullsized":"2024-09-21T11:54:50.656104+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:55.486676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:55.486676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:12.803070+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":147,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.668431+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:54:50.668431+0000","last_peered":"2024-09-21T11:54:50.668431+0000","last_clean":"2024-09-21T11:54:50.668431+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:54:50.668431+0000","last_undegraded":"2024-09-21T11:54:50.668431+0000","last_fullsized":"2024-09-21T11:54:50.668431+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.101192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":347,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.662802+0000","last_change":"2024-09-21T11:54:33.391503+0000","last_active":"2024-09-21T11:54:50.662802+0000","last_peered":"2024-09-21T11:54:50.662802+0000","last_clean":"2024-09-21T11:54:50.662802+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:54:50.662802+0000","last_undegraded":"2024-09-21T11:54:50.662802+0000","last_fullsized":"2024-09-21T11:54:50.662802+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:54:33.391425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:33.391425+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:48.197809+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":676,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.666727+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:54:50.666727+0000","last_peered":"2024-09-21T11:54:50.666727+0000","last_clean":"2024-09-21T11:54:50.666727+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:54:50.666727+0000","last_undegraded":"2024-09-21T11:54:50.666727+0000","last_fullsized":"2024-09-21T11:54:50.666727+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":840,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.654217+0000","last_change":"2024-09-21T11:51:44.478918+0000","last_active":"2024-09-21T11:54:50.654217+0000","last_peered":"2024-09-21T11:54:50.654217+0000","last_clean":"2024-09-21T11:54:50.654217+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:54:50.654217+0000","last_undegraded":"2024-09-21T11:54:50.654217+0000","last_fullsized":"2024-09-21T11:54:50.654217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:44.478855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.478855+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.275587+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"355'9599","reported_seq":9763,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.184002+0000","last_change":"2024-09-21T11:53:57.419202+0000","last_active":"2024-09-21T11:54:51.184002+0000","last_peered":"2024-09-21T11:54:51.184002+0000","last_clean":"2024-09-21T11:54:51.184002+0000","last_became_active":"2024-09-21T11:52:45.608448+0000","last_became_peered":"2024-09-21T11:52:45.608448+0000","last_unstale":"2024-09-21T11:54:51.184002+0000","last_undegraded":"2024-09-21T11:54:51.184002+0000","last_fullsized":"2024-09-21T11:54:51.184002+0000","mapping_epoch":313,"log_start":"331'4300","ondisk_log_start":"331'4300","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'6082","last_scrub_stamp":"2024-09-21T11:53:57.418961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:57.418961+0000","objects_scrubbed":6033,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:20.883049+0000","scrub_duration":2364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368115712,"num_objects":5617,"num_object_clones":0,"num_object_copies":16851,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":486912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":377,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.661981+0000","last_change":"2024-09-21T11:52:39.116264+0000","last_active":"2024-09-21T11:54:50.661981+0000","last_peered":"2024-09-21T11:54:50.661981+0000","last_clean":"2024-09-21T11:54:50.661981+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:54:50.661981+0000","last_undegraded":"2024-09-21T11:54:50.661981+0000","last_fullsized":"2024-09-21T11:54:50.661981+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:52:39.116197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:39.116197+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:47.545993+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":664,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.655523+0000","last_change":"2024-09-21T11:53:30.353707+0000","last_active":"2024-09-21T11:54:50.655523+0000","last_peered":"2024-09-21T11:54:50.655523+0000","last_clean":"2024-09-21T11:54:50.655523+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:54:50.655523+0000","last_undegraded":"2024-09-21T11:54:50.655523+0000","last_fullsized":"2024-09-21T11:54:50.655523+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:30.353632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:30.353632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"294'9","reported_seq":201,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.652278+0000","last_change":"2024-09-21T11:51:43.526079+0000","last_active":"2024-09-21T11:54:50.652278+0000","last_peered":"2024-09-21T11:54:50.652278+0000","last_clean":"2024-09-21T11:54:50.652278+0000","last_became_active":"2024-09-21T11:51:43.525566+0000","last_became_peered":"2024-09-21T11:51:43.525566+0000","last_unstale":"2024-09-21T11:54:50.652278+0000","last_undegraded":"2024-09-21T11:54:50.652278+0000","last_fullsized":"2024-09-21T11:54:50.652278+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:11.932479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:11.932479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:16.143901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":659,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650267+0000","last_change":"2024-09-21T11:53:52.178001+0000","last_active":"2024-09-21T11:54:50.650267+0000","last_peered":"2024-09-21T11:54:50.650267+0000","last_clean":"2024-09-21T11:54:50.650267+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:54:50.650267+0000","last_undegraded":"2024-09-21T11:54:50.650267+0000","last_fullsized":"2024-09-21T11:54:50.650267+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:52.177812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:52.177812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:05.487897+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"355'9668","reported_seq":9835,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.279269+0000","last_change":"2024-09-21T11:54:02.094582+0000","last_active":"2024-09-21T11:54:51.279269+0000","last_peered":"2024-09-21T11:54:51.279269+0000","last_clean":"2024-09-21T11:54:51.279269+0000","last_became_active":"2024-09-21T11:52:45.606226+0000","last_became_peered":"2024-09-21T11:52:45.606226+0000","last_unstale":"2024-09-21T11:54:51.279269+0000","last_undegraded":"2024-09-21T11:54:51.279269+0000","last_fullsized":"2024-09-21T11:54:51.279269+0000","mapping_epoch":313,"log_start":"327'3600","ondisk_log_start":"327'3600","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"340'6367","last_scrub_stamp":"2024-09-21T11:54:02.094356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:02.094356+0000","objects_scrubbed":6327,"log_size":6068,"log_dups_size":0,"ondisk_log_size":6068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.011413+0000","scrub_duration":1665,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373555200,"num_objects":5700,"num_object_clones":0,"num_object_copies":17100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9668,"num_write_kb":491776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":697,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.654987+0000","last_change":"2024-09-21T11:52:47.279483+0000","last_active":"2024-09-21T11:54:50.654987+0000","last_peered":"2024-09-21T11:54:50.654987+0000","last_clean":"2024-09-21T11:54:50.654987+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:54:50.654987+0000","last_undegraded":"2024-09-21T11:54:50.654987+0000","last_fullsized":"2024-09-21T11:54:50.654987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:52:47.279423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:47.279423+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":650,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.670697+0000","last_change":"2024-09-21T11:52:42.279545+0000","last_active":"2024-09-21T11:54:50.670697+0000","last_peered":"2024-09-21T11:54:50.670697+0000","last_clean":"2024-09-21T11:54:50.670697+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:54:50.670697+0000","last_undegraded":"2024-09-21T11:54:50.670697+0000","last_fullsized":"2024-09-21T11:54:50.670697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:42.279496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:42.279496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":136,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.657334+0000","last_change":"2024-09-21T11:51:43.542323+0000","last_active":"2024-09-21T11:54:50.657334+0000","last_peered":"2024-09-21T11:54:50.657334+0000","last_clean":"2024-09-21T11:54:50.657334+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:54:50.657334+0000","last_undegraded":"2024-09-21T11:54:50.657334+0000","last_fullsized":"2024-09-21T11:54:50.657334+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.940335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.940335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:50.467868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":138,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.646936+0000","last_change":"2024-09-21T11:53:27.999159+0000","last_active":"2024-09-21T11:54:50.646936+0000","last_peered":"2024-09-21T11:54:50.646936+0000","last_clean":"2024-09-21T11:54:50.646936+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:54:50.646936+0000","last_undegraded":"2024-09-21T11:54:50.646936+0000","last_fullsized":"2024-09-21T11:54:50.646936+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.999084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.999084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"355'9903","reported_seq":9998,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.281454+0000","last_change":"2024-09-21T11:52:45.635813+0000","last_active":"2024-09-21T11:54:51.281454+0000","last_peered":"2024-09-21T11:54:51.281454+0000","last_clean":"2024-09-21T11:54:51.281454+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:54:51.281454+0000","last_undegraded":"2024-09-21T11:54:51.281454+0000","last_fullsized":"2024-09-21T11:54:51.281454+0000","mapping_epoch":313,"log_start":"328'3900","ondisk_log_start":"328'3900","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":6003,"log_dups_size":0,"ondisk_log_size":6003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":379781120,"num_objects":5795,"num_object_clones":0,"num_object_copies":17385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9903,"num_write_kb":502336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":648,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650162+0000","last_change":"2024-09-21T11:51:43.525478+0000","last_active":"2024-09-21T11:54:50.650162+0000","last_peered":"2024-09-21T11:54:50.650162+0000","last_clean":"2024-09-21T11:54:50.650162+0000","last_became_active":"2024-09-21T11:51:43.523852+0000","last_became_peered":"2024-09-21T11:51:43.523852+0000","last_unstale":"2024-09-21T11:54:50.650162+0000","last_undegraded":"2024-09-21T11:54:50.650162+0000","last_fullsized":"2024-09-21T11:54:50.650162+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.142710+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":672,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.667012+0000","last_change":"2024-09-21T11:50:28.657341+0000","last_active":"2024-09-21T11:54:50.667012+0000","last_peered":"2024-09-21T11:54:50.667012+0000","last_clean":"2024-09-21T11:54:50.667012+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:54:50.667012+0000","last_undegraded":"2024-09-21T11:54:50.667012+0000","last_fullsized":"2024-09-21T11:54:50.667012+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:28.657261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.657261+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.097296+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":845,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.655161+0000","last_change":"2024-09-21T11:51:20.675165+0000","last_active":"2024-09-21T11:54:50.655161+0000","last_peered":"2024-09-21T11:54:50.655161+0000","last_clean":"2024-09-21T11:54:50.655161+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:54:50.655161+0000","last_undegraded":"2024-09-21T11:54:50.655161+0000","last_fullsized":"2024-09-21T11:54:50.655161+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:51:20.675101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.675101+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:48.486444+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":641,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637012+0000","last_change":"2024-09-21T11:52:14.010215+0000","last_active":"2024-09-21T11:54:50.637012+0000","last_peered":"2024-09-21T11:54:50.637012+0000","last_clean":"2024-09-21T11:54:50.637012+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:54:50.637012+0000","last_undegraded":"2024-09-21T11:54:50.637012+0000","last_fullsized":"2024-09-21T11:54:50.637012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:14.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:14.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:45.192898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"355'9575","reported_seq":9670,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.201790+0000","last_change":"2024-09-21T11:52:45.608507+0000","last_active":"2024-09-21T11:54:51.201790+0000","last_peered":"2024-09-21T11:54:51.201790+0000","last_clean":"2024-09-21T11:54:51.201790+0000","last_became_active":"2024-09-21T11:52:45.608219+0000","last_became_peered":"2024-09-21T11:52:45.608219+0000","last_unstale":"2024-09-21T11:54:51.201790+0000","last_undegraded":"2024-09-21T11:54:51.201790+0000","last_fullsized":"2024-09-21T11:54:51.201790+0000","mapping_epoch":313,"log_start":"331'4300","ondisk_log_start":"331'4300","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5275,"log_dups_size":0,"ondisk_log_size":5275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.363111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":375586816,"num_objects":5731,"num_object_clones":0,"num_object_copies":17193,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9575,"num_write_kb":489792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":362,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.663337+0000","last_change":"2024-09-21T11:53:30.179784+0000","last_active":"2024-09-21T11:54:50.663337+0000","last_peered":"2024-09-21T11:54:50.663337+0000","last_clean":"2024-09-21T11:54:50.663337+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:54:50.663337+0000","last_undegraded":"2024-09-21T11:54:50.663337+0000","last_fullsized":"2024-09-21T11:54:50.663337+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:30.179690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:30.179690+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:59.902366+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":674,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650687+0000","last_change":"2024-09-21T11:54:22.095001+0000","last_active":"2024-09-21T11:54:50.650687+0000","last_peered":"2024-09-21T11:54:50.650687+0000","last_clean":"2024-09-21T11:54:50.650687+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:54:50.650687+0000","last_undegraded":"2024-09-21T11:54:50.650687+0000","last_fullsized":"2024-09-21T11:54:50.650687+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:22.094918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:22.094918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:40.780472+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":824,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.674503+0000","last_change":"2024-09-21T11:53:56.181634+0000","last_active":"2024-09-21T11:54:50.674503+0000","last_peered":"2024-09-21T11:54:50.674503+0000","last_clean":"2024-09-21T11:54:50.674503+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:54:50.674503+0000","last_undegraded":"2024-09-21T11:54:50.674503+0000","last_fullsized":"2024-09-21T11:54:50.674503+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:53:56.181577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:56.181577+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:17.783176+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":646,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.667855+0000","last_change":"2024-09-21T11:53:48.256119+0000","last_active":"2024-09-21T11:54:50.667855+0000","last_peered":"2024-09-21T11:54:50.667855+0000","last_clean":"2024-09-21T11:54:50.667855+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:54:50.667855+0000","last_undegraded":"2024-09-21T11:54:50.667855+0000","last_fullsized":"2024-09-21T11:54:50.667855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:48.255951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:48.255951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.617117+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"355'9541","reported_seq":9637,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.257686+0000","last_change":"2024-09-21T11:52:45.607195+0000","last_active":"2024-09-21T11:54:51.257686+0000","last_peered":"2024-09-21T11:54:51.257686+0000","last_clean":"2024-09-21T11:54:51.257686+0000","last_became_active":"2024-09-21T11:52:45.606865+0000","last_became_peered":"2024-09-21T11:52:45.606865+0000","last_unstale":"2024-09-21T11:54:51.257686+0000","last_undegraded":"2024-09-21T11:54:51.257686+0000","last_fullsized":"2024-09-21T11:54:51.257686+0000","mapping_epoch":313,"log_start":"327'3500","ondisk_log_start":"327'3500","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":6041,"log_dups_size":0,"ondisk_log_size":6041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:44.275772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364511256,"num_objects":5563,"num_object_clones":0,"num_object_copies":16689,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5563,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9541,"num_write_kb":483265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":697,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.675222+0000","last_change":"2024-09-21T11:53:22.488389+0000","last_active":"2024-09-21T11:54:50.675222+0000","last_peered":"2024-09-21T11:54:50.675222+0000","last_clean":"2024-09-21T11:54:50.675222+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:54:50.675222+0000","last_undegraded":"2024-09-21T11:54:50.675222+0000","last_fullsized":"2024-09-21T11:54:50.675222+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:53:22.488238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:22.488238+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":335,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.662427+0000","last_change":"2024-09-21T11:53:27.160558+0000","last_active":"2024-09-21T11:54:50.662427+0000","last_peered":"2024-09-21T11:54:50.662427+0000","last_clean":"2024-09-21T11:54:50.662427+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:54:50.662427+0000","last_undegraded":"2024-09-21T11:54:50.662427+0000","last_fullsized":"2024-09-21T11:54:50.662427+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.160486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.160486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:44.486333+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":634,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.653585+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:54:50.653585+0000","last_peered":"2024-09-21T11:54:50.653585+0000","last_clean":"2024-09-21T11:54:50.653585+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:54:50.653585+0000","last_undegraded":"2024-09-21T11:54:50.653585+0000","last_fullsized":"2024-09-21T11:54:50.653585+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:42.643962+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":335,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.662516+0000","last_change":"2024-09-21T11:52:09.412149+0000","last_active":"2024-09-21T11:54:50.662516+0000","last_peered":"2024-09-21T11:54:50.662516+0000","last_clean":"2024-09-21T11:54:50.662516+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:54:50.662516+0000","last_undegraded":"2024-09-21T11:54:50.662516+0000","last_fullsized":"2024-09-21T11:54:50.662516+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:09.412091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:09.412091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:41.503668+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"355'9604","reported_seq":9699,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.268548+0000","last_change":"2024-09-21T11:52:45.607383+0000","last_active":"2024-09-21T11:54:51.268548+0000","last_peered":"2024-09-21T11:54:51.268548+0000","last_clean":"2024-09-21T11:54:51.268548+0000","last_became_active":"2024-09-21T11:52:45.607000+0000","last_became_peered":"2024-09-21T11:52:45.607000+0000","last_unstale":"2024-09-21T11:54:51.268548+0000","last_undegraded":"2024-09-21T11:54:51.268548+0000","last_fullsized":"2024-09-21T11:54:51.268548+0000","mapping_epoch":313,"log_start":"327'3600","ondisk_log_start":"327'3600","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":6004,"log_dups_size":0,"ondisk_log_size":6004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:49.450213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368574464,"num_objects":5624,"num_object_clones":0,"num_object_copies":16872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9604,"num_write_kb":487296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":461,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.662334+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:54:50.662334+0000","last_peered":"2024-09-21T11:54:50.662334+0000","last_clean":"2024-09-21T11:54:50.662334+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:54:50.662334+0000","last_undegraded":"2024-09-21T11:54:50.662334+0000","last_fullsized":"2024-09-21T11:54:50.662334+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:17.332467+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":161,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:52.312620+0000","last_change":"2024-09-21T11:54:52.312620+0000","last_active":"2024-09-21T11:54:52.312620+0000","last_peered":"2024-09-21T11:54:52.312620+0000","last_clean":"2024-09-21T11:54:52.312620+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:54:52.312620+0000","last_undegraded":"2024-09-21T11:54:52.312620+0000","last_fullsized":"2024-09-21T11:54:52.312620+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:54:52.310060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:52.310060+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:21.355022+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":666,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650408+0000","last_change":"2024-09-21T11:51:12.562026+0000","last_active":"2024-09-21T11:54:50.650408+0000","last_peered":"2024-09-21T11:54:50.650408+0000","last_clean":"2024-09-21T11:54:50.650408+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:54:50.650408+0000","last_undegraded":"2024-09-21T11:54:50.650408+0000","last_fullsized":"2024-09-21T11:54:50.650408+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:51:12.561961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.561961+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:17.150594+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":640,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.666542+0000","last_change":"2024-09-21T11:50:37.858401+0000","last_active":"2024-09-21T11:54:50.666542+0000","last_peered":"2024-09-21T11:54:50.666542+0000","last_clean":"2024-09-21T11:54:50.666542+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:54:50.666542+0000","last_undegraded":"2024-09-21T11:54:50.666542+0000","last_fullsized":"2024-09-21T11:54:50.666542+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:21.058181+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":622,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.655648+0000","last_change":"2024-09-21T11:54:34.452033+0000","last_active":"2024-09-21T11:54:50.655648+0000","last_peered":"2024-09-21T11:54:50.655648+0000","last_clean":"2024-09-21T11:54:50.655648+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:54:50.655648+0000","last_undegraded":"2024-09-21T11:54:50.655648+0000","last_fullsized":"2024-09-21T11:54:50.655648+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:34.451939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:34.451939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:53.805403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"355'9910","reported_seq":10086,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.356801+0000","last_change":"2024-09-21T11:54:31.986720+0000","last_active":"2024-09-21T11:54:51.356801+0000","last_peered":"2024-09-21T11:54:51.356801+0000","last_clean":"2024-09-21T11:54:51.356801+0000","last_became_active":"2024-09-21T11:52:45.604600+0000","last_became_peered":"2024-09-21T11:52:45.604600+0000","last_unstale":"2024-09-21T11:54:51.356801+0000","last_undegraded":"2024-09-21T11:54:51.356801+0000","last_fullsized":"2024-09-21T11:54:51.356801+0000","mapping_epoch":313,"log_start":"330'4300","ondisk_log_start":"330'4300","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"349'8491","last_scrub_stamp":"2024-09-21T11:54:31.986575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:31.986575+0000","objects_scrubbed":7209,"log_size":5610,"log_dups_size":0,"ondisk_log_size":5610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:50.526900+0000","scrub_duration":1918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":374865920,"num_objects":5720,"num_object_clones":0,"num_object_copies":17160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9910,"num_write_kb":500160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":278,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.660052+0000","last_change":"2024-09-21T11:51:41.006710+0000","last_active":"2024-09-21T11:54:50.660052+0000","last_peered":"2024-09-21T11:54:50.660052+0000","last_clean":"2024-09-21T11:54:50.660052+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:54:50.660052+0000","last_undegraded":"2024-09-21T11:54:50.660052+0000","last_fullsized":"2024-09-21T11:54:50.660052+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:51:41.006622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:41.006622+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:21.055561+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":339,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.661453+0000","last_change":"2024-09-21T11:51:43.346789+0000","last_active":"2024-09-21T11:54:50.661453+0000","last_peered":"2024-09-21T11:54:50.661453+0000","last_clean":"2024-09-21T11:54:50.661453+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:54:50.661453+0000","last_undegraded":"2024-09-21T11:54:50.661453+0000","last_fullsized":"2024-09-21T11:54:50.661453+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:43.346718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.346718+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.466879+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":676,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.647343+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:54:50.647343+0000","last_peered":"2024-09-21T11:54:50.647343+0000","last_clean":"2024-09-21T11:54:50.647343+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:54:50.647343+0000","last_undegraded":"2024-09-21T11:54:50.647343+0000","last_fullsized":"2024-09-21T11:54:50.647343+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:44.933216+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":634,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.649824+0000","last_change":"2024-09-21T11:49:47.008541+0000","last_active":"2024-09-21T11:54:50.649824+0000","last_peered":"2024-09-21T11:54:50.649824+0000","last_clean":"2024-09-21T11:54:50.649824+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:54:50.649824+0000","last_undegraded":"2024-09-21T11:54:50.649824+0000","last_fullsized":"2024-09-21T11:54:50.649824+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.008369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.008369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:03.066869+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":638,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637333+0000","last_change":"2024-09-21T11:54:33.253430+0000","last_active":"2024-09-21T11:54:50.637333+0000","last_peered":"2024-09-21T11:54:50.637333+0000","last_clean":"2024-09-21T11:54:50.637333+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:54:50.637333+0000","last_undegraded":"2024-09-21T11:54:50.637333+0000","last_fullsized":"2024-09-21T11:54:50.637333+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:33.253368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:33.253368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:58.365025+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"355'9687","reported_seq":9782,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:52.316447+0000","last_change":"2024-09-21T11:52:45.604487+0000","last_active":"2024-09-21T11:54:52.316447+0000","last_peered":"2024-09-21T11:54:52.316447+0000","last_clean":"2024-09-21T11:54:52.316447+0000","last_became_active":"2024-09-21T11:52:45.604244+0000","last_became_peered":"2024-09-21T11:52:45.604244+0000","last_unstale":"2024-09-21T11:54:52.316447+0000","last_undegraded":"2024-09-21T11:54:52.316447+0000","last_fullsized":"2024-09-21T11:54:52.316447+0000","mapping_epoch":313,"log_start":"331'4400","ondisk_log_start":"331'4400","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5287,"log_dups_size":0,"ondisk_log_size":5287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368377856,"num_objects":5621,"num_object_clones":0,"num_object_copies":16863,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9687,"num_write_kb":489856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":715,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.666816+0000","last_change":"2024-09-21T11:53:20.438262+0000","last_active":"2024-09-21T11:54:50.666816+0000","last_peered":"2024-09-21T11:54:50.666816+0000","last_clean":"2024-09-21T11:54:50.666816+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:54:50.666816+0000","last_undegraded":"2024-09-21T11:54:50.666816+0000","last_fullsized":"2024-09-21T11:54:50.666816+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:53:20.438174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.438174+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":658,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.666174+0000","last_change":"2024-09-21T11:54:32.060813+0000","last_active":"2024-09-21T11:54:50.666174+0000","last_peered":"2024-09-21T11:54:50.666174+0000","last_clean":"2024-09-21T11:54:50.666174+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:54:50.666174+0000","last_undegraded":"2024-09-21T11:54:50.666174+0000","last_fullsized":"2024-09-21T11:54:50.666174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:32.060638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:32.060638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:43.748498+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1574,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.636816+0000","last_change":"2024-09-21T11:54:38.270801+0000","last_active":"2024-09-21T11:54:50.636816+0000","last_peered":"2024-09-21T11:54:50.636816+0000","last_clean":"2024-09-21T11:54:50.636816+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:54:50.636816+0000","last_undegraded":"2024-09-21T11:54:50.636816+0000","last_fullsized":"2024-09-21T11:54:50.636816+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:54:38.270748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.270748+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:51.553375+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":881,"num_read_kb":876,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":647,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.651276+0000","last_change":"2024-09-21T11:51:17.645920+0000","last_active":"2024-09-21T11:54:50.651276+0000","last_peered":"2024-09-21T11:54:50.651276+0000","last_clean":"2024-09-21T11:54:50.651276+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:54:50.651276+0000","last_undegraded":"2024-09-21T11:54:50.651276+0000","last_fullsized":"2024-09-21T11:54:50.651276+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.645823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.645823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.111560+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":343,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.662240+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:54:50.662240+0000","last_peered":"2024-09-21T11:54:50.662240+0000","last_clean":"2024-09-21T11:54:50.662240+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:54:50.662240+0000","last_undegraded":"2024-09-21T11:54:50.662240+0000","last_fullsized":"2024-09-21T11:54:50.662240+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.479171+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"355'9616","reported_seq":9711,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.200803+0000","last_change":"2024-09-21T11:52:45.607096+0000","last_active":"2024-09-21T11:54:51.200803+0000","last_peered":"2024-09-21T11:54:51.200803+0000","last_clean":"2024-09-21T11:54:51.200803+0000","last_became_active":"2024-09-21T11:52:45.606777+0000","last_became_peered":"2024-09-21T11:52:45.606777+0000","last_unstale":"2024-09-21T11:54:51.200803+0000","last_undegraded":"2024-09-21T11:54:51.200803+0000","last_fullsized":"2024-09-21T11:54:51.200803+0000","mapping_epoch":313,"log_start":"327'3600","ondisk_log_start":"327'3600","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":6016,"log_dups_size":0,"ondisk_log_size":6016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:48.437837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":365821952,"num_objects":5582,"num_object_clones":0,"num_object_copies":16746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9616,"num_write_kb":486336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":865,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.653922+0000","last_change":"2024-09-21T11:54:35.232137+0000","last_active":"2024-09-21T11:54:50.653922+0000","last_peered":"2024-09-21T11:54:50.653922+0000","last_clean":"2024-09-21T11:54:50.653922+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:54:50.653922+0000","last_undegraded":"2024-09-21T11:54:50.653922+0000","last_fullsized":"2024-09-21T11:54:50.653922+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:54:35.231924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:35.231924+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:51.395140+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":643,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.654834+0000","last_change":"2024-09-21T11:50:59.680952+0000","last_active":"2024-09-21T11:54:50.654834+0000","last_peered":"2024-09-21T11:54:50.654834+0000","last_clean":"2024-09-21T11:54:50.654834+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:54:50.654834+0000","last_undegraded":"2024-09-21T11:54:50.654834+0000","last_fullsized":"2024-09-21T11:54:50.654834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:59.680755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:59.680755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:21.608988+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":713,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.636913+0000","last_change":"2024-09-21T11:54:22.360499+0000","last_active":"2024-09-21T11:54:50.636913+0000","last_peered":"2024-09-21T11:54:50.636913+0000","last_clean":"2024-09-21T11:54:50.636913+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:54:50.636913+0000","last_undegraded":"2024-09-21T11:54:50.636913+0000","last_fullsized":"2024-09-21T11:54:50.636913+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:54:22.360410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:54:22.360410+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:32.516643+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":642,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637287+0000","last_change":"2024-09-21T11:51:18.318095+0000","last_active":"2024-09-21T11:54:50.637287+0000","last_peered":"2024-09-21T11:54:50.637287+0000","last_clean":"2024-09-21T11:54:50.637287+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:54:50.637287+0000","last_undegraded":"2024-09-21T11:54:50.637287+0000","last_fullsized":"2024-09-21T11:54:50.637287+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.317950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.317950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:11.261207+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":637,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.651133+0000","last_change":"2024-09-21T11:53:31.966692+0000","last_active":"2024-09-21T11:54:50.651133+0000","last_peered":"2024-09-21T11:54:50.651133+0000","last_clean":"2024-09-21T11:54:50.651133+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:54:50.651133+0000","last_undegraded":"2024-09-21T11:54:50.651133+0000","last_fullsized":"2024-09-21T11:54:50.651133+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:31.966508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:31.966508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"355'9813","reported_seq":9976,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.236914+0000","last_change":"2024-09-21T11:53:53.487635+0000","last_active":"2024-09-21T11:54:51.236914+0000","last_peered":"2024-09-21T11:54:51.236914+0000","last_clean":"2024-09-21T11:54:51.236914+0000","last_became_active":"2024-09-21T11:52:45.605575+0000","last_became_peered":"2024-09-21T11:52:45.605575+0000","last_unstale":"2024-09-21T11:54:51.236914+0000","last_undegraded":"2024-09-21T11:54:51.236914+0000","last_fullsized":"2024-09-21T11:54:51.236914+0000","mapping_epoch":313,"log_start":"327'3800","ondisk_log_start":"327'3800","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'5948","last_scrub_stamp":"2024-09-21T11:53:53.487473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:53.487473+0000","objects_scrubbed":5891,"log_size":6013,"log_dups_size":0,"ondisk_log_size":6013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:06.519835+0000","scrub_duration":1988,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":372310016,"num_objects":5681,"num_object_clones":0,"num_object_copies":17043,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9813,"num_write_kb":495808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":659,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.654070+0000","last_change":"2024-09-21T11:53:58.427767+0000","last_active":"2024-09-21T11:54:50.654070+0000","last_peered":"2024-09-21T11:54:50.654070+0000","last_clean":"2024-09-21T11:54:50.654070+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:54:50.654070+0000","last_undegraded":"2024-09-21T11:54:50.654070+0000","last_fullsized":"2024-09-21T11:54:50.654070+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:58.427708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:58.427708+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:08.495842+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":144,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.647146+0000","last_change":"2024-09-21T11:52:13.709151+0000","last_active":"2024-09-21T11:54:50.647146+0000","last_peered":"2024-09-21T11:54:50.647146+0000","last_clean":"2024-09-21T11:54:50.647146+0000","last_became_active":"2024-09-21T11:51:43.532027+0000","last_became_peered":"2024-09-21T11:51:43.532027+0000","last_unstale":"2024-09-21T11:54:50.647146+0000","last_undegraded":"2024-09-21T11:54:50.647146+0000","last_fullsized":"2024-09-21T11:54:50.647146+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:13.709074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:13.709074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:38.928791+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":625,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650950+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:54:50.650950+0000","last_peered":"2024-09-21T11:54:50.650950+0000","last_clean":"2024-09-21T11:54:50.650950+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:54:50.650950+0000","last_undegraded":"2024-09-21T11:54:50.650950+0000","last_fullsized":"2024-09-21T11:54:50.650950+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:05.297159+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":644,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.656025+0000","last_change":"2024-09-21T11:53:52.437233+0000","last_active":"2024-09-21T11:54:50.656025+0000","last_peered":"2024-09-21T11:54:50.656025+0000","last_clean":"2024-09-21T11:54:50.656025+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:54:50.656025+0000","last_undegraded":"2024-09-21T11:54:50.656025+0000","last_fullsized":"2024-09-21T11:54:50.656025+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:52.437166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:52.437166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:02.147272+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"355'9609","reported_seq":9775,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.565512+0000","last_change":"2024-09-21T11:54:00.488606+0000","last_active":"2024-09-21T11:54:51.565512+0000","last_peered":"2024-09-21T11:54:51.565512+0000","last_clean":"2024-09-21T11:54:51.565512+0000","last_became_active":"2024-09-21T11:52:45.637825+0000","last_became_peered":"2024-09-21T11:52:45.637825+0000","last_unstale":"2024-09-21T11:54:51.565512+0000","last_undegraded":"2024-09-21T11:54:51.565512+0000","last_fullsized":"2024-09-21T11:54:51.565512+0000","mapping_epoch":313,"log_start":"324'2900","ondisk_log_start":"324'2900","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'6224","last_scrub_stamp":"2024-09-21T11:54:00.488535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.488535+0000","objects_scrubbed":6155,"log_size":6709,"log_dups_size":0,"ondisk_log_size":6709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:21.189007+0000","scrub_duration":2345,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363921408,"num_objects":5553,"num_object_clones":0,"num_object_copies":16659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9609,"num_write_kb":485184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":722,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.649328+0000","last_change":"2024-09-21T11:51:43.533280+0000","last_active":"2024-09-21T11:54:50.649328+0000","last_peered":"2024-09-21T11:54:50.649328+0000","last_clean":"2024-09-21T11:54:50.649328+0000","last_became_active":"2024-09-21T11:51:43.525941+0000","last_became_peered":"2024-09-21T11:51:43.525941+0000","last_unstale":"2024-09-21T11:54:50.649328+0000","last_undegraded":"2024-09-21T11:54:50.649328+0000","last_fullsized":"2024-09-21T11:54:50.649328+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:50:36.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.385902+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:08.215334+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":671,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.653915+0000","last_change":"2024-09-21T11:54:36.208783+0000","last_active":"2024-09-21T11:54:50.653915+0000","last_peered":"2024-09-21T11:54:50.653915+0000","last_clean":"2024-09-21T11:54:50.653915+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:54:50.653915+0000","last_undegraded":"2024-09-21T11:54:50.653915+0000","last_fullsized":"2024-09-21T11:54:50.653915+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:36.208720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:36.208720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:46.586464+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":335,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.662151+0000","last_change":"2024-09-21T11:52:44.976410+0000","last_active":"2024-09-21T11:54:50.662151+0000","last_peered":"2024-09-21T11:54:50.662151+0000","last_clean":"2024-09-21T11:54:50.662151+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:54:50.662151+0000","last_undegraded":"2024-09-21T11:54:50.662151+0000","last_fullsized":"2024-09-21T11:54:50.662151+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.976363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.976363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:06.448787+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":638,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.660235+0000","last_change":"2024-09-21T11:51:47.944684+0000","last_active":"2024-09-21T11:54:50.660235+0000","last_peered":"2024-09-21T11:54:50.660235+0000","last_clean":"2024-09-21T11:54:50.660235+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:54:50.660235+0000","last_undegraded":"2024-09-21T11:54:50.660235+0000","last_fullsized":"2024-09-21T11:54:50.660235+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:47.944609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.944609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.139462+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"355'9779","reported_seq":9941,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.393829+0000","last_change":"2024-09-21T11:53:53.011097+0000","last_active":"2024-09-21T11:54:51.393829+0000","last_peered":"2024-09-21T11:54:51.393829+0000","last_clean":"2024-09-21T11:54:51.393829+0000","last_became_active":"2024-09-21T11:52:45.601350+0000","last_became_peered":"2024-09-21T11:52:45.601350+0000","last_unstale":"2024-09-21T11:54:51.393829+0000","last_undegraded":"2024-09-21T11:54:51.393829+0000","last_fullsized":"2024-09-21T11:54:51.393829+0000","mapping_epoch":313,"log_start":"332'4500","ondisk_log_start":"332'4500","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'5855","last_scrub_stamp":"2024-09-21T11:53:53.011008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:53.011008+0000","objects_scrubbed":5813,"log_size":5279,"log_dups_size":0,"ondisk_log_size":5279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:10.597953+0000","scrub_duration":1614,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369164288,"num_objects":5633,"num_object_clones":0,"num_object_copies":16899,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":493184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":636,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.656325+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:54:50.656325+0000","last_peered":"2024-09-21T11:54:50.656325+0000","last_clean":"2024-09-21T11:54:50.656325+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:54:50.656325+0000","last_undegraded":"2024-09-21T11:54:50.656325+0000","last_fullsized":"2024-09-21T11:54:50.656325+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:19.572942+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":634,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.674043+0000","last_change":"2024-09-21T11:53:22.315543+0000","last_active":"2024-09-21T11:54:50.674043+0000","last_peered":"2024-09-21T11:54:50.674043+0000","last_clean":"2024-09-21T11:54:50.674043+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:54:50.674043+0000","last_undegraded":"2024-09-21T11:54:50.674043+0000","last_fullsized":"2024-09-21T11:54:50.674043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:22.315149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:22.315149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":837,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.649297+0000","last_change":"2024-09-21T11:53:31.991392+0000","last_active":"2024-09-21T11:54:50.649297+0000","last_peered":"2024-09-21T11:54:50.649297+0000","last_clean":"2024-09-21T11:54:50.649297+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:54:50.649297+0000","last_undegraded":"2024-09-21T11:54:50.649297+0000","last_fullsized":"2024-09-21T11:54:50.649297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:53:31.991316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:31.991316+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:39.887323+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":455,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.671482+0000","last_change":"2024-09-21T11:52:52.981662+0000","last_active":"2024-09-21T11:54:50.671482+0000","last_peered":"2024-09-21T11:54:50.671482+0000","last_clean":"2024-09-21T11:54:50.671482+0000","last_became_active":"2024-09-21T11:51:43.543072+0000","last_became_peered":"2024-09-21T11:51:43.543072+0000","last_unstale":"2024-09-21T11:54:50.671482+0000","last_undegraded":"2024-09-21T11:54:50.671482+0000","last_fullsized":"2024-09-21T11:54:50.671482+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:52.981586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:52.981586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:05.493607+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"355'9656","reported_seq":9751,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.578415+0000","last_change":"2024-09-21T11:52:45.635909+0000","last_active":"2024-09-21T11:54:51.578415+0000","last_peered":"2024-09-21T11:54:51.578415+0000","last_clean":"2024-09-21T11:54:51.578415+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:54:51.578415+0000","last_undegraded":"2024-09-21T11:54:51.578415+0000","last_fullsized":"2024-09-21T11:54:51.578415+0000","mapping_epoch":313,"log_start":"324'2900","ondisk_log_start":"324'2900","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":6756,"log_dups_size":0,"ondisk_log_size":6756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:12.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367263744,"num_objects":5604,"num_object_clones":0,"num_object_copies":16812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9656,"num_write_kb":488320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":680,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.674606+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:54:50.674606+0000","last_peered":"2024-09-21T11:54:50.674606+0000","last_clean":"2024-09-21T11:54:50.674606+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:54:50.674606+0000","last_undegraded":"2024-09-21T11:54:50.674606+0000","last_fullsized":"2024-09-21T11:54:50.674606+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.191640+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":665,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650135+0000","last_change":"2024-09-21T11:53:56.166604+0000","last_active":"2024-09-21T11:54:50.650135+0000","last_peered":"2024-09-21T11:54:50.650135+0000","last_clean":"2024-09-21T11:54:50.650135+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:54:50.650135+0000","last_undegraded":"2024-09-21T11:54:50.650135+0000","last_fullsized":"2024-09-21T11:54:50.650135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:56.166533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:56.166533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.929990+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":634,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.647134+0000","last_change":"2024-09-21T11:53:26.990691+0000","last_active":"2024-09-21T11:54:50.647134+0000","last_peered":"2024-09-21T11:54:50.647134+0000","last_clean":"2024-09-21T11:54:50.647134+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:54:50.647134+0000","last_undegraded":"2024-09-21T11:54:50.647134+0000","last_fullsized":"2024-09-21T11:54:50.647134+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.990626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.990626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:45.922336+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":621,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.666070+0000","last_change":"2024-09-21T11:53:31.442098+0000","last_active":"2024-09-21T11:54:50.666070+0000","last_peered":"2024-09-21T11:54:50.666070+0000","last_clean":"2024-09-21T11:54:50.666070+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:54:50.666070+0000","last_undegraded":"2024-09-21T11:54:50.666070+0000","last_fullsized":"2024-09-21T11:54:50.666070+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:31.442027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:31.442027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"355'9774","reported_seq":9940,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:52.319180+0000","last_change":"2024-09-21T11:54:01.780305+0000","last_active":"2024-09-21T11:54:52.319180+0000","last_peered":"2024-09-21T11:54:52.319180+0000","last_clean":"2024-09-21T11:54:52.319180+0000","last_became_active":"2024-09-21T11:52:45.636291+0000","last_became_peered":"2024-09-21T11:52:45.636291+0000","last_unstale":"2024-09-21T11:54:52.319180+0000","last_undegraded":"2024-09-21T11:54:52.319180+0000","last_fullsized":"2024-09-21T11:54:52.319180+0000","mapping_epoch":313,"log_start":"331'4500","ondisk_log_start":"331'4500","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"340'6353","last_scrub_stamp":"2024-09-21T11:54:01.780225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.780225+0000","objects_scrubbed":6266,"log_size":5274,"log_dups_size":0,"ondisk_log_size":5274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:15.413281+0000","scrub_duration":2578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368574464,"num_objects":5624,"num_object_clones":0,"num_object_copies":16872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9774,"num_write_kb":492736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":754,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.653739+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:54:50.653739+0000","last_peered":"2024-09-21T11:54:50.653739+0000","last_clean":"2024-09-21T11:54:50.653739+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:54:50.653739+0000","last_undegraded":"2024-09-21T11:54:50.653739+0000","last_fullsized":"2024-09-21T11:54:50.653739+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.382566+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":641,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637492+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:54:50.637492+0000","last_peered":"2024-09-21T11:54:50.637492+0000","last_clean":"2024-09-21T11:54:50.637492+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:54:50.637492+0000","last_undegraded":"2024-09-21T11:54:50.637492+0000","last_fullsized":"2024-09-21T11:54:50.637492+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:43.232503+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":650,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.647580+0000","last_change":"2024-09-21T11:51:48.941202+0000","last_active":"2024-09-21T11:54:50.647580+0000","last_peered":"2024-09-21T11:54:50.647580+0000","last_clean":"2024-09-21T11:54:50.647580+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:54:50.647580+0000","last_undegraded":"2024-09-21T11:54:50.647580+0000","last_fullsized":"2024-09-21T11:54:50.647580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:48.941128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:48.941128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:17.103034+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":637,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.636851+0000","last_change":"2024-09-21T11:52:12.100354+0000","last_active":"2024-09-21T11:54:50.636851+0000","last_peered":"2024-09-21T11:54:50.636851+0000","last_clean":"2024-09-21T11:54:50.636851+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:54:50.636851+0000","last_undegraded":"2024-09-21T11:54:50.636851+0000","last_fullsized":"2024-09-21T11:54:50.636851+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:44.217224+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"355'9736","reported_seq":9831,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.272954+0000","last_change":"2024-09-21T11:52:45.636994+0000","last_active":"2024-09-21T11:54:51.272954+0000","last_peered":"2024-09-21T11:54:51.272954+0000","last_clean":"2024-09-21T11:54:51.272954+0000","last_became_active":"2024-09-21T11:52:45.636672+0000","last_became_peered":"2024-09-21T11:52:45.636672+0000","last_unstale":"2024-09-21T11:54:51.272954+0000","last_undegraded":"2024-09-21T11:54:51.272954+0000","last_fullsized":"2024-09-21T11:54:51.272954+0000","mapping_epoch":313,"log_start":"327'3700","ondisk_log_start":"327'3700","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":6036,"log_dups_size":0,"ondisk_log_size":6036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":375259136,"num_objects":5726,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9736,"num_write_kb":494784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":676,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.662700+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:54:50.662700+0000","last_peered":"2024-09-21T11:54:50.662700+0000","last_clean":"2024-09-21T11:54:50.662700+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:54:50.662700+0000","last_undegraded":"2024-09-21T11:54:50.662700+0000","last_fullsized":"2024-09-21T11:54:50.662700+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:11.511710+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":679,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637740+0000","last_change":"2024-09-21T11:53:58.230600+0000","last_active":"2024-09-21T11:54:50.637740+0000","last_peered":"2024-09-21T11:54:50.637740+0000","last_clean":"2024-09-21T11:54:50.637740+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:54:50.637740+0000","last_undegraded":"2024-09-21T11:54:50.637740+0000","last_fullsized":"2024-09-21T11:54:50.637740+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:58.230513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:58.230513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.582409+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":664,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.653374+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:54:50.653374+0000","last_peered":"2024-09-21T11:54:50.653374+0000","last_clean":"2024-09-21T11:54:50.653374+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:54:50.653374+0000","last_undegraded":"2024-09-21T11:54:50.653374+0000","last_fullsized":"2024-09-21T11:54:50.653374+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.693324+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":644,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.669539+0000","last_change":"2024-09-21T11:51:09.660956+0000","last_active":"2024-09-21T11:54:50.669539+0000","last_peered":"2024-09-21T11:54:50.669539+0000","last_clean":"2024-09-21T11:54:50.669539+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:54:50.669539+0000","last_undegraded":"2024-09-21T11:54:50.669539+0000","last_fullsized":"2024-09-21T11:54:50.669539+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:09.660880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.660880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:47.501067+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"355'9602","reported_seq":9763,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:51.321680+0000","last_change":"2024-09-21T11:53:51.496758+0000","last_active":"2024-09-21T11:54:51.321680+0000","last_peered":"2024-09-21T11:54:51.321680+0000","last_clean":"2024-09-21T11:54:51.321680+0000","last_became_active":"2024-09-21T11:52:45.603101+0000","last_became_peered":"2024-09-21T11:52:45.603101+0000","last_unstale":"2024-09-21T11:54:51.321680+0000","last_undegraded":"2024-09-21T11:54:51.321680+0000","last_fullsized":"2024-09-21T11:54:51.321680+0000","mapping_epoch":313,"log_start":"329'4000","ondisk_log_start":"329'4000","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"337'5707","last_scrub_stamp":"2024-09-21T11:53:51.496521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:51.496521+0000","objects_scrubbed":5625,"log_size":5602,"log_dups_size":0,"ondisk_log_size":5602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.557812+0000","scrub_duration":2450,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370409472,"num_objects":5652,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9602,"num_write_kb":488128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":680,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.649719+0000","last_change":"2024-09-21T11:53:29.958829+0000","last_active":"2024-09-21T11:54:50.649719+0000","last_peered":"2024-09-21T11:54:50.649719+0000","last_clean":"2024-09-21T11:54:50.649719+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:54:50.649719+0000","last_undegraded":"2024-09-21T11:54:50.649719+0000","last_fullsized":"2024-09-21T11:54:50.649719+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:53:29.958746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.958746+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":669,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637650+0000","last_change":"2024-09-21T11:52:43.086438+0000","last_active":"2024-09-21T11:54:50.637650+0000","last_peered":"2024-09-21T11:54:50.637650+0000","last_clean":"2024-09-21T11:54:50.637650+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:54:50.637650+0000","last_undegraded":"2024-09-21T11:54:50.637650+0000","last_fullsized":"2024-09-21T11:54:50.637650+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:52:43.086376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:43.086376+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:15.310400+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":828,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.636959+0000","last_change":"2024-09-21T11:54:23.346794+0000","last_active":"2024-09-21T11:54:50.636959+0000","last_peered":"2024-09-21T11:54:50.636959+0000","last_clean":"2024-09-21T11:54:50.636959+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:54:50.636959+0000","last_undegraded":"2024-09-21T11:54:50.636959+0000","last_fullsized":"2024-09-21T11:54:50.636959+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:54:23.346727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.346727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:23.671194+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":629,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650059+0000","last_change":"2024-09-21T11:53:27.992553+0000","last_active":"2024-09-21T11:54:50.650059+0000","last_peered":"2024-09-21T11:54:50.650059+0000","last_clean":"2024-09-21T11:54:50.650059+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:54:50.650059+0000","last_undegraded":"2024-09-21T11:54:50.650059+0000","last_fullsized":"2024-09-21T11:54:50.650059+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.992489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.992489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:39.963992+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":665,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.653603+0000","last_change":"2024-09-21T11:54:38.208884+0000","last_active":"2024-09-21T11:54:50.653603+0000","last_peered":"2024-09-21T11:54:50.653603+0000","last_clean":"2024-09-21T11:54:50.653603+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:54:50.653603+0000","last_undegraded":"2024-09-21T11:54:50.653603+0000","last_fullsized":"2024-09-21T11:54:50.653603+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:38.208809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.208809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:03.474977+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":678,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.670551+0000","last_change":"2024-09-21T11:54:02.489875+0000","last_active":"2024-09-21T11:54:50.670551+0000","last_peered":"2024-09-21T11:54:50.670551+0000","last_clean":"2024-09-21T11:54:50.670551+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:54:50.670551+0000","last_undegraded":"2024-09-21T11:54:50.670551+0000","last_fullsized":"2024-09-21T11:54:50.670551+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:54:02.489798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:02.489798+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.297459+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":671,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.652354+0000","last_change":"2024-09-21T11:54:32.067297+0000","last_active":"2024-09-21T11:54:50.652354+0000","last_peered":"2024-09-21T11:54:50.652354+0000","last_clean":"2024-09-21T11:54:50.652354+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:54:50.652354+0000","last_undegraded":"2024-09-21T11:54:50.652354+0000","last_fullsized":"2024-09-21T11:54:50.652354+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:32.067168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:32.067168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:41.374600+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":645,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637296+0000","last_change":"2024-09-21T11:53:26.139062+0000","last_active":"2024-09-21T11:54:50.637296+0000","last_peered":"2024-09-21T11:54:50.637296+0000","last_clean":"2024-09-21T11:54:50.637296+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:54:50.637296+0000","last_undegraded":"2024-09-21T11:54:50.637296+0000","last_fullsized":"2024-09-21T11:54:50.637296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.138988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.138988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:46.217683+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":679,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.666914+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:54:50.666914+0000","last_peered":"2024-09-21T11:54:50.666914+0000","last_clean":"2024-09-21T11:54:50.666914+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:54:50.666914+0000","last_undegraded":"2024-09-21T11:54:50.666914+0000","last_fullsized":"2024-09-21T11:54:50.666914+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":641,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637215+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:54:50.637215+0000","last_peered":"2024-09-21T11:54:50.637215+0000","last_clean":"2024-09-21T11:54:50.637215+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:54:50.637215+0000","last_undegraded":"2024-09-21T11:54:50.637215+0000","last_fullsized":"2024-09-21T11:54:50.637215+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.263136+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":656,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.669828+0000","last_change":"2024-09-21T11:52:41.245695+0000","last_active":"2024-09-21T11:54:50.669828+0000","last_peered":"2024-09-21T11:54:50.669828+0000","last_clean":"2024-09-21T11:54:50.669828+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:54:50.669828+0000","last_undegraded":"2024-09-21T11:54:50.669828+0000","last_fullsized":"2024-09-21T11:54:50.669828+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:41.245632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:41.245632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":638,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.647375+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:54:50.647375+0000","last_peered":"2024-09-21T11:54:50.647375+0000","last_clean":"2024-09-21T11:54:50.647375+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:54:50.647375+0000","last_undegraded":"2024-09-21T11:54:50.647375+0000","last_fullsized":"2024-09-21T11:54:50.647375+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.101504+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":662,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650562+0000","last_change":"2024-09-21T11:53:30.947056+0000","last_active":"2024-09-21T11:54:50.650562+0000","last_peered":"2024-09-21T11:54:50.650562+0000","last_clean":"2024-09-21T11:54:50.650562+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:54:50.650562+0000","last_undegraded":"2024-09-21T11:54:50.650562+0000","last_fullsized":"2024-09-21T11:54:50.650562+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:30.946969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:30.946969+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":657,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.672054+0000","last_change":"2024-09-21T11:52:45.688676+0000","last_active":"2024-09-21T11:54:50.672054+0000","last_peered":"2024-09-21T11:54:50.672054+0000","last_clean":"2024-09-21T11:54:50.672054+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:54:50.672054+0000","last_undegraded":"2024-09-21T11:54:50.672054+0000","last_fullsized":"2024-09-21T11:54:50.672054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:45.688617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:45.688617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":805,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.656178+0000","last_change":"2024-09-21T11:51:08.629172+0000","last_active":"2024-09-21T11:54:50.656178+0000","last_peered":"2024-09-21T11:54:50.656178+0000","last_clean":"2024-09-21T11:54:50.656178+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:54:50.656178+0000","last_undegraded":"2024-09-21T11:54:50.656178+0000","last_fullsized":"2024-09-21T11:54:50.656178+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:08.628986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.628986+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:40.424036+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":136,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.660154+0000","last_change":"2024-09-21T11:51:43.531318+0000","last_active":"2024-09-21T11:54:50.660154+0000","last_peered":"2024-09-21T11:54:50.660154+0000","last_clean":"2024-09-21T11:54:50.660154+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:54:50.660154+0000","last_undegraded":"2024-09-21T11:54:50.660154+0000","last_fullsized":"2024-09-21T11:54:50.660154+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.826834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.826834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:11.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":698,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.656409+0000","last_change":"2024-09-21T11:53:25.264037+0000","last_active":"2024-09-21T11:54:50.656409+0000","last_peered":"2024-09-21T11:54:50.656409+0000","last_clean":"2024-09-21T11:54:50.656409+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:54:50.656409+0000","last_undegraded":"2024-09-21T11:54:50.656409+0000","last_fullsized":"2024-09-21T11:54:50.656409+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:25.263955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:25.263955+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":641,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637295+0000","last_change":"2024-09-21T11:54:35.264828+0000","last_active":"2024-09-21T11:54:50.637295+0000","last_peered":"2024-09-21T11:54:50.637295+0000","last_clean":"2024-09-21T11:54:50.637295+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:54:50.637295+0000","last_undegraded":"2024-09-21T11:54:50.637295+0000","last_fullsized":"2024-09-21T11:54:50.637295+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:35.264755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:35.264755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:02.953376+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":851,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.650393+0000","last_change":"2024-09-21T11:53:24.947837+0000","last_active":"2024-09-21T11:54:50.650393+0000","last_peered":"2024-09-21T11:54:50.650393+0000","last_clean":"2024-09-21T11:54:50.650393+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:54:50.650393+0000","last_undegraded":"2024-09-21T11:54:50.650393+0000","last_fullsized":"2024-09-21T11:54:50.650393+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:53:24.947744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:24.947744+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:54.797043+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":637,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.666494+0000","last_change":"2024-09-21T11:50:02.782565+0000","last_active":"2024-09-21T11:54:50.666494+0000","last_peered":"2024-09-21T11:54:50.666494+0000","last_clean":"2024-09-21T11:54:50.666494+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:54:50.666494+0000","last_undegraded":"2024-09-21T11:54:50.666494+0000","last_fullsized":"2024-09-21T11:54:50.666494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.782475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.782475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:46.128440+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":686,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.653927+0000","last_change":"2024-09-21T11:51:43.533694+0000","last_active":"2024-09-21T11:54:50.653927+0000","last_peered":"2024-09-21T11:54:50.653927+0000","last_clean":"2024-09-21T11:54:50.653927+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:54:50.653927+0000","last_undegraded":"2024-09-21T11:54:50.653927+0000","last_fullsized":"2024-09-21T11:54:50.653927+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:08.366846+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":144,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637373+0000","last_change":"2024-09-21T11:51:52.159555+0000","last_active":"2024-09-21T11:54:50.637373+0000","last_peered":"2024-09-21T11:54:50.637373+0000","last_clean":"2024-09-21T11:54:50.637373+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:54:50.637373+0000","last_undegraded":"2024-09-21T11:54:50.637373+0000","last_fullsized":"2024-09-21T11:54:50.637373+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:52.159433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:52.159433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:16.279585+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":343,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.663243+0000","last_change":"2024-09-21T11:54:35.426101+0000","last_active":"2024-09-21T11:54:50.663243+0000","last_peered":"2024-09-21T11:54:50.663243+0000","last_clean":"2024-09-21T11:54:50.663243+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:54:50.663243+0000","last_undegraded":"2024-09-21T11:54:50.663243+0000","last_fullsized":"2024-09-21T11:54:50.663243+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:35.426030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:35.426030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:52.101778+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":622,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.655548+0000","last_change":"2024-09-21T11:52:48.256878+0000","last_active":"2024-09-21T11:54:50.655548+0000","last_peered":"2024-09-21T11:54:50.655548+0000","last_clean":"2024-09-21T11:54:50.655548+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:54:50.655548+0000","last_undegraded":"2024-09-21T11:54:50.655548+0000","last_fullsized":"2024-09-21T11:54:50.655548+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:48.256830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:48.256830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":354,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.663470+0000","last_change":"2024-09-21T11:54:38.447060+0000","last_active":"2024-09-21T11:54:50.663470+0000","last_peered":"2024-09-21T11:54:50.663470+0000","last_clean":"2024-09-21T11:54:50.663470+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:54:50.663470+0000","last_undegraded":"2024-09-21T11:54:50.663470+0000","last_fullsized":"2024-09-21T11:54:50.663470+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:38.446684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.446684+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:45.619387+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":641,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.637458+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:54:50.637458+0000","last_peered":"2024-09-21T11:54:50.637458+0000","last_clean":"2024-09-21T11:54:50.637458+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:54:50.637458+0000","last_undegraded":"2024-09-21T11:54:50.637458+0000","last_fullsized":"2024-09-21T11:54:50.637458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:13.281117+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":648,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.673775+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:54:50.673775+0000","last_peered":"2024-09-21T11:54:50.673775+0000","last_clean":"2024-09-21T11:54:50.673775+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:54:50.673775+0000","last_undegraded":"2024-09-21T11:54:50.673775+0000","last_fullsized":"2024-09-21T11:54:50.673775+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:15.498922+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":668,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.667570+0000","last_change":"2024-09-21T11:54:31.029959+0000","last_active":"2024-09-21T11:54:50.667570+0000","last_peered":"2024-09-21T11:54:50.667570+0000","last_clean":"2024-09-21T11:54:50.667570+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:54:50.667570+0000","last_undegraded":"2024-09-21T11:54:50.667570+0000","last_fullsized":"2024-09-21T11:54:50.667570+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:54:31.029780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:31.029780+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:49.683352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":677,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.655382+0000","last_change":"2024-09-21T11:53:32.308905+0000","last_active":"2024-09-21T11:54:50.655382+0000","last_peered":"2024-09-21T11:54:50.655382+0000","last_clean":"2024-09-21T11:54:50.655382+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:54:50.655382+0000","last_undegraded":"2024-09-21T11:54:50.655382+0000","last_fullsized":"2024-09-21T11:54:50.655382+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:32.308827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:32.308827+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:50.813704+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":663,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.652377+0000","last_change":"2024-09-21T11:53:28.945615+0000","last_active":"2024-09-21T11:54:50.652377+0000","last_peered":"2024-09-21T11:54:50.652377+0000","last_clean":"2024-09-21T11:54:50.652377+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:54:50.652377+0000","last_undegraded":"2024-09-21T11:54:50.652377+0000","last_fullsized":"2024-09-21T11:54:50.652377+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:28.945564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:28.945564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:46.973609+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":625,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.636550+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:54:50.636550+0000","last_peered":"2024-09-21T11:54:50.636550+0000","last_clean":"2024-09-21T11:54:50.636550+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:54:50.636550+0000","last_undegraded":"2024-09-21T11:54:50.636550+0000","last_fullsized":"2024-09-21T11:54:50.636550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:10.309619+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":638,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.673960+0000","last_change":"2024-09-21T11:54:25.551920+0000","last_active":"2024-09-21T11:54:50.673960+0000","last_peered":"2024-09-21T11:54:50.673960+0000","last_clean":"2024-09-21T11:54:50.673960+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:54:50.673960+0000","last_undegraded":"2024-09-21T11:54:50.673960+0000","last_fullsized":"2024-09-21T11:54:50.673960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.551852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.551852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:43.602512+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":689,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.655268+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:54:50.655268+0000","last_peered":"2024-09-21T11:54:50.655268+0000","last_clean":"2024-09-21T11:54:50.655268+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:54:50.655268+0000","last_undegraded":"2024-09-21T11:54:50.655268+0000","last_fullsized":"2024-09-21T11:54:50.655268+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:00.664484+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":633,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.667364+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:54:50.667364+0000","last_peered":"2024-09-21T11:54:50.667364+0000","last_clean":"2024-09-21T11:54:50.667364+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:54:50.667364+0000","last_undegraded":"2024-09-21T11:54:50.667364+0000","last_fullsized":"2024-09-21T11:54:50.667364+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:49.212310+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":633,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.653544+0000","last_change":"2024-09-21T11:50:38.405183+0000","last_active":"2024-09-21T11:54:50.653544+0000","last_peered":"2024-09-21T11:54:50.653544+0000","last_clean":"2024-09-21T11:54:50.653544+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:54:50.653544+0000","last_undegraded":"2024-09-21T11:54:50.653544+0000","last_fullsized":"2024-09-21T11:54:50.653544+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:38.405118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.405118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:17.298725+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":637,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.636781+0000","last_change":"2024-09-21T11:53:53.105703+0000","last_active":"2024-09-21T11:54:50.636781+0000","last_peered":"2024-09-21T11:54:50.636781+0000","last_clean":"2024-09-21T11:54:50.636781+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:54:50.636781+0000","last_undegraded":"2024-09-21T11:54:50.636781+0000","last_fullsized":"2024-09-21T11:54:50.636781+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:53.105624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:53.105624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:57.009776+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":343,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.661505+0000","last_change":"2024-09-21T11:52:12.348687+0000","last_active":"2024-09-21T11:54:50.661505+0000","last_peered":"2024-09-21T11:54:50.661505+0000","last_clean":"2024-09-21T11:54:50.661505+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:54:50.661505+0000","last_undegraded":"2024-09-21T11:54:50.661505+0000","last_fullsized":"2024-09-21T11:54:50.661505+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.324575+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":664,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.655441+0000","last_change":"2024-09-21T11:54:23.601389+0000","last_active":"2024-09-21T11:54:50.655441+0000","last_peered":"2024-09-21T11:54:50.655441+0000","last_clean":"2024-09-21T11:54:50.655441+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:54:50.655441+0000","last_undegraded":"2024-09-21T11:54:50.655441+0000","last_fullsized":"2024-09-21T11:54:50.655441+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:23.601318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.601318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:26.191120+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":731,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.654239+0000","last_change":"2024-09-21T11:53:20.555366+0000","last_active":"2024-09-21T11:54:50.654239+0000","last_peered":"2024-09-21T11:54:50.654239+0000","last_clean":"2024-09-21T11:54:50.654239+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:54:50.654239+0000","last_undegraded":"2024-09-21T11:54:50.654239+0000","last_fullsized":"2024-09-21T11:54:50.654239+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:20.554779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.554779+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":642,"reported_epoch":355,"state":"active+clean","last_fresh":"2024-09-21T11:54:50.666643+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:54:50.666643+0000","last_peered":"2024-09-21T11:54:50.666643+0000","last_clean":"2024-09-21T11:54:50.666643+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:54:50.666643+0000","last_undegraded":"2024-09-21T11:54:50.666643+0000","last_fullsized":"2024-09-21T11:54:50.666643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:43.127854+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":5926092824,"num_objects":90426,"num_object_clones":0,"num_object_copies":271278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90426,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":155072,"num_write_kb":7855873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17732481024,"data_stored":17732468808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93272,"ondisk_log_size":93272,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1113,"num_read_kb":870,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":881,"num_read_kb":876,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234167,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2071400,"kb_used_data":1476568,"kb_used_omap":1,"kb_used_meta":594814,"kb_avail":91673752,"statfs":{"total":95995035648,"available":93873922048,"internally_reserved":0,"allocated":1512005632,"data_stored":1511613938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":609089994},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463100,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3053216,"kb_used_data":2566608,"kb_used_omap":1,"kb_used_meta":486526,"kb_avail":90691936,"statfs":{"total":95995035648,"available":92868542464,"internally_reserved":0,"allocated":2628206592,"data_stored":2627823458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":498203078},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724737,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2374264,"kb_used_data":1819684,"kb_used_omap":1,"kb_used_meta":554558,"kb_avail":91370888,"statfs":{"total":95995035648,"available":93563789312,"internally_reserved":0,"allocated":1863356416,"data_stored":1862958230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":567867846},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920967,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3369796,"kb_used_data":2911024,"kb_used_omap":1,"kb_used_meta":458750,"kb_avail":90375356,"statfs":{"total":95995035648,"available":92544364544,"internally_reserved":0,"allocated":2980888576,"data_stored":2980479287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":469760453},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117197,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2404848,"kb_used_data":1791964,"kb_used_omap":1,"kb_used_meta":612862,"kb_avail":91340304,"statfs":{"total":95995035648,"available":93532471296,"internally_reserved":0,"allocated":1834971136,"data_stored":1834581128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":627571143},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411539,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2792756,"kb_used_data":2197920,"kb_used_omap":1,"kb_used_meta":594814,"kb_avail":90952396,"statfs":{"total":95995035648,"available":93135253504,"internally_reserved":0,"allocated":2250670080,"data_stored":2250276687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":609089989},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575066,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1687044,"kb_used_data":1092212,"kb_used_omap":1,"kb_used_meta":594814,"kb_avail":92058108,"statfs":{"total":95995035648,"available":94267502592,"internally_reserved":0,"allocated":1118425088,"data_stored":1118004777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":609089990},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738592,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4121612,"kb_used_data":3562744,"kb_used_omap":1,"kb_used_meta":558718,"kb_avail":89623540,"statfs":{"total":95995035648,"available":91774504960,"internally_reserved":0,"allocated":3648249856,"data_stored":3647833524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":572127685},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3645575168,"data_stored":3645571096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1105526784,"data_stored":1105526784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2225864704,"data_stored":2225864704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1834090496,"data_stored":1834090496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2976387072,"data_stored":2976383000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1853292544,"data_stored":1853292544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2599686144,"data_stored":2599682072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1492058112,"data_stored":1492058112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:55:01.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:55:01.519 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:01.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:01.525 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:01.554 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:01.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:01.675 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:01.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:01.676 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:01.777 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:01.806 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:01.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:01.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:01.907 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:01.933 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:01.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:01.934 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:01.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[36454]: pgmap v722: 148 pgs: 148 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 750 op/s 2024-09-21T11:55:01.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[36454]: 6.18 scrub starts 2024-09-21T11:55:01.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[36454]: 6.18 scrub ok 2024-09-21T11:55:01.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[36454]: 4.8 scrub starts 2024-09-21T11:55:01.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[36454]: 4.8 scrub ok 2024-09-21T11:55:01.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:01.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[29710]: pgmap v722: 148 pgs: 148 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 750 op/s 2024-09-21T11:55:01.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[29710]: 6.18 scrub starts 2024-09-21T11:55:01.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[29710]: 6.18 scrub ok 2024-09-21T11:55:01.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[29710]: 4.8 scrub starts 2024-09-21T11:55:01.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[29710]: 4.8 scrub ok 2024-09-21T11:55:01.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:01 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:02.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:55:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:55:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:01 smithi175 ceph-mon[32613]: pgmap v722: 148 pgs: 148 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 750 op/s 2024-09-21T11:55:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:01 smithi175 ceph-mon[32613]: 6.18 scrub starts 2024-09-21T11:55:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:01 smithi175 ceph-mon[32613]: 6.18 scrub ok 2024-09-21T11:55:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:01 smithi175 ceph-mon[32613]: 4.8 scrub starts 2024-09-21T11:55:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:01 smithi175 ceph-mon[32613]: 4.8 scrub ok 2024-09-21T11:55:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:01 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:02.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:02.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:02.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:02.176 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:02.209 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:02.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:02.210 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:02.311 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:02.338 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:02.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:02.339 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:02.440 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:02.458 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:55:02.458 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:55:02.458 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:55:02.458 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:55:02.458 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:55:02.470 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:02.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:02.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:02.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:02.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:02.611 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:02.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:02.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:02.738 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:02.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:02.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:02.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:02.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:02.879 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:02.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:02.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:02.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[36454]: 6.11 scrub starts 2024-09-21T11:55:02.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[36454]: 6.11 scrub ok 2024-09-21T11:55:02.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[36454]: 4.2 scrub starts 2024-09-21T11:55:02.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[36454]: 4.2 scrub ok 2024-09-21T11:55:02.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:02.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[36454]: pgmap v723: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:55:02.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[29710]: 6.11 scrub starts 2024-09-21T11:55:02.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[29710]: 6.11 scrub ok 2024-09-21T11:55:02.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[29710]: 4.2 scrub starts 2024-09-21T11:55:02.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[29710]: 4.2 scrub ok 2024-09-21T11:55:02.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:02.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:02 smithi043 ceph-mon[29710]: pgmap v723: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:55:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:02 smithi175 ceph-mon[32613]: 6.11 scrub starts 2024-09-21T11:55:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:02 smithi175 ceph-mon[32613]: 6.11 scrub ok 2024-09-21T11:55:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:02 smithi175 ceph-mon[32613]: 4.2 scrub starts 2024-09-21T11:55:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:02 smithi175 ceph-mon[32613]: 4.2 scrub ok 2024-09-21T11:55:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:02 smithi175 ceph-mon[32613]: pgmap v723: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-21T11:55:03.012 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:03.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:03.013 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:03.114 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:03.159 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:03.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:03.160 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:03.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:03.296 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:03.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:03.297 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:03.398 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:03.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:03.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:03.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:03.528 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:03.560 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:03.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:03.562 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:03.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:55:03.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:03.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:03.708 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:03.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:03.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:03.852 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:03.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:03.853 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:03.953 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:03.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:03 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:55:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:03 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:55:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:03 smithi043 ceph-mon[29710]: osdmap e358: 8 total, 8 up, 8 in 2024-09-21T11:55:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:03 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:55:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:03 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:55:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:03 smithi043 ceph-mon[36454]: osdmap e358: 8 total, 8 up, 8 in 2024-09-21T11:55:03.992 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:03.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:03.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:04.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:03 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:55:04.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:03 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:55:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:03 smithi175 ceph-mon[32613]: osdmap e358: 8 total, 8 up, 8 in 2024-09-21T11:55:04.093 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:04.114 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:04.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:04.115 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:04.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:04.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:04.244 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:04.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:04.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:04.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:04.380 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:04.380 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:04.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:04.513 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:04.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:04.514 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:04.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:04.651 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:04.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:04.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:04.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:04.783 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:04.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:04.784 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:04.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:04.917 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:04.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:04.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:04.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:04 smithi043 ceph-mon[36454]: 5.17 scrub starts 2024-09-21T11:55:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:04 smithi043 ceph-mon[36454]: 5.17 scrub ok 2024-09-21T11:55:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:04 smithi043 ceph-mon[36454]: pgmap v725: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-21T11:55:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:04 smithi043 ceph-mon[29710]: 5.17 scrub starts 2024-09-21T11:55:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:04 smithi043 ceph-mon[29710]: 5.17 scrub ok 2024-09-21T11:55:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:04 smithi043 ceph-mon[29710]: pgmap v725: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-21T11:55:05.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:04 smithi175 ceph-mon[32613]: 5.17 scrub starts 2024-09-21T11:55:05.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:04 smithi175 ceph-mon[32613]: 5.17 scrub ok 2024-09-21T11:55:05.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:04 smithi175 ceph-mon[32613]: pgmap v725: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-21T11:55:05.019 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:05.045 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:05.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:05.046 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:05.055 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:55:05.058 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:55:05.058 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:55:05.058 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:55:05.058 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:55:05.058 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:55:05.058 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:55:05.059 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:55:05.075 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:55:05.075 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:55:05.091 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:05.128 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:55:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:55:05.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:05.169 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:05.177 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:55:05.178 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:55:05.218 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:55:05.218 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:55:05.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:05.228 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:05.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:05.241 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:55:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:05.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:05.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:05.372 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:05.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:05.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:05.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:05.522 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:05.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:05.623 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:05.655 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:05.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:05.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:05.749 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:55:05.757 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:05.783 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:05.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:05.784 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:05.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:05.928 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:05.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:05.929 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[36454]: 3.12 scrub starts 2024-09-21T11:55:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[36454]: 3.12 scrub ok 2024-09-21T11:55:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[36454]: 4.1b scrub starts 2024-09-21T11:55:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[36454]: 4.1b scrub ok 2024-09-21T11:55:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[36454]: 6.1e scrub starts 2024-09-21T11:55:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[36454]: 6.1e scrub ok 2024-09-21T11:55:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2074875707' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3735746586' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[29710]: 3.12 scrub starts 2024-09-21T11:55:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[29710]: 3.12 scrub ok 2024-09-21T11:55:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[29710]: 4.1b scrub starts 2024-09-21T11:55:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[29710]: 4.1b scrub ok 2024-09-21T11:55:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[29710]: 6.1e scrub starts 2024-09-21T11:55:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[29710]: 6.1e scrub ok 2024-09-21T11:55:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2074875707' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:05 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3735746586' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:06.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:05 smithi175 ceph-mon[32613]: 3.12 scrub starts 2024-09-21T11:55:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:05 smithi175 ceph-mon[32613]: 3.12 scrub ok 2024-09-21T11:55:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:05 smithi175 ceph-mon[32613]: 4.1b scrub starts 2024-09-21T11:55:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:05 smithi175 ceph-mon[32613]: 4.1b scrub ok 2024-09-21T11:55:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:05 smithi175 ceph-mon[32613]: 6.1e scrub starts 2024-09-21T11:55:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:05 smithi175 ceph-mon[32613]: 6.1e scrub ok 2024-09-21T11:55:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:05 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2074875707' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:05 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3735746586' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:06.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:06.057 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:06.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:06.058 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:06.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:06.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:06.198 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:06.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:06.337 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:06.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:06.338 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:06.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:06.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:06.479 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:06.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:06.624 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:06.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:06.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:06.725 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:06.760 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:06.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:06.761 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[36454]: 5.1b scrub starts 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[36454]: 5.1b scrub ok 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[36454]: 4.0 scrub starts 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[36454]: 4.0 scrub ok 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3735746586' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[36454]: osdmap e359: 8 total, 8 up, 8 in 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[36454]: pgmap v727: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 692 op/s 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[29710]: 5.1b scrub starts 2024-09-21T11:55:06.829 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[29710]: 5.1b scrub ok 2024-09-21T11:55:06.830 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[29710]: 4.0 scrub starts 2024-09-21T11:55:06.830 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[29710]: 4.0 scrub ok 2024-09-21T11:55:06.830 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:55:06.830 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:55:06.830 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3735746586' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:06.830 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[29710]: osdmap e359: 8 total, 8 up, 8 in 2024-09-21T11:55:06.830 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:06 smithi043 ceph-mon[29710]: pgmap v727: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 692 op/s 2024-09-21T11:55:06.861 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:06.890 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:06.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:06.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:06.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:06 smithi175 ceph-mon[32613]: 5.1b scrub starts 2024-09-21T11:55:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:06 smithi175 ceph-mon[32613]: 5.1b scrub ok 2024-09-21T11:55:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:06 smithi175 ceph-mon[32613]: 4.0 scrub starts 2024-09-21T11:55:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:06 smithi175 ceph-mon[32613]: 4.0 scrub ok 2024-09-21T11:55:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:06 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:55:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:06 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:55:07.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:06 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3735746586' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:07.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:06 smithi175 ceph-mon[32613]: osdmap e359: 8 total, 8 up, 8 in 2024-09-21T11:55:07.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:06 smithi175 ceph-mon[32613]: pgmap v727: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 692 op/s 2024-09-21T11:55:07.050 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:07.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:07.051 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:07.151 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:07.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:07.192 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:07.192 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:07.293 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:07.326 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:07.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:07.327 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:07.427 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:07.454 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:55:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:55:07.461 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:07.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:07.462 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:07.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:07.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:07.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:07.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:07.739 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:55:07.739 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:55:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:07.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:07.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:07.847 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:07.879 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:07.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:07.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:07.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:07 smithi043 ceph-mon[29710]: 3.0 scrub starts 2024-09-21T11:55:07.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:07 smithi043 ceph-mon[29710]: 3.0 scrub ok 2024-09-21T11:55:07.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:07 smithi043 ceph-mon[29710]: 5.6 scrub starts 2024-09-21T11:55:07.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:07 smithi043 ceph-mon[29710]: 5.6 scrub ok 2024-09-21T11:55:07.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:07 smithi043 ceph-mon[29710]: from='client.16329 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:55:07.937 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:07 smithi043 ceph-mon[36454]: 3.0 scrub starts 2024-09-21T11:55:07.937 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:07 smithi043 ceph-mon[36454]: 3.0 scrub ok 2024-09-21T11:55:07.937 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:07 smithi043 ceph-mon[36454]: 5.6 scrub starts 2024-09-21T11:55:07.937 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:07 smithi043 ceph-mon[36454]: 5.6 scrub ok 2024-09-21T11:55:07.937 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:07 smithi043 ceph-mon[36454]: from='client.16329 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:55:07.981 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:07 smithi175 ceph-mon[32613]: 3.0 scrub starts 2024-09-21T11:55:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:07 smithi175 ceph-mon[32613]: 3.0 scrub ok 2024-09-21T11:55:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:07 smithi175 ceph-mon[32613]: 5.6 scrub starts 2024-09-21T11:55:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:07 smithi175 ceph-mon[32613]: 5.6 scrub ok 2024-09-21T11:55:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:07 smithi175 ceph-mon[32613]: from='client.16329 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:55:08.016 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:08.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:08.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:08.117 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:08.151 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:08.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:08.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:08.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:08.296 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:08.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:08.298 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:08.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:08.434 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:08.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:08.435 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:08.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:08.574 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:08.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:08.575 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:08.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:55:08.676 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:08.704 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:08.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:08.705 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:08.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:55:08.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:08.844 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:08.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:08.845 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:08.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:08.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:08.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[29710]: 4.b scrub starts 2024-09-21T11:55:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[29710]: 4.b scrub ok 2024-09-21T11:55:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[29710]: 6.1f scrub starts 2024-09-21T11:55:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[29710]: 6.1f scrub ok 2024-09-21T11:55:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[29710]: 6.e scrub starts 2024-09-21T11:55:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[29710]: 6.e scrub ok 2024-09-21T11:55:08.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[29710]: pgmap v728: 148 pgs: 148 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-21T11:55:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[36454]: 4.b scrub starts 2024-09-21T11:55:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[36454]: 4.b scrub ok 2024-09-21T11:55:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[36454]: 6.1f scrub starts 2024-09-21T11:55:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[36454]: 6.1f scrub ok 2024-09-21T11:55:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[36454]: 6.e scrub starts 2024-09-21T11:55:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[36454]: 6.e scrub ok 2024-09-21T11:55:08.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:08 smithi043 ceph-mon[36454]: pgmap v728: 148 pgs: 148 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-21T11:55:09.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:09.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:09.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:09.196 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:09.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:09.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:08 smithi175 ceph-mon[32613]: 4.b scrub starts 2024-09-21T11:55:09.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:08 smithi175 ceph-mon[32613]: 4.b scrub ok 2024-09-21T11:55:09.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:08 smithi175 ceph-mon[32613]: 6.1f scrub starts 2024-09-21T11:55:09.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:08 smithi175 ceph-mon[32613]: 6.1f scrub ok 2024-09-21T11:55:09.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:08 smithi175 ceph-mon[32613]: 6.e scrub starts 2024-09-21T11:55:09.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:08 smithi175 ceph-mon[32613]: 6.e scrub ok 2024-09-21T11:55:09.263 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:08 smithi175 ceph-mon[32613]: pgmap v728: 148 pgs: 148 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-21T11:55:09.292 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:09.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:09.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:09.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:55:09.393 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:09.428 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:09.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:09.429 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:09.431 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":727,"stamp":"2024-09-21T11:55:06.082271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5217393203,"num_objects":79802,"num_object_clones":0,"num_object_copies":239406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79802,"num_whiteouts":0,"num_read":2141,"num_read_kb":1998,"num_write":166730,"num_write_kb":7857546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":597267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94441,"ondisk_log_size":94441,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":19661024,"kb_used_data":15176988,"kb_used_omap":12,"kb_used_meta":4483699,"kb_avail":730300192,"statfs":{"total":767960285184,"available":747827396608,"internally_reserved":0,"allocated":15541235712,"data_stored":15537980580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4591308338},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,10,11,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-362807296,"num_objects":-5536,"num_object_clones":0,"num_object_copies":-16608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5536,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5536,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002101"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":646,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.679037+0000","last_change":"2024-09-21T11:54:53.303631+0000","last_active":"2024-09-21T11:54:54.679037+0000","last_peered":"2024-09-21T11:54:54.679037+0000","last_clean":"2024-09-21T11:54:54.679037+0000","last_became_active":"2024-09-21T11:40:00.312822+0000","last_became_peered":"2024-09-21T11:40:00.312822+0000","last_unstale":"2024-09-21T11:54:54.679037+0000","last_undegraded":"2024-09-21T11:54:54.679037+0000","last_fullsized":"2024-09-21T11:54:54.679037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.303555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.303555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:15.140395+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":677,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.680210+0000","last_change":"2024-09-21T11:54:27.283019+0000","last_active":"2024-09-21T11:54:54.680210+0000","last_peered":"2024-09-21T11:54:54.680210+0000","last_clean":"2024-09-21T11:54:54.680210+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:54:54.680210+0000","last_undegraded":"2024-09-21T11:54:54.680210+0000","last_fullsized":"2024-09-21T11:54:54.680210+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:27.282826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:27.282826+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:43.536640+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":355,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.671000+0000","last_change":"2024-09-21T11:54:52.816248+0000","last_active":"2024-09-21T11:54:54.671000+0000","last_peered":"2024-09-21T11:54:54.671000+0000","last_clean":"2024-09-21T11:54:54.671000+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:54:54.671000+0000","last_undegraded":"2024-09-21T11:54:54.671000+0000","last_fullsized":"2024-09-21T11:54:54.671000+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:52.816156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:52.816156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:01.547204+0000","scrub_duration":226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":450,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.689146+0000","last_change":"2024-09-21T11:51:43.528594+0000","last_active":"2024-09-21T11:54:54.689146+0000","last_peered":"2024-09-21T11:54:54.689146+0000","last_clean":"2024-09-21T11:54:54.689146+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:54:54.689146+0000","last_undegraded":"2024-09-21T11:54:54.689146+0000","last_fullsized":"2024-09-21T11:54:54.689146+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:06.268427+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":649,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687942+0000","last_change":"2024-09-21T11:54:53.491138+0000","last_active":"2024-09-21T11:54:54.687942+0000","last_peered":"2024-09-21T11:54:54.687942+0000","last_clean":"2024-09-21T11:54:54.687942+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:54:54.687942+0000","last_undegraded":"2024-09-21T11:54:54.687942+0000","last_fullsized":"2024-09-21T11:54:54.687942+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.490941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.490941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:05.702659+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":631,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687801+0000","last_change":"2024-09-21T11:51:01.667426+0000","last_active":"2024-09-21T11:54:54.687801+0000","last_peered":"2024-09-21T11:54:54.687801+0000","last_clean":"2024-09-21T11:54:54.687801+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:54:54.687801+0000","last_undegraded":"2024-09-21T11:54:54.687801+0000","last_fullsized":"2024-09-21T11:54:54.687801+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:01.667366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:01.667366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:20.606273+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":685,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688524+0000","last_change":"2024-09-21T11:53:54.438887+0000","last_active":"2024-09-21T11:54:54.688524+0000","last_peered":"2024-09-21T11:54:54.688524+0000","last_clean":"2024-09-21T11:54:54.688524+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:54:54.688524+0000","last_undegraded":"2024-09-21T11:54:54.688524+0000","last_fullsized":"2024-09-21T11:54:54.688524+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:53:54.438798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:54.438798+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:20.197167+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":478,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.680296+0000","last_change":"2024-09-21T11:54:28.252705+0000","last_active":"2024-09-21T11:54:54.680296+0000","last_peered":"2024-09-21T11:54:54.680296+0000","last_clean":"2024-09-21T11:54:54.680296+0000","last_became_active":"2024-09-21T11:51:43.526945+0000","last_became_peered":"2024-09-21T11:51:43.526945+0000","last_unstale":"2024-09-21T11:54:54.680296+0000","last_undegraded":"2024-09-21T11:54:54.680296+0000","last_fullsized":"2024-09-21T11:54:54.680296+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:28.252642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:28.252642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:41.593068+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":637,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.689108+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:54:54.689108+0000","last_peered":"2024-09-21T11:54:54.689108+0000","last_clean":"2024-09-21T11:54:54.689108+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:54:54.689108+0000","last_undegraded":"2024-09-21T11:54:54.689108+0000","last_fullsized":"2024-09-21T11:54:54.689108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:15.115091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":635,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687456+0000","last_change":"2024-09-21T11:52:55.214031+0000","last_active":"2024-09-21T11:54:54.687456+0000","last_peered":"2024-09-21T11:54:54.687456+0000","last_clean":"2024-09-21T11:54:54.687456+0000","last_became_active":"2024-09-21T11:51:43.545021+0000","last_became_peered":"2024-09-21T11:51:43.545021+0000","last_unstale":"2024-09-21T11:54:54.687456+0000","last_undegraded":"2024-09-21T11:54:54.687456+0000","last_fullsized":"2024-09-21T11:54:54.687456+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:52:55.213938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:55.213938+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":628,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.690302+0000","last_change":"2024-09-21T11:46:19.236380+0000","last_active":"2024-09-21T11:54:54.690302+0000","last_peered":"2024-09-21T11:54:54.690302+0000","last_clean":"2024-09-21T11:54:54.690302+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:54:54.690302+0000","last_undegraded":"2024-09-21T11:54:54.690302+0000","last_fullsized":"2024-09-21T11:54:54.690302+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:46:19.236309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:46:19.236309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.252990+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":663,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:00.276976+0000","last_change":"2024-09-21T11:55:00.276976+0000","last_active":"2024-09-21T11:55:00.276976+0000","last_peered":"2024-09-21T11:55:00.276976+0000","last_clean":"2024-09-21T11:55:00.276976+0000","last_became_active":"2024-09-21T11:47:03.384811+0000","last_became_peered":"2024-09-21T11:47:03.384811+0000","last_unstale":"2024-09-21T11:55:00.276976+0000","last_undegraded":"2024-09-21T11:55:00.276976+0000","last_fullsized":"2024-09-21T11:55:00.276976+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:00.276898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:00.276898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:30.032507+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":675,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688670+0000","last_change":"2024-09-21T11:54:33.055621+0000","last_active":"2024-09-21T11:54:54.688670+0000","last_peered":"2024-09-21T11:54:54.688670+0000","last_clean":"2024-09-21T11:54:54.688670+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:54:54.688670+0000","last_undegraded":"2024-09-21T11:54:54.688670+0000","last_fullsized":"2024-09-21T11:54:54.688670+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:33.055554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:33.055554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:02.654956+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":641,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688189+0000","last_change":"2024-09-21T11:51:43.547924+0000","last_active":"2024-09-21T11:54:54.688189+0000","last_peered":"2024-09-21T11:54:54.688189+0000","last_clean":"2024-09-21T11:54:54.688189+0000","last_became_active":"2024-09-21T11:51:43.545191+0000","last_became_peered":"2024-09-21T11:51:43.545191+0000","last_unstale":"2024-09-21T11:54:54.688189+0000","last_undegraded":"2024-09-21T11:54:54.688189+0000","last_fullsized":"2024-09-21T11:54:54.688189+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:49:14.010600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:49:14.010600+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:03.470173+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":643,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.672720+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:54:54.672720+0000","last_peered":"2024-09-21T11:54:54.672720+0000","last_clean":"2024-09-21T11:54:54.672720+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:54:54.672720+0000","last_undegraded":"2024-09-21T11:54:54.672720+0000","last_fullsized":"2024-09-21T11:54:54.672720+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.290553+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":647,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.686889+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:54:54.686889+0000","last_peered":"2024-09-21T11:54:54.686889+0000","last_clean":"2024-09-21T11:54:54.686889+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:54:54.686889+0000","last_undegraded":"2024-09-21T11:54:54.686889+0000","last_fullsized":"2024-09-21T11:54:54.686889+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.222133+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":140,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.679367+0000","last_change":"2024-09-21T11:51:43.524599+0000","last_active":"2024-09-21T11:54:54.679367+0000","last_peered":"2024-09-21T11:54:54.679367+0000","last_clean":"2024-09-21T11:54:54.679367+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:54:54.679367+0000","last_undegraded":"2024-09-21T11:54:54.679367+0000","last_fullsized":"2024-09-21T11:54:54.679367+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:32.302842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:32.302842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.350209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":637,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.679690+0000","last_change":"2024-09-21T11:54:23.070082+0000","last_active":"2024-09-21T11:54:54.679690+0000","last_peered":"2024-09-21T11:54:54.679690+0000","last_clean":"2024-09-21T11:54:54.679690+0000","last_became_active":"2024-09-21T11:40:00.311853+0000","last_became_peered":"2024-09-21T11:40:00.311853+0000","last_unstale":"2024-09-21T11:54:54.679690+0000","last_undegraded":"2024-09-21T11:54:54.679690+0000","last_fullsized":"2024-09-21T11:54:54.679690+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:23.069995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.069995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:24.259600+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":761,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.686471+0000","last_change":"2024-09-21T11:50:25.966805+0000","last_active":"2024-09-21T11:54:54.686471+0000","last_peered":"2024-09-21T11:54:54.686471+0000","last_clean":"2024-09-21T11:54:54.686471+0000","last_became_active":"2024-09-21T11:47:03.385286+0000","last_became_peered":"2024-09-21T11:47:03.385286+0000","last_unstale":"2024-09-21T11:54:54.686471+0000","last_undegraded":"2024-09-21T11:54:54.686471+0000","last_fullsized":"2024-09-21T11:54:54.686471+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:50:25.966740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.966740+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:06.137315+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":347,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.672029+0000","last_change":"2024-09-21T11:53:23.118402+0000","last_active":"2024-09-21T11:54:54.672029+0000","last_peered":"2024-09-21T11:54:54.672029+0000","last_clean":"2024-09-21T11:54:54.672029+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:54:54.672029+0000","last_undegraded":"2024-09-21T11:54:54.672029+0000","last_fullsized":"2024-09-21T11:54:54.672029+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:23.118315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:23.118315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:48.518742+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":635,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.689901+0000","last_change":"2024-09-21T11:52:44.264180+0000","last_active":"2024-09-21T11:54:54.689901+0000","last_peered":"2024-09-21T11:54:54.689901+0000","last_clean":"2024-09-21T11:54:54.689901+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:54:54.689901+0000","last_undegraded":"2024-09-21T11:54:54.689901+0000","last_fullsized":"2024-09-21T11:54:54.689901+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.264114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.264114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":462,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.671296+0000","last_change":"2024-09-21T11:54:02.517182+0000","last_active":"2024-09-21T11:54:54.671296+0000","last_peered":"2024-09-21T11:54:54.671296+0000","last_clean":"2024-09-21T11:54:54.671296+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:54:54.671296+0000","last_undegraded":"2024-09-21T11:54:54.671296+0000","last_fullsized":"2024-09-21T11:54:54.671296+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:02.517119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:02.517119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:26.878233+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":711,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687592+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:54:54.687592+0000","last_peered":"2024-09-21T11:54:54.687592+0000","last_clean":"2024-09-21T11:54:54.687592+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:54:54.687592+0000","last_undegraded":"2024-09-21T11:54:54.687592+0000","last_fullsized":"2024-09-21T11:54:54.687592+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:04.849420+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":148,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687355+0000","last_change":"2024-09-21T11:53:55.486849+0000","last_active":"2024-09-21T11:54:54.687355+0000","last_peered":"2024-09-21T11:54:54.687355+0000","last_clean":"2024-09-21T11:54:54.687355+0000","last_became_active":"2024-09-21T11:51:43.548726+0000","last_became_peered":"2024-09-21T11:51:43.548726+0000","last_unstale":"2024-09-21T11:54:54.687355+0000","last_undegraded":"2024-09-21T11:54:54.687355+0000","last_fullsized":"2024-09-21T11:54:54.687355+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:55.486676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:55.486676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:12.803070+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":151,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.681500+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:54:54.681500+0000","last_peered":"2024-09-21T11:54:54.681500+0000","last_clean":"2024-09-21T11:54:54.681500+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:54:54.681500+0000","last_undegraded":"2024-09-21T11:54:54.681500+0000","last_fullsized":"2024-09-21T11:54:54.681500+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.101192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":351,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.671190+0000","last_change":"2024-09-21T11:54:33.391503+0000","last_active":"2024-09-21T11:54:54.671190+0000","last_peered":"2024-09-21T11:54:54.671190+0000","last_clean":"2024-09-21T11:54:54.671190+0000","last_became_active":"2024-09-21T11:47:03.371208+0000","last_became_peered":"2024-09-21T11:47:03.371208+0000","last_unstale":"2024-09-21T11:54:54.671190+0000","last_undegraded":"2024-09-21T11:54:54.671190+0000","last_fullsized":"2024-09-21T11:54:54.671190+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:54:33.391425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:33.391425+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:48.197809+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":680,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.680575+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:54:54.680575+0000","last_peered":"2024-09-21T11:54:54.680575+0000","last_clean":"2024-09-21T11:54:54.680575+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:54:54.680575+0000","last_undegraded":"2024-09-21T11:54:54.680575+0000","last_fullsized":"2024-09-21T11:54:54.680575+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":846,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.550032+0000","last_change":"2024-09-21T11:51:44.478918+0000","last_active":"2024-09-21T11:54:55.550032+0000","last_peered":"2024-09-21T11:54:55.550032+0000","last_clean":"2024-09-21T11:54:55.550032+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:54:55.550032+0000","last_undegraded":"2024-09-21T11:54:55.550032+0000","last_fullsized":"2024-09-21T11:54:55.550032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:44.478855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.478855+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.275587+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"357'10281","reported_seq":10449,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.225763+0000","last_change":"2024-09-21T11:53:57.419202+0000","last_active":"2024-09-21T11:55:01.225763+0000","last_peered":"2024-09-21T11:55:01.225763+0000","last_clean":"2024-09-21T11:55:01.225763+0000","last_became_active":"2024-09-21T11:52:45.608448+0000","last_became_peered":"2024-09-21T11:52:45.608448+0000","last_unstale":"2024-09-21T11:55:01.225763+0000","last_undegraded":"2024-09-21T11:55:01.225763+0000","last_fullsized":"2024-09-21T11:55:01.225763+0000","mapping_epoch":313,"log_start":"335'5000","ondisk_log_start":"335'5000","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'6082","last_scrub_stamp":"2024-09-21T11:53:57.418961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:57.418961+0000","objects_scrubbed":6033,"log_size":5281,"log_dups_size":0,"ondisk_log_size":5281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:20.883049+0000","scrub_duration":2364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":323420160,"num_objects":4935,"num_object_clones":0,"num_object_copies":14805,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10281,"num_write_kb":486912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":381,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.672761+0000","last_change":"2024-09-21T11:52:39.116264+0000","last_active":"2024-09-21T11:54:54.672761+0000","last_peered":"2024-09-21T11:54:54.672761+0000","last_clean":"2024-09-21T11:54:54.672761+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:54:54.672761+0000","last_undegraded":"2024-09-21T11:54:54.672761+0000","last_fullsized":"2024-09-21T11:54:54.672761+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:52:39.116197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:39.116197+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:47.545993+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":668,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688219+0000","last_change":"2024-09-21T11:53:30.353707+0000","last_active":"2024-09-21T11:54:54.688219+0000","last_peered":"2024-09-21T11:54:54.688219+0000","last_clean":"2024-09-21T11:54:54.688219+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:54:54.688219+0000","last_undegraded":"2024-09-21T11:54:54.688219+0000","last_fullsized":"2024-09-21T11:54:54.688219+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:30.353632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:30.353632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"294'9","reported_seq":207,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.549678+0000","last_change":"2024-09-21T11:51:43.526079+0000","last_active":"2024-09-21T11:54:55.549678+0000","last_peered":"2024-09-21T11:54:55.549678+0000","last_clean":"2024-09-21T11:54:55.549678+0000","last_became_active":"2024-09-21T11:51:43.525566+0000","last_became_peered":"2024-09-21T11:51:43.525566+0000","last_unstale":"2024-09-21T11:54:55.549678+0000","last_undegraded":"2024-09-21T11:54:55.549678+0000","last_fullsized":"2024-09-21T11:54:55.549678+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:11.932479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:11.932479+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:16.143901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":663,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688849+0000","last_change":"2024-09-21T11:53:52.178001+0000","last_active":"2024-09-21T11:54:54.688849+0000","last_peered":"2024-09-21T11:54:54.688849+0000","last_clean":"2024-09-21T11:54:54.688849+0000","last_became_active":"2024-09-21T11:47:03.384576+0000","last_became_peered":"2024-09-21T11:47:03.384576+0000","last_unstale":"2024-09-21T11:54:54.688849+0000","last_undegraded":"2024-09-21T11:54:54.688849+0000","last_fullsized":"2024-09-21T11:54:54.688849+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:52.177812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:52.177812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:05.487897+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"357'10383","reported_seq":10554,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.271489+0000","last_change":"2024-09-21T11:54:02.094582+0000","last_active":"2024-09-21T11:55:01.271489+0000","last_peered":"2024-09-21T11:55:01.271489+0000","last_clean":"2024-09-21T11:55:01.271489+0000","last_became_active":"2024-09-21T11:52:45.606226+0000","last_became_peered":"2024-09-21T11:52:45.606226+0000","last_unstale":"2024-09-21T11:55:01.271489+0000","last_undegraded":"2024-09-21T11:55:01.271489+0000","last_fullsized":"2024-09-21T11:55:01.271489+0000","mapping_epoch":313,"log_start":"331'4300","ondisk_log_start":"331'4300","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"340'6367","last_scrub_stamp":"2024-09-21T11:54:02.094356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:02.094356+0000","objects_scrubbed":6327,"log_size":6083,"log_dups_size":0,"ondisk_log_size":6083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.011413+0000","scrub_duration":1665,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":326696960,"num_objects":4985,"num_object_clones":0,"num_object_copies":14955,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10383,"num_write_kb":491776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":701,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688050+0000","last_change":"2024-09-21T11:52:47.279483+0000","last_active":"2024-09-21T11:54:54.688050+0000","last_peered":"2024-09-21T11:54:54.688050+0000","last_clean":"2024-09-21T11:54:54.688050+0000","last_became_active":"2024-09-21T11:40:02.398987+0000","last_became_peered":"2024-09-21T11:40:02.398987+0000","last_unstale":"2024-09-21T11:54:54.688050+0000","last_undegraded":"2024-09-21T11:54:54.688050+0000","last_fullsized":"2024-09-21T11:54:54.688050+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:52:47.279423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:47.279423+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":662,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.512377+0000","last_change":"2024-09-21T11:54:55.512377+0000","last_active":"2024-09-21T11:54:55.512377+0000","last_peered":"2024-09-21T11:54:55.512377+0000","last_clean":"2024-09-21T11:54:55.512377+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:54:55.512377+0000","last_undegraded":"2024-09-21T11:54:55.512377+0000","last_fullsized":"2024-09-21T11:54:55.512377+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:55.512314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:55.512314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:09.527517+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":140,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688050+0000","last_change":"2024-09-21T11:51:43.542323+0000","last_active":"2024-09-21T11:54:54.688050+0000","last_peered":"2024-09-21T11:54:54.688050+0000","last_clean":"2024-09-21T11:54:54.688050+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:54:54.688050+0000","last_undegraded":"2024-09-21T11:54:54.688050+0000","last_fullsized":"2024-09-21T11:54:54.688050+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.940335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.940335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:50.467868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":150,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.683681+0000","last_change":"2024-09-21T11:54:53.844954+0000","last_active":"2024-09-21T11:54:54.683681+0000","last_peered":"2024-09-21T11:54:54.683681+0000","last_clean":"2024-09-21T11:54:54.683681+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:54:54.683681+0000","last_undegraded":"2024-09-21T11:54:54.683681+0000","last_fullsized":"2024-09-21T11:54:54.683681+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.844876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.844876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:54.854366+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"357'10561","reported_seq":10721,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.291145+0000","last_change":"2024-09-21T11:54:54.009954+0000","last_active":"2024-09-21T11:55:01.291145+0000","last_peered":"2024-09-21T11:55:01.291145+0000","last_clean":"2024-09-21T11:55:01.291145+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:55:01.291145+0000","last_undegraded":"2024-09-21T11:55:01.291145+0000","last_fullsized":"2024-09-21T11:55:01.291145+0000","mapping_epoch":313,"log_start":"331'4500","ondisk_log_start":"331'4500","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"356'10086","last_scrub_stamp":"2024-09-21T11:54:54.009664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:54.009664+0000","objects_scrubbed":5657,"log_size":6061,"log_dups_size":0,"ondisk_log_size":6061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:57.111381+0000","scrub_duration":1483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":336658432,"num_objects":5137,"num_object_clones":0,"num_object_copies":15411,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10561,"num_write_kb":502336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":652,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687222+0000","last_change":"2024-09-21T11:51:43.525478+0000","last_active":"2024-09-21T11:54:54.687222+0000","last_peered":"2024-09-21T11:54:54.687222+0000","last_clean":"2024-09-21T11:54:54.687222+0000","last_became_active":"2024-09-21T11:51:43.523852+0000","last_became_peered":"2024-09-21T11:51:43.523852+0000","last_unstale":"2024-09-21T11:54:54.687222+0000","last_undegraded":"2024-09-21T11:54:54.687222+0000","last_fullsized":"2024-09-21T11:54:54.687222+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.142710+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":676,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.680855+0000","last_change":"2024-09-21T11:50:28.657341+0000","last_active":"2024-09-21T11:54:54.680855+0000","last_peered":"2024-09-21T11:54:54.680855+0000","last_clean":"2024-09-21T11:54:54.680855+0000","last_became_active":"2024-09-21T11:47:03.385602+0000","last_became_peered":"2024-09-21T11:47:03.385602+0000","last_unstale":"2024-09-21T11:54:54.680855+0000","last_undegraded":"2024-09-21T11:54:54.680855+0000","last_fullsized":"2024-09-21T11:54:54.680855+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:28.657261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.657261+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.097296+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":851,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.550147+0000","last_change":"2024-09-21T11:51:20.675165+0000","last_active":"2024-09-21T11:54:55.550147+0000","last_peered":"2024-09-21T11:54:55.550147+0000","last_clean":"2024-09-21T11:54:55.550147+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:54:55.550147+0000","last_undegraded":"2024-09-21T11:54:55.550147+0000","last_fullsized":"2024-09-21T11:54:55.550147+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:51:20.675101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.675101+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:48.486444+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":645,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.672886+0000","last_change":"2024-09-21T11:52:14.010215+0000","last_active":"2024-09-21T11:54:54.672886+0000","last_peered":"2024-09-21T11:54:54.672886+0000","last_clean":"2024-09-21T11:54:54.672886+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:54:54.672886+0000","last_undegraded":"2024-09-21T11:54:54.672886+0000","last_fullsized":"2024-09-21T11:54:54.672886+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:14.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:14.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:45.192898+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"357'10246","reported_seq":10344,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.217789+0000","last_change":"2024-09-21T11:52:45.608507+0000","last_active":"2024-09-21T11:55:01.217789+0000","last_peered":"2024-09-21T11:55:01.217789+0000","last_clean":"2024-09-21T11:55:01.217789+0000","last_became_active":"2024-09-21T11:52:45.608219+0000","last_became_peered":"2024-09-21T11:52:45.608219+0000","last_unstale":"2024-09-21T11:55:01.217789+0000","last_undegraded":"2024-09-21T11:55:01.217789+0000","last_fullsized":"2024-09-21T11:55:01.217789+0000","mapping_epoch":313,"log_start":"334'4900","ondisk_log_start":"334'4900","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5346,"log_dups_size":0,"ondisk_log_size":5346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.363111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331612160,"num_objects":5060,"num_object_clones":0,"num_object_copies":15180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10246,"num_write_kb":489792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":366,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673480+0000","last_change":"2024-09-21T11:53:30.179784+0000","last_active":"2024-09-21T11:54:54.673480+0000","last_peered":"2024-09-21T11:54:54.673480+0000","last_clean":"2024-09-21T11:54:54.673480+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:54:54.673480+0000","last_undegraded":"2024-09-21T11:54:54.673480+0000","last_fullsized":"2024-09-21T11:54:54.673480+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:30.179690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:30.179690+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:54:59.902366+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":678,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.689131+0000","last_change":"2024-09-21T11:54:22.095001+0000","last_active":"2024-09-21T11:54:54.689131+0000","last_peered":"2024-09-21T11:54:54.689131+0000","last_clean":"2024-09-21T11:54:54.689131+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:54:54.689131+0000","last_undegraded":"2024-09-21T11:54:54.689131+0000","last_fullsized":"2024-09-21T11:54:54.689131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:22.094918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:22.094918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:40.780472+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":830,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.550728+0000","last_change":"2024-09-21T11:53:56.181634+0000","last_active":"2024-09-21T11:54:55.550728+0000","last_peered":"2024-09-21T11:54:55.550728+0000","last_clean":"2024-09-21T11:54:55.550728+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:54:55.550728+0000","last_undegraded":"2024-09-21T11:54:55.550728+0000","last_fullsized":"2024-09-21T11:54:55.550728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:53:56.181577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:56.181577+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:17.783176+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":650,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.679852+0000","last_change":"2024-09-21T11:53:48.256119+0000","last_active":"2024-09-21T11:54:54.679852+0000","last_peered":"2024-09-21T11:54:54.679852+0000","last_clean":"2024-09-21T11:54:54.679852+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:54:54.679852+0000","last_undegraded":"2024-09-21T11:54:54.679852+0000","last_fullsized":"2024-09-21T11:54:54.679852+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:48.255951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:48.255951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.617117+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"357'10244","reported_seq":10345,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.299510+0000","last_change":"2024-09-21T11:52:45.607195+0000","last_active":"2024-09-21T11:55:01.299510+0000","last_peered":"2024-09-21T11:55:01.299510+0000","last_clean":"2024-09-21T11:55:01.299510+0000","last_became_active":"2024-09-21T11:52:45.606865+0000","last_became_peered":"2024-09-21T11:52:45.606865+0000","last_unstale":"2024-09-21T11:55:01.299510+0000","last_undegraded":"2024-09-21T11:55:01.299510+0000","last_fullsized":"2024-09-21T11:55:01.299510+0000","mapping_epoch":313,"log_start":"331'4200","ondisk_log_start":"331'4200","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:44.275772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318439448,"num_objects":4860,"num_object_clones":0,"num_object_copies":14580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10244,"num_write_kb":483265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":701,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.679500+0000","last_change":"2024-09-21T11:53:22.488389+0000","last_active":"2024-09-21T11:54:54.679500+0000","last_peered":"2024-09-21T11:54:54.679500+0000","last_clean":"2024-09-21T11:54:54.679500+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:54:54.679500+0000","last_undegraded":"2024-09-21T11:54:54.679500+0000","last_fullsized":"2024-09-21T11:54:54.679500+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:53:22.488238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:22.488238+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":339,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.672303+0000","last_change":"2024-09-21T11:53:27.160558+0000","last_active":"2024-09-21T11:54:54.672303+0000","last_peered":"2024-09-21T11:54:54.672303+0000","last_clean":"2024-09-21T11:54:54.672303+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:54:54.672303+0000","last_undegraded":"2024-09-21T11:54:54.672303+0000","last_fullsized":"2024-09-21T11:54:54.672303+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.160486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.160486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:44.486333+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":638,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.678309+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:54:54.678309+0000","last_peered":"2024-09-21T11:54:54.678309+0000","last_clean":"2024-09-21T11:54:54.678309+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:54:54.678309+0000","last_undegraded":"2024-09-21T11:54:54.678309+0000","last_fullsized":"2024-09-21T11:54:54.678309+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:42.643962+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":339,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.670892+0000","last_change":"2024-09-21T11:52:09.412149+0000","last_active":"2024-09-21T11:54:54.670892+0000","last_peered":"2024-09-21T11:54:54.670892+0000","last_clean":"2024-09-21T11:54:54.670892+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:54:54.670892+0000","last_undegraded":"2024-09-21T11:54:54.670892+0000","last_fullsized":"2024-09-21T11:54:54.670892+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:09.412091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:09.412091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:41.503668+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"357'10294","reported_seq":10394,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.231060+0000","last_change":"2024-09-21T11:52:45.607383+0000","last_active":"2024-09-21T11:55:01.231060+0000","last_peered":"2024-09-21T11:55:01.231060+0000","last_clean":"2024-09-21T11:55:01.231060+0000","last_became_active":"2024-09-21T11:52:45.607000+0000","last_became_peered":"2024-09-21T11:52:45.607000+0000","last_unstale":"2024-09-21T11:55:01.231060+0000","last_undegraded":"2024-09-21T11:55:01.231060+0000","last_fullsized":"2024-09-21T11:55:01.231060+0000","mapping_epoch":313,"log_start":"331'4200","ondisk_log_start":"331'4200","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":6094,"log_dups_size":0,"ondisk_log_size":6094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:49.450213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":323354624,"num_objects":4934,"num_object_clones":0,"num_object_copies":14802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10294,"num_write_kb":487296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":467,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.549742+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:54:55.549742+0000","last_peered":"2024-09-21T11:54:55.549742+0000","last_clean":"2024-09-21T11:54:55.549742+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:54:55.549742+0000","last_undegraded":"2024-09-21T11:54:55.549742+0000","last_fullsized":"2024-09-21T11:54:55.549742+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:17.332467+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":165,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.679533+0000","last_change":"2024-09-21T11:54:52.312620+0000","last_active":"2024-09-21T11:54:54.679533+0000","last_peered":"2024-09-21T11:54:54.679533+0000","last_clean":"2024-09-21T11:54:54.679533+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:54:54.679533+0000","last_undegraded":"2024-09-21T11:54:54.679533+0000","last_fullsized":"2024-09-21T11:54:54.679533+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:54:52.310060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:52.310060+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:21.355022+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":670,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688516+0000","last_change":"2024-09-21T11:51:12.562026+0000","last_active":"2024-09-21T11:54:54.688516+0000","last_peered":"2024-09-21T11:54:54.688516+0000","last_clean":"2024-09-21T11:54:54.688516+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:54:54.688516+0000","last_undegraded":"2024-09-21T11:54:54.688516+0000","last_fullsized":"2024-09-21T11:54:54.688516+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:51:12.561961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.561961+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:17.150594+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":644,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.681658+0000","last_change":"2024-09-21T11:50:37.858401+0000","last_active":"2024-09-21T11:54:54.681658+0000","last_peered":"2024-09-21T11:54:54.681658+0000","last_clean":"2024-09-21T11:54:54.681658+0000","last_became_active":"2024-09-21T11:47:03.384824+0000","last_became_peered":"2024-09-21T11:47:03.384824+0000","last_unstale":"2024-09-21T11:54:54.681658+0000","last_undegraded":"2024-09-21T11:54:54.681658+0000","last_fullsized":"2024-09-21T11:54:54.681658+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:21.058181+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":626,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687649+0000","last_change":"2024-09-21T11:54:34.452033+0000","last_active":"2024-09-21T11:54:54.687649+0000","last_peered":"2024-09-21T11:54:54.687649+0000","last_clean":"2024-09-21T11:54:54.687649+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:54:54.687649+0000","last_undegraded":"2024-09-21T11:54:54.687649+0000","last_fullsized":"2024-09-21T11:54:54.687649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:34.451939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:34.451939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:53.805403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"357'10591","reported_seq":10771,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.389403+0000","last_change":"2024-09-21T11:54:31.986720+0000","last_active":"2024-09-21T11:55:01.389403+0000","last_peered":"2024-09-21T11:55:01.389403+0000","last_clean":"2024-09-21T11:55:01.389403+0000","last_became_active":"2024-09-21T11:52:45.604600+0000","last_became_peered":"2024-09-21T11:52:45.604600+0000","last_unstale":"2024-09-21T11:55:01.389403+0000","last_undegraded":"2024-09-21T11:55:01.389403+0000","last_fullsized":"2024-09-21T11:55:01.389403+0000","mapping_epoch":313,"log_start":"334'5000","ondisk_log_start":"334'5000","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"349'8491","last_scrub_stamp":"2024-09-21T11:54:31.986575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:31.986575+0000","objects_scrubbed":7209,"log_size":5591,"log_dups_size":0,"ondisk_log_size":5591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:50.526900+0000","scrub_duration":1918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330235904,"num_objects":5039,"num_object_clones":0,"num_object_copies":15117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10591,"num_write_kb":500160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":282,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.686864+0000","last_change":"2024-09-21T11:51:41.006710+0000","last_active":"2024-09-21T11:54:54.686864+0000","last_peered":"2024-09-21T11:54:54.686864+0000","last_clean":"2024-09-21T11:54:54.686864+0000","last_became_active":"2024-09-21T11:48:33.028750+0000","last_became_peered":"2024-09-21T11:48:33.028750+0000","last_unstale":"2024-09-21T11:54:54.686864+0000","last_undegraded":"2024-09-21T11:54:54.686864+0000","last_fullsized":"2024-09-21T11:54:54.686864+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:51:41.006622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:41.006622+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:21.055561+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":343,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673588+0000","last_change":"2024-09-21T11:51:43.346789+0000","last_active":"2024-09-21T11:54:54.673588+0000","last_peered":"2024-09-21T11:54:54.673588+0000","last_clean":"2024-09-21T11:54:54.673588+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:54:54.673588+0000","last_undegraded":"2024-09-21T11:54:54.673588+0000","last_fullsized":"2024-09-21T11:54:54.673588+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:51:43.346718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:43.346718+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.466879+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":680,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.683932+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:54:54.683932+0000","last_peered":"2024-09-21T11:54:54.683932+0000","last_clean":"2024-09-21T11:54:54.683932+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:54:54.683932+0000","last_undegraded":"2024-09-21T11:54:54.683932+0000","last_fullsized":"2024-09-21T11:54:54.683932+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:44.933216+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":638,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687886+0000","last_change":"2024-09-21T11:49:47.008541+0000","last_active":"2024-09-21T11:54:54.687886+0000","last_peered":"2024-09-21T11:54:54.687886+0000","last_clean":"2024-09-21T11:54:54.687886+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:54:54.687886+0000","last_undegraded":"2024-09-21T11:54:54.687886+0000","last_fullsized":"2024-09-21T11:54:54.687886+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:47.008369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:47.008369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:03.066869+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":642,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673562+0000","last_change":"2024-09-21T11:54:33.253430+0000","last_active":"2024-09-21T11:54:54.673562+0000","last_peered":"2024-09-21T11:54:54.673562+0000","last_clean":"2024-09-21T11:54:54.673562+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:54:54.673562+0000","last_undegraded":"2024-09-21T11:54:54.673562+0000","last_fullsized":"2024-09-21T11:54:54.673562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:33.253368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:33.253368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:58.365025+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"357'10364","reported_seq":10528,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:02.279767+0000","last_change":"2024-09-21T11:54:57.284460+0000","last_active":"2024-09-21T11:55:02.279767+0000","last_peered":"2024-09-21T11:55:02.279767+0000","last_clean":"2024-09-21T11:55:02.279767+0000","last_became_active":"2024-09-21T11:52:45.604244+0000","last_became_peered":"2024-09-21T11:52:45.604244+0000","last_unstale":"2024-09-21T11:55:02.279767+0000","last_undegraded":"2024-09-21T11:55:02.279767+0000","last_fullsized":"2024-09-21T11:55:02.279767+0000","mapping_epoch":313,"log_start":"336'5100","ondisk_log_start":"336'5100","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"357'10040","last_scrub_stamp":"2024-09-21T11:54:57.284313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:57.284313+0000","objects_scrubbed":5333,"log_size":5264,"log_dups_size":0,"ondisk_log_size":5264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:19.005227+0000","scrub_duration":2019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324009984,"num_objects":4944,"num_object_clones":0,"num_object_copies":14832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10364,"num_write_kb":489856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":719,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.680660+0000","last_change":"2024-09-21T11:53:20.438262+0000","last_active":"2024-09-21T11:54:54.680660+0000","last_peered":"2024-09-21T11:54:54.680660+0000","last_clean":"2024-09-21T11:54:54.680660+0000","last_became_active":"2024-09-21T11:47:03.385096+0000","last_became_peered":"2024-09-21T11:47:03.385096+0000","last_unstale":"2024-09-21T11:54:54.680660+0000","last_undegraded":"2024-09-21T11:54:54.680660+0000","last_fullsized":"2024-09-21T11:54:54.680660+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:53:20.438174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.438174+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":662,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.678885+0000","last_change":"2024-09-21T11:54:32.060813+0000","last_active":"2024-09-21T11:54:54.678885+0000","last_peered":"2024-09-21T11:54:54.678885+0000","last_clean":"2024-09-21T11:54:54.678885+0000","last_became_active":"2024-09-21T11:40:00.311937+0000","last_became_peered":"2024-09-21T11:40:00.311937+0000","last_unstale":"2024-09-21T11:54:54.678885+0000","last_undegraded":"2024-09-21T11:54:54.678885+0000","last_fullsized":"2024-09-21T11:54:54.678885+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:32.060638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:32.060638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:43.748498+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1588,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:59.325334+0000","last_change":"2024-09-21T11:54:38.270801+0000","last_active":"2024-09-21T11:54:59.325334+0000","last_peered":"2024-09-21T11:54:59.325334+0000","last_clean":"2024-09-21T11:54:59.325334+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:54:59.325334+0000","last_undegraded":"2024-09-21T11:54:59.325334+0000","last_fullsized":"2024-09-21T11:54:59.325334+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:54:38.270748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.270748+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:51.553375+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":891,"num_read_kb":886,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":651,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.689531+0000","last_change":"2024-09-21T11:51:17.645920+0000","last_active":"2024-09-21T11:54:54.689531+0000","last_peered":"2024-09-21T11:54:54.689531+0000","last_clean":"2024-09-21T11:54:54.689531+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:54:54.689531+0000","last_undegraded":"2024-09-21T11:54:54.689531+0000","last_fullsized":"2024-09-21T11:54:54.689531+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.645823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.645823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.111560+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":347,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.671294+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:54:54.671294+0000","last_peered":"2024-09-21T11:54:54.671294+0000","last_clean":"2024-09-21T11:54:54.671294+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:54:54.671294+0000","last_undegraded":"2024-09-21T11:54:54.671294+0000","last_fullsized":"2024-09-21T11:54:54.671294+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.479171+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"357'10260","reported_seq":10358,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.299595+0000","last_change":"2024-09-21T11:52:45.607096+0000","last_active":"2024-09-21T11:55:01.299595+0000","last_peered":"2024-09-21T11:55:01.299595+0000","last_clean":"2024-09-21T11:55:01.299595+0000","last_became_active":"2024-09-21T11:52:45.606777+0000","last_became_peered":"2024-09-21T11:52:45.606777+0000","last_unstale":"2024-09-21T11:55:01.299595+0000","last_undegraded":"2024-09-21T11:55:01.299595+0000","last_fullsized":"2024-09-21T11:55:01.299595+0000","mapping_epoch":313,"log_start":"331'4200","ondisk_log_start":"331'4200","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":6060,"log_dups_size":0,"ondisk_log_size":6060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:48.437837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":323616768,"num_objects":4938,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10260,"num_write_kb":486336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":871,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.549857+0000","last_change":"2024-09-21T11:54:35.232137+0000","last_active":"2024-09-21T11:54:55.549857+0000","last_peered":"2024-09-21T11:54:55.549857+0000","last_clean":"2024-09-21T11:54:55.549857+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:54:55.549857+0000","last_undegraded":"2024-09-21T11:54:55.549857+0000","last_fullsized":"2024-09-21T11:54:55.549857+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:54:35.231924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:35.231924+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:51.395140+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":647,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687550+0000","last_change":"2024-09-21T11:50:59.680952+0000","last_active":"2024-09-21T11:54:54.687550+0000","last_peered":"2024-09-21T11:54:54.687550+0000","last_clean":"2024-09-21T11:54:54.687550+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:54:54.687550+0000","last_undegraded":"2024-09-21T11:54:54.687550+0000","last_fullsized":"2024-09-21T11:54:54.687550+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:59.680755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:59.680755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:21.608988+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":717,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673480+0000","last_change":"2024-09-21T11:54:22.360499+0000","last_active":"2024-09-21T11:54:54.673480+0000","last_peered":"2024-09-21T11:54:54.673480+0000","last_clean":"2024-09-21T11:54:54.673480+0000","last_became_active":"2024-09-21T11:39:51.236482+0000","last_became_peered":"2024-09-21T11:39:51.236482+0000","last_unstale":"2024-09-21T11:54:54.673480+0000","last_undegraded":"2024-09-21T11:54:54.673480+0000","last_fullsized":"2024-09-21T11:54:54.673480+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:54:22.360410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:54:22.360410+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:32.516643+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":646,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.674574+0000","last_change":"2024-09-21T11:51:18.318095+0000","last_active":"2024-09-21T11:54:54.674574+0000","last_peered":"2024-09-21T11:54:54.674574+0000","last_clean":"2024-09-21T11:54:54.674574+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:54:54.674574+0000","last_undegraded":"2024-09-21T11:54:54.674574+0000","last_fullsized":"2024-09-21T11:54:54.674574+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.317950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.317950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:11.261207+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":649,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:58.013361+0000","last_change":"2024-09-21T11:54:58.013361+0000","last_active":"2024-09-21T11:54:58.013361+0000","last_peered":"2024-09-21T11:54:58.013361+0000","last_clean":"2024-09-21T11:54:58.013361+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:54:58.013361+0000","last_undegraded":"2024-09-21T11:54:58.013361+0000","last_fullsized":"2024-09-21T11:54:58.013361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:58.013113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:58.013113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:02.912267+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"357'10473","reported_seq":10640,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.294047+0000","last_change":"2024-09-21T11:53:53.487635+0000","last_active":"2024-09-21T11:55:01.294047+0000","last_peered":"2024-09-21T11:55:01.294047+0000","last_clean":"2024-09-21T11:55:01.294047+0000","last_became_active":"2024-09-21T11:52:45.605575+0000","last_became_peered":"2024-09-21T11:52:45.605575+0000","last_unstale":"2024-09-21T11:55:01.294047+0000","last_undegraded":"2024-09-21T11:55:01.294047+0000","last_fullsized":"2024-09-21T11:55:01.294047+0000","mapping_epoch":313,"log_start":"331'4400","ondisk_log_start":"331'4400","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'5948","last_scrub_stamp":"2024-09-21T11:53:53.487473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:53.487473+0000","objects_scrubbed":5891,"log_size":6073,"log_dups_size":0,"ondisk_log_size":6073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:06.519835+0000","scrub_duration":1988,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329056256,"num_objects":5021,"num_object_clones":0,"num_object_copies":15063,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10473,"num_write_kb":495808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":663,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.679447+0000","last_change":"2024-09-21T11:53:58.427767+0000","last_active":"2024-09-21T11:54:54.679447+0000","last_peered":"2024-09-21T11:54:54.679447+0000","last_clean":"2024-09-21T11:54:54.679447+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:54:54.679447+0000","last_undegraded":"2024-09-21T11:54:54.679447+0000","last_fullsized":"2024-09-21T11:54:54.679447+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:58.427708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:58.427708+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:08.495842+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":148,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.683725+0000","last_change":"2024-09-21T11:52:13.709151+0000","last_active":"2024-09-21T11:54:54.683725+0000","last_peered":"2024-09-21T11:54:54.683725+0000","last_clean":"2024-09-21T11:54:54.683725+0000","last_became_active":"2024-09-21T11:51:43.532027+0000","last_became_peered":"2024-09-21T11:51:43.532027+0000","last_unstale":"2024-09-21T11:54:54.683725+0000","last_undegraded":"2024-09-21T11:54:54.683725+0000","last_fullsized":"2024-09-21T11:54:54.683725+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:13.709074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:13.709074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:38.928791+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":629,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.689352+0000","last_change":"2024-09-21T11:48:23.020005+0000","last_active":"2024-09-21T11:54:54.689352+0000","last_peered":"2024-09-21T11:54:54.689352+0000","last_clean":"2024-09-21T11:54:54.689352+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:54:54.689352+0000","last_undegraded":"2024-09-21T11:54:54.689352+0000","last_fullsized":"2024-09-21T11:54:54.689352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:48:23.019951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:48:23.019951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:05.297159+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":648,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687302+0000","last_change":"2024-09-21T11:53:52.437233+0000","last_active":"2024-09-21T11:54:54.687302+0000","last_peered":"2024-09-21T11:54:54.687302+0000","last_clean":"2024-09-21T11:54:54.687302+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:54:54.687302+0000","last_undegraded":"2024-09-21T11:54:54.687302+0000","last_fullsized":"2024-09-21T11:54:54.687302+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:52.437166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:52.437166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:02.147272+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"357'10259","reported_seq":10428,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.578138+0000","last_change":"2024-09-21T11:54:00.488606+0000","last_active":"2024-09-21T11:55:01.578138+0000","last_peered":"2024-09-21T11:55:01.578138+0000","last_clean":"2024-09-21T11:55:01.578138+0000","last_became_active":"2024-09-21T11:52:45.637825+0000","last_became_peered":"2024-09-21T11:52:45.637825+0000","last_unstale":"2024-09-21T11:55:01.578138+0000","last_undegraded":"2024-09-21T11:55:01.578138+0000","last_fullsized":"2024-09-21T11:55:01.578138+0000","mapping_epoch":313,"log_start":"327'3500","ondisk_log_start":"327'3500","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'6224","last_scrub_stamp":"2024-09-21T11:54:00.488535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.488535+0000","objects_scrubbed":6155,"log_size":6759,"log_dups_size":0,"ondisk_log_size":6759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:21.189007+0000","scrub_duration":2345,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321323008,"num_objects":4903,"num_object_clones":0,"num_object_copies":14709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10259,"num_write_kb":485184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":726,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.686571+0000","last_change":"2024-09-21T11:51:43.533280+0000","last_active":"2024-09-21T11:54:54.686571+0000","last_peered":"2024-09-21T11:54:54.686571+0000","last_clean":"2024-09-21T11:54:54.686571+0000","last_became_active":"2024-09-21T11:51:43.525941+0000","last_became_peered":"2024-09-21T11:51:43.525941+0000","last_unstale":"2024-09-21T11:54:54.686571+0000","last_undegraded":"2024-09-21T11:54:54.686571+0000","last_fullsized":"2024-09-21T11:54:54.686571+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:50:36.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.385902+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:08.215334+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":675,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.678759+0000","last_change":"2024-09-21T11:54:36.208783+0000","last_active":"2024-09-21T11:54:54.678759+0000","last_peered":"2024-09-21T11:54:54.678759+0000","last_clean":"2024-09-21T11:54:54.678759+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:54:54.678759+0000","last_undegraded":"2024-09-21T11:54:54.678759+0000","last_fullsized":"2024-09-21T11:54:54.678759+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:36.208720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:36.208720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:46.586464+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":339,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.671189+0000","last_change":"2024-09-21T11:52:44.976410+0000","last_active":"2024-09-21T11:54:54.671189+0000","last_peered":"2024-09-21T11:54:54.671189+0000","last_clean":"2024-09-21T11:54:54.671189+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:54:54.671189+0000","last_undegraded":"2024-09-21T11:54:54.671189+0000","last_fullsized":"2024-09-21T11:54:54.671189+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.976363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.976363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:06.448787+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":642,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687035+0000","last_change":"2024-09-21T11:51:47.944684+0000","last_active":"2024-09-21T11:54:54.687035+0000","last_peered":"2024-09-21T11:54:54.687035+0000","last_clean":"2024-09-21T11:54:54.687035+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:54:54.687035+0000","last_undegraded":"2024-09-21T11:54:54.687035+0000","last_fullsized":"2024-09-21T11:54:54.687035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:47.944609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.944609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.139462+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"357'10464","reported_seq":10629,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.439114+0000","last_change":"2024-09-21T11:53:53.011097+0000","last_active":"2024-09-21T11:55:01.439114+0000","last_peered":"2024-09-21T11:55:01.439114+0000","last_clean":"2024-09-21T11:55:01.439114+0000","last_became_active":"2024-09-21T11:52:45.601350+0000","last_became_peered":"2024-09-21T11:52:45.601350+0000","last_unstale":"2024-09-21T11:55:01.439114+0000","last_undegraded":"2024-09-21T11:55:01.439114+0000","last_fullsized":"2024-09-21T11:55:01.439114+0000","mapping_epoch":313,"log_start":"336'5200","ondisk_log_start":"336'5200","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'5855","last_scrub_stamp":"2024-09-21T11:53:53.011008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:53.011008+0000","objects_scrubbed":5813,"log_size":5264,"log_dups_size":0,"ondisk_log_size":5264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:10.597953+0000","scrub_duration":1614,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324272128,"num_objects":4948,"num_object_clones":0,"num_object_copies":14844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":493184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":640,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688941+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:54:54.688941+0000","last_peered":"2024-09-21T11:54:54.688941+0000","last_clean":"2024-09-21T11:54:54.688941+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:54:54.688941+0000","last_undegraded":"2024-09-21T11:54:54.688941+0000","last_fullsized":"2024-09-21T11:54:54.688941+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:19.572942+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":638,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688323+0000","last_change":"2024-09-21T11:53:22.315543+0000","last_active":"2024-09-21T11:54:54.688323+0000","last_peered":"2024-09-21T11:54:54.688323+0000","last_clean":"2024-09-21T11:54:54.688323+0000","last_became_active":"2024-09-21T11:40:00.307458+0000","last_became_peered":"2024-09-21T11:40:00.307458+0000","last_unstale":"2024-09-21T11:54:54.688323+0000","last_undegraded":"2024-09-21T11:54:54.688323+0000","last_fullsized":"2024-09-21T11:54:54.688323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:22.315149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:22.315149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":843,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.554514+0000","last_change":"2024-09-21T11:53:31.991392+0000","last_active":"2024-09-21T11:54:55.554514+0000","last_peered":"2024-09-21T11:54:55.554514+0000","last_clean":"2024-09-21T11:54:55.554514+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:54:55.554514+0000","last_undegraded":"2024-09-21T11:54:55.554514+0000","last_fullsized":"2024-09-21T11:54:55.554514+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:53:31.991316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:31.991316+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:39.887323+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":459,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.672452+0000","last_change":"2024-09-21T11:52:52.981662+0000","last_active":"2024-09-21T11:54:54.672452+0000","last_peered":"2024-09-21T11:54:54.672452+0000","last_clean":"2024-09-21T11:54:54.672452+0000","last_became_active":"2024-09-21T11:51:43.543072+0000","last_became_peered":"2024-09-21T11:51:43.543072+0000","last_unstale":"2024-09-21T11:54:54.672452+0000","last_undegraded":"2024-09-21T11:54:54.672452+0000","last_fullsized":"2024-09-21T11:54:54.672452+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:52.981586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:52.981586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:05.493607+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"357'10346","reported_seq":10445,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.586352+0000","last_change":"2024-09-21T11:52:45.635909+0000","last_active":"2024-09-21T11:55:01.586352+0000","last_peered":"2024-09-21T11:55:01.586352+0000","last_clean":"2024-09-21T11:55:01.586352+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:55:01.586352+0000","last_undegraded":"2024-09-21T11:55:01.586352+0000","last_fullsized":"2024-09-21T11:55:01.586352+0000","mapping_epoch":313,"log_start":"327'3600","ondisk_log_start":"327'3600","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":6746,"log_dups_size":0,"ondisk_log_size":6746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:12.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322043904,"num_objects":4914,"num_object_clones":0,"num_object_copies":14742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10346,"num_write_kb":488320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":684,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.679430+0000","last_change":"2024-09-21T11:48:21.207732+0000","last_active":"2024-09-21T11:54:54.679430+0000","last_peered":"2024-09-21T11:54:54.679430+0000","last_clean":"2024-09-21T11:54:54.679430+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:54:54.679430+0000","last_undegraded":"2024-09-21T11:54:54.679430+0000","last_fullsized":"2024-09-21T11:54:54.679430+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-21T11:48:21.207648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:48:21.207648+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.191640+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":669,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688770+0000","last_change":"2024-09-21T11:53:56.166604+0000","last_active":"2024-09-21T11:54:54.688770+0000","last_peered":"2024-09-21T11:54:54.688770+0000","last_clean":"2024-09-21T11:54:54.688770+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:54:54.688770+0000","last_undegraded":"2024-09-21T11:54:54.688770+0000","last_fullsized":"2024-09-21T11:54:54.688770+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:56.166533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:56.166533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.929990+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":638,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.683840+0000","last_change":"2024-09-21T11:53:26.990691+0000","last_active":"2024-09-21T11:54:54.683840+0000","last_peered":"2024-09-21T11:54:54.683840+0000","last_clean":"2024-09-21T11:54:54.683840+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:54:54.683840+0000","last_undegraded":"2024-09-21T11:54:54.683840+0000","last_fullsized":"2024-09-21T11:54:54.683840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.990626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.990626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:45.922336+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":625,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.678789+0000","last_change":"2024-09-21T11:53:31.442098+0000","last_active":"2024-09-21T11:54:54.678789+0000","last_peered":"2024-09-21T11:54:54.678789+0000","last_clean":"2024-09-21T11:54:54.678789+0000","last_became_active":"2024-09-21T11:40:06.341630+0000","last_became_peered":"2024-09-21T11:40:06.341630+0000","last_unstale":"2024-09-21T11:54:54.678789+0000","last_undegraded":"2024-09-21T11:54:54.678789+0000","last_fullsized":"2024-09-21T11:54:54.678789+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:31.442027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:31.442027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"357'10470","reported_seq":10641,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:02.322571+0000","last_change":"2024-09-21T11:54:01.780305+0000","last_active":"2024-09-21T11:55:02.322571+0000","last_peered":"2024-09-21T11:55:02.322571+0000","last_clean":"2024-09-21T11:55:02.322571+0000","last_became_active":"2024-09-21T11:52:45.636291+0000","last_became_peered":"2024-09-21T11:52:45.636291+0000","last_unstale":"2024-09-21T11:55:02.322571+0000","last_undegraded":"2024-09-21T11:55:02.322571+0000","last_fullsized":"2024-09-21T11:55:02.322571+0000","mapping_epoch":313,"log_start":"336'5200","ondisk_log_start":"336'5200","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"340'6353","last_scrub_stamp":"2024-09-21T11:54:01.780225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.780225+0000","objects_scrubbed":6266,"log_size":5270,"log_dups_size":0,"ondisk_log_size":5270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:15.413281+0000","scrub_duration":2578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322961408,"num_objects":4928,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10470,"num_write_kb":492736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":758,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.678563+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:54:54.678563+0000","last_peered":"2024-09-21T11:54:54.678563+0000","last_clean":"2024-09-21T11:54:54.678563+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:54:54.678563+0000","last_undegraded":"2024-09-21T11:54:54.678563+0000","last_fullsized":"2024-09-21T11:54:54.678563+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.382566+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":645,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673876+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:54:54.673876+0000","last_peered":"2024-09-21T11:54:54.673876+0000","last_clean":"2024-09-21T11:54:54.673876+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:54:54.673876+0000","last_undegraded":"2024-09-21T11:54:54.673876+0000","last_fullsized":"2024-09-21T11:54:54.673876+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:43.232503+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":654,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.686641+0000","last_change":"2024-09-21T11:51:48.941202+0000","last_active":"2024-09-21T11:54:54.686641+0000","last_peered":"2024-09-21T11:54:54.686641+0000","last_clean":"2024-09-21T11:54:54.686641+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:54:54.686641+0000","last_undegraded":"2024-09-21T11:54:54.686641+0000","last_fullsized":"2024-09-21T11:54:54.686641+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:48.941128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:48.941128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:17.103034+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":641,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673503+0000","last_change":"2024-09-21T11:52:12.100354+0000","last_active":"2024-09-21T11:54:54.673503+0000","last_peered":"2024-09-21T11:54:54.673503+0000","last_clean":"2024-09-21T11:54:54.673503+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:54:54.673503+0000","last_undegraded":"2024-09-21T11:54:54.673503+0000","last_fullsized":"2024-09-21T11:54:54.673503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.100292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:44.217224+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"357'10426","reported_seq":10588,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.294052+0000","last_change":"2024-09-21T11:54:55.485474+0000","last_active":"2024-09-21T11:55:01.294052+0000","last_peered":"2024-09-21T11:55:01.294052+0000","last_clean":"2024-09-21T11:55:01.294052+0000","last_became_active":"2024-09-21T11:52:45.636672+0000","last_became_peered":"2024-09-21T11:52:45.636672+0000","last_unstale":"2024-09-21T11:55:01.294052+0000","last_undegraded":"2024-09-21T11:55:01.294052+0000","last_fullsized":"2024-09-21T11:55:01.294052+0000","mapping_epoch":313,"log_start":"331'4400","ondisk_log_start":"331'4400","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"357'10034","last_scrub_stamp":"2024-09-21T11:54:55.485108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:55.485108+0000","objects_scrubbed":5494,"log_size":6026,"log_dups_size":0,"ondisk_log_size":6026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:10.702275+0000","scrub_duration":1931,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330039296,"num_objects":5036,"num_object_clones":0,"num_object_copies":15108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10426,"num_write_kb":494784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":680,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.671100+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:54:54.671100+0000","last_peered":"2024-09-21T11:54:54.671100+0000","last_clean":"2024-09-21T11:54:54.671100+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:54:54.671100+0000","last_undegraded":"2024-09-21T11:54:54.671100+0000","last_fullsized":"2024-09-21T11:54:54.671100+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:11.511710+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":683,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.674190+0000","last_change":"2024-09-21T11:53:58.230600+0000","last_active":"2024-09-21T11:54:54.674190+0000","last_peered":"2024-09-21T11:54:54.674190+0000","last_clean":"2024-09-21T11:54:54.674190+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:54:54.674190+0000","last_undegraded":"2024-09-21T11:54:54.674190+0000","last_fullsized":"2024-09-21T11:54:54.674190+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:58.230513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:58.230513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.582409+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":668,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.684203+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:54:54.684203+0000","last_peered":"2024-09-21T11:54:54.684203+0000","last_clean":"2024-09-21T11:54:54.684203+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:54:54.684203+0000","last_undegraded":"2024-09-21T11:54:54.684203+0000","last_fullsized":"2024-09-21T11:54:54.684203+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.693324+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":648,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688053+0000","last_change":"2024-09-21T11:51:09.660956+0000","last_active":"2024-09-21T11:54:54.688053+0000","last_peered":"2024-09-21T11:54:54.688053+0000","last_clean":"2024-09-21T11:54:54.688053+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:54:54.688053+0000","last_undegraded":"2024-09-21T11:54:54.688053+0000","last_fullsized":"2024-09-21T11:54:54.688053+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:09.660880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.660880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:47.501067+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"357'10233","reported_seq":10398,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.377599+0000","last_change":"2024-09-21T11:53:51.496758+0000","last_active":"2024-09-21T11:55:01.377599+0000","last_peered":"2024-09-21T11:55:01.377599+0000","last_clean":"2024-09-21T11:55:01.377599+0000","last_became_active":"2024-09-21T11:52:45.603101+0000","last_became_peered":"2024-09-21T11:52:45.603101+0000","last_unstale":"2024-09-21T11:55:01.377599+0000","last_undegraded":"2024-09-21T11:55:01.377599+0000","last_fullsized":"2024-09-21T11:55:01.377599+0000","mapping_epoch":313,"log_start":"332'4600","ondisk_log_start":"332'4600","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"337'5707","last_scrub_stamp":"2024-09-21T11:53:51.496521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:51.496521+0000","objects_scrubbed":5625,"log_size":5633,"log_dups_size":0,"ondisk_log_size":5633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.557812+0000","scrub_duration":2450,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329056256,"num_objects":5021,"num_object_clones":0,"num_object_copies":15063,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10233,"num_write_kb":488128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":692,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.019421+0000","last_change":"2024-09-21T11:55:01.019421+0000","last_active":"2024-09-21T11:55:01.019421+0000","last_peered":"2024-09-21T11:55:01.019421+0000","last_clean":"2024-09-21T11:55:01.019421+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:55:01.019421+0000","last_undegraded":"2024-09-21T11:55:01.019421+0000","last_fullsized":"2024-09-21T11:55:01.019421+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:55:01.019348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.019348+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:13.582637+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":673,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.674317+0000","last_change":"2024-09-21T11:52:43.086438+0000","last_active":"2024-09-21T11:54:54.674317+0000","last_peered":"2024-09-21T11:54:54.674317+0000","last_clean":"2024-09-21T11:54:54.674317+0000","last_became_active":"2024-09-21T11:40:00.310449+0000","last_became_peered":"2024-09-21T11:40:00.310449+0000","last_unstale":"2024-09-21T11:54:54.674317+0000","last_undegraded":"2024-09-21T11:54:54.674317+0000","last_fullsized":"2024-09-21T11:54:54.674317+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:52:43.086376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:43.086376+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:15.310400+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":834,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.549947+0000","last_change":"2024-09-21T11:54:23.346794+0000","last_active":"2024-09-21T11:54:55.549947+0000","last_peered":"2024-09-21T11:54:55.549947+0000","last_clean":"2024-09-21T11:54:55.549947+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:54:55.549947+0000","last_undegraded":"2024-09-21T11:54:55.549947+0000","last_fullsized":"2024-09-21T11:54:55.549947+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:54:23.346727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.346727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:23.671194+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":633,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688162+0000","last_change":"2024-09-21T11:53:27.992553+0000","last_active":"2024-09-21T11:54:54.688162+0000","last_peered":"2024-09-21T11:54:54.688162+0000","last_clean":"2024-09-21T11:54:54.688162+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:54:54.688162+0000","last_undegraded":"2024-09-21T11:54:54.688162+0000","last_fullsized":"2024-09-21T11:54:54.688162+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.992489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.992489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:39.963992+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":669,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.680429+0000","last_change":"2024-09-21T11:54:38.208884+0000","last_active":"2024-09-21T11:54:54.680429+0000","last_peered":"2024-09-21T11:54:54.680429+0000","last_clean":"2024-09-21T11:54:54.680429+0000","last_became_active":"2024-09-21T11:40:00.310298+0000","last_became_peered":"2024-09-21T11:40:00.310298+0000","last_unstale":"2024-09-21T11:54:54.680429+0000","last_undegraded":"2024-09-21T11:54:54.680429+0000","last_fullsized":"2024-09-21T11:54:54.680429+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:38.208809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.208809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:03.474977+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":682,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.690209+0000","last_change":"2024-09-21T11:54:02.489875+0000","last_active":"2024-09-21T11:54:54.690209+0000","last_peered":"2024-09-21T11:54:54.690209+0000","last_clean":"2024-09-21T11:54:54.690209+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:54:54.690209+0000","last_undegraded":"2024-09-21T11:54:54.690209+0000","last_fullsized":"2024-09-21T11:54:54.690209+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:54:02.489798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:02.489798+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:07.297459+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":675,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.689538+0000","last_change":"2024-09-21T11:54:32.067297+0000","last_active":"2024-09-21T11:54:54.689538+0000","last_peered":"2024-09-21T11:54:54.689538+0000","last_clean":"2024-09-21T11:54:54.689538+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:54:54.689538+0000","last_undegraded":"2024-09-21T11:54:54.689538+0000","last_fullsized":"2024-09-21T11:54:54.689538+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:32.067168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:32.067168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:41.374600+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":649,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673482+0000","last_change":"2024-09-21T11:53:26.139062+0000","last_active":"2024-09-21T11:54:54.673482+0000","last_peered":"2024-09-21T11:54:54.673482+0000","last_clean":"2024-09-21T11:54:54.673482+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:54:54.673482+0000","last_undegraded":"2024-09-21T11:54:54.673482+0000","last_fullsized":"2024-09-21T11:54:54.673482+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.138988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.138988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:46.217683+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":683,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.680751+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:54:54.680751+0000","last_peered":"2024-09-21T11:54:54.680751+0000","last_clean":"2024-09-21T11:54:54.680751+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:54:54.680751+0000","last_undegraded":"2024-09-21T11:54:54.680751+0000","last_fullsized":"2024-09-21T11:54:54.680751+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":645,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673240+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:54:54.673240+0000","last_peered":"2024-09-21T11:54:54.673240+0000","last_clean":"2024-09-21T11:54:54.673240+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:54:54.673240+0000","last_undegraded":"2024-09-21T11:54:54.673240+0000","last_fullsized":"2024-09-21T11:54:54.673240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:14.263136+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":660,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.690056+0000","last_change":"2024-09-21T11:52:41.245695+0000","last_active":"2024-09-21T11:54:54.690056+0000","last_peered":"2024-09-21T11:54:54.690056+0000","last_clean":"2024-09-21T11:54:54.690056+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:54:54.690056+0000","last_undegraded":"2024-09-21T11:54:54.690056+0000","last_fullsized":"2024-09-21T11:54:54.690056+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:41.245632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:52:41.245632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":642,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.686250+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:54:54.686250+0000","last_peered":"2024-09-21T11:54:54.686250+0000","last_clean":"2024-09-21T11:54:54.686250+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:54:54.686250+0000","last_undegraded":"2024-09-21T11:54:54.686250+0000","last_fullsized":"2024-09-21T11:54:54.686250+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:04.101504+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":674,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.999030+0000","last_change":"2024-09-21T11:54:54.999030+0000","last_active":"2024-09-21T11:54:54.999030+0000","last_peered":"2024-09-21T11:54:54.999030+0000","last_clean":"2024-09-21T11:54:54.999030+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:54:54.999030+0000","last_undegraded":"2024-09-21T11:54:54.999030+0000","last_fullsized":"2024-09-21T11:54:54.999030+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:54.998955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:54.998955+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:22.083273+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":661,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.682831+0000","last_change":"2024-09-21T11:52:45.688676+0000","last_active":"2024-09-21T11:54:54.682831+0000","last_peered":"2024-09-21T11:54:54.682831+0000","last_clean":"2024-09-21T11:54:54.682831+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:54:54.682831+0000","last_undegraded":"2024-09-21T11:54:54.682831+0000","last_fullsized":"2024-09-21T11:54:54.682831+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:45.688617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:45.688617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":811,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.550413+0000","last_change":"2024-09-21T11:51:08.629172+0000","last_active":"2024-09-21T11:54:55.550413+0000","last_peered":"2024-09-21T11:54:55.550413+0000","last_clean":"2024-09-21T11:54:55.550413+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:54:55.550413+0000","last_undegraded":"2024-09-21T11:54:55.550413+0000","last_fullsized":"2024-09-21T11:54:55.550413+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:08.628986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:08.628986+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:40.424036+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":140,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.686956+0000","last_change":"2024-09-21T11:51:43.531318+0000","last_active":"2024-09-21T11:54:54.686956+0000","last_peered":"2024-09-21T11:54:54.686956+0000","last_clean":"2024-09-21T11:54:54.686956+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:54:54.686956+0000","last_undegraded":"2024-09-21T11:54:54.686956+0000","last_fullsized":"2024-09-21T11:54:54.686956+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.826834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.826834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:11.200154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":702,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.689014+0000","last_change":"2024-09-21T11:53:25.264037+0000","last_active":"2024-09-21T11:54:54.689014+0000","last_peered":"2024-09-21T11:54:54.689014+0000","last_clean":"2024-09-21T11:54:54.689014+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:54:54.689014+0000","last_undegraded":"2024-09-21T11:54:54.689014+0000","last_fullsized":"2024-09-21T11:54:54.689014+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:25.263955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:25.263955+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":645,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673153+0000","last_change":"2024-09-21T11:54:35.264828+0000","last_active":"2024-09-21T11:54:54.673153+0000","last_peered":"2024-09-21T11:54:54.673153+0000","last_clean":"2024-09-21T11:54:54.673153+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:54:54.673153+0000","last_undegraded":"2024-09-21T11:54:54.673153+0000","last_fullsized":"2024-09-21T11:54:54.673153+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:35.264755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:35.264755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:02.953376+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":857,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.549694+0000","last_change":"2024-09-21T11:53:24.947837+0000","last_active":"2024-09-21T11:54:55.549694+0000","last_peered":"2024-09-21T11:54:55.549694+0000","last_clean":"2024-09-21T11:54:55.549694+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:54:55.549694+0000","last_undegraded":"2024-09-21T11:54:55.549694+0000","last_fullsized":"2024-09-21T11:54:55.549694+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:53:24.947744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:24.947744+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":641,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.680357+0000","last_change":"2024-09-21T11:50:02.782565+0000","last_active":"2024-09-21T11:54:54.680357+0000","last_peered":"2024-09-21T11:54:54.680357+0000","last_clean":"2024-09-21T11:54:54.680357+0000","last_became_active":"2024-09-21T11:40:06.351674+0000","last_became_peered":"2024-09-21T11:40:06.351674+0000","last_unstale":"2024-09-21T11:54:54.680357+0000","last_undegraded":"2024-09-21T11:54:54.680357+0000","last_fullsized":"2024-09-21T11:54:54.680357+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:02.782475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:02.782475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:46.128440+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":690,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.679204+0000","last_change":"2024-09-21T11:51:43.533694+0000","last_active":"2024-09-21T11:54:54.679204+0000","last_peered":"2024-09-21T11:54:54.679204+0000","last_clean":"2024-09-21T11:54:54.679204+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:54:54.679204+0000","last_undegraded":"2024-09-21T11:54:54.679204+0000","last_fullsized":"2024-09-21T11:54:54.679204+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:08.366846+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":148,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.674691+0000","last_change":"2024-09-21T11:51:52.159555+0000","last_active":"2024-09-21T11:54:54.674691+0000","last_peered":"2024-09-21T11:54:54.674691+0000","last_clean":"2024-09-21T11:54:54.674691+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:54:54.674691+0000","last_undegraded":"2024-09-21T11:54:54.674691+0000","last_fullsized":"2024-09-21T11:54:54.674691+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:52.159433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:52.159433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:16.279585+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":347,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.671486+0000","last_change":"2024-09-21T11:54:35.426101+0000","last_active":"2024-09-21T11:54:54.671486+0000","last_peered":"2024-09-21T11:54:54.671486+0000","last_clean":"2024-09-21T11:54:54.671486+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:54:54.671486+0000","last_undegraded":"2024-09-21T11:54:54.671486+0000","last_fullsized":"2024-09-21T11:54:54.671486+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:35.426030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:35.426030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:52.101778+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":634,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:55:01.434194+0000","last_change":"2024-09-21T11:55:01.434194+0000","last_active":"2024-09-21T11:55:01.434194+0000","last_peered":"2024-09-21T11:55:01.434194+0000","last_clean":"2024-09-21T11:55:01.434194+0000","last_became_active":"2024-09-21T11:40:06.353239+0000","last_became_peered":"2024-09-21T11:40:06.353239+0000","last_unstale":"2024-09-21T11:55:01.434194+0000","last_undegraded":"2024-09-21T11:55:01.434194+0000","last_fullsized":"2024-09-21T11:55:01.434194+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:01.434123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.434123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:28.376247+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":358,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673632+0000","last_change":"2024-09-21T11:54:38.447060+0000","last_active":"2024-09-21T11:54:54.673632+0000","last_peered":"2024-09-21T11:54:54.673632+0000","last_clean":"2024-09-21T11:54:54.673632+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:54:54.673632+0000","last_undegraded":"2024-09-21T11:54:54.673632+0000","last_fullsized":"2024-09-21T11:54:54.673632+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:38.446684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.446684+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:45.619387+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":645,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.673740+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:54:54.673740+0000","last_peered":"2024-09-21T11:54:54.673740+0000","last_clean":"2024-09-21T11:54:54.673740+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:54:54.673740+0000","last_undegraded":"2024-09-21T11:54:54.673740+0000","last_fullsized":"2024-09-21T11:54:54.673740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:13.281117+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":652,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688153+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:54:54.688153+0000","last_peered":"2024-09-21T11:54:54.688153+0000","last_clean":"2024-09-21T11:54:54.688153+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:54:54.688153+0000","last_undegraded":"2024-09-21T11:54:54.688153+0000","last_fullsized":"2024-09-21T11:54:54.688153+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:15.498922+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":672,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.676398+0000","last_change":"2024-09-21T11:54:31.029959+0000","last_active":"2024-09-21T11:54:54.676398+0000","last_peered":"2024-09-21T11:54:54.676398+0000","last_clean":"2024-09-21T11:54:54.676398+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:54:54.676398+0000","last_undegraded":"2024-09-21T11:54:54.676398+0000","last_fullsized":"2024-09-21T11:54:54.676398+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:54:31.029780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:31.029780+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:49.683352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":681,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688239+0000","last_change":"2024-09-21T11:53:32.308905+0000","last_active":"2024-09-21T11:54:54.688239+0000","last_peered":"2024-09-21T11:54:54.688239+0000","last_clean":"2024-09-21T11:54:54.688239+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:54:54.688239+0000","last_undegraded":"2024-09-21T11:54:54.688239+0000","last_fullsized":"2024-09-21T11:54:54.688239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:32.308827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:32.308827+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":667,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.689609+0000","last_change":"2024-09-21T11:53:28.945615+0000","last_active":"2024-09-21T11:54:54.689609+0000","last_peered":"2024-09-21T11:54:54.689609+0000","last_clean":"2024-09-21T11:54:54.689609+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:54:54.689609+0000","last_undegraded":"2024-09-21T11:54:54.689609+0000","last_fullsized":"2024-09-21T11:54:54.689609+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:28.945564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:28.945564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:46.973609+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":629,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.672496+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:54:54.672496+0000","last_peered":"2024-09-21T11:54:54.672496+0000","last_clean":"2024-09-21T11:54:54.672496+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:54:54.672496+0000","last_undegraded":"2024-09-21T11:54:54.672496+0000","last_fullsized":"2024-09-21T11:54:54.672496+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:10.309619+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":642,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.686845+0000","last_change":"2024-09-21T11:54:25.551920+0000","last_active":"2024-09-21T11:54:54.686845+0000","last_peered":"2024-09-21T11:54:54.686845+0000","last_clean":"2024-09-21T11:54:54.686845+0000","last_became_active":"2024-09-21T11:40:06.487829+0000","last_became_peered":"2024-09-21T11:40:06.487829+0000","last_unstale":"2024-09-21T11:54:54.686845+0000","last_undegraded":"2024-09-21T11:54:54.686845+0000","last_fullsized":"2024-09-21T11:54:54.686845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.551852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.551852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:43.602512+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":693,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.687954+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:54:54.687954+0000","last_peered":"2024-09-21T11:54:54.687954+0000","last_clean":"2024-09-21T11:54:54.687954+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:54:54.687954+0000","last_undegraded":"2024-09-21T11:54:54.687954+0000","last_fullsized":"2024-09-21T11:54:54.687954+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:00.664484+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":637,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.679122+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:54:54.679122+0000","last_peered":"2024-09-21T11:54:54.679122+0000","last_clean":"2024-09-21T11:54:54.679122+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:54:54.679122+0000","last_undegraded":"2024-09-21T11:54:54.679122+0000","last_fullsized":"2024-09-21T11:54:54.679122+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:49.212310+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":637,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.678355+0000","last_change":"2024-09-21T11:50:38.405183+0000","last_active":"2024-09-21T11:54:54.678355+0000","last_peered":"2024-09-21T11:54:54.678355+0000","last_clean":"2024-09-21T11:54:54.678355+0000","last_became_active":"2024-09-21T11:40:04.332156+0000","last_became_peered":"2024-09-21T11:40:04.332156+0000","last_unstale":"2024-09-21T11:54:54.678355+0000","last_undegraded":"2024-09-21T11:54:54.678355+0000","last_fullsized":"2024-09-21T11:54:54.678355+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:38.405118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.405118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:17.298725+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":649,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:57.381670+0000","last_change":"2024-09-21T11:54:57.381670+0000","last_active":"2024-09-21T11:54:57.381670+0000","last_peered":"2024-09-21T11:54:57.381670+0000","last_clean":"2024-09-21T11:54:57.381670+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:54:57.381670+0000","last_undegraded":"2024-09-21T11:54:57.381670+0000","last_fullsized":"2024-09-21T11:54:57.381670+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:57.381605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:57.381605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:21.092123+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":347,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.671878+0000","last_change":"2024-09-21T11:52:12.348687+0000","last_active":"2024-09-21T11:54:54.671878+0000","last_peered":"2024-09-21T11:54:54.671878+0000","last_clean":"2024-09-21T11:54:54.671878+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:54:54.671878+0000","last_undegraded":"2024-09-21T11:54:54.671878+0000","last_fullsized":"2024-09-21T11:54:54.671878+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:18.324575+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":668,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.688123+0000","last_change":"2024-09-21T11:54:23.601389+0000","last_active":"2024-09-21T11:54:54.688123+0000","last_peered":"2024-09-21T11:54:54.688123+0000","last_clean":"2024-09-21T11:54:54.688123+0000","last_became_active":"2024-09-21T11:47:03.373184+0000","last_became_peered":"2024-09-21T11:47:03.373184+0000","last_unstale":"2024-09-21T11:54:54.688123+0000","last_undegraded":"2024-09-21T11:54:54.688123+0000","last_fullsized":"2024-09-21T11:54:54.688123+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:23.601318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.601318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:26.191120+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":743,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:55.510472+0000","last_change":"2024-09-21T11:54:55.510472+0000","last_active":"2024-09-21T11:54:55.510472+0000","last_peered":"2024-09-21T11:54:55.510472+0000","last_clean":"2024-09-21T11:54:55.510472+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:54:55.510472+0000","last_undegraded":"2024-09-21T11:54:55.510472+0000","last_fullsized":"2024-09-21T11:54:55.510472+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:54:55.509769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:55.509769+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:58.331474+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":646,"reported_epoch":357,"state":"active+clean","last_fresh":"2024-09-21T11:54:54.680496+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:54:54.680496+0000","last_peered":"2024-09-21T11:54:54.680496+0000","last_clean":"2024-09-21T11:54:54.680496+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:54:54.680496+0000","last_undegraded":"2024-09-21T11:54:54.680496+0000","last_fullsized":"2024-09-21T11:54:54.680496+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:43.127854+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":5216796696,"num_objects":79603,"num_object_clones":0,"num_object_copies":238809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79603,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165895,"num_write_kb":7855873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15434067968,"data_stored":15434055752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93595,"ondisk_log_size":93595,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1113,"num_read_kb":870,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":5034,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":620,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":891,"num_read_kb":886,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234169,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1893132,"kb_used_data":1302460,"kb_used_omap":1,"kb_used_meta":590654,"kb_avail":91852020,"statfs":{"total":95995035648,"available":94056468480,"internally_reserved":0,"allocated":1333719040,"data_stored":1333311530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":604830154},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463102,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2747524,"kb_used_data":2260980,"kb_used_omap":1,"kb_used_meta":486526,"kb_avail":90997628,"statfs":{"total":95995035648,"available":93181571072,"internally_reserved":0,"allocated":2315243520,"data_stored":2314844570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":498203078},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724739,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2158412,"kb_used_data":1603836,"kb_used_omap":1,"kb_used_meta":554558,"kb_avail":91586740,"statfs":{"total":95995035648,"available":93784821760,"internally_reserved":0,"allocated":1642328064,"data_stored":1641919362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":567867846},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920970,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2884144,"kb_used_data":2393436,"kb_used_omap":1,"kb_used_meta":490686,"kb_avail":90861008,"statfs":{"total":95995035648,"available":93041672192,"internally_reserved":0,"allocated":2450878464,"data_stored":2450505902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":502462917},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117199,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2186436,"kb_used_data":1577588,"kb_used_omap":1,"kb_used_meta":608702,"kb_avail":91558716,"statfs":{"total":95995035648,"available":93756125184,"internally_reserved":0,"allocated":1615450112,"data_stored":1615049588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":623311303},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411541,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2531848,"kb_used_data":1936952,"kb_used_omap":1,"kb_used_meta":594814,"kb_avail":91213304,"statfs":{"total":95995035648,"available":93402423296,"internally_reserved":0,"allocated":1983438848,"data_stored":1983034939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":609089989},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1550824,"kb_used_data":964312,"kb_used_omap":1,"kb_used_meta":586494,"kb_avail":92194328,"statfs":{"total":95995035648,"available":94406991872,"internally_reserved":0,"allocated":987455488,"data_stored":987019361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":600570310},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738594,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3708704,"kb_used_data":3137424,"kb_used_omap":1,"kb_used_meta":571262,"kb_avail":90036448,"statfs":{"total":95995035648,"available":92197322752,"internally_reserved":0,"allocated":3212722176,"data_stored":3212295328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":584972741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3207270400,"data_stored":3207266328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":974651392,"data_stored":974651392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1960312832,"data_stored":1960312832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1614020608,"data_stored":1614020608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2441744384,"data_stored":2441740312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1634205696,"data_stored":1634205696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2287603712,"data_stored":2287599640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1314258944,"data_stored":1314258944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:55:09.438 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-21T11:55:09.438 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:55:09.439 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-21T11:55:09.439 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T11:55:09.439 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T11:55:09.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-21T11:55:09.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:09.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:09.584 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:09.584 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:09.629 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:09.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:09.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:09.755 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:09.755 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:09.807 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:55:09.807 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:55:09.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:09.889 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:09.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:09.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:09.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:10.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:09 smithi175 ceph-mon[32613]: 5.f scrub starts 2024-09-21T11:55:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:09 smithi175 ceph-mon[32613]: 5.f scrub ok 2024-09-21T11:55:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:09 smithi175 ceph-mon[32613]: 4.3 scrub starts 2024-09-21T11:55:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:09 smithi175 ceph-mon[32613]: 4.3 scrub ok 2024-09-21T11:55:10.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:10.028 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:10.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:10.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:09 smithi043 ceph-mon[29710]: 5.f scrub starts 2024-09-21T11:55:10.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:09 smithi043 ceph-mon[29710]: 5.f scrub ok 2024-09-21T11:55:10.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:09 smithi043 ceph-mon[29710]: 4.3 scrub starts 2024-09-21T11:55:10.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:09 smithi043 ceph-mon[29710]: 4.3 scrub ok 2024-09-21T11:55:10.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:09 smithi043 ceph-mon[36454]: 5.f scrub starts 2024-09-21T11:55:10.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:09 smithi043 ceph-mon[36454]: 5.f scrub ok 2024-09-21T11:55:10.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:09 smithi043 ceph-mon[36454]: 4.3 scrub starts 2024-09-21T11:55:10.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:09 smithi043 ceph-mon[36454]: 4.3 scrub ok 2024-09-21T11:55:10.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:10.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:10.170 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:10.271 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:10.302 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:10.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:10.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:10.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:10.442 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:10.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:10.443 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:10.544 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:10.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:10.578 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:10.578 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:10.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:10.720 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:10.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:10.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:10.756 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:55:10.781 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:55:10.821 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:10.869 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:10.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:10.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:10.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:10 smithi043 ceph-mon[29710]: pgmap v729: 148 pgs: 148 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 777 op/s 2024-09-21T11:55:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2052913271' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:10 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:10 smithi043 ceph-mon[36454]: pgmap v729: 148 pgs: 148 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 777 op/s 2024-09-21T11:55:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2052913271' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:10 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:10 smithi175 ceph-mon[32613]: pgmap v729: 148 pgs: 148 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 777 op/s 2024-09-21T11:55:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2052913271' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:10 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:11.023 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:11.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:11.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:11.040 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:11.076 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:11.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:11.175 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:11.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:11.176 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:11.195 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:11.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:11.309 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:11.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:11.310 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:11.411 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:11.451 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:11.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:11.452 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:11.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:11.592 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:11.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:11.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:11.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:11.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:11.734 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:11.734 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:11.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:11.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:11.873 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:11.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:11.918 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[29710]: 4.6 scrub starts 2024-09-21T11:55:11.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[29710]: 4.6 scrub ok 2024-09-21T11:55:11.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[29710]: 4.1a scrub starts 2024-09-21T11:55:11.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[29710]: 4.1a scrub ok 2024-09-21T11:55:11.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[29710]: 5.c scrub starts 2024-09-21T11:55:11.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[29710]: 5.c scrub ok 2024-09-21T11:55:11.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:11.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:55:11.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[29710]: osdmap e360: 8 total, 8 up, 8 in 2024-09-21T11:55:11.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:11.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[36454]: 4.6 scrub starts 2024-09-21T11:55:11.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[36454]: 4.6 scrub ok 2024-09-21T11:55:11.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[36454]: 4.1a scrub starts 2024-09-21T11:55:11.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[36454]: 4.1a scrub ok 2024-09-21T11:55:11.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[36454]: 5.c scrub starts 2024-09-21T11:55:11.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[36454]: 5.c scrub ok 2024-09-21T11:55:11.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:11.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:55:11.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[36454]: osdmap e360: 8 total, 8 up, 8 in 2024-09-21T11:55:11.920 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:11 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:11.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:11.979 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-mon[32613]: 4.6 scrub starts 2024-09-21T11:55:11.979 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-mon[32613]: 4.6 scrub ok 2024-09-21T11:55:11.979 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-mon[32613]: 4.1a scrub starts 2024-09-21T11:55:11.979 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-mon[32613]: 4.1a scrub ok 2024-09-21T11:55:11.979 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-mon[32613]: 5.c scrub starts 2024-09-21T11:55:11.979 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-mon[32613]: 5.c scrub ok 2024-09-21T11:55:11.979 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:11.979 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:55:11.979 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-mon[32613]: osdmap e360: 8 total, 8 up, 8 in 2024-09-21T11:55:11.979 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:11.980 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:55:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:55:12.011 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:12.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:12.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:12.113 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:12.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:12.147 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:12.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:12.236 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:55:12.237 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T11:55:12.237 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:55:12.248 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:12.285 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:12.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:12.286 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:12.312 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:55:12.312 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:55:12.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:12.418 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:12.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:12.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:12.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:12.554 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:12.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:12.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:12.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:12.697 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:12.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:12.698 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:12.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:12.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:12.845 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:12.845 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:12.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:12.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:12.982 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:12.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:13.083 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:13.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:55:13.138 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:13.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:13.139 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:13.240 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:13.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:13.280 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:13.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:13.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:13.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:13.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:13.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:13.565 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:13.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:13.566 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:13.567 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:13.647 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:55:13.647 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:55:13.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:13 smithi043 ceph-mon[36454]: pgmap v731: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-21T11:55:13.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:13 smithi043 ceph-mon[29710]: pgmap v731: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-21T11:55:13.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:55:13.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:13.678 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:55:13.678 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:55:13.706 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:13 smithi175 ceph-mon[32613]: pgmap v731: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-21T11:55:13.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:13.715 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:13.715 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:13.717 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:55:13.717 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:55:13.753 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:55:13.753 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:55:13.797 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:55:13.798 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:55:13.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:13.833 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:55:13.833 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:55:13.867 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:55:13.867 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:55:13.875 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:13.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:13.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:13.901 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:55:13.901 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:55:13.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:14.015 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:14.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:14.017 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:14.118 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:14.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:14.160 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:14.160 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:14.261 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:14.294 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:14.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:14.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:14.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:14.429 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:14.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:14.430 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:14.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3552753817' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:14.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3552753817' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:14.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:14.568 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:14.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:14.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:14.575 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:55:14.575 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:55:14.575 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:55:14.575 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:55:14.575 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.575 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43899 8.2 8:43c05007:::benchmark_data_smithi043_97042_object43898:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:55:14.575 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.986744+0000", 2024-09-21T11:55:14.575 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 593.25015577700003, 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.132095406, 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43899 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.576 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986744+0000", 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986744+0000", 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986745+0000", 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.632e-06 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.577 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986800+0000", 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4793999999999997e-05 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986816+0000", 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5444e-05 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986824+0000", 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5799999999999992e-06 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.578 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.986889+0000", 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4658000000000003e-05 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987030+0000", 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014126500000000001 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987181+0000", 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015071799999999999 2024-09-21T11:55:14.579 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989750+0000", 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025688159999999998 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.991302+0000", 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0015523080000000001 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118726+0000", 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12742430799999999 2024-09-21T11:55:14.580 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118787+0000", 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1141000000000003e-05 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.118839+0000", 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1742e-05 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.581 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:43905 8.2 8:49886794:::benchmark_data_smithi043_97042_object43904:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e172)", 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:20.987297+0000", 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 593.24960214600003, 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13219620100000001, 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 43905 2024-09-21T11:55:14.582 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987297+0000", 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987297+0000", 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.583 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987298+0000", 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9999999999999996e-07 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987377+0000", 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8947000000000002e-05 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.584 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987387+0000", 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4960000000000002e-06 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.987394+0000", 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3159999999999999e-06 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.988450+0000", 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001055802 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.585 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.988542+0000", 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1652999999999998e-05 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.988627+0000", 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5554000000000001e-05 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.989839+0000", 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001212288 2024-09-21T11:55:14.586 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:20.991597+0000", 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001757303 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.119404+0000", 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12780698200000001 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.119460+0000", 2024-09-21T11:55:14.587 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.622e-05 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:21.119494+0000", 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3739999999999999e-05 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46235 8.2 8:4e6f27b7:::benchmark_data_smithi043_97042_object46234:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.589574+0000", 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 591.647325927, 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.126942478, 2024-09-21T11:55:14.588 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46235 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589574+0000", 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.589 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589574+0000", 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589585+0000", 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0998999999999999e-05 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589616+0000", 2024-09-21T11:55:14.590 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1123e-05 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589627+0000", 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1486e-05 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589634+0000", 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0740000000000004e-06 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.591 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589713+0000", 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8753999999999998e-05 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.589860+0000", 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014660000000000001 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590003+0000", 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000143516 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.592 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596149+0000", 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0061461579999999997 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596193+0000", 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3176000000000001e-05 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716435+0000", 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.120242746 2024-09-21T11:55:14.593 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716472+0000", 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6868e-05 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716516+0000", 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3977999999999998e-05 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.594 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:46237 8.2 8:4ddff5f6:::benchmark_data_smithi043_97042_object46236:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:22.590497+0000", 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 591.64640252100003, 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.126123824, 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 46237 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.595 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590497+0000", 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590497+0000", 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590498+0000", 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0729999999999999e-06 2024-09-21T11:55:14.596 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590524+0000", 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6284000000000001e-05 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590535+0000", 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0913999999999999e-05 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.590545+0000", 2024-09-21T11:55:14.597 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0054999999999999e-05 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.591661+0000", 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001115324 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.591765+0000", 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104269 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.591884+0000", 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000118842 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596154+0000", 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0042705729999999997 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.596233+0000", 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8449999999999996e-05 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.599 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716553+0000", 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.12031977000000001 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716582+0000", 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9628000000000002e-05 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:22.716621+0000", 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8642e-05 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.600 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:47783 8.2 8:4b213eb9:::benchmark_data_smithi043_97042_object47782:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e173)", 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:23.709555+0000", 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 590.52734470600001, 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.135854843, 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47783 2024-09-21T11:55:14.601 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709555+0000", 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709555+0000", 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.602 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709556+0000", 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4160000000000001e-06 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709667+0000", 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110912 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709676+0000", 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6349999999999995e-06 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.603 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.709682+0000", 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8649999999999998e-06 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.710257+0000", 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057514299999999997 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.710363+0000", 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010655200000000001 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.604 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.710488+0000", 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124943 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712767+0000", 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022785959999999999 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.712779+0000", 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2417e-05 2024-09-21T11:55:14.605 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845284+0000", 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.132504967 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845364+0000", 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9629000000000006e-05 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.606 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:23.845410+0000", 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5768e-05 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.15150.0:66128 8.2 8:4c9d601c:::benchmark_data_smithi043_97042_object66127:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:45:35.918161+0000", 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 578.31873827300001, 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.13079232700000001, 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.607 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.15150", 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 66128 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918161+0000", 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918161+0000", 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.608 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918162+0000", 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0610000000000001e-06 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918211+0000", 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9125000000000001e-05 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.609 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918221+0000", 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.183e-06 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.918227+0000", 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5470000000000002e-06 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919404+0000", 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011771520000000001 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.610 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919510+0000", 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106103 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.919632+0000", 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012106400000000001 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.921625+0000", 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019929919999999999 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.611 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:35.923993+0000", 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0023686279999999998 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048774+0000", 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.124780935 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048871+0000", 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6514000000000005e-05 2024-09-21T11:55:14.612 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:45:36.048954+0000", 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3022999999999994e-05 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67179 10.d e330/313 10:bb0d89d0:::benchmark_data_smithi043_150678_object67178:head v 330'4110, mlcod=330'4108)", 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.332772+0000", 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 105.904127909, 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.20463724599999999, 2024-09-21T11:55:14.613 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.332772+0000", 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.332772+0000", 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.332773+0000", 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9699999999999994e-07 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.332817+0000", 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4765999999999998e-05 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.332833+0000", 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5305999999999999e-05 2024-09-21T11:55:14.615 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.332840+0000", 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8680000000000001e-06 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.332901+0000", 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1367999999999999e-05 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.332920+0000", 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9272000000000001e-05 2024-09-21T11:55:14.616 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.537366+0000", 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.20444574200000001 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.537409+0000", 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2926999999999997e-05 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.617 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67184 10.1 e330/313 10:888d6a11:::benchmark_data_smithi043_150678_object67183:head v 330'4255, mlcod=330'4253)", 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.335234+0000", 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 105.90166557800001, 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.20628911999999999, 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.335234+0000", 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.618 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.335234+0000", 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.335235+0000", 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4300000000000001e-07 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.335255+0000", 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0191999999999999e-05 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.619 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.335265+0000", 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9170000000000001e-06 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.537249+0000", 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.20198387000000001 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.537300+0000", 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1504e-05 2024-09-21T11:55:14.620 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.537317+0000", 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6314e-05 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.541479+0000", 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.004162481 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.621 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.541523+0000", 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3899000000000001e-05 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67189 10.6 e330/313 10:65d25444:::benchmark_data_smithi043_150678_object67188:head v 330'4105, mlcod=330'4103)", 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.338976+0000", 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 105.89792385600001, 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.20983970399999999, 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.622 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338976+0000", 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338976+0000", 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338977+0000", 2024-09-21T11:55:14.623 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4330000000000001e-06 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.339012+0000", 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4919999999999998e-05 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.536925+0000", 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19791267200000001 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.624 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.536943+0000", 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7989e-05 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.537168+0000", 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022506300000000001 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.537192+0000", 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3830999999999999e-05 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.625 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.548596+0000", 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.011404354 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.548815+0000", 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000219442 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:72203 10.0 e332/313 10:0f581edb:::benchmark_data_smithi043_150678_object72202:head v 332'4553, mlcod=332'4550)", 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.965097+0000", 2024-09-21T11:55:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 101.271802343, 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23741131600000001, 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.965097+0000", 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.965097+0000", 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.627 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.965098+0000", 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.16e-07 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.965122+0000", 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3893e-05 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.965132+0000", 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9059999999999997e-06 2024-09-21T11:55:14.628 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.965139+0000", 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3180000000000001e-06 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.965211+0000", 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2103999999999999e-05 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.965230+0000", 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8936999999999998e-05 2024-09-21T11:55:14.629 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968072+0000", 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028422199999999999 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202509+0000", 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.234436122 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.630 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:72204 10.2 e332/313 10:40424d9c:::benchmark_data_smithi043_150678_object72203:head v 332'4538, mlcod=332'4535)", 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.967096+0000", 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 101.26980402300001, 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.238745981, 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967096+0000", 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.631 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967096+0000", 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967097+0000", 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2419999999999999e-06 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967137+0000", 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0500999999999997e-05 2024-09-21T11:55:14.632 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967149+0000", 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1425e-05 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967157+0000", 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9200000000000004e-06 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967210+0000", 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2982000000000002e-05 2024-09-21T11:55:14.633 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967227+0000", 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7847e-05 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.205773+0000", 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23854528899999999 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.634 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.205842+0000", 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8775000000000005e-05 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:72207 10.d e332/313 10:b16de966:::benchmark_data_smithi043_150678_object72206:head v 332'4410, mlcod=332'4407)", 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.968002+0000", 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 101.268897598, 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24189200199999999, 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.635 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968002+0000", 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968002+0000", 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968003+0000", 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6899999999999997e-07 2024-09-21T11:55:14.636 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968026+0000", 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3283999999999999e-05 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968033+0000", 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2919999999999998e-06 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968039+0000", 2024-09-21T11:55:14.637 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2040000000000002e-06 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202497+0000", 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23445785699999999 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202522+0000", 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5185e-05 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.638 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.209835+0000", 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0073125050000000004 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.209894+0000", 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9005999999999997e-05 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:72208 10.c e332/313 10:34d133c3:::benchmark_data_smithi043_150678_object72207:head v 332'4425, mlcod=332'4422)", 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.968750+0000", 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 101.268149982, 2024-09-21T11:55:14.639 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24312957700000001, 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968750+0000", 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968750+0000", 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.640 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968750+0000", 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4700000000000001e-07 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968772+0000", 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2141e-05 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202512+0000", 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23373949599999999 2024-09-21T11:55:14.641 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202524+0000", 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1568e-05 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202588+0000", 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4232000000000004e-05 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202609+0000", 2024-09-21T11:55:14.642 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0764e-05 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.211827+0000", 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0092183019999999994 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.211879+0000", 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2327000000000002e-05 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.643 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:72213 10.6 e332/313 10:6e0a855e:::benchmark_data_smithi043_150678_object72212:head v 332'4430, mlcod=332'4428)", 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.970359+0000", 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 101.26654056, 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.235848903, 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970359+0000", 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.644 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970359+0000", 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970360+0000", 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.85e-07 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970382+0000", 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2297999999999999e-05 2024-09-21T11:55:14.645 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202560+0000", 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23217784399999999 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202570+0000", 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0406e-05 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.646 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.203010+0000", 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00043942999999999999 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.203031+0000", 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1043000000000002e-05 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206107+0000", 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003075618 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.647 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206208+0000", 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010137900000000001 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:97349 10.1 e339/313 10:874816ec:::benchmark_data_smithi043_150678_object97348:head v 339'6071, mlcod=339'6067)", 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:56.999453+0000", 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 77.237446392999999, 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15410347899999999, 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.648 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.999453+0000", 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.999453+0000", 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.999454+0000", 2024-09-21T11:55:14.649 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1599999999999999e-06 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.999480+0000", 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6109e-05 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.999492+0000", 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1498000000000001e-05 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.650 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.999503+0000", 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1518999999999999e-05 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.000615+0000", 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011110689999999999 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.000635+0000", 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0516000000000001e-05 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.651 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153524+0000", 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15288933800000001 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153557+0000", 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2270000000000001e-05 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:97352 10.c e339/313 10:31cbdf0b:::benchmark_data_smithi043_150678_object97351:head v 339'6078, mlcod=339'6075)", 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:57.001726+0000", 2024-09-21T11:55:14.652 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 77.235173336000003, 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.151778997, 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.001726+0000", 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.001726+0000", 2024-09-21T11:55:14.653 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.001727+0000", 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3900000000000004e-07 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.001769+0000", 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2441000000000002e-05 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.654 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.001779+0000", 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3530000000000001e-06 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.001786+0000", 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1690000000000003e-06 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.001851+0000", 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5276000000000003e-05 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.655 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.001870+0000", 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8669e-05 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.005980+0000", 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0041100440000000002 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153505+0000", 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14752530599999999 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.656 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:97357 10.6 e339/313 10:6512073b:::benchmark_data_smithi043_150678_object97356:head v 339'6017, mlcod=339'6015)", 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:57.004273+0000", 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 77.232626459000002, 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15261097500000001, 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.657 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.004273+0000", 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.004273+0000", 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.004274+0000", 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6700000000000002e-07 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.658 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.004300+0000", 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6112999999999999e-05 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.004311+0000", 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0907000000000001e-05 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.004318+0000", 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8709999999999999e-06 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.659 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.004402+0000", 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3641999999999996e-05 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.004421+0000", 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9337000000000001e-05 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.156726+0000", 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15230460400000001 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.660 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.156884+0000", 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000158534 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:97354 10.e e339/313 10:723b16cb:::benchmark_data_smithi043_150678_object97353:head v 339'6110, mlcod=339'6108)", 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:57.005824+0000", 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 77.231075281000003, 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15797861399999999, 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.661 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.005824+0000", 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.005824+0000", 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.005825+0000", 2024-09-21T11:55:14.662 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.063e-06 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.005846+0000", 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0899000000000001e-05 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.005856+0000", 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0022e-05 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.663 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.005863+0000", 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9360000000000002e-06 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153531+0000", 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14766821399999999 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153555+0000", 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3842e-05 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.664 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.163762+0000", 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01020688 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.163803+0000", 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0757999999999998e-05 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:97360 10.0 e339/313 10:0fbeaa83:::benchmark_data_smithi043_150678_object97359:head v 339'6183, mlcod=339'6181)", 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:57.007086+0000", 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 77.229813903999997, 2024-09-21T11:55:14.665 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15010037900000001, 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.007086+0000", 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.007086+0000", 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.666 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.007086+0000", 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9800000000000003e-07 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.007109+0000", 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2750000000000001e-05 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153491+0000", 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14638209299999999 2024-09-21T11:55:14.667 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153504+0000", 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2258e-05 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153571+0000", 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7263999999999998e-05 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153591+0000", 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9966000000000002e-05 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.668 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.157012+0000", 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0034209819999999999 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.157186+0000", 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017426799999999999 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:97363 10.6 e339/313 10:6d716e9e:::benchmark_data_smithi043_150678_object97362:head v 339'6018, mlcod=339'6015)", 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:57.009331+0000", 2024-09-21T11:55:14.669 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 77.227568266999995, 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.157862797, 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.009331+0000", 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.009331+0000", 2024-09-21T11:55:14.670 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.009332+0000", 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9999999999999996e-07 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.009352+0000", 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0220999999999998e-05 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153495+0000", 2024-09-21T11:55:14.671 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.14414308100000001 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153503+0000", 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8089999999999993e-06 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153542+0000", 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9218999999999998e-05 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.153558+0000", 2024-09-21T11:55:14.672 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5662e-05 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.166454+0000", 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012896090000000001 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.167194+0000", 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00073991499999999995 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:14.673 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:55:14.674 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:14.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:14.708 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:14.708 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3552753817' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:14.809 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:14.832 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:14.839 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:14.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:14.841 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:14.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:14.980 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:14.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:14.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:15.083 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:15.119 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:15.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:15.120 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:15.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:15.247 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:15.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:15.248 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:15.280 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:55:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:55:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:15.349 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:15.388 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:15.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:15.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:15.490 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:15.519 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:15.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:15.520 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:15.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:15.662 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:15.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:15.663 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:15.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:15 smithi043 ceph-mon[36454]: pgmap v732: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-21T11:55:15.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:15 smithi043 ceph-mon[29710]: pgmap v732: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-21T11:55:15.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:15 smithi175 ceph-mon[32613]: pgmap v732: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.32k op/s 2024-09-21T11:55:15.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:15.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:15.794 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:15.794 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:15.895 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:15.933 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:15.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:15.934 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:16.035 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:16.078 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:16.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:16.079 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:16.180 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:16.211 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:16.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:16.212 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:16.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:16.353 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:16.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:16.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:16.447 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:55:16.454 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:16.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:16.484 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:16.484 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:16.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:16.628 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:16.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:16.629 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:16.730 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:16 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:55:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:16 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:55:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:16 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/496172080' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:16 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:55:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:16 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:55:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:16 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/496172080' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:16.759 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:16.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:16.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:16.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:16 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:55:16.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:16 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:55:16.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:16 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/496172080' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:16.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:55:16.823 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:55:16.860 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:16.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:16.909 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:16.909 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:17.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:17.046 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:17.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:17.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:17.148 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:17.202 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:17.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:17.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:17.287 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:17.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:17.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:17.340 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:17.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:17.476 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:17.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:17.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:17.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:17.608 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:17.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:17.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:17.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:17.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:17.738 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:17.738 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:17.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:17 smithi043 ceph-mon[36454]: pgmap v733: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-21T11:55:17.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:17 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:17.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:17 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/496172080' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:17.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:17 smithi043 ceph-mon[36454]: osdmap e361: 8 total, 8 up, 8 in 2024-09-21T11:55:17.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:17 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:17.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:17 smithi043 ceph-mon[29710]: pgmap v733: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-21T11:55:17.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:17 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:17.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/496172080' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:17.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:17 smithi043 ceph-mon[29710]: osdmap e361: 8 total, 8 up, 8 in 2024-09-21T11:55:17.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:17 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:17.753 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:17 smithi175 ceph-mon[32613]: pgmap v733: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-21T11:55:17.753 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:17 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:17.753 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:17 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/496172080' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:17.753 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:17 smithi175 ceph-mon[32613]: osdmap e361: 8 total, 8 up, 8 in 2024-09-21T11:55:17.753 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:17 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:17.839 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:17.873 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:17.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:17.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:183998 10.a e361/313 10:5ef26de0:::benchmark_data_smithi043_150678_object61247:head v 361'11406, mlcod=361'11402)", 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:55:17.915311+0000", 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0052056130000000004, 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0052331010000000004, 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:17.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915311+0000", 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915311+0000", 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915312+0000", 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.879999999999999e-07 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915318+0000", 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3720000000000001e-06 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915330+0000", 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1776e-05 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915338+0000", 2024-09-21T11:55:17.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1270000000000003e-06 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915462+0000", 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012370600000000001 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915480+0000", 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8411999999999999e-05 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:183995 10.3 e361/313 10:cd303718:::benchmark_data_smithi043_150678_object61244:head v 361'11385, mlcod=361'11381)", 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:55:17.915400+0000", 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0051161430000000001, 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0052050220000000001, 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915400+0000", 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915400+0000", 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915401+0000", 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4200000000000001e-07 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915406+0000", 2024-09-21T11:55:17.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.074e-06 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915416+0000", 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0461e-05 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915422+0000", 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3029999999999999e-06 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:17.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915628+0000", 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00020594499999999999 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:184002 10.7 e361/313 10:e60b02cb:::benchmark_data_smithi043_150678_object61251:head v 361'11584, mlcod=361'11580)", 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:55:17.915663+0000", 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0048533889999999996, 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0049897539999999999, 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:55:17.931 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915663+0000", 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915663+0000", 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915664+0000", 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6699999999999999e-07 2024-09-21T11:55:17.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915667+0000", 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9269999999999998e-06 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915675+0000", 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7800000000000001e-06 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915680+0000", 2024-09-21T11:55:17.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1780000000000002e-06 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915834+0000", 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015347000000000001 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.915876+0000", 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1857999999999998e-05 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:17.934 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:183990 10.7 e361/313 10:ea54498b:::benchmark_data_smithi043_150678_object61239:head v 361'11583, mlcod=361'11578)", 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:55:17.910554+0000", 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0099624199999999996, 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010149981000000001, 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910554+0000", 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910554+0000", 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910555+0000", 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2099999999999996e-07 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910559+0000", 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8029999999999996e-06 2024-09-21T11:55:17.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910569+0000", 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.149e-06 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910576+0000", 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9070000000000001e-06 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910678+0000", 2024-09-21T11:55:17.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102859 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910697+0000", 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8719000000000002e-05 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.918781+0000", 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0080834640000000003 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:183986 10.a e361/313 10:56c2fa7a:::benchmark_data_smithi043_150678_object61235:head v 361'11405, mlcod=361'11401)", 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:55:17.910593+0000", 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0099233089999999999, 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.01016765, 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910593+0000", 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:17.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910593+0000", 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910593+0000", 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.0699999999999998e-07 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910597+0000", 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7950000000000001e-06 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:17.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910603+0000", 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9959999999999999e-06 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.910609+0000", 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4829999999999999e-06 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.911127+0000", 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00051789800000000003 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.911145+0000", 2024-09-21T11:55:17.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8782000000000001e-05 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.918906+0000", 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0077608249999999998 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:183992 10.3 e361/313 10:c46adbd4:::benchmark_data_smithi043_150678_object61241:head v 361'11384, mlcod=361'11381)", 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:55:17.911250+0000", 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0092666659999999998, 2024-09-21T11:55:17.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0095671090000000007, 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.911250+0000", 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.911250+0000", 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.911250+0000", 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8599999999999998e-07 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.911256+0000", 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8730000000000004e-06 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:17.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.911268+0000", 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2014999999999999e-05 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.911275+0000", 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.674e-06 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.911436+0000", 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016076999999999999 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:17.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:17.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:55:17.911486+0000", 2024-09-21T11:55:17.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9984000000000001e-05 2024-09-21T11:55:17.946 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.946 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:17.946 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.946 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:17.946 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:55:17.946 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 6 2024-09-21T11:55:17.946 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:55:17.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:18.019 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:18.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:18.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:18.120 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:18.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:18.166 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:18.167 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:18.267 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:18.296 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:18.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:18.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:18.376 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:55:18.376 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:55:18.376 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-21T11:55:18.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight-by-utilization 1000 0.05 2024-09-21T11:55:18.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:18.466 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:18.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:18.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:18.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:18.597 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:18.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:18.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:18.690 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:18.699 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:18.733 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:18.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:18.734 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:18.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:18.880 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:18.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:18.881 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:18.910 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:55:18.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:19.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:19.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:19.133 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:19.164 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:19.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:19.165 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:19.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:19.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:19.308 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:19.308 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:19.319 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:19.372 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:55:19.372 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:55:19.376 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:19.409 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:19.409 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:55:19.410 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:55:19.445 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:19.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:19.446 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:19.449 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:55:19.449 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:55:19.490 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:55:19.490 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:55:19.522 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:55:19.522 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:55:19.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:19.559 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:55:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:55:19.581 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:19.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:19.582 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:19.594 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:55:19.594 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:55:19.628 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:55:19.629 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:55:19.683 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:19.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:19 smithi043 ceph-mon[36454]: pgmap v735: 148 pgs: 148 active+clean; 3.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-21T11:55:19.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:19 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:55:19.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:19 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:55:19.703 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:19 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/123248598' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:19.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:19 smithi043 ceph-mon[29710]: pgmap v735: 148 pgs: 148 active+clean; 3.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-21T11:55:19.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:19 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:55:19.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:19 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:55:19.703 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:19 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/123248598' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:19.718 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:19.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:19.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:19.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:19 smithi175 ceph-mon[32613]: pgmap v735: 148 pgs: 148 active+clean; 3.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-21T11:55:19.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:19 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:55:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:19 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:55:19.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:19 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/123248598' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:19.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:19.849 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:19.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:19.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:19.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:19.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:19.988 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:20.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:20.134 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:20.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:20.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:20.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:20.271 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:20.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:20.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:20.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:20.376 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:55:20.376 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:55:20.416 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:20.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:20.417 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:20.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:55:20.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:20.602 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:20.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:20.603 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:20.637 INFO:teuthology.orchestra.run.smithi043.stdout:moved 0 / 444 (0%) 2024-09-21T11:55:20.637 INFO:teuthology.orchestra.run.smithi043.stdout:avg 55.5 2024-09-21T11:55:20.638 INFO:teuthology.orchestra.run.smithi043.stdout:stddev 5.76628 -> 5.76628 (expected baseline 6.96868) 2024-09-21T11:55:20.638 INFO:teuthology.orchestra.run.smithi043.stdout:min osd.2 with 45 -> 45 pgs (0.810811 -> 0.810811 * mean) 2024-09-21T11:55:20.638 INFO:teuthology.orchestra.run.smithi043.stdout:max osd.4 with 66 -> 66 pgs (1.18919 -> 1.18919 * mean) 2024-09-21T11:55:20.638 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:55:20.638 INFO:teuthology.orchestra.run.smithi043.stdout:oload 1000 2024-09-21T11:55:20.638 INFO:teuthology.orchestra.run.smithi043.stdout:max_change 0.05 2024-09-21T11:55:20.638 INFO:teuthology.orchestra.run.smithi043.stdout:max_change_osds 4 2024-09-21T11:55:20.638 INFO:teuthology.orchestra.run.smithi043.stdout:average_utilization 0.0222 2024-09-21T11:55:20.638 INFO:teuthology.orchestra.run.smithi043.stdout:overload_utilization 0.2225 2024-09-21T11:55:20.638 INFO:teuthology.orchestra.run.smithi043.stderr:no change 2024-09-21T11:55:20.639 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:20 smithi043 ceph-mon[36454]: osdmap e362: 8 total, 8 up, 8 in 2024-09-21T11:55:20.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:20 smithi043 ceph-mon[29710]: osdmap e362: 8 total, 8 up, 8 in 2024-09-21T11:55:20.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:20.750 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:20.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:20.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:20.755 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:20 smithi175 ceph-mon[32613]: osdmap e362: 8 total, 8 up, 8 in 2024-09-21T11:55:20.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:20.901 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:20.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:20.903 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:21.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:21.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:21.042 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:21.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:21.143 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:21.174 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:21.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:21.175 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:21.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:55:21.276 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:21.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:21.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:21.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:21.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:21 smithi175 ceph-mon[32613]: pgmap v737: 148 pgs: 148 active+clean; 3.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 787 op/s 2024-09-21T11:55:21.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:21 smithi175 ceph-mon[32613]: from='client.26168 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:21.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:21 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:55:21.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:21.508 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:21.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:21.582 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-21T11:55:21.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd test-reweight-by-utilization 1000 3.0 2024-09-21T11:55:21.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:21.682 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:21.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:21.691 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:21.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:21.712 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:21 smithi043 ceph-mon[29710]: pgmap v737: 148 pgs: 148 active+clean; 3.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 787 op/s 2024-09-21T11:55:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:21 smithi043 ceph-mon[29710]: from='client.26168 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:21 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:55:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:21 smithi043 ceph-mon[36454]: pgmap v737: 148 pgs: 148 active+clean; 3.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 787 op/s 2024-09-21T11:55:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:21 smithi043 ceph-mon[36454]: from='client.26168 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:21.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:21 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:55:21.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:55:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:55:21.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:21.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:21.841 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:21.841 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:21.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:21.987 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:21.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:21.989 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:22.089 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:22.128 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:22.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:22.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:22.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:22.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:22.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:22.321 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:55:22.321 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:55:22.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:22.445 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:22.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:22.446 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:22.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:22.594 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:22.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:22.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:22.696 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:22.727 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:22.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:22.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:22.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:55:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:22 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:55:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:22 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:55:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:22 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:55:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:22 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:55:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:22 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:55:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:22 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:55:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:22 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:55:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:22 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:55:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:22 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:55:22.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:22.878 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:22.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:22.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:22.979 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:23.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:23.048 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:23.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:23.163 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:55:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:23.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:23.199 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:23.225 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:23.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:23.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:23.340 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:23.341 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:23.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:23.447 INFO:teuthology.orchestra.run.smithi043.stdout:moved 0 / 444 (0%) 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stdout:avg 55.5 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stdout:stddev 5.76628 -> 5.76628 (expected baseline 6.96868) 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stdout:min osd.2 with 45 -> 45 pgs (0.810811 -> 0.810811 * mean) 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stdout:max osd.4 with 66 -> 66 pgs (1.18919 -> 1.18919 * mean) 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stdout:oload 1000 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stdout:max_change 3 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stdout:max_change_osds 4 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stdout:average_utilization 0.0213 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stdout:overload_utilization 0.2127 2024-09-21T11:55:23.448 INFO:teuthology.orchestra.run.smithi043.stderr:no change 2024-09-21T11:55:23.485 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:23.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:23.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:55:23.587 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:23.593 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:23.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:23.631 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:23.631 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:23.728 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:23 smithi175 ceph-mon[32613]: pgmap v738: 148 pgs: 148 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-21T11:55:23.728 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/335660867' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:23.728 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:23 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:23.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:23 smithi043 ceph-mon[36454]: pgmap v738: 148 pgs: 148 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-21T11:55:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/335660867' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:23.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:23 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:23 smithi043 ceph-mon[29710]: pgmap v738: 148 pgs: 148 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s 2024-09-21T11:55:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/335660867' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:23.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:23 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:23.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:23.767 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:23.767 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:23.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:23.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:23.910 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:24.010 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:24.042 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:24.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:24.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:24.143 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:24.182 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:24.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:24.183 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:24.259 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:55:24.259 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:55:24.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:24.321 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:24.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:24.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:24.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:24.460 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:24.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:24.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:24.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:24.599 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:24.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:24.600 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:24.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:24.742 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:24.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:24.743 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:24.819 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:55:24.819 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:55:24.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:24 smithi043 ceph-mon[36454]: from='client.16359 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:24.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:24 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:24.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:24 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:55:24.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:24 smithi043 ceph-mon[36454]: osdmap e363: 8 total, 8 up, 8 in 2024-09-21T11:55:24.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:24 smithi043 ceph-mon[29710]: from='client.16359 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:24.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:24 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:24.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:24 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:55:24.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:24 smithi043 ceph-mon[29710]: osdmap e363: 8 total, 8 up, 8 in 2024-09-21T11:55:24.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:24.853 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:55:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:55:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:24.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:24.880 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:24.889 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:55:24.889 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:55:24.927 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:55:24.928 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:55:24.937 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:24 smithi175 ceph-mon[32613]: from='client.16359 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:24.937 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:24 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:24.937 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:24 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:55:24.937 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:24 smithi175 ceph-mon[32613]: osdmap e363: 8 total, 8 up, 8 in 2024-09-21T11:55:24.960 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:55:24.960 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:55:24.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:25.003 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:55:25.003 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:55:25.018 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:25.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:25.019 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:25.038 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:55:25.038 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:55:25.069 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:55:25.073 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:55:25.073 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:55:25.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:25.158 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:25.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:25.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:25.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:25.302 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:25.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:25.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:25.348 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:55:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:25.402 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-21T11:55:25.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight-by-utilization 110 1.0 2024-09-21T11:55:25.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:25.444 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:25.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:25.445 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:25.468 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:25.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:25.586 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:25.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:25.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:25.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:25.720 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[29710]: pgmap v740: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 669 B/s rd, 0 B/s wr, 935 op/s 2024-09-21T11:55:25.720 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[29710]: 3.14 scrub starts 2024-09-21T11:55:25.720 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[29710]: 3.14 scrub ok 2024-09-21T11:55:25.721 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/31782664' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:25.721 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[29710]: 5.7 scrub starts 2024-09-21T11:55:25.721 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[29710]: 5.7 scrub ok 2024-09-21T11:55:25.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[36454]: pgmap v740: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 669 B/s rd, 0 B/s wr, 935 op/s 2024-09-21T11:55:25.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[36454]: 3.14 scrub starts 2024-09-21T11:55:25.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[36454]: 3.14 scrub ok 2024-09-21T11:55:25.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/31782664' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:25.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[36454]: 5.7 scrub starts 2024-09-21T11:55:25.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:25 smithi043 ceph-mon[36454]: 5.7 scrub ok 2024-09-21T11:55:25.739 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:25.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:25.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:25.792 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:25 smithi175 ceph-mon[32613]: pgmap v740: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 669 B/s rd, 0 B/s wr, 935 op/s 2024-09-21T11:55:25.792 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:25 smithi175 ceph-mon[32613]: 3.14 scrub starts 2024-09-21T11:55:25.792 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:25 smithi175 ceph-mon[32613]: 3.14 scrub ok 2024-09-21T11:55:25.792 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:25 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/31782664' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:25.792 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:25 smithi175 ceph-mon[32613]: 5.7 scrub starts 2024-09-21T11:55:25.793 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:25 smithi175 ceph-mon[32613]: 5.7 scrub ok 2024-09-21T11:55:25.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:25.905 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:25.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:25.906 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:26.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:26.042 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:26.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:26.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:26.079 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:55:26.079 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:55:26.079 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:55:26.079 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:55:26.080 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:55:26.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:26.177 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:26.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:26.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:26.280 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:26.303 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:26.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:26.331 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:26.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:26.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:26.466 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:26.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:26.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:26.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:55:26.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:26.597 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:26.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:26.598 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:26.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:26.740 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:26.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:26.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:26.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[29710]: 6.f scrub starts 2024-09-21T11:55:26.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[29710]: 6.f scrub ok 2024-09-21T11:55:26.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[29710]: 3.1e scrub starts 2024-09-21T11:55:26.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[29710]: 3.1e scrub ok 2024-09-21T11:55:26.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[29710]: 5.3 scrub starts 2024-09-21T11:55:26.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[29710]: 5.3 scrub ok 2024-09-21T11:55:26.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[36454]: 6.f scrub starts 2024-09-21T11:55:26.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[36454]: 6.f scrub ok 2024-09-21T11:55:26.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[36454]: 3.1e scrub starts 2024-09-21T11:55:26.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[36454]: 3.1e scrub ok 2024-09-21T11:55:26.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[36454]: 5.3 scrub starts 2024-09-21T11:55:26.744 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:26 smithi043 ceph-mon[36454]: 5.3 scrub ok 2024-09-21T11:55:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:26 smithi175 ceph-mon[32613]: 6.f scrub starts 2024-09-21T11:55:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:26 smithi175 ceph-mon[32613]: 6.f scrub ok 2024-09-21T11:55:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:26 smithi175 ceph-mon[32613]: 3.1e scrub starts 2024-09-21T11:55:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:26 smithi175 ceph-mon[32613]: 3.1e scrub ok 2024-09-21T11:55:26.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:26 smithi175 ceph-mon[32613]: 5.3 scrub starts 2024-09-21T11:55:26.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:26 smithi175 ceph-mon[32613]: 5.3 scrub ok 2024-09-21T11:55:26.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:26.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:26.904 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:26.904 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:26.922 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:55:27.005 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:27.034 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:27.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:27.035 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:27.136 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:27.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:27.171 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:27.171 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:27.272 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:27.324 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:27.328 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:27.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:27.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:27.430 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:27.465 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:27.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:27.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:27.556 INFO:teuthology.orchestra.run.smithi043.stdout:moved 27 / 444 (6.08108%) 2024-09-21T11:55:27.556 INFO:teuthology.orchestra.run.smithi043.stdout:avg 55.5 2024-09-21T11:55:27.556 INFO:teuthology.orchestra.run.smithi043.stdout:stddev 5.76628 -> 7.31437 (expected baseline 6.96868) 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout:min osd.2 with 45 -> 54 pgs (0.810811 -> 0.972973 * mean) 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout:max osd.4 with 66 -> 60 pgs (1.18919 -> 1.08108 * mean) 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout:oload 110 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout:max_change 1 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout:max_change_osds 4 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout:average_utilization 0.0207 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout:overload_utilization 0.0227 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0 weight 1.0000 -> 0.6837 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4 weight 1.0000 -> 0.8904 2024-09-21T11:55:27.557 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6 weight 1.0000 -> 0.9072 2024-09-21T11:55:27.567 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:27.597 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:27.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:27.599 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:27.699 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:27.722 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:27.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:27.723 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:27.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:27 smithi175 ceph-mon[32613]: pgmap v741: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 679 op/s 2024-09-21T11:55:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:27 smithi175 ceph-mon[32613]: 5.15 scrub starts 2024-09-21T11:55:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:27 smithi175 ceph-mon[32613]: 5.15 scrub ok 2024-09-21T11:55:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:27 smithi175 ceph-mon[32613]: 3.1b scrub starts 2024-09-21T11:55:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:27 smithi175 ceph-mon[32613]: 3.1b scrub ok 2024-09-21T11:55:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:27 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'0':'44804','4':'58353','6':'59453'}"} : dispatch 2024-09-21T11:55:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:27 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'0':'44804','4':'58353','6':'59453'}"} : dispatch 2024-09-21T11:55:27.818 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[29710]: pgmap v741: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 679 op/s 2024-09-21T11:55:27.818 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[29710]: 5.15 scrub starts 2024-09-21T11:55:27.818 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[29710]: 5.15 scrub ok 2024-09-21T11:55:27.818 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[29710]: 3.1b scrub starts 2024-09-21T11:55:27.818 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[29710]: 3.1b scrub ok 2024-09-21T11:55:27.818 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'0':'44804','4':'58353','6':'59453'}"} : dispatch 2024-09-21T11:55:27.818 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'0':'44804','4':'58353','6':'59453'}"} : dispatch 2024-09-21T11:55:27.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[36454]: pgmap v741: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 679 op/s 2024-09-21T11:55:27.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[36454]: 5.15 scrub starts 2024-09-21T11:55:27.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[36454]: 5.15 scrub ok 2024-09-21T11:55:27.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[36454]: 3.1b scrub starts 2024-09-21T11:55:27.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[36454]: 3.1b scrub ok 2024-09-21T11:55:27.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'0':'44804','4':'58353','6':'59453'}"} : dispatch 2024-09-21T11:55:27.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:27 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'0':'44804','4':'58353','6':'59453'}"} : dispatch 2024-09-21T11:55:27.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:27.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:27.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:27.920 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:55:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:55:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:55:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:55:27.921 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:47541 10.f 10:f5e3899d:::benchmark_data_smithi043_150678_object47540:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:13.848208+0000", 2024-09-21T11:55:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 134.045250708, 2024-09-21T11:55:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28965287499999998, 2024-09-21T11:55:27.922 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.923 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47541 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.848208+0000", 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.848208+0000", 2024-09-21T11:55:27.924 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.848209+0000", 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.35e-06 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.848267+0000", 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8189000000000001e-05 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.925 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.848280+0000", 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2689e-05 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.848287+0000", 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4109999999999999e-06 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.848343+0000", 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6042999999999999e-05 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.926 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.848474+0000", 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000131023 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.848651+0000", 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00017651900000000001 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.854379+0000", 2024-09-21T11:55:27.927 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057285840000000001 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856442+0000", 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020620600000000001 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.137779+0000", 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28133745700000001 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.928 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.137816+0000", 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7067000000000003e-05 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.137861+0000", 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4483000000000001e-05 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:47547 10.f 10:fa7229d3:::benchmark_data_smithi043_150678_object47546:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:13.849076+0000", 2024-09-21T11:55:27.929 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 134.04438191899999, 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.294471396, 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47547 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.849076+0000", 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.930 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.849076+0000", 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.849077+0000", 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3099999999999996e-07 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.849139+0000", 2024-09-21T11:55:27.931 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1523e-05 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.849152+0000", 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3264999999999999e-05 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.849159+0000", 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8390000000000001e-06 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.850503+0000", 2024-09-21T11:55:27.932 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001344338 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.850720+0000", 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021672999999999999 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.850907+0000", 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00018716300000000001 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.854418+0000", 2024-09-21T11:55:27.933 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0035113309999999999 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.858362+0000", 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0039432060000000003 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143426+0000", 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.285064331 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143494+0000", 2024-09-21T11:55:27.934 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7631000000000004e-05 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143548+0000", 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4208e-05 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:47550 10.f 10:fa6f9d44:::benchmark_data_smithi043_150678_object47549:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:13.850905+0000", 2024-09-21T11:55:27.935 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 134.04255312399999, 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29278101099999998, 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47550 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.936 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.850905+0000", 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.850905+0000", 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.850906+0000", 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0139999999999999e-06 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.937 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.851000+0000", 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4118999999999998e-05 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.851010+0000", 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6139999999999998e-06 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.851018+0000", 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8420000000000005e-06 2024-09-21T11:55:27.938 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.854449+0000", 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0034309010000000001 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.854585+0000", 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000136441 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.854716+0000", 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000130648 2024-09-21T11:55:27.939 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.858410+0000", 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003694003 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.858423+0000", 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3441000000000001e-05 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143598+0000", 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28517517999999997 2024-09-21T11:55:27.940 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143648+0000", 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9276000000000001e-05 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143686+0000", 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8532000000000003e-05 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.941 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:47555 10.7 10:eaeff710:::benchmark_data_smithi043_150678_object47554:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:13.855044+0000", 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 134.03841424800001, 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28883321000000001, 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47555 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.942 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855044+0000", 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855044+0000", 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855045+0000", 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.99e-07 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.943 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855071+0000", 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5928999999999999e-05 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855080+0000", 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9919999999999996e-06 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855086+0000", 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3879999999999997e-06 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.944 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855138+0000", 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2003999999999999e-05 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855222+0000", 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3776000000000006e-05 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855547+0000", 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00032499299999999998 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.945 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.858401+0000", 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028546130000000002 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.858428+0000", 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6458e-05 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143794+0000", 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28536626900000001 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.946 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143843+0000", 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8643999999999997e-05 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143877+0000", 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4445e-05 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:47559 10.f 10:fadc3654:::benchmark_data_smithi043_150678_object47558:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:27.947 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:13.855950+0000", 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 134.037507912, 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28807566200000001, 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47559 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855950+0000", 2024-09-21T11:55:27.948 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855950+0000", 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.855951+0000", 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.09e-07 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856014+0000", 2024-09-21T11:55:27.949 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2303000000000003e-05 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856035+0000", 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1614999999999999e-05 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856044+0000", 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6130000000000004e-06 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.950 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856465+0000", 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042129500000000002 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856560+0000", 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4788000000000001e-05 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,3", 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856674+0000", 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114521 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.951 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.858498+0000", 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018232960000000001 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.858962+0000", 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046445200000000002 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143923+0000", 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28496084900000002 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.952 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143986+0000", 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2539999999999994e-05 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.144026+0000", 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0481000000000001e-05 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:47560 10.7 10:eff58e28:::benchmark_data_smithi043_150678_object47559:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:13.856776+0000", 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 134.036682711, 2024-09-21T11:55:27.953 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28725373599999998, 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47560 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856776+0000", 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.954 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856776+0000", 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856777+0000", 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.372e-06 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856836+0000", 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9199e-05 2024-09-21T11:55:27.955 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856847+0000", 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1005999999999999e-05 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.856856+0000", 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5439999999999999e-06 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.956 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.858458+0000", 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001602537 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.858556+0000", 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7924999999999995e-05 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.858642+0000", 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5618000000000006e-05 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.957 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.860305+0000", 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00166329 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.860757+0000", 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00045195400000000002 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143921+0000", 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28316366599999998 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.958 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.143989+0000", 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8439e-05 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.144029+0000", 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0185999999999998e-05 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:47565 10.7 10:e627394e:::benchmark_data_smithi043_150678_object47564:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:27.959 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:13.863716+0000", 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 134.029741874, 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28043058500000001, 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 47565 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.960 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.863716+0000", 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.863716+0000", 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.863718+0000", 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.539e-06 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.961 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.863770+0000", 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2391e-05 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.863782+0000", 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1535999999999999e-05 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.863787+0000", 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0150000000000003e-06 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.962 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.863833+0000", 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6508999999999998e-05 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.863961+0000", 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012710600000000001 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.864047+0000", 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6661000000000003e-05 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.963 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.865546+0000", 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001498779 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:13.865920+0000", 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00037431900000000002 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.144065+0000", 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27814480899999999 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.964 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.144111+0000", 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5794000000000002e-05 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:14.144147+0000", 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6127000000000002e-05 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67837 10.a 10:5163063c:::benchmark_data_smithi043_150678_object67836:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046032+0000", 2024-09-21T11:55:27.965 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 118.84742638500001, 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28443493600000003, 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67837 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046032+0000", 2024-09-21T11:55:27.966 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046032+0000", 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046034+0000", 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6610000000000001e-06 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046106+0000", 2024-09-21T11:55:27.967 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2730000000000003e-05 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046120+0000", 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3154e-05 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046127+0000", 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5989999999999996e-06 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046234+0000", 2024-09-21T11:55:27.968 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000107353 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046344+0000", 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010936400000000001 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046643+0000", 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029910400000000001 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.051199+0000", 2024-09-21T11:55:27.969 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045556199999999998 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052422+0000", 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012237249999999999 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329800+0000", 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27737740300000002 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.970 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.330413+0000", 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00061291799999999995 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.330467+0000", 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4305000000000003e-05 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67842 10.7 10:e64204f5:::benchmark_data_smithi043_150678_object67841:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:55:27.971 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046901+0000", 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 118.846557734, 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29040000399999999, 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67842 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.972 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046901+0000", 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046901+0000", 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046901+0000", 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7700000000000005e-07 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.973 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046959+0000", 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7772000000000002e-05 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046965+0000", 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9209999999999997e-06 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046970+0000", 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.887e-06 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.974 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047161+0000", 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000191597 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047257+0000", 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5379999999999998e-05 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047363+0000", 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106601 2024-09-21T11:55:27.975 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.051954+0000", 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045905260000000002 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053167+0000", 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001213656 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337195+0000", 2024-09-21T11:55:27.976 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28402736299999998 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337255+0000", 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0362000000000002e-05 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337301+0000", 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5461999999999999e-05 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.977 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67849 10.a 10:51c5c732:::benchmark_data_smithi043_150678_object67848:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.052391+0000", 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 118.84106704600001, 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28774486199999999, 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67849 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.978 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052391+0000", 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052391+0000", 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052392+0000", 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1600000000000001e-07 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.979 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.980 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.980 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052429+0000", 2024-09-21T11:55:27.980 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6918999999999997e-05 2024-09-21T11:55:27.980 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.981 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052435+0000", 2024-09-21T11:55:27.981 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5390000000000004e-06 2024-09-21T11:55:27.981 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.981 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.981 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052441+0000", 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4229999999999996e-06 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052510+0000", 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8796999999999999e-05 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052632+0000", 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012236300000000001 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.982 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052776+0000", 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014414399999999999 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.055195+0000", 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024188249999999999 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.055867+0000", 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00067204600000000004 2024-09-21T11:55:27.983 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.340022+0000", 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28415448999999998 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.340086+0000", 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4700000000000001e-05 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.340136+0000", 2024-09-21T11:55:27.984 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.99e-05 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67869 10.a 10:5b2a1c6d:::benchmark_data_smithi043_150678_object67868:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.068678+0000", 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 118.82478004799999, 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27168131099999998, 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.985 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67869 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.068678+0000", 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.068678+0000", 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.986 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.068680+0000", 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.279e-06 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.068718+0000", 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8628999999999999e-05 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.068727+0000", 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4710000000000007e-06 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.987 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.068732+0000", 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8999999999999997e-06 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.068772+0000", 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0259000000000002e-05 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.068852+0000", 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0490999999999999e-05 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.988 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:55:27.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.068947+0000", 2024-09-21T11:55:27.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4778000000000006e-05 2024-09-21T11:55:27.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.989 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.989 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.072694+0000", 2024-09-21T11:55:27.989 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0037472949999999999 2024-09-21T11:55:27.989 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.989 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.989 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:27.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.073720+0000", 2024-09-21T11:55:27.991 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001025506 2024-09-21T11:55:27.991 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.991 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.991 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.991 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.340210+0000", 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.26648993799999998 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.340320+0000", 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000110016 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.340360+0000", 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9749000000000002e-05 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.992 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67871 10.a 10:56e52118:::benchmark_data_smithi043_150678_object67870:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.069944+0000", 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 118.82351481400001, 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27057836000000002, 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67871 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:27.993 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.069944+0000", 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.069944+0000", 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.069944+0000", 2024-09-21T11:55:27.994 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9899999999999999e-07 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.069999+0000", 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4471000000000001e-05 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.070009+0000", 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0078000000000001e-05 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.070015+0000", 2024-09-21T11:55:27.995 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4269999999999996e-06 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.071125+0000", 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001109425 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.071228+0000", 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103662 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,5", 2024-09-21T11:55:27.996 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.071310+0000", 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1934999999999994e-05 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.072857+0000", 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001546368 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.073788+0000", 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00093119100000000005 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:27.997 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.340403+0000", 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.266614921 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.340484+0000", 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1167000000000004e-05 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.340522+0000", 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7916000000000001e-05 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:27.998 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67878 10.7 10:e82489f1:::benchmark_data_smithi043_150678_object67877:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.079204+0000", 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 118.81425467, 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.25827462400000001, 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67878 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:27.999 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.079204+0000", 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.079204+0000", 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.079205+0000", 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1790000000000001e-06 2024-09-21T11:55:28.000 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.079260+0000", 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4802000000000001e-05 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.079270+0000", 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0329999999999999e-05 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.079276+0000", 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1360000000000002e-06 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.001 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.079332+0000", 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5871000000000001e-05 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.079468+0000", 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013616499999999999 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.079567+0000", 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8907999999999996e-05 2024-09-21T11:55:28.002 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.083504+0000", 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0039364919999999998 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.084121+0000", 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00061694799999999998 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337349+0000", 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.25322892200000002 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.003 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337446+0000", 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6093000000000003e-05 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337478+0000", 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2777999999999997e-05 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.004 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72203 10.0 10:0f581edb:::benchmark_data_smithi043_150678_object72202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.964491+0000", 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 114.928967159, 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.238550227, 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72203 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.005 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.964491+0000", 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.964491+0000", 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.964492+0000", 2024-09-21T11:55:28.006 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6000000000000003e-07 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.964566+0000", 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4151000000000002e-05 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.964583+0000", 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6489000000000001e-05 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.964589+0000", 2024-09-21T11:55:28.007 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6919999999999996e-06 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.964643+0000", 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3372000000000003e-05 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.964771+0000", 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012788599999999999 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.964978+0000", 2024-09-21T11:55:28.008 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000207846 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967599+0000", 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026203450000000001 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.974457+0000", 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0068583569999999998 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202854+0000", 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.22839657799999999 2024-09-21T11:55:28.009 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.202970+0000", 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000116205 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.203041+0000", 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1545999999999996e-05 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.010 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72207 10.d 10:b16de966:::benchmark_data_smithi043_150678_object72206:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.967092+0000", 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 114.92636647499999, 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24868158100000001, 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72207 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.011 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967092+0000", 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967092+0000", 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967093+0000", 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.621e-06 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.012 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967248+0000", 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015449999999999999 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967259+0000", 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1436e-05 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967267+0000", 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0209999999999999e-06 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.013 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967519+0000", 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025169 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967646+0000", 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000127336 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.967780+0000", 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000133702 2024-09-21T11:55:28.014 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.972454+0000", 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0046735769999999999 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.973338+0000", 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000884639 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.215669+0000", 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24233020799999999 2024-09-21T11:55:28.015 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.215739+0000", 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0720000000000001e-05 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.215773+0000", 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4131000000000002e-05 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.016 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72209 10.d 10:b3bcc1b2:::benchmark_data_smithi043_150678_object72208:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.968645+0000", 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 114.924813106, 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.247306584, 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72209 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:28.017 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968645+0000", 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968645+0000", 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968646+0000", 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.147e-06 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.018 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968935+0000", 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028859000000000003 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968944+0000", 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.4129999999999995e-06 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968950+0000", 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1760000000000001e-06 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.019 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970355+0000", 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001405157 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970470+0000", 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000115266 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970610+0000", 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013994800000000001 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.020 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.972585+0000", 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019750309999999999 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.977961+0000", 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0053757830000000003 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.215809+0000", 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23784873500000001 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.021 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.215900+0000", 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0839000000000002e-05 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.215952+0000", 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1499000000000003e-05 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72210 10.0 10:07e5b872:::benchmark_data_smithi043_150678_object72209:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:55:28.022 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.968960+0000", 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 114.924498125, 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24006982499999999, 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72210 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968960+0000", 2024-09-21T11:55:28.023 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968960+0000", 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968961+0000", 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6800000000000001e-07 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968981+0000", 2024-09-21T11:55:28.024 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0026000000000001e-05 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968987+0000", 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5230000000000001e-06 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968991+0000", 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8639999999999998e-06 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969037+0000", 2024-09-21T11:55:28.025 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5575999999999999e-05 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969155+0000", 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011827100000000001 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969317+0000", 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016205599999999999 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:28.026 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.974471+0000", 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0051542510000000003 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.978176+0000", 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0037045799999999999 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.208944+0000", 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23076802799999999 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:28.027 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.208991+0000", 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7466999999999998e-05 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.209030+0000", 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8714999999999998e-05 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72211 10.0 10:0a0bffc7:::benchmark_data_smithi043_150678_object72210:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.969251+0000", 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 114.924207021, 2024-09-21T11:55:28.028 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24017171900000001, 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72211 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969251+0000", 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.029 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969251+0000", 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969252+0000", 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8199999999999999e-07 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969324+0000", 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2015999999999996e-05 2024-09-21T11:55:28.030 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969331+0000", 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0589999999999997e-06 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969338+0000", 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7580000000000003e-06 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.972160+0000", 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0028221259999999999 2024-09-21T11:55:28.031 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.972299+0000", 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013937899999999999 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.972444+0000", 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014454600000000001 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.974477+0000", 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0020326620000000002 2024-09-21T11:55:28.032 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.978311+0000", 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0038344999999999998 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.209335+0000", 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.231024023 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.209383+0000", 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7712e-05 2024-09-21T11:55:28.033 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.209423+0000", 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0256000000000002e-05 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72224 10.d 10:b8b4116a:::benchmark_data_smithi043_150678_object72223:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.979575+0000", 2024-09-21T11:55:28.034 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 114.91388381, 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23653006500000001, 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72224 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.979575+0000", 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.035 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.979575+0000", 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.979576+0000", 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.319e-06 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.979644+0000", 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7698000000000001e-05 2024-09-21T11:55:28.036 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.979656+0000", 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2328e-05 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.979664+0000", 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2050000000000002e-06 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.979710+0000", 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6156000000000003e-05 2024-09-21T11:55:28.037 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.979800+0000", 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9949000000000002e-05 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.979888+0000", 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7904000000000004e-05 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.981692+0000", 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018042379999999999 2024-09-21T11:55:28.038 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.982711+0000", 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001018316 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.215995+0000", 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.233284296 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.216072+0000", 2024-09-21T11:55:28.039 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7528999999999996e-05 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.216105+0000", 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2126999999999999e-05 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:126300 10.b 10:d6278167:::benchmark_data_smithi043_150678_object3549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:54:25.231599+0000", 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 62.661859481, 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23982671, 2024-09-21T11:55:28.040 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 126300 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.231599+0000", 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:55:28.041 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.231599+0000", 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.231600+0000", 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4640000000000001e-06 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.231606+0000", 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7130000000000002e-06 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.042 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.231617+0000", 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1260999999999999e-05 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.231625+0000", 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3599999999999998e-06 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.232074+0000", 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044959399999999999 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.043 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.232174+0000", 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9275000000000001e-05 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,1", 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.232276+0000", 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000102817 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.235801+0000", 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003524957 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.044 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.237636+0000", 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001835075 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.471365+0000", 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23372907100000001 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.471398+0000", 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2858000000000003e-05 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:55:28.045 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:55:28.046 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:55:28.046 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:54:25.471426+0000", 2024-09-21T11:55:28.046 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7265000000000001e-05 2024-09-21T11:55:28.046 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.046 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:28.046 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.046 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:55:28.046 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:55:28.046 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:55:28.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:28.048 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:28.048 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:28.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:55:28.148 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:28.169 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:28.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:28.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:28.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:28.292 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:28.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:28.293 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:28.394 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:28.415 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:28.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:28.416 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:28.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:28.542 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:28.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:28.543 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:28.619 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-21T11:55:28.619 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd test-reweight-by-utilization 1000 0.05 2024-09-21T11:55:28.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:28.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:28.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:28.742 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:28 smithi175 ceph-mon[32613]: from='client.26183 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:28.742 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:28 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'0':'44804','4':'58353','6':'59453'}"}]': finished 2024-09-21T11:55:28.742 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:28 smithi175 ceph-mon[32613]: osdmap e364: 8 total, 8 up, 8 in 2024-09-21T11:55:28.742 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:28 smithi175 ceph-mon[32613]: 10.c scrub starts 2024-09-21T11:55:28.742 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:28 smithi175 ceph-mon[32613]: osdmap e365: 8 total, 8 up, 8 in 2024-09-21T11:55:28.771 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:55:28.789 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:28.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:28.835 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:28.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:28.869 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:28 smithi043 ceph-mon[36454]: from='client.26183 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:28.870 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:28 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'0':'44804','4':'58353','6':'59453'}"}]': finished 2024-09-21T11:55:28.870 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:28 smithi043 ceph-mon[36454]: osdmap e364: 8 total, 8 up, 8 in 2024-09-21T11:55:28.870 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:28 smithi043 ceph-mon[36454]: 10.c scrub starts 2024-09-21T11:55:28.870 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:28 smithi043 ceph-mon[36454]: osdmap e365: 8 total, 8 up, 8 in 2024-09-21T11:55:28.870 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:28 smithi043 ceph-mon[29710]: from='client.26183 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:28.870 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:28 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'0':'44804','4':'58353','6':'59453'}"}]': finished 2024-09-21T11:55:28.870 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:28 smithi043 ceph-mon[29710]: osdmap e364: 8 total, 8 up, 8 in 2024-09-21T11:55:28.870 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:28 smithi043 ceph-mon[29710]: 10.c scrub starts 2024-09-21T11:55:28.870 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:28 smithi043 ceph-mon[29710]: osdmap e365: 8 total, 8 up, 8 in 2024-09-21T11:55:28.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:28.963 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:28.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:28.965 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:28.979 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:29.023 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:29.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:29.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:29.090 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:29.090 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:29.190 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:29.212 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:29.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:29.213 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:29.314 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:29.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:29.339 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:29.339 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:29.440 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:29.462 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:29.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:29.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:29.486 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:55:29.486 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:55:29.486 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:55:29.486 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:55:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:55:29.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:29.587 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:29.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:29.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:29.651 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:55:29.651 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:55:29.670 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:55:29.671 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:55:29.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:29.703 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:55:29.703 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:55:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:29.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:29.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:29.745 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:55:29.745 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:55:29.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: 10.c scrub ok 2024-09-21T11:55:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: pgmap v743: 148 pgs: 148 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-21T11:55:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: 5.e scrub starts 2024-09-21T11:55:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: 5.e scrub ok 2024-09-21T11:55:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:55:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:55:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/140353513' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:55:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:55:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/140353513' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:29 smithi175 ceph-mon[32613]: osdmap e366: 8 total, 8 up, 8 in 2024-09-21T11:55:29.782 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:55:29.782 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:55:29.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: 10.c scrub ok 2024-09-21T11:55:29.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: pgmap v743: 148 pgs: 148 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-21T11:55:29.782 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: 5.e scrub starts 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: 5.e scrub ok 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/140353513' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/140353513' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[29710]: osdmap e366: 8 total, 8 up, 8 in 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: 10.c scrub ok 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: pgmap v743: 148 pgs: 148 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: 5.e scrub starts 2024-09-21T11:55:29.783 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: 5.e scrub ok 2024-09-21T11:55:29.784 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:55:29.784 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:55:29.784 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/140353513' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:29.784 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:55:29.784 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:55:29.784 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/140353513' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:29.784 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:29 smithi043 ceph-mon[36454]: osdmap e366: 8 total, 8 up, 8 in 2024-09-21T11:55:29.823 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:55:29.823 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:55:29.832 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:29.860 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:55:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:55:29.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:29.866 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:29.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:29.901 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:55:29.901 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:55:29.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:30.009 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:30.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:30.010 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:30.110 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:30.136 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:30.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:30.137 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:30.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:30.281 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:30.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:30.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:30.341 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:55:30.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:30.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:30.425 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:30.526 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:30.565 INFO:teuthology.orchestra.run.smithi043.stdout:moved 0 / 444 (0%) 2024-09-21T11:55:30.565 INFO:teuthology.orchestra.run.smithi043.stdout:avg 55.5 2024-09-21T11:55:30.565 INFO:teuthology.orchestra.run.smithi043.stdout:stddev 7.31437 -> 7.31437 (expected baseline 6.96868) 2024-09-21T11:55:30.565 INFO:teuthology.orchestra.run.smithi043.stdout:min osd.0 with 38 -> 38 pgs (0.684685 -> 0.684685 * mean) 2024-09-21T11:55:30.566 INFO:teuthology.orchestra.run.smithi043.stdout:max osd.1 with 63 -> 63 pgs (1.13514 -> 1.13514 * mean) 2024-09-21T11:55:30.566 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:55:30.566 INFO:teuthology.orchestra.run.smithi043.stdout:oload 1000 2024-09-21T11:55:30.566 INFO:teuthology.orchestra.run.smithi043.stdout:max_change 0.05 2024-09-21T11:55:30.566 INFO:teuthology.orchestra.run.smithi043.stdout:max_change_osds 4 2024-09-21T11:55:30.566 INFO:teuthology.orchestra.run.smithi043.stdout:average_utilization 0.0196 2024-09-21T11:55:30.566 INFO:teuthology.orchestra.run.smithi043.stdout:overload_utilization 0.1955 2024-09-21T11:55:30.566 INFO:teuthology.orchestra.run.smithi043.stderr:no change 2024-09-21T11:55:30.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:30.567 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:30.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:30.668 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:30.708 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:30.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:30.709 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:30.733 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:30.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:30.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:30.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:30.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:30 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:55:30.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:30 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:55:30.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:30 smithi175 ceph-mon[32613]: 3.2 scrub starts 2024-09-21T11:55:30.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:30 smithi175 ceph-mon[32613]: 3.2 scrub ok 2024-09-21T11:55:30.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:30 smithi175 ceph-mon[32613]: 10.a starting backfill to osd.2 from (0'0,0'0] MAX to 363'12043 2024-09-21T11:55:30.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:30 smithi175 ceph-mon[32613]: 10.4 starting backfill to osd.1 from (0'0,0'0] MAX to 363'12217 2024-09-21T11:55:30.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:30 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1266322605' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:30.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:30.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:55:30.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:55:30.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[29710]: 3.2 scrub starts 2024-09-21T11:55:30.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[29710]: 3.2 scrub ok 2024-09-21T11:55:30.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[29710]: 10.a starting backfill to osd.2 from (0'0,0'0] MAX to 363'12043 2024-09-21T11:55:30.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[29710]: 10.4 starting backfill to osd.1 from (0'0,0'0] MAX to 363'12217 2024-09-21T11:55:30.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1266322605' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:30.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:55:30.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:55:30.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[36454]: 3.2 scrub starts 2024-09-21T11:55:30.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[36454]: 3.2 scrub ok 2024-09-21T11:55:30.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[36454]: 10.a starting backfill to osd.2 from (0'0,0'0] MAX to 363'12043 2024-09-21T11:55:30.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[36454]: 10.4 starting backfill to osd.1 from (0'0,0'0] MAX to 363'12217 2024-09-21T11:55:30.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:30 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1266322605' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:30.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:30.995 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:31.097 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:31.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:31.134 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:31.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:31.271 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:31.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:31.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:31.372 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:31.412 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:31.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:31.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:31.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:31.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:31.541 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:31.541 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:31.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:31.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:31 smithi043 ceph-mon[36454]: pgmap v746: 148 pgs: 148 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 872 op/s 2024-09-21T11:55:31.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:31 smithi043 ceph-mon[36454]: from='client.26201 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:31.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:31 smithi043 ceph-mon[36454]: 10.6 scrub starts 2024-09-21T11:55:31.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:31 smithi043 ceph-mon[29710]: pgmap v746: 148 pgs: 148 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 872 op/s 2024-09-21T11:55:31.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:31 smithi043 ceph-mon[29710]: from='client.26201 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:31.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:31 smithi043 ceph-mon[29710]: 10.6 scrub starts 2024-09-21T11:55:31.715 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:31.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:31.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:31.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:31.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:31.849 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:31.850 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:31.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:31 smithi175 ceph-mon[32613]: pgmap v746: 148 pgs: 148 active+clean; 3.2 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 872 op/s 2024-09-21T11:55:31.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:31 smithi175 ceph-mon[32613]: from='client.26201 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:31.916 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:31 smithi175 ceph-mon[32613]: 10.6 scrub starts 2024-09-21T11:55:31.917 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:55:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:55:31.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:31.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:31.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:32.088 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:32.133 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:32.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:32.134 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:32.157 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-21T11:55:32.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd test-reweight-by-utilization 1000 1.0 2024-09-21T11:55:32.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:32.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:32.286 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:32.287 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:32.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:32.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:32.439 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:32.439 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:32.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:32.567 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:32.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:32.568 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:32.571 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:32.669 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:32.701 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:32.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:32.702 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:32.707 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:55:32.707 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:55:32.803 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:32.837 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:32.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:32.838 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:32.939 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:32.960 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:32.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:32.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:55:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[29710]: 10.6 scrub ok 2024-09-21T11:55:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:55:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[36454]: 10.6 scrub ok 2024-09-21T11:55:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:32 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:33.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:32 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:55:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:32 smithi175 ceph-mon[32613]: 10.6 scrub ok 2024-09-21T11:55:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:33.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:32 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]} : dispatch 2024-09-21T11:55:33.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:55:33.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:33.083 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:33.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:33.084 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:33.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:33.215 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:33.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:33.216 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:33.317 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:33.354 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:33.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:33.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:33.456 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:33.483 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:33.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:33.484 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:33.585 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:33.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:33.621 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:33.621 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:33.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:55:33.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:33.791 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:33.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:33.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:33.893 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:33.928 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:33.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:33.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:33.975 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:33 smithi175 ceph-mon[32613]: pgmap v747: 148 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 145 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.29k op/s; 6006/150057 objects misplaced (4.002%); 2.4 MiB/s, 43 objects/s recovering 2024-09-21T11:55:33.975 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-21T11:55:33.975 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:33 smithi175 ceph-mon[32613]: osdmap e367: 8 total, 8 up, 8 in 2024-09-21T11:55:33.975 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:33 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T11:55:33.975 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:33 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T11:55:33.976 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:55:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:33 smithi043 ceph-mon[29710]: pgmap v747: 148 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 145 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.29k op/s; 6006/150057 objects misplaced (4.002%); 2.4 MiB/s, 43 objects/s recovering 2024-09-21T11:55:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-21T11:55:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:33 smithi043 ceph-mon[29710]: osdmap e367: 8 total, 8 up, 8 in 2024-09-21T11:55:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:33 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T11:55:33.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:33 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T11:55:33.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:33 smithi043 ceph-mon[36454]: pgmap v747: 148 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 145 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.29k op/s; 6006/150057 objects misplaced (4.002%); 2.4 MiB/s, 43 objects/s recovering 2024-09-21T11:55:33.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-21T11:55:33.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:33 smithi043 ceph-mon[36454]: osdmap e367: 8 total, 8 up, 8 in 2024-09-21T11:55:33.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:33 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T11:55:33.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:33 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T11:55:34.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:34.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:34.065 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:34.065 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:34.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:34.188 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:34.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:34.189 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:34.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:34.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:34.330 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:34.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:34.334 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:34.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:34.474 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:34.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:34.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:34.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:34.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:34.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:34.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:34.769 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:34.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:34.771 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:34.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:34.902 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:34.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:34.903 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:34.942 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:55:34.942 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:55:34.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:34 smithi043 ceph-mon[29710]: 6.16 scrub starts 2024-09-21T11:55:34.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:34 smithi043 ceph-mon[29710]: 6.16 scrub ok 2024-09-21T11:55:34.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:34 smithi043 ceph-mon[29710]: osdmap e368: 8 total, 8 up, 8 in 2024-09-21T11:55:34.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:34 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T11:55:34.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:34 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T11:55:34.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:34 smithi043 ceph-mon[36454]: 6.16 scrub starts 2024-09-21T11:55:34.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:34 smithi043 ceph-mon[36454]: 6.16 scrub ok 2024-09-21T11:55:34.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:34 smithi043 ceph-mon[36454]: osdmap e368: 8 total, 8 up, 8 in 2024-09-21T11:55:34.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:34 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T11:55:34.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:34 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T11:55:35.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:35.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:34 smithi175 ceph-mon[32613]: 6.16 scrub starts 2024-09-21T11:55:35.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:34 smithi175 ceph-mon[32613]: 6.16 scrub ok 2024-09-21T11:55:35.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:34 smithi175 ceph-mon[32613]: osdmap e368: 8 total, 8 up, 8 in 2024-09-21T11:55:35.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:34 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T11:55:35.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:34 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T11:55:35.045 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:35.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:35.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:35.147 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:35.178 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:35.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:35.179 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:35.280 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:35.320 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:35.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:35.321 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:35.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:35.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:35.464 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:35.490 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:55:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:55:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:35.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:35.600 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:35.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:35.601 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:35.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:55:35.702 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:35.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:35.753 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:35.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:35.853 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:35.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:35.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:35.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:35 smithi043 ceph-mon[29710]: pgmap v750: 148 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 145 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 738 B/s rd, 0 B/s wr, 487 op/s; 6006/149496 objects misplaced (4.017%); 2.6 MiB/s, 46 objects/s recovering 2024-09-21T11:55:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:35 smithi043 ceph-mon[29710]: 3.f scrub starts 2024-09-21T11:55:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:35 smithi043 ceph-mon[29710]: 3.f scrub ok 2024-09-21T11:55:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:35 smithi043 ceph-mon[36454]: pgmap v750: 148 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 145 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 738 B/s rd, 0 B/s wr, 487 op/s; 6006/149496 objects misplaced (4.017%); 2.6 MiB/s, 46 objects/s recovering 2024-09-21T11:55:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:35 smithi043 ceph-mon[36454]: 3.f scrub starts 2024-09-21T11:55:35.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:35 smithi043 ceph-mon[36454]: 3.f scrub ok 2024-09-21T11:55:36.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:35 smithi175 ceph-mon[32613]: pgmap v750: 148 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 145 active+clean; 3.0 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 738 B/s rd, 0 B/s wr, 487 op/s; 6006/149496 objects misplaced (4.017%); 2.6 MiB/s, 46 objects/s recovering 2024-09-21T11:55:36.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:35 smithi175 ceph-mon[32613]: 3.f scrub starts 2024-09-21T11:55:36.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:35 smithi175 ceph-mon[32613]: 3.f scrub ok 2024-09-21T11:55:36.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:36.048 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:36.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:36.049 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:36.149 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:36.181 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:36.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:36.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:36.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:36.331 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:36.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:36.332 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:36.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:36.469 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:36.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:36.470 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:36.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:36.577 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:36.584 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:36.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:36.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:36.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:36.632 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:36.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:36.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:36.753 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:36.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:36.848 INFO:teuthology.orchestra.run.smithi043.stdout:moved 0 / 444 (0%) 2024-09-21T11:55:36.848 INFO:teuthology.orchestra.run.smithi043.stdout:avg 55.5 2024-09-21T11:55:36.848 INFO:teuthology.orchestra.run.smithi043.stdout:stddev 7.19375 -> 7.19375 (expected baseline 6.96868) 2024-09-21T11:55:36.848 INFO:teuthology.orchestra.run.smithi043.stdout:min osd.0 with 38 -> 38 pgs (0.684685 -> 0.684685 * mean) 2024-09-21T11:55:36.848 INFO:teuthology.orchestra.run.smithi043.stdout:max osd.1 with 62 -> 62 pgs (1.11712 -> 1.11712 * mean) 2024-09-21T11:55:36.848 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:55:36.848 INFO:teuthology.orchestra.run.smithi043.stdout:oload 1000 2024-09-21T11:55:36.848 INFO:teuthology.orchestra.run.smithi043.stdout:max_change 1 2024-09-21T11:55:36.848 INFO:teuthology.orchestra.run.smithi043.stdout:max_change_osds 4 2024-09-21T11:55:36.849 INFO:teuthology.orchestra.run.smithi043.stdout:average_utilization 0.0184 2024-09-21T11:55:36.849 INFO:teuthology.orchestra.run.smithi043.stdout:overload_utilization 0.1838 2024-09-21T11:55:36.849 INFO:teuthology.orchestra.run.smithi043.stderr:no change 2024-09-21T11:55:36.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:36.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:36.884 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:36.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:36.985 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:36 smithi043 ceph-mon[36454]: 6.9 scrub starts 2024-09-21T11:55:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:36 smithi043 ceph-mon[36454]: 6.9 scrub ok 2024-09-21T11:55:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:36 smithi043 ceph-mon[36454]: pgmap v751: 148 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 145 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 626 B/s rd, 0 B/s wr, 793 op/s; 4019/142032 objects misplaced (2.830%); 16 MiB/s, 266 objects/s recovering 2024-09-21T11:55:36.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:36 smithi043 ceph-mon[29710]: 6.9 scrub starts 2024-09-21T11:55:36.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:36 smithi043 ceph-mon[29710]: 6.9 scrub ok 2024-09-21T11:55:36.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:36 smithi043 ceph-mon[29710]: pgmap v751: 148 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 145 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 626 B/s rd, 0 B/s wr, 793 op/s; 4019/142032 objects misplaced (2.830%); 16 MiB/s, 266 objects/s recovering 2024-09-21T11:55:37.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:36 smithi175 ceph-mon[32613]: 6.9 scrub starts 2024-09-21T11:55:37.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:36 smithi175 ceph-mon[32613]: 6.9 scrub ok 2024-09-21T11:55:37.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:36 smithi175 ceph-mon[32613]: pgmap v751: 148 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 145 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 626 B/s rd, 0 B/s wr, 793 op/s; 4019/142032 objects misplaced (2.830%); 16 MiB/s, 266 objects/s recovering 2024-09-21T11:55:37.015 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:37.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:37.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:37.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:37.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:37.159 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:37.159 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:37.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:37.294 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:37.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:37.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:37.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:37.432 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:37.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:37.433 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:37.533 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:37.571 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:37.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:37.573 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:37.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:37.689 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:55:37.690 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:55:37.699 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:37.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:37.699 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:37.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:37.843 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:37.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:37.844 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:37.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:37.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:37.984 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:37.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:38.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:38.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:38.119 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:38.119 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:38.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:38.274 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:38.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:38.275 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:38.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:38.427 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:38.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:38.428 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:38.472 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:55:38.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:55:38.530 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:38.559 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:38.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:38.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:38.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:38.687 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:38.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:38.688 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:38.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:38 smithi043 ceph-mon[36454]: from='client.26207 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:38.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:38 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/847720348' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:38.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:38 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:38.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:38 smithi043 ceph-mon[29710]: from='client.26207 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:38.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:38 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/847720348' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:38.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:38 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:38.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:38 smithi175 ceph-mon[32613]: from='client.26207 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T11:55:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:38 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/847720348' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:38 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:55:38.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:38.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:38.845 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:38.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:38.946 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:38.986 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:38.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:38.988 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:39.088 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:39.120 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:39.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:39.121 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:39.221 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:39.253 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:39.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:39.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:39.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:39.397 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:39.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:39.398 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:39.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:39.545 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:39.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:39.546 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:39.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:39.666 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:39 smithi043 ceph-mon[36454]: pgmap v752: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 946 op/s; 4019/134904 objects misplaced (2.979%); 13 MiB/s, 218 objects/s recovering 2024-09-21T11:55:39.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:39 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:39.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:39 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:55:39.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:39 smithi043 ceph-mon[36454]: osdmap e369: 8 total, 8 up, 8 in 2024-09-21T11:55:39.667 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/435056907' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:39.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:39 smithi043 ceph-mon[29710]: pgmap v752: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 946 op/s; 4019/134904 objects misplaced (2.979%); 13 MiB/s, 218 objects/s recovering 2024-09-21T11:55:39.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:39 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:39.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:39 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:55:39.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:39 smithi043 ceph-mon[29710]: osdmap e369: 8 total, 8 up, 8 in 2024-09-21T11:55:39.667 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/435056907' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:39.695 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:39.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:39.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:39 smithi175 ceph-mon[32613]: pgmap v752: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 946 op/s; 4019/134904 objects misplaced (2.979%); 13 MiB/s, 218 objects/s recovering 2024-09-21T11:55:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:39 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:39 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:55:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:39 smithi175 ceph-mon[32613]: osdmap e369: 8 total, 8 up, 8 in 2024-09-21T11:55:39.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/435056907' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:39.797 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:39.828 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:39.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:39.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:39.930 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:39.970 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:39.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:39.971 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:40.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:40.114 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:40.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:40.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:40.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:40.248 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:40.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:40.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:40.350 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:40.391 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:40.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:40.392 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:55:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:55:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:55:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:55:40.439 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:55:40.439 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:55:40.439 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:55:40.439 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:55:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:55:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:55:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:55:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:55:40.440 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:55:40.440 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:55:40.440 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:55:40.440 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:55:40.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:40.540 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:40.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:40.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:40.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:40.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:40.696 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:40.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:40.797 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:40.829 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:40.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:40.830 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:40.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:40.972 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:40.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:40.973 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:41.074 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:41.106 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:41.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:41.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:41.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:41.220 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:41.261 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:41.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:41.262 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:41.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:41.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:41.412 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:41.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:41.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:41.547 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:41.548 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:41.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:55:41.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:41.656 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:41 smithi043 ceph-mon[29710]: pgmap v754: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 693 B/s rd, 0 B/s wr, 659 op/s; 4019/134904 objects misplaced (2.979%); 13 MiB/s, 201 objects/s recovering 2024-09-21T11:55:41.657 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:41 smithi043 ceph-mon[36454]: pgmap v754: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 693 B/s rd, 0 B/s wr, 659 op/s; 4019/134904 objects misplaced (2.979%); 13 MiB/s, 201 objects/s recovering 2024-09-21T11:55:41.697 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:41.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:41.698 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:41 smithi175 ceph-mon[32613]: pgmap v754: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 693 B/s rd, 0 B/s wr, 659 op/s; 4019/134904 objects misplaced (2.979%); 13 MiB/s, 201 objects/s recovering 2024-09-21T11:55:41.761 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:55:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:55:41.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:41.827 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:41.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:41.828 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:41.915 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:55:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:55:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:55:41.915 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:55:41.915 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:55:41.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:41.961 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:41.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:41.962 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:42.063 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:42.106 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:42.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:42.107 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:42.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:42.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:42.251 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:42.252 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:42.352 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:42.388 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:42.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:42.389 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:42.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:42.529 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:42.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:42.530 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:42.631 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:42.665 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:42.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:42.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:42.731 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:42 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:42.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:42 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:42.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:42 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:42.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:42.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:55:42.814 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:42.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:42.815 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:42.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:42.939 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:42.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:42.984 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:42.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:43.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:43.117 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:43.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:43.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:43.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:55:43.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:43.262 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:43.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:43.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:43.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:43.396 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:43.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:43.397 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:43.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:43.538 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:43.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:43.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:43.640 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:43.664 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:43.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:43.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:43.675 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:43 smithi043 ceph-mon[29710]: pgmap v755: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 2408/119919 objects misplaced (2.008%); 20 MiB/s, 326 objects/s recovering 2024-09-21T11:55:43.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:43 smithi043 ceph-mon[36454]: pgmap v755: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 2408/119919 objects misplaced (2.008%); 20 MiB/s, 326 objects/s recovering 2024-09-21T11:55:43.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:43 smithi175 ceph-mon[32613]: pgmap v755: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 2408/119919 objects misplaced (2.008%); 20 MiB/s, 326 objects/s recovering 2024-09-21T11:55:43.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:43.797 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:43.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:43.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:43.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:43.936 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:43.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:43.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:44.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:44.072 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:44.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:44.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:44.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:44.220 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:44.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:44.221 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:44.322 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:44.347 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:44.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:44.349 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:44.367 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:55:44.367 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:55:44.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119715 9.b 9:d24acb6c:::benchmark_data_smithi043_126002_object119714:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 307.70970759199997, 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.205727302, 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:55:44.368 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119715 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651647+0000", 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651670+0000", 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2603999999999999e-05 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651679+0000", 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2490000000000007e-06 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651719+0000", 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0123999999999997e-05 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652107+0000", 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00038846399999999999 2024-09-21T11:55:44.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652254+0000", 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014700100000000001 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652440+0000", 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018522499999999999 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851244+0000", 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.198804392 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851269+0000", 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5381999999999999e-05 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857251+0000", 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059817589999999997 2024-09-21T11:55:44.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857317+0000", 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5874e-05 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857373+0000", 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6014000000000003e-05 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119721 9.5 9:a09f84b6:::benchmark_data_smithi043_126002_object119720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 307.70493030300003, 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20659565899999999, 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:55:44.375 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119721 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656425+0000", 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.725e-06 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656448+0000", 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3238999999999999e-05 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656461+0000", 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.331e-05 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656470+0000", 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7940000000000008e-06 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656563+0000", 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2993999999999998e-05 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656754+0000", 2024-09-21T11:55:44.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019041800000000001 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656917+0000", 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016343600000000001 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.856079+0000", 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.199162268 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.858867+0000", 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002787253 2024-09-21T11:55:44.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.862926+0000", 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040595980000000002 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.862979+0000", 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3369999999999999e-05 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.863019+0000", 2024-09-21T11:55:44.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9254000000000001e-05 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119722 9.9 9:9c70bd2d:::benchmark_data_smithi043_126002_object119721:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 307.70485699400001, 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.21088224999999999, 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.381 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119722 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656497+0000", 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4499999999999997e-07 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656518+0000", 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1294e-05 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656526+0000", 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7570000000000004e-06 2024-09-21T11:55:44.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656532+0000", 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656639+0000", 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010705999999999999 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656753+0000", 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011366099999999999 2024-09-21T11:55:44.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657088+0000", 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00033480000000000001 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.860812+0000", 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.203723983 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.860856+0000", 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3983999999999998e-05 2024-09-21T11:55:44.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867254+0000", 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063985939999999996 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867337+0000", 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2649000000000001e-05 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867379+0000", 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1854000000000003e-05 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:119715 9.b e277/247 ondisk, result = 0)", 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 307.70479165199998, 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19469081599999999, 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:55:44.387 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656563+0000", 2024-09-21T11:55:44.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2699999999999998e-07 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656565+0000", 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5900000000000002e-06 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656572+0000", 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8730000000000001e-06 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656577+0000", 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3020000000000002e-06 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851228+0000", 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19465021800000001 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851237+0000", 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2639999999999997e-06 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851253+0000", 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5741999999999999e-05 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:119715 9.b e277/247 ondisk, result = 0)", 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 307.70399257000003, 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20004322699999999, 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.391 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657362+0000", 2024-09-21T11:55:44.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2799999999999995e-07 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657398+0000", 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6180000000000003e-05 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851962+0000", 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19456380700000001 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851969+0000", 2024-09-21T11:55:44.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2470000000000002e-06 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857236+0000", 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052674760000000001 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857245+0000", 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3350000000000007e-06 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857404+0000", 2024-09-21T11:55:44.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015935400000000001 2024-09-21T11:55:44.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.395 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.395 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119726 9.9 9:9360eb09:::benchmark_data_smithi043_126002_object119725:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 307.70219124699997, 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20832704299999999, 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119726 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659164+0000", 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2450000000000001e-06 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659190+0000", 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5653999999999999e-05 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851168+0000", 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.191977544 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851185+0000", 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7118000000000001e-05 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854231+0000", 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003045889 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854382+0000", 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015179700000000001 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854513+0000", 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013041499999999999 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867241+0000", 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012728013 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867388+0000", 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147128 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867435+0000", 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6724000000000002e-05 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867467+0000", 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1986000000000002e-05 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867489+0000", 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2529999999999999e-05 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47521 10.4 10:207ae421:::benchmark_data_smithi043_150678_object47520:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.835329+0000", 2024-09-21T11:55:44.401 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 150.52602472199999, 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.30159633899999999, 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47521 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835329+0000", 2024-09-21T11:55:44.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835329+0000", 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835330+0000", 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-06 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835511+0000", 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000180896 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835570+0000", 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.914e-05 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835577+0000", 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4549999999999998e-06 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.839940+0000", 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043626489999999997 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.840064+0000", 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012452599999999999 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.840193+0000", 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012866900000000001 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843992+0000", 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037985509999999998 2024-09-21T11:55:44.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.845585+0000", 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015931949999999999 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136797+0000", 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29121228599999999 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136857+0000", 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0177999999999999e-05 2024-09-21T11:55:44.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136925+0000", 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7770999999999999e-05 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47525 10.4 10:29bd7892:::benchmark_data_smithi043_150678_object47524:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.841626+0000", 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 150.51972700499999, 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29545084599999999, 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.407 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47525 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841626+0000", 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841626+0000", 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841628+0000", 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4610000000000001e-06 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841655+0000", 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6658000000000001e-05 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841682+0000", 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7674999999999998e-05 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841691+0000", 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5520000000000005e-06 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841931+0000", 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023983500000000001 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842058+0000", 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012696499999999999 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842192+0000", 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013453200000000001 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.845735+0000", 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035430750000000001 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.847002+0000", 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012665899999999999 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136972+0000", 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28997065100000002 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137022+0000", 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9577999999999999e-05 2024-09-21T11:55:44.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137077+0000", 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5274e-05 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47558 10.4 10:2e79000f:::benchmark_data_smithi043_150678_object47557:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.855768+0000", 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 150.50558571100001, 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28151058899999998, 2024-09-21T11:55:44.413 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47558 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855768+0000", 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855768+0000", 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855769+0000", 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.33e-06 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855788+0000", 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9289000000000001e-05 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855811+0000", 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3139e-05 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855819+0000", 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2819999999999999e-06 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855885+0000", 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5813000000000002e-05 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.856043+0000", 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015814 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.856194+0000", 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015090899999999999 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.859384+0000", 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031903259999999998 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860384+0000", 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00099984199999999992 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137145+0000", 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27676123800000002 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137217+0000", 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2330999999999996e-05 2024-09-21T11:55:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137278+0000", 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0949999999999998e-05 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47561 10.4 10:2c90ec34:::benchmark_data_smithi043_150678_object47560:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.857677+0000", 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 150.503676412, 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27975456399999998, 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.419 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47561 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857677+0000", 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857677+0000", 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857678+0000", 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.297e-06 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857699+0000", 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1055000000000001e-05 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857714+0000", 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4229e-05 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857721+0000", 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8399999999999995e-06 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857847+0000", 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125618 2024-09-21T11:55:44.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857956+0000", 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109215 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.858055+0000", 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8906999999999993e-05 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.859507+0000", 2024-09-21T11:55:44.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014520920000000001 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860490+0000", 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098292600000000007 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137335+0000", 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27684498899999999 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137393+0000", 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8255999999999999e-05 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137432+0000", 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8139999999999997e-05 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:67192 10.4 10:20483ca3:::benchmark_data_smithi043_150678_object67191:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:28.342058+0000", 2024-09-21T11:55:44.425 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 136.019295365, 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19868539599999999, 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 67192 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342058+0000", 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342058+0000", 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342060+0000", 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6700000000000001e-06 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342080+0000", 2024-09-21T11:55:44.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0582e-05 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342095+0000", 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4853e-05 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342102+0000", 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2069999999999998e-06 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342158+0000", 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599000000000001e-05 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342286+0000", 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012840200000000001 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342431+0000", 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014455499999999999 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.345136+0000", 2024-09-21T11:55:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027053469999999999 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.345250+0000", 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114228 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.540632+0000", 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.195381519 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.540694+0000", 2024-09-21T11:55:44.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1975000000000002e-05 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.540743+0000", 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9459000000000002e-05 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:85498 10.8 10:1841726b:::benchmark_data_smithi043_150678_object85497:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:44.793462+0000", 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.567891117, 2024-09-21T11:55:44.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28091800300000003, 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 85498 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793462+0000", 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793462+0000", 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793464+0000", 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.282e-06 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793488+0000", 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4277000000000002e-05 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793498+0000", 2024-09-21T11:55:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0564000000000001e-05 2024-09-21T11:55:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793512+0000", 2024-09-21T11:55:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3858e-05 2024-09-21T11:55:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793594+0000", 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1499e-05 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793742+0000", 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014823599999999999 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793877+0000", 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000135227 2024-09-21T11:55:44.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.797141+0000", 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032633900000000001 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.798058+0000", 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00091767599999999997 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074222+0000", 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27616407700000001 2024-09-21T11:55:44.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074310+0000", 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7566999999999997e-05 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074380+0000", 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0350000000000002e-05 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:85533 10.8 10:1dd7dbdf:::benchmark_data_smithi043_150678_object85532:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:44.822341+0000", 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 119.53901207299999, 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.25225375999999999, 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 85533 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822341+0000", 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822341+0000", 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822342+0000", 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.164e-06 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822370+0000", 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7681000000000001e-05 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822381+0000", 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1032999999999999e-05 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822388+0000", 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0400000000000004e-06 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822464+0000", 2024-09-21T11:55:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5916999999999998e-05 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822690+0000", 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225388 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822866+0000", 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017633899999999999 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.826112+0000", 2024-09-21T11:55:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032457430000000002 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.826939+0000", 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082743300000000003 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074462+0000", 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.24752277 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074555+0000", 2024-09-21T11:55:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2961000000000007e-05 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074595+0000", 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0290999999999998e-05 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126842 10.4 10:2c1c1280:::benchmark_data_smithi043_150678_object4091:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.884029+0000", 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.477324487000004, 2024-09-21T11:55:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.23360112199999999, 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126842 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884029+0000", 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884029+0000", 2024-09-21T11:55:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884030+0000", 2024-09-21T11:55:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4499999999999997e-07 2024-09-21T11:55:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884032+0000", 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8200000000000001e-06 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884039+0000", 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5910000000000001e-06 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884044+0000", 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3539999999999999e-06 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.887155+0000", 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031106319999999999 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.887284+0000", 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000128634 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.887406+0000", 2024-09-21T11:55:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012195 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.890430+0000", 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030242569999999998 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895372+0000", 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049420849999999997 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117520+0000", 2024-09-21T11:55:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.22214824699999999 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117582+0000", 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1618999999999994e-05 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117630+0000", 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8288000000000003e-05 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126858 10.4 10:2178d15c:::benchmark_data_smithi043_150678_object4107:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.895928+0000", 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.465425178000004, 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.22202257, 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126858 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895928+0000", 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895928+0000", 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895929+0000", 2024-09-21T11:55:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4700000000000001e-07 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895932+0000", 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4089999999999999e-06 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895941+0000", 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895948+0000", 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2639999999999997e-06 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.896006+0000", 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969e-05 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.896110+0000", 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010406 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:55:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.896213+0000", 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010307899999999999 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.899350+0000", 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031370899999999999 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.901791+0000", 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024410310000000002 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117743+0000", 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.215951699 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117903+0000", 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016066600000000001 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117951+0000", 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7432000000000003e-05 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126885 10.4 10:2892f7f4:::benchmark_data_smithi043_150678_object4134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.918908+0000", 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.442445853999999, 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20485952199999999, 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126885 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918908+0000", 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918908+0000", 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918909+0000", 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2750000000000001e-06 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918915+0000", 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7660000000000004e-06 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918928+0000", 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3444e-05 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918937+0000", 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8040000000000007e-06 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918996+0000", 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9425000000000001e-05 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.919146+0000", 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015016399999999999 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.919290+0000", 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000143234 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.921837+0000", 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025476990000000001 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.922942+0000", 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011045759999999999 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123550+0000", 2024-09-21T11:55:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.200607601 2024-09-21T11:55:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.461 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123718+0000", 2024-09-21T11:55:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000168708 2024-09-21T11:55:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123767+0000", 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8826000000000003e-05 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126898 10.4 10:2c034698:::benchmark_data_smithi043_150678_object4147:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.930744+0000", 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 78.430609755000006, 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19317583399999999, 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126898 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930744+0000", 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930744+0000", 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930745+0000", 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.499e-06 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930749+0000", 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2259999999999999e-06 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930763+0000", 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3512e-05 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930770+0000", 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2899999999999997e-06 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930833+0000", 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3257999999999996e-05 2024-09-21T11:55:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930950+0000", 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011681 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.931074+0000", 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012377699999999999 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.933746+0000", 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026722899999999999 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.934205+0000", 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00045867100000000001 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123819+0000", 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.18961380999999999 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123882+0000", 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2739999999999999e-05 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123919+0000", 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7951000000000003e-05 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:168906 10.8 10:1f344508:::benchmark_data_smithi043_150678_object46155:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e358)", 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:04.074915+0000", 2024-09-21T11:55:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 40.286438416999999, 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.167228132, 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 168906 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074915+0000", 2024-09-21T11:55:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074915+0000", 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074916+0000", 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4020000000000001e-06 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074921+0000", 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.865e-06 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074932+0000", 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0550000000000001e-05 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074939+0000", 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.571e-06 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.075042+0000", 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102817 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.075156+0000", 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113479 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.075301+0000", 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014525000000000001 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.077475+0000", 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021743520000000001 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.078271+0000", 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00079528200000000002 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.242030+0000", 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.16375980200000001 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.242090+0000", 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9790000000000002e-05 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.242143+0000", 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2972e-05 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:205288 10.8 10:1fa98cb4:::benchmark_data_smithi043_150678_object82537:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e369)", 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:40.458665+0000", 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 3.902688022, 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.042295259000000002, 2024-09-21T11:55:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 205288 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.458665+0000", 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.458665+0000", 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.458666+0000", 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2500000000000001e-07 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.458667+0000", 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.688e-06 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.458672+0000", 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5009999999999998e-06 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.458676+0000", 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1989999999999999e-06 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.480624+0000", 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.021947947999999998 2024-09-21T11:55:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.480730+0000", 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105634 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.480830+0000", 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100169 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.484769+0000", 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039391210000000003 2024-09-21T11:55:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.498850+0000", 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014080619000000001 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.500870+0000", 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020199509999999999 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.500919+0000", 2024-09-21T11:55:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9777000000000001e-05 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:40.500961+0000", 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1326999999999999e-05 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:208579 10.8 10:1744c1c0:::benchmark_data_smithi043_150678_object85828:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e369)", 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:43.452729+0000", 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.90862390000000004, 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.045077023000000001, 2024-09-21T11:55:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 208579 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.452729+0000", 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.452729+0000", 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.452730+0000", 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2800000000000002e-07 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.452732+0000", 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4229999999999999e-06 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.452738+0000", 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6300000000000003e-06 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.452743+0000", 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.848e-06 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.471600+0000", 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.018857133000000002 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.471711+0000", 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011110799999999999 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.471813+0000", 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010220799999999999 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.478681+0000", 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0068673989999999997 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.491488+0000", 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012807565 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.497706+0000", 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0062182210000000003 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.497763+0000", 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6725999999999999e-05 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:55:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:55:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:55:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:43.497807+0000", 2024-09-21T11:55:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3334000000000002e-05 2024-09-21T11:55:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:55:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:55:44.487 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:55:44.489 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:44.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:44.490 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:44.502 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:55:44.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:44.633 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:44.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:44.634 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:44.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/423749236' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:44.699 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/423749236' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:44.735 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:44.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/423749236' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:55:44.773 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:44.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:44.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:44.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:44.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:44.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:44.912 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:45.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:45.042 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:45.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:45.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:45.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:45.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:45.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:45.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:45.326 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:55:45.326 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:55:45.333 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:55:45.333 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:55:45.333 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-21T11:55:45.333 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T11:55:45.333 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T11:55:45.333 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-21T11:55:45.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:45.344 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:45.344 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:45.369 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:55:45.369 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:55:45.370 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:55:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:45.404 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:55:45.404 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:55:45.437 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:55:45.437 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:55:45.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:45.473 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:55:45.474 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:55:45.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:45.481 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:45.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:45.510 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:55:45.510 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:55:45.556 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:55:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:55:45.581 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:45.592 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:55:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:55:45.621 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:45.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:45.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:45.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:45.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-mon[29710]: pgmap v756: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s; 2408/118857 objects misplaced (2.026%); 17 MiB/s, 273 objects/s recovering 2024-09-21T11:55:45.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:45.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/423749236' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:45.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-mon[29710]: osdmap e370: 8 total, 8 up, 8 in 2024-09-21T11:55:45.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2332422053' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:45.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-mon[36454]: pgmap v756: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s; 2408/118857 objects misplaced (2.026%); 17 MiB/s, 273 objects/s recovering 2024-09-21T11:55:45.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:45.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/423749236' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:45.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-mon[36454]: osdmap e370: 8 total, 8 up, 8 in 2024-09-21T11:55:45.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2332422053' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:45.753 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:55:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:45 smithi175 ceph-mon[32613]: pgmap v756: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.02k op/s; 2408/118857 objects misplaced (2.026%); 17 MiB/s, 273 objects/s recovering 2024-09-21T11:55:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:45 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/423749236' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:55:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:45 smithi175 ceph-mon[32613]: osdmap e370: 8 total, 8 up, 8 in 2024-09-21T11:55:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2332422053' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:45.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:45.762 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:45.762 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:45.862 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:45.901 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:45.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:45.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:46.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:46.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:46.035 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:46.035 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:46.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:46.176 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:46.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:46.177 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:46.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:46.315 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:46.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:46.316 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:46.403 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:55:46.403 INFO:teuthology.orchestra.run.smithi175.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-21T11:55:46.403 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:55:46.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:46.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:46.455 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:46.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:46.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:46.593 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:46.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:46.594 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:46.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:46.735 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:46.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:46.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:46.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:46.871 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:46.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:46.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:46.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:47.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:47.013 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:47.114 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:47.143 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:47.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:47.144 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:47.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:47.283 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:47.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:47.284 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:47.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:47.417 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:47.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:47.418 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:47.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:47.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:47.542 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:47.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:47.664 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:47.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:47.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:47.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:47.794 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:47.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:47.795 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:47.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:47.918 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:47.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:47.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:47 smithi043 ceph-mon[29710]: pgmap v758: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.00k op/s; 872/110847 objects misplaced (0.787%); 20 MiB/s, 318 objects/s recovering 2024-09-21T11:55:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:47 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:55:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:47 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:47 smithi043 ceph-mon[29710]: osdmap e371: 8 total, 8 up, 8 in 2024-09-21T11:55:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:47 smithi043 ceph-mon[36454]: pgmap v758: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.00k op/s; 872/110847 objects misplaced (0.787%); 20 MiB/s, 318 objects/s recovering 2024-09-21T11:55:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:47 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:55:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:47 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:47 smithi043 ceph-mon[36454]: osdmap e371: 8 total, 8 up, 8 in 2024-09-21T11:55:48.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:47 smithi175 ceph-mon[32613]: pgmap v758: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.00k op/s; 872/110847 objects misplaced (0.787%); 20 MiB/s, 318 objects/s recovering 2024-09-21T11:55:48.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:47 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:55:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:47 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:55:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:47 smithi175 ceph-mon[32613]: osdmap e371: 8 total, 8 up, 8 in 2024-09-21T11:55:48.020 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:48.041 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:48.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:48.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:48.142 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:48.163 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:48.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:48.164 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:48.220 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:55:48.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:48.286 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:48.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:48.287 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:48.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:48.412 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:48.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:48.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:48.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:48.543 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:48.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:48.544 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:48.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:48.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:55:48.678 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:48.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:48.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:48.737 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:48.781 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:48.807 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:48.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:48.808 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:48.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:48.931 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:48.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:48.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:49.033 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:49.054 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:49.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:49.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:49.156 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:49.179 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:49.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:49.180 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:49.280 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:49.304 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:49.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:49.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:49.315 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:49.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:49.427 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:49.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:49.428 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:49.529 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:49.564 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:49.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:49.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:49.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:49.702 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:49.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:49.703 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:49.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:49 smithi043 ceph-mon[29710]: pgmap v760: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 872/102915 objects misplaced (0.847%); 20 MiB/s, 318 objects/s recovering 2024-09-21T11:55:49.718 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:49 smithi043 ceph-mon[29710]: osdmap e372: 8 total, 8 up, 8 in 2024-09-21T11:55:49.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:49 smithi043 ceph-mon[36454]: pgmap v760: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 872/102915 objects misplaced (0.847%); 20 MiB/s, 318 objects/s recovering 2024-09-21T11:55:49.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:49 smithi043 ceph-mon[36454]: osdmap e372: 8 total, 8 up, 8 in 2024-09-21T11:55:49.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:49 smithi175 ceph-mon[32613]: pgmap v760: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 872/102915 objects misplaced (0.847%); 20 MiB/s, 318 objects/s recovering 2024-09-21T11:55:49.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:49 smithi175 ceph-mon[32613]: osdmap e372: 8 total, 8 up, 8 in 2024-09-21T11:55:49.804 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:49.838 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:49.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:49.839 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:49.882 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:55:49.882 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:55:49.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:49.974 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:49.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:49.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:50.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:55:50.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:50.101 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:50.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:50.103 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:50.203 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:50.240 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:50.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:50.241 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:50.342 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:50.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:50.394 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:50.395 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:50.468 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:55:50.495 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:50.538 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:50.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:50.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:50.640 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:50.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:50.677 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:50.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:50.725 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:50 smithi043 ceph-mon[29710]: osdmap e373: 8 total, 8 up, 8 in 2024-09-21T11:55:50.725 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/229659013' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:50.725 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:50 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:50.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:50 smithi043 ceph-mon[36454]: osdmap e373: 8 total, 8 up, 8 in 2024-09-21T11:55:50.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/229659013' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:50.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:50 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:50.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:50 smithi175 ceph-mon[32613]: osdmap e373: 8 total, 8 up, 8 in 2024-09-21T11:55:50.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/229659013' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:50.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:50 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:55:50.778 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:50.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:55:50.807 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:50.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:50.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:50.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:50.942 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:50.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:50.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:51.043 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:51.073 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:51.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:51.074 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:51.175 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:51.209 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:51.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:51.210 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:51.311 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:51.345 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:51.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:51.346 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:51.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:51.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:51.485 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:51.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:51.626 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:51.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:51.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:51.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:51 smithi175 ceph-mon[32613]: pgmap v763: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 914 B/s rd, 0 B/s wr, 950 op/s; 872/102915 objects misplaced (0.847%); 15 MiB/s, 232 objects/s recovering 2024-09-21T11:55:51.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:51 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:51.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:51 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:55:51.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:51 smithi175 ceph-mon[32613]: osdmap e374: 8 total, 8 up, 8 in 2024-09-21T11:55:51.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:51 smithi175 ceph-mon[32613]: 6.8 scrub starts 2024-09-21T11:55:51.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:51 smithi175 ceph-mon[32613]: 6.8 scrub ok 2024-09-21T11:55:51.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:51.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[36454]: pgmap v763: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 914 B/s rd, 0 B/s wr, 950 op/s; 872/102915 objects misplaced (0.847%); 15 MiB/s, 232 objects/s recovering 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[36454]: osdmap e374: 8 total, 8 up, 8 in 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[36454]: 6.8 scrub starts 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[36454]: 6.8 scrub ok 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[29710]: pgmap v763: 148 pgs: 2 active+remapped+backfilling, 146 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 914 B/s rd, 0 B/s wr, 950 op/s; 872/102915 objects misplaced (0.847%); 15 MiB/s, 232 objects/s recovering 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[29710]: osdmap e374: 8 total, 8 up, 8 in 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[29710]: 6.8 scrub starts 2024-09-21T11:55:51.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:51 smithi043 ceph-mon[29710]: 6.8 scrub ok 2024-09-21T11:55:51.759 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:51.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:51.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:51.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:51.899 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:51.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:51.900 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:52.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:52.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:55:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:55:52.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:52.041 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:52.041 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:52.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:52.169 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:52.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:52.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:52.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:52.344 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:52.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:52.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:52.446 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:52.482 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:52.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:52.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:52.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:52.621 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:52.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:52.622 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:52.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[36454]: 5.14 scrub starts 2024-09-21T11:55:52.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[36454]: 5.14 scrub ok 2024-09-21T11:55:52.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[36454]: 6.3 scrub starts 2024-09-21T11:55:52.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[36454]: 6.3 scrub ok 2024-09-21T11:55:52.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:52.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[36454]: 5.11 scrub starts 2024-09-21T11:55:52.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[36454]: 5.11 scrub ok 2024-09-21T11:55:52.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[29710]: 5.14 scrub starts 2024-09-21T11:55:52.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[29710]: 5.14 scrub ok 2024-09-21T11:55:52.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[29710]: 6.3 scrub starts 2024-09-21T11:55:52.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[29710]: 6.3 scrub ok 2024-09-21T11:55:52.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:52.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[29710]: 5.11 scrub starts 2024-09-21T11:55:52.625 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:52 smithi043 ceph-mon[29710]: 5.11 scrub ok 2024-09-21T11:55:52.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:52.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:52 smithi175 ceph-mon[32613]: 5.14 scrub starts 2024-09-21T11:55:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:52 smithi175 ceph-mon[32613]: 5.14 scrub ok 2024-09-21T11:55:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:52 smithi175 ceph-mon[32613]: 6.3 scrub starts 2024-09-21T11:55:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:52 smithi175 ceph-mon[32613]: 6.3 scrub ok 2024-09-21T11:55:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:55:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:52 smithi175 ceph-mon[32613]: 5.11 scrub starts 2024-09-21T11:55:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:52 smithi175 ceph-mon[32613]: 5.11 scrub ok 2024-09-21T11:55:52.765 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:52.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:52.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:52.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:52.889 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:52.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:52.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:52.930 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:55:52.931 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:55:52.943 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:55:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:55:52.981 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:55:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:55:52.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:53.017 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:55:53.017 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:55:53.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:53.027 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:53.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:53.048 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:55:53.048 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:55:53.087 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:55:53.087 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:55:53.121 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:55:53.122 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:55:53.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:53.157 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:55:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:55:53.164 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:53.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:53.166 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:53.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:53.299 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:53.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:53.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:53.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:53.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:53.440 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:53.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:53.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:53.579 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:53.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:53.580 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:53.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:53.720 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:53.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:53.722 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:53.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:53 smithi043 ceph-mon[36454]: 6.1 scrub starts 2024-09-21T11:55:53.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:53 smithi043 ceph-mon[36454]: 6.1 scrub ok 2024-09-21T11:55:53.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:53 smithi043 ceph-mon[36454]: pgmap v765: 148 pgs: 148 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 870 op/s; 11 MiB/s, 178 objects/s recovering 2024-09-21T11:55:53.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:53 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1692115429' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:53 smithi043 ceph-mon[29710]: 6.1 scrub starts 2024-09-21T11:55:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:53 smithi043 ceph-mon[29710]: 6.1 scrub ok 2024-09-21T11:55:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:53 smithi043 ceph-mon[29710]: pgmap v765: 148 pgs: 148 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 870 op/s; 11 MiB/s, 178 objects/s recovering 2024-09-21T11:55:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:53 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1692115429' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:53.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:53 smithi175 ceph-mon[32613]: 6.1 scrub starts 2024-09-21T11:55:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:53 smithi175 ceph-mon[32613]: 6.1 scrub ok 2024-09-21T11:55:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:53 smithi175 ceph-mon[32613]: pgmap v765: 148 pgs: 148 active+clean; 1.9 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 870 op/s; 11 MiB/s, 178 objects/s recovering 2024-09-21T11:55:53.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:53 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1692115429' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:55:53.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:53.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:53.857 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:53.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:53.938 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:55:53.938 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:55:53.938 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-21T11:55:53.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:55:53.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:53.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:55:53.989 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:53.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:53.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:54.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:54.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:54.162 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:54.163 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:54.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:54.342 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:54.342 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:55:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:55:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:55:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:55:54.342 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:55:54.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:54.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:54.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:54.482 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:54.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:54.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:54.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:54.640 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:54.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:54.641 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:54 smithi043 ceph-mon[36454]: 4.f scrub starts 2024-09-21T11:55:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:54 smithi043 ceph-mon[36454]: 4.f scrub ok 2024-09-21T11:55:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:54 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T11:55:54.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:54 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T11:55:54.742 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:54 smithi043 ceph-mon[29710]: 4.f scrub starts 2024-09-21T11:55:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:54 smithi043 ceph-mon[29710]: 4.f scrub ok 2024-09-21T11:55:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:54 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T11:55:54.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:54 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T11:55:54.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:54 smithi175 ceph-mon[32613]: 4.f scrub starts 2024-09-21T11:55:54.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:54 smithi175 ceph-mon[32613]: 4.f scrub ok 2024-09-21T11:55:54.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:54 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T11:55:54.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:54 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T11:55:54.773 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:54.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:54.774 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:54.875 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:54.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:54.904 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:54.904 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:55.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:55.041 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:55.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:55.042 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:55.143 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:55.181 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:55.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:55.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:55.199 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:55:55.214 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:55.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:55.288 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:55:55.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:55.344 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:55.344 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:55.417 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:55:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:55:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:55:55.445 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:55.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:55.483 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:55.483 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:55.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:55.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:55.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:55.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:55.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:55 smithi043 ceph-mon[36454]: pgmap v766: 148 pgs: 148 active+clean; 1.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 661 op/s; 8.7 MiB/s, 138 objects/s recovering 2024-09-21T11:55:55.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:55 smithi043 ceph-mon[29710]: pgmap v766: 148 pgs: 148 active+clean; 1.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 661 op/s; 8.7 MiB/s, 138 objects/s recovering 2024-09-21T11:55:55.759 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:55.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:55.760 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:55.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:55 smithi175 ceph-mon[32613]: pgmap v766: 148 pgs: 148 active+clean; 1.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 661 op/s; 8.7 MiB/s, 138 objects/s recovering 2024-09-21T11:55:55.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:55.899 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:55.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:55.900 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:56.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:56.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:56.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:56.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:55:56.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:56.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:56.189 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:56.189 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:56.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:56.322 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:56.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:56.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:56.424 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:56.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:56.452 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:56.452 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:56.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:56.578 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:56.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:56.579 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:56.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:56.715 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:56.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:56.716 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:56.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:56 smithi043 ceph-mon[29710]: 10.9 scrub starts 2024-09-21T11:55:56.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:56 smithi043 ceph-mon[36454]: 10.9 scrub starts 2024-09-21T11:55:56.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:56 smithi175 ceph-mon[32613]: 10.9 scrub starts 2024-09-21T11:55:56.817 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:56.869 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:56.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:56.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:56.899 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:55:56.901 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:55:56.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:57.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:57.005 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:57.006 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:57.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:57.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:57.143 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:57.143 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:57.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:57.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:57.268 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:57.268 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:57.369 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:57.405 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:57.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:57.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:57.507 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:57.542 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:57.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:57.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:57.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:57.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:57.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:57 smithi175 ceph-mon[32613]: 3.4 scrub starts 2024-09-21T11:55:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:57 smithi175 ceph-mon[32613]: 3.4 scrub ok 2024-09-21T11:55:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:57 smithi175 ceph-mon[32613]: pgmap v767: 148 pgs: 148 active+clean; 1.7 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 668 B/s rd, 0 B/s wr, 924 op/s; 6.8 MiB/s, 108 objects/s recovering 2024-09-21T11:55:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:57 smithi175 ceph-mon[32613]: 10.9 scrub ok 2024-09-21T11:55:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:57 smithi175 ceph-mon[32613]: 5.1d scrub starts 2024-09-21T11:55:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:57 smithi175 ceph-mon[32613]: 5.1d scrub ok 2024-09-21T11:55:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:57 smithi175 ceph-mon[32613]: 5.1f scrub starts 2024-09-21T11:55:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:57 smithi175 ceph-mon[32613]: 5.1f scrub ok 2024-09-21T11:55:57.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:57.791 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[29710]: 3.4 scrub starts 2024-09-21T11:55:57.791 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[29710]: 3.4 scrub ok 2024-09-21T11:55:57.791 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[29710]: pgmap v767: 148 pgs: 148 active+clean; 1.7 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 668 B/s rd, 0 B/s wr, 924 op/s; 6.8 MiB/s, 108 objects/s recovering 2024-09-21T11:55:57.791 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[29710]: 10.9 scrub ok 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[29710]: 5.1d scrub starts 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[29710]: 5.1d scrub ok 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[29710]: 5.1f scrub starts 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[29710]: 5.1f scrub ok 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[36454]: 3.4 scrub starts 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[36454]: 3.4 scrub ok 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[36454]: pgmap v767: 148 pgs: 148 active+clean; 1.7 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 668 B/s rd, 0 B/s wr, 924 op/s; 6.8 MiB/s, 108 objects/s recovering 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[36454]: 10.9 scrub ok 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[36454]: 5.1d scrub starts 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[36454]: 5.1d scrub ok 2024-09-21T11:55:57.792 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[36454]: 5.1f scrub starts 2024-09-21T11:55:57.793 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:57 smithi043 ceph-mon[36454]: 5.1f scrub ok 2024-09-21T11:55:57.811 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:57.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:57.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:57.913 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:57.947 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:57.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:57.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:58.049 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:55:58.088 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:58.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:58.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:55:58.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:58.225 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:58.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:58.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:58.327 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:58.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:58.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:58.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:58.493 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:58.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:58.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:58.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:58.627 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:58.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:58.628 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:58.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:55:58.750 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:58.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:58.751 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:55:58.852 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:55:58.888 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:58.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:58.889 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:55:58.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:58 smithi043 ceph-mon[29710]: from='client.16428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:55:58.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:58 smithi043 ceph-mon[29710]: 3.1a scrub starts 2024-09-21T11:55:58.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:58 smithi043 ceph-mon[29710]: 3.1a scrub ok 2024-09-21T11:55:58.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:58 smithi043 ceph-mon[29710]: 3.1 scrub starts 2024-09-21T11:55:58.904 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:58 smithi043 ceph-mon[29710]: 3.1 scrub ok 2024-09-21T11:55:58.904 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:58 smithi043 ceph-mon[36454]: from='client.16428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:55:58.904 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:58 smithi043 ceph-mon[36454]: 3.1a scrub starts 2024-09-21T11:55:58.904 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:58 smithi043 ceph-mon[36454]: 3.1a scrub ok 2024-09-21T11:55:58.904 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:58 smithi043 ceph-mon[36454]: 3.1 scrub starts 2024-09-21T11:55:58.904 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:58 smithi043 ceph-mon[36454]: 3.1 scrub ok 2024-09-21T11:55:58.989 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:59.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:58 smithi175 ceph-mon[32613]: from='client.16428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:55:59.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:58 smithi175 ceph-mon[32613]: 3.1a scrub starts 2024-09-21T11:55:59.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:58 smithi175 ceph-mon[32613]: 3.1a scrub ok 2024-09-21T11:55:59.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:58 smithi175 ceph-mon[32613]: 3.1 scrub starts 2024-09-21T11:55:59.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:58 smithi175 ceph-mon[32613]: 3.1 scrub ok 2024-09-21T11:55:59.018 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:59.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:59.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:59.121 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:55:59.151 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:59.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:59.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:55:59.253 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:59.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:59.286 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:59.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:55:59.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:59.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:59.433 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:59.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:55:59.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:59.576 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:59.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:55:59.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:55:59.714 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:59.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:59.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:55:59.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:55:59.849 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:55:59.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:59.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:55:59.951 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:55:59.990 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:55:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:59 smithi043 ceph-mon[29710]: pgmap v768: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s; 6.0 MiB/s, 96 objects/s recovering 2024-09-21T11:55:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:59 smithi043 ceph-mon[29710]: 6.10 scrub starts 2024-09-21T11:55:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:55:59 smithi043 ceph-mon[29710]: 6.10 scrub ok 2024-09-21T11:55:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:59 smithi043 ceph-mon[36454]: pgmap v768: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s; 6.0 MiB/s, 96 objects/s recovering 2024-09-21T11:55:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:59 smithi043 ceph-mon[36454]: 6.10 scrub starts 2024-09-21T11:55:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:55:59 smithi043 ceph-mon[36454]: 6.10 scrub ok 2024-09-21T11:55:59.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:55:59.992 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:00.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:59 smithi175 ceph-mon[32613]: pgmap v768: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s; 6.0 MiB/s, 96 objects/s recovering 2024-09-21T11:56:00.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:59 smithi175 ceph-mon[32613]: 6.10 scrub starts 2024-09-21T11:56:00.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:55:59 smithi175 ceph-mon[32613]: 6.10 scrub ok 2024-09-21T11:56:00.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:00.129 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:00.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:00.130 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:00.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:00.254 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":767,"stamp":"2024-09-21T11:55:56.104225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1771706931,"num_objects":27225,"num_object_clones":0,"num_object_copies":81675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27225,"num_whiteouts":0,"num_read":2190,"num_read_kb":2047,"num_write":219307,"num_write_kb":7857546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5382,"num_bytes_recovered":334504749,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88940,"ondisk_log_size":88940,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":446,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":10008644,"kb_used_data":5382888,"kb_used_omap":12,"kb_used_meta":4625267,"kb_avail":739952572,"statfs":{"total":767960285184,"available":757711433728,"internally_reserved":0,"allocated":5512077312,"data_stored":5508986817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4736273970},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,6,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-463994880,"num_objects":-7080,"num_object_clones":0,"num_object_copies":-21240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-872,"num_objects_unfound":0,"num_objects_dirty":-7080,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7080,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":833,"num_bytes_recovered":54591488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.663813"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472239+0000","last_change":"2024-09-21T11:55:28.565270+0000","last_active":"2024-09-21T11:55:50.472239+0000","last_peered":"2024-09-21T11:55:50.472239+0000","last_clean":"2024-09-21T11:55:50.472239+0000","last_became_active":"2024-09-21T11:55:28.564399+0000","last_became_peered":"2024-09-21T11:55:28.564399+0000","last_unstale":"2024-09-21T11:55:50.472239+0000","last_undegraded":"2024-09-21T11:55:50.472239+0000","last_fullsized":"2024-09-21T11:55:50.472239+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.303555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.303555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:22.256231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":711,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475253+0000","last_change":"2024-09-21T11:54:27.283019+0000","last_active":"2024-09-21T11:55:50.475253+0000","last_peered":"2024-09-21T11:55:50.475253+0000","last_clean":"2024-09-21T11:55:50.475253+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:55:50.475253+0000","last_undegraded":"2024-09-21T11:55:50.475253+0000","last_fullsized":"2024-09-21T11:55:50.475253+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:27.282826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:27.282826+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":389,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.474807+0000","last_change":"2024-09-21T11:54:52.816248+0000","last_active":"2024-09-21T11:55:50.474807+0000","last_peered":"2024-09-21T11:55:50.474807+0000","last_clean":"2024-09-21T11:55:50.474807+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:55:50.474807+0000","last_undegraded":"2024-09-21T11:55:50.474807+0000","last_fullsized":"2024-09-21T11:55:50.474807+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:52.816156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:52.816156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:01.547204+0000","scrub_duration":226,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":484,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.470911+0000","last_change":"2024-09-21T11:51:43.528594+0000","last_active":"2024-09-21T11:55:50.470911+0000","last_peered":"2024-09-21T11:55:50.470911+0000","last_clean":"2024-09-21T11:55:50.470911+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:55:50.470911+0000","last_undegraded":"2024-09-21T11:55:50.470911+0000","last_fullsized":"2024-09-21T11:55:50.470911+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:13.165993+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":684,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469434+0000","last_change":"2024-09-21T11:54:53.491138+0000","last_active":"2024-09-21T11:55:50.469434+0000","last_peered":"2024-09-21T11:55:50.469434+0000","last_clean":"2024-09-21T11:55:50.469434+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:55:50.469434+0000","last_undegraded":"2024-09-21T11:55:50.469434+0000","last_fullsized":"2024-09-21T11:55:50.469434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.490941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.490941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:05.702659+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":673,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469175+0000","last_change":"2024-09-21T11:55:25.429625+0000","last_active":"2024-09-21T11:55:50.469175+0000","last_peered":"2024-09-21T11:55:50.469175+0000","last_clean":"2024-09-21T11:55:50.469175+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:55:50.469175+0000","last_undegraded":"2024-09-21T11:55:50.469175+0000","last_fullsized":"2024-09-21T11:55:50.469175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:25.429559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:25.429559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:49.460771+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":719,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.468814+0000","last_change":"2024-09-21T11:53:54.438887+0000","last_active":"2024-09-21T11:55:50.468814+0000","last_peered":"2024-09-21T11:55:50.468814+0000","last_clean":"2024-09-21T11:55:50.468814+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:55:50.468814+0000","last_undegraded":"2024-09-21T11:55:50.468814+0000","last_fullsized":"2024-09-21T11:55:50.468814+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:53:54.438798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:54.438798+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":519,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475337+0000","last_change":"2024-09-21T11:55:28.561709+0000","last_active":"2024-09-21T11:55:50.475337+0000","last_peered":"2024-09-21T11:55:50.475337+0000","last_clean":"2024-09-21T11:55:50.475337+0000","last_became_active":"2024-09-21T11:55:28.561338+0000","last_became_peered":"2024-09-21T11:55:28.561338+0000","last_unstale":"2024-09-21T11:55:50.475337+0000","last_undegraded":"2024-09-21T11:55:50.475337+0000","last_fullsized":"2024-09-21T11:55:50.475337+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:28.252642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:28.252642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":671,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473301+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:55:50.473301+0000","last_peered":"2024-09-21T11:55:50.473301+0000","last_clean":"2024-09-21T11:55:50.473301+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:55:50.473301+0000","last_undegraded":"2024-09-21T11:55:50.473301+0000","last_fullsized":"2024-09-21T11:55:50.473301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:17.087313+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":694,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.470161+0000","last_change":"2024-09-21T11:55:28.584595+0000","last_active":"2024-09-21T11:55:50.470161+0000","last_peered":"2024-09-21T11:55:50.470161+0000","last_clean":"2024-09-21T11:55:50.470161+0000","last_became_active":"2024-09-21T11:55:28.575920+0000","last_became_peered":"2024-09-21T11:55:28.575920+0000","last_unstale":"2024-09-21T11:55:50.470161+0000","last_undegraded":"2024-09-21T11:55:50.470161+0000","last_fullsized":"2024-09-21T11:55:50.470161+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:10.424192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:10.424192+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:38.749738+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":671,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473537+0000","last_change":"2024-09-21T11:55:05.096606+0000","last_active":"2024-09-21T11:55:50.473537+0000","last_peered":"2024-09-21T11:55:50.473537+0000","last_clean":"2024-09-21T11:55:50.473537+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:55:50.473537+0000","last_undegraded":"2024-09-21T11:55:50.473537+0000","last_fullsized":"2024-09-21T11:55:50.473537+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:05.096542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:05.096542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:27.440887+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":340,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475236+0000","last_change":"2024-09-21T11:55:28.567246+0000","last_active":"2024-09-21T11:55:50.475236+0000","last_peered":"2024-09-21T11:55:50.475236+0000","last_clean":"2024-09-21T11:55:50.475236+0000","last_became_active":"2024-09-21T11:55:28.566921+0000","last_became_peered":"2024-09-21T11:55:28.566921+0000","last_unstale":"2024-09-21T11:55:50.475236+0000","last_undegraded":"2024-09-21T11:55:50.475236+0000","last_fullsized":"2024-09-21T11:55:50.475236+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:00.276898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:00.276898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:05.231007+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":709,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.471446+0000","last_change":"2024-09-21T11:54:33.055621+0000","last_active":"2024-09-21T11:55:50.471446+0000","last_peered":"2024-09-21T11:55:50.471446+0000","last_clean":"2024-09-21T11:55:50.471446+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:55:50.471446+0000","last_undegraded":"2024-09-21T11:55:50.471446+0000","last_fullsized":"2024-09-21T11:55:50.471446+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:33.055554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:33.055554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:02.654956+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":683,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469733+0000","last_change":"2024-09-21T11:55:04.399234+0000","last_active":"2024-09-21T11:55:50.469733+0000","last_peered":"2024-09-21T11:55:50.469733+0000","last_clean":"2024-09-21T11:55:50.469733+0000","last_became_active":"2024-09-21T11:51:43.545191+0000","last_became_peered":"2024-09-21T11:51:43.545191+0000","last_unstale":"2024-09-21T11:55:50.469733+0000","last_undegraded":"2024-09-21T11:55:50.469733+0000","last_fullsized":"2024-09-21T11:55:50.469733+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'7","last_scrub_stamp":"2024-09-21T11:55:04.399020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:04.399020+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:23.894944+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":677,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.471860+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:55:50.471860+0000","last_peered":"2024-09-21T11:55:50.471860+0000","last_clean":"2024-09-21T11:55:50.471860+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:55:50.471860+0000","last_undegraded":"2024-09-21T11:55:50.471860+0000","last_fullsized":"2024-09-21T11:55:50.471860+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:11.590706+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":681,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473671+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:55:50.473671+0000","last_peered":"2024-09-21T11:55:50.473671+0000","last_clean":"2024-09-21T11:55:50.473671+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:55:50.473671+0000","last_undegraded":"2024-09-21T11:55:50.473671+0000","last_fullsized":"2024-09-21T11:55:50.473671+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:14.121214+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":183,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473089+0000","last_change":"2024-09-21T11:55:04.496118+0000","last_active":"2024-09-21T11:55:50.473089+0000","last_peered":"2024-09-21T11:55:50.473089+0000","last_clean":"2024-09-21T11:55:50.473089+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:55:50.473089+0000","last_undegraded":"2024-09-21T11:55:50.473089+0000","last_fullsized":"2024-09-21T11:55:50.473089+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:04.496040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:04.496040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:05.376248+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472130+0000","last_change":"2024-09-21T11:55:28.562731+0000","last_active":"2024-09-21T11:55:50.472130+0000","last_peered":"2024-09-21T11:55:50.472130+0000","last_clean":"2024-09-21T11:55:50.472130+0000","last_became_active":"2024-09-21T11:55:28.562057+0000","last_became_peered":"2024-09-21T11:55:28.562057+0000","last_unstale":"2024-09-21T11:55:50.472130+0000","last_undegraded":"2024-09-21T11:55:50.472130+0000","last_fullsized":"2024-09-21T11:55:50.472130+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:27.176320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:27.176320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:40.048471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":819,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.462829+0000","last_change":"2024-09-21T11:55:33.768876+0000","last_active":"2024-09-21T11:55:50.462829+0000","last_peered":"2024-09-21T11:55:50.462829+0000","last_clean":"2024-09-21T11:55:50.462829+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T11:55:50.462829+0000","last_undegraded":"2024-09-21T11:55:50.462829+0000","last_fullsized":"2024-09-21T11:55:50.462829+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:50:25.966740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.966740+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":381,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.474962+0000","last_change":"2024-09-21T11:53:23.118402+0000","last_active":"2024-09-21T11:55:50.474962+0000","last_peered":"2024-09-21T11:55:50.474962+0000","last_clean":"2024-09-21T11:55:50.474962+0000","last_became_active":"2024-09-21T11:47:03.377366+0000","last_became_peered":"2024-09-21T11:47:03.377366+0000","last_unstale":"2024-09-21T11:55:50.474962+0000","last_undegraded":"2024-09-21T11:55:50.474962+0000","last_fullsized":"2024-09-21T11:55:50.474962+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:23.118315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:23.118315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":677,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.468957+0000","last_change":"2024-09-21T11:55:07.393341+0000","last_active":"2024-09-21T11:55:50.468957+0000","last_peered":"2024-09-21T11:55:50.468957+0000","last_clean":"2024-09-21T11:55:50.468957+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:55:50.468957+0000","last_undegraded":"2024-09-21T11:55:50.468957+0000","last_fullsized":"2024-09-21T11:55:50.468957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:07.393249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:07.393249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:35.621347+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":496,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.476787+0000","last_change":"2024-09-21T11:54:02.517182+0000","last_active":"2024-09-21T11:55:50.476787+0000","last_peered":"2024-09-21T11:55:50.476787+0000","last_clean":"2024-09-21T11:55:50.476787+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:55:50.476787+0000","last_undegraded":"2024-09-21T11:55:50.476787+0000","last_fullsized":"2024-09-21T11:55:50.476787+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:02.517119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:02.517119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":746,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469591+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:55:50.469591+0000","last_peered":"2024-09-21T11:55:50.469591+0000","last_clean":"2024-09-21T11:55:50.469591+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:55:50.469591+0000","last_undegraded":"2024-09-21T11:55:50.469591+0000","last_fullsized":"2024-09-21T11:55:50.469591+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:04.849420+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":182,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469986+0000","last_change":"2024-09-21T11:53:55.486849+0000","last_active":"2024-09-21T11:55:50.469986+0000","last_peered":"2024-09-21T11:55:50.469986+0000","last_clean":"2024-09-21T11:55:50.469986+0000","last_became_active":"2024-09-21T11:51:43.548726+0000","last_became_peered":"2024-09-21T11:51:43.548726+0000","last_unstale":"2024-09-21T11:55:50.469986+0000","last_undegraded":"2024-09-21T11:55:50.469986+0000","last_fullsized":"2024-09-21T11:55:50.469986+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:55.486676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:55.486676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":185,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465245+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:55:50.465245+0000","last_peered":"2024-09-21T11:55:50.465245+0000","last_clean":"2024-09-21T11:55:50.465245+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:55:50.465245+0000","last_undegraded":"2024-09-21T11:55:50.465245+0000","last_fullsized":"2024-09-21T11:55:50.465245+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:18.213996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":396,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475003+0000","last_change":"2024-09-21T11:55:28.573969+0000","last_active":"2024-09-21T11:55:50.475003+0000","last_peered":"2024-09-21T11:55:50.475003+0000","last_clean":"2024-09-21T11:55:50.475003+0000","last_became_active":"2024-09-21T11:55:28.568598+0000","last_became_peered":"2024-09-21T11:55:28.568598+0000","last_unstale":"2024-09-21T11:55:50.475003+0000","last_undegraded":"2024-09-21T11:55:50.475003+0000","last_fullsized":"2024-09-21T11:55:50.475003+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:54:33.391425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:33.391425+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:56.634062+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":714,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465700+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:55:50.465700+0000","last_peered":"2024-09-21T11:55:50.465700+0000","last_clean":"2024-09-21T11:55:50.465700+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:55:50.465700+0000","last_undegraded":"2024-09-21T11:55:50.465700+0000","last_fullsized":"2024-09-21T11:55:50.465700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:11.327043+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":891,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.563611+0000","last_change":"2024-09-21T11:51:44.478918+0000","last_active":"2024-09-21T11:55:50.563611+0000","last_peered":"2024-09-21T11:55:50.563611+0000","last_clean":"2024-09-21T11:55:50.563611+0000","last_became_active":"2024-09-21T11:40:04.327681+0000","last_became_peered":"2024-09-21T11:40:04.327681+0000","last_unstale":"2024-09-21T11:55:50.563611+0000","last_undegraded":"2024-09-21T11:55:50.563611+0000","last_fullsized":"2024-09-21T11:55:50.563611+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:51:44.478855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:44.478855+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"374'13370","reported_seq":13572,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:51.244101+0000","last_change":"2024-09-21T11:53:57.419202+0000","last_active":"2024-09-21T11:55:51.244101+0000","last_peered":"2024-09-21T11:55:51.244101+0000","last_clean":"2024-09-21T11:55:51.244101+0000","last_became_active":"2024-09-21T11:52:45.608448+0000","last_became_peered":"2024-09-21T11:52:45.608448+0000","last_unstale":"2024-09-21T11:55:51.244101+0000","last_undegraded":"2024-09-21T11:55:51.244101+0000","last_fullsized":"2024-09-21T11:55:51.244101+0000","mapping_epoch":313,"log_start":"349'8319","ondisk_log_start":"349'8319","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'6082","last_scrub_stamp":"2024-09-21T11:53:57.418961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:57.418961+0000","objects_scrubbed":6033,"log_size":5051,"log_dups_size":0,"ondisk_log_size":5051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:41.553020+0000","scrub_duration":2364,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":1846,"num_object_clones":0,"num_object_copies":5538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13370,"num_write_kb":486912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":415,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475857+0000","last_change":"2024-09-21T11:52:39.116264+0000","last_active":"2024-09-21T11:55:50.475857+0000","last_peered":"2024-09-21T11:55:50.475857+0000","last_clean":"2024-09-21T11:55:50.475857+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:55:50.475857+0000","last_undegraded":"2024-09-21T11:55:50.475857+0000","last_fullsized":"2024-09-21T11:55:50.475857+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:52:39.116197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:52:39.116197+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":703,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469334+0000","last_change":"2024-09-21T11:53:30.353707+0000","last_active":"2024-09-21T11:55:50.469334+0000","last_peered":"2024-09-21T11:55:50.469334+0000","last_clean":"2024-09-21T11:55:50.469334+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:55:50.469334+0000","last_undegraded":"2024-09-21T11:55:50.469334+0000","last_fullsized":"2024-09-21T11:55:50.469334+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:30.353632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:30.353632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"366'10","reported_seq":270,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.476734+0000","last_change":"2024-09-21T11:55:28.951330+0000","last_active":"2024-09-21T11:55:50.476734+0000","last_peered":"2024-09-21T11:55:50.476734+0000","last_clean":"2024-09-21T11:55:50.476734+0000","last_became_active":"2024-09-21T11:55:28.575209+0000","last_became_peered":"2024-09-21T11:55:28.575209+0000","last_unstale":"2024-09-21T11:55:50.476734+0000","last_undegraded":"2024-09-21T11:55:50.476734+0000","last_fullsized":"2024-09-21T11:55:50.476734+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'9","last_scrub_stamp":"2024-09-21T11:55:28.951276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:28.951276+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:43.105898+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":712,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473457+0000","last_change":"2024-09-21T11:55:33.000584+0000","last_active":"2024-09-21T11:55:50.473457+0000","last_peered":"2024-09-21T11:55:50.473457+0000","last_clean":"2024-09-21T11:55:50.473457+0000","last_became_active":"2024-09-21T11:55:28.564378+0000","last_became_peered":"2024-09-21T11:55:28.564378+0000","last_unstale":"2024-09-21T11:55:50.473457+0000","last_undegraded":"2024-09-21T11:55:50.473457+0000","last_fullsized":"2024-09-21T11:55:50.473457+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:33.000521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:33.000521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:46.499959+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"374'13833","reported_seq":14038,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:55.739071+0000","last_change":"2024-09-21T11:54:02.094582+0000","last_active":"2024-09-21T11:55:55.739071+0000","last_peered":"2024-09-21T11:55:55.739071+0000","last_clean":"2024-09-21T11:55:55.739071+0000","last_became_active":"2024-09-21T11:52:45.606226+0000","last_became_peered":"2024-09-21T11:52:45.606226+0000","last_unstale":"2024-09-21T11:55:55.739071+0000","last_undegraded":"2024-09-21T11:55:55.739071+0000","last_fullsized":"2024-09-21T11:55:55.739071+0000","mapping_epoch":313,"log_start":"349'8248","ondisk_log_start":"349'8248","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"340'6367","last_scrub_stamp":"2024-09-21T11:54:02.094356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:02.094356+0000","objects_scrubbed":6327,"log_size":5585,"log_dups_size":0,"ondisk_log_size":5585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1665,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100597760,"num_objects":1535,"num_object_clones":0,"num_object_copies":4605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13833,"num_write_kb":491776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":768,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.468660+0000","last_change":"2024-09-21T11:55:28.585148+0000","last_active":"2024-09-21T11:55:50.468660+0000","last_peered":"2024-09-21T11:55:50.468660+0000","last_clean":"2024-09-21T11:55:50.468660+0000","last_became_active":"2024-09-21T11:55:28.563496+0000","last_became_peered":"2024-09-21T11:55:28.563496+0000","last_unstale":"2024-09-21T11:55:50.468660+0000","last_undegraded":"2024-09-21T11:55:50.468660+0000","last_fullsized":"2024-09-21T11:55:50.468660+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:55:05.451051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:05.451051+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:27.192714+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":697,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469234+0000","last_change":"2024-09-21T11:54:55.512377+0000","last_active":"2024-09-21T11:55:50.469234+0000","last_peered":"2024-09-21T11:55:50.469234+0000","last_clean":"2024-09-21T11:55:50.469234+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:55:50.469234+0000","last_undegraded":"2024-09-21T11:55:50.469234+0000","last_fullsized":"2024-09-21T11:55:50.469234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:55.512314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:55.512314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:09.527517+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":175,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.468603+0000","last_change":"2024-09-21T11:51:43.542323+0000","last_active":"2024-09-21T11:55:50.468603+0000","last_peered":"2024-09-21T11:55:50.468603+0000","last_clean":"2024-09-21T11:55:50.468603+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:55:50.468603+0000","last_undegraded":"2024-09-21T11:55:50.468603+0000","last_fullsized":"2024-09-21T11:55:50.468603+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:17.940335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:17.940335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":184,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.464077+0000","last_change":"2024-09-21T11:54:53.844954+0000","last_active":"2024-09-21T11:55:50.464077+0000","last_peered":"2024-09-21T11:55:50.464077+0000","last_clean":"2024-09-21T11:55:50.464077+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:55:50.464077+0000","last_undegraded":"2024-09-21T11:55:50.464077+0000","last_fullsized":"2024-09-21T11:55:50.464077+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.844876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.844876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:54.854366+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.f","version":"374'14115","reported_seq":14312,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:55.734072+0000","last_change":"2024-09-21T11:54:54.009954+0000","last_active":"2024-09-21T11:55:55.734072+0000","last_peered":"2024-09-21T11:55:55.734072+0000","last_clean":"2024-09-21T11:55:55.734072+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:55:55.734072+0000","last_undegraded":"2024-09-21T11:55:55.734072+0000","last_fullsized":"2024-09-21T11:55:55.734072+0000","mapping_epoch":313,"log_start":"349'8526","ondisk_log_start":"349'8526","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"356'10086","last_scrub_stamp":"2024-09-21T11:54:54.009664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:54.009664+0000","objects_scrubbed":5657,"log_size":5589,"log_dups_size":0,"ondisk_log_size":5589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:57.111381+0000","scrub_duration":1483,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103743488,"num_objects":1583,"num_object_clones":0,"num_object_copies":4749,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14115,"num_write_kb":502336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":686,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473764+0000","last_change":"2024-09-21T11:51:43.525478+0000","last_active":"2024-09-21T11:55:50.473764+0000","last_peered":"2024-09-21T11:55:50.473764+0000","last_clean":"2024-09-21T11:55:50.473764+0000","last_became_active":"2024-09-21T11:51:43.523852+0000","last_became_peered":"2024-09-21T11:51:43.523852+0000","last_unstale":"2024-09-21T11:55:50.473764+0000","last_undegraded":"2024-09-21T11:55:50.473764+0000","last_fullsized":"2024-09-21T11:55:50.473764+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:16.103621+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":370,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475064+0000","last_change":"2024-09-21T11:55:28.573493+0000","last_active":"2024-09-21T11:55:50.475064+0000","last_peered":"2024-09-21T11:55:50.475064+0000","last_clean":"2024-09-21T11:55:50.475064+0000","last_became_active":"2024-09-21T11:55:28.567133+0000","last_became_peered":"2024-09-21T11:55:28.567133+0000","last_unstale":"2024-09-21T11:55:50.475064+0000","last_undegraded":"2024-09-21T11:55:50.475064+0000","last_fullsized":"2024-09-21T11:55:50.475064+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:28.657261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.657261+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:47.626565+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":896,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.561579+0000","last_change":"2024-09-21T11:51:20.675165+0000","last_active":"2024-09-21T11:55:50.561579+0000","last_peered":"2024-09-21T11:55:50.561579+0000","last_clean":"2024-09-21T11:55:50.561579+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:55:50.561579+0000","last_undegraded":"2024-09-21T11:55:50.561579+0000","last_fullsized":"2024-09-21T11:55:50.561579+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:51:20.675101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.675101+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":679,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.471922+0000","last_change":"2024-09-21T11:52:14.010215+0000","last_active":"2024-09-21T11:55:50.471922+0000","last_peered":"2024-09-21T11:55:50.471922+0000","last_clean":"2024-09-21T11:55:50.471922+0000","last_became_active":"2024-09-21T11:40:06.488968+0000","last_became_peered":"2024-09-21T11:40:06.488968+0000","last_unstale":"2024-09-21T11:55:50.471922+0000","last_undegraded":"2024-09-21T11:55:50.471922+0000","last_fullsized":"2024-09-21T11:55:50.471922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:14.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:14.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"374'13446","reported_seq":13619,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:51.198596+0000","last_change":"2024-09-21T11:55:27.966176+0000","last_active":"2024-09-21T11:55:51.198596+0000","last_peered":"2024-09-21T11:55:51.198596+0000","last_clean":"2024-09-21T11:55:51.198596+0000","last_became_active":"2024-09-21T11:52:45.608219+0000","last_became_peered":"2024-09-21T11:52:45.608219+0000","last_unstale":"2024-09-21T11:55:51.198596+0000","last_undegraded":"2024-09-21T11:55:51.198596+0000","last_fullsized":"2024-09-21T11:55:51.198596+0000","mapping_epoch":313,"log_start":"350'8347","ondisk_log_start":"350'8347","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"364'12047","last_scrub_stamp":"2024-09-21T11:55:27.966084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:55:27.966084+0000","objects_scrubbed":3259,"log_size":5099,"log_dups_size":0,"ondisk_log_size":5099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:27.977871+0000","scrub_duration":394,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121896960,"num_objects":1860,"num_object_clones":0,"num_object_copies":5580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13446,"num_write_kb":489792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":409,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.477062+0000","last_change":"2024-09-21T11:55:01.632777+0000","last_active":"2024-09-21T11:55:50.477062+0000","last_peered":"2024-09-21T11:55:50.477062+0000","last_clean":"2024-09-21T11:55:50.477062+0000","last_became_active":"2024-09-21T11:47:03.378579+0000","last_became_peered":"2024-09-21T11:47:03.378579+0000","last_unstale":"2024-09-21T11:55:50.477062+0000","last_undegraded":"2024-09-21T11:55:50.477062+0000","last_fullsized":"2024-09-21T11:55:50.477062+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:55:01.631695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.631695+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:04.702899+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":712,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473386+0000","last_change":"2024-09-21T11:54:22.095001+0000","last_active":"2024-09-21T11:55:50.473386+0000","last_peered":"2024-09-21T11:55:50.473386+0000","last_clean":"2024-09-21T11:55:50.473386+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:55:50.473386+0000","last_undegraded":"2024-09-21T11:55:50.473386+0000","last_fullsized":"2024-09-21T11:55:50.473386+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:22.094918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:22.094918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:50.020352+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":882,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465609+0000","last_change":"2024-09-21T11:55:26.222904+0000","last_active":"2024-09-21T11:55:50.465609+0000","last_peered":"2024-09-21T11:55:50.465609+0000","last_clean":"2024-09-21T11:55:50.465609+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:55:50.465609+0000","last_undegraded":"2024-09-21T11:55:50.465609+0000","last_fullsized":"2024-09-21T11:55:50.465609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:26.222823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:26.222823+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:27.737656+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":684,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465868+0000","last_change":"2024-09-21T11:53:48.256119+0000","last_active":"2024-09-21T11:55:50.465868+0000","last_peered":"2024-09-21T11:55:50.465868+0000","last_clean":"2024-09-21T11:55:50.465868+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:55:50.465868+0000","last_undegraded":"2024-09-21T11:55:50.465868+0000","last_fullsized":"2024-09-21T11:55:50.465868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:48.255951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:48.255951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:20.226002+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"374'13621","reported_seq":13756,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:55.708415+0000","last_change":"2024-09-21T11:52:45.607195+0000","last_active":"2024-09-21T11:55:55.708415+0000","last_peered":"2024-09-21T11:55:55.708415+0000","last_clean":"2024-09-21T11:55:55.708415+0000","last_became_active":"2024-09-21T11:52:45.606865+0000","last_became_peered":"2024-09-21T11:52:45.606865+0000","last_unstale":"2024-09-21T11:55:55.708415+0000","last_undegraded":"2024-09-21T11:55:55.708415+0000","last_fullsized":"2024-09-21T11:55:55.708415+0000","mapping_epoch":313,"log_start":"348'7970","ondisk_log_start":"348'7970","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5651,"log_dups_size":0,"ondisk_log_size":5651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97124376,"num_objects":1483,"num_object_clones":0,"num_object_copies":4449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13621,"num_write_kb":483265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":743,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465659+0000","last_change":"2024-09-21T11:55:08.281432+0000","last_active":"2024-09-21T11:55:50.465659+0000","last_peered":"2024-09-21T11:55:50.465659+0000","last_clean":"2024-09-21T11:55:50.465659+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:55:50.465659+0000","last_undegraded":"2024-09-21T11:55:50.465659+0000","last_fullsized":"2024-09-21T11:55:50.465659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:55:08.281339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:08.281339+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:27.455274+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":374,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.476022+0000","last_change":"2024-09-21T11:53:27.160558+0000","last_active":"2024-09-21T11:55:50.476022+0000","last_peered":"2024-09-21T11:55:50.476022+0000","last_clean":"2024-09-21T11:55:50.476022+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:55:50.476022+0000","last_undegraded":"2024-09-21T11:55:50.476022+0000","last_fullsized":"2024-09-21T11:55:50.476022+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.160486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.160486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":672,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472105+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:55:50.472105+0000","last_peered":"2024-09-21T11:55:50.472105+0000","last_clean":"2024-09-21T11:55:50.472105+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:55:50.472105+0000","last_undegraded":"2024-09-21T11:55:50.472105+0000","last_fullsized":"2024-09-21T11:55:50.472105+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":374,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.474707+0000","last_change":"2024-09-21T11:52:09.412149+0000","last_active":"2024-09-21T11:55:50.474707+0000","last_peered":"2024-09-21T11:55:50.474707+0000","last_clean":"2024-09-21T11:55:50.474707+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:55:50.474707+0000","last_undegraded":"2024-09-21T11:55:50.474707+0000","last_fullsized":"2024-09-21T11:55:50.474707+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:09.412091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:09.412091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"374'13703","reported_seq":18800,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:55.728006+0000","last_change":"2024-09-21T11:55:49.480866+0000","last_active":"2024-09-21T11:55:55.728006+0000","last_peered":"2024-09-21T11:55:55.728006+0000","last_clean":"2024-09-21T11:55:55.728006+0000","last_became_active":"2024-09-21T11:55:49.448578+0000","last_became_peered":"2024-09-21T11:55:49.448578+0000","last_unstale":"2024-09-21T11:55:55.728006+0000","last_undegraded":"2024-09-21T11:55:55.728006+0000","last_fullsized":"2024-09-21T11:55:55.728006+0000","mapping_epoch":372,"log_start":"348'8088","ondisk_log_start":"348'8088","created":313,"last_epoch_clean":373,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5615,"log_dups_size":0,"ondisk_log_size":5615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99942400,"num_objects":1525,"num_object_clones":0,"num_object_copies":4575,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13703,"num_write_kb":487296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2473,"num_bytes_recovered":162070528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":512,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475978+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:55:50.475978+0000","last_peered":"2024-09-21T11:55:50.475978+0000","last_clean":"2024-09-21T11:55:50.475978+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:55:50.475978+0000","last_undegraded":"2024-09-21T11:55:50.475978+0000","last_fullsized":"2024-09-21T11:55:50.475978+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:17.628254+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":199,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465697+0000","last_change":"2024-09-21T11:54:52.312620+0000","last_active":"2024-09-21T11:55:50.465697+0000","last_peered":"2024-09-21T11:55:50.465697+0000","last_clean":"2024-09-21T11:55:50.465697+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:55:50.465697+0000","last_undegraded":"2024-09-21T11:55:50.465697+0000","last_fullsized":"2024-09-21T11:55:50.465697+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:54:52.310060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:52.310060+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:21.355022+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":704,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473740+0000","last_change":"2024-09-21T11:51:12.562026+0000","last_active":"2024-09-21T11:55:50.473740+0000","last_peered":"2024-09-21T11:55:50.473740+0000","last_clean":"2024-09-21T11:55:50.473740+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:55:50.473740+0000","last_undegraded":"2024-09-21T11:55:50.473740+0000","last_fullsized":"2024-09-21T11:55:50.473740+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:51:12.561961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:51:12.561961+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:21.069629+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":345,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.474558+0000","last_change":"2024-09-21T11:55:28.562691+0000","last_active":"2024-09-21T11:55:50.474558+0000","last_peered":"2024-09-21T11:55:50.474558+0000","last_clean":"2024-09-21T11:55:50.474558+0000","last_became_active":"2024-09-21T11:55:28.562061+0000","last_became_peered":"2024-09-21T11:55:28.562061+0000","last_unstale":"2024-09-21T11:55:50.474558+0000","last_undegraded":"2024-09-21T11:55:50.474558+0000","last_fullsized":"2024-09-21T11:55:50.474558+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:49.587631+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":661,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.468973+0000","last_change":"2024-09-21T11:54:34.452033+0000","last_active":"2024-09-21T11:55:50.468973+0000","last_peered":"2024-09-21T11:55:50.468973+0000","last_clean":"2024-09-21T11:55:50.468973+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:55:50.468973+0000","last_undegraded":"2024-09-21T11:55:50.468973+0000","last_fullsized":"2024-09-21T11:55:50.468973+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:34.451939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:34.451939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:53.805403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"374'13728","reported_seq":13942,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:51.407517+0000","last_change":"2024-09-21T11:54:31.986720+0000","last_active":"2024-09-21T11:55:51.407517+0000","last_peered":"2024-09-21T11:55:51.407517+0000","last_clean":"2024-09-21T11:55:51.407517+0000","last_became_active":"2024-09-21T11:52:45.604600+0000","last_became_peered":"2024-09-21T11:52:45.604600+0000","last_unstale":"2024-09-21T11:55:51.407517+0000","last_undegraded":"2024-09-21T11:55:51.407517+0000","last_fullsized":"2024-09-21T11:55:51.407517+0000","mapping_epoch":313,"log_start":"348'8277","ondisk_log_start":"348'8277","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"349'8491","last_scrub_stamp":"2024-09-21T11:54:31.986575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:31.986575+0000","objects_scrubbed":7209,"log_size":5451,"log_dups_size":0,"ondisk_log_size":5451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124649472,"num_objects":1902,"num_object_clones":0,"num_object_copies":5706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13728,"num_write_kb":500160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":336,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.462974+0000","last_change":"2024-09-21T11:55:28.581193+0000","last_active":"2024-09-21T11:55:50.462974+0000","last_peered":"2024-09-21T11:55:50.462974+0000","last_clean":"2024-09-21T11:55:50.462974+0000","last_became_active":"2024-09-21T11:55:28.569095+0000","last_became_peered":"2024-09-21T11:55:28.569095+0000","last_unstale":"2024-09-21T11:55:50.462974+0000","last_undegraded":"2024-09-21T11:55:50.462974+0000","last_fullsized":"2024-09-21T11:55:50.462974+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:51:41.006622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:41.006622+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":386,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.474233+0000","last_change":"2024-09-21T11:55:06.628254+0000","last_active":"2024-09-21T11:55:50.474233+0000","last_peered":"2024-09-21T11:55:50.474233+0000","last_clean":"2024-09-21T11:55:50.474233+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:55:50.474233+0000","last_undegraded":"2024-09-21T11:55:50.474233+0000","last_fullsized":"2024-09-21T11:55:50.474233+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:55:06.628029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:06.628029+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:14.321228+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":714,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.463549+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:55:50.463549+0000","last_peered":"2024-09-21T11:55:50.463549+0000","last_clean":"2024-09-21T11:55:50.463549+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:55:50.463549+0000","last_undegraded":"2024-09-21T11:55:50.463549+0000","last_fullsized":"2024-09-21T11:55:50.463549+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":681,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.463845+0000","last_change":"2024-09-21T11:55:06.871048+0000","last_active":"2024-09-21T11:55:50.463845+0000","last_peered":"2024-09-21T11:55:50.463845+0000","last_clean":"2024-09-21T11:55:50.463845+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:55:50.463845+0000","last_undegraded":"2024-09-21T11:55:50.463845+0000","last_fullsized":"2024-09-21T11:55:50.463845+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:06.870710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:06.870710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:28.922671+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":676,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472551+0000","last_change":"2024-09-21T11:54:33.253430+0000","last_active":"2024-09-21T11:55:50.472551+0000","last_peered":"2024-09-21T11:55:50.472551+0000","last_clean":"2024-09-21T11:55:50.472551+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:55:50.472551+0000","last_undegraded":"2024-09-21T11:55:50.472551+0000","last_fullsized":"2024-09-21T11:55:50.472551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:33.253368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:33.253368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:58.365025+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"374'13516","reported_seq":13714,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:52.443285+0000","last_change":"2024-09-21T11:54:57.284460+0000","last_active":"2024-09-21T11:55:52.443285+0000","last_peered":"2024-09-21T11:55:52.443285+0000","last_clean":"2024-09-21T11:55:52.443285+0000","last_became_active":"2024-09-21T11:52:45.604244+0000","last_became_peered":"2024-09-21T11:52:45.604244+0000","last_unstale":"2024-09-21T11:55:52.443285+0000","last_undegraded":"2024-09-21T11:55:52.443285+0000","last_fullsized":"2024-09-21T11:55:52.443285+0000","mapping_epoch":313,"log_start":"342'6800","ondisk_log_start":"342'6800","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"357'10040","last_scrub_stamp":"2024-09-21T11:54:57.284313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:57.284313+0000","objects_scrubbed":5333,"log_size":6716,"log_dups_size":0,"ondisk_log_size":6716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:19.005227+0000","scrub_duration":2019,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":1792,"num_object_clones":0,"num_object_copies":5376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13516,"num_write_kb":489856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":568,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475150+0000","last_change":"2024-09-21T11:55:28.581087+0000","last_active":"2024-09-21T11:55:50.475150+0000","last_peered":"2024-09-21T11:55:50.475150+0000","last_clean":"2024-09-21T11:55:50.475150+0000","last_became_active":"2024-09-21T11:55:28.566613+0000","last_became_peered":"2024-09-21T11:55:28.566613+0000","last_unstale":"2024-09-21T11:55:50.475150+0000","last_undegraded":"2024-09-21T11:55:50.475150+0000","last_fullsized":"2024-09-21T11:55:50.475150+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:55:10.349240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:10.349240+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:33.498867+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":703,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.467045+0000","last_change":"2024-09-21T11:55:28.572360+0000","last_active":"2024-09-21T11:55:50.467045+0000","last_peered":"2024-09-21T11:55:50.467045+0000","last_clean":"2024-09-21T11:55:50.467045+0000","last_became_active":"2024-09-21T11:55:28.572037+0000","last_became_peered":"2024-09-21T11:55:28.572037+0000","last_unstale":"2024-09-21T11:55:50.467045+0000","last_undegraded":"2024-09-21T11:55:50.467045+0000","last_fullsized":"2024-09-21T11:55:50.467045+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:32.060638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:32.060638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1672,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.474109+0000","last_change":"2024-09-21T11:54:38.270801+0000","last_active":"2024-09-21T11:55:50.474109+0000","last_peered":"2024-09-21T11:55:50.474109+0000","last_clean":"2024-09-21T11:55:50.474109+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:55:50.474109+0000","last_undegraded":"2024-09-21T11:55:50.474109+0000","last_fullsized":"2024-09-21T11:55:50.474109+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:54:38.270748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.270748+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:51.553375+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":941,"num_read_kb":936,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":694,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472149+0000","last_change":"2024-09-21T11:55:24.998887+0000","last_active":"2024-09-21T11:55:50.472149+0000","last_peered":"2024-09-21T11:55:50.472149+0000","last_clean":"2024-09-21T11:55:50.472149+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:55:50.472149+0000","last_undegraded":"2024-09-21T11:55:50.472149+0000","last_fullsized":"2024-09-21T11:55:50.472149+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.998737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.998737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:45.936303+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":382,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475854+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:55:50.475854+0000","last_peered":"2024-09-21T11:55:50.475854+0000","last_clean":"2024-09-21T11:55:50.475854+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:55:50.475854+0000","last_undegraded":"2024-09-21T11:55:50.475854+0000","last_fullsized":"2024-09-21T11:55:50.475854+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:15.701086+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"374'13663","reported_seq":13797,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:55.728166+0000","last_change":"2024-09-21T11:52:45.607096+0000","last_active":"2024-09-21T11:55:55.728166+0000","last_peered":"2024-09-21T11:55:55.728166+0000","last_clean":"2024-09-21T11:55:55.728166+0000","last_became_active":"2024-09-21T11:52:45.606777+0000","last_became_peered":"2024-09-21T11:52:45.606777+0000","last_unstale":"2024-09-21T11:55:55.728166+0000","last_undegraded":"2024-09-21T11:55:55.728166+0000","last_fullsized":"2024-09-21T11:55:55.728166+0000","mapping_epoch":313,"log_start":"348'8068","ondisk_log_start":"348'8068","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5595,"log_dups_size":0,"ondisk_log_size":5595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100597760,"num_objects":1535,"num_object_clones":0,"num_object_copies":4605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13663,"num_write_kb":486336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":916,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.563273+0000","last_change":"2024-09-21T11:54:35.232137+0000","last_active":"2024-09-21T11:55:50.563273+0000","last_peered":"2024-09-21T11:55:50.563273+0000","last_clean":"2024-09-21T11:55:50.563273+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:55:50.563273+0000","last_undegraded":"2024-09-21T11:55:50.563273+0000","last_fullsized":"2024-09-21T11:55:50.563273+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:54:35.231924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:35.231924+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":689,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.470308+0000","last_change":"2024-09-21T11:55:29.461801+0000","last_active":"2024-09-21T11:55:50.470308+0000","last_peered":"2024-09-21T11:55:50.470308+0000","last_clean":"2024-09-21T11:55:50.470308+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:55:50.470308+0000","last_undegraded":"2024-09-21T11:55:50.470308+0000","last_fullsized":"2024-09-21T11:55:50.470308+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:29.461745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:29.461745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:40.699332+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":764,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.474230+0000","last_change":"2024-09-21T11:55:28.581579+0000","last_active":"2024-09-21T11:55:50.474230+0000","last_peered":"2024-09-21T11:55:50.474230+0000","last_clean":"2024-09-21T11:55:50.474230+0000","last_became_active":"2024-09-21T11:55:28.570670+0000","last_became_peered":"2024-09-21T11:55:28.570670+0000","last_unstale":"2024-09-21T11:55:50.474230+0000","last_undegraded":"2024-09-21T11:55:50.474230+0000","last_fullsized":"2024-09-21T11:55:50.474230+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:54:22.360410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:54:22.360410+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":680,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472575+0000","last_change":"2024-09-21T11:51:18.318095+0000","last_active":"2024-09-21T11:55:50.472575+0000","last_peered":"2024-09-21T11:55:50.472575+0000","last_clean":"2024-09-21T11:55:50.472575+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:55:50.472575+0000","last_undegraded":"2024-09-21T11:55:50.472575+0000","last_fullsized":"2024-09-21T11:55:50.472575+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.317950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.317950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:21.581023+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":684,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.471500+0000","last_change":"2024-09-21T11:54:58.013361+0000","last_active":"2024-09-21T11:55:50.471500+0000","last_peered":"2024-09-21T11:55:50.471500+0000","last_clean":"2024-09-21T11:55:50.471500+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:55:50.471500+0000","last_undegraded":"2024-09-21T11:55:50.471500+0000","last_fullsized":"2024-09-21T11:55:50.471500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:58.013113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:58.013113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:02.912267+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"374'13975","reported_seq":14176,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:55.732208+0000","last_change":"2024-09-21T11:53:53.487635+0000","last_active":"2024-09-21T11:55:55.732208+0000","last_peered":"2024-09-21T11:55:55.732208+0000","last_clean":"2024-09-21T11:55:55.732208+0000","last_became_active":"2024-09-21T11:52:45.605575+0000","last_became_peered":"2024-09-21T11:52:45.605575+0000","last_unstale":"2024-09-21T11:55:55.732208+0000","last_undegraded":"2024-09-21T11:55:55.732208+0000","last_fullsized":"2024-09-21T11:55:55.732208+0000","mapping_epoch":313,"log_start":"349'8404","ondisk_log_start":"349'8404","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'5948","last_scrub_stamp":"2024-09-21T11:53:53.487473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:53.487473+0000","objects_scrubbed":5891,"log_size":5571,"log_dups_size":0,"ondisk_log_size":5571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:17.287595+0000","scrub_duration":1988,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":99549184,"num_objects":1519,"num_object_clones":0,"num_object_copies":4557,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13975,"num_write_kb":495808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":697,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473138+0000","last_change":"2024-09-21T11:53:58.427767+0000","last_active":"2024-09-21T11:55:50.473138+0000","last_peered":"2024-09-21T11:55:50.473138+0000","last_clean":"2024-09-21T11:55:50.473138+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:55:50.473138+0000","last_undegraded":"2024-09-21T11:55:50.473138+0000","last_fullsized":"2024-09-21T11:55:50.473138+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:58.427708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:58.427708+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":182,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.463441+0000","last_change":"2024-09-21T11:52:13.709151+0000","last_active":"2024-09-21T11:55:50.463441+0000","last_peered":"2024-09-21T11:55:50.463441+0000","last_clean":"2024-09-21T11:55:50.463441+0000","last_became_active":"2024-09-21T11:51:43.532027+0000","last_became_peered":"2024-09-21T11:51:43.532027+0000","last_unstale":"2024-09-21T11:55:50.463441+0000","last_undegraded":"2024-09-21T11:55:50.463441+0000","last_fullsized":"2024-09-21T11:55:50.463441+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:13.709074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:52:13.709074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":672,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.471404+0000","last_change":"2024-09-21T11:55:08.075348+0000","last_active":"2024-09-21T11:55:50.471404+0000","last_peered":"2024-09-21T11:55:50.471404+0000","last_clean":"2024-09-21T11:55:50.471404+0000","last_became_active":"2024-09-21T11:40:04.329210+0000","last_became_peered":"2024-09-21T11:40:04.329210+0000","last_unstale":"2024-09-21T11:55:50.471404+0000","last_undegraded":"2024-09-21T11:55:50.471404+0000","last_fullsized":"2024-09-21T11:55:50.471404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:08.075277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:08.075277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:13.181057+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":683,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469938+0000","last_change":"2024-09-21T11:53:52.437233+0000","last_active":"2024-09-21T11:55:50.469938+0000","last_peered":"2024-09-21T11:55:50.469938+0000","last_clean":"2024-09-21T11:55:50.469938+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:55:50.469938+0000","last_undegraded":"2024-09-21T11:55:50.469938+0000","last_fullsized":"2024-09-21T11:55:50.469938+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:52.437166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:52.437166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"374'13380","reported_seq":13584,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:51.701214+0000","last_change":"2024-09-21T11:54:00.488606+0000","last_active":"2024-09-21T11:55:51.701214+0000","last_peered":"2024-09-21T11:55:51.701214+0000","last_clean":"2024-09-21T11:55:51.701214+0000","last_became_active":"2024-09-21T11:52:45.637825+0000","last_became_peered":"2024-09-21T11:52:45.637825+0000","last_unstale":"2024-09-21T11:55:51.701214+0000","last_undegraded":"2024-09-21T11:55:51.701214+0000","last_fullsized":"2024-09-21T11:55:51.701214+0000","mapping_epoch":313,"log_start":"348'7732","ondisk_log_start":"348'7732","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"339'6224","last_scrub_stamp":"2024-09-21T11:54:00.488535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.488535+0000","objects_scrubbed":6155,"log_size":5648,"log_dups_size":0,"ondisk_log_size":5648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2345,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":116785152,"num_objects":1782,"num_object_clones":0,"num_object_copies":5346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13380,"num_write_kb":485184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":785,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.462817+0000","last_change":"2024-09-21T11:55:28.579501+0000","last_active":"2024-09-21T11:55:50.462817+0000","last_peered":"2024-09-21T11:55:50.462817+0000","last_clean":"2024-09-21T11:55:50.462817+0000","last_became_active":"2024-09-21T11:55:28.561575+0000","last_became_peered":"2024-09-21T11:55:28.561575+0000","last_unstale":"2024-09-21T11:55:50.462817+0000","last_undegraded":"2024-09-21T11:55:50.462817+0000","last_fullsized":"2024-09-21T11:55:50.462817+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:50:36.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.385902+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":710,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472899+0000","last_change":"2024-09-21T11:54:36.208783+0000","last_active":"2024-09-21T11:55:50.472899+0000","last_peered":"2024-09-21T11:55:50.472899+0000","last_clean":"2024-09-21T11:55:50.472899+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:55:50.472899+0000","last_undegraded":"2024-09-21T11:55:50.472899+0000","last_fullsized":"2024-09-21T11:55:50.472899+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:36.208720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:36.208720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":381,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475772+0000","last_change":"2024-09-21T11:55:10.605948+0000","last_active":"2024-09-21T11:55:50.475772+0000","last_peered":"2024-09-21T11:55:50.475772+0000","last_clean":"2024-09-21T11:55:50.475772+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:55:50.475772+0000","last_undegraded":"2024-09-21T11:55:50.475772+0000","last_fullsized":"2024-09-21T11:55:50.475772+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:10.605869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:10.605869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:11.530111+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":685,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.463097+0000","last_change":"2024-09-21T11:55:24.583429+0000","last_active":"2024-09-21T11:55:50.463097+0000","last_peered":"2024-09-21T11:55:50.463097+0000","last_clean":"2024-09-21T11:55:50.463097+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:55:50.463097+0000","last_undegraded":"2024-09-21T11:55:50.463097+0000","last_fullsized":"2024-09-21T11:55:50.463097+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.583208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.583208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:40.880319+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"374'13546","reported_seq":13746,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:51.451520+0000","last_change":"2024-09-21T11:53:53.011097+0000","last_active":"2024-09-21T11:55:51.451520+0000","last_peered":"2024-09-21T11:55:51.451520+0000","last_clean":"2024-09-21T11:55:51.451520+0000","last_became_active":"2024-09-21T11:52:45.601350+0000","last_became_peered":"2024-09-21T11:52:45.601350+0000","last_unstale":"2024-09-21T11:55:51.451520+0000","last_undegraded":"2024-09-21T11:55:51.451520+0000","last_fullsized":"2024-09-21T11:55:51.451520+0000","mapping_epoch":313,"log_start":"350'8646","ondisk_log_start":"350'8646","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"338'5855","last_scrub_stamp":"2024-09-21T11:53:53.011008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:53:53.011008+0000","objects_scrubbed":5813,"log_size":4900,"log_dups_size":0,"ondisk_log_size":4900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:16.674654+0000","scrub_duration":1614,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":122290176,"num_objects":1866,"num_object_clones":0,"num_object_copies":5598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13546,"num_write_kb":493184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":674,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.470192+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:55:50.470192+0000","last_peered":"2024-09-21T11:55:50.470192+0000","last_clean":"2024-09-21T11:55:50.470192+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:55:50.470192+0000","last_undegraded":"2024-09-21T11:55:50.470192+0000","last_fullsized":"2024-09-21T11:55:50.470192+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:21.451083+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":680,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469899+0000","last_change":"2024-09-21T11:55:28.568637+0000","last_active":"2024-09-21T11:55:50.469899+0000","last_peered":"2024-09-21T11:55:50.469899+0000","last_clean":"2024-09-21T11:55:50.469899+0000","last_became_active":"2024-09-21T11:55:28.568312+0000","last_became_peered":"2024-09-21T11:55:28.568312+0000","last_unstale":"2024-09-21T11:55:50.469899+0000","last_undegraded":"2024-09-21T11:55:50.469899+0000","last_fullsized":"2024-09-21T11:55:50.469899+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:22.315149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:22.315149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":888,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.560756+0000","last_change":"2024-09-21T11:53:31.991392+0000","last_active":"2024-09-21T11:55:50.560756+0000","last_peered":"2024-09-21T11:55:50.560756+0000","last_clean":"2024-09-21T11:55:50.560756+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:55:50.560756+0000","last_undegraded":"2024-09-21T11:55:50.560756+0000","last_fullsized":"2024-09-21T11:55:50.560756+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:53:31.991316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:31.991316+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":501,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475470+0000","last_change":"2024-09-21T11:55:07.658109+0000","last_active":"2024-09-21T11:55:50.475470+0000","last_peered":"2024-09-21T11:55:50.475470+0000","last_clean":"2024-09-21T11:55:50.475470+0000","last_became_active":"2024-09-21T11:51:43.543072+0000","last_became_peered":"2024-09-21T11:51:43.543072+0000","last_unstale":"2024-09-21T11:55:50.475470+0000","last_undegraded":"2024-09-21T11:55:50.475470+0000","last_fullsized":"2024-09-21T11:55:50.475470+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:07.658022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:07.658022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:27.233153+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.5","version":"374'13441","reported_seq":13575,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:51.699689+0000","last_change":"2024-09-21T11:52:45.635909+0000","last_active":"2024-09-21T11:55:51.699689+0000","last_peered":"2024-09-21T11:55:51.699689+0000","last_clean":"2024-09-21T11:55:51.699689+0000","last_became_active":"2024-09-21T11:52:45.635590+0000","last_became_peered":"2024-09-21T11:52:45.635590+0000","last_unstale":"2024-09-21T11:55:51.699689+0000","last_undegraded":"2024-09-21T11:55:51.699689+0000","last_fullsized":"2024-09-21T11:55:51.699689+0000","mapping_epoch":313,"log_start":"348'7843","ondisk_log_start":"348'7843","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:52:44.568274+0000","objects_scrubbed":0,"log_size":5598,"log_dups_size":0,"ondisk_log_size":5598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119209984,"num_objects":1819,"num_object_clones":0,"num_object_copies":5457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13441,"num_write_kb":488320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":727,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465585+0000","last_change":"2024-09-21T11:55:07.300834+0000","last_active":"2024-09-21T11:55:50.465585+0000","last_peered":"2024-09-21T11:55:50.465585+0000","last_clean":"2024-09-21T11:55:50.465585+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:55:50.465585+0000","last_undegraded":"2024-09-21T11:55:50.465585+0000","last_fullsized":"2024-09-21T11:55:50.465585+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:07.300758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:07.300758+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:36.342250+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":712,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472885+0000","last_change":"2024-09-21T11:55:34.034149+0000","last_active":"2024-09-21T11:55:50.472885+0000","last_peered":"2024-09-21T11:55:50.472885+0000","last_clean":"2024-09-21T11:55:50.472885+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:55:50.472885+0000","last_undegraded":"2024-09-21T11:55:50.472885+0000","last_fullsized":"2024-09-21T11:55:50.472885+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:34.034076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:34.034076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:51.437569+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":673,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.464167+0000","last_change":"2024-09-21T11:53:26.990691+0000","last_active":"2024-09-21T11:55:50.464167+0000","last_peered":"2024-09-21T11:55:50.464167+0000","last_clean":"2024-09-21T11:55:50.464167+0000","last_became_active":"2024-09-21T11:40:04.329726+0000","last_became_peered":"2024-09-21T11:40:04.329726+0000","last_unstale":"2024-09-21T11:55:50.464167+0000","last_undegraded":"2024-09-21T11:55:50.464167+0000","last_fullsized":"2024-09-21T11:55:50.464167+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.990626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.990626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":39,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475547+0000","last_change":"2024-09-21T11:55:35.532779+0000","last_active":"2024-09-21T11:55:50.475547+0000","last_peered":"2024-09-21T11:55:50.475547+0000","last_clean":"2024-09-21T11:55:50.475547+0000","last_became_active":"2024-09-21T11:55:28.566108+0000","last_became_peered":"2024-09-21T11:55:28.566108+0000","last_unstale":"2024-09-21T11:55:50.475547+0000","last_undegraded":"2024-09-21T11:55:50.475547+0000","last_fullsized":"2024-09-21T11:55:50.475547+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:35.532577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:35.532577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:37.885219+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"374'13824","reported_seq":19268,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:55.740541+0000","last_change":"2024-09-21T11:55:48.469988+0000","last_active":"2024-09-21T11:55:55.740541+0000","last_peered":"2024-09-21T11:55:55.740541+0000","last_clean":"2024-09-21T11:55:55.740541+0000","last_became_active":"2024-09-21T11:55:48.447302+0000","last_became_peered":"2024-09-21T11:55:48.447302+0000","last_unstale":"2024-09-21T11:55:55.740541+0000","last_undegraded":"2024-09-21T11:55:55.740541+0000","last_fullsized":"2024-09-21T11:55:55.740541+0000","mapping_epoch":371,"log_start":"351'8817","ondisk_log_start":"351'8817","created":313,"last_epoch_clean":372,"parent":"0.0","parent_split_bits":0,"last_scrub":"340'6353","last_scrub_stamp":"2024-09-21T11:54:01.780225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.780225+0000","objects_scrubbed":6266,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2578,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103153664,"num_objects":1574,"num_object_clones":0,"num_object_copies":4722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13824,"num_write_kb":492736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2613,"num_bytes_recovered":171245568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":793,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472347+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:55:50.472347+0000","last_peered":"2024-09-21T11:55:50.472347+0000","last_clean":"2024-09-21T11:55:50.472347+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:55:50.472347+0000","last_undegraded":"2024-09-21T11:55:50.472347+0000","last_fullsized":"2024-09-21T11:55:50.472347+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:15.354715+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":680,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473228+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:55:50.473228+0000","last_peered":"2024-09-21T11:55:50.473228+0000","last_clean":"2024-09-21T11:55:50.473228+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:55:50.473228+0000","last_undegraded":"2024-09-21T11:55:50.473228+0000","last_fullsized":"2024-09-21T11:55:50.473228+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:49.576280+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":689,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.466956+0000","last_change":"2024-09-21T11:51:48.941202+0000","last_active":"2024-09-21T11:55:50.466956+0000","last_peered":"2024-09-21T11:55:50.466956+0000","last_clean":"2024-09-21T11:55:50.466956+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:55:50.466956+0000","last_undegraded":"2024-09-21T11:55:50.466956+0000","last_fullsized":"2024-09-21T11:55:50.466956+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:48.941128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:48.941128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:21.610005+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":684,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.647537+0000","last_change":"2024-09-21T11:55:50.647537+0000","last_active":"2024-09-21T11:55:50.647537+0000","last_peered":"2024-09-21T11:55:50.647537+0000","last_clean":"2024-09-21T11:55:50.647537+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:55:50.647537+0000","last_undegraded":"2024-09-21T11:55:50.647537+0000","last_fullsized":"2024-09-21T11:55:50.647537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:50.647374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:50.647374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:53.911019+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"374'13915","reported_seq":14111,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:55.722632+0000","last_change":"2024-09-21T11:54:55.485474+0000","last_active":"2024-09-21T11:55:55.722632+0000","last_peered":"2024-09-21T11:55:55.722632+0000","last_clean":"2024-09-21T11:55:55.722632+0000","last_became_active":"2024-09-21T11:52:45.636672+0000","last_became_peered":"2024-09-21T11:52:45.636672+0000","last_unstale":"2024-09-21T11:55:55.722632+0000","last_undegraded":"2024-09-21T11:55:55.722632+0000","last_fullsized":"2024-09-21T11:55:55.722632+0000","mapping_epoch":313,"log_start":"349'8277","ondisk_log_start":"349'8277","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"357'10034","last_scrub_stamp":"2024-09-21T11:54:55.485108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:54:55.485108+0000","objects_scrubbed":5494,"log_size":5638,"log_dups_size":0,"ondisk_log_size":5638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:10.702275+0000","scrub_duration":1931,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101384192,"num_objects":1547,"num_object_clones":0,"num_object_copies":4641,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13915,"num_write_kb":494784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":714,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.474897+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:55:50.474897+0000","last_peered":"2024-09-21T11:55:50.474897+0000","last_clean":"2024-09-21T11:55:50.474897+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:55:50.474897+0000","last_undegraded":"2024-09-21T11:55:50.474897+0000","last_fullsized":"2024-09-21T11:55:50.474897+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:11.603969+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":718,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473400+0000","last_change":"2024-09-21T11:53:58.230600+0000","last_active":"2024-09-21T11:55:50.473400+0000","last_peered":"2024-09-21T11:55:50.473400+0000","last_clean":"2024-09-21T11:55:50.473400+0000","last_became_active":"2024-09-21T11:47:03.386179+0000","last_became_peered":"2024-09-21T11:47:03.386179+0000","last_unstale":"2024-09-21T11:55:50.473400+0000","last_undegraded":"2024-09-21T11:55:50.473400+0000","last_fullsized":"2024-09-21T11:55:50.473400+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:58.230513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:58.230513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:40.547123+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":703,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.462335+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:55:50.462335+0000","last_peered":"2024-09-21T11:55:50.462335+0000","last_clean":"2024-09-21T11:55:50.462335+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:55:50.462335+0000","last_undegraded":"2024-09-21T11:55:50.462335+0000","last_fullsized":"2024-09-21T11:55:50.462335+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":683,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469555+0000","last_change":"2024-09-21T11:51:09.660956+0000","last_active":"2024-09-21T11:55:50.469555+0000","last_peered":"2024-09-21T11:55:50.469555+0000","last_clean":"2024-09-21T11:55:50.469555+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:55:50.469555+0000","last_undegraded":"2024-09-21T11:55:50.469555+0000","last_fullsized":"2024-09-21T11:55:50.469555+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:09.660880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:09.660880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"374'13396","reported_seq":13635,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:51.444503+0000","last_change":"2024-09-21T11:55:31.963410+0000","last_active":"2024-09-21T11:55:51.444503+0000","last_peered":"2024-09-21T11:55:51.444503+0000","last_clean":"2024-09-21T11:55:51.444503+0000","last_became_active":"2024-09-21T11:52:45.603101+0000","last_became_peered":"2024-09-21T11:52:45.603101+0000","last_unstale":"2024-09-21T11:55:51.444503+0000","last_undegraded":"2024-09-21T11:55:51.444503+0000","last_fullsized":"2024-09-21T11:55:51.444503+0000","mapping_epoch":313,"log_start":"348'8017","ondisk_log_start":"348'8017","created":313,"last_epoch_clean":314,"parent":"0.0","parent_split_bits":0,"last_scrub":"366'12172","last_scrub_stamp":"2024-09-21T11:55:31.963108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:52:44.568274+0000","last_clean_scrub_stamp":"2024-09-21T11:55:31.963108+0000","objects_scrubbed":3116,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:32.899257+0000","scrub_duration":1041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121765888,"num_objects":1858,"num_object_clones":0,"num_object_copies":5574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13396,"num_write_kb":488128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":727,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.470436+0000","last_change":"2024-09-21T11:55:01.019421+0000","last_active":"2024-09-21T11:55:50.470436+0000","last_peered":"2024-09-21T11:55:50.470436+0000","last_clean":"2024-09-21T11:55:50.470436+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:55:50.470436+0000","last_undegraded":"2024-09-21T11:55:50.470436+0000","last_fullsized":"2024-09-21T11:55:50.470436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:55:01.019348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.019348+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:13.582637+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":729,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473747+0000","last_change":"2024-09-21T11:55:34.552520+0000","last_active":"2024-09-21T11:55:50.473747+0000","last_peered":"2024-09-21T11:55:50.473747+0000","last_clean":"2024-09-21T11:55:50.473747+0000","last_became_active":"2024-09-21T11:55:28.561948+0000","last_became_peered":"2024-09-21T11:55:28.561948+0000","last_unstale":"2024-09-21T11:55:50.473747+0000","last_undegraded":"2024-09-21T11:55:50.473747+0000","last_fullsized":"2024-09-21T11:55:50.473747+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:55:34.552437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:34.552437+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:01.097111+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":879,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475766+0000","last_change":"2024-09-21T11:54:23.346794+0000","last_active":"2024-09-21T11:55:50.475766+0000","last_peered":"2024-09-21T11:55:50.475766+0000","last_clean":"2024-09-21T11:55:50.475766+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:55:50.475766+0000","last_undegraded":"2024-09-21T11:55:50.475766+0000","last_fullsized":"2024-09-21T11:55:50.475766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:54:23.346727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.346727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":668,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.471003+0000","last_change":"2024-09-21T11:53:27.992553+0000","last_active":"2024-09-21T11:55:50.471003+0000","last_peered":"2024-09-21T11:55:50.471003+0000","last_clean":"2024-09-21T11:55:50.471003+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:55:50.471003+0000","last_undegraded":"2024-09-21T11:55:50.471003+0000","last_fullsized":"2024-09-21T11:55:50.471003+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.992489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.992489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:40.984476+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":710,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475498+0000","last_change":"2024-09-21T11:55:28.557481+0000","last_active":"2024-09-21T11:55:50.475498+0000","last_peered":"2024-09-21T11:55:50.475498+0000","last_clean":"2024-09-21T11:55:50.475498+0000","last_became_active":"2024-09-21T11:55:28.557190+0000","last_became_peered":"2024-09-21T11:55:28.557190+0000","last_unstale":"2024-09-21T11:55:50.475498+0000","last_undegraded":"2024-09-21T11:55:50.475498+0000","last_fullsized":"2024-09-21T11:55:50.475498+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:38.208809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.208809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:08.071556+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":716,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469138+0000","last_change":"2024-09-21T11:54:02.489875+0000","last_active":"2024-09-21T11:55:50.469138+0000","last_peered":"2024-09-21T11:55:50.469138+0000","last_clean":"2024-09-21T11:55:50.469138+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:55:50.469138+0000","last_undegraded":"2024-09-21T11:55:50.469138+0000","last_fullsized":"2024-09-21T11:55:50.469138+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:54:02.489798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:02.489798+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":709,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.476781+0000","last_change":"2024-09-21T11:54:32.067297+0000","last_active":"2024-09-21T11:55:50.476781+0000","last_peered":"2024-09-21T11:55:50.476781+0000","last_clean":"2024-09-21T11:55:50.476781+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:55:50.476781+0000","last_undegraded":"2024-09-21T11:55:50.476781+0000","last_fullsized":"2024-09-21T11:55:50.476781+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:32.067168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:32.067168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":683,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472453+0000","last_change":"2024-09-21T11:53:26.139062+0000","last_active":"2024-09-21T11:55:50.472453+0000","last_peered":"2024-09-21T11:55:50.472453+0000","last_clean":"2024-09-21T11:55:50.472453+0000","last_became_active":"2024-09-21T11:40:06.488126+0000","last_became_peered":"2024-09-21T11:40:06.488126+0000","last_unstale":"2024-09-21T11:55:50.472453+0000","last_undegraded":"2024-09-21T11:55:50.472453+0000","last_fullsized":"2024-09-21T11:55:50.472453+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.138988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.138988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":717,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465748+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:55:50.465748+0000","last_peered":"2024-09-21T11:55:50.465748+0000","last_clean":"2024-09-21T11:55:50.465748+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:55:50.465748+0000","last_undegraded":"2024-09-21T11:55:50.465748+0000","last_fullsized":"2024-09-21T11:55:50.465748+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:16.276458+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":679,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.475913+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:55:50.475913+0000","last_peered":"2024-09-21T11:55:50.475913+0000","last_clean":"2024-09-21T11:55:50.475913+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:55:50.475913+0000","last_undegraded":"2024-09-21T11:55:50.475913+0000","last_fullsized":"2024-09-21T11:55:50.475913+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:22.581352+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":702,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469051+0000","last_change":"2024-09-21T11:55:03.415840+0000","last_active":"2024-09-21T11:55:50.469051+0000","last_peered":"2024-09-21T11:55:50.469051+0000","last_clean":"2024-09-21T11:55:50.469051+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:55:50.469051+0000","last_undegraded":"2024-09-21T11:55:50.469051+0000","last_fullsized":"2024-09-21T11:55:50.469051+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:03.415774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:03.415774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:05.969592+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":676,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.462104+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:55:50.462104+0000","last_peered":"2024-09-21T11:55:50.462104+0000","last_clean":"2024-09-21T11:55:50.462104+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:55:50.462104+0000","last_undegraded":"2024-09-21T11:55:50.462104+0000","last_fullsized":"2024-09-21T11:55:50.462104+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:10.810579+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":708,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472276+0000","last_change":"2024-09-21T11:54:54.999030+0000","last_active":"2024-09-21T11:55:50.472276+0000","last_peered":"2024-09-21T11:55:50.472276+0000","last_clean":"2024-09-21T11:55:50.472276+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:55:50.472276+0000","last_undegraded":"2024-09-21T11:55:50.472276+0000","last_fullsized":"2024-09-21T11:55:50.472276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:54.998955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:54.998955+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:22.083273+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":703,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.466107+0000","last_change":"2024-09-21T11:55:04.304207+0000","last_active":"2024-09-21T11:55:50.466107+0000","last_peered":"2024-09-21T11:55:50.466107+0000","last_clean":"2024-09-21T11:55:50.466107+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:55:50.466107+0000","last_undegraded":"2024-09-21T11:55:50.466107+0000","last_fullsized":"2024-09-21T11:55:50.466107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:04.304136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:04.304136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:30.701698+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":864,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:51.317638+0000","last_change":"2024-09-21T11:55:51.317564+0000","last_active":"2024-09-21T11:55:51.317638+0000","last_peered":"2024-09-21T11:55:51.317638+0000","last_clean":"2024-09-21T11:55:51.317638+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:55:51.317638+0000","last_undegraded":"2024-09-21T11:55:51.317638+0000","last_fullsized":"2024-09-21T11:55:51.317638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:51.317473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.317473+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:12.961860+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":174,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.462999+0000","last_change":"2024-09-21T11:51:43.531318+0000","last_active":"2024-09-21T11:55:50.462999+0000","last_peered":"2024-09-21T11:55:50.462999+0000","last_clean":"2024-09-21T11:55:50.462999+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:55:50.462999+0000","last_undegraded":"2024-09-21T11:55:50.462999+0000","last_fullsized":"2024-09-21T11:55:50.462999+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.826834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.826834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:14.770427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":736,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.470265+0000","last_change":"2024-09-21T11:53:25.264037+0000","last_active":"2024-09-21T11:55:50.470265+0000","last_peered":"2024-09-21T11:55:50.470265+0000","last_clean":"2024-09-21T11:55:50.470265+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:55:50.470265+0000","last_undegraded":"2024-09-21T11:55:50.470265+0000","last_fullsized":"2024-09-21T11:55:50.470265+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:25.263955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:25.263955+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":679,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472266+0000","last_change":"2024-09-21T11:54:35.264828+0000","last_active":"2024-09-21T11:55:50.472266+0000","last_peered":"2024-09-21T11:55:50.472266+0000","last_clean":"2024-09-21T11:55:50.472266+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:55:50.472266+0000","last_undegraded":"2024-09-21T11:55:50.472266+0000","last_fullsized":"2024-09-21T11:55:50.472266+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:35.264755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:35.264755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:02.953376+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":909,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472113+0000","last_change":"2024-09-21T11:55:26.971522+0000","last_active":"2024-09-21T11:55:50.472113+0000","last_peered":"2024-09-21T11:55:50.472113+0000","last_clean":"2024-09-21T11:55:50.472113+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:55:50.472113+0000","last_undegraded":"2024-09-21T11:55:50.472113+0000","last_fullsized":"2024-09-21T11:55:50.472113+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:55:26.971440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:26.971440+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:44.656785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":39,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472688+0000","last_change":"2024-09-21T11:55:33.546350+0000","last_active":"2024-09-21T11:55:50.472688+0000","last_peered":"2024-09-21T11:55:50.472688+0000","last_clean":"2024-09-21T11:55:50.472688+0000","last_became_active":"2024-09-21T11:55:28.565700+0000","last_became_peered":"2024-09-21T11:55:28.565700+0000","last_unstale":"2024-09-21T11:55:50.472688+0000","last_undegraded":"2024-09-21T11:55:50.472688+0000","last_fullsized":"2024-09-21T11:55:50.472688+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:33.546264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:33.546264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:57.443968+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":724,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472636+0000","last_change":"2024-09-21T11:51:43.533694+0000","last_active":"2024-09-21T11:55:50.472636+0000","last_peered":"2024-09-21T11:55:50.472636+0000","last_clean":"2024-09-21T11:55:50.472636+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:55:50.472636+0000","last_undegraded":"2024-09-21T11:55:50.472636+0000","last_fullsized":"2024-09-21T11:55:50.472636+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:14.386783+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":190,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.472709+0000","last_change":"2024-09-21T11:55:24.538954+0000","last_active":"2024-09-21T11:55:50.472709+0000","last_peered":"2024-09-21T11:55:50.472709+0000","last_clean":"2024-09-21T11:55:50.472709+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:55:50.472709+0000","last_undegraded":"2024-09-21T11:55:50.472709+0000","last_fullsized":"2024-09-21T11:55:50.472709+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.538880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.538880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:33.029522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":381,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.476989+0000","last_change":"2024-09-21T11:54:35.426101+0000","last_active":"2024-09-21T11:55:50.476989+0000","last_peered":"2024-09-21T11:55:50.476989+0000","last_clean":"2024-09-21T11:55:50.476989+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:55:50.476989+0000","last_undegraded":"2024-09-21T11:55:50.476989+0000","last_fullsized":"2024-09-21T11:55:50.476989+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:35.426030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:35.426030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:52.101778+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":675,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.468457+0000","last_change":"2024-09-21T11:55:28.563670+0000","last_active":"2024-09-21T11:55:50.468457+0000","last_peered":"2024-09-21T11:55:50.468457+0000","last_clean":"2024-09-21T11:55:50.468457+0000","last_became_active":"2024-09-21T11:55:28.563340+0000","last_became_peered":"2024-09-21T11:55:28.563340+0000","last_unstale":"2024-09-21T11:55:50.468457+0000","last_undegraded":"2024-09-21T11:55:50.468457+0000","last_fullsized":"2024-09-21T11:55:50.468457+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:01.434123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.434123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:04.638785+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":392,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.477208+0000","last_change":"2024-09-21T11:54:38.447060+0000","last_active":"2024-09-21T11:55:50.477208+0000","last_peered":"2024-09-21T11:55:50.477208+0000","last_clean":"2024-09-21T11:55:50.477208+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:55:50.477208+0000","last_undegraded":"2024-09-21T11:55:50.477208+0000","last_fullsized":"2024-09-21T11:55:50.477208+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:38.446684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.446684+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":679,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473049+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:55:50.473049+0000","last_peered":"2024-09-21T11:55:50.473049+0000","last_clean":"2024-09-21T11:55:50.473049+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:55:50.473049+0000","last_undegraded":"2024-09-21T11:55:50.473049+0000","last_fullsized":"2024-09-21T11:55:50.473049+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:13.620163+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":686,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469652+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:55:50.469652+0000","last_peered":"2024-09-21T11:55:50.469652+0000","last_clean":"2024-09-21T11:55:50.469652+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:55:50.469652+0000","last_undegraded":"2024-09-21T11:55:50.469652+0000","last_fullsized":"2024-09-21T11:55:50.469652+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:19.480627+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":706,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465372+0000","last_change":"2024-09-21T11:54:31.029959+0000","last_active":"2024-09-21T11:55:50.465372+0000","last_peered":"2024-09-21T11:55:50.465372+0000","last_clean":"2024-09-21T11:55:50.465372+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:55:50.465372+0000","last_undegraded":"2024-09-21T11:55:50.465372+0000","last_fullsized":"2024-09-21T11:55:50.465372+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:54:31.029780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:31.029780+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":715,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.468634+0000","last_change":"2024-09-21T11:53:32.308905+0000","last_active":"2024-09-21T11:55:50.468634+0000","last_peered":"2024-09-21T11:55:50.468634+0000","last_clean":"2024-09-21T11:55:50.468634+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:55:50.468634+0000","last_undegraded":"2024-09-21T11:55:50.468634+0000","last_fullsized":"2024-09-21T11:55:50.468634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:32.308827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:32.308827+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":701,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.476860+0000","last_change":"2024-09-21T11:53:28.945615+0000","last_active":"2024-09-21T11:55:50.476860+0000","last_peered":"2024-09-21T11:55:50.476860+0000","last_clean":"2024-09-21T11:55:50.476860+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:55:50.476860+0000","last_undegraded":"2024-09-21T11:55:50.476860+0000","last_fullsized":"2024-09-21T11:55:50.476860+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:28.945564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:28.945564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:47.015587+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":663,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.471284+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:55:50.471284+0000","last_peered":"2024-09-21T11:55:50.471284+0000","last_clean":"2024-09-21T11:55:50.471284+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:55:50.471284+0000","last_undegraded":"2024-09-21T11:55:50.471284+0000","last_fullsized":"2024-09-21T11:55:50.471284+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:15.583628+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":683,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.469815+0000","last_change":"2024-09-21T11:55:28.564578+0000","last_active":"2024-09-21T11:55:50.469815+0000","last_peered":"2024-09-21T11:55:50.469815+0000","last_clean":"2024-09-21T11:55:50.469815+0000","last_became_active":"2024-09-21T11:55:28.564261+0000","last_became_peered":"2024-09-21T11:55:28.564261+0000","last_unstale":"2024-09-21T11:55:50.469815+0000","last_undegraded":"2024-09-21T11:55:50.469815+0000","last_fullsized":"2024-09-21T11:55:50.469815+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.551852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.551852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":727,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.468545+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:55:50.468545+0000","last_peered":"2024-09-21T11:55:50.468545+0000","last_clean":"2024-09-21T11:55:50.468545+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:55:50.468545+0000","last_undegraded":"2024-09-21T11:55:50.468545+0000","last_fullsized":"2024-09-21T11:55:50.468545+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":671,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465707+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:55:50.465707+0000","last_peered":"2024-09-21T11:55:50.465707+0000","last_clean":"2024-09-21T11:55:50.465707+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:55:50.465707+0000","last_undegraded":"2024-09-21T11:55:50.465707+0000","last_fullsized":"2024-09-21T11:55:50.465707+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:50.171680+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":686,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:52.217954+0000","last_change":"2024-09-21T11:55:52.217954+0000","last_active":"2024-09-21T11:55:52.217954+0000","last_peered":"2024-09-21T11:55:52.217954+0000","last_clean":"2024-09-21T11:55:52.217954+0000","last_became_active":"2024-09-21T11:55:28.575256+0000","last_became_peered":"2024-09-21T11:55:28.575256+0000","last_unstale":"2024-09-21T11:55:52.217954+0000","last_undegraded":"2024-09-21T11:55:52.217954+0000","last_fullsized":"2024-09-21T11:55:52.217954+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:52.217898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.217898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.160221+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":683,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.471766+0000","last_change":"2024-09-21T11:54:57.381670+0000","last_active":"2024-09-21T11:55:50.471766+0000","last_peered":"2024-09-21T11:55:50.471766+0000","last_clean":"2024-09-21T11:55:50.471766+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:55:50.471766+0000","last_undegraded":"2024-09-21T11:55:50.471766+0000","last_fullsized":"2024-09-21T11:55:50.471766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:57.381605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:57.381605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:21.092123+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":381,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.474793+0000","last_change":"2024-09-21T11:52:12.348687+0000","last_active":"2024-09-21T11:55:50.474793+0000","last_peered":"2024-09-21T11:55:50.474793+0000","last_clean":"2024-09-21T11:55:50.474793+0000","last_became_active":"2024-09-21T11:47:03.373890+0000","last_became_peered":"2024-09-21T11:47:03.373890+0000","last_unstale":"2024-09-21T11:55:50.474793+0000","last_undegraded":"2024-09-21T11:55:50.474793+0000","last_fullsized":"2024-09-21T11:55:50.474793+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:18.584379+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":709,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.468815+0000","last_change":"2024-09-21T11:55:28.563845+0000","last_active":"2024-09-21T11:55:50.468815+0000","last_peered":"2024-09-21T11:55:50.468815+0000","last_clean":"2024-09-21T11:55:50.468815+0000","last_became_active":"2024-09-21T11:55:28.563540+0000","last_became_peered":"2024-09-21T11:55:28.563540+0000","last_unstale":"2024-09-21T11:55:50.468815+0000","last_undegraded":"2024-09-21T11:55:50.468815+0000","last_fullsized":"2024-09-21T11:55:50.468815+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:23.601318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.601318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":777,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.473634+0000","last_change":"2024-09-21T11:54:55.510472+0000","last_active":"2024-09-21T11:55:50.473634+0000","last_peered":"2024-09-21T11:55:50.473634+0000","last_clean":"2024-09-21T11:55:50.473634+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:55:50.473634+0000","last_undegraded":"2024-09-21T11:55:50.473634+0000","last_fullsized":"2024-09-21T11:55:50.473634+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:54:55.509769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:55.509769+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:55:58.331474+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":680,"reported_epoch":374,"state":"active+clean","last_fresh":"2024-09-21T11:55:50.465547+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:55:50.465547+0000","last_peered":"2024-09-21T11:55:50.465547+0000","last_clean":"2024-09-21T11:55:50.465547+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:55:50.465547+0000","last_undegraded":"2024-09-21T11:55:50.465547+0000","last_fullsized":"2024-09-21T11:55:50.465547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:46.131671+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":1771110424,"num_objects":27026,"num_object_clones":0,"num_object_copies":81078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218472,"num_write_kb":7855873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5086,"num_bytes_recovered":333316096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":5385367552,"data_stored":5385355336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88093,"ondisk_log_size":88093,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":5914,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":758,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":941,"num_read_kb":936,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234179,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1085676,"kb_used_data":494992,"kb_used_omap":1,"kb_used_meta":590654,"kb_avail":92659476,"statfs":{"total":95995035648,"available":94883303424,"internally_reserved":0,"allocated":506871808,"data_stored":506474775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":604830154},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463113,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1203856,"kb_used_data":707584,"kb_used_omap":1,"kb_used_meta":496254,"kb_avail":92541296,"statfs":{"total":95995035648,"available":94762287104,"internally_reserved":0,"allocated":724566016,"data_stored":724185515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":508164550},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724749,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1187096,"kb_used_data":606148,"kb_used_omap":1,"kb_used_meta":580926,"kb_avail":92558056,"statfs":{"total":95995035648,"available":94779449344,"internally_reserved":0,"allocated":620695552,"data_stored":620309349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":594868678},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920980,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1320464,"kb_used_data":815808,"kb_used_omap":1,"kb_used_meta":504574,"kb_avail":92424688,"statfs":{"total":95995035648,"available":94642880512,"internally_reserved":0,"allocated":835387392,"data_stored":835006094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":516684229},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117210,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1149192,"kb_used_data":508536,"kb_used_omap":1,"kb_used_meta":640638,"kb_avail":92595960,"statfs":{"total":95995035648,"available":94818263040,"internally_reserved":0,"allocated":520740864,"data_stored":520366825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":656013767},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411551,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1451668,"kb_used_data":829056,"kb_used_omap":1,"kb_used_meta":622590,"kb_avail":92293484,"statfs":{"total":95995035648,"available":94508527616,"internally_reserved":0,"allocated":848953344,"data_stored":848571934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":637532613},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1118724,"kb_used_data":501484,"kb_used_omap":1,"kb_used_meta":617022,"kb_avail":92626428,"statfs":{"total":95995035648,"available":94849462272,"internally_reserved":0,"allocated":513519616,"data_stored":513094336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":631830982},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738604,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1491968,"kb_used_data":919280,"kb_used_omap":1,"kb_used_meta":572606,"kb_avail":92253184,"statfs":{"total":95995035648,"available":94467260416,"internally_reserved":0,"allocated":941342720,"data_stored":940977989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":586348997},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":930156544,"data_stored":930152472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":492240896,"data_stored":492240896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":844038144,"data_stored":844038144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":508493824,"data_stored":508493824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":806752256,"data_stored":806748184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":608501760,"data_stored":608501760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":702746624,"data_stored":702742552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":492437504,"data_stored":492437504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:56:00.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool set .mgr pgp_num 1 2024-09-21T11:56:00.273 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:00.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:00.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:00.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:00.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:00.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:00.535 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:00.568 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:00.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:00.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:00.611 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:00.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:00.685 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:00.726 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:00.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:00.727 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:00 smithi043 ceph-mon[29710]: 4.12 scrub starts 2024-09-21T11:56:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:00 smithi043 ceph-mon[29710]: 4.12 scrub ok 2024-09-21T11:56:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:00 smithi043 ceph-mon[29710]: 6.15 scrub starts 2024-09-21T11:56:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:00 smithi043 ceph-mon[29710]: 6.15 scrub ok 2024-09-21T11:56:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:00 smithi043 ceph-mon[29710]: 10.d scrub starts 2024-09-21T11:56:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:00 smithi043 ceph-mon[36454]: 4.12 scrub starts 2024-09-21T11:56:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:00 smithi043 ceph-mon[36454]: 4.12 scrub ok 2024-09-21T11:56:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:00 smithi043 ceph-mon[36454]: 6.15 scrub starts 2024-09-21T11:56:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:00 smithi043 ceph-mon[36454]: 6.15 scrub ok 2024-09-21T11:56:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:00 smithi043 ceph-mon[36454]: 10.d scrub starts 2024-09-21T11:56:00.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:00.863 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:00.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:00.864 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:00.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:01.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:00 smithi175 ceph-mon[32613]: 4.12 scrub starts 2024-09-21T11:56:01.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:00 smithi175 ceph-mon[32613]: 4.12 scrub ok 2024-09-21T11:56:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:00 smithi175 ceph-mon[32613]: 6.15 scrub starts 2024-09-21T11:56:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:00 smithi175 ceph-mon[32613]: 6.15 scrub ok 2024-09-21T11:56:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:00 smithi175 ceph-mon[32613]: 10.d scrub starts 2024-09-21T11:56:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:01.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:01.031 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:01.132 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:01.168 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:01.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:01.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:01.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:01.309 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:01.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:01.310 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:01.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:01.455 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:01.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:01.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:34879 9.4 9:2d14d963:::benchmark_data_smithi043_126002_object34878:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:49:28.278865+0000", 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 393.21652956200001, 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.16660401999999999, 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.501 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 34879 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278865+0000", 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278865+0000", 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278866+0000", 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6899999999999996e-07 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278888+0000", 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1725e-05 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278897+0000", 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2739999999999996e-06 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.278904+0000", 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6019999999999996e-06 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.280772+0000", 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0018679180000000001 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.280895+0000", 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000122831 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.281041+0000", 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014673799999999999 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.283617+0000", 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025755309999999998 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.286918+0000", 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003300786 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.445111+0000", 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.15819361500000001 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.445414+0000", 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030314399999999998 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:49:28.445469+0000", 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4987000000000001e-05 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:166714 9.0 9:0db06241:::benchmark_data_smithi043_126002_object39648:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:18.113330+0000", 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 283.38206525499999, 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.182493339, 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:56:01.507 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 166714 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113330+0000", 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113330+0000", 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113331+0000", 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.068e-06 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113335+0000", 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6059999999999996e-06 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113349+0000", 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.323e-05 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113356+0000", 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9920000000000001e-06 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113412+0000", 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6793999999999999e-05 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113528+0000", 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115573 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113645+0000", 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116731 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.116389+0000", 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027440640000000001 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.116875+0000", 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00048631400000000001 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295714+0000", 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17883933099999999 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295772+0000", 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7215e-05 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295823+0000", 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1421000000000001e-05 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:166729 9.0 9:0da96deb:::benchmark_data_smithi043_126002_object39663:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:18.125977+0000", 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 283.36941803899998, 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17002272299999999, 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 166729 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.513 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.125977+0000", 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.125977+0000", 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.125979+0000", 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.581e-06 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.125984+0000", 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7379999999999999e-06 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.125999+0000", 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4229e-05 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126006+0000", 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8509999999999999e-06 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126073+0000", 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6190000000000002e-05 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126203+0000", 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013019799999999999 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126333+0000", 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013044200000000001 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.130722+0000", 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043886250000000002 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.132475+0000", 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017531580000000001 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295911+0000", 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.16343649900000001 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295962+0000", 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0099000000000003e-05 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.296000+0000", 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8112999999999999e-05 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.518 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:166730 9.0 9:0f1e7941:::benchmark_data_smithi043_126002_object39664:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:18.126334+0000", 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 283.36906119299999, 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17205351699999999, 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 166730 2024-09-21T11:56:01.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126334+0000", 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126334+0000", 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126335+0000", 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9199999999999999e-07 2024-09-21T11:56:01.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126340+0000", 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9749999999999998e-06 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126369+0000", 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.955e-05 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126379+0000", 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4700000000000008e-06 2024-09-21T11:56:01.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.128585+0000", 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022058529999999998 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.128691+0000", 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106638 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.128800+0000", 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108323 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.130823+0000", 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020232050000000001 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.132530+0000", 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017074970000000001 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.298292+0000", 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.16576127099999999 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.298345+0000", 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3437999999999999e-05 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.298387+0000", 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2404999999999998e-05 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.524 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:211057 9.e e296/247 9:722bc66e:::benchmark_data_smithi043_126002_object83991:head v 296'13171, mlcod=296'13169)", 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409320+0000", 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 246.08607479700001, 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29737601699999999, 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409320+0000", 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409320+0000", 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409321+0000", 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0800000000000004e-07 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409325+0000", 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-06 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409333+0000", 2024-09-21T11:56:01.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4669999999999998e-06 2024-09-21T11:56:01.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409338+0000", 2024-09-21T11:56:01.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.609e-06 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409445+0000", 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106464 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409463+0000", 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8719000000000002e-05 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706603+0000", 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.297139561 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706696+0000", 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3189000000000005e-05 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:211059 9.0 9:08679dd9:::benchmark_data_smithi043_126002_object83993:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409680+0000", 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 246.08571468599999, 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29635377000000002, 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 211059 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.530 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409680+0000", 2024-09-21T11:56:01.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409680+0000", 2024-09-21T11:56:01.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409682+0000", 2024-09-21T11:56:01.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3379999999999999e-06 2024-09-21T11:56:01.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409687+0000", 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4929999999999998e-06 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409698+0000", 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409706+0000", 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9680000000000006e-06 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.410134+0000", 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000427494 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.410245+0000", 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111241 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.410365+0000", 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120077 2024-09-21T11:56:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705854+0000", 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29548879099999997 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705865+0000", 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1168999999999999e-05 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705894+0000", 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9088999999999999e-05 2024-09-21T11:56:01.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705988+0000", 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4019999999999998e-05 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706034+0000", 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6109000000000002e-05 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:211058 9.d e296/247 9:b1ac4faf:::benchmark_data_smithi043_126002_object83992:head v 296'13146, mlcod=296'13144)", 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409691+0000", 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 246.085704056, 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29170147000000002, 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409691+0000", 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409691+0000", 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409692+0000", 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0200000000000001e-07 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409698+0000", 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3860000000000004e-06 2024-09-21T11:56:01.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409708+0000", 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3540000000000006e-06 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409714+0000", 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8120000000000002e-06 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409791+0000", 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6365999999999994e-05 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409815+0000", 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4087000000000002e-05 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413064+0000", 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032496109999999999 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701392+0000", 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.288328052 2024-09-21T11:56:01.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:211060 9.0 9:0749ae3e:::benchmark_data_smithi043_126002_object83994:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409751+0000", 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 246.08564375500001, 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.296470603, 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:01.543 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 211060 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409751+0000", 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409751+0000", 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409752+0000", 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9299999999999997e-07 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409756+0000", 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8349999999999997e-06 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409764+0000", 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9580000000000007e-06 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409771+0000", 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6819999999999997e-06 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701791+0000", 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29201998200000001 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701945+0000", 2024-09-21T11:56:01.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000154518 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.702140+0000", 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019451399999999999 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706071+0000", 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039315360000000002 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706081+0000", 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6900000000000004e-06 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706111+0000", 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9796999999999999e-05 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706188+0000", 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7074000000000001e-05 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706222+0000", 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4224000000000003e-05 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:211067 9.4 9:2a2e5d4d:::benchmark_data_smithi043_126002_object84001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.412385+0000", 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 246.083010333, 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.30543494500000001, 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 211067 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412385+0000", 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412385+0000", 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412386+0000", 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8029999999999999e-06 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412912+0000", 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052593400000000004 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701342+0000", 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28842914800000002 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701361+0000", 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9381999999999998e-05 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701424+0000", 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2634999999999999e-05 2024-09-21T11:56:01.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701566+0000", 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014276 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701694+0000", 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012741900000000001 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.707583+0000", 2024-09-21T11:56:01.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058892190000000002 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.707636+0000", 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3359999999999997e-05 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.717703+0000", 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010067066 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.717763+0000", 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9849999999999998e-05 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.717820+0000", 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6369000000000002e-05 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:211066 9.a e296/247 9:51f636b8:::benchmark_data_smithi043_126002_object84000:head v 296'13283, mlcod=296'13281)", 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.412517+0000", 2024-09-21T11:56:01.555 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 246.08287774999999, 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29723240400000001, 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412517+0000", 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412517+0000", 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412518+0000", 2024-09-21T11:56:01.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-21T11:56:01.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412621+0000", 2024-09-21T11:56:01.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:01.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102394 2024-09-21T11:56:01.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412632+0000", 2024-09-21T11:56:01.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0849000000000001e-05 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412639+0000", 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7769999999999997e-06 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701367+0000", 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28872841500000002 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701394+0000", 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7345999999999999e-05 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.709721+0000", 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083270189999999997 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.709750+0000", 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8390000000000001e-05 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.560 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:211059 9.0 e296/247 ondisk, result = 0)", 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 246.08210234399999, 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.292746124, 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1399999999999996e-07 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.562 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413297+0000", 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8070000000000001e-06 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701343+0000", 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28804594700000002 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701357+0000", 2024-09-21T11:56:01.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3959e-05 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705885+0000", 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0045275940000000002 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705890+0000", 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706039+0000", 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014890199999999999 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:211059 9.0 e296/247 ondisk, result = 0)", 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.414118+0000", 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 246.08127705300001, 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29174787600000002, 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414118+0000", 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414118+0000", 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414119+0000", 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5899999999999995e-07 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414123+0000", 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5170000000000001e-06 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701349+0000", 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28722547300000001 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701359+0000", 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0664e-05 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705840+0000", 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044810070000000004 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705848+0000", 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3900000000000004e-06 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705866+0000", 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7966e-05 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:126300 10.b e348/313 10:d6278167:::benchmark_data_smithi043_150678_object3549:head v 348'7832, mlcod=348'7829)", 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.262324363000005, 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.23630744300000001, 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1999999999999999e-07 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233078+0000", 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6150000000000002e-06 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233088+0000", 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7499999999999998e-06 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233094+0000", 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233148+0000", 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3943000000000002e-05 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233168+0000", 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9822000000000001e-05 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.469306+0000", 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.236138347 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.469378+0000", 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2000000000000002e-05 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126310 10.2 10:429ff067:::benchmark_data_smithi043_150678_object3559:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:56:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.234619+0000", 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.260775902999995, 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.23258517300000001, 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126310 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234619+0000", 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234619+0000", 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234621+0000", 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0660000000000002e-06 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234627+0000", 2024-09-21T11:56:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3090000000000001e-06 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456221+0000", 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.22159378199999999 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456236+0000", 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304999999999999e-05 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456300+0000", 2024-09-21T11:56:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4294999999999993e-05 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456588+0000", 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028769499999999998 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456701+0000", 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113275 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.464708+0000", 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080072679999999997 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.464728+0000", 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9700000000000001e-05 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467080+0000", 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00235214 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467137+0000", 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7194999999999997e-05 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467204+0000", 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7143000000000004e-05 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:126311 10.0 e348/313 10:03ca3f46:::benchmark_data_smithi043_150678_object3560:head v 348'7980, mlcod=348'7978)", 2024-09-21T11:56:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.238509+0000", 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 96.256885804000007, 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.22861142000000001, 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238509+0000", 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238509+0000", 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238511+0000", 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4559999999999999e-06 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238524+0000", 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2875e-05 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.455912+0000", 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.21738855400000001 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.455934+0000", 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1517000000000001e-05 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456941+0000", 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010073829999999999 2024-09-21T11:56:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456966+0000", 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4556e-05 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467084+0000", 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010118240000000001 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467121+0000", 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6838999999999997e-05 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.583 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:130557 10.2 10:4741c7dc:::benchmark_data_smithi043_150678_object7806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.493746+0000", 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 92.001648865999996, 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.18472519600000001, 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 130557 2024-09-21T11:56:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493746+0000", 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493746+0000", 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493748+0000", 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0480000000000001e-06 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493754+0000", 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0769999999999996e-06 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493795+0000", 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0853999999999999e-05 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493804+0000", 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4950000000000008e-06 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493862+0000", 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8749999999999998e-05 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493968+0000", 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105327 2024-09-21T11:56:01.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.494070+0000", 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010192 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497356+0000", 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003286027 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678126+0000", 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.18076989199999999 2024-09-21T11:56:01.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678302+0000", 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017640099999999999 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678415+0000", 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112965 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678471+0000", 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6440000000000002e-05 2024-09-21T11:56:01.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:130563 10.2 10:4b5202a2:::benchmark_data_smithi043_150678_object7812:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.497727+0000", 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.997668371000003, 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.183693038, 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:01.590 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 130563 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497727+0000", 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497727+0000", 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497728+0000", 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.807e-06 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497735+0000", 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497748+0000", 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2532e-05 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497756+0000", 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1100000000000003e-06 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.500514+0000", 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027577190000000001 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675393+0000", 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17487897299999999 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675484+0000", 2024-09-21T11:56:01.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1860999999999993e-05 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678546+0000", 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030618709999999999 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681236+0000", 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026896289999999998 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681299+0000", 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2710000000000001e-05 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681371+0000", 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2080000000000001e-05 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681420+0000", 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8996999999999999e-05 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:130569 10.2 10:46c1164d:::benchmark_data_smithi043_150678_object7818:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.502522+0000", 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.992873407000005, 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17911437499999999, 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 130569 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.596 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502522+0000", 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502522+0000", 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502524+0000", 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1940000000000001e-06 2024-09-21T11:56:01.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502531+0000", 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0960000000000003e-06 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675246+0000", 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.172715127 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675264+0000", 2024-09-21T11:56:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8306999999999999e-05 2024-09-21T11:56:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678690+0000", 2024-09-21T11:56:01.600 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:56:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034253310000000002 2024-09-21T11:56:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678797+0000", 2024-09-21T11:56:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107679 2024-09-21T11:56:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678873+0000", 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5402999999999996e-05 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681486+0000", 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026133100000000002 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681534+0000", 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8352000000000001e-05 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681547+0000", 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2784e-05 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681595+0000", 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7403e-05 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681636+0000", 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1389e-05 2024-09-21T11:56:01.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:130568 10.0 e348/313 10:0d347089:::benchmark_data_smithi043_150678_object7817:head v 348'8247, mlcod=348'8245)", 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.502849+0000", 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.992546051000005, 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.180754058, 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502849+0000", 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502849+0000", 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502850+0000", 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-06 2024-09-21T11:56:01.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502860+0000", 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0000000000000001e-05 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675242+0000", 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17238205500000001 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675254+0000", 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1874e-05 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675376+0000", 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012161 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675398+0000", 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2949999999999999e-05 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.683567+0000", 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0081684180000000002 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.683603+0000", 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6127999999999997e-05 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25910.0:130557 10.2 e348/313 ondisk, result = 0)", 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.504828+0000", 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 91.990566568000006, 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17365343999999999, 2024-09-21T11:56:01.607 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.504828+0000", 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.504828+0000", 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.504829+0000", 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.012e-06 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.504836+0000", 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1859999999999997e-06 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675240+0000", 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17040451500000001 2024-09-21T11:56:01.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675265+0000", 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4658999999999999e-05 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678286+0000", 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003021087 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678294+0000", 2024-09-21T11:56:01.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6350000000000006e-06 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678482+0000", 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018834599999999999 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:01.611 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:56:01.611 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: pgmap v769: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 995 op/s; 5.2 MiB/s, 83 objects/s recovering 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: 5.12 scrub starts 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: 5.12 scrub ok 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: osdmap e375: 8 total, 8 up, 8 in 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: 10.d scrub ok 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2913520512' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[29710]: 10.b scrub starts 2024-09-21T11:56:01.612 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:56:01.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:56:01.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: pgmap v769: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 995 op/s; 5.2 MiB/s, 83 objects/s recovering 2024-09-21T11:56:01.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: 5.12 scrub starts 2024-09-21T11:56:01.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: 5.12 scrub ok 2024-09-21T11:56:01.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: osdmap e375: 8 total, 8 up, 8 in 2024-09-21T11:56:01.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: 10.d scrub ok 2024-09-21T11:56:01.613 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:56:01.641 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:01.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:01.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:01.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:01.750 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:56:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:56:01.784 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:01.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:01.785 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:01.791 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:56:01.792 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:56:01.827 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:56:01.828 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:56:01.866 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:56:01.866 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:56:01.867 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:56:01.867 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2913520512' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:01.867 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:01 smithi043 ceph-mon[36454]: 10.b scrub starts 2024-09-21T11:56:01.886 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:01.905 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:56:01.906 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:56:01.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:01.930 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:01.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:01.936 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:56:01.936 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:56:01.972 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:56:01.972 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:56:02.008 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:56:02.009 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:56:02.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: pgmap v769: 148 pgs: 1 active+clean+scrubbing, 147 active+clean; 1.5 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 995 op/s; 5.2 MiB/s, 83 objects/s recovering 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: 5.12 scrub starts 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: 5.12 scrub ok 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: osdmap e375: 8 total, 8 up, 8 in 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: 10.d scrub ok 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2913520512' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:02.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:01 smithi175 ceph-mon[32613]: 10.b scrub starts 2024-09-21T11:56:02.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:02.071 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:02.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:02.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:02.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:02.216 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:02.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:02.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:02.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:02.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:02.355 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:02.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:02.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:02.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:02.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:02.597 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:02.631 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:02.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:02.632 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:02.733 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:02.771 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:02.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:02.772 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:02.872 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:02.906 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:02.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:02.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:56:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:56:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[36454]: 10.b scrub ok 2024-09-21T11:56:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2913520512' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[36454]: osdmap e376: 8 total, 8 up, 8 in 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2224982977' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[29710]: 10.b scrub ok 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2913520512' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[29710]: osdmap e376: 8 total, 8 up, 8 in 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2224982977' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:02 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:03.008 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:03.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:02 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:56:03.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:02 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:56:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:02 smithi175 ceph-mon[32613]: 10.b scrub ok 2024-09-21T11:56:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2913520512' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:02 smithi175 ceph-mon[32613]: osdmap e376: 8 total, 8 up, 8 in 2024-09-21T11:56:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2224982977' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:02 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:02 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:03.043 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:03.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:03.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:03.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:03.180 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:03.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:03.181 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:03.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:03.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:03.321 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:03.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:03.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:03.460 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:03.460 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:03.561 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:03.589 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:03.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:03.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:03.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[29710]: 4.1c scrub starts 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[29710]: 4.1c scrub ok 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[29710]: pgmap v772: 148 pgs: 148 active+clean; 1.1 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[29710]: 6.b scrub starts 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[29710]: 6.b scrub ok 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[29710]: 3.1c scrub starts 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[29710]: 3.1c scrub ok 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[36454]: 4.1c scrub starts 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[36454]: 4.1c scrub ok 2024-09-21T11:56:03.698 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:03.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[36454]: pgmap v772: 148 pgs: 148 active+clean; 1.1 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-21T11:56:03.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[36454]: 6.b scrub starts 2024-09-21T11:56:03.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[36454]: 6.b scrub ok 2024-09-21T11:56:03.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[36454]: 3.1c scrub starts 2024-09-21T11:56:03.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:03 smithi043 ceph-mon[36454]: 3.1c scrub ok 2024-09-21T11:56:03.729 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:03.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:03.730 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:03.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:03.864 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:03.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:03.865 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:03.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:04.007 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:04.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:04.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:04.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:03 smithi175 ceph-mon[32613]: 4.1c scrub starts 2024-09-21T11:56:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:03 smithi175 ceph-mon[32613]: 4.1c scrub ok 2024-09-21T11:56:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:03 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:03 smithi175 ceph-mon[32613]: pgmap v772: 148 pgs: 148 active+clean; 1.1 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s 2024-09-21T11:56:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:03 smithi175 ceph-mon[32613]: 6.b scrub starts 2024-09-21T11:56:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:03 smithi175 ceph-mon[32613]: 6.b scrub ok 2024-09-21T11:56:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:03 smithi175 ceph-mon[32613]: 3.1c scrub starts 2024-09-21T11:56:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:03 smithi175 ceph-mon[32613]: 3.1c scrub ok 2024-09-21T11:56:04.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:04.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:04.157 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:04.157 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:04.258 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:04.289 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:04.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:04.290 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:04.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:04.432 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:04.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:04.433 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:04.534 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:04.555 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:04.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:04.556 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:04.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:04.679 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:04.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:04.680 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:04.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:04.801 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:04.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:04.803 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:04.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:04.931 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:04.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:04.932 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:04 smithi043 ceph-mon[36454]: 3.19 scrub starts 2024-09-21T11:56:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:04 smithi043 ceph-mon[36454]: 3.19 scrub ok 2024-09-21T11:56:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:04 smithi043 ceph-mon[29710]: 3.19 scrub starts 2024-09-21T11:56:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:04 smithi043 ceph-mon[29710]: 3.19 scrub ok 2024-09-21T11:56:05.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:04 smithi175 ceph-mon[32613]: 3.19 scrub starts 2024-09-21T11:56:05.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:04 smithi175 ceph-mon[32613]: 3.19 scrub ok 2024-09-21T11:56:05.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:05.068 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:05.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:05.069 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:05.170 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:05.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:05.207 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:05.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:05.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:05.342 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:05.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:05.343 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:05.445 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:05.475 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:05.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:05.476 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:56:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:05.561 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 1 pgp_num to 1 2024-09-21T11:56:05.577 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:05.609 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:05.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:05.610 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:05.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:05.747 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:05.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:05.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:05.848 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:05.884 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:05.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:05.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:05.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:05.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[29710]: 4.d scrub starts 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[29710]: 4.d scrub ok 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[29710]: pgmap v773: 148 pgs: 148 active+clean; 1.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/208165388' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[29710]: 5.9 scrub starts 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[29710]: 5.9 scrub ok 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[29710]: 6.7 scrub starts 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[29710]: 6.7 scrub ok 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[36454]: 4.d scrub starts 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[36454]: 4.d scrub ok 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[36454]: pgmap v773: 148 pgs: 148 active+clean; 1.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s 2024-09-21T11:56:05.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/208165388' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:56:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[36454]: 5.9 scrub starts 2024-09-21T11:56:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[36454]: 5.9 scrub ok 2024-09-21T11:56:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[36454]: 6.7 scrub starts 2024-09-21T11:56:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:05 smithi043 ceph-mon[36454]: 6.7 scrub ok 2024-09-21T11:56:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:05 smithi175 ceph-mon[32613]: 4.d scrub starts 2024-09-21T11:56:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:05 smithi175 ceph-mon[32613]: 4.d scrub ok 2024-09-21T11:56:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:05 smithi175 ceph-mon[32613]: pgmap v773: 148 pgs: 148 active+clean; 1.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s 2024-09-21T11:56:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:05 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/208165388' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:56:06.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:05 smithi175 ceph-mon[32613]: 5.9 scrub starts 2024-09-21T11:56:06.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:05 smithi175 ceph-mon[32613]: 5.9 scrub ok 2024-09-21T11:56:06.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:05 smithi175 ceph-mon[32613]: 6.7 scrub starts 2024-09-21T11:56:06.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:05 smithi175 ceph-mon[32613]: 6.7 scrub ok 2024-09-21T11:56:06.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:06.030 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:06.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:06.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:56:06.132 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:06.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:06.165 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:06.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:06.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:06.316 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:06.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:06.317 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:06.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:06.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:06.468 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:06.469 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:06.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:06.602 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:06.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:06.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:06.704 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:06.735 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:06.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:06.736 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[29710]: 4.7 scrub starts 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[29710]: 4.7 scrub ok 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/208165388' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[29710]: 5.0 scrub starts 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[29710]: osdmap e377: 8 total, 8 up, 8 in 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[29710]: 5.0 scrub ok 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[36454]: 4.7 scrub starts 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[36454]: 4.7 scrub ok 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/208165388' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[36454]: 5.0 scrub starts 2024-09-21T11:56:06.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[36454]: osdmap e377: 8 total, 8 up, 8 in 2024-09-21T11:56:06.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:06 smithi043 ceph-mon[36454]: 5.0 scrub ok 2024-09-21T11:56:06.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:06.869 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:06.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:06.870 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:06.960 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:06.971 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:07.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:07.000 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:07.001 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:07.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:06 smithi175 ceph-mon[32613]: 4.7 scrub starts 2024-09-21T11:56:07.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:06 smithi175 ceph-mon[32613]: 4.7 scrub ok 2024-09-21T11:56:07.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:06 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/208165388' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:56:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:06 smithi175 ceph-mon[32613]: 5.0 scrub starts 2024-09-21T11:56:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:06 smithi175 ceph-mon[32613]: osdmap e377: 8 total, 8 up, 8 in 2024-09-21T11:56:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:06 smithi175 ceph-mon[32613]: 5.0 scrub ok 2024-09-21T11:56:07.101 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:07.137 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:07.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:07.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:07.239 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:07.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:07.276 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:07.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:07.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:07.407 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:07.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:07.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:07.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:56:07.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:07.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:07.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:07.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:56:07.632 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:07.632 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:07.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:07.708 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:07.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:07.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:07 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T11:56:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:07 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T11:56:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:07 smithi043 ceph-mon[29710]: pgmap v775: 148 pgs: 148 active+clean; 970 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.44k op/s 2024-09-21T11:56:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:07 smithi043 ceph-mon[29710]: 5.1 scrub starts 2024-09-21T11:56:07.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:07 smithi043 ceph-mon[29710]: 5.1 scrub ok 2024-09-21T11:56:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:07 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T11:56:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:07 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T11:56:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:07 smithi043 ceph-mon[36454]: pgmap v775: 148 pgs: 148 active+clean; 970 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.44k op/s 2024-09-21T11:56:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:07 smithi043 ceph-mon[36454]: 5.1 scrub starts 2024-09-21T11:56:07.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:07 smithi043 ceph-mon[36454]: 5.1 scrub ok 2024-09-21T11:56:07.810 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:07.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:07.840 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:07.840 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:07.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:07.974 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:07.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:07.975 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:08.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:07 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T11:56:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:07 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T11:56:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:07 smithi175 ceph-mon[32613]: pgmap v775: 148 pgs: 148 active+clean; 970 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.44k op/s 2024-09-21T11:56:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:07 smithi175 ceph-mon[32613]: 5.1 scrub starts 2024-09-21T11:56:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:07 smithi175 ceph-mon[32613]: 5.1 scrub ok 2024-09-21T11:56:08.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:08.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:08.112 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:08.112 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:08.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:08.246 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:08.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:08.247 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:08.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:08.350 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:56:08.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:08.413 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:08.413 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:08.498 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:08.499 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:08.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:08.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:08 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:56:08.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:08 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:56:08.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:08 smithi043 ceph-mon[29710]: 10.a scrub starts 2024-09-21T11:56:08.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:08 smithi043 ceph-mon[29710]: 5.19 scrub starts 2024-09-21T11:56:08.591 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:08 smithi043 ceph-mon[29710]: 5.19 scrub ok 2024-09-21T11:56:08.591 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:08 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:56:08.591 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:08 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:56:08.592 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:08 smithi043 ceph-mon[36454]: 10.a scrub starts 2024-09-21T11:56:08.592 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:08 smithi043 ceph-mon[36454]: 5.19 scrub starts 2024-09-21T11:56:08.592 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:08 smithi043 ceph-mon[36454]: 5.19 scrub ok 2024-09-21T11:56:08.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:08.607 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:08.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:08.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:08.752 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:08.852 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:08.882 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:08.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:08.883 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:08.984 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:09.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:08 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:56:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:08 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:56:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:08 smithi175 ceph-mon[32613]: 10.a scrub starts 2024-09-21T11:56:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:08 smithi175 ceph-mon[32613]: 5.19 scrub starts 2024-09-21T11:56:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:08 smithi175 ceph-mon[32613]: 5.19 scrub ok 2024-09-21T11:56:09.019 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:09.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:09.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:09.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:09.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:09.163 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:09.164 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:09.265 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:09.298 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:09.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:09.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:09.399 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:09.440 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:09.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:09.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:09.509 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:56:09.510 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:56:09.542 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:09.546 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:56:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:56:09.573 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:09.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:09.574 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:09.587 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:56:09.588 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:56:09.629 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:56:09.629 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:56:09.668 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:56:09.669 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:56:09.674 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:09.706 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:09.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:09.707 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:09.714 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:56:09.715 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:56:09.761 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:56:09.761 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: 10.5 scrub starts 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: 10.5 scrub ok 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: pgmap v776: 148 pgs: 148 active+clean; 787 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: 10.a scrub ok 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: 5.d scrub starts 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: 5.d scrub ok 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: 4.e scrub starts 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: 4.e scrub ok 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: 4.2 scrub starts 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: 4.2 scrub ok 2024-09-21T11:56:09.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3355162755' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: 10.5 scrub starts 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: 10.5 scrub ok 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: pgmap v776: 148 pgs: 148 active+clean; 787 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: 10.a scrub ok 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: 5.d scrub starts 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: 5.d scrub ok 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: 4.e scrub starts 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: 4.e scrub ok 2024-09-21T11:56:09.763 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: 4.2 scrub starts 2024-09-21T11:56:09.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: 4.2 scrub ok 2024-09-21T11:56:09.764 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3355162755' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:09.771 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:09.799 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:56:09.799 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:56:09.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:09.867 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:09.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:09.871 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:09.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:10.007 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:10.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:10.008 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: 10.5 scrub starts 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: 10.5 scrub ok 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: pgmap v776: 148 pgs: 148 active+clean; 787 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: 10.a scrub ok 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: 5.d scrub starts 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: 5.d scrub ok 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: 4.e scrub starts 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: 4.e scrub ok 2024-09-21T11:56:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: 4.2 scrub starts 2024-09-21T11:56:10.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: 4.2 scrub ok 2024-09-21T11:56:10.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3355162755' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:10.109 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:10.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:10.144 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:10.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:10.245 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:10.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:10.275 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:10.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:10.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:10.412 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:10.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:10.413 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:10.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:10.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:10.556 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:10.557 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:10.633 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:56:10.658 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:10.682 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:10.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:10.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:10.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:10.816 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:10.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:10.817 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:10.918 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:10.948 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:10.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:10.949 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[36454]: 3.b scrub starts 2024-09-21T11:56:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[36454]: 3.b scrub ok 2024-09-21T11:56:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[36454]: 2.0 scrub starts 2024-09-21T11:56:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[36454]: 2.0 scrub ok 2024-09-21T11:56:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[36454]: osdmap e378: 8 total, 8 up, 8 in 2024-09-21T11:56:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1232002891' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[29710]: 3.b scrub starts 2024-09-21T11:56:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[29710]: 3.b scrub ok 2024-09-21T11:56:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[29710]: 2.0 scrub starts 2024-09-21T11:56:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[29710]: 2.0 scrub ok 2024-09-21T11:56:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[29710]: osdmap e378: 8 total, 8 up, 8 in 2024-09-21T11:56:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1232002891' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:10.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:10 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:11.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-mon[32613]: 3.b scrub starts 2024-09-21T11:56:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-mon[32613]: 3.b scrub ok 2024-09-21T11:56:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-mon[32613]: 2.0 scrub starts 2024-09-21T11:56:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-mon[32613]: 2.0 scrub ok 2024-09-21T11:56:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-mon[32613]: osdmap e378: 8 total, 8 up, 8 in 2024-09-21T11:56:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1232002891' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:11.011 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore.transactions t=2024-09-21T11:56:10.659932235Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-21T11:56:11.011 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore.transactions t=2024-09-21T11:56:10.670453949Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-21T11:56:11.011 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore.transactions t=2024-09-21T11:56:10.682730271Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-09-21T11:56:11.011 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore.transactions t=2024-09-21T11:56:10.694291384Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=3 code="database is locked" 2024-09-21T11:56:11.011 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=sqlstore.transactions t=2024-09-21T11:56:10.705610275Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=4 code="database is locked" 2024-09-21T11:56:11.012 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 11:56:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ngalert.notifier.alertmanager org=1 t=2024-09-21T11:56:10.7063501Z level=error msg="Running maintenance failed" err="database is locked" 2024-09-21T11:56:11.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:11.089 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:11.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:11.090 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:11.191 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:11.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:11.215 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:11.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:11.316 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:11.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:11.351 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:11.352 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:11.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:11.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:11.492 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:11.492 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:11.593 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:11.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:11.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:11.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:11.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:11.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:11.765 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:11.833 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:56:11.833 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:56:11.833 INFO:tasks.thrashosds.thrasher:Reweighting osd 4 to 0.6174647741144306 2024-09-21T11:56:11.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 4 0.6174647741144306 2024-09-21T11:56:11.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:11.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[29710]: pgmap v778: 148 pgs: 148 active+clean; 787 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 776 op/s 2024-09-21T11:56:11.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[29710]: 6.5 scrub starts 2024-09-21T11:56:11.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[29710]: 6.5 scrub ok 2024-09-21T11:56:11.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:11.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:56:11.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[29710]: osdmap e379: 8 total, 8 up, 8 in 2024-09-21T11:56:11.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[36454]: pgmap v778: 148 pgs: 148 active+clean; 787 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 776 op/s 2024-09-21T11:56:11.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[36454]: 6.5 scrub starts 2024-09-21T11:56:11.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[36454]: 6.5 scrub ok 2024-09-21T11:56:11.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:11.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:56:11.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:11 smithi043 ceph-mon[36454]: osdmap e379: 8 total, 8 up, 8 in 2024-09-21T11:56:11.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:11.921 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:11 smithi175 ceph-mon[32613]: pgmap v778: 148 pgs: 148 active+clean; 787 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 776 op/s 2024-09-21T11:56:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:11 smithi175 ceph-mon[32613]: 6.5 scrub starts 2024-09-21T11:56:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:11 smithi175 ceph-mon[32613]: 6.5 scrub ok 2024-09-21T11:56:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:11 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:11 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:56:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:11 smithi175 ceph-mon[32613]: osdmap e379: 8 total, 8 up, 8 in 2024-09-21T11:56:12.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:56:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:56:12.022 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:12.055 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:12.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:12.056 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:12.156 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:12.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:12.197 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:12.197 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:12.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:12.335 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:12.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:12.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:12.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:12.490 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:12.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:12.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:12.592 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:12.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:12.622 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:12.622 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:12.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:12.769 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:12.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:12.770 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:12.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:12.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:12.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:12.913 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:13.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:13.052 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:13.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:13.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:13.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:13.200 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:13.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:13.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:13.303 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:13.339 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:13.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:13.340 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:13.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:13.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:13.490 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:13.490 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:13.539 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:56:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:56:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:56:13.539 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:56:13.539 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:56:13.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:13.643 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:13.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:13.644 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:13 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:13 smithi043 ceph-mon[36454]: pgmap v780: 148 pgs: 148 active+clean; 449 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.72k op/s 2024-09-21T11:56:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:13 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:13 smithi043 ceph-mon[29710]: pgmap v780: 148 pgs: 148 active+clean; 449 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.72k op/s 2024-09-21T11:56:13.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:13.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:13.793 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:13.794 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:13.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:13.922 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:13.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:13.923 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:13 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:13 smithi175 ceph-mon[32613]: pgmap v780: 148 pgs: 148 active+clean; 449 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.72k op/s 2024-09-21T11:56:14.023 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:14.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:14.053 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:14.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:14.105 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:56:14.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:14.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:14.201 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:14.301 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:14.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:14.329 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:14.330 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:14.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:14.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:14.484 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:14.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:14.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:56:14.585 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:14.615 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:14.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:14.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:14.705 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:14 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:56:14.705 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:14 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:56:14.705 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:14 smithi043 ceph-mon[36454]: osdmap e380: 8 total, 8 up, 8 in 2024-09-21T11:56:14.706 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:14 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:56:14.706 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:14 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:56:14.706 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:14 smithi043 ceph-mon[29710]: osdmap e380: 8 total, 8 up, 8 in 2024-09-21T11:56:14.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:14.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:14.783 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:14.783 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:14.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:14.912 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:14.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:14.914 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:15.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:14 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:56:15.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:14 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:56:15.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:14 smithi175 ceph-mon[32613]: osdmap e380: 8 total, 8 up, 8 in 2024-09-21T11:56:15.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:15.065 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:15.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:15.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:15.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:15.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:56:15.209 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:15.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:15.211 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:15.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:15.360 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:56:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:56:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:15.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:15.370 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:15.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:15.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:15.526 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:15.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:15.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:15.535 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:15.581 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:56:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:56:15.628 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:15.630 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:56:15.630 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:56:15.669 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.4 to 0.617465 (9e12) 2024-09-21T11:56:15.677 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:15.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:15.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:15.693 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:56:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:56:15.726 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:56:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:56:15.761 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:56:15.761 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:56:15.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:15.793 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:56:15.793 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:56:15.811 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:15.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:15.813 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:15.820 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:56:15.821 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:56:15.838 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:56:15.838 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:56:15.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:15.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:15.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:15.951 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:15 smithi043 ceph-mon[29710]: pgmap v782: 148 pgs: 148 active+clean; 425 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 963 op/s 2024-09-21T11:56:15.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4209433936' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 0.6174647741144306} : dispatch 2024-09-21T11:56:15.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:15 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 0.6174647741144306} : dispatch 2024-09-21T11:56:15.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/387882481' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:15.952 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:15 smithi043 ceph-mon[36454]: pgmap v782: 148 pgs: 148 active+clean; 425 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 963 op/s 2024-09-21T11:56:15.952 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4209433936' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 0.6174647741144306} : dispatch 2024-09-21T11:56:15.952 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:15 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 0.6174647741144306} : dispatch 2024-09-21T11:56:15.952 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/387882481' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:15 smithi175 ceph-mon[32613]: pgmap v782: 148 pgs: 148 active+clean; 425 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 963 op/s 2024-09-21T11:56:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4209433936' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 0.6174647741144306} : dispatch 2024-09-21T11:56:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:15 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 0.6174647741144306} : dispatch 2024-09-21T11:56:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/387882481' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:16.037 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:16.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:16.060 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:16.161 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:16.183 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:16.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:16.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:16.285 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:16.308 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:16.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:16.309 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:16.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:16.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:16.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:16.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:16.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:16.559 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:16.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:16.689 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:16.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:16.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:16.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:16.814 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:16.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:16.815 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:16.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:16.938 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:16.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:16.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:16 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 0.6174647741144306}]': finished 2024-09-21T11:56:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:16 smithi043 ceph-mon[36454]: osdmap e381: 8 total, 8 up, 8 in 2024-09-21T11:56:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:16 smithi043 ceph-mon[36454]: osdmap e382: 8 total, 8 up, 8 in 2024-09-21T11:56:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:16 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 0.6174647741144306}]': finished 2024-09-21T11:56:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:16 smithi043 ceph-mon[29710]: osdmap e381: 8 total, 8 up, 8 in 2024-09-21T11:56:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:16 smithi043 ceph-mon[29710]: osdmap e382: 8 total, 8 up, 8 in 2024-09-21T11:56:17.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:16 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 0.6174647741144306}]': finished 2024-09-21T11:56:17.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:16 smithi175 ceph-mon[32613]: osdmap e381: 8 total, 8 up, 8 in 2024-09-21T11:56:17.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:16 smithi175 ceph-mon[32613]: osdmap e382: 8 total, 8 up, 8 in 2024-09-21T11:56:17.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:17.062 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:17.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:17.063 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:17.164 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:17.188 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:17.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:17.189 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:17.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:17.321 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:17.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:17.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:17.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:17.448 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:17.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:17.449 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:17.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:17.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:17.578 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:17.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:17.706 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:17.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:17.707 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:17.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:17.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:17.845 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:17.846 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:17.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:17.969 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:17.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:17.970 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:17.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:17 smithi043 ceph-mon[29710]: pgmap v784: 148 pgs: 4 peering, 144 active+clean; 249 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.43k op/s 2024-09-21T11:56:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:17 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:17 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:17 smithi043 ceph-mon[29710]: osdmap e383: 8 total, 8 up, 8 in 2024-09-21T11:56:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:17 smithi043 ceph-mon[36454]: pgmap v784: 148 pgs: 4 peering, 144 active+clean; 249 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.43k op/s 2024-09-21T11:56:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:17 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:17 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:17 smithi043 ceph-mon[36454]: osdmap e383: 8 total, 8 up, 8 in 2024-09-21T11:56:18.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:17 smithi175 ceph-mon[32613]: pgmap v784: 148 pgs: 4 peering, 144 active+clean; 249 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.43k op/s 2024-09-21T11:56:18.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:17 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:17 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:17 smithi175 ceph-mon[32613]: osdmap e383: 8 total, 8 up, 8 in 2024-09-21T11:56:18.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:18.103 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:18.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:18.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:18.204 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:18.239 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:18.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:18.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:18.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:18.386 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:18.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:18.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:18.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:18.528 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:18.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:18.529 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:18.615 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:18.615 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:18.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:18.638 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:18.681 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:18.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:18.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:18.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:18.806 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:18.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:18.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:18.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:18.929 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:18.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:18.931 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:18.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:18 smithi043 ceph-mon[36454]: 10.1 starting backfill to osd.1 from (0'0,0'0] MAX to 380'15094 2024-09-21T11:56:18.959 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:18 smithi043 ceph-mon[36454]: 10.0 starting backfill to osd.7 from (0'0,0'0] MAX to 380'15393 2024-09-21T11:56:18.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:18 smithi043 ceph-mon[29710]: 10.1 starting backfill to osd.1 from (0'0,0'0] MAX to 380'15094 2024-09-21T11:56:18.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:18 smithi043 ceph-mon[29710]: 10.0 starting backfill to osd.7 from (0'0,0'0] MAX to 380'15393 2024-09-21T11:56:19.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:18 smithi175 ceph-mon[32613]: 10.1 starting backfill to osd.1 from (0'0,0'0] MAX to 380'15094 2024-09-21T11:56:19.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:18 smithi175 ceph-mon[32613]: 10.0 starting backfill to osd.7 from (0'0,0'0] MAX to 380'15393 2024-09-21T11:56:19.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:19.054 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:19.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:19.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:19.156 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:19.179 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:19.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:19.180 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:19.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:19.303 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:19.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:19.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:19.405 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:19.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:19.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:19.428 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:19.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:56:19.529 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:19.551 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:19.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:19.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:19.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:19.674 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:19.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:19.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:19.776 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:19.805 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:19.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:19.806 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:19.853 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:19 smithi043 ceph-mon[36454]: pgmap v787: 148 pgs: 14 peering, 134 active+clean; 99 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:56:19.853 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:19 smithi043 ceph-mon[36454]: osdmap e384: 8 total, 8 up, 8 in 2024-09-21T11:56:19.853 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:19 smithi043 ceph-mon[29710]: pgmap v787: 148 pgs: 14 peering, 134 active+clean; 99 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:56:19.853 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:19 smithi043 ceph-mon[29710]: osdmap e384: 8 total, 8 up, 8 in 2024-09-21T11:56:19.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:19.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:19.949 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:20.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:19 smithi175 ceph-mon[32613]: pgmap v787: 148 pgs: 14 peering, 134 active+clean; 99 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-21T11:56:20.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:19 smithi175 ceph-mon[32613]: osdmap e384: 8 total, 8 up, 8 in 2024-09-21T11:56:20.050 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:20.051 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:20.078 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:20.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:20.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:20.135 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Removed 122748 objects 2024-09-21T11:56:20.135 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Clean up completed and total clean up time :117.807 2024-09-21T11:56:20.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:20.203 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:20.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:20.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:20.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:20.327 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:20.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:20.328 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:20.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:56:20.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:20.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:20.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:20.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:20.578 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:20.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:20.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:20.617 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:20.618 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:20.680 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:20.703 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:20.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:20.704 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:20.705 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:56:20.805 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:20.829 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:20.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:20.830 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:20.848 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:20 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:56:20.849 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:20 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:56:20.849 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:20 smithi043 ceph-mon[36454]: osdmap e385: 8 total, 8 up, 8 in 2024-09-21T11:56:20.849 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2199968943' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:20.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:20 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:56:20.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:20 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:56:20.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:20 smithi043 ceph-mon[29710]: osdmap e385: 8 total, 8 up, 8 in 2024-09-21T11:56:20.849 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2199968943' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:20.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:20.953 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:20.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:20.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:20 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:56:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:20 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:56:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:20 smithi175 ceph-mon[32613]: osdmap e385: 8 total, 8 up, 8 in 2024-09-21T11:56:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2199968943' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:21.054 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:21.081 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:21.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:21.082 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:21.183 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:21.208 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:21.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:21.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:21.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:21.335 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:21.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:21.336 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:21.437 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:21.462 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:21.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:21.463 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:21.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:21.589 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:21.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:21.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:21.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:21.708 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:56:21.708 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:56:21.709 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-21T11:56:21.709 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T11:56:21.709 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T11:56:21.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-21T11:56:21.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:21.719 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:21.719 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:21.820 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:21.846 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:21.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:21.847 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:21.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:21 smithi043 ceph-mon[29710]: pgmap v790: 148 pgs: 14 peering, 134 active+clean; 99 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 599 op/s 2024-09-21T11:56:21.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:21 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:21.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2199968943' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:21.942 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:21 smithi043 ceph-mon[29710]: osdmap e386: 8 total, 8 up, 8 in 2024-09-21T11:56:21.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:21 smithi043 ceph-mon[36454]: pgmap v790: 148 pgs: 14 peering, 134 active+clean; 99 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 599 op/s 2024-09-21T11:56:21.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:21 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:21.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:21 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2199968943' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:21.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:21 smithi043 ceph-mon[36454]: osdmap e386: 8 total, 8 up, 8 in 2024-09-21T11:56:21.948 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:21.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:56:21.973 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:21.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:21.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:22.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:21 smithi175 ceph-mon[32613]: pgmap v790: 148 pgs: 14 peering, 134 active+clean; 99 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 599 op/s 2024-09-21T11:56:22.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:21 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:22.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:21 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2199968943' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:22.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:21 smithi175 ceph-mon[32613]: osdmap e386: 8 total, 8 up, 8 in 2024-09-21T11:56:22.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:56:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:56:22.013 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:22.075 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:22.100 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:22.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:22.101 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:22.202 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:22.228 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:22.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:22.229 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:22.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:22.355 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:22.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:22.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:22.457 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:22.482 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:22.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:22.483 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:22.563 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:56:22.563 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:56:22.583 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:56:22.583 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:56:22.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:22.606 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:56:22.606 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:56:22.609 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:22.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:22.610 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:22.622 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:56:22.622 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:56:22.639 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:56:22.639 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:56:22.656 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:56:22.656 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:56:22.673 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:56:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:56:22.690 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:56:22.691 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:56:22.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:22.735 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:22.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:22.736 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:22.829 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2882243722' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:22.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2882243722' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:22.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:22.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:22.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:22.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:22.988 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:22.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:22.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2882243722' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:23.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:23.113 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:23.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:23.114 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:23.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:23.236 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:23.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:23.237 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:23.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:23.361 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:23.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:23.362 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:23.463 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:23.484 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:23.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:23.485 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:23.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:56:23.585 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:23.606 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:23.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:23.607 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:23.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:23.730 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:23.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:23.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:23.805 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:56:23.805 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "" 2024-09-21T11:56:23.806 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:56:23.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:23.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:23.856 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:23.856 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:23.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:23.979 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:23.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:23.980 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[29710]: pgmap v792: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.8 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 335 op/s; 1.5 MiB/s, 26 objects/s recovering 2024-09-21T11:56:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[29710]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:56:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:56:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[36454]: pgmap v792: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.8 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 335 op/s; 1.5 MiB/s, 26 objects/s recovering 2024-09-21T11:56:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[36454]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:56:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:56:23.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:23.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:23 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:24.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:23 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:24.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:23 smithi175 ceph-mon[32613]: pgmap v792: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 5.8 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 335 op/s; 1.5 MiB/s, 26 objects/s recovering 2024-09-21T11:56:24.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:23 smithi175 ceph-mon[32613]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:56:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:23 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:56:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:23 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:23 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:24.046 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:24.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:24.105 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:24.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:24.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:24.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:24.232 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:24.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:24.233 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:24.333 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:24.356 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:24.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:24.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:24.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:24.480 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:24.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:24.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:24.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:24.603 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:24.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:24.604 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:24.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:24.727 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:24.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:24.729 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:24.829 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:24.851 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:24.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:24.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:24.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:24.976 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:24.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:24.977 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:25.025 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:56:25.025 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:56:25.025 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:56:25.025 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:56:25.025 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:56:25.029 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:25.077 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:25.099 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:25.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:25.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:25.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:25.223 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:25.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:25.224 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:25 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:25 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:25 smithi043 ceph-mon[36454]: pgmap v793: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 263 op/s; 1.1 MiB/s, 19 objects/s recovering 2024-09-21T11:56:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:25 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:25 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:25 smithi043 ceph-mon[29710]: pgmap v793: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 263 op/s; 1.1 MiB/s, 19 objects/s recovering 2024-09-21T11:56:25.324 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:25.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:25.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:25.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:25.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:25.470 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:25.490 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:56:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:56:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:25.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:25 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:25.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:25 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:25.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:25 smithi175 ceph-mon[32613]: pgmap v793: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 263 op/s; 1.1 MiB/s, 19 objects/s recovering 2024-09-21T11:56:25.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:56:25.571 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:25.596 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:25.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:25.597 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:25.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:25.721 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:25.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:25.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:25.727 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-21T11:56:25.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-21T11:56:25.823 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:25.848 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:25.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:25.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:25.950 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:25.974 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:25.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:25.975 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:26.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:26.098 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:26.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:26.099 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:26.175 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:56:26.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:26 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:56:26.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:26 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:56:26.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:26 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:26.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:26 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/373189536' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:56:26.176 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:26 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:56:26.176 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:26 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:56:26.176 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:26 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:26.177 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:26 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/373189536' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:56:26.200 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:26.225 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:26.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:26.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:26.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:56:26.326 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:26.350 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:26.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:26.351 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:26.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:26.476 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:26.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:26.477 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:26 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:56:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:26 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:56:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:26 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:26.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:26 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/373189536' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:56:26.577 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:26.601 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:26.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:26.602 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:26.702 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:26.725 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:26.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:26.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:26.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:26.851 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:26.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:26.852 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:26.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:26.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:26.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:26.976 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:27.065 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:27.071 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:27.077 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:27.100 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:27.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:27.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:27.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:27.223 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:27.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:27.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:27.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:27 smithi043 ceph-mon[36454]: pgmap v794: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 689 B/s rd, 0 B/s wr, 212 op/s; 897 KiB/s, 16 objects/s recovering 2024-09-21T11:56:27.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:27 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:27.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:27 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/373189536' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:56:27.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:27 smithi043 ceph-mon[36454]: osdmap e387: 8 total, 8 up, 8 in 2024-09-21T11:56:27.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:27 smithi043 ceph-mon[29710]: pgmap v794: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 689 B/s rd, 0 B/s wr, 212 op/s; 897 KiB/s, 16 objects/s recovering 2024-09-21T11:56:27.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:27 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:27.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:27 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/373189536' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:56:27.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:27 smithi043 ceph-mon[29710]: osdmap e387: 8 total, 8 up, 8 in 2024-09-21T11:56:27.324 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:27.348 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:27.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:27.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:27.449 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:27.472 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:27.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:27.473 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:27.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:27 smithi175 ceph-mon[32613]: pgmap v794: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 689 B/s rd, 0 B/s wr, 212 op/s; 897 KiB/s, 16 objects/s recovering 2024-09-21T11:56:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:27 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:27 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/373189536' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:56:27.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:27 smithi175 ceph-mon[32613]: osdmap e387: 8 total, 8 up, 8 in 2024-09-21T11:56:27.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:27.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:27.831 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:27.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:27.958 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:27.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:27.960 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:27.962 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:56:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:56:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:56:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:56:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245657 8.b 8:db11300a:::benchmark_data_smithi043_97042_object108563:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:56:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:56:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.22619483, 2024-09-21T11:56:27.962 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167927969999999, 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245657 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.963 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565747+0000", 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5340000000000001e-06 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565752+0000", 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8459999999999999e-06 2024-09-21T11:56:27.964 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565762+0000", 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6029999999999994e-06 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565769+0000", 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3019999999999997e-06 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:27.965 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565833+0000", 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4139000000000004e-05 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565842+0000", 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2550000000000003e-06 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576490+0000", 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.010647965 2024-09-21T11:56:27.966 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576618+0000", 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000128233 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576763+0000", 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000144845 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582393+0000", 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056301229999999999 2024-09-21T11:56:27.967 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582458+0000", 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4715000000000006e-05 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582472+0000", 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3985000000000001e-05 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582509+0000", 2024-09-21T11:56:27.968 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7063000000000001e-05 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582538+0000", 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9189000000000001e-05 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245660 8.b 8:d0b775d8:::benchmark_data_smithi043_97042_object108566:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.22326979299999, 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0141818570000001, 2024-09-21T11:56:27.969 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245660 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.970 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0200000000000003e-07 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568673+0000", 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2029999999999999e-06 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568679+0000", 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7590000000000003e-06 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568684+0000", 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1800000000000004e-06 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568765+0000", 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0898999999999999e-05 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568774+0000", 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2e-06 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.579784+0000", 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0110101520000001 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.579907+0000", 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000122404 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.973 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.580042+0000", 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000135205 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582646+0000", 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026039240000000001 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582746+0000", 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100447 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.974 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582764+0000", 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7265000000000002e-05 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582813+0000", 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9487999999999998e-05 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582852+0000", 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9329000000000002e-05 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:27.975 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245661 8.b 8:d3f46cc5:::benchmark_data_smithi043_97042_object108567:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.22324097900002, 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026528527, 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:56:27.976 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245661 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:27.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568700+0000", 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9200000000000001e-07 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568702+0000", 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4049999999999998e-06 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568708+0000", 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0900000000000001e-06 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.978 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568715+0000", 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3319999999999997e-06 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568799+0000", 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4618999999999997e-05 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568806+0000", 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3239999999999999e-06 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.979 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582891+0000", 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0140855200000001 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.583014+0000", 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012294699999999999 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.583136+0000", 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121651 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.980 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588881+0000", 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057450959999999999 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588920+0000", 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9140999999999997e-05 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595113+0000", 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061930309999999999 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.981 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595177+0000", 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4243000000000001e-05 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595228+0000", 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0636000000000001e-05 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.982 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245694 8.b 8:d4a2706b:::benchmark_data_smithi043_97042_object108600:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:56:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:56:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.19382767399998, 2024-09-21T11:56:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99723669299999995, 2024-09-21T11:56:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:27.983 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:56:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:56:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245694 2024-09-21T11:56:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.984 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:56:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:56:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:27.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598114+0000", 2024-09-21T11:56:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.203e-06 2024-09-21T11:56:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598118+0000", 2024-09-21T11:56:27.986 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3760000000000001e-06 2024-09-21T11:56:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598129+0000", 2024-09-21T11:56:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0373999999999999e-05 2024-09-21T11:56:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598137+0000", 2024-09-21T11:56:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-21T11:56:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:27.988 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598208+0000", 2024-09-21T11:56:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1235000000000005e-05 2024-09-21T11:56:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.989 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598217+0000", 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0480000000000004e-06 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585700+0000", 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98748349499999999 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585809+0000", 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010897500000000001 2024-09-21T11:56:27.990 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585910+0000", 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100326 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588958+0000", 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030481290000000001 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595096+0000", 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.006137829 2024-09-21T11:56:27.991 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595241+0000", 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014534899999999999 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595298+0000", 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7256999999999997e-05 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595349+0000", 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1267999999999998e-05 2024-09-21T11:56:27.992 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245711 8.b 8:d32e232b:::benchmark_data_smithi043_97042_object108617:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.177585798, 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98116202900000005, 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:56:27.993 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245711 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.994 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614357+0000", 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9910000000000001e-06 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614362+0000", 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8119999999999996e-06 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614381+0000", 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8799000000000001e-05 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.995 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614390+0000", 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6289999999999999e-06 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614452+0000", 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2620999999999996e-05 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614462+0000", 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9869999999999994e-06 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588986+0000", 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.974523374 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.589275+0000", 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028897299999999999 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.589384+0000", 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109497 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595361+0000", 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059765479999999999 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595400+0000", 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8847999999999998e-05 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.998 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595444+0000", 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4270999999999997e-05 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595473+0000", 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9339000000000001e-05 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595517+0000", 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3340000000000002e-05 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:27.999 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245713 8.b 8:d46d9f23:::benchmark_data_smithi043_97042_object108619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.17678264300002, 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98051237099999999, 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245713 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.000 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615159+0000", 2024-09-21T11:56:28.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.339e-06 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615164+0000", 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6659999999999999e-06 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615180+0000", 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5835e-05 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615188+0000", 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7600000000000008e-06 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615244+0000", 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5172e-05 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615252+0000", 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6969999999999999e-06 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592015+0000", 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97676251999999997 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592146+0000", 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000131094 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592299+0000", 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015273399999999999 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595406+0000", 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031073920000000001 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595425+0000", 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9119000000000001e-05 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595582+0000", 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001573 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595632+0000", 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9282e-05 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595670+0000", 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8461000000000003e-05 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245721 8.b 8:de2b861f:::benchmark_data_smithi043_97042_object108627:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:56:28.006 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.16887691400001, 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98811751400000003, 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245721 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:56:28.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623065+0000", 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.553e-06 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623072+0000", 2024-09-21T11:56:28.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4729999999999997e-06 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623087+0000", 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5197999999999999e-05 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623095+0000", 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2369999999999992e-06 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623159+0000", 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3953000000000002e-05 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623168+0000", 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0350000000000007e-06 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595704+0000", 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97253655100000003 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595813+0000", 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108168 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595916+0000", 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103482 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604767+0000", 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088508360000000008 2024-09-21T11:56:28.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604858+0000", 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0768999999999998e-05 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611090+0000", 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0062325690000000003 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611136+0000", 2024-09-21T11:56:28.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5197000000000001e-05 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611181+0000", 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5493e-05 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245797 8.b 8:d43bc7ef:::benchmark_data_smithi043_97042_object108703:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.09063289800002, 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.910005379, 2024-09-21T11:56:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245797 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701309+0000", 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1999999999999999e-06 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701315+0000", 2024-09-21T11:56:28.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701326+0000", 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0956e-05 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701334+0000", 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7570000000000004e-06 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701426+0000", 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2200999999999994e-05 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701436+0000", 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0175e-05 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.598806+0000", 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.89737045500000001 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.598902+0000", 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5308000000000003e-05 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.599142+0000", 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024078399999999999 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604823+0000", 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056808700000000002 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.610984+0000", 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061603750000000001 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611191+0000", 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000207612 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611273+0000", 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1924999999999999e-05 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611313+0000", 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9651999999999999e-05 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245872 8.b 8:d0b8f221:::benchmark_data_smithi043_97042_object108778:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.00328408000001, 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.82278842799999996, 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.020 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245872 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788658+0000", 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.708e-06 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788664+0000", 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5140000000000001e-06 2024-09-21T11:56:28.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788679+0000", 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5764e-05 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788691+0000", 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.173e-05 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788774+0000", 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3029999999999996e-05 2024-09-21T11:56:28.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788783+0000", 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9840000000000007e-06 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.601802+0000", 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81301899799999999 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.601917+0000", 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011464 2024-09-21T11:56:28.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.602046+0000", 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000129228 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604894+0000", 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028480250000000001 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611032+0000", 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061384710000000004 2024-09-21T11:56:28.025 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611319+0000", 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028687299999999999 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611405+0000", 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5921999999999995e-05 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.026 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611445+0000", 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9541e-05 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245873 8.b 8:dccc5dce:::benchmark_data_smithi043_97042_object108779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.00322722700002, 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.82286793400000002, 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.027 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245873 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:56:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788714+0000", 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5300000000000004e-07 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788717+0000", 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2849999999999999e-06 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788723+0000", 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3300000000000004e-06 2024-09-21T11:56:28.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788729+0000", 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3979999999999998e-06 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788805+0000", 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6068000000000004e-05 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788810+0000", 2024-09-21T11:56:28.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4720000000000003e-06 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604923+0000", 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81611315500000003 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.605021+0000", 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7503999999999993e-05 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.605122+0000", 2024-09-21T11:56:28.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010059400000000001 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.610912+0000", 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057902530000000004 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611078+0000", 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016624000000000001 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611455+0000", 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037679500000000003 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611538+0000", 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3288999999999995e-05 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611581+0000", 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2997999999999997e-05 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.033 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194299 10.4 10:28bf67cd:::benchmark_data_smithi043_150678_object71548:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.581309+0000", 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.210631775000003, 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1019873919999998, 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:28.034 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194299 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581309+0000", 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581309+0000", 2024-09-21T11:56:28.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581310+0000", 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3769999999999999e-06 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581319+0000", 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4210000000000001e-06 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581327+0000", 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7389999999999999e-06 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581334+0000", 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1080000000000004e-06 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581401+0000", 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6445000000000005e-05 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581411+0000", 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9939999999999995e-06 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556211+0000", 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97480030200000001 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556223+0000", 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1844e-05 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.675650+0000", 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1194268890000001 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.677880+0000", 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00223026 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.678201+0000", 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032071399999999999 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.682954+0000", 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047530539999999996 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683014+0000", 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0504999999999998e-05 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683069+0000", 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4252000000000002e-05 2024-09-21T11:56:28.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683086+0000", 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7492000000000001e-05 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683148+0000", 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1648000000000003e-05 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683296+0000", 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148348 2024-09-21T11:56:28.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194318 10.4 10:2adde221:::benchmark_data_smithi043_150678_object71567:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.600714+0000", 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.191226331999999, 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0883899850000001, 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194318 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600714+0000", 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600714+0000", 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600716+0000", 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600754+0000", 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7953000000000001e-05 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600787+0000", 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2662e-05 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600797+0000", 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0239e-05 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600975+0000", 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017730400000000001 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600985+0000", 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0231000000000001e-05 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556251+0000", 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.95526609299999998 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556266+0000", 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4663e-05 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683516+0000", 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.127250428 2024-09-21T11:56:28.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683657+0000", 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014146699999999999 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683806+0000", 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148131 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688816+0000", 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050102640000000004 2024-09-21T11:56:28.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688855+0000", 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9101999999999999e-05 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688866+0000", 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1002000000000001e-05 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688937+0000", 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0576999999999999e-05 2024-09-21T11:56:28.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688994+0000", 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7679999999999997e-05 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689104+0000", 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010983400000000001 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.049 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194320 10.4 10:287edeca:::benchmark_data_smithi043_150678_object71569:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.600823+0000", 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.191117495, 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.088657736, 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194320 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600823+0000", 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600823+0000", 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600823+0000", 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6199999999999998e-07 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600827+0000", 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600833+0000", 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0599999999999996e-06 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600838+0000", 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.601008+0000", 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000169968 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.601014+0000", 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4969999999999998e-06 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556293+0000", 2024-09-21T11:56:28.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.95527826599999999 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556301+0000", 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2570000000000007e-06 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.686056+0000", 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1297548079999999 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.054 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.686143+0000", 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6985999999999997e-05 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.686286+0000", 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014329699999999999 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688875+0000", 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025888059999999999 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689175+0000", 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000300167 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689228+0000", 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3185e-05 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689294+0000", 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5982000000000006e-05 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689359+0000", 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4987000000000006e-05 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689481+0000", 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012158 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.057 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194340 10.4 10:236291d1:::benchmark_data_smithi043_150678_object71589:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.620209+0000", 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.171731852000001, 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.101492264, 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194340 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620209+0000", 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620209+0000", 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620211+0000", 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-06 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620217+0000", 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2060000000000004e-06 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620234+0000", 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7153999999999999e-05 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620242+0000", 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3960000000000006e-06 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620374+0000", 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013184700000000001 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620384+0000", 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7019999999999996e-06 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556329+0000", 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.93594473600000005 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556337+0000", 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9750000000000007e-06 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689522+0000", 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1331854770000001 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689648+0000", 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125826 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689794+0000", 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000146305 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698662+0000", 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088681130000000004 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698680+0000", 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7312e-05 2024-09-21T11:56:28.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698782+0000", 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010262199999999999 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.721411+0000", 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022628385000000001 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.721581+0000", 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017054099999999999 2024-09-21T11:56:28.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.721701+0000", 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119497 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194360 10.4 10:22a7317d:::benchmark_data_smithi043_150678_object71609:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.638142+0000", 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.153798809000001, 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0838764699999999, 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.065 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194360 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638142+0000", 2024-09-21T11:56:28.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638142+0000", 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638144+0000", 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.984e-06 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638150+0000", 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3969999999999999e-06 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638170+0000", 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9559e-05 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638178+0000", 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5499999999999995e-06 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638293+0000", 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114521 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638304+0000", 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287000000000001e-05 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556366+0000", 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.91806218399999995 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556375+0000", 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7190000000000007e-06 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.692612+0000", 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.136237707 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.692769+0000", 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015609 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.692911+0000", 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014248599999999999 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698686+0000", 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057744409999999999 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698734+0000", 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8278000000000001e-05 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698864+0000", 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013069199999999999 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.721750+0000", 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022885784999999999 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.721905+0000", 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015452599999999999 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722018+0000", 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011326400000000001 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194367 10.4 10:201541d4:::benchmark_data_smithi043_150678_object71616:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.646661+0000", 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.145279256999999, 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0756208169999999, 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194367 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646661+0000", 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646661+0000", 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646663+0000", 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4500000000000001e-06 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646676+0000", 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2972999999999999e-05 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646690+0000", 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4127999999999999e-05 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646696+0000", 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4099999999999996e-06 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646766+0000", 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0028000000000001e-05 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646775+0000", 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7290000000000006e-06 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556406+0000", 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.90963151200000003 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556416+0000", 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4350000000000003e-06 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.695728+0000", 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1393118389999999 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.695829+0000", 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101401 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.695908+0000", 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9169e-05 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.078 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698691+0000", 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027825929999999999 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706414+0000", 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077236359999999999 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706463+0000", 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8832999999999998e-05 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722087+0000", 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015623662 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722156+0000", 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9255e-05 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722282+0000", 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125764 2024-09-21T11:56:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194376 10.4 10:26311a81:::benchmark_data_smithi043_150678_object71625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.658363+0000", 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.133577623000001, 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.064158495, 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194376 2024-09-21T11:56:28.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658363+0000", 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658363+0000", 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658364+0000", 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.655e-06 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658371+0000", 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4239999999999998e-06 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658389+0000", 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8346e-05 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658397+0000", 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5070000000000003e-06 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658468+0000", 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0887000000000001e-05 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658480+0000", 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2006e-05 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556442+0000", 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.89796246999999996 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556449+0000", 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4109999999999999e-06 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698913+0000", 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.142463979 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.699050+0000", 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136286 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.699189+0000", 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013936299999999999 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706428+0000", 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072391399999999998 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706501+0000", 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170999999999995e-05 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706541+0000", 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9657000000000003e-05 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722333+0000", 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015791667999999998 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722392+0000", 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9348000000000002e-05 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722521+0000", 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012917699999999999 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194389 10.4 10:2e49ec61:::benchmark_data_smithi043_150678_object71638:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.674458+0000", 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.117482639000002, 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0482976590000002, 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194389 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.089 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674458+0000", 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674458+0000", 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674460+0000", 2024-09-21T11:56:28.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9829999999999999e-06 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674467+0000", 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0389999999999999e-06 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674491+0000", 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3977000000000001e-05 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674500+0000", 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9409999999999996e-06 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674578+0000", 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8226999999999995e-05 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674591+0000", 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3202e-05 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556476+0000", 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.88188519700000001 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556484+0000", 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.396e-06 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.703276+0000", 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.146791801 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.703405+0000", 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000129615 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.703540+0000", 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000134778 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.712886+0000", 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093464740000000004 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.712929+0000", 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2154999999999999e-05 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.720772+0000", 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007843104 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722572+0000", 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001800711 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722634+0000", 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1736000000000007e-05 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722755+0000", 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121323 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194396 10.4 10:2cc12a85:::benchmark_data_smithi043_150678_object71645:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.689435+0000", 2024-09-21T11:56:28.096 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.10250576, 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0335722509999998, 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194396 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689435+0000", 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689435+0000", 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689436+0000", 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4890000000000001e-06 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689442+0000", 2024-09-21T11:56:28.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.818e-06 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689455+0000", 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2819999999999999e-05 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689462+0000", 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4580000000000004e-06 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689528+0000", 2024-09-21T11:56:28.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6221000000000002e-05 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689536+0000", 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9929999999999995e-06 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556510+0000", 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.86697341500000003 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556518+0000", 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-21T11:56:28.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706571+0000", 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.150052691 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706680+0000", 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010967999999999999 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706801+0000", 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120394 2024-09-21T11:56:28.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.712969+0000", 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061681189999999997 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.713014+0000", 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4926999999999999e-05 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.720775+0000", 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077615119999999999 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722800+0000", 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002024311 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722857+0000", 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7161999999999999e-05 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.723007+0000", 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015021699999999999 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194404 10.4 10:2c63f935:::benchmark_data_smithi043_150678_object71653:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.718920+0000", 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 60.073020247000002, 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0043187869999999, 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:56:28.104 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194404 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718920+0000", 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718920+0000", 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:56:28.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718922+0000", 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4100000000000001e-06 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718927+0000", 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7749999999999998e-06 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718939+0000", 2024-09-21T11:56:28.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2038000000000001e-05 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718948+0000", 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.719014+0000", 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6309000000000005e-05 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.719021+0000", 2024-09-21T11:56:28.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0700000000000001e-06 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.558746+0000", 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.839724688 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.558767+0000", 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0749000000000001e-05 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:56:28.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.709717+0000", 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1509502739999999 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.709865+0000", 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148589 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.710005+0000", 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001392 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.720727+0000", 2024-09-21T11:56:28.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010722687999999999 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.720764+0000", 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6353000000000002e-05 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.720780+0000", 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5795e-05 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:28.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.723059+0000", 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022793800000000001 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.723117+0000", 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8557000000000002e-05 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.723239+0000", 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121462 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.111 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:56:28.112 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:56:28.112 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:56:28.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:28.135 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:28.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:28.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:28.239 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:28.264 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:28.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:28.265 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:28.366 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:28.391 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:28.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:28.392 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:28.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:28.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:28.518 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:28.619 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:28.645 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:28.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:28.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:28.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:28 smithi043 ceph-mon[29710]: 3.a scrub starts 2024-09-21T11:56:28.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:28 smithi043 ceph-mon[29710]: 3.a scrub ok 2024-09-21T11:56:28.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:28 smithi043 ceph-mon[29710]: 10.8 scrub starts 2024-09-21T11:56:28.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:28 smithi043 ceph-mon[29710]: 10.8 scrub ok 2024-09-21T11:56:28.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:28 smithi043 ceph-mon[36454]: 3.a scrub starts 2024-09-21T11:56:28.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:28 smithi043 ceph-mon[36454]: 3.a scrub ok 2024-09-21T11:56:28.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:28 smithi043 ceph-mon[36454]: 10.8 scrub starts 2024-09-21T11:56:28.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:28 smithi043 ceph-mon[36454]: 10.8 scrub ok 2024-09-21T11:56:28.747 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:28.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:28 smithi175 ceph-mon[32613]: 3.a scrub starts 2024-09-21T11:56:28.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:28 smithi175 ceph-mon[32613]: 3.a scrub ok 2024-09-21T11:56:28.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:28 smithi175 ceph-mon[32613]: 10.8 scrub starts 2024-09-21T11:56:28.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:28 smithi175 ceph-mon[32613]: 10.8 scrub ok 2024-09-21T11:56:28.772 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:28.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:28.773 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:28.874 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:28.899 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:28.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:28.900 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:28.930 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:56:28.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:56:29.001 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:29.023 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:29.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:29.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:29.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:29.148 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:29.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:29.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:29.202 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:29.250 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:29.259 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:56:29.260 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:56:29.273 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:29.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:29.274 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:29.286 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:56:29.286 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:56:29.303 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:56:29.303 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:56:29.319 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:56:29.319 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:56:29.335 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:56:29.335 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:56:29.352 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:56:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:56:29.369 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:56:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:56:29.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:29.387 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:56:29.388 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:56:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:29.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:29.401 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:29.501 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:29.524 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:29.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:29.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:29.589 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:56:29.589 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:56:29.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:29.640 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 10.2 scrub starts 2024-09-21T11:56:29.640 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 10.2 scrub ok 2024-09-21T11:56:29.640 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 10.3 scrub starts 2024-09-21T11:56:29.640 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 10.3 scrub ok 2024-09-21T11:56:29.641 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 2.2 scrub starts 2024-09-21T11:56:29.641 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 2.2 scrub ok 2024-09-21T11:56:29.641 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: pgmap v796: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 198 op/s; 832 KiB/s, 14 objects/s recovering 2024-09-21T11:56:29.641 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 6.c scrub starts 2024-09-21T11:56:29.641 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 6.c scrub ok 2024-09-21T11:56:29.641 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 4.8 scrub starts 2024-09-21T11:56:29.641 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 4.8 scrub ok 2024-09-21T11:56:29.642 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 4.5 scrub starts 2024-09-21T11:56:29.642 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 4.5 scrub ok 2024-09-21T11:56:29.642 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3440132860' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:29.642 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 3.3 scrub starts 2024-09-21T11:56:29.642 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:29 smithi175 ceph-mon[32613]: 3.3 scrub ok 2024-09-21T11:56:29.648 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:29.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:29.650 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 10.2 scrub starts 2024-09-21T11:56:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 10.2 scrub ok 2024-09-21T11:56:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 10.3 scrub starts 2024-09-21T11:56:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 10.3 scrub ok 2024-09-21T11:56:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 2.2 scrub starts 2024-09-21T11:56:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 2.2 scrub ok 2024-09-21T11:56:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: pgmap v796: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 198 op/s; 832 KiB/s, 14 objects/s recovering 2024-09-21T11:56:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 6.c scrub starts 2024-09-21T11:56:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 6.c scrub ok 2024-09-21T11:56:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 4.8 scrub starts 2024-09-21T11:56:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 4.8 scrub ok 2024-09-21T11:56:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 4.5 scrub starts 2024-09-21T11:56:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 4.5 scrub ok 2024-09-21T11:56:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3440132860' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 3.3 scrub starts 2024-09-21T11:56:29.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[36454]: 3.3 scrub ok 2024-09-21T11:56:29.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 10.2 scrub starts 2024-09-21T11:56:29.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 10.2 scrub ok 2024-09-21T11:56:29.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 10.3 scrub starts 2024-09-21T11:56:29.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 10.3 scrub ok 2024-09-21T11:56:29.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 2.2 scrub starts 2024-09-21T11:56:29.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 2.2 scrub ok 2024-09-21T11:56:29.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: pgmap v796: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 198 op/s; 832 KiB/s, 14 objects/s recovering 2024-09-21T11:56:29.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 6.c scrub starts 2024-09-21T11:56:29.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 6.c scrub ok 2024-09-21T11:56:29.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 4.8 scrub starts 2024-09-21T11:56:29.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 4.8 scrub ok 2024-09-21T11:56:29.745 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 4.5 scrub starts 2024-09-21T11:56:29.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 4.5 scrub ok 2024-09-21T11:56:29.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3440132860' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:29.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 3.3 scrub starts 2024-09-21T11:56:29.746 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:29 smithi043 ceph-mon[29710]: 3.3 scrub ok 2024-09-21T11:56:29.747 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:29.747 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:29.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:29.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:29.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:29.876 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:29.899 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:29.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:29.901 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:30.002 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:30.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:30.025 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:30.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:30.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:30.150 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:30.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:30.152 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:30.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:30.276 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:30.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:30.277 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:30.378 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:30.401 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:30.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:30.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:30.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:56:30.465 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:30.504 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:30.524 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:30.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:30.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:30.626 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:30.647 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:30.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:30.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:30.690 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[36454]: 3.e scrub starts 2024-09-21T11:56:30.690 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[36454]: 3.e scrub ok 2024-09-21T11:56:30.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[36454]: 4.17 scrub starts 2024-09-21T11:56:30.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[36454]: 4.17 scrub ok 2024-09-21T11:56:30.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:56:30.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:56:30.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[29710]: 3.e scrub starts 2024-09-21T11:56:30.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[29710]: 3.e scrub ok 2024-09-21T11:56:30.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[29710]: 4.17 scrub starts 2024-09-21T11:56:30.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[29710]: 4.17 scrub ok 2024-09-21T11:56:30.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:56:30.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:30 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:56:30.706 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:30.749 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:30.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:30 smithi175 ceph-mon[32613]: 3.e scrub starts 2024-09-21T11:56:30.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:30 smithi175 ceph-mon[32613]: 3.e scrub ok 2024-09-21T11:56:30.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:30 smithi175 ceph-mon[32613]: 4.17 scrub starts 2024-09-21T11:56:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:30 smithi175 ceph-mon[32613]: 4.17 scrub ok 2024-09-21T11:56:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:30 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:56:30.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:30 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:56:30.770 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:30.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:30.771 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:30.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:30.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:30.897 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:30.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:31.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:31.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:31.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:31.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:31.144 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:31.145 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:31.189 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:31.189 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:31.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:31.268 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:31.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:31.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:31.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:31.392 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:31.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:31.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:31.471 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:56:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'unique_pool_3' removed 2024-09-21T11:56:31.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:31.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:31.534 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:31.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:31.658 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:31.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:31.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:31.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[29710]: pgmap v797: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 168 op/s; 707 KiB/s, 12 objects/s recovering 2024-09-21T11:56:31.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[29710]: osdmap e388: 8 total, 8 up, 8 in 2024-09-21T11:56:31.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1848402123' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:56:31.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[29710]: 5.f scrub starts 2024-09-21T11:56:31.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[29710]: 5.f scrub ok 2024-09-21T11:56:31.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[29710]: 10.f scrub starts 2024-09-21T11:56:31.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[29710]: 10.f scrub ok 2024-09-21T11:56:31.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3088332498' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[36454]: pgmap v797: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 168 op/s; 707 KiB/s, 12 objects/s recovering 2024-09-21T11:56:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[36454]: osdmap e388: 8 total, 8 up, 8 in 2024-09-21T11:56:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1848402123' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:56:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[36454]: 5.f scrub starts 2024-09-21T11:56:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[36454]: 5.f scrub ok 2024-09-21T11:56:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[36454]: 10.f scrub starts 2024-09-21T11:56:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[36454]: 10.f scrub ok 2024-09-21T11:56:31.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3088332498' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:31.751 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:31 smithi175 ceph-mon[32613]: pgmap v797: 148 pgs: 1 active+remapped+backfill_toofull, 147 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 168 op/s; 707 KiB/s, 12 objects/s recovering 2024-09-21T11:56:31.751 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:31 smithi175 ceph-mon[32613]: osdmap e388: 8 total, 8 up, 8 in 2024-09-21T11:56:31.751 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1848402123' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T11:56:31.751 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:31 smithi175 ceph-mon[32613]: 5.f scrub starts 2024-09-21T11:56:31.751 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:31 smithi175 ceph-mon[32613]: 5.f scrub ok 2024-09-21T11:56:31.751 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:31 smithi175 ceph-mon[32613]: 10.f scrub starts 2024-09-21T11:56:31.751 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:31 smithi175 ceph-mon[32613]: 10.f scrub ok 2024-09-21T11:56:31.751 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3088332498' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:31.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:31.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:31.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:31.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:31.913 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:31.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:31.914 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:31.915 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:56:32.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:56:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:56:32.015 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:32.036 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:32.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:32.037 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:32.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:32.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:56:32.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:32.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:32.204 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:32.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:32.286 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:32.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:32.287 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:32.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:32.410 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:32.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:32.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:32.512 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:32.534 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:32.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:32.535 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:32.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:32 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:56:32.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:32 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1848402123' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:56:32.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:32 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3088332498' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:32.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:32 smithi175 ceph-mon[32613]: osdmap e389: 8 total, 8 up, 8 in 2024-09-21T11:56:32.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:32 smithi175 ceph-mon[32613]: 4.19 scrub starts 2024-09-21T11:56:32.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:32 smithi175 ceph-mon[32613]: 4.19 scrub ok 2024-09-21T11:56:32.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:32 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:32.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:32.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:32.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:32.665 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:32.665 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:32.688 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:32.688 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:56:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1848402123' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:56:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3088332498' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[36454]: osdmap e389: 8 total, 8 up, 8 in 2024-09-21T11:56:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[36454]: 4.19 scrub starts 2024-09-21T11:56:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[36454]: 4.19 scrub ok 2024-09-21T11:56:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:32.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:32.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:56:32.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1848402123' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T11:56:32.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3088332498' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:32.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[29710]: osdmap e389: 8 total, 8 up, 8 in 2024-09-21T11:56:32.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[29710]: 4.19 scrub starts 2024-09-21T11:56:32.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[29710]: 4.19 scrub ok 2024-09-21T11:56:32.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:32.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:32.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:32.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:32.792 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:32.792 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:32.893 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:32.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:32.917 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:32.917 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:32.987 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-21T11:56:32.988 INFO:tasks.radosbench:Beginning radosbench... 2024-09-21T11:56:32.988 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-21T11:56:32.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool create unique_pool_4 16 2024-09-21T11:56:33.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:33.032 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:33.055 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:33.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:33.056 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:33.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:33.180 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:33.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:33.181 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:33.282 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:33.305 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:33.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:33.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:33.315 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:56:33.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:33.430 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:33.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:33.431 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:33.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:33.554 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:33.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:33.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:33.604 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:33.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:33.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[29710]: pgmap v800: 132 pgs: 132 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T11:56:33.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.3", "id": [2, 0]} : dispatch 2024-09-21T11:56:33.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.3", "id": [2, 0]} : dispatch 2024-09-21T11:56:33.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[29710]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-21T11:56:33.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:56:33.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:56:33.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:56:33.623 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:33.623 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[36454]: pgmap v800: 132 pgs: 132 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T11:56:33.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.3", "id": [2, 0]} : dispatch 2024-09-21T11:56:33.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.3", "id": [2, 0]} : dispatch 2024-09-21T11:56:33.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[36454]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-21T11:56:33.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:56:33.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:56:33.624 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:33 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:56:33.656 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:33.677 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:33.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:33.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:33.725 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:56:33.725 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:56:33.726 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:33 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:33.726 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:33 smithi175 ceph-mon[32613]: pgmap v800: 132 pgs: 132 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-21T11:56:33.726 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:33 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.3", "id": [2, 0]} : dispatch 2024-09-21T11:56:33.726 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.3", "id": [2, 0]} : dispatch 2024-09-21T11:56:33.726 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:33 smithi175 ceph-mon[32613]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-21T11:56:33.726 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:33 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:56:33.726 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:33 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:56:33.726 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:33 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:56:33.744 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:56:33.744 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:56:33.759 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:33.762 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:56:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:56:33.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:33.792 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:56:33.793 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:56:33.801 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:33.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:33.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:33.810 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:56:33.810 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:56:33.827 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:56:33.828 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:56:33.845 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:56:33.845 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:56:33.861 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:56:33.862 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:56:33.903 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:33.927 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:33.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:33.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:34.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:34.050 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:34.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:34.051 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:34.108 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:56:34.108 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:56:34.108 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:56:34.109 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:56:34.152 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:34.174 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:34.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:34.175 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:34.276 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:34.297 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:34.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:34.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:34.398 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:34.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:34.421 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:34.421 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:34.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:56:34.522 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:34.543 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:34.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:34.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:34.645 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:34.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[36454]: 3.13 scrub starts 2024-09-21T11:56:34.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[36454]: 3.13 scrub ok 2024-09-21T11:56:34.647 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.3", "id": [2, 0]}]': finished 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[36454]: osdmap e390: 8 total, 8 up, 8 in 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[36454]: 5.1c scrub starts 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[36454]: 5.1c scrub ok 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2763895717' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[36454]: 6.1e scrub starts 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[36454]: 6.1e scrub ok 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[29710]: 3.13 scrub starts 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[29710]: 3.13 scrub ok 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.3", "id": [2, 0]}]': finished 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[29710]: osdmap e390: 8 total, 8 up, 8 in 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[29710]: 5.1c scrub starts 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[29710]: 5.1c scrub ok 2024-09-21T11:56:34.648 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2763895717' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[29710]: 6.1e scrub starts 2024-09-21T11:56:34.649 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:34 smithi043 ceph-mon[29710]: 6.1e scrub ok 2024-09-21T11:56:34.666 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:34.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:34.667 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:34 smithi175 ceph-mon[32613]: 3.13 scrub starts 2024-09-21T11:56:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:34 smithi175 ceph-mon[32613]: 3.13 scrub ok 2024-09-21T11:56:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:34 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.3", "id": [2, 0]}]': finished 2024-09-21T11:56:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:34 smithi175 ceph-mon[32613]: osdmap e390: 8 total, 8 up, 8 in 2024-09-21T11:56:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:34 smithi175 ceph-mon[32613]: 5.1c scrub starts 2024-09-21T11:56:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:34 smithi175 ceph-mon[32613]: 5.1c scrub ok 2024-09-21T11:56:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2763895717' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:34 smithi175 ceph-mon[32613]: 6.1e scrub starts 2024-09-21T11:56:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:34 smithi175 ceph-mon[32613]: 6.1e scrub ok 2024-09-21T11:56:34.768 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:34.789 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:34.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:34.790 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:34.890 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:34.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:56:34.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:34.916 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:34.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:35.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:35.041 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:35.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:35.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:35.113 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:35.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:35.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:35.167 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:35.202 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:35.267 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:35.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:35.289 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:35.289 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:35.368 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:56:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:35.391 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:35.412 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:35.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:35.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:35.492 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'unique_pool_4' created 2024-09-21T11:56:35.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:35.539 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:35.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:35.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:35.579 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:35 smithi175 ceph-mon[32613]: 4.15 scrub starts 2024-09-21T11:56:35.579 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:35 smithi175 ceph-mon[32613]: pgmap v802: 132 pgs: 132 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:56:35.579 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:35 smithi175 ceph-mon[32613]: 4.15 scrub ok 2024-09-21T11:56:35.579 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:35 smithi175 ceph-mon[32613]: osdmap e391: 8 total, 8 up, 8 in 2024-09-21T11:56:35.579 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:35 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/905109374' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16} : dispatch 2024-09-21T11:56:35.579 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:35 smithi175 ceph-mon[32613]: 4.3 scrub starts 2024-09-21T11:56:35.579 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:35 smithi175 ceph-mon[32613]: 4.3 scrub ok 2024-09-21T11:56:35.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:35.664 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:35.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:35.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:35.729 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:56:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:56:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:56:35.729 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:56:35.729 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:56:35.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[29710]: 4.15 scrub starts 2024-09-21T11:56:35.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[29710]: pgmap v802: 132 pgs: 132 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:56:35.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[29710]: 4.15 scrub ok 2024-09-21T11:56:35.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[29710]: osdmap e391: 8 total, 8 up, 8 in 2024-09-21T11:56:35.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/905109374' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16} : dispatch 2024-09-21T11:56:35.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[29710]: 4.3 scrub starts 2024-09-21T11:56:35.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[29710]: 4.3 scrub ok 2024-09-21T11:56:35.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[36454]: 4.15 scrub starts 2024-09-21T11:56:35.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[36454]: pgmap v802: 132 pgs: 132 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-21T11:56:35.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[36454]: 4.15 scrub ok 2024-09-21T11:56:35.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[36454]: osdmap e391: 8 total, 8 up, 8 in 2024-09-21T11:56:35.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/905109374' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16} : dispatch 2024-09-21T11:56:35.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[36454]: 4.3 scrub starts 2024-09-21T11:56:35.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:35 smithi043 ceph-mon[36454]: 4.3 scrub ok 2024-09-21T11:56:35.765 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:35.786 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:35.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:35.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:35.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:35.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:35.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:35.912 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:36.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:36.035 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:36.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:36.036 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:36.137 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:36.159 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:36.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:36.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:36.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-21T11:56:36.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:36.283 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:36.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:36.284 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:36.377 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:56:36.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:36.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:36.407 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:36.502 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:56:36.509 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:36.517 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:36.529 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:36.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:36.530 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:36.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:36.632 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:36.653 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:36.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:36.654 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:36.753 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:36 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/905109374' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-21T11:56:36.753 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:36 smithi175 ceph-mon[32613]: osdmap e392: 8 total, 8 up, 8 in 2024-09-21T11:56:36.753 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:36 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1241844214' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:36.753 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:36 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:36.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:36.777 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:36.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:36.777 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:36.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:36 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/905109374' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-21T11:56:36.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:36 smithi043 ceph-mon[29710]: osdmap e392: 8 total, 8 up, 8 in 2024-09-21T11:56:36.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:36 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1241844214' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:36.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:36 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:36.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:36 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/905109374' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-21T11:56:36.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:36 smithi043 ceph-mon[36454]: osdmap e392: 8 total, 8 up, 8 in 2024-09-21T11:56:36.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:36 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1241844214' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:36.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:36 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:56:36.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:36.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:36.901 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:36.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:37.002 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:37.024 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:37.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:37.025 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:37.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:56:37.104 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:56:37.104 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:166707 9.6 9:627f9972:::benchmark_data_smithi043_126002_object39641:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:18.109124+0000", 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 318.99251449799999, 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18514305, 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 166707 2024-09-21T11:56:37.105 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109124+0000", 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109124+0000", 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.106 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109125+0000", 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.362e-06 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109130+0000", 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3789999999999999e-06 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109140+0000", 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9739999999999997e-06 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.107 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109147+0000", 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2289999999999997e-06 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109198+0000", 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1131999999999998e-05 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109305+0000", 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010662099999999999 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.108 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109527+0000", 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022260199999999999 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.112844+0000", 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003316608 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.113270+0000", 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042589800000000002 2024-09-21T11:56:37.109 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.294193+0000", 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.180923054 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.294233+0000", 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9733000000000001e-05 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.294267+0000", 2024-09-21T11:56:37.110 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4458000000000001e-05 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:188198 9.7 9:ed836cf1:::benchmark_data_smithi043_126002_object61132:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e291)", 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:36.279169+0000", 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 300.82246949300003, 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.179365786, 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:37.111 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 188198 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.279169+0000", 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.279169+0000", 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.112 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.279170+0000", 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5099999999999998e-07 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.279174+0000", 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8589999999999998e-06 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.279186+0000", 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1892999999999999e-05 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.279192+0000", 2024-09-21T11:56:37.113 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5760000000000002e-06 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.285016+0000", 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0058240339999999996 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.285156+0000", 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000139345 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.285282+0000", 2024-09-21T11:56:37.114 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000126551 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.291699+0000", 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0064168189999999998 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.291765+0000", 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6000999999999994e-05 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.458441+0000", 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.16667607000000001 2024-09-21T11:56:37.115 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.458489+0000", 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8250000000000001e-05 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:36.458535+0000", 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5436999999999999e-05 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.116 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211066 9.a 9:51f636b8:::benchmark_data_smithi043_126002_object84000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 281.68966677600002, 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.30144848499999999, 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:37.117 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211066 2024-09-21T11:56:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:56:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.118 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411975+0000", 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.401e-06 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.119 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411979+0000", 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4889999999999997e-06 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411984+0000", 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2530000000000004e-06 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412132+0000", 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014723399999999999 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.120 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412228+0000", 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6713999999999996e-05 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,7", 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412367+0000", 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013867400000000001 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415401+0000", 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030337630000000001 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.121 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.421153+0000", 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057522909999999997 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.713334+0000", 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29218145600000001 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.713381+0000", 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6492000000000001e-05 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.122 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.713420+0000", 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9387000000000001e-05 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67189 10.6 10:65d25444:::benchmark_data_smithi043_150678_object67188:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.338348+0000", 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.763290127, 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.210603178, 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.123 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67189 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338348+0000", 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338348+0000", 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.124 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338350+0000", 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4670000000000001e-06 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338456+0000", 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010598399999999999 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338466+0000", 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0482999999999999e-05 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338473+0000", 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6810000000000001e-06 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338544+0000", 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0992000000000001e-05 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338681+0000", 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000137059 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:37.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338828+0000", 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146582 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.340772+0000", 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019443959999999999 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.341306+0000", 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00053431300000000002 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.548824+0000", 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.207518006 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.548901+0000", 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6970000000000003e-05 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.548952+0000", 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0244999999999998e-05 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.128 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67201 10.6 10:64739ad8:::benchmark_data_smithi043_150678_object67200:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.347253+0000", 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.754385657, 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.20183933500000001, 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67201 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347253+0000", 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347253+0000", 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347255+0000", 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.835e-06 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347359+0000", 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104629 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347374+0000", 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4418e-05 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347385+0000", 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0807e-05 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347457+0000", 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2028999999999998e-05 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347597+0000", 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013995899999999999 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347765+0000", 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016854899999999999 2024-09-21T11:56:37.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.350194+0000", 2024-09-21T11:56:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002429198 2024-09-21T11:56:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.350953+0000", 2024-09-21T11:56:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075869499999999999 2024-09-21T11:56:37.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.548992+0000", 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.198038465 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549053+0000", 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1884999999999994e-05 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549092+0000", 2024-09-21T11:56:37.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8865999999999997e-05 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67203 10.6 10:6478c686:::benchmark_data_smithi043_150678_object67202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.351169+0000", 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.75046996699999, 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19805109200000001, 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.135 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67203 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351169+0000", 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351169+0000", 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351170+0000", 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5770000000000001e-06 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351238+0000", 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7508999999999993e-05 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351249+0000", 2024-09-21T11:56:37.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1793e-05 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351257+0000", 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3409999999999997e-06 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351300+0000", 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3575000000000002e-05 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351396+0000", 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5711000000000006e-05 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351484+0000", 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8200000000000003e-05 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.353164+0000", 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001679435 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.353614+0000", 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044980699999999998 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549142+0000", 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19552818499999999 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549185+0000", 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3698000000000001e-05 2024-09-21T11:56:37.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549220+0000", 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4261000000000003e-05 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67206 10.6 10:650072e7:::benchmark_data_smithi043_150678_object67205:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.356129+0000", 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.74550949499999, 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19332545400000001, 2024-09-21T11:56:37.141 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67206 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356129+0000", 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356129+0000", 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356130+0000", 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2440000000000001e-06 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356159+0000", 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8768e-05 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356170+0000", 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.111e-05 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356176+0000", 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.305e-06 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356217+0000", 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1125999999999999e-05 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356308+0000", 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1793999999999996e-05 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356392+0000", 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3438999999999999e-05 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.357662+0000", 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012698469999999999 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.358436+0000", 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00077422400000000001 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549312+0000", 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19087628200000001 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549413+0000", 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101264 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549455+0000", 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1050999999999997e-05 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67834 10.6 10:6b4bd2cf:::benchmark_data_smithi043_150678_object67833:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.042578+0000", 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.059060434, 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28913464900000002, 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.147 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67834 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042578+0000", 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042578+0000", 2024-09-21T11:56:37.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042580+0000", 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.443e-06 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042613+0000", 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3355000000000002e-05 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042623+0000", 2024-09-21T11:56:37.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8290000000000003e-06 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042630+0000", 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7519999999999999e-06 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042846+0000", 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021617899999999999 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042970+0000", 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123807 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043305+0000", 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033573499999999998 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331521+0000", 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28821593200000001 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331551+0000", 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9621999999999999e-05 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331587+0000", 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6513000000000001e-05 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331664+0000", 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6893999999999999e-05 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331713+0000", 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8588000000000003e-05 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67836 10.6 10:654cbc8c:::benchmark_data_smithi043_150678_object67835:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.043478+0000", 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.058160945, 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28847367400000001, 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.153 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67836 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043478+0000", 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043478+0000", 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043479+0000", 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0780000000000001e-06 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043547+0000", 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7952000000000003e-05 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043557+0000", 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0547000000000001e-05 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043564+0000", 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0550000000000002e-06 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046156+0000", 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002592123 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046282+0000", 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125747 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046402+0000", 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012027999999999999 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331787+0000", 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28538471399999998 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331797+0000", 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6050000000000004e-06 2024-09-21T11:56:37.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331831+0000", 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4458000000000001e-05 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331911+0000", 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9338999999999994e-05 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331951+0000", 2024-09-21T11:56:37.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0775999999999997e-05 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:67834 10.6 e331/313 ondisk, result = 0)", 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046066+0000", 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.05557237900001, 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28548575300000001, 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046066+0000", 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046066+0000", 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046067+0000", 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0210000000000001e-06 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046072+0000", 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046082+0000", 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3449999999999995e-06 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046089+0000", 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0740000000000004e-06 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331506+0000", 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28541750199999999 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331515+0000", 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8950000000000004e-06 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331552+0000", 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6764000000000002e-05 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.162 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:67834 10.6 e331/313 ondisk, result = 0)", 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046496+0000", 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.055142166, 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.285227234, 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046496+0000", 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046496+0000", 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046497+0000", 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0299999999999999e-07 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046501+0000", 2024-09-21T11:56:37.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2359999999999998e-06 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046509+0000", 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9719999999999992e-06 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046516+0000", 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3679999999999998e-06 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331575+0000", 2024-09-21T11:56:37.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28505966199999999 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331582+0000", 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2790000000000004e-06 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331724+0000", 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014211399999999999 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67837 10.a e331/313 10:5163063c:::benchmark_data_smithi043_150678_object67836:head v 331'4187, mlcod=331'4185)", 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046734+0000", 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.054904088, 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28267978599999999, 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046734+0000", 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046734+0000", 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046735+0000", 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7300000000000005e-07 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046777+0000", 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1236999999999998e-05 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046786+0000", 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6069999999999997e-06 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046793+0000", 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3990000000000001e-06 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046836+0000", 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2938999999999997e-05 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046853+0000", 2024-09-21T11:56:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7424000000000001e-05 2024-09-21T11:56:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.170 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329365+0000", 2024-09-21T11:56:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28251246899999999 2024-09-21T11:56:37.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329414+0000", 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8838000000000002e-05 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67842 10.7 e331/313 10:e64204f5:::benchmark_data_smithi043_150678_object67841:head v 331'4261, mlcod=331'4259)", 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.05418984799999, 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28956349100000001, 2024-09-21T11:56:37.172 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047488+0000", 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8732000000000001e-05 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047497+0000", 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8599999999999999e-06 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047503+0000", 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7289999999999997e-06 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047550+0000", 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7571000000000002e-05 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329323+0000", 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28177280300000002 2024-09-21T11:56:37.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.336914+0000", 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0075908069999999998 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337012+0000", 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8308999999999997e-05 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.176 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67844 10.3 e331/313 10:c750a974:::benchmark_data_smithi043_150678_object67843:head v 331'4259, mlcod=331'4257)", 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.047834+0000", 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.05380453399999, 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29078393699999999, 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047834+0000", 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047834+0000", 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047835+0000", 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9999999999999996e-07 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047887+0000", 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1690999999999997e-05 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329326+0000", 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28143986999999998 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329343+0000", 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6688999999999999e-05 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329400+0000", 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6820999999999997e-05 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329424+0000", 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3693000000000002e-05 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.338574+0000", 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091507940000000003 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.338618+0000", 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3578999999999997e-05 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67847 10.6 10:65456b12:::benchmark_data_smithi043_150678_object67846:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.052219+0000", 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.04941960400001, 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28538154100000002, 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.181 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67847 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052219+0000", 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052219+0000", 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052221+0000", 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.108e-06 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052286+0000", 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4874000000000003e-05 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329342+0000", 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27705603200000001 2024-09-21T11:56:37.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329356+0000", 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4419e-05 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331988+0000", 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026312319999999998 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.332118+0000", 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012998 2024-09-21T11:56:37.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.332241+0000", 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012306900000000001 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337426+0000", 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005185449 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337464+0000", 2024-09-21T11:56:37.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7957999999999998e-05 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337515+0000", 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1017000000000003e-05 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337562+0000", 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7200999999999998e-05 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337601+0000", 2024-09-21T11:56:37.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8201999999999998e-05 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:67836 10.6 e331/313 ondisk, result = 0)", 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.053932+0000", 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 188.04770667700001, 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27786533400000002, 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053932+0000", 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053932+0000", 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053933+0000", 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.381e-06 2024-09-21T11:56:37.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053940+0000", 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1810000000000004e-06 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329343+0000", 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27540220300000001 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329355+0000", 2024-09-21T11:56:37.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2237e-05 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331776+0000", 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024212000000000001 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331783+0000", 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5429999999999999e-06 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331797+0000", 2024-09-21T11:56:37.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4589e-05 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72213 10.6 10:6e0a855e:::benchmark_data_smithi043_150678_object72212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.969718+0000", 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 184.131920347, 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23677020700000001, 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.191 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72213 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969718+0000", 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969718+0000", 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969720+0000", 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.522e-06 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969793+0000", 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3422000000000003e-05 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969810+0000", 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7014e-05 2024-09-21T11:56:37.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969819+0000", 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.653e-06 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969874+0000", 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5568000000000001e-05 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970035+0000", 2024-09-21T11:56:37.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016056600000000001 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970175+0000", 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014015300000000001 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.972425+0000", 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022500749999999998 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.973522+0000", 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010972670000000001 2024-09-21T11:56:37.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206358+0000", 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23283527300000001 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206438+0000", 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9804999999999999e-05 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206488+0000", 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0889e-05 2024-09-21T11:56:37.196 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:85555 10.9 10:9e997645:::benchmark_data_smithi043_150678_object85554:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.837759+0000", 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 172.26387930000001, 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24446227800000001, 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.197 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 85555 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837759+0000", 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837759+0000", 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837761+0000", 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1500000000000002e-06 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837839+0000", 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7088999999999993e-05 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837852+0000", 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3409e-05 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837859+0000", 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.635e-06 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837907+0000", 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8232000000000002e-05 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.838017+0000", 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011044000000000001 2024-09-21T11:56:37.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.838100+0000", 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3128999999999996e-05 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.840553+0000", 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024521650000000001 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.840749+0000", 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019673600000000001 2024-09-21T11:56:37.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082065+0000", 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.241316108 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082158+0000", 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2916000000000003e-05 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082222+0000", 2024-09-21T11:56:37.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3268999999999993e-05 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:85563 10.9 10:97d65cde:::benchmark_data_smithi043_150678_object85562:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.847840+0000", 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 172.253798222, 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.234536209, 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.203 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 85563 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847840+0000", 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847840+0000", 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847842+0000", 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.482e-06 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847894+0000", 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2577999999999997e-05 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847905+0000", 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0764000000000001e-05 2024-09-21T11:56:37.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847911+0000", 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7960000000000001e-06 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847964+0000", 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3365000000000002e-05 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848068+0000", 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103872 2024-09-21T11:56:37.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848154+0000", 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6026000000000006e-05 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.851858+0000", 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0037039949999999999 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.851882+0000", 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3261999999999999e-05 2024-09-21T11:56:37.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082261+0000", 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23037903200000001 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082340+0000", 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9379e-05 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082377+0000", 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6658e-05 2024-09-21T11:56:37.208 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:85564 10.9 10:98f1ca55:::benchmark_data_smithi043_150678_object85563:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.848489+0000", 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 172.253149705, 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24391052699999999, 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:37.209 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 85564 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848489+0000", 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848489+0000", 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848490+0000", 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8100000000000002e-07 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848561+0000", 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1595999999999997e-05 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848569+0000", 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6760000000000007e-06 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848573+0000", 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4850000000000003e-06 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.850273+0000", 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001699952 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.850369+0000", 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5712999999999997e-05 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.850456+0000", 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7298000000000003e-05 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.851904+0000", 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001447439 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.852025+0000", 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000121432 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:37.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.092269+0000", 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24024353500000001 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.092357+0000", 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8367000000000003e-05 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.092399+0000", 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2252999999999997e-05 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.214 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:37.215 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:37.215 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:56:37.235 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:37.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:37.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:37.337 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:37.358 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:37.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:37.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:37.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:37.483 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:37.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:37.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:37.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:37.609 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:37.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:37.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:37.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:37.732 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:37.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:37.733 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[29710]: pgmap v805: 148 pgs: 6 creating+peering, 10 unknown, 132 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail 2024-09-21T11:56:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[29710]: 6.18 scrub starts 2024-09-21T11:56:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[29710]: 6.18 scrub ok 2024-09-21T11:56:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[29710]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:56:37.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[29710]: osdmap e393: 8 total, 8 up, 8 in 2024-09-21T11:56:37.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[36454]: pgmap v805: 148 pgs: 6 creating+peering, 10 unknown, 132 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail 2024-09-21T11:56:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[36454]: 6.18 scrub starts 2024-09-21T11:56:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[36454]: 6.18 scrub ok 2024-09-21T11:56:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[36454]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:56:37.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:37 smithi043 ceph-mon[36454]: osdmap e393: 8 total, 8 up, 8 in 2024-09-21T11:56:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:37 smithi175 ceph-mon[32613]: pgmap v805: 148 pgs: 6 creating+peering, 10 unknown, 132 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail 2024-09-21T11:56:37.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:37 smithi175 ceph-mon[32613]: 6.18 scrub starts 2024-09-21T11:56:37.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:37 smithi175 ceph-mon[32613]: 6.18 scrub ok 2024-09-21T11:56:37.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:37 smithi175 ceph-mon[32613]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-21T11:56:37.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:37 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:37.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:37 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:56:37.764 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:37 smithi175 ceph-mon[32613]: osdmap e393: 8 total, 8 up, 8 in 2024-09-21T11:56:37.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:37.858 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:37.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:37.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:37.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:38.004 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:38.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:38.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:38.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:38.128 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:38.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:38.129 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:38.212 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:38.230 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:38.254 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:38.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:38.255 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:38.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:56:38.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:38.379 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:38.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:38.380 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:38.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:38.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:38.504 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:38.504 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:38.537 INFO:teuthology.orchestra.run.smithi043.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-21T11:56:38.548 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:38.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:38.626 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:38.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:38.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:38.724 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:56:38.724 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:56:38.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:38 smithi043 ceph-mon[29710]: osdmap e394: 8 total, 8 up, 8 in 2024-09-21T11:56:38.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:38 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3961353146' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:38.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:38 smithi043 ceph-mon[36454]: osdmap e394: 8 total, 8 up, 8 in 2024-09-21T11:56:38.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:38 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3961353146' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:38.741 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:56:38.742 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:56:38.749 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:38.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:38.750 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:38.758 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:56:38.758 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:56:38.774 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:56:38.774 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:56:38.791 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:56:38.792 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:56:38.808 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:56:38.808 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:56:38.825 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:56:38.825 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:56:38.842 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:56:38.842 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:56:38.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:38.874 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:38.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:38.875 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:38.875 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:38 smithi175 ceph-mon[32613]: osdmap e394: 8 total, 8 up, 8 in 2024-09-21T11:56:38.875 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:38 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3961353146' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-21T11:56:38.976 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:38.998 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:38.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:38.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:39.082 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:39.083 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:39.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:39.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:39.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:39.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:56:39.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:39.281 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:39.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:39.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:39.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:39.405 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:39.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:39.406 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:39.506 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:39.528 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:39.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:39.529 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:39.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:39.652 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:39.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:39.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:39 smithi043 ceph-mon[29710]: pgmap v808: 148 pgs: 10 creating+peering, 138 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3961353146' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:39 smithi043 ceph-mon[29710]: osdmap e395: 8 total, 8 up, 8 in 2024-09-21T11:56:39.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2914240631' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:39 smithi043 ceph-mon[36454]: pgmap v808: 148 pgs: 10 creating+peering, 138 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3961353146' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:39 smithi043 ceph-mon[36454]: osdmap e395: 8 total, 8 up, 8 in 2024-09-21T11:56:39.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2914240631' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:39.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:39.776 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:39.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:39.777 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:39.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:39.900 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:39.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:39.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:39.968 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:39 smithi175 ceph-mon[32613]: pgmap v808: 148 pgs: 10 creating+peering, 138 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T11:56:39.968 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3961353146' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-21T11:56:39.968 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:39 smithi175 ceph-mon[32613]: osdmap e395: 8 total, 8 up, 8 in 2024-09-21T11:56:39.968 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2914240631' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:39.996 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:56:40.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:40.024 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:40.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:40.025 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:40.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:40.148 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:40.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:40.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:40.164 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:40.239 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:40.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:40.272 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:40.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:40.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:40.374 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:40.396 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:40.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:40.397 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:40.488 DEBUG:teuthology.orchestra.run.smithi043:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-21T11:56:40.489 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-21T11:56:40.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:56:40.491 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-21T11:56:40.491 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-21T11:56:40.497 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:40.554 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:40.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:40.555 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:40.647 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:40 smithi175 ceph-mon[32613]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:56:40.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:40.680 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:hints = 1 2024-09-21T11:56:40.680 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-21T11:56:40.680 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Object prefix: benchmark_data_smithi043_178089 2024-09-21T11:56:40.680 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:56:40.681 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 0 0 0 0 0 0 - 0 2024-09-21T11:56:40.682 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:40.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:40.684 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:40.780 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:40.780 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:40.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:40.829 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:40.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:40.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:40.884 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:40.930 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:40.959 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:40.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:40.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:40 smithi043 ceph-mon[36454]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:56:40.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:40 smithi043 ceph-mon[29710]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-21T11:56:41.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:41.102 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:41.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:41.103 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:41.204 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:41.231 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:41.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:41.232 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:41.333 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:41.363 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:41.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:41.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:41.465 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:41.483 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:56:41.493 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:41.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:41.495 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:41.595 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:41.597 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:56:41.627 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:41.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:41.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:41.681 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 1 16 2998 2982 186.365 186.375 0.0130085 0.00533812 2024-09-21T11:56:41.729 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:41 smithi175 ceph-mon[32613]: pgmap v810: 148 pgs: 10 creating+peering, 138 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:56:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:41 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2259444778' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:41 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:41.764 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:41.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:41.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:41.859 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:41.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:41.905 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:41 smithi043 ceph-mon[29710]: pgmap v810: 148 pgs: 10 creating+peering, 138 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:56:41.906 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:41 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2259444778' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:41.906 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:41 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:41.906 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:41 smithi043 ceph-mon[36454]: pgmap v810: 148 pgs: 10 creating+peering, 138 active+clean; 583 KiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T11:56:41.906 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:41 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2259444778' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:41.906 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:41 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:56:41.909 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:56:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:41.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:41.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:56:42.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:42.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:56:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:56:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:42.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:42.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:42.130 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:56:42.154 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:42.187 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:42.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:42.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:42.289 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:42.320 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:42.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:42.321 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:42.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:42.449 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:42.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:42.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:42.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:42.590 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:42.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:42.591 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:42.661 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:42.662 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:42.681 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 2 16 6070 6054 189.175 192 0.00708503 0.00527482 2024-09-21T11:56:42.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:42.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:42.732 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:42.732 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:42 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:42 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:42.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:42 smithi043 ceph-mon[36454]: osdmap e396: 8 total, 8 up, 8 in 2024-09-21T11:56:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:42 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:42 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:42.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:42 smithi043 ceph-mon[29710]: osdmap e396: 8 total, 8 up, 8 in 2024-09-21T11:56:42.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:42 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:42 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:56:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:42 smithi175 ceph-mon[32613]: osdmap e396: 8 total, 8 up, 8 in 2024-09-21T11:56:42.833 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:42.863 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:42.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:42.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:42.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:42.997 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:42.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:42.998 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:43.099 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:43.140 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:43.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:43.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:43.242 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:43.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:43.265 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:43.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:43.366 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:43.387 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:43.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:43.388 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:43.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:43.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:43.524 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:43.524 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:43.625 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:43.656 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:43.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:43.657 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:43.681 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 3 16 8751 8735 181.967 167.562 0.00689897 0.0054851 2024-09-21T11:56:43.757 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:43.778 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:43.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:43.779 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:43.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:43.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:43.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:43.947 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:43 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:43.947 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:43 smithi175 ceph-mon[32613]: from='client.26345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:56:43.947 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:43 smithi175 ceph-mon[32613]: pgmap v812: 148 pgs: 148 active+clean; 104 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 910 B/s rd, 18 MiB/s wr, 295 op/s 2024-09-21T11:56:43.975 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:56:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:43 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:43 smithi043 ceph-mon[36454]: from='client.26345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:56:43.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:43 smithi043 ceph-mon[36454]: pgmap v812: 148 pgs: 148 active+clean; 104 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 910 B/s rd, 18 MiB/s wr, 295 op/s 2024-09-21T11:56:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:43 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:43 smithi043 ceph-mon[29710]: from='client.26345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:56:43.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:43 smithi043 ceph-mon[29710]: pgmap v812: 148 pgs: 148 active+clean; 104 MiB data, 4.7 GiB used, 711 GiB / 715 GiB avail; 910 B/s rd, 18 MiB/s wr, 295 op/s 2024-09-21T11:56:44.022 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:44.060 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:44.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:44.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:44.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:44.183 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:44.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:44.184 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:44.285 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:44.321 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:44.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:44.322 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:44.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:44.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:44.443 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:44.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:44.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:44.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:56:44.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:44.578 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:44.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:44.579 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:44.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:44.681 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 4 16 10920 10904 170.363 135.562 0.006258 0.00585704 2024-09-21T11:56:44.726 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:44.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:44.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:44.828 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:44.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:44.870 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:44.870 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:44.912 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:44.913 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:44.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:45.012 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:45.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:45.013 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:45.082 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":811,"stamp":"2024-09-21T11:56:41.595755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":68033051,"num_objects":1228,"num_object_clones":0,"num_object_copies":3684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1228,"num_whiteouts":0,"num_read":2239,"num_read_kb":2096,"num_write":1864,"num_write_kb":67529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":1188763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1877,"ondisk_log_size":1877,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":443,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":4844364,"kb_used_data":103628,"kb_used_omap":12,"kb_used_meta":4740467,"kb_avail":745116852,"statfs":{"total":767960285184,"available":762999656448,"internally_reserved":0,"allocated":106115072,"data_stored":103037461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4854238770},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":67436544,"num_objects":1029,"num_object_clones":0,"num_object_copies":3087,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1029,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":1029,"num_write_kb":65856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.477376"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":74,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553795+0000","last_change":"2024-09-21T11:55:28.565270+0000","last_active":"2024-09-21T11:56:38.553795+0000","last_peered":"2024-09-21T11:56:38.553795+0000","last_clean":"2024-09-21T11:56:38.553795+0000","last_became_active":"2024-09-21T11:55:28.564399+0000","last_became_peered":"2024-09-21T11:55:28.564399+0000","last_unstale":"2024-09-21T11:56:38.553795+0000","last_undegraded":"2024-09-21T11:56:38.553795+0000","last_fullsized":"2024-09-21T11:56:38.553795+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.303555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.303555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.026551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":754,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.006529+0000","last_change":"2024-09-21T11:54:27.283019+0000","last_active":"2024-09-21T11:56:39.006529+0000","last_peered":"2024-09-21T11:56:39.006529+0000","last_clean":"2024-09-21T11:56:39.006529+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:56:39.006529+0000","last_undegraded":"2024-09-21T11:56:39.006529+0000","last_fullsized":"2024-09-21T11:56:39.006529+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:27.282826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:27.282826+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:26.053550+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":439,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181238+0000","last_change":"2024-09-21T11:56:08.429983+0000","last_active":"2024-09-21T11:56:39.181238+0000","last_peered":"2024-09-21T11:56:39.181238+0000","last_clean":"2024-09-21T11:56:39.181238+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:56:39.181238+0000","last_undegraded":"2024-09-21T11:56:39.181238+0000","last_fullsized":"2024-09-21T11:56:39.181238+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:08.429912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.429912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:17.067807+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":526,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.552631+0000","last_change":"2024-09-21T11:51:43.528594+0000","last_active":"2024-09-21T11:56:38.552631+0000","last_peered":"2024-09-21T11:56:38.552631+0000","last_clean":"2024-09-21T11:56:38.552631+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:56:38.552631+0000","last_undegraded":"2024-09-21T11:56:38.552631+0000","last_fullsized":"2024-09-21T11:56:38.552631+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:17.102412+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":726,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540199+0000","last_change":"2024-09-21T11:54:53.491138+0000","last_active":"2024-09-21T11:56:38.540199+0000","last_peered":"2024-09-21T11:56:38.540199+0000","last_clean":"2024-09-21T11:56:38.540199+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:56:38.540199+0000","last_undegraded":"2024-09-21T11:56:38.540199+0000","last_fullsized":"2024-09-21T11:56:38.540199+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.490941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.490941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":716,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540175+0000","last_change":"2024-09-21T11:55:25.429625+0000","last_active":"2024-09-21T11:56:38.540175+0000","last_peered":"2024-09-21T11:56:38.540175+0000","last_clean":"2024-09-21T11:56:38.540175+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:56:38.540175+0000","last_undegraded":"2024-09-21T11:56:38.540175+0000","last_fullsized":"2024-09-21T11:56:38.540175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:25.429559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:25.429559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:49.460771+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":769,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.541033+0000","last_change":"2024-09-21T11:56:31.572626+0000","last_active":"2024-09-21T11:56:38.541033+0000","last_peered":"2024-09-21T11:56:38.541033+0000","last_clean":"2024-09-21T11:56:38.541033+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:56:38.541033+0000","last_undegraded":"2024-09-21T11:56:38.541033+0000","last_fullsized":"2024-09-21T11:56:38.541033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:31.572525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:31.572525+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:41.538532+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":562,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.006672+0000","last_change":"2024-09-21T11:55:28.561709+0000","last_active":"2024-09-21T11:56:39.006672+0000","last_peered":"2024-09-21T11:56:39.006672+0000","last_clean":"2024-09-21T11:56:39.006672+0000","last_became_active":"2024-09-21T11:55:28.561338+0000","last_became_peered":"2024-09-21T11:55:28.561338+0000","last_unstale":"2024-09-21T11:56:39.006672+0000","last_undegraded":"2024-09-21T11:56:39.006672+0000","last_fullsized":"2024-09-21T11:56:39.006672+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:28.252642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:28.252642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:12.260547+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":713,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553478+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:56:38.553478+0000","last_peered":"2024-09-21T11:56:38.553478+0000","last_clean":"2024-09-21T11:56:38.553478+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:56:38.553478+0000","last_undegraded":"2024-09-21T11:56:38.553478+0000","last_fullsized":"2024-09-21T11:56:38.553478+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:22.116605+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":736,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.539461+0000","last_change":"2024-09-21T11:55:28.584595+0000","last_active":"2024-09-21T11:56:38.539461+0000","last_peered":"2024-09-21T11:56:38.539461+0000","last_clean":"2024-09-21T11:56:38.539461+0000","last_became_active":"2024-09-21T11:55:28.575920+0000","last_became_peered":"2024-09-21T11:55:28.575920+0000","last_unstale":"2024-09-21T11:56:38.539461+0000","last_undegraded":"2024-09-21T11:56:38.539461+0000","last_fullsized":"2024-09-21T11:56:38.539461+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:10.424192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:10.424192+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:38.749738+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":713,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.552911+0000","last_change":"2024-09-21T11:55:05.096606+0000","last_active":"2024-09-21T11:56:38.552911+0000","last_peered":"2024-09-21T11:56:38.552911+0000","last_clean":"2024-09-21T11:56:38.552911+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:56:38.552911+0000","last_undegraded":"2024-09-21T11:56:38.552911+0000","last_fullsized":"2024-09-21T11:56:38.552911+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:05.096542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:05.096542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":390,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181100+0000","last_change":"2024-09-21T11:56:36.422776+0000","last_active":"2024-09-21T11:56:39.181100+0000","last_peered":"2024-09-21T11:56:39.181100+0000","last_clean":"2024-09-21T11:56:39.181100+0000","last_became_active":"2024-09-21T11:55:28.566921+0000","last_became_peered":"2024-09-21T11:55:28.566921+0000","last_unstale":"2024-09-21T11:56:39.181100+0000","last_undegraded":"2024-09-21T11:56:39.181100+0000","last_fullsized":"2024-09-21T11:56:39.181100+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:36.422726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:36.422726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.958604+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":759,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553091+0000","last_change":"2024-09-21T11:56:02.994041+0000","last_active":"2024-09-21T11:56:38.553091+0000","last_peered":"2024-09-21T11:56:38.553091+0000","last_clean":"2024-09-21T11:56:38.553091+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:56:38.553091+0000","last_undegraded":"2024-09-21T11:56:38.553091+0000","last_fullsized":"2024-09-21T11:56:38.553091+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.993966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.993966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:09.442277+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":738,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540507+0000","last_change":"2024-09-21T11:56:16.673919+0000","last_active":"2024-09-21T11:56:38.540507+0000","last_peered":"2024-09-21T11:56:38.540507+0000","last_clean":"2024-09-21T11:56:38.540507+0000","last_became_active":"2024-09-21T11:56:16.667750+0000","last_became_peered":"2024-09-21T11:56:16.667750+0000","last_unstale":"2024-09-21T11:56:38.540507+0000","last_undegraded":"2024-09-21T11:56:38.540507+0000","last_fullsized":"2024-09-21T11:56:38.540507+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'7","last_scrub_stamp":"2024-09-21T11:55:04.399020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:04.399020+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":719,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002213+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:56:39.002213+0000","last_peered":"2024-09-21T11:56:39.002213+0000","last_clean":"2024-09-21T11:56:39.002213+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:56:39.002213+0000","last_undegraded":"2024-09-21T11:56:39.002213+0000","last_fullsized":"2024-09-21T11:56:39.002213+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:12.606428+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":723,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.552659+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:56:38.552659+0000","last_peered":"2024-09-21T11:56:38.552659+0000","last_clean":"2024-09-21T11:56:38.552659+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:56:38.552659+0000","last_undegraded":"2024-09-21T11:56:38.552659+0000","last_fullsized":"2024-09-21T11:56:38.552659+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:19.025012+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":233,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.007175+0000","last_change":"2024-09-21T11:56:34.073296+0000","last_active":"2024-09-21T11:56:39.007175+0000","last_peered":"2024-09-21T11:56:39.007175+0000","last_clean":"2024-09-21T11:56:39.007175+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:56:39.007175+0000","last_undegraded":"2024-09-21T11:56:39.007175+0000","last_fullsized":"2024-09-21T11:56:39.007175+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:34.073249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:34.073249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:46.860083+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":74,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002548+0000","last_change":"2024-09-21T11:55:28.562731+0000","last_active":"2024-09-21T11:56:39.002548+0000","last_peered":"2024-09-21T11:56:39.002548+0000","last_clean":"2024-09-21T11:56:39.002548+0000","last_became_active":"2024-09-21T11:55:28.562057+0000","last_became_peered":"2024-09-21T11:55:28.562057+0000","last_unstale":"2024-09-21T11:56:39.002548+0000","last_undegraded":"2024-09-21T11:56:39.002548+0000","last_fullsized":"2024-09-21T11:56:39.002548+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:27.176320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:27.176320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:40.048471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":865,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.502796+0000","last_change":"2024-09-21T11:56:01.647675+0000","last_active":"2024-09-21T11:56:36.502796+0000","last_peered":"2024-09-21T11:56:36.502796+0000","last_clean":"2024-09-21T11:56:36.502796+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T11:56:36.502796+0000","last_undegraded":"2024-09-21T11:56:36.502796+0000","last_fullsized":"2024-09-21T11:56:36.502796+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:01.647580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:01.647580+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:23.282522+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":438,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181448+0000","last_change":"2024-09-21T11:56:16.669857+0000","last_active":"2024-09-21T11:56:39.181448+0000","last_peered":"2024-09-21T11:56:39.181448+0000","last_clean":"2024-09-21T11:56:39.181448+0000","last_became_active":"2024-09-21T11:56:16.669551+0000","last_became_peered":"2024-09-21T11:56:16.669551+0000","last_unstale":"2024-09-21T11:56:39.181448+0000","last_undegraded":"2024-09-21T11:56:39.181448+0000","last_fullsized":"2024-09-21T11:56:39.181448+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:56.501356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:56.501356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:14.605337+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":719,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.539788+0000","last_change":"2024-09-21T11:55:07.393341+0000","last_active":"2024-09-21T11:56:38.539788+0000","last_peered":"2024-09-21T11:56:38.539788+0000","last_clean":"2024-09-21T11:56:38.539788+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:56:38.539788+0000","last_undegraded":"2024-09-21T11:56:38.539788+0000","last_fullsized":"2024-09-21T11:56:38.539788+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:07.393249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:07.393249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":546,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181537+0000","last_change":"2024-09-21T11:55:57.442188+0000","last_active":"2024-09-21T11:56:39.181537+0000","last_peered":"2024-09-21T11:56:39.181537+0000","last_clean":"2024-09-21T11:56:39.181537+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:56:39.181537+0000","last_undegraded":"2024-09-21T11:56:39.181537+0000","last_fullsized":"2024-09-21T11:56:39.181537+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:57.441965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.441965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:22.964661+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":788,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.539750+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:56:38.539750+0000","last_peered":"2024-09-21T11:56:38.539750+0000","last_clean":"2024-09-21T11:56:38.539750+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:56:38.539750+0000","last_undegraded":"2024-09-21T11:56:38.539750+0000","last_fullsized":"2024-09-21T11:56:38.539750+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":239,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540578+0000","last_change":"2024-09-21T11:56:33.487190+0000","last_active":"2024-09-21T11:56:38.540578+0000","last_peered":"2024-09-21T11:56:38.540578+0000","last_clean":"2024-09-21T11:56:38.540578+0000","last_became_active":"2024-09-21T11:56:16.666765+0000","last_became_peered":"2024-09-21T11:56:16.666765+0000","last_unstale":"2024-09-21T11:56:38.540578+0000","last_undegraded":"2024-09-21T11:56:38.540578+0000","last_fullsized":"2024-09-21T11:56:38.540578+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.487132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.487132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:03.363944+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":224,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.500402+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:56:36.500402+0000","last_peered":"2024-09-21T11:56:36.500402+0000","last_clean":"2024-09-21T11:56:36.500402+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:56:36.500402+0000","last_undegraded":"2024-09-21T11:56:36.500402+0000","last_fullsized":"2024-09-21T11:56:36.500402+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:23.147359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":446,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181358+0000","last_change":"2024-09-21T11:56:04.425734+0000","last_active":"2024-09-21T11:56:39.181358+0000","last_peered":"2024-09-21T11:56:39.181358+0000","last_clean":"2024-09-21T11:56:39.181358+0000","last_became_active":"2024-09-21T11:55:28.568598+0000","last_became_peered":"2024-09-21T11:55:28.568598+0000","last_unstale":"2024-09-21T11:56:39.181358+0000","last_undegraded":"2024-09-21T11:56:39.181358+0000","last_fullsized":"2024-09-21T11:56:39.181358+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:04.425651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:04.425651+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:05.921552+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":753,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.500779+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:56:36.500779+0000","last_peered":"2024-09-21T11:56:36.500779+0000","last_clean":"2024-09-21T11:56:36.500779+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:56:36.500779+0000","last_undegraded":"2024-09-21T11:56:36.500779+0000","last_fullsized":"2024-09-21T11:56:36.500779+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:13.139725+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"383'9","reported_seq":963,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:40.571782+0000","last_change":"2024-09-21T11:56:16.677508+0000","last_active":"2024-09-21T11:56:40.571782+0000","last_peered":"2024-09-21T11:56:40.571782+0000","last_clean":"2024-09-21T11:56:40.571782+0000","last_became_active":"2024-09-21T11:56:16.673446+0000","last_became_peered":"2024-09-21T11:56:16.673446+0000","last_unstale":"2024-09-21T11:56:40.571782+0000","last_undegraded":"2024-09-21T11:56:40.571782+0000","last_fullsized":"2024-09-21T11:56:40.571782+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:57.232943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.232943+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:20.776566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"395'135","reported_seq":152,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:41.539769+0000","last_change":"2024-09-21T11:56:36.507880+0000","last_active":"2024-09-21T11:56:41.539769+0000","last_peered":"2024-09-21T11:56:41.539769+0000","last_clean":"2024-09-21T11:56:41.539769+0000","last_became_active":"2024-09-21T11:56:36.507606+0000","last_became_peered":"2024-09-21T11:56:36.507606+0000","last_unstale":"2024-09-21T11:56:41.539769+0000","last_undegraded":"2024-09-21T11:56:41.539769+0000","last_fullsized":"2024-09-21T11:56:41.539769+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:44.536721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8847360,"num_objects":135,"num_object_clones":0,"num_object_copies":405,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":466,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181930+0000","last_change":"2024-09-21T11:55:52.562992+0000","last_active":"2024-09-21T11:56:39.181930+0000","last_peered":"2024-09-21T11:56:39.181930+0000","last_clean":"2024-09-21T11:56:39.181930+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:56:39.181930+0000","last_undegraded":"2024-09-21T11:56:39.181930+0000","last_fullsized":"2024-09-21T11:56:39.181930+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:55:52.562928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.562928+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:22.482120+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":745,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540178+0000","last_change":"2024-09-21T11:53:30.353707+0000","last_active":"2024-09-21T11:56:38.540178+0000","last_peered":"2024-09-21T11:56:38.540178+0000","last_clean":"2024-09-21T11:56:38.540178+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:56:38.540178+0000","last_undegraded":"2024-09-21T11:56:38.540178+0000","last_fullsized":"2024-09-21T11:56:38.540178+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:30.353632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:30.353632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:13.479595+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"366'10","reported_seq":323,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553447+0000","last_change":"2024-09-21T11:55:28.951330+0000","last_active":"2024-09-21T11:56:38.553447+0000","last_peered":"2024-09-21T11:56:38.553447+0000","last_clean":"2024-09-21T11:56:38.553447+0000","last_became_active":"2024-09-21T11:55:28.575209+0000","last_became_peered":"2024-09-21T11:55:28.575209+0000","last_unstale":"2024-09-21T11:56:38.553447+0000","last_undegraded":"2024-09-21T11:56:38.553447+0000","last_fullsized":"2024-09-21T11:56:38.553447+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"294'9","last_scrub_stamp":"2024-09-21T11:55:28.951276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:28.951276+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:43.105898+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":755,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553196+0000","last_change":"2024-09-21T11:55:33.000584+0000","last_active":"2024-09-21T11:56:38.553196+0000","last_peered":"2024-09-21T11:56:38.553196+0000","last_clean":"2024-09-21T11:56:38.553196+0000","last_became_active":"2024-09-21T11:55:28.564378+0000","last_became_peered":"2024-09-21T11:55:28.564378+0000","last_unstale":"2024-09-21T11:56:38.553196+0000","last_undegraded":"2024-09-21T11:56:38.553196+0000","last_fullsized":"2024-09-21T11:56:38.553196+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:33.000521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:33.000521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:46.499959+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"395'120","reported_seq":137,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:41.247972+0000","last_change":"2024-09-21T11:56:36.511425+0000","last_active":"2024-09-21T11:56:41.247972+0000","last_peered":"2024-09-21T11:56:41.247972+0000","last_clean":"2024-09-21T11:56:41.247972+0000","last_became_active":"2024-09-21T11:56:36.511191+0000","last_became_peered":"2024-09-21T11:56:36.511191+0000","last_unstale":"2024-09-21T11:56:41.247972+0000","last_undegraded":"2024-09-21T11:56:41.247972+0000","last_fullsized":"2024-09-21T11:56:41.247972+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:04.769503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7864320,"num_objects":120,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":7680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":811,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.539964+0000","last_change":"2024-09-21T11:55:28.585148+0000","last_active":"2024-09-21T11:56:38.539964+0000","last_peered":"2024-09-21T11:56:38.539964+0000","last_clean":"2024-09-21T11:56:38.539964+0000","last_became_active":"2024-09-21T11:55:28.563496+0000","last_became_peered":"2024-09-21T11:55:28.563496+0000","last_unstale":"2024-09-21T11:56:38.539964+0000","last_undegraded":"2024-09-21T11:56:38.539964+0000","last_fullsized":"2024-09-21T11:56:38.539964+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:55:05.451051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:05.451051+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":739,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540064+0000","last_change":"2024-09-21T11:54:55.512377+0000","last_active":"2024-09-21T11:56:38.540064+0000","last_peered":"2024-09-21T11:56:38.540064+0000","last_clean":"2024-09-21T11:56:38.540064+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:56:38.540064+0000","last_undegraded":"2024-09-21T11:56:38.540064+0000","last_fullsized":"2024-09-21T11:56:38.540064+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:55.512314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:55.512314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":225,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540586+0000","last_change":"2024-09-21T11:56:06.506517+0000","last_active":"2024-09-21T11:56:38.540586+0000","last_peered":"2024-09-21T11:56:38.540586+0000","last_clean":"2024-09-21T11:56:38.540586+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:56:38.540586+0000","last_undegraded":"2024-09-21T11:56:38.540586+0000","last_fullsized":"2024-09-21T11:56:38.540586+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:06.506449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:06.506449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:31.003771+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":223,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.503857+0000","last_change":"2024-09-21T11:54:53.844954+0000","last_active":"2024-09-21T11:56:36.503857+0000","last_peered":"2024-09-21T11:56:36.503857+0000","last_clean":"2024-09-21T11:56:36.503857+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:56:36.503857+0000","last_undegraded":"2024-09-21T11:56:36.503857+0000","last_fullsized":"2024-09-21T11:56:36.503857+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.844876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.844876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"395'146","reported_seq":163,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:41.534154+0000","last_change":"2024-09-21T11:56:36.508433+0000","last_active":"2024-09-21T11:56:41.534154+0000","last_peered":"2024-09-21T11:56:41.534154+0000","last_clean":"2024-09-21T11:56:41.534154+0000","last_became_active":"2024-09-21T11:56:36.507746+0000","last_became_peered":"2024-09-21T11:56:36.507746+0000","last_unstale":"2024-09-21T11:56:41.534154+0000","last_undegraded":"2024-09-21T11:56:41.534154+0000","last_fullsized":"2024-09-21T11:56:41.534154+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:54.005355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9568256,"num_objects":146,"num_object_clones":0,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":752,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.552767+0000","last_change":"2024-09-21T11:56:16.682561+0000","last_active":"2024-09-21T11:56:38.552767+0000","last_peered":"2024-09-21T11:56:38.552767+0000","last_clean":"2024-09-21T11:56:38.552767+0000","last_became_active":"2024-09-21T11:56:16.670451+0000","last_became_peered":"2024-09-21T11:56:16.670451+0000","last_unstale":"2024-09-21T11:56:38.552767+0000","last_undegraded":"2024-09-21T11:56:38.552767+0000","last_fullsized":"2024-09-21T11:56:38.552767+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:21.067144+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":412,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181540+0000","last_change":"2024-09-21T11:55:28.573493+0000","last_active":"2024-09-21T11:56:39.181540+0000","last_peered":"2024-09-21T11:56:39.181540+0000","last_clean":"2024-09-21T11:56:39.181540+0000","last_became_active":"2024-09-21T11:55:28.567133+0000","last_became_peered":"2024-09-21T11:55:28.567133+0000","last_unstale":"2024-09-21T11:56:39.181540+0000","last_undegraded":"2024-09-21T11:56:39.181540+0000","last_fullsized":"2024-09-21T11:56:39.181540+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:50:28.657261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:28.657261+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:47.626565+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":956,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.539798+0000","last_change":"2024-09-21T11:56:05.567407+0000","last_active":"2024-09-21T11:56:38.539798+0000","last_peered":"2024-09-21T11:56:38.539798+0000","last_clean":"2024-09-21T11:56:38.539798+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:56:38.539798+0000","last_undegraded":"2024-09-21T11:56:38.539798+0000","last_fullsized":"2024-09-21T11:56:38.539798+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:05.567332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.567332+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.902664+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":736,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002384+0000","last_change":"2024-09-21T11:56:34.581966+0000","last_active":"2024-09-21T11:56:39.002384+0000","last_peered":"2024-09-21T11:56:39.002384+0000","last_clean":"2024-09-21T11:56:39.002384+0000","last_became_active":"2024-09-21T11:56:34.581626+0000","last_became_peered":"2024-09-21T11:56:34.581626+0000","last_unstale":"2024-09-21T11:56:39.002384+0000","last_undegraded":"2024-09-21T11:56:39.002384+0000","last_fullsized":"2024-09-21T11:56:39.002384+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.643323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.643323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:21.060965+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.624333+0000","last_change":"2024-09-21T11:56:36.623897+0000","last_active":"2024-09-21T11:56:36.624333+0000","last_peered":"2024-09-21T11:56:36.624333+0000","last_clean":"2024-09-21T11:56:36.624333+0000","last_became_active":"2024-09-21T11:56:36.623581+0000","last_became_peered":"2024-09-21T11:56:36.623581+0000","last_unstale":"2024-09-21T11:56:36.624333+0000","last_undegraded":"2024-09-21T11:56:36.624333+0000","last_fullsized":"2024-09-21T11:56:36.624333+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:35.850166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":478,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181996+0000","last_change":"2024-09-21T11:56:16.679360+0000","last_active":"2024-09-21T11:56:39.181996+0000","last_peered":"2024-09-21T11:56:39.181996+0000","last_clean":"2024-09-21T11:56:39.181996+0000","last_became_active":"2024-09-21T11:56:16.668951+0000","last_became_peered":"2024-09-21T11:56:16.668951+0000","last_unstale":"2024-09-21T11:56:39.181996+0000","last_undegraded":"2024-09-21T11:56:39.181996+0000","last_fullsized":"2024-09-21T11:56:39.181996+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:56:09.421193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.421193+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:32.891630+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":755,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553611+0000","last_change":"2024-09-21T11:54:22.095001+0000","last_active":"2024-09-21T11:56:38.553611+0000","last_peered":"2024-09-21T11:56:38.553611+0000","last_clean":"2024-09-21T11:56:38.553611+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:56:38.553611+0000","last_undegraded":"2024-09-21T11:56:38.553611+0000","last_fullsized":"2024-09-21T11:56:38.553611+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:22.094918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:22.094918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:50.020352+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":929,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.501185+0000","last_change":"2024-09-21T11:55:26.222904+0000","last_active":"2024-09-21T11:56:36.501185+0000","last_peered":"2024-09-21T11:56:36.501185+0000","last_clean":"2024-09-21T11:56:36.501185+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:56:36.501185+0000","last_undegraded":"2024-09-21T11:56:36.501185+0000","last_fullsized":"2024-09-21T11:56:36.501185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:26.222823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:26.222823+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":723,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.497930+0000","last_change":"2024-09-21T11:53:48.256119+0000","last_active":"2024-09-21T11:56:36.497930+0000","last_peered":"2024-09-21T11:56:36.497930+0000","last_clean":"2024-09-21T11:56:36.497930+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:56:36.497930+0000","last_undegraded":"2024-09-21T11:56:36.497930+0000","last_fullsized":"2024-09-21T11:56:36.497930+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:48.255951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:48.255951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:25.212487+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"395'164","reported_seq":181,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:41.535394+0000","last_change":"2024-09-21T11:56:36.508895+0000","last_active":"2024-09-21T11:56:41.535394+0000","last_peered":"2024-09-21T11:56:41.535394+0000","last_clean":"2024-09-21T11:56:41.535394+0000","last_became_active":"2024-09-21T11:56:36.508626+0000","last_became_peered":"2024-09-21T11:56:36.508626+0000","last_unstale":"2024-09-21T11:56:41.535394+0000","last_undegraded":"2024-09-21T11:56:41.535394+0000","last_fullsized":"2024-09-21T11:56:41.535394+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:50.842122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10747904,"num_objects":164,"num_object_clones":0,"num_object_copies":492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":164,"num_write_kb":10496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":789,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.501549+0000","last_change":"2024-09-21T11:56:35.177700+0000","last_active":"2024-09-21T11:56:36.501549+0000","last_peered":"2024-09-21T11:56:36.501549+0000","last_clean":"2024-09-21T11:56:36.501549+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:56:36.501549+0000","last_undegraded":"2024-09-21T11:56:36.501549+0000","last_fullsized":"2024-09-21T11:56:36.501549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:56:35.177639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.177639+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:45.157349+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":424,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.180983+0000","last_change":"2024-09-21T11:55:55.527775+0000","last_active":"2024-09-21T11:56:39.180983+0000","last_peered":"2024-09-21T11:56:39.180983+0000","last_clean":"2024-09-21T11:56:39.180983+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:56:39.180983+0000","last_undegraded":"2024-09-21T11:56:39.180983+0000","last_fullsized":"2024-09-21T11:56:39.180983+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:55.527584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:55.527584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:57.534058+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":714,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.006680+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:56:39.006680+0000","last_peered":"2024-09-21T11:56:39.006680+0000","last_clean":"2024-09-21T11:56:39.006680+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:56:39.006680+0000","last_undegraded":"2024-09-21T11:56:39.006680+0000","last_fullsized":"2024-09-21T11:56:39.006680+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:13.302731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":424,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181061+0000","last_change":"2024-09-21T11:55:51.602665+0000","last_active":"2024-09-21T11:56:39.181061+0000","last_peered":"2024-09-21T11:56:39.181061+0000","last_clean":"2024-09-21T11:56:39.181061+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:56:39.181061+0000","last_undegraded":"2024-09-21T11:56:39.181061+0000","last_fullsized":"2024-09-21T11:56:39.181061+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.602587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.602587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.913702+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"395'15","reported_seq":32,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:40.763316+0000","last_change":"2024-09-21T11:56:36.510297+0000","last_active":"2024-09-21T11:56:40.763316+0000","last_peered":"2024-09-21T11:56:40.763316+0000","last_clean":"2024-09-21T11:56:40.763316+0000","last_became_active":"2024-09-21T11:56:36.510018+0000","last_became_peered":"2024-09-21T11:56:36.510018+0000","last_unstale":"2024-09-21T11:56:40.763316+0000","last_undegraded":"2024-09-21T11:56:40.763316+0000","last_fullsized":"2024-09-21T11:56:40.763316+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:04.522929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":983040,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":564,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.180730+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:56:39.180730+0000","last_peered":"2024-09-21T11:56:39.180730+0000","last_clean":"2024-09-21T11:56:39.180730+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:56:39.180730+0000","last_undegraded":"2024-09-21T11:56:39.180730+0000","last_fullsized":"2024-09-21T11:56:39.180730+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:19.535620+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":246,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.501640+0000","last_change":"2024-09-21T11:56:29.248925+0000","last_active":"2024-09-21T11:56:36.501640+0000","last_peered":"2024-09-21T11:56:36.501640+0000","last_clean":"2024-09-21T11:56:36.501640+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:56:36.501640+0000","last_undegraded":"2024-09-21T11:56:36.501640+0000","last_fullsized":"2024-09-21T11:56:36.501640+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:29.248875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.248875+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:51.923814+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":755,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.552837+0000","last_change":"2024-09-21T11:56:28.083398+0000","last_active":"2024-09-21T11:56:38.552837+0000","last_peered":"2024-09-21T11:56:38.552837+0000","last_clean":"2024-09-21T11:56:38.552837+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:56:38.552837+0000","last_undegraded":"2024-09-21T11:56:38.552837+0000","last_fullsized":"2024-09-21T11:56:38.552837+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:56:28.083324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.083324+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.861453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":394,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181074+0000","last_change":"2024-09-21T11:56:16.667228+0000","last_active":"2024-09-21T11:56:39.181074+0000","last_peered":"2024-09-21T11:56:39.181074+0000","last_clean":"2024-09-21T11:56:39.181074+0000","last_became_active":"2024-09-21T11:56:16.666883+0000","last_became_peered":"2024-09-21T11:56:16.666883+0000","last_unstale":"2024-09-21T11:56:39.181074+0000","last_undegraded":"2024-09-21T11:56:39.181074+0000","last_fullsized":"2024-09-21T11:56:39.181074+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:18.570168+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":703,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.539961+0000","last_change":"2024-09-21T11:54:34.452033+0000","last_active":"2024-09-21T11:56:38.539961+0000","last_peered":"2024-09-21T11:56:38.539961+0000","last_clean":"2024-09-21T11:56:38.539961+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:56:38.539961+0000","last_undegraded":"2024-09-21T11:56:38.539961+0000","last_fullsized":"2024-09-21T11:56:38.539961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:34.451939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:34.451939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"395'11","reported_seq":28,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:40.749537+0000","last_change":"2024-09-21T11:56:36.509094+0000","last_active":"2024-09-21T11:56:40.749537+0000","last_peered":"2024-09-21T11:56:40.749537+0000","last_clean":"2024-09-21T11:56:40.749537+0000","last_became_active":"2024-09-21T11:56:36.508840+0000","last_became_peered":"2024-09-21T11:56:36.508840+0000","last_unstale":"2024-09-21T11:56:40.749537+0000","last_undegraded":"2024-09-21T11:56:40.749537+0000","last_fullsized":"2024-09-21T11:56:40.749537+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:01.292057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":720896,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":382,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.503916+0000","last_change":"2024-09-21T11:56:04.625590+0000","last_active":"2024-09-21T11:56:36.503916+0000","last_peered":"2024-09-21T11:56:36.503916+0000","last_clean":"2024-09-21T11:56:36.503916+0000","last_became_active":"2024-09-21T11:55:28.569095+0000","last_became_peered":"2024-09-21T11:55:28.569095+0000","last_unstale":"2024-09-21T11:56:36.503916+0000","last_undegraded":"2024-09-21T11:56:36.503916+0000","last_fullsized":"2024-09-21T11:56:36.503916+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:56:04.625379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:04.625379+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:07.127924+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":428,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.179889+0000","last_change":"2024-09-21T11:55:06.628254+0000","last_active":"2024-09-21T11:56:39.179889+0000","last_peered":"2024-09-21T11:56:39.179889+0000","last_clean":"2024-09-21T11:56:39.179889+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:56:39.179889+0000","last_undegraded":"2024-09-21T11:56:39.179889+0000","last_fullsized":"2024-09-21T11:56:39.179889+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:55:06.628029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:06.628029+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":752,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.503704+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:56:36.503704+0000","last_peered":"2024-09-21T11:56:36.503704+0000","last_clean":"2024-09-21T11:56:36.503704+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:56:36.503704+0000","last_undegraded":"2024-09-21T11:56:36.503704+0000","last_fullsized":"2024-09-21T11:56:36.503704+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:10.681786+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":719,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.505318+0000","last_change":"2024-09-21T11:55:06.871048+0000","last_active":"2024-09-21T11:56:36.505318+0000","last_peered":"2024-09-21T11:56:36.505318+0000","last_clean":"2024-09-21T11:56:36.505318+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:56:36.505318+0000","last_undegraded":"2024-09-21T11:56:36.505318+0000","last_fullsized":"2024-09-21T11:56:36.505318+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:06.870710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:06.870710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":726,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.001898+0000","last_change":"2024-09-21T11:56:10.595430+0000","last_active":"2024-09-21T11:56:39.001898+0000","last_peered":"2024-09-21T11:56:39.001898+0000","last_clean":"2024-09-21T11:56:39.001898+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:56:39.001898+0000","last_undegraded":"2024-09-21T11:56:39.001898+0000","last_fullsized":"2024-09-21T11:56:39.001898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:10.595373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:10.595373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:19.061656+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"395'10","reported_seq":27,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:40.755982+0000","last_change":"2024-09-21T11:56:36.509416+0000","last_active":"2024-09-21T11:56:40.755982+0000","last_peered":"2024-09-21T11:56:40.755982+0000","last_clean":"2024-09-21T11:56:40.755982+0000","last_became_active":"2024-09-21T11:56:36.509115+0000","last_became_peered":"2024-09-21T11:56:36.509115+0000","last_unstale":"2024-09-21T11:56:40.755982+0000","last_undegraded":"2024-09-21T11:56:40.755982+0000","last_fullsized":"2024-09-21T11:56:40.755982+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:39.857708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":655360,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":610,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181647+0000","last_change":"2024-09-21T11:55:28.581087+0000","last_active":"2024-09-21T11:56:39.181647+0000","last_peered":"2024-09-21T11:56:39.181647+0000","last_clean":"2024-09-21T11:56:39.181647+0000","last_became_active":"2024-09-21T11:55:28.566613+0000","last_became_peered":"2024-09-21T11:55:28.566613+0000","last_unstale":"2024-09-21T11:56:39.181647+0000","last_undegraded":"2024-09-21T11:56:39.181647+0000","last_fullsized":"2024-09-21T11:56:39.181647+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:55:10.349240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:10.349240+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":750,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.500320+0000","last_change":"2024-09-21T11:55:58.281352+0000","last_active":"2024-09-21T11:56:36.500320+0000","last_peered":"2024-09-21T11:56:36.500320+0000","last_clean":"2024-09-21T11:56:36.500320+0000","last_became_active":"2024-09-21T11:55:28.572037+0000","last_became_peered":"2024-09-21T11:55:28.572037+0000","last_unstale":"2024-09-21T11:56:36.500320+0000","last_undegraded":"2024-09-21T11:56:36.500320+0000","last_fullsized":"2024-09-21T11:56:36.500320+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:58.281280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:58.281280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:01.019344+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1773,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.483243+0000","last_change":"2024-09-21T11:56:09.604887+0000","last_active":"2024-09-21T11:56:39.483243+0000","last_peered":"2024-09-21T11:56:39.483243+0000","last_clean":"2024-09-21T11:56:39.483243+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:56:39.483243+0000","last_undegraded":"2024-09-21T11:56:39.483243+0000","last_fullsized":"2024-09-21T11:56:39.483243+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:56:09.604808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.604808+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:21.526417+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":991,"num_read_kb":986,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":736,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553751+0000","last_change":"2024-09-21T11:55:24.998887+0000","last_active":"2024-09-21T11:56:38.553751+0000","last_peered":"2024-09-21T11:56:38.553751+0000","last_clean":"2024-09-21T11:56:38.553751+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:56:38.553751+0000","last_undegraded":"2024-09-21T11:56:38.553751+0000","last_fullsized":"2024-09-21T11:56:38.553751+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.998737+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.998737+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:45.936303+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":424,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.180601+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:56:39.180601+0000","last_peered":"2024-09-21T11:56:39.180601+0000","last_clean":"2024-09-21T11:56:39.180601+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:56:39.180601+0000","last_undegraded":"2024-09-21T11:56:39.180601+0000","last_fullsized":"2024-09-21T11:56:39.180601+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:25.527742+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"395'19","reported_seq":36,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:40.765073+0000","last_change":"2024-09-21T11:56:36.509270+0000","last_active":"2024-09-21T11:56:40.765073+0000","last_peered":"2024-09-21T11:56:40.765073+0000","last_clean":"2024-09-21T11:56:40.765073+0000","last_became_active":"2024-09-21T11:56:36.508533+0000","last_became_peered":"2024-09-21T11:56:36.508533+0000","last_unstale":"2024-09-21T11:56:40.765073+0000","last_undegraded":"2024-09-21T11:56:40.765073+0000","last_fullsized":"2024-09-21T11:56:40.765073+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.121205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1245184,"num_objects":19,"num_object_clones":0,"num_object_copies":57,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":975,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.007352+0000","last_change":"2024-09-21T11:56:29.095770+0000","last_active":"2024-09-21T11:56:39.007352+0000","last_peered":"2024-09-21T11:56:39.007352+0000","last_clean":"2024-09-21T11:56:39.007352+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:56:39.007352+0000","last_undegraded":"2024-09-21T11:56:39.007352+0000","last_fullsized":"2024-09-21T11:56:39.007352+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:56:29.095713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.095713+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.293114+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":732,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.539839+0000","last_change":"2024-09-21T11:55:29.461801+0000","last_active":"2024-09-21T11:56:38.539839+0000","last_peered":"2024-09-21T11:56:38.539839+0000","last_clean":"2024-09-21T11:56:38.539839+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:56:38.539839+0000","last_undegraded":"2024-09-21T11:56:38.539839+0000","last_fullsized":"2024-09-21T11:56:38.539839+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:29.461745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:29.461745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":816,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002292+0000","last_change":"2024-09-21T11:56:07.609933+0000","last_active":"2024-09-21T11:56:39.002292+0000","last_peered":"2024-09-21T11:56:39.002292+0000","last_clean":"2024-09-21T11:56:39.002292+0000","last_became_active":"2024-09-21T11:55:28.570670+0000","last_became_peered":"2024-09-21T11:55:28.570670+0000","last_unstale":"2024-09-21T11:56:39.002292+0000","last_undegraded":"2024-09-21T11:56:39.002292+0000","last_fullsized":"2024-09-21T11:56:39.002292+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:56:07.609858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:56:07.609858+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:25.633763+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":722,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002763+0000","last_change":"2024-09-21T11:51:18.318095+0000","last_active":"2024-09-21T11:56:39.002763+0000","last_peered":"2024-09-21T11:56:39.002763+0000","last_clean":"2024-09-21T11:56:39.002763+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:56:39.002763+0000","last_undegraded":"2024-09-21T11:56:39.002763+0000","last_fullsized":"2024-09-21T11:56:39.002763+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.317950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.317950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:22.592787+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":734,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553700+0000","last_change":"2024-09-21T11:56:05.003664+0000","last_active":"2024-09-21T11:56:38.553700+0000","last_peered":"2024-09-21T11:56:38.553700+0000","last_clean":"2024-09-21T11:56:38.553700+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:56:38.553700+0000","last_undegraded":"2024-09-21T11:56:38.553700+0000","last_fullsized":"2024-09-21T11:56:38.553700+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:05.003603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.003603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:05.356475+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"395'17","reported_seq":34,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:40.804114+0000","last_change":"2024-09-21T11:56:36.508185+0000","last_active":"2024-09-21T11:56:40.804114+0000","last_peered":"2024-09-21T11:56:40.804114+0000","last_clean":"2024-09-21T11:56:40.804114+0000","last_became_active":"2024-09-21T11:56:36.507849+0000","last_became_peered":"2024-09-21T11:56:36.507849+0000","last_unstale":"2024-09-21T11:56:40.804114+0000","last_undegraded":"2024-09-21T11:56:40.804114+0000","last_fullsized":"2024-09-21T11:56:40.804114+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.525964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":1088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":748,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.007237+0000","last_change":"2024-09-21T11:56:09.204336+0000","last_active":"2024-09-21T11:56:39.007237+0000","last_peered":"2024-09-21T11:56:39.007237+0000","last_clean":"2024-09-21T11:56:39.007237+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:56:39.007237+0000","last_undegraded":"2024-09-21T11:56:39.007237+0000","last_fullsized":"2024-09-21T11:56:39.007237+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:56:09.204270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.204270+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:17.886802+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":236,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.503556+0000","last_change":"2024-09-21T11:56:16.671887+0000","last_active":"2024-09-21T11:56:36.503556+0000","last_peered":"2024-09-21T11:56:36.503556+0000","last_clean":"2024-09-21T11:56:36.503556+0000","last_became_active":"2024-09-21T11:56:16.671572+0000","last_became_peered":"2024-09-21T11:56:16.671572+0000","last_unstale":"2024-09-21T11:56:36.503556+0000","last_undegraded":"2024-09-21T11:56:36.503556+0000","last_fullsized":"2024-09-21T11:56:36.503556+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:05.663102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.663102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:27.961529+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":729,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553657+0000","last_change":"2024-09-21T11:56:30.997926+0000","last_active":"2024-09-21T11:56:38.553657+0000","last_peered":"2024-09-21T11:56:38.553657+0000","last_clean":"2024-09-21T11:56:38.553657+0000","last_became_active":"2024-09-21T11:56:16.671360+0000","last_became_peered":"2024-09-21T11:56:16.671360+0000","last_unstale":"2024-09-21T11:56:38.553657+0000","last_undegraded":"2024-09-21T11:56:38.553657+0000","last_fullsized":"2024-09-21T11:56:38.553657+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:30.997880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:30.997880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:34.105058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":733,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.539333+0000","last_change":"2024-09-21T11:56:28.556475+0000","last_active":"2024-09-21T11:56:38.539333+0000","last_peered":"2024-09-21T11:56:38.539333+0000","last_clean":"2024-09-21T11:56:38.539333+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:56:38.539333+0000","last_undegraded":"2024-09-21T11:56:38.539333+0000","last_fullsized":"2024-09-21T11:56:38.539333+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.556423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.556423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:33.767058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.512630+0000","last_change":"2024-09-21T11:56:36.511935+0000","last_active":"2024-09-21T11:56:36.512630+0000","last_peered":"2024-09-21T11:56:36.512630+0000","last_clean":"2024-09-21T11:56:36.512630+0000","last_became_active":"2024-09-21T11:56:36.511642+0000","last_became_peered":"2024-09-21T11:56:36.511642+0000","last_unstale":"2024-09-21T11:56:36.512630+0000","last_undegraded":"2024-09-21T11:56:36.512630+0000","last_fullsized":"2024-09-21T11:56:36.512630+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:51.191998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":832,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.504833+0000","last_change":"2024-09-21T11:56:03.668325+0000","last_active":"2024-09-21T11:56:36.504833+0000","last_peered":"2024-09-21T11:56:36.504833+0000","last_clean":"2024-09-21T11:56:36.504833+0000","last_became_active":"2024-09-21T11:55:28.561575+0000","last_became_peered":"2024-09-21T11:55:28.561575+0000","last_unstale":"2024-09-21T11:56:36.504833+0000","last_undegraded":"2024-09-21T11:56:36.504833+0000","last_fullsized":"2024-09-21T11:56:36.504833+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:56:03.667750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:03.667750+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:16.766955+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":760,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.006923+0000","last_change":"2024-09-21T11:56:28.045735+0000","last_active":"2024-09-21T11:56:39.006923+0000","last_peered":"2024-09-21T11:56:39.006923+0000","last_clean":"2024-09-21T11:56:39.006923+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:56:39.006923+0000","last_undegraded":"2024-09-21T11:56:39.006923+0000","last_fullsized":"2024-09-21T11:56:39.006923+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.045670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.045670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:35.259699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":424,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.180516+0000","last_change":"2024-09-21T11:55:10.605948+0000","last_active":"2024-09-21T11:56:39.180516+0000","last_peered":"2024-09-21T11:56:39.180516+0000","last_clean":"2024-09-21T11:56:39.180516+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:56:39.180516+0000","last_undegraded":"2024-09-21T11:56:39.180516+0000","last_fullsized":"2024-09-21T11:56:39.180516+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:10.605869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:10.605869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.452804+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":723,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.504096+0000","last_change":"2024-09-21T11:55:24.583429+0000","last_active":"2024-09-21T11:56:36.504096+0000","last_peered":"2024-09-21T11:56:36.504096+0000","last_clean":"2024-09-21T11:56:36.504096+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:56:36.504096+0000","last_undegraded":"2024-09-21T11:56:36.504096+0000","last_fullsized":"2024-09-21T11:56:36.504096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.583208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.583208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:40.880319+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"395'21","reported_seq":38,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:40.806377+0000","last_change":"2024-09-21T11:56:36.504822+0000","last_active":"2024-09-21T11:56:40.806377+0000","last_peered":"2024-09-21T11:56:40.806377+0000","last_clean":"2024-09-21T11:56:40.806377+0000","last_became_active":"2024-09-21T11:56:36.504516+0000","last_became_peered":"2024-09-21T11:56:36.504516+0000","last_unstale":"2024-09-21T11:56:40.806377+0000","last_undegraded":"2024-09-21T11:56:40.806377+0000","last_fullsized":"2024-09-21T11:56:40.806377+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:54.965928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1376256,"num_objects":21,"num_object_clones":0,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":717,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540292+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:56:38.540292+0000","last_peered":"2024-09-21T11:56:38.540292+0000","last_clean":"2024-09-21T11:56:38.540292+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:56:38.540292+0000","last_undegraded":"2024-09-21T11:56:38.540292+0000","last_fullsized":"2024-09-21T11:56:38.540292+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:22.507278+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":730,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540690+0000","last_change":"2024-09-21T11:56:09.486510+0000","last_active":"2024-09-21T11:56:38.540690+0000","last_peered":"2024-09-21T11:56:38.540690+0000","last_clean":"2024-09-21T11:56:38.540690+0000","last_became_active":"2024-09-21T11:55:28.568312+0000","last_became_peered":"2024-09-21T11:55:28.568312+0000","last_unstale":"2024-09-21T11:56:38.540690+0000","last_undegraded":"2024-09-21T11:56:38.540690+0000","last_fullsized":"2024-09-21T11:56:38.540690+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:09.486386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.486386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:29.172746+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":943,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.504742+0000","last_change":"2024-09-21T11:56:08.680971+0000","last_active":"2024-09-21T11:56:36.504742+0000","last_peered":"2024-09-21T11:56:36.504742+0000","last_clean":"2024-09-21T11:56:36.504742+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:56:36.504742+0000","last_undegraded":"2024-09-21T11:56:36.504742+0000","last_fullsized":"2024-09-21T11:56:36.504742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:56:08.680766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.680766+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:16.508041+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":551,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181973+0000","last_change":"2024-09-21T11:56:16.669124+0000","last_active":"2024-09-21T11:56:39.181973+0000","last_peered":"2024-09-21T11:56:39.181973+0000","last_clean":"2024-09-21T11:56:39.181973+0000","last_became_active":"2024-09-21T11:56:16.668785+0000","last_became_peered":"2024-09-21T11:56:16.668785+0000","last_unstale":"2024-09-21T11:56:39.181973+0000","last_undegraded":"2024-09-21T11:56:39.181973+0000","last_fullsized":"2024-09-21T11:56:39.181973+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:07.658022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:07.658022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"395'129","reported_seq":146,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:41.241915+0000","last_change":"2024-09-21T11:56:36.508655+0000","last_active":"2024-09-21T11:56:41.241915+0000","last_peered":"2024-09-21T11:56:41.241915+0000","last_clean":"2024-09-21T11:56:41.241915+0000","last_became_active":"2024-09-21T11:56:36.508299+0000","last_became_peered":"2024-09-21T11:56:36.508299+0000","last_unstale":"2024-09-21T11:56:41.241915+0000","last_undegraded":"2024-09-21T11:56:41.241915+0000","last_fullsized":"2024-09-21T11:56:41.241915+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:05.125216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8454144,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":8256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":765,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.501455+0000","last_change":"2024-09-21T11:55:07.300834+0000","last_active":"2024-09-21T11:56:36.501455+0000","last_peered":"2024-09-21T11:56:36.501455+0000","last_clean":"2024-09-21T11:56:36.501455+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:56:36.501455+0000","last_undegraded":"2024-09-21T11:56:36.501455+0000","last_fullsized":"2024-09-21T11:56:36.501455+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:07.300758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:07.300758+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":754,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553162+0000","last_change":"2024-09-21T11:55:34.034149+0000","last_active":"2024-09-21T11:56:38.553162+0000","last_peered":"2024-09-21T11:56:38.553162+0000","last_clean":"2024-09-21T11:56:38.553162+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:56:38.553162+0000","last_undegraded":"2024-09-21T11:56:38.553162+0000","last_fullsized":"2024-09-21T11:56:38.553162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:34.034076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:34.034076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:51.437569+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":718,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.504006+0000","last_change":"2024-09-21T11:56:16.668904+0000","last_active":"2024-09-21T11:56:36.504006+0000","last_peered":"2024-09-21T11:56:36.504006+0000","last_clean":"2024-09-21T11:56:36.504006+0000","last_became_active":"2024-09-21T11:56:16.668588+0000","last_became_peered":"2024-09-21T11:56:16.668588+0000","last_unstale":"2024-09-21T11:56:36.504006+0000","last_undegraded":"2024-09-21T11:56:36.504006+0000","last_fullsized":"2024-09-21T11:56:36.504006+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.990626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.990626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:18.813963+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":81,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002107+0000","last_change":"2024-09-21T11:55:35.532779+0000","last_active":"2024-09-21T11:56:39.002107+0000","last_peered":"2024-09-21T11:56:39.002107+0000","last_clean":"2024-09-21T11:56:39.002107+0000","last_became_active":"2024-09-21T11:55:28.566108+0000","last_became_peered":"2024-09-21T11:55:28.566108+0000","last_unstale":"2024-09-21T11:56:39.002107+0000","last_undegraded":"2024-09-21T11:56:39.002107+0000","last_fullsized":"2024-09-21T11:56:39.002107+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:35.532577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:35.532577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"395'108","reported_seq":125,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:41.268980+0000","last_change":"2024-09-21T11:56:36.506979+0000","last_active":"2024-09-21T11:56:41.268980+0000","last_peered":"2024-09-21T11:56:41.268980+0000","last_clean":"2024-09-21T11:56:41.268980+0000","last_became_active":"2024-09-21T11:56:36.506694+0000","last_became_peered":"2024-09-21T11:56:36.506694+0000","last_unstale":"2024-09-21T11:56:41.268980+0000","last_undegraded":"2024-09-21T11:56:41.268980+0000","last_fullsized":"2024-09-21T11:56:41.268980+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.639288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":835,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.006809+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:56:39.006809+0000","last_peered":"2024-09-21T11:56:39.006809+0000","last_clean":"2024-09-21T11:56:39.006809+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:56:39.006809+0000","last_undegraded":"2024-09-21T11:56:39.006809+0000","last_fullsized":"2024-09-21T11:56:39.006809+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:18.201979+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":722,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002799+0000","last_change":"2024-09-21T11:49:49.480836+0000","last_active":"2024-09-21T11:56:39.002799+0000","last_peered":"2024-09-21T11:56:39.002799+0000","last_clean":"2024-09-21T11:56:39.002799+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:56:39.002799+0000","last_undegraded":"2024-09-21T11:56:39.002799+0000","last_fullsized":"2024-09-21T11:56:39.002799+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.480775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.480775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:49.576280+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":727,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.505075+0000","last_change":"2024-09-21T11:51:48.941202+0000","last_active":"2024-09-21T11:56:36.505075+0000","last_peered":"2024-09-21T11:56:36.505075+0000","last_clean":"2024-09-21T11:56:36.505075+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:56:36.505075+0000","last_undegraded":"2024-09-21T11:56:36.505075+0000","last_fullsized":"2024-09-21T11:56:36.505075+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:48.941128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:48.941128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:24.866192+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":726,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002234+0000","last_change":"2024-09-21T11:55:50.647537+0000","last_active":"2024-09-21T11:56:39.002234+0000","last_peered":"2024-09-21T11:56:39.002234+0000","last_clean":"2024-09-21T11:56:39.002234+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:56:39.002234+0000","last_undegraded":"2024-09-21T11:56:39.002234+0000","last_fullsized":"2024-09-21T11:56:39.002234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:50.647374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:50.647374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:53.911019+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.624348+0000","last_change":"2024-09-21T11:56:36.623904+0000","last_active":"2024-09-21T11:56:36.624348+0000","last_peered":"2024-09-21T11:56:36.624348+0000","last_clean":"2024-09-21T11:56:36.624348+0000","last_became_active":"2024-09-21T11:56:36.623611+0000","last_became_peered":"2024-09-21T11:56:36.623611+0000","last_unstale":"2024-09-21T11:56:36.624348+0000","last_undegraded":"2024-09-21T11:56:36.624348+0000","last_fullsized":"2024-09-21T11:56:36.624348+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:03.170336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":757,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181350+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:56:39.181350+0000","last_peered":"2024-09-21T11:56:39.181350+0000","last_clean":"2024-09-21T11:56:39.181350+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:56:39.181350+0000","last_undegraded":"2024-09-21T11:56:39.181350+0000","last_fullsized":"2024-09-21T11:56:39.181350+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:16.543488+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":775,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.003053+0000","last_change":"2024-09-21T11:56:29.486116+0000","last_active":"2024-09-21T11:56:39.003053+0000","last_peered":"2024-09-21T11:56:39.003053+0000","last_clean":"2024-09-21T11:56:39.003053+0000","last_became_active":"2024-09-21T11:56:16.670969+0000","last_became_peered":"2024-09-21T11:56:16.670969+0000","last_unstale":"2024-09-21T11:56:39.003053+0000","last_undegraded":"2024-09-21T11:56:39.003053+0000","last_fullsized":"2024-09-21T11:56:39.003053+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:29.486067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.486067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:47.036894+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":741,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.503171+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:56:36.503171+0000","last_peered":"2024-09-21T11:56:36.503171+0000","last_clean":"2024-09-21T11:56:36.503171+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:56:36.503171+0000","last_undegraded":"2024-09-21T11:56:36.503171+0000","last_fullsized":"2024-09-21T11:56:36.503171+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:13.728141+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":733,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540374+0000","last_change":"2024-09-21T11:56:02.541452+0000","last_active":"2024-09-21T11:56:38.540374+0000","last_peered":"2024-09-21T11:56:38.540374+0000","last_clean":"2024-09-21T11:56:38.540374+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:56:38.540374+0000","last_undegraded":"2024-09-21T11:56:38.540374+0000","last_fullsized":"2024-09-21T11:56:38.540374+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.541232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.541232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:16.599825+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"395'134","reported_seq":151,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:41.452443+0000","last_change":"2024-09-21T11:56:36.624091+0000","last_active":"2024-09-21T11:56:41.452443+0000","last_peered":"2024-09-21T11:56:41.452443+0000","last_clean":"2024-09-21T11:56:41.452443+0000","last_became_active":"2024-09-21T11:56:36.623765+0000","last_became_peered":"2024-09-21T11:56:36.623765+0000","last_unstale":"2024-09-21T11:56:41.452443+0000","last_undegraded":"2024-09-21T11:56:41.452443+0000","last_fullsized":"2024-09-21T11:56:41.452443+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:59.412761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8781824,"num_objects":134,"num_object_clones":0,"num_object_copies":402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":8576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":777,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.552544+0000","last_change":"2024-09-21T11:56:29.063211+0000","last_active":"2024-09-21T11:56:38.552544+0000","last_peered":"2024-09-21T11:56:38.552544+0000","last_clean":"2024-09-21T11:56:38.552544+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:56:38.552544+0000","last_undegraded":"2024-09-21T11:56:38.552544+0000","last_fullsized":"2024-09-21T11:56:38.552544+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:29.063150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.063150+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:45.931544+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":771,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.003141+0000","last_change":"2024-09-21T11:55:34.552520+0000","last_active":"2024-09-21T11:56:39.003141+0000","last_peered":"2024-09-21T11:56:39.003141+0000","last_clean":"2024-09-21T11:56:39.003141+0000","last_became_active":"2024-09-21T11:55:28.561948+0000","last_became_peered":"2024-09-21T11:55:28.561948+0000","last_unstale":"2024-09-21T11:56:39.003141+0000","last_undegraded":"2024-09-21T11:56:39.003141+0000","last_fullsized":"2024-09-21T11:56:39.003141+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:55:34.552437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:34.552437+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:01.097111+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":939,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002383+0000","last_change":"2024-09-21T11:56:04.614322+0000","last_active":"2024-09-21T11:56:39.002383+0000","last_peered":"2024-09-21T11:56:39.002383+0000","last_clean":"2024-09-21T11:56:39.002383+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:56:39.002383+0000","last_undegraded":"2024-09-21T11:56:39.002383+0000","last_fullsized":"2024-09-21T11:56:39.002383+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:56:04.614247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:04.614247+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:09.974717+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":710,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.552636+0000","last_change":"2024-09-21T11:53:27.992553+0000","last_active":"2024-09-21T11:56:38.552636+0000","last_peered":"2024-09-21T11:56:38.552636+0000","last_clean":"2024-09-21T11:56:38.552636+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:56:38.552636+0000","last_undegraded":"2024-09-21T11:56:38.552636+0000","last_fullsized":"2024-09-21T11:56:38.552636+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.992489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.992489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:40.984476+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":753,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.006776+0000","last_change":"2024-09-21T11:55:28.557481+0000","last_active":"2024-09-21T11:56:39.006776+0000","last_peered":"2024-09-21T11:56:39.006776+0000","last_clean":"2024-09-21T11:56:39.006776+0000","last_became_active":"2024-09-21T11:55:28.557190+0000","last_became_peered":"2024-09-21T11:55:28.557190+0000","last_unstale":"2024-09-21T11:56:39.006776+0000","last_undegraded":"2024-09-21T11:56:39.006776+0000","last_fullsized":"2024-09-21T11:56:39.006776+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:38.208809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.208809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.969866+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":766,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540007+0000","last_change":"2024-09-21T11:56:29.528662+0000","last_active":"2024-09-21T11:56:38.540007+0000","last_peered":"2024-09-21T11:56:38.540007+0000","last_clean":"2024-09-21T11:56:38.540007+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:56:38.540007+0000","last_undegraded":"2024-09-21T11:56:38.540007+0000","last_fullsized":"2024-09-21T11:56:38.540007+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:56:29.528598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.528598+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:47.850219+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":759,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553389+0000","last_change":"2024-09-21T11:55:54.003787+0000","last_active":"2024-09-21T11:56:38.553389+0000","last_peered":"2024-09-21T11:56:38.553389+0000","last_clean":"2024-09-21T11:56:38.553389+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:56:38.553389+0000","last_undegraded":"2024-09-21T11:56:38.553389+0000","last_fullsized":"2024-09-21T11:56:38.553389+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:54.003722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:54.003722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:06.305761+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":740,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002537+0000","last_change":"2024-09-21T11:56:16.670921+0000","last_active":"2024-09-21T11:56:39.002537+0000","last_peered":"2024-09-21T11:56:39.002537+0000","last_clean":"2024-09-21T11:56:39.002537+0000","last_became_active":"2024-09-21T11:56:16.670467+0000","last_became_peered":"2024-09-21T11:56:16.670467+0000","last_unstale":"2024-09-21T11:56:39.002537+0000","last_undegraded":"2024-09-21T11:56:39.002537+0000","last_fullsized":"2024-09-21T11:56:39.002537+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:59.640643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.640643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:03.630415+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":755,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.500882+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:56:36.500882+0000","last_peered":"2024-09-21T11:56:36.500882+0000","last_clean":"2024-09-21T11:56:36.500882+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:56:36.500882+0000","last_undegraded":"2024-09-21T11:56:36.500882+0000","last_fullsized":"2024-09-21T11:56:36.500882+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:18.135115+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":722,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002472+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:56:39.002472+0000","last_peered":"2024-09-21T11:56:39.002472+0000","last_clean":"2024-09-21T11:56:39.002472+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:56:39.002472+0000","last_undegraded":"2024-09-21T11:56:39.002472+0000","last_fullsized":"2024-09-21T11:56:39.002472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:25.568680+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":744,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.539887+0000","last_change":"2024-09-21T11:55:03.415840+0000","last_active":"2024-09-21T11:56:38.539887+0000","last_peered":"2024-09-21T11:56:38.539887+0000","last_clean":"2024-09-21T11:56:38.539887+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:56:38.539887+0000","last_undegraded":"2024-09-21T11:56:38.539887+0000","last_fullsized":"2024-09-21T11:56:38.539887+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:03.415774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:03.415774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":715,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.502426+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:56:36.502426+0000","last_peered":"2024-09-21T11:56:36.502426+0000","last_clean":"2024-09-21T11:56:36.502426+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:56:36.502426+0000","last_undegraded":"2024-09-21T11:56:36.502426+0000","last_fullsized":"2024-09-21T11:56:36.502426+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:11.666188+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":758,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553433+0000","last_change":"2024-09-21T11:56:34.119200+0000","last_active":"2024-09-21T11:56:38.553433+0000","last_peered":"2024-09-21T11:56:38.553433+0000","last_clean":"2024-09-21T11:56:38.553433+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:56:38.553433+0000","last_undegraded":"2024-09-21T11:56:38.553433+0000","last_fullsized":"2024-09-21T11:56:38.553433+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:34.119128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:34.119128+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.898211+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":742,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.500708+0000","last_change":"2024-09-21T11:55:04.304207+0000","last_active":"2024-09-21T11:56:36.500708+0000","last_peered":"2024-09-21T11:56:36.500708+0000","last_clean":"2024-09-21T11:56:36.500708+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:56:36.500708+0000","last_undegraded":"2024-09-21T11:56:36.500708+0000","last_fullsized":"2024-09-21T11:56:36.500708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:04.304136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:04.304136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":916,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540184+0000","last_change":"2024-09-21T11:55:51.317564+0000","last_active":"2024-09-21T11:56:38.540184+0000","last_peered":"2024-09-21T11:56:38.540184+0000","last_clean":"2024-09-21T11:56:38.540184+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:56:38.540184+0000","last_undegraded":"2024-09-21T11:56:38.540184+0000","last_fullsized":"2024-09-21T11:56:38.540184+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:51.317473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.317473+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:12.961860+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":213,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.504048+0000","last_change":"2024-09-21T11:51:43.531318+0000","last_active":"2024-09-21T11:56:36.504048+0000","last_peered":"2024-09-21T11:56:36.504048+0000","last_clean":"2024-09-21T11:56:36.504048+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:56:36.504048+0000","last_undegraded":"2024-09-21T11:56:36.504048+0000","last_fullsized":"2024-09-21T11:56:36.504048+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.826834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.826834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.778367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":779,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540375+0000","last_change":"2024-09-21T11:53:25.264037+0000","last_active":"2024-09-21T11:56:38.540375+0000","last_peered":"2024-09-21T11:56:38.540375+0000","last_clean":"2024-09-21T11:56:38.540375+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:56:38.540375+0000","last_undegraded":"2024-09-21T11:56:38.540375+0000","last_fullsized":"2024-09-21T11:56:38.540375+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:25.263955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:25.263955+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.464657+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":730,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002658+0000","last_change":"2024-09-21T11:56:33.473303+0000","last_active":"2024-09-21T11:56:39.002658+0000","last_peered":"2024-09-21T11:56:39.002658+0000","last_clean":"2024-09-21T11:56:39.002658+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:56:39.002658+0000","last_undegraded":"2024-09-21T11:56:39.002658+0000","last_fullsized":"2024-09-21T11:56:39.002658+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.473240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.473240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.182237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":961,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553362+0000","last_change":"2024-09-21T11:55:26.971522+0000","last_active":"2024-09-21T11:56:38.553362+0000","last_peered":"2024-09-21T11:56:38.553362+0000","last_clean":"2024-09-21T11:56:38.553362+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:56:38.553362+0000","last_undegraded":"2024-09-21T11:56:38.553362+0000","last_fullsized":"2024-09-21T11:56:38.553362+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:55:26.971440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:26.971440+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:44.656785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":81,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002805+0000","last_change":"2024-09-21T11:55:33.546350+0000","last_active":"2024-09-21T11:56:39.002805+0000","last_peered":"2024-09-21T11:56:39.002805+0000","last_clean":"2024-09-21T11:56:39.002805+0000","last_became_active":"2024-09-21T11:55:28.565700+0000","last_became_peered":"2024-09-21T11:55:28.565700+0000","last_unstale":"2024-09-21T11:56:39.002805+0000","last_undegraded":"2024-09-21T11:56:39.002805+0000","last_fullsized":"2024-09-21T11:56:39.002805+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:33.546264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:33.546264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:57.443968+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":767,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.007264+0000","last_change":"2024-09-21T11:51:43.533694+0000","last_active":"2024-09-21T11:56:39.007264+0000","last_peered":"2024-09-21T11:56:39.007264+0000","last_clean":"2024-09-21T11:56:39.007264+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:56:39.007264+0000","last_undegraded":"2024-09-21T11:56:39.007264+0000","last_fullsized":"2024-09-21T11:56:39.007264+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:17.242122+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":233,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002926+0000","last_change":"2024-09-21T11:55:24.538954+0000","last_active":"2024-09-21T11:56:39.002926+0000","last_peered":"2024-09-21T11:56:39.002926+0000","last_clean":"2024-09-21T11:56:39.002926+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:56:39.002926+0000","last_undegraded":"2024-09-21T11:56:39.002926+0000","last_fullsized":"2024-09-21T11:56:39.002926+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.538880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.538880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":432,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181738+0000","last_change":"2024-09-21T11:56:00.441644+0000","last_active":"2024-09-21T11:56:39.181738+0000","last_peered":"2024-09-21T11:56:39.181738+0000","last_clean":"2024-09-21T11:56:39.181738+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:56:39.181738+0000","last_undegraded":"2024-09-21T11:56:39.181738+0000","last_fullsized":"2024-09-21T11:56:39.181738+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:00.441565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:00.441565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:27.350768+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":718,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540493+0000","last_change":"2024-09-21T11:55:28.563670+0000","last_active":"2024-09-21T11:56:38.540493+0000","last_peered":"2024-09-21T11:56:38.540493+0000","last_clean":"2024-09-21T11:56:38.540493+0000","last_became_active":"2024-09-21T11:55:28.563340+0000","last_became_peered":"2024-09-21T11:55:28.563340+0000","last_unstale":"2024-09-21T11:56:38.540493+0000","last_undegraded":"2024-09-21T11:56:38.540493+0000","last_fullsized":"2024-09-21T11:56:38.540493+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:01.434123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.434123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":443,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181921+0000","last_change":"2024-09-21T11:55:59.419239+0000","last_active":"2024-09-21T11:56:39.181921+0000","last_peered":"2024-09-21T11:56:39.181921+0000","last_clean":"2024-09-21T11:56:39.181921+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:56:39.181921+0000","last_undegraded":"2024-09-21T11:56:39.181921+0000","last_fullsized":"2024-09-21T11:56:39.181921+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:59.419148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.419148+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:24.349135+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":721,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002645+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:56:39.002645+0000","last_peered":"2024-09-21T11:56:39.002645+0000","last_clean":"2024-09-21T11:56:39.002645+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:56:39.002645+0000","last_undegraded":"2024-09-21T11:56:39.002645+0000","last_fullsized":"2024-09-21T11:56:39.002645+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.630324+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":729,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.541084+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:56:38.541084+0000","last_peered":"2024-09-21T11:56:38.541084+0000","last_clean":"2024-09-21T11:56:38.541084+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:56:38.541084+0000","last_undegraded":"2024-09-21T11:56:38.541084+0000","last_fullsized":"2024-09-21T11:56:38.541084+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:19.549269+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":753,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.497759+0000","last_change":"2024-09-21T11:55:59.288321+0000","last_active":"2024-09-21T11:56:36.497759+0000","last_peered":"2024-09-21T11:56:36.497759+0000","last_clean":"2024-09-21T11:56:36.497759+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:56:36.497759+0000","last_undegraded":"2024-09-21T11:56:36.497759+0000","last_fullsized":"2024-09-21T11:56:36.497759+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:55:59.288240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.288240+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:13.275297+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":758,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540668+0000","last_change":"2024-09-21T11:53:32.308905+0000","last_active":"2024-09-21T11:56:38.540668+0000","last_peered":"2024-09-21T11:56:38.540668+0000","last_clean":"2024-09-21T11:56:38.540668+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:56:38.540668+0000","last_undegraded":"2024-09-21T11:56:38.540668+0000","last_fullsized":"2024-09-21T11:56:38.540668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:32.308827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:32.308827+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:16.498027+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":743,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.553515+0000","last_change":"2024-09-21T11:53:28.945615+0000","last_active":"2024-09-21T11:56:38.553515+0000","last_peered":"2024-09-21T11:56:38.553515+0000","last_clean":"2024-09-21T11:56:38.553515+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:56:38.553515+0000","last_undegraded":"2024-09-21T11:56:38.553515+0000","last_fullsized":"2024-09-21T11:56:38.553515+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:28.945564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:53:28.945564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:47.015587+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":706,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.001964+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:56:39.001964+0000","last_peered":"2024-09-21T11:56:39.001964+0000","last_clean":"2024-09-21T11:56:39.001964+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:56:39.001964+0000","last_undegraded":"2024-09-21T11:56:39.001964+0000","last_fullsized":"2024-09-21T11:56:39.001964+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:19.625346+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":726,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540603+0000","last_change":"2024-09-21T11:55:28.564578+0000","last_active":"2024-09-21T11:56:38.540603+0000","last_peered":"2024-09-21T11:56:38.540603+0000","last_clean":"2024-09-21T11:56:38.540603+0000","last_became_active":"2024-09-21T11:55:28.564261+0000","last_became_peered":"2024-09-21T11:55:28.564261+0000","last_unstale":"2024-09-21T11:56:38.540603+0000","last_undegraded":"2024-09-21T11:56:38.540603+0000","last_fullsized":"2024-09-21T11:56:38.540603+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.551852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.551852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":770,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.539851+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:56:38.539851+0000","last_peered":"2024-09-21T11:56:38.539851+0000","last_clean":"2024-09-21T11:56:38.539851+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:56:38.539851+0000","last_undegraded":"2024-09-21T11:56:38.539851+0000","last_fullsized":"2024-09-21T11:56:38.539851+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:25.518080+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":709,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.499968+0000","last_change":"2024-09-21T11:50:25.708558+0000","last_active":"2024-09-21T11:56:36.499968+0000","last_peered":"2024-09-21T11:56:36.499968+0000","last_clean":"2024-09-21T11:56:36.499968+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:56:36.499968+0000","last_undegraded":"2024-09-21T11:56:36.499968+0000","last_fullsized":"2024-09-21T11:56:36.499968+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:25.708485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:50:25.708485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:50.171680+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":729,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.006741+0000","last_change":"2024-09-21T11:55:52.217954+0000","last_active":"2024-09-21T11:56:39.006741+0000","last_peered":"2024-09-21T11:56:39.006741+0000","last_clean":"2024-09-21T11:56:39.006741+0000","last_became_active":"2024-09-21T11:55:28.575256+0000","last_became_peered":"2024-09-21T11:55:28.575256+0000","last_unstale":"2024-09-21T11:56:39.006741+0000","last_undegraded":"2024-09-21T11:56:39.006741+0000","last_fullsized":"2024-09-21T11:56:39.006741+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:52.217898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.217898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.160221+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":726,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.002182+0000","last_change":"2024-09-21T11:54:57.381670+0000","last_active":"2024-09-21T11:56:39.002182+0000","last_peered":"2024-09-21T11:56:39.002182+0000","last_clean":"2024-09-21T11:56:39.002182+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:56:39.002182+0000","last_undegraded":"2024-09-21T11:56:39.002182+0000","last_fullsized":"2024-09-21T11:56:39.002182+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:57.381605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:57.381605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.460530+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":430,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.181293+0000","last_change":"2024-09-21T11:56:16.669730+0000","last_active":"2024-09-21T11:56:39.181293+0000","last_peered":"2024-09-21T11:56:39.181293+0000","last_clean":"2024-09-21T11:56:39.181293+0000","last_became_active":"2024-09-21T11:56:16.669407+0000","last_became_peered":"2024-09-21T11:56:16.669407+0000","last_unstale":"2024-09-21T11:56:39.181293+0000","last_undegraded":"2024-09-21T11:56:39.181293+0000","last_fullsized":"2024-09-21T11:56:39.181293+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:24.528780+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":751,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:38.540076+0000","last_change":"2024-09-21T11:55:28.563845+0000","last_active":"2024-09-21T11:56:38.540076+0000","last_peered":"2024-09-21T11:56:38.540076+0000","last_clean":"2024-09-21T11:56:38.540076+0000","last_became_active":"2024-09-21T11:55:28.563540+0000","last_became_peered":"2024-09-21T11:55:28.563540+0000","last_unstale":"2024-09-21T11:56:38.540076+0000","last_undegraded":"2024-09-21T11:56:38.540076+0000","last_fullsized":"2024-09-21T11:56:38.540076+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:23.601318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.601318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.497705+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":828,"reported_epoch":395,"state":"active+clean","last_fresh":"2024-09-21T11:56:39.007450+0000","last_change":"2024-09-21T11:56:33.048379+0000","last_active":"2024-09-21T11:56:39.007450+0000","last_peered":"2024-09-21T11:56:39.007450+0000","last_clean":"2024-09-21T11:56:39.007450+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:56:39.007450+0000","last_undegraded":"2024-09-21T11:56:39.007450+0000","last_fullsized":"2024-09-21T11:56:39.007450+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:56:33.048316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.048316+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.955454+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":719,"reported_epoch":393,"state":"active+clean","last_fresh":"2024-09-21T11:56:36.500281+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:56:36.500281+0000","last_peered":"2024-09-21T11:56:36.500281+0000","last_clean":"2024-09-21T11:56:36.500281+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:56:36.500281+0000","last_undegraded":"2024-09-21T11:56:36.500281+0000","last_fullsized":"2024-09-21T11:56:36.500281+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:56:46.131671+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":67436544,"num_objects":1029,"num_object_clones":0,"num_object_copies":3087,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1029,"num_write_kb":65856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":205062144,"data_stored":205062144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1029,"ondisk_log_size":1029,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":6024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":758,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":991,"num_read_kb":986,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234189,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":628144,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":93117008,"statfs":{"total":95995035648,"available":95351816192,"internally_reserved":0,"allocated":1396736,"data_stored":997325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":641792458},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463122,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":523996,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":522622,"kb_avail":93221156,"statfs":{"total":95995035648,"available":95458463744,"internally_reserved":0,"allocated":1376256,"data_stored":997449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":535165382},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724759,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":594136,"kb_used_data":21448,"kb_used_omap":1,"kb_used_meta":572606,"kb_avail":93151016,"statfs":{"total":95995035648,"available":95386640384,"internally_reserved":0,"allocated":21962752,"data_stored":21574171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":586348998},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920989,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":501208,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":500414,"kb_avail":93243944,"statfs":{"total":95995035648,"available":95481798656,"internally_reserved":0,"allocated":782336,"data_stored":406718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":512424389},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117219,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":653912,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":653118,"kb_avail":93091240,"statfs":{"total":95995035648,"available":95325429760,"internally_reserved":0,"allocated":782336,"data_stored":405919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":668793287},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411560,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":635844,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":635070,"kb_avail":93109308,"statfs":{"total":95995035648,"available":95343931392,"internally_reserved":0,"allocated":770048,"data_stored":392120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":650312133},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575088,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":711552,"kb_used_data":76464,"kb_used_omap":1,"kb_used_meta":635070,"kb_avail":93033600,"statfs":{"total":95995035648,"available":95266406400,"internally_reserved":0,"allocated":78299136,"data_stored":77871478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":650312134},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738613,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":595572,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":594814,"kb_avail":93149580,"statfs":{"total":95995035648,"available":95385169920,"internally_reserved":0,"allocated":745472,"data_stored":392281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":609089989},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":93323264,"data_stored":93323264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7012352,"data_stored":7012352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":2162688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40042496,"data_stored":40042496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12124160,"data_stored":12124160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":50397184,"data_stored":50397184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:56:45.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool set .mgr pgp_num 1 2024-09-21T11:56:45.109 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:45.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:45.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:45.191 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:45.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:45.320 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:56:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:56:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:45.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:45.342 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:45.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:45.443 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:45.473 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:45.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:45.475 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:45.576 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:45.612 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:45.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:45.613 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:45.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:56:45.682 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 5 16 13423 13407 167.551 156.438 0.00554364 0.00595947 2024-09-21T11:56:45.714 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:45.748 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:45.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:45.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:45 smithi175 ceph-mon[32613]: pgmap v813: 148 pgs: 148 active+clean; 104 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 775 B/s rd, 16 MiB/s wr, 251 op/s 2024-09-21T11:56:45.822 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:56:45.823 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:56:45.841 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:45.851 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:45.864 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:56:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:56:45.879 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:45.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:45.880 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:45.906 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:56:45.907 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:56:45.907 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:45 smithi043 ceph-mon[36454]: pgmap v813: 148 pgs: 148 active+clean; 104 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 775 B/s rd, 16 MiB/s wr, 251 op/s 2024-09-21T11:56:45.908 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:45 smithi043 ceph-mon[29710]: pgmap v813: 148 pgs: 148 active+clean; 104 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 775 B/s rd, 16 MiB/s wr, 251 op/s 2024-09-21T11:56:45.944 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:56:45.945 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:56:45.979 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:56:45.979 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:56:45.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:46.017 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:56:46.017 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:56:46.020 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:46.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:46.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:46.032 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:46.051 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:56:46.052 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:56:46.085 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:56:46.086 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:56:46.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:46.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:46.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:46.273 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:46.306 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:46.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:46.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:46.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:46.445 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:46.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:46.446 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:46.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:46.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:46.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:46.621 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:56:46.621 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:56:46.621 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:56:46.621 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:56:46.621 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:56:46.682 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 6 16 15753 15737 163.895 145.625 0.00378403 0.00609086 2024-09-21T11:56:46.688 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:46.716 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:46.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:46.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:46.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:46 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4187383197' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:46.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:46 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4187383197' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:46.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:46 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4187383197' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:46.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:46.847 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:46.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:46.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:46.948 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:46.984 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:46.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:46.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:47.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:47.126 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:47.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:47.127 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:47.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:47.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:47.269 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:47.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:47.402 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:47.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:47.403 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:47.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:47.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:47.544 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:47.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:47.682 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 7 16 18232 18216 162.597 154.938 0.00311802 0.00614145 2024-09-21T11:56:47.684 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:47.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:47.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:47.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:56:47.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:47.824 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:47.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:47.825 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:47.838 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:56:47.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:47 smithi043 ceph-mon[36454]: pgmap v814: 148 pgs: 148 active+clean; 424 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 846 op/s 2024-09-21T11:56:47.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:47 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:47.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:47 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:47.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1390030708' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:56:47.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:47 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:56:47.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:47 smithi043 ceph-mon[29710]: pgmap v814: 148 pgs: 148 active+clean; 424 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 846 op/s 2024-09-21T11:56:47.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:47 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:47.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:47 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:47.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1390030708' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:56:47.897 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:47 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:56:47.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:47.977 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:47.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:47.978 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:47 smithi175 ceph-mon[32613]: pgmap v814: 148 pgs: 148 active+clean; 424 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 53 MiB/s wr, 846 op/s 2024-09-21T11:56:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:47 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:56:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:47 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:56:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1390030708' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:56:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:47 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:56:48.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:48.114 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:48.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:48.116 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:48.118 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:56:48.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:48.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:48.248 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:48.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:48.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:48.388 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:48.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:48.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:48.490 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:48.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:48.523 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:48.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:48.624 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:48.657 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:48.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:48.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:48.683 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 8 16 20553 20537 160.404 145.062 0.00402631 0.00622817 2024-09-21T11:56:48.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: osdmap e397: 8 total, 8 up, 8 in 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: 3.16 scrub starts 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: 3.16 scrub ok 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: 4.b scrub starts 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: 4.b scrub ok 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: 3.6 scrub starts 2024-09-21T11:56:48.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:48 smithi175 ceph-mon[32613]: 3.6 scrub ok 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:119810 9.3 9:c8160ff6:::benchmark_data_smithi043_126002_object119809:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:50:36.892458+0000", 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 371.89138248400002, 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.16747954200000001, 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.790 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 119810 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892458+0000", 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.791 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892458+0000", 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892460+0000", 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7179999999999999e-06 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892523+0000", 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3162000000000003e-05 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.792 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892534+0000", 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1422e-05 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.892541+0000", 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9890000000000003e-06 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.896348+0000", 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0038067639999999998 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.793 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.896494+0000", 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000145592 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,3", 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.896641+0000", 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014765899999999999 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.904387+0000", 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0077451430000000003 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.794 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:36.905317+0000", 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00093077800000000005 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.059807+0000", 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.15448937300000001 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.059867+0000", 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0757000000000001e-05 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.795 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:50:37.059938+0000", 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0185000000000006e-05 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211057 9.e 9:722bc66e:::benchmark_data_smithi043_126002_object83991:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.408880+0000", 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 293.37496085800001, 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.30000521899999999, 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.796 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211057 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408880+0000", 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408880+0000", 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.797 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408881+0000", 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4759999999999999e-06 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408886+0000", 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7199999999999997e-06 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.798 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408895+0000", 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0399999999999998e-06 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408902+0000", 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7490000000000001e-06 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.408968+0000", 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6713999999999999e-05 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409115+0000", 2024-09-21T11:56:48.799 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014610400000000001 2024-09-21T11:56:48.800 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.800 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.800 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:48.800 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409227+0000", 2024-09-21T11:56:48.800 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011280700000000001 2024-09-21T11:56:48.801 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.801 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.801 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415783+0000", 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0065555459999999998 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415844+0000", 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0736000000000003e-05 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.708802+0000", 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.292957984 2024-09-21T11:56:48.802 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.708841+0000", 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8881999999999998e-05 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.708885+0000", 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4461e-05 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.803 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211058 9.d 9:b1ac4faf:::benchmark_data_smithi043_126002_object83992:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.409106+0000", 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 293.37473416699999, 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29279477300000001, 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211058 2024-09-21T11:56:48.804 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409106+0000", 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409106+0000", 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.805 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409107+0000", 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4199999999999995e-07 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409111+0000", 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7239999999999998e-06 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409120+0000", 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7299999999999994e-06 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.806 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409126+0000", 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2840000000000003e-06 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409187+0000", 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1270000000000001e-05 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409347+0000", 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016009800000000001 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.807 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.409487+0000", 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140097 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.419524+0000", 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.010036953 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.419667+0000", 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000142695 2024-09-21T11:56:48.808 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.701808+0000", 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.282140687 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.701849+0000", 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1121000000000002e-05 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.701901+0000", 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2472000000000002e-05 2024-09-21T11:56:48.809 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211061 9.e 9:76133603:::benchmark_data_smithi043_126002_object83995:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.410901+0000", 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 293.37294009999999, 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.298113611, 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:48.810 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211061 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410901+0000", 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410901+0000", 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.811 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410902+0000", 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1880000000000001e-06 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410906+0000", 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.25e-06 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410922+0000", 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5665999999999999e-05 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.812 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410935+0000", 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3487e-05 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411705+0000", 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00076967500000000005 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411820+0000", 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011558 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:48.813 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411944+0000", 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123359 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.416307+0000", 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043633209999999999 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422021+0000", 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00571434 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.814 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.708922+0000", 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.286900975 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.708971+0000", 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8937000000000003e-05 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.709014+0000", 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2833000000000001e-05 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.815 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211064 9.d 9:b03adb7e:::benchmark_data_smithi043_126002_object83998:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.410999+0000", 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 293.37284199300001, 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.30108133799999998, 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211064 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.816 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410999+0000", 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410999+0000", 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.410999+0000", 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8199999999999999e-07 2024-09-21T11:56:48.817 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.818 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411003+0000", 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6859999999999999e-06 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411012+0000", 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.0489999999999993e-06 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411019+0000", 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9539999999999998e-06 2024-09-21T11:56:48.819 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415773+0000", 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0047537710000000004 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415901+0000", 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012853800000000001 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.416029+0000", 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012789999999999999 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.820 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.419685+0000", 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036557690000000001 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.421768+0000", 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002082754 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.711984+0000", 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29021645800000001 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.821 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712036+0000", 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1657999999999999e-05 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712080+0000", 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4119000000000002e-05 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211071 9.e 9:74ef5582:::benchmark_data_smithi043_126002_object84005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.418278+0000", 2024-09-21T11:56:48.822 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 293.36556255900001, 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29953004500000002, 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211071 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418278+0000", 2024-09-21T11:56:48.823 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418278+0000", 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418280+0000", 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5239999999999999e-06 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418284+0000", 2024-09-21T11:56:48.824 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3209999999999998e-06 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418294+0000", 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8709999999999992e-06 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418301+0000", 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.328e-06 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.825 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418410+0000", 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000108491 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418527+0000", 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011742499999999999 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.418644+0000", 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000117092 2024-09-21T11:56:48.826 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.421227+0000", 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002582754 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422100+0000", 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00087292900000000004 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.717710+0000", 2024-09-21T11:56:48.827 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29561067699999999 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.717764+0000", 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3575000000000001e-05 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.717808+0000", 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4057999999999997e-05 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.828 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211076 9.d 9:b6dd861d:::benchmark_data_smithi043_126002_object84010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.422179+0000", 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 293.361661932, 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29006326399999999, 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211076 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.829 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422179+0000", 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422179+0000", 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.830 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422180+0000", 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1400000000000001e-06 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422184+0000", 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0749999999999999e-06 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422192+0000", 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1280000000000008e-06 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.831 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422197+0000", 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5920000000000002e-06 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422236+0000", 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9332000000000002e-05 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422315+0000", 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8918000000000006e-05 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.832 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422403+0000", 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7632000000000004e-05 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428577+0000", 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.006174995 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428852+0000", 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00027460599999999999 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.833 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712126+0000", 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28327408799999998 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712205+0000", 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8461999999999995e-05 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712242+0000", 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7295999999999997e-05 2024-09-21T11:56:48.834 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211077 9.d 9:b5166759:::benchmark_data_smithi043_126002_object84011:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.422212+0000", 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 293.361628145, 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.290211045, 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:48.835 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211077 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422212+0000", 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422212+0000", 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.836 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422213+0000", 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8599999999999998e-07 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422216+0000", 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7860000000000001e-06 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422220+0000", 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9870000000000001e-06 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.837 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422224+0000", 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6190000000000002e-06 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.424411+0000", 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021861010000000002 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.424540+0000", 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000129636 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.838 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.424662+0000", 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000122139 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428585+0000", 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0039231659999999996 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428888+0000", 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030278100000000001 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.839 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712282+0000", 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28339345399999999 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712387+0000", 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000105746 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712424+0000", 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6044000000000003e-05 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.840 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211078 9.d 9:bb97decd:::benchmark_data_smithi043_126002_object84012:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.422243+0000", 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 293.36159756699999, 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29034121800000001, 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211078 2024-09-21T11:56:48.841 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422243+0000", 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422243+0000", 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.842 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422244+0000", 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2900000000000004e-07 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422246+0000", 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.695e-06 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422252+0000", 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4439999999999999e-06 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.843 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.422255+0000", 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3100000000000001e-06 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.426651+0000", 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043957340000000001 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.426730+0000", 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8875999999999995e-05 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,5", 2024-09-21T11:56:48.844 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.426812+0000", 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2638000000000004e-05 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428716+0000", 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019032879999999999 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.428958+0000", 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00024197500000000001 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.845 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712476+0000", 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28351836899999999 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712538+0000", 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2286999999999995e-05 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.712584+0000", 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6072999999999997e-05 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.846 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67182 10.c 10:3edf8765:::benchmark_data_smithi043_150678_object67181:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.333828+0000", 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 200.450012348, 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.20815579300000001, 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67182 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.847 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.333828+0000", 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.333828+0000", 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.333830+0000", 2024-09-21T11:56:48.848 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.875e-06 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.333900+0000", 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.9605999999999997e-05 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.333910+0000", 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0103e-05 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.333916+0000", 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5060000000000001e-06 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.849 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334475+0000", 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00055855900000000003 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334629+0000", 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015380799999999999 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334769+0000", 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140233 2024-09-21T11:56:48.850 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.337276+0000", 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002506552 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338085+0000", 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00080974499999999995 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.541793+0000", 2024-09-21T11:56:48.851 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.20370775099999999 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.541942+0000", 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014904100000000001 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.541984+0000", 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2014000000000001e-05 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.852 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67184 10.1 10:888d6a11:::benchmark_data_smithi043_150678_object67183:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.334274+0000", 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 200.44956692400001, 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.20784313300000001, 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67184 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.853 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334274+0000", 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334274+0000", 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334275+0000", 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0330000000000001e-06 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.854 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334307+0000", 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2320999999999998e-05 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334324+0000", 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6997e-05 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334332+0000", 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3890000000000005e-06 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.855 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334403+0000", 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0251000000000002e-05 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334562+0000", 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00015885300000000001 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.334699+0000", 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000137071 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.856 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.337415+0000", 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0027167910000000001 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.337974+0000", 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00055876899999999998 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.857 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.542037+0000", 2024-09-21T11:56:48.858 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.204063045 2024-09-21T11:56:48.858 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.858 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:48.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.542085+0000", 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7927999999999999e-05 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.542117+0000", 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1684999999999999e-05 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.862 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67200 10.1 10:864f0d67:::benchmark_data_smithi043_150678_object67199:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.347145+0000", 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 200.43669533799999, 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19524280099999999, 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67200 2024-09-21T11:56:48.863 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347145+0000", 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347145+0000", 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347147+0000", 2024-09-21T11:56:48.864 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.934e-06 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347246+0000", 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9166000000000006e-05 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347278+0000", 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1801000000000003e-05 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.865 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347292+0000", 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3529e-05 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347355+0000", 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3355000000000006e-05 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347498+0000", 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014314199999999999 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.866 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:48.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347635+0000", 2024-09-21T11:56:48.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013686899999999999 2024-09-21T11:56:48.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.867 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.349773+0000", 2024-09-21T11:56:48.867 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021376120000000001 2024-09-21T11:56:48.867 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.867 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.867 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351040+0000", 2024-09-21T11:56:48.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012673420000000001 2024-09-21T11:56:48.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.868 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.542298+0000", 2024-09-21T11:56:48.868 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19125804099999999 2024-09-21T11:56:48.868 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.868 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.868 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.542348+0000", 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0328000000000003e-05 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.542388+0000", 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9681999999999997e-05 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72208 10.c 10:34d133c3:::benchmark_data_smithi043_150678_object72207:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:56:48.869 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.968180+0000", 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 195.81566077799999, 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24401014700000001, 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72208 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.870 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968180+0000", 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968180+0000", 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968182+0000", 2024-09-21T11:56:48.871 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.137e-06 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968247+0000", 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5423e-05 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968259+0000", 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2088e-05 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.872 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968271+0000", 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1622e-05 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968339+0000", 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8087e-05 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968487+0000", 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014787700000000001 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.873 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.968634+0000", 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000147408 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.973246+0000", 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0046115619999999996 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.977878+0000", 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0046320240000000002 2024-09-21T11:56:48.874 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.212045+0000", 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23416742400000001 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.212142+0000", 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6385999999999995e-05 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.875 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.212190+0000", 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8109000000000003e-05 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72216 10.c 10:30a9b016:::benchmark_data_smithi043_150678_object72215:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.970152+0000", 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 195.81368853800001, 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24217161800000001, 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.876 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72216 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970152+0000", 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970152+0000", 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.877 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970153+0000", 2024-09-21T11:56:48.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2139999999999999e-06 2024-09-21T11:56:48.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.878 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.878 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.878 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970219+0000", 2024-09-21T11:56:48.878 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5835999999999998e-05 2024-09-21T11:56:48.878 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970233+0000", 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3869999999999999e-05 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970241+0000", 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0139999999999998e-06 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.879 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.971032+0000", 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00079130900000000002 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.971120+0000", 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7843999999999994e-05 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.971218+0000", 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8160999999999997e-05 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.880 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.973391+0000", 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0021727949999999999 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.977924+0000", 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0045324370000000003 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.212250+0000", 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.234326214 2024-09-21T11:56:48.881 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.212298+0000", 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8143000000000003e-05 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.212324+0000", 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5780999999999999e-05 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.882 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72226 10.c 10:3cc42f96:::benchmark_data_smithi043_150678_object72225:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.982531+0000", 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 195.801309833, 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.22990877000000001, 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72226 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.883 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.982531+0000", 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.982531+0000", 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.982533+0000", 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1550000000000001e-06 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.884 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.982582+0000", 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8609000000000002e-05 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.982596+0000", 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4493e-05 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.982604+0000", 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4419999999999998e-06 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.885 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.982657+0000", 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2213000000000003e-05 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.982748+0000", 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1025000000000003e-05 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.982828+0000", 2024-09-21T11:56:48.886 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.0452999999999997e-05 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.984759+0000", 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019311599999999999 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.986242+0000", 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001482188 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.212362+0000", 2024-09-21T11:56:48.887 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.22612083599999999 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.212408+0000", 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5705999999999999e-05 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.212440+0000", 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.1489999999999998e-05 2024-09-21T11:56:48.888 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:85500 10.c 10:31f0f4f1:::benchmark_data_smithi043_150678_object85499:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.793963+0000", 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 183.98987735599999, 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28503999200000002, 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.889 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 85500 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.793963+0000", 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.793963+0000", 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.890 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.793970+0000", 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6660000000000002e-06 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.794013+0000", 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3442999999999997e-05 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.794030+0000", 2024-09-21T11:56:48.891 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6444e-05 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.794040+0000", 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0516000000000001e-05 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.794108+0000", 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7417999999999997e-05 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.068881+0000", 2024-09-21T11:56:48.892 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27477348499999998 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.069047+0000", 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000165756 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.078823+0000", 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097760610000000008 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.078838+0000", 2024-09-21T11:56:48.893 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.491e-05 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.078890+0000", 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2138999999999997e-05 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.078958+0000", 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7689999999999997e-05 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.894 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.079003+0000", 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5463999999999997e-05 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:85498 10.8 e336/313 10:1841726b:::benchmark_data_smithi043_150678_object85497:head v 336'5296, mlcod=336'5294)", 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.794300+0000", 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 183.989540568, 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27969625799999998, 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.895 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.794300+0000", 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.794300+0000", 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.794301+0000", 2024-09-21T11:56:48.896 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.7199999999999997e-07 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.794358+0000", 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7364999999999997e-05 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.068670+0000", 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27431190999999999 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.897 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.068686+0000", 2024-09-21T11:56:48.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5707000000000001e-05 2024-09-21T11:56:48.898 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.898 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.898 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.898 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.068805+0000", 2024-09-21T11:56:48.898 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000119064 2024-09-21T11:56:48.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.068831+0000", 2024-09-21T11:56:48.899 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.5466999999999999e-05 2024-09-21T11:56:48.899 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.899 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.899 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.899 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.073957+0000", 2024-09-21T11:56:48.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0051265720000000002 2024-09-21T11:56:48.900 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.900 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.900 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.900 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.073996+0000", 2024-09-21T11:56:48.900 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9201e-05 2024-09-21T11:56:48.900 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.900 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.901 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.901 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.901 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.901 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:85555 10.9 e336/313 10:9e997645:::benchmark_data_smithi043_150678_object85554:head v 336'5442, mlcod=336'5440)", 2024-09-21T11:56:48.901 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.838235+0000", 2024-09-21T11:56:48.901 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 183.94560547, 2024-09-21T11:56:48.901 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24360511300000001, 2024-09-21T11:56:48.901 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.901 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.838235+0000", 2024-09-21T11:56:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.902 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.838235+0000", 2024-09-21T11:56:48.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.903 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.903 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.903 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.838237+0000", 2024-09-21T11:56:48.903 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7940000000000001e-06 2024-09-21T11:56:48.903 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.838270+0000", 2024-09-21T11:56:48.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3550000000000002e-05 2024-09-21T11:56:48.904 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.904 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.904 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.904 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.068674+0000", 2024-09-21T11:56:48.904 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.230403622 2024-09-21T11:56:48.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.905 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.905 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.068686+0000", 2024-09-21T11:56:48.905 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2179e-05 2024-09-21T11:56:48.905 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.905 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.068877+0000", 2024-09-21T11:56:48.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019037200000000001 2024-09-21T11:56:48.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.906 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.906 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.906 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.068894+0000", 2024-09-21T11:56:48.906 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7808999999999998e-05 2024-09-21T11:56:48.906 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.081804+0000", 2024-09-21T11:56:48.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012909039000000001 2024-09-21T11:56:48.907 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.907 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.907 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.907 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.081840+0000", 2024-09-21T11:56:48.907 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6748000000000001e-05 2024-09-21T11:56:48.908 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.908 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.908 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.908 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.908 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.908 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:97306 10.9 e339/313 10:965ea9a6:::benchmark_data_smithi043_150678_object97305:head v 339'6197, mlcod=339'6195)", 2024-09-21T11:56:48.908 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:56.829206+0000", 2024-09-21T11:56:48.909 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 171.954634314, 2024-09-21T11:56:48.909 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.162669707, 2024-09-21T11:56:48.909 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.909 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.909 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.909 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.909 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.909 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.909 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.829206+0000", 2024-09-21T11:56:48.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.910 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.910 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.829206+0000", 2024-09-21T11:56:48.910 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.910 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.910 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.829207+0000", 2024-09-21T11:56:48.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5300000000000003e-07 2024-09-21T11:56:48.911 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.911 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.911 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.911 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.829234+0000", 2024-09-21T11:56:48.911 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7163000000000001e-05 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.829245+0000", 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0485e-05 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.829252+0000", 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0550000000000002e-06 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.912 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.829308+0000", 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6455000000000001e-05 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.829321+0000", 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2971e-05 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.991824+0000", 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.16250298499999999 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.913 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:56.991876+0000", 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1839999999999998e-05 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:97359 10.1 10:8ade32fa:::benchmark_data_smithi043_150678_object97358:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e339)", 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:57.006945+0000", 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 171.77689530399999, 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.16278736599999999, 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:56:48.914 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 97359 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.006945+0000", 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.915 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.006945+0000", 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.006947+0000", 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9180000000000001e-06 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.006980+0000", 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3158999999999999e-05 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.916 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.006998+0000", 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7861000000000001e-05 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.007007+0000", 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2979999999999992e-06 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.007091+0000", 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.4154999999999996e-05 2024-09-21T11:56:48.917 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.007216+0000", 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012549300000000001 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.007354+0000", 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000138162 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:56:48.918 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.010558+0000", 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003203502 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.011019+0000", 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00046119899999999997 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.169567+0000", 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.158547471 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:56:48.919 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.169678+0000", 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111783 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:57.169733+0000", 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4364999999999999e-05 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:56:48.920 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:56:48.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:56:48.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:56:48.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:48.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:56:48.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: osdmap e397: 8 total, 8 up, 8 in 2024-09-21T11:56:48.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: 3.16 scrub starts 2024-09-21T11:56:48.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: 3.16 scrub ok 2024-09-21T11:56:48.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: 4.b scrub starts 2024-09-21T11:56:48.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: 4.b scrub ok 2024-09-21T11:56:48.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: 3.6 scrub starts 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[29710]: 3.6 scrub ok 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: osdmap e397: 8 total, 8 up, 8 in 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: 3.16 scrub starts 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: 3.16 scrub ok 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: 4.b scrub starts 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: 4.b scrub ok 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: 3.6 scrub starts 2024-09-21T11:56:48.922 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:48 smithi043 ceph-mon[36454]: 3.6 scrub ok 2024-09-21T11:56:48.961 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:48.991 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:48.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:48.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:49.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:49.136 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:49.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:49.137 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:49.238 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:49.269 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:49.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:49.270 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:49.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:49.399 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:49.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:49.400 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:49.501 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:49.532 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:49.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:49.533 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:49.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:49.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:49.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:49.683 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 9 16 22733 22717 157.719 136.25 0.00384912 0.00633358 2024-09-21T11:56:49.773 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:49.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:56:49.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:49.815 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:49.815 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:49.830 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 1 pgp_num to 1 2024-09-21T11:56:49.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:56:49.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:49.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:49 smithi043 ceph-mon[29710]: pgmap v816: 148 pgs: 148 active+clean; 924 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-21T11:56:49.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:49 smithi043 ceph-mon[29710]: 5.e scrub starts 2024-09-21T11:56:49.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:49 smithi043 ceph-mon[29710]: 5.e scrub ok 2024-09-21T11:56:49.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1328430223' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:56:49.941 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:49 smithi043 ceph-mon[36454]: pgmap v816: 148 pgs: 148 active+clean; 924 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-21T11:56:49.941 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:49 smithi043 ceph-mon[36454]: 5.e scrub starts 2024-09-21T11:56:49.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:49 smithi043 ceph-mon[36454]: 5.e scrub ok 2024-09-21T11:56:49.942 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1328430223' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:56:49.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:49.994 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:50.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:49 smithi175 ceph-mon[32613]: pgmap v816: 148 pgs: 148 active+clean; 924 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-21T11:56:50.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:49 smithi175 ceph-mon[32613]: 5.e scrub starts 2024-09-21T11:56:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:49 smithi175 ceph-mon[32613]: 5.e scrub ok 2024-09-21T11:56:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1328430223' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"} : dispatch 2024-09-21T11:56:50.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:50.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:50.162 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:50.162 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:50.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:50.295 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:50.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:50.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:50.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:50.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:50.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:50.428 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:50.529 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:50.563 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:50.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:50.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:50.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:56:50.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:50.685 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 10 16 24870 24854 155.302 133.562 0.00804416 0.00643149 2024-09-21T11:56:50.751 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:50.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:50.752 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:50.828 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:50.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:50.881 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:50.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:50.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:50.891 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:50.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:50 smithi043 ceph-mon[29710]: 6.1b scrub starts 2024-09-21T11:56:50.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:50 smithi043 ceph-mon[29710]: 6.1b scrub ok 2024-09-21T11:56:50.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1328430223' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:56:50.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:50 smithi043 ceph-mon[29710]: osdmap e398: 8 total, 8 up, 8 in 2024-09-21T11:56:50.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:50 smithi043 ceph-mon[29710]: pgmap v818: 148 pgs: 148 active+clean; 924 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-21T11:56:50.978 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:50 smithi043 ceph-mon[36454]: 6.1b scrub starts 2024-09-21T11:56:50.978 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:50 smithi043 ceph-mon[36454]: 6.1b scrub ok 2024-09-21T11:56:50.978 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1328430223' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:56:50.978 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:50 smithi043 ceph-mon[36454]: osdmap e398: 8 total, 8 up, 8 in 2024-09-21T11:56:50.978 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:50 smithi043 ceph-mon[36454]: pgmap v818: 148 pgs: 148 active+clean; 924 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-21T11:56:50.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:51.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:51.032 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:51.032 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:51.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:51.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:51.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:51.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:50 smithi175 ceph-mon[32613]: 6.1b scrub starts 2024-09-21T11:56:51.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:50 smithi175 ceph-mon[32613]: 6.1b scrub ok 2024-09-21T11:56:51.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1328430223' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "1"}]': finished 2024-09-21T11:56:51.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:50 smithi175 ceph-mon[32613]: osdmap e398: 8 total, 8 up, 8 in 2024-09-21T11:56:51.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:50 smithi175 ceph-mon[32613]: pgmap v818: 148 pgs: 148 active+clean; 924 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-21T11:56:51.283 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:51.313 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:51.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:51.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:51.414 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:51.447 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:51.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:51.448 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:51.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:51.576 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:51.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:51.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:51.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:51.683 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 11 15 26735 26720 151.786 116.625 0.00634208 0.0065813 2024-09-21T11:56:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:51.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:51.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:51.771 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:56:51.771 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:56:51.805 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:56:51.805 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:56:51.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:51.844 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:56:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:56:51.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:51.846 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:51.846 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:51.890 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:56:51.891 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:56:51.936 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:56:51.936 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:56:51.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:51.975 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:56:51.975 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:56:51.978 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:56:51.978 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:56:52.007 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:52.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:52.008 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:56:52.012 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:56:52.049 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:56:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:56:52.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:52.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:52.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[36454]: 3.d scrub starts 2024-09-21T11:56:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[36454]: 3.d scrub ok 2024-09-21T11:56:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[36454]: 5.17 scrub starts 2024-09-21T11:56:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[36454]: 5.17 scrub ok 2024-09-21T11:56:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T11:56:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T11:56:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2071757871' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[29710]: 3.d scrub starts 2024-09-21T11:56:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[29710]: 3.d scrub ok 2024-09-21T11:56:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[29710]: 5.17 scrub starts 2024-09-21T11:56:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[29710]: 5.17 scrub ok 2024-09-21T11:56:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T11:56:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T11:56:52.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:51 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2071757871' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:52.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:52.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:51 smithi175 ceph-mon[32613]: 3.d scrub starts 2024-09-21T11:56:52.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:51 smithi175 ceph-mon[32613]: 3.d scrub ok 2024-09-21T11:56:52.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:51 smithi175 ceph-mon[32613]: 5.17 scrub starts 2024-09-21T11:56:52.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:51 smithi175 ceph-mon[32613]: 5.17 scrub ok 2024-09-21T11:56:52.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:51 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T11:56:52.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:51 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T11:56:52.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:51 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2071757871' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:56:52.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:56:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:56:52.297 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:52.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:52.299 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:52.399 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:52.431 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:52.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:52.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:52.526 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:56:52.533 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:52.564 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:52.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:52.565 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:52.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:52.683 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 12 15 28758 28743 149.673 126.438 0.00958734 0.00667474 2024-09-21T11:56:52.700 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:52.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:52.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:52.802 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:52.822 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:52.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:52.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:52.924 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:52.957 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:52.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:52.958 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:53.059 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:53.090 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:53.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:53.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:53.192 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:53.221 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:53.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:53.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:52 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:53.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:52 smithi043 ceph-mon[29710]: pgmap v819: 148 pgs: 148 active+clean; 1.6 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 189 MiB/s wr, 3.03k op/s 2024-09-21T11:56:53.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:52 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:53.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:52 smithi043 ceph-mon[36454]: pgmap v819: 148 pgs: 148 active+clean; 1.6 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 189 MiB/s wr, 3.03k op/s 2024-09-21T11:56:53.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:52 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:56:53.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:52 smithi175 ceph-mon[32613]: pgmap v819: 148 pgs: 148 active+clean; 1.6 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 189 MiB/s wr, 3.03k op/s 2024-09-21T11:56:53.324 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:53.354 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:53.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:53.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:53.455 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:53.485 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:53.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:53.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:53.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:53.626 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:53.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:53.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:53.683 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 13 16 30610 30594 147.058 115.688 0.00952729 0.0067935 2024-09-21T11:56:53.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:53.757 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:53.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:53.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:53.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:53.897 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:53.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:53.898 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:53.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:54.021 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:54.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:54.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:54.122 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:54.155 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:54.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:54.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:54.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:53 smithi043 ceph-mon[29710]: 3.17 scrub starts 2024-09-21T11:56:54.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:53 smithi043 ceph-mon[29710]: 3.17 scrub ok 2024-09-21T11:56:54.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:53 smithi043 ceph-mon[29710]: 3.7 scrub starts 2024-09-21T11:56:54.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:53 smithi043 ceph-mon[29710]: 3.7 scrub ok 2024-09-21T11:56:54.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:53 smithi043 ceph-mon[36454]: 3.17 scrub starts 2024-09-21T11:56:54.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:53 smithi043 ceph-mon[36454]: 3.17 scrub ok 2024-09-21T11:56:54.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:53 smithi043 ceph-mon[36454]: 3.7 scrub starts 2024-09-21T11:56:54.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:53 smithi043 ceph-mon[36454]: 3.7 scrub ok 2024-09-21T11:56:54.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:54.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:53 smithi175 ceph-mon[32613]: 3.17 scrub starts 2024-09-21T11:56:54.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:53 smithi175 ceph-mon[32613]: 3.17 scrub ok 2024-09-21T11:56:54.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:53 smithi175 ceph-mon[32613]: 3.7 scrub starts 2024-09-21T11:56:54.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:53 smithi175 ceph-mon[32613]: 3.7 scrub ok 2024-09-21T11:56:54.295 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:54.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:54.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:54.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:54.428 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:54.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:54.429 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:54.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:54.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:54.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:54.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:54.683 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 14 16 32375 32359 144.433 110.312 0.0105325 0.00691806 2024-09-21T11:56:54.700 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:54.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:54.702 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:54.802 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:54.836 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:54.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:54.837 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:54.937 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:54.965 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:54.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:54.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:55.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:55.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:55.105 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:55.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:55.206 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:54 smithi043 ceph-mon[36454]: pgmap v820: 148 pgs: 148 active+clean; 1.6 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 149 MiB/s wr, 2.39k op/s 2024-09-21T11:56:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:54 smithi043 ceph-mon[29710]: pgmap v820: 148 pgs: 148 active+clean; 1.6 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 149 MiB/s wr, 2.39k op/s 2024-09-21T11:56:55.237 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:55.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:55.238 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:55.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:54 smithi175 ceph-mon[32613]: pgmap v820: 148 pgs: 148 active+clean; 1.6 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 149 MiB/s wr, 2.39k op/s 2024-09-21T11:56:55.339 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:55.360 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:55.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:55.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:55.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:56:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:56:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:56:55.492 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:55.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:55.493 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:55.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:55.620 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:55.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:55.621 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:55.683 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 15 16 34041 34025 141.745 104.125 0.00205072 0.00703131 2024-09-21T11:56:55.721 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:56:55.721 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:56:55.721 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-21T11:56:55.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-21T11:56:55.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:55.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:55.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:55.765 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:55.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:55.894 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:55.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:55.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:55.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:56.020 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:56.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:56.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:56.047 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:55 smithi043 ceph-mon[36454]: 6.8 scrub starts 2024-09-21T11:56:56.048 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:56 smithi043 ceph-mon[36454]: 6.8 scrub ok 2024-09-21T11:56:56.048 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:55 smithi043 ceph-mon[29710]: 6.8 scrub starts 2024-09-21T11:56:56.048 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:56 smithi043 ceph-mon[29710]: 6.8 scrub ok 2024-09-21T11:56:56.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:56.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:56.165 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:56.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:56.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:55 smithi175 ceph-mon[32613]: 6.8 scrub starts 2024-09-21T11:56:56.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:55 smithi175 ceph-mon[32613]: 6.8 scrub ok 2024-09-21T11:56:56.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:56.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:56.295 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:56.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:56.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:56.439 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:56.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:56.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:56.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:56.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:56.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:56.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:56.683 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 16 16 35815 35799 139.813 110.875 0.0116611 0.00714636 2024-09-21T11:56:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:56.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:56.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:56.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:56.843 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:56.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:56.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:56.945 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:56.977 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:56.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:56.978 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:57.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:57.111 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:57.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:57.113 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:57.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:57.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:56 smithi043 ceph-mon[29710]: 5.15 scrub starts 2024-09-21T11:56:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:56 smithi043 ceph-mon[29710]: 5.15 scrub ok 2024-09-21T11:56:57.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:56 smithi043 ceph-mon[29710]: pgmap v821: 148 pgs: 148 active+clean; 1.8 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 616 B/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-21T11:56:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:56 smithi043 ceph-mon[36454]: 5.15 scrub starts 2024-09-21T11:56:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:56 smithi043 ceph-mon[36454]: 5.15 scrub ok 2024-09-21T11:56:57.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:56 smithi043 ceph-mon[36454]: pgmap v821: 148 pgs: 148 active+clean; 1.8 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 616 B/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-21T11:56:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:57.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:57.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:57.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:56 smithi175 ceph-mon[32613]: 5.15 scrub starts 2024-09-21T11:56:57.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:56 smithi175 ceph-mon[32613]: 5.15 scrub ok 2024-09-21T11:56:57.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:56 smithi175 ceph-mon[32613]: pgmap v821: 148 pgs: 148 active+clean; 1.8 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 616 B/s rd, 110 MiB/s wr, 1.75k op/s 2024-09-21T11:56:57.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:57.388 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:57.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:57.389 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:57.490 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:57.519 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:57.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:57.520 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:57.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:57.644 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:57.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:57.646 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:57.684 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 17 16 37450 37434 137.599 102.188 0.00941319 0.0072609 2024-09-21T11:56:57.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:57.783 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:57.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:57.784 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:57.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:57.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:57.907 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:57.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:58.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:58.039 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:58.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:58.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:58.141 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:56:58.174 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:58.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:58.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:56:58.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:58 smithi175 ceph-mon[32613]: 5.7 scrub starts 2024-09-21T11:56:58.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:58 smithi175 ceph-mon[32613]: 5.7 scrub ok 2024-09-21T11:56:58.276 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:56:58.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:56:58.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:58.378 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:58.378 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:56:58.416 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:58 smithi043 ceph-mon[36454]: 5.7 scrub starts 2024-09-21T11:56:58.417 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:58 smithi043 ceph-mon[36454]: 5.7 scrub ok 2024-09-21T11:56:58.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:58 smithi043 ceph-mon[29710]: 5.7 scrub starts 2024-09-21T11:56:58.417 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:58 smithi043 ceph-mon[29710]: 5.7 scrub ok 2024-09-21T11:56:58.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:58.529 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:58.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:58.531 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:58.631 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:58.678 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:58.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:58.679 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:58.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 18 16 38986 38970 135.273 96 0.00669698 0.00738684 2024-09-21T11:56:58.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:56:58.801 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:58.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:58.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:56:58.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:58.932 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:58.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:58.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:59.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:59.076 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:59.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:59.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:59.178 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:59.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:59.221 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:59 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:56:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:59 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:56:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:59 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T11:56:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:59 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T11:56:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:59 smithi175 ceph-mon[32613]: pgmap v822: 148 pgs: 148 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-21T11:56:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:59 smithi175 ceph-mon[32613]: 6.16 scrub starts 2024-09-21T11:56:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:59 smithi175 ceph-mon[32613]: 6.16 scrub ok 2024-09-21T11:56:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:59 smithi175 ceph-mon[32613]: 3.4 scrub starts 2024-09-21T11:56:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:59 smithi175 ceph-mon[32613]: 3.4 scrub ok 2024-09-21T11:56:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:56:59 smithi175 ceph-mon[32613]: osdmap e399: 8 total, 8 up, 8 in 2024-09-21T11:56:59.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:56:59.360 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:59.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:59.361 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:56:59.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:56:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:56:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:56:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T11:56:59.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[29710]: pgmap v822: 148 pgs: 148 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[29710]: 6.16 scrub starts 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[29710]: 6.16 scrub ok 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[29710]: 3.4 scrub starts 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[29710]: 3.4 scrub ok 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[29710]: osdmap e399: 8 total, 8 up, 8 in 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T11:56:59.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[36454]: pgmap v822: 148 pgs: 148 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-21T11:56:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[36454]: 6.16 scrub starts 2024-09-21T11:56:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[36454]: 6.16 scrub ok 2024-09-21T11:56:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[36454]: 3.4 scrub starts 2024-09-21T11:56:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[36454]: 3.4 scrub ok 2024-09-21T11:56:59.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:56:59 smithi043 ceph-mon[36454]: osdmap e399: 8 total, 8 up, 8 in 2024-09-21T11:56:59.495 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:59.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:59.497 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:56:59.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:56:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:59.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:59.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:56:59.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 19 16 40437 40421 132.926 90.6875 0.0187623 0.00751693 2024-09-21T11:56:59.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:56:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:56:59.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:59.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:56:59.860 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:56:59.894 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:56:59.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:56:59.895 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:56:59.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:00.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:00.039 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:00.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:00.177 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:00.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:00.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:00.280 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:00.311 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:00.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:00.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:00.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:00.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:00.447 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:00 smithi043 ceph-mon[36454]: 4.1b scrub starts 2024-09-21T11:57:00.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:00 smithi043 ceph-mon[36454]: 4.1b scrub ok 2024-09-21T11:57:00.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:00 smithi043 ceph-mon[29710]: 4.1b scrub starts 2024-09-21T11:57:00.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:00 smithi043 ceph-mon[29710]: 4.1b scrub ok 2024-09-21T11:57:00.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:00 smithi175 ceph-mon[32613]: 4.1b scrub starts 2024-09-21T11:57:00.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:00 smithi175 ceph-mon[32613]: 4.1b scrub ok 2024-09-21T11:57:00.548 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:00.580 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:00.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:00.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:00.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:00.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:57:00.685548+0000 min lat: 0.00120284 max lat: 0.171457 avg lat: 0.00758645 2024-09-21T11:57:00.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:57:00.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 20 15 42155 42140 131.651 107.438 0.00273625 0.00758645 2024-09-21T11:57:00.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:00.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:00.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:00.834 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:00.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:00.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:00.936 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:00.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:00.972 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:00.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:01.073 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:01.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:01.108 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:01.108 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:01.209 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:01.241 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:01.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:01.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:01.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:01.384 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:01.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:01.386 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:01.487 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:01 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T11:57:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:01 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T11:57:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:01 smithi043 ceph-mon[36454]: pgmap v824: 148 pgs: 148 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-21T11:57:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:01 smithi043 ceph-mon[36454]: 4.0 scrub starts 2024-09-21T11:57:01.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:01 smithi043 ceph-mon[36454]: 4.0 scrub ok 2024-09-21T11:57:01.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:01 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T11:57:01.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:01 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T11:57:01.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:01 smithi043 ceph-mon[29710]: pgmap v824: 148 pgs: 148 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-21T11:57:01.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:01 smithi043 ceph-mon[29710]: 4.0 scrub starts 2024-09-21T11:57:01.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:01 smithi043 ceph-mon[29710]: 4.0 scrub ok 2024-09-21T11:57:01.509 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:01.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:01.510 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:01.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:01 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T11:57:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:01 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T11:57:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:01 smithi175 ceph-mon[32613]: pgmap v824: 148 pgs: 148 active+clean; 2.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 126 MiB/s wr, 2.02k op/s 2024-09-21T11:57:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:01 smithi175 ceph-mon[32613]: 4.0 scrub starts 2024-09-21T11:57:01.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:01 smithi175 ceph-mon[32613]: 4.0 scrub ok 2024-09-21T11:57:01.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:01.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:01.653 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:01.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 21 16 43587 43571 129.641 89.4375 0.00831485 0.00770796 2024-09-21T11:57:01.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:01.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:57:01.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:01.823 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:01.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:01.924 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:01.948 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:01.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:01.950 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:02.050 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:02.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:02.080 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:02.080 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:02.172 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:02 smithi043 ceph-mon[36454]: 3.1 scrub starts 2024-09-21T11:57:02.172 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:02 smithi043 ceph-mon[36454]: 3.1 scrub ok 2024-09-21T11:57:02.172 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:02 smithi043 ceph-mon[36454]: 3.f scrub starts 2024-09-21T11:57:02.172 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:02 smithi043 ceph-mon[36454]: 3.f scrub ok 2024-09-21T11:57:02.172 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:02 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:02.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:02 smithi043 ceph-mon[29710]: 3.1 scrub starts 2024-09-21T11:57:02.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:02 smithi043 ceph-mon[29710]: 3.1 scrub ok 2024-09-21T11:57:02.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:02 smithi043 ceph-mon[29710]: 3.f scrub starts 2024-09-21T11:57:02.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:02 smithi043 ceph-mon[29710]: 3.f scrub ok 2024-09-21T11:57:02.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:02 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:02.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:02.245 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:02.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:02.247 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:02.249 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:02.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:02 smithi175 ceph-mon[32613]: 3.1 scrub starts 2024-09-21T11:57:02.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:02 smithi175 ceph-mon[32613]: 3.1 scrub ok 2024-09-21T11:57:02.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:02 smithi175 ceph-mon[32613]: 3.f scrub starts 2024-09-21T11:57:02.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:02 smithi175 ceph-mon[32613]: 3.f scrub ok 2024-09-21T11:57:02.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:02 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:02.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:57:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:57:02.288 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:02.348 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:02.383 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:02.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:02.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:02.486 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:02.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:02.515 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:02.515 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:02.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:02.643 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:02.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:02.644 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:02.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 22 16 44943 44927 127.601 84.75 0.0186687 0.00783045 2024-09-21T11:57:02.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:02.795 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:02.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:02.796 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:02.897 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:02.925 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:02.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:02.926 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:02.930 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:02.950 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:57:02.950 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:57:02.988 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:57:02.989 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:57:03.026 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:57:03.027 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:57:03.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:03.045 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:57:03.078 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:57:03.078 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:57:03.081 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:03.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:03.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:03.117 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:57:03.117 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:57:03.156 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:57:03.156 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:57:03.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:03.203 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:57:03.203 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:57:03.224 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:03.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:03.226 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:03.238 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:57:03.238 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:57:03.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:03.340 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:03 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:03.340 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:03 smithi043 ceph-mon[29710]: pgmap v825: 148 pgs: 148 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-21T11:57:03.340 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:03 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2324168350' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:03.340 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:03 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3935913773' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:03.340 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:03 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:03.340 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:03 smithi043 ceph-mon[36454]: pgmap v825: 148 pgs: 148 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-21T11:57:03.341 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:03 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2324168350' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:03.341 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:03 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3935913773' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:03.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:03.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:03.470 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:03.504 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:03.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:03.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:03.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:03 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:03.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:03 smithi175 ceph-mon[32613]: pgmap v825: 148 pgs: 148 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-21T11:57:03.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:03 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2324168350' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:03.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:03 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3935913773' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:03.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:03.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:03.644 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:03.644 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:03.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 23 16 46489 46473 126.254 96.625 0.00982254 0.00791476 2024-09-21T11:57:03.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:03.752 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:57:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:57:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:57:03.752 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:57:03.752 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:57:03.786 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:03.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:03.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:03.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:03.931 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:03.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:03.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:04.034 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:04.064 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:04.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:04.066 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:04.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:04.209 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:04.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:04.210 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:04.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:04.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:04 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:57:04.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:04 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:57:04.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:04 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2324168350' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:57:04.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:04 smithi043 ceph-mon[36454]: osdmap e400: 8 total, 8 up, 8 in 2024-09-21T11:57:04.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:04 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:57:04.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:04 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:57:04.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:04 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2324168350' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:57:04.319 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:04 smithi043 ceph-mon[29710]: osdmap e400: 8 total, 8 up, 8 in 2024-09-21T11:57:04.342 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:04.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:04.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:04.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:04.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:04.489 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:04.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:04.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:04 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:57:04.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:04 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:57:04.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:04 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2324168350' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:57:04.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:04 smithi175 ceph-mon[32613]: osdmap e400: 8 total, 8 up, 8 in 2024-09-21T11:57:04.589 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:04.621 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:04.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:04.622 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:04.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 24 15 48190 48175 125.425 106.375 0.00569109 0.00796756 2024-09-21T11:57:04.723 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:04.750 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:04.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:04.751 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:04.852 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:04.883 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:04.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:04.884 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:04.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:05.028 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:05.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:05.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:05.129 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:05.161 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:05.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:05.162 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:05.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:05.294 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:05.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:05.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:05.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:05.436 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:05.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:05.438 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:57:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:57:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:57:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[36454]: osdmap e401: 8 total, 8 up, 8 in 2024-09-21T11:57:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[36454]: pgmap v828: 148 pgs: 148 active+clean; 2.6 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-21T11:57:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[36454]: 6.15 scrub starts 2024-09-21T11:57:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[36454]: 6.15 scrub ok 2024-09-21T11:57:05.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[36454]: 6.1f scrub starts 2024-09-21T11:57:05.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[36454]: 6.1f scrub ok 2024-09-21T11:57:05.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:57:05.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:57:05.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[29710]: osdmap e401: 8 total, 8 up, 8 in 2024-09-21T11:57:05.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[29710]: pgmap v828: 148 pgs: 148 active+clean; 2.6 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-21T11:57:05.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[29710]: 6.15 scrub starts 2024-09-21T11:57:05.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[29710]: 6.15 scrub ok 2024-09-21T11:57:05.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[29710]: 6.1f scrub starts 2024-09-21T11:57:05.494 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:05 smithi043 ceph-mon[29710]: 6.1f scrub ok 2024-09-21T11:57:05.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:05 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:57:05.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:05 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:57:05.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:05 smithi175 ceph-mon[32613]: osdmap e401: 8 total, 8 up, 8 in 2024-09-21T11:57:05.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:05 smithi175 ceph-mon[32613]: pgmap v828: 148 pgs: 148 active+clean; 2.6 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-21T11:57:05.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:05 smithi175 ceph-mon[32613]: 6.15 scrub starts 2024-09-21T11:57:05.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:05 smithi175 ceph-mon[32613]: 6.15 scrub ok 2024-09-21T11:57:05.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:05 smithi175 ceph-mon[32613]: 6.1f scrub starts 2024-09-21T11:57:05.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:05 smithi175 ceph-mon[32613]: 6.1f scrub ok 2024-09-21T11:57:05.538 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:05.570 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:05.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:05.571 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:05.671 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:05.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 25 16 49808 49792 124.45 101.062 0.0268078 0.00802999 2024-09-21T11:57:05.700 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:05.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:05.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:05.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:57:05.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:05.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:05.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:05.953 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:05.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:05.982 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:05.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:06.083 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:06.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:06.128 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:06.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:06.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:06.279 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:06.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:06.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:06.385 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:06 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:57:06.385 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:06 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:57:06.385 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:06 smithi043 ceph-mon[36454]: osdmap e402: 8 total, 8 up, 8 in 2024-09-21T11:57:06.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:06 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:57:06.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:06 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:57:06.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:06 smithi043 ceph-mon[29710]: osdmap e402: 8 total, 8 up, 8 in 2024-09-21T11:57:06.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:06.413 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:06.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:06 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:57:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:06 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:57:06.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:06 smithi175 ceph-mon[32613]: osdmap e402: 8 total, 8 up, 8 in 2024-09-21T11:57:06.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:06.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:06.556 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:06.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:06.657 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:06.686 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 26 16 51254 51238 123.14 90.375 0.00759536 0.00811532 2024-09-21T11:57:06.694 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:06.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:06.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:06.710 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:57:06.710 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:57:06.710 INFO:teuthology.orchestra.run.smithi043.stdout: "epoch": 402, 2024-09-21T11:57:06.710 INFO:teuthology.orchestra.run.smithi043.stdout: "fsid": "4f2c5252-780d-11ef-baf6-efdc52797490", 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "created": "2024-09-21T11:34:07.249195+0000", 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "modified": "2024-09-21T11:57:05.047116+0000", 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "last_up_change": "2024-09-21T11:39:49.206441+0000", 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "last_in_change": "2024-09-21T11:47:02.337266+0000", 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_num": 5799936, 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_set": [ 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "pglog_hardlimit", 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "purged_snapdirs", 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_deletes", 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "sortbitwise" 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_version": 18, 2024-09-21T11:57:06.711 INFO:teuthology.orchestra.run.smithi043.stdout: "full_ratio": 0.94999998807907104, 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_snapshot": "", 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_max": 11, 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "max_osd": 8, 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "require_min_compat_client": "luminous", 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "min_compat_client": "luminous", 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "require_osd_release": "squid", 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "allow_crimson": false, 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "pools": [ 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 1, 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".mgr", 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:37:26.023430+0000", 2024-09-21T11:57:06.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 1, 2024-09-21T11:57:06.713 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 1, 2024-09-21T11:57:06.714 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 1, 2024-09-21T11:57:06.714 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 1, 2024-09-21T11:57:06.714 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 1, 2024-09-21T11:57:06.714 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:57:06.714 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:57:06.714 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:57:06.714 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:57:06.714 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:57:06.714 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:57:06.714 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:57:06.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.715 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "398", 2024-09-21T11:57:06.715 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:57:06.715 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:57:06.715 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:57:06.715 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:57:06.715 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:57:06.715 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:57:06.715 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:57:06.715 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:57:06.716 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:57:06.717 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:57:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:57:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:57:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:57:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:57:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:57:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:57:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_max": 32, 2024-09-21T11:57:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_min": 1 2024-09-21T11:57:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.718 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": {} 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 6.9800000190734863, 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 6.9800000190734863, 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 0.43000000715255737, 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 3, 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 3, 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 2, 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "datapool", 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:58.262982+0000", 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 8193, 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:57:06.720 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 3, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 3, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 3, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 3, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 3, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:57:06.721 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "61", 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 2, 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 61, 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:57:06.722 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:57:06.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:57:06.724 INFO:teuthology.orchestra.run.smithi043.stdout: "rbd": {} 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.3299999237060547, 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.3299999237060547, 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1, 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.725 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 3, 2024-09-21T11:57:06.726 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": ".rgw.root", 2024-09-21T11:57:06.726 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:39:59.229530+0000", 2024-09-21T11:57:06.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:57:06.726 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:57:06.726 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:57:06.726 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:57:06.726 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:57:06.726 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:57:06.727 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:57:06.727 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:57:06.727 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:57:06.727 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:57:06.727 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:57:06.727 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:57:06.727 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:57:06.727 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:57:06.727 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:57:06.728 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:57:06.728 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:57:06.728 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:57:06.728 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:57:06.728 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:57:06.728 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:57:06.728 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:57:06.728 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:57:06.728 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:57:06.729 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:57:06.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.729 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "60", 2024-09-21T11:57:06.729 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:57:06.729 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:57:06.729 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:57:06.729 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:57:06.729 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:57:06.729 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:57:06.730 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:57:06.730 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:57:06.730 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:57:06.730 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:57:06.730 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:57:06.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:57:06.730 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:57:06.730 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:57:06.731 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:57:06.731 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:57:06.731 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:57:06.731 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:57:06.731 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:57:06.731 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:57:06.731 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:57:06.731 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:57:06.732 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:57:06.732 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:57:06.732 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:57:06.732 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:57:06.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.732 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:57:06.732 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:57:06.732 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:57:06.732 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:57:06.733 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:57:06.733 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:57:06.733 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:57:06.733 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:57:06.733 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:57:06.733 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:57:06.733 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:57:06.733 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:57:06.733 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:57:06.734 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:57:06.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.734 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:57:06.734 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:57:06.734 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.75, 2024-09-21T11:57:06.734 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.75, 2024-09-21T11:57:06.734 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:57:06.734 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2, 2024-09-21T11:57:06.734 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2, 2024-09-21T11:57:06.735 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:57:06.735 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:57:06.735 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:57:06.735 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.735 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 4, 2024-09-21T11:57:06.736 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.log", 2024-09-21T11:57:06.736 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:00.380059+0000", 2024-09-21T11:57:06.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:57:06.736 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:57:06.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:57:06.736 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:57:06.736 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:57:06.736 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:57:06.736 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:57:06.737 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:57:06.737 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:57:06.737 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:57:06.737 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:57:06.737 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:57:06.737 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:57:06.737 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:57:06.737 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:57:06.737 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.738 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "62", 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:57:06.739 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.740 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:57:06.741 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 2.190000057220459, 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 2.190000057220459, 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2.5, 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2.5, 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 5, 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.control", 2024-09-21T11:57:06.742 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:02.330041+0000", 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:57:06.743 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "64", 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:57:06.744 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:57:06.745 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:57:06.746 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.3200000524520874, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.3200000524520874, 2024-09-21T11:57:06.747 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:57:06.748 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.5, 2024-09-21T11:57:06.748 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.5, 2024-09-21T11:57:06.748 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:57:06.748 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:57:06.748 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 6, 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "default.rgw.meta", 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:40:04.395413+0000", 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:57:06.749 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 32, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 32, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 32, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 32, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 32, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:57:06.750 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "66", 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:57:06.751 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:57:06.752 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:57:06.753 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:57:06.753 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:57:06.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:57:06.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.753 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:57:06.753 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:57:06.753 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "options": { 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_bias": 4 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:57:06.754 INFO:teuthology.orchestra.run.smithi043.stdout: "rgw": {} 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.5399999618530273, 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.5399999618530273, 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 1.75, 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 1.75, 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.755 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "pool": 11, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_name": "unique_pool_4", 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "create_time": "2024-09-21T11:56:34.685583+0000", 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "flags": 1, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "flags_names": "hashpspool", 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "type": 1, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "min_size": 2, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_rule": 0, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_count": 0, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_target": 0, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_barrier": 0, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "is_stretch_pool": false, 2024-09-21T11:57:06.756 INFO:teuthology.orchestra.run.smithi043.stdout: "object_hash": 2, 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_autoscale_mode": "off", 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num": 16, 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num": 16, 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_placement_num_target": 16, 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_target": 16, 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_num_pending": 16, 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "last_pg_merge_meta": { 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "source_pgid": "0.0", 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "ready_epoch": 0, 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_started": 0, 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "last_epoch_clean": 0, 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "source_version": "0'0", 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "target_version": "0'0" 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.757 INFO:teuthology.orchestra.run.smithi043.stdout: "last_change": "395", 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend": "0", 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "last_force_op_resend_preluminous": "0", 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "auid": 0, 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_mode": "selfmanaged", 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_seq": 0, 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "snap_epoch": 0, 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_snaps": [], 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps": "[]", 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_bytes": 0, 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "quota_max_objects": 0, 2024-09-21T11:57:06.758 INFO:teuthology.orchestra.run.smithi043.stdout: "tiers": [], 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "tier_of": -1, 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "read_tier": -1, 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "write_tier": -1, 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_mode": "none", 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_bytes": 0, 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "target_max_objects": 0, 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_flush_age": 0, 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "cache_min_evict_age": 0, 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profile": "", 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_params": { 2024-09-21T11:57:06.759 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "none" 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_period": 0, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_count": 0, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "use_gmt_hitset": true, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "min_read_recency_for_promote": 0, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "min_write_recency_for_promote": 0, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_grade_decay_rate": 0, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "hit_set_search_last_n": 0, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "grade_table": [], 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_width": 0, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "expected_num_objects": 0, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "fast_read": false, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "options": {}, 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "application_metadata": { 2024-09-21T11:57:06.760 INFO:teuthology.orchestra.run.smithi043.stdout: "rados": {} 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "read_balance": { 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "score_type": "Fair distribution", 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "score_acting": 1.75, 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "score_stable": 1.75, 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "optimal_score": 1.1399999856948853, 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_acting": 2, 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "raw_score_stable": 2, 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity_weighted": 0.87172126770019531, 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity": 1, 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "average_primary_affinity_weighted": 0.87000000476837158 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:57:06.761 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "5229029d-4fd6-46b6-8325-a3e004df09b4", 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 0.68365478515625, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 8, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 364, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.762 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6801", 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6802", 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.763 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6804", 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6803", 2024-09-21T11:57:06.764 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 817279193 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6801/817279193", 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6802/817279193", 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6804/817279193", 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6803/817279193", 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.765 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "a2a7b58b-ba99-4383-a6d3-3b19388b2607", 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 13, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 392, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.766 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6805", 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6806", 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.767 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6808", 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6807", 2024-09-21T11:57:06.768 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 263756171 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6805/263756171", 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6806/263756171", 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6808/263756171", 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6807/263756171", 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.769 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c24a60b5-5713-492b-82ee-576ea7a743ce", 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 18, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 392, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:57:06.770 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6809", 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6810", 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.771 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6812", 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.772 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6811", 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 687577863 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6809/687577863", 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6810/687577863", 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6812/687577863", 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6811/687577863", 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.773 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.774 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:57:06.774 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "c5d43cf8-1c72-4d7d-ae41-e8136b47177d", 2024-09-21T11:57:06.774 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:57:06.774 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:57:06.774 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:57:06.774 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:57:06.774 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:57:06.775 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:57:06.775 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 27, 2024-09-21T11:57:06.775 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 392, 2024-09-21T11:57:06.775 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:57:06.775 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:57:06.775 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:57:06.775 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.776 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.776 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.776 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6813", 2024-09-21T11:57:06.776 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:57:06.776 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.776 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.777 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:57:06.777 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.777 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.777 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.777 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6814", 2024-09-21T11:57:06.777 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:57:06.777 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.777 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.778 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:57:06.778 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.778 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.778 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.778 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6816", 2024-09-21T11:57:06.778 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:57:06.779 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.779 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.779 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:57:06.779 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.779 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.779 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.780 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.43:6815", 2024-09-21T11:57:06.780 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 129493158 2024-09-21T11:57:06.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.780 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.780 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.43:6813/129493158", 2024-09-21T11:57:06.780 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.43:6814/129493158", 2024-09-21T11:57:06.780 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.43:6816/129493158", 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.43:6815/129493158", 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "79ee85c5-afdc-4d68-a253-89b7b0eb652c", 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:57:06.781 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 0.617462158203125, 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 0, 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 33, 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 235, 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6800", 2024-09-21T11:57:06.782 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6801", 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:57:06.783 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6803", 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6802", 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1323992640 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.784 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6800/1323992640", 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6801/1323992640", 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6803/1323992640", 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6802/1323992640", 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "4d647321-71a1-412b-bbf5-921441923a65", 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:57:06.785 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 39, 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 392, 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6804", 2024-09-21T11:57:06.786 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:57:06.787 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.787 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.787 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.787 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:57:06.787 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.787 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.788 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.788 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6805", 2024-09-21T11:57:06.788 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:57:06.788 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.788 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.788 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6807", 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.789 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6806", 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1163208458 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6804/1163208458", 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6805/1163208458", 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6807/1163208458", 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6806/1163208458", 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.790 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "3c33bea7-94b5-46c1-9905-81b4612a2354", 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 0.9071807861328125, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 0.9737701416015625, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 47, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 392, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:57:06.791 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6808", 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6809", 2024-09-21T11:57:06.792 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6811", 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.793 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6810", 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 932395625 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6808/932395625", 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6809/932395625", 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6811/932395625", 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6810/932395625", 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:57:06.794 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "uuid": "41bc9817-d9d1-4c11-94ff-7de332af6d22", 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "up": 1, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "in": 1, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "weight": 1, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_affinity": 1, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_begin": 0, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "last_clean_end": 0, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "up_from": 54, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "up_thru": 392, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "down_at": 0, 2024-09-21T11:57:06.795 INFO:teuthology.orchestra.run.smithi043.stdout: "lost_at": 0, 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addrs": { 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6812", 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addrs": { 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.796 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6813", 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addrs": { 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6815", 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.797 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addrs": { 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: "addrvec": [ 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v2", 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "172.21.15.175:6814", 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3676713275 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: "public_addr": "172.21.15.175:6812/3676713275", 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_addr": "172.21.15.175:6813/3676713275", 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_back_addr": "172.21.15.175:6815/3676713275", 2024-09-21T11:57:06.798 INFO:teuthology.orchestra.run.smithi043.stdout: "heartbeat_front_addr": "172.21.15.175:6814/3676713275", 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: "state": [ 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: "exists", 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: "up" 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_xinfo": [ 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 0, 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:57:06.799 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:56:09.033392+0000", 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 1, 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:57:04.683086+0000", 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:57:06.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 2, 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:57:03.886703+0000", 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 3, 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:57:06.801 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:56:47.486234+0000", 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 4, 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:56:29.776098+0000", 2024-09-21T11:57:06.802 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 5, 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:56:57.708199+0000", 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 6, 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:57:06.803 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:56:19.599687+0000", 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": 7, 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "down_stamp": "0.000000", 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_probability": 0, 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "laggy_interval": 0, 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "features": 4540719139924082687, 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "old_weight": 0, 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "last_purged_snaps_scrub": "2024-09-21T11:56:13.323550+0000", 2024-09-21T11:57:06.804 INFO:teuthology.orchestra.run.smithi043.stdout: "dead_epoch": 0 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap": [ 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.2", 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: 2, 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: 4, 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: 5 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.1c", 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: "osds": [ 2024-09-21T11:57:06.805 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: 7, 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: 2 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_items": [ 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.4", 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:57:06.806 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.7", 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.d", 2024-09-21T11:57:06.807 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.13", 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.808 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.16", 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 3, 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 2 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "4.1c", 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:57:06.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 1, 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 5 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: "pgid": "6.3", 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: "mappings": [ 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: "from": 2, 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: "to": 0 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_upmap_primaries": [], 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: "pg_temp": [], 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: "primary_temp": [], 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklist": { 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2419845767": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3249046048": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2518306973": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3115176321": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/2583179759": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:57:06.811 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/4225358170": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/347827280": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/1513342881": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/1113684867": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/3345448895": "2024-09-22T11:35:03.554546+0000", 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/2454875216": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/911630798": "2024-09-22T11:34:37.015137+0000", 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:0/983028036": "2024-09-22T11:40:29.670859+0000", 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "172.21.15.43:6800/4145025121": "2024-09-22T11:40:29.670859+0000" 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "range_blocklist": {}, 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "erasure_code_profiles": { 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "default": { 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "crush-failure-domain": "osd", 2024-09-21T11:57:06.812 INFO:teuthology.orchestra.run.smithi043.stdout: "k": "2", 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "m": "1", 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "plugin": "jerasure", 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "technique": "reed_sol_van" 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "removed_snaps_queue": [], 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "new_removed_snaps": [], 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "new_purged_snaps": [], 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "crush_node_flags": {}, 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "device_class_flags": {}, 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode": { 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_enabled": false, 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_bucket_count": 0, 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "degraded_stretch_mode": 0, 2024-09-21T11:57:06.813 INFO:teuthology.orchestra.run.smithi043.stdout: "recovering_stretch_mode": 0, 2024-09-21T11:57:06.814 INFO:teuthology.orchestra.run.smithi043.stdout: "stretch_mode_bucket": 0 2024-09-21T11:57:06.814 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:06.814 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:57:06.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:57:06.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:06.845 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:06.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:06.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:06.947 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:06.976 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:06.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:06.977 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:07.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:07.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:07.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:07.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:07.272 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:07.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:07.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:07.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:07.411 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:07.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:07.412 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:07 smithi043 ceph-mon[36454]: 6.7 scrub starts 2024-09-21T11:57:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:07 smithi043 ceph-mon[36454]: 6.7 scrub ok 2024-09-21T11:57:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:07 smithi043 ceph-mon[36454]: pgmap v830: 148 pgs: 148 active+clean; 2.8 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 113 MiB/s wr, 1.80k op/s 2024-09-21T11:57:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/268543750' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:57:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:07 smithi043 ceph-mon[29710]: 6.7 scrub starts 2024-09-21T11:57:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:07 smithi043 ceph-mon[29710]: 6.7 scrub ok 2024-09-21T11:57:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:07 smithi043 ceph-mon[29710]: pgmap v830: 148 pgs: 148 active+clean; 2.8 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 113 MiB/s wr, 1.80k op/s 2024-09-21T11:57:07.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/268543750' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:57:07.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:07 smithi175 ceph-mon[32613]: 6.7 scrub starts 2024-09-21T11:57:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:07 smithi175 ceph-mon[32613]: 6.7 scrub ok 2024-09-21T11:57:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:07 smithi175 ceph-mon[32613]: pgmap v830: 148 pgs: 148 active+clean; 2.8 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 113 MiB/s wr, 1.80k op/s 2024-09-21T11:57:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/268543750' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-21T11:57:07.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:07.541 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:07.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:07.542 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:07.643 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:07.672 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:07.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:07.673 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:07.687 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 27 16 52728 52712 121.991 92.125 0.0172089 0.00819252 2024-09-21T11:57:07.700 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:07.758 INFO:tasks.thrashosds.thrasher:j is {'epoch': 402, 'fsid': '4f2c5252-780d-11ef-baf6-efdc52797490', 'created': '2024-09-21T11:34:07.249195+0000', 'modified': '2024-09-21T11:57:05.047116+0000', 'last_up_change': '2024-09-21T11:39:49.206441+0000', 'last_in_change': '2024-09-21T11:47:02.337266+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T11:37:26.023430+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '398', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.980000019073486, 'score_stable': 6.980000019073486, 'optimal_score': 0.4300000071525574, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-21T11:39:58.262982+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 1, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-21T11:39:59.229530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-21T11:40:00.380059+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.190000057220459, 'score_stable': 2.190000057220459, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-21T11:40:02.330041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3200000524520874, 'score_stable': 1.3200000524520874, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-21T11:40:04.395413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-09-21T11:56:34.685583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '395', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8717212677001953, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8700000047683716}}], 'osds': [{'osd': 0, 'uuid': '5229029d-4fd6-46b6-8325-a3e004df09b4', 'up': 1, 'in': 1, 'weight': 0.68365478515625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 364, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6801', 'nonce': 817279193}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6802', 'nonce': 817279193}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6804', 'nonce': 817279193}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6803', 'nonce': 817279193}]}, 'public_addr': '172.21.15.43:6801/817279193', 'cluster_addr': '172.21.15.43:6802/817279193', 'heartbeat_back_addr': '172.21.15.43:6804/817279193', 'heartbeat_front_addr': '172.21.15.43:6803/817279193', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a2a7b58b-ba99-4383-a6d3-3b19388b2607', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 392, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6805', 'nonce': 263756171}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6806', 'nonce': 263756171}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6808', 'nonce': 263756171}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6807', 'nonce': 263756171}]}, 'public_addr': '172.21.15.43:6805/263756171', 'cluster_addr': '172.21.15.43:6806/263756171', 'heartbeat_back_addr': '172.21.15.43:6808/263756171', 'heartbeat_front_addr': '172.21.15.43:6807/263756171', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c24a60b5-5713-492b-82ee-576ea7a743ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 392, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6809', 'nonce': 687577863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6810', 'nonce': 687577863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6812', 'nonce': 687577863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6811', 'nonce': 687577863}]}, 'public_addr': '172.21.15.43:6809/687577863', 'cluster_addr': '172.21.15.43:6810/687577863', 'heartbeat_back_addr': '172.21.15.43:6812/687577863', 'heartbeat_front_addr': '172.21.15.43:6811/687577863', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c5d43cf8-1c72-4d7d-ae41-e8136b47177d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 27, 'up_thru': 392, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6813', 'nonce': 129493158}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6814', 'nonce': 129493158}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6816', 'nonce': 129493158}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6815', 'nonce': 129493158}]}, 'public_addr': '172.21.15.43:6813/129493158', 'cluster_addr': '172.21.15.43:6814/129493158', 'heartbeat_back_addr': '172.21.15.43:6816/129493158', 'heartbeat_front_addr': '172.21.15.43:6815/129493158', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '79ee85c5-afdc-4d68-a253-89b7b0eb652c', 'up': 1, 'in': 1, 'weight': 0.617462158203125, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 33, 'up_thru': 235, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6800', 'nonce': 1323992640}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6801', 'nonce': 1323992640}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6803', 'nonce': 1323992640}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6802', 'nonce': 1323992640}]}, 'public_addr': '172.21.15.175:6800/1323992640', 'cluster_addr': '172.21.15.175:6801/1323992640', 'heartbeat_back_addr': '172.21.15.175:6803/1323992640', 'heartbeat_front_addr': '172.21.15.175:6802/1323992640', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4d647321-71a1-412b-bbf5-921441923a65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 392, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6804', 'nonce': 1163208458}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6805', 'nonce': 1163208458}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6807', 'nonce': 1163208458}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6806', 'nonce': 1163208458}]}, 'public_addr': '172.21.15.175:6804/1163208458', 'cluster_addr': '172.21.15.175:6805/1163208458', 'heartbeat_back_addr': '172.21.15.175:6807/1163208458', 'heartbeat_front_addr': '172.21.15.175:6806/1163208458', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '3c33bea7-94b5-46c1-9905-81b4612a2354', 'up': 1, 'in': 1, 'weight': 0.9071807861328125, 'primary_affinity': 0.9737701416015625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 47, 'up_thru': 392, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6808', 'nonce': 932395625}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6809', 'nonce': 932395625}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6811', 'nonce': 932395625}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6810', 'nonce': 932395625}]}, 'public_addr': '172.21.15.175:6808/932395625', 'cluster_addr': '172.21.15.175:6809/932395625', 'heartbeat_back_addr': '172.21.15.175:6811/932395625', 'heartbeat_front_addr': '172.21.15.175:6810/932395625', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '41bc9817-d9d1-4c11-94ff-7de332af6d22', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 54, 'up_thru': 392, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6812', 'nonce': 3676713275}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6813', 'nonce': 3676713275}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6815', 'nonce': 3676713275}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6814', 'nonce': 3676713275}]}, 'public_addr': '172.21.15.175:6812/3676713275', 'cluster_addr': '172.21.15.175:6813/3676713275', 'heartbeat_back_addr': '172.21.15.175:6815/3676713275', 'heartbeat_front_addr': '172.21.15.175:6814/3676713275', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:56:09.033392+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:57:04.683086+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:57:03.886703+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:56:47.486234+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:56:29.776098+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:56:57.708199+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:56:19.599687+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-21T11:56:13.323550+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '6.2', 'osds': [2, 4, 5]}, {'pgid': '6.1c', 'osds': [0, 7, 2]}], 'pg_upmap_items': [{'pgid': '4.4', 'mappings': [{'from': 3, 'to': 5}]}, {'pgid': '4.7', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.d', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.16', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '4.1c', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '6.3', 'mappings': [{'from': 2, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.43:0/2419845767': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3249046048': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/2518306973': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/3115176321': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:6800/2583179759': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/4225358170': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/347827280': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/1513342881': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/1113684867': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/3345448895': '2024-09-22T11:35:03.554546+0000', '172.21.15.43:0/2454875216': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:0/911630798': '2024-09-22T11:34:37.015137+0000', '172.21.15.43:0/983028036': '2024-09-22T11:40:29.670859+0000', '172.21.15.43:6800/4145025121': '2024-09-22T11:40:29.670859+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-21T11:57:07.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:57:07.774 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:07.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:07.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:07.961 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:07.989 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:07.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:07.991 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:08.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:08.114 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:08.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:08.115 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:08.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:08.249 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:08.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:08.250 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:08.346 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:08.350 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:08.374 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:08.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:08.376 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:08.413 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:57:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:57:08.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:57:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:57:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119715 9.b 9:d24acb6c:::benchmark_data_smithi043_126002_object119714:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:57:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:57:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 391.75656074199998, 2024-09-21T11:57:08.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.205727302, 2024-09-21T11:57:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:57:08.415 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:57:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119715 2024-09-21T11:57:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:57:08.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651646+0000", 2024-09-21T11:57:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651647+0000", 2024-09-21T11:57:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-21T11:57:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651670+0000", 2024-09-21T11:57:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2603999999999999e-05 2024-09-21T11:57:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651679+0000", 2024-09-21T11:57:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2490000000000007e-06 2024-09-21T11:57:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.651719+0000", 2024-09-21T11:57:08.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0123999999999997e-05 2024-09-21T11:57:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652107+0000", 2024-09-21T11:57:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00038846399999999999 2024-09-21T11:57:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652254+0000", 2024-09-21T11:57:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014700100000000001 2024-09-21T11:57:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:57:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.652440+0000", 2024-09-21T11:57:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018522499999999999 2024-09-21T11:57:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851244+0000", 2024-09-21T11:57:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.198804392 2024-09-21T11:57:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851269+0000", 2024-09-21T11:57:08.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5381999999999999e-05 2024-09-21T11:57:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857251+0000", 2024-09-21T11:57:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059817589999999997 2024-09-21T11:57:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857317+0000", 2024-09-21T11:57:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5874e-05 2024-09-21T11:57:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857373+0000", 2024-09-21T11:57:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6014000000000003e-05 2024-09-21T11:57:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119721 9.5 9:a09f84b6:::benchmark_data_smithi043_126002_object119720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:57:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:57:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 391.75178345299997, 2024-09-21T11:57:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20659565899999999, 2024-09-21T11:57:08.428 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:57:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:57:08.429 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119721 2024-09-21T11:57:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:57:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656423+0000", 2024-09-21T11:57:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656425+0000", 2024-09-21T11:57:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.725e-06 2024-09-21T11:57:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656448+0000", 2024-09-21T11:57:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3238999999999999e-05 2024-09-21T11:57:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656461+0000", 2024-09-21T11:57:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.331e-05 2024-09-21T11:57:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656470+0000", 2024-09-21T11:57:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7940000000000008e-06 2024-09-21T11:57:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656563+0000", 2024-09-21T11:57:08.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2993999999999998e-05 2024-09-21T11:57:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656754+0000", 2024-09-21T11:57:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019041800000000001 2024-09-21T11:57:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:57:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656917+0000", 2024-09-21T11:57:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016343600000000001 2024-09-21T11:57:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.856079+0000", 2024-09-21T11:57:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.199162268 2024-09-21T11:57:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.858867+0000", 2024-09-21T11:57:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002787253 2024-09-21T11:57:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.862926+0000", 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040595980000000002 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.862979+0000", 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3369999999999999e-05 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.863019+0000", 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9254000000000001e-05 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.438 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119722 9.9 9:9c70bd2d:::benchmark_data_smithi043_126002_object119721:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:57:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:57:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 391.75171014400001, 2024-09-21T11:57:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.21088224999999999, 2024-09-21T11:57:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:57:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:57:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119722 2024-09-21T11:57:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656496+0000", 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656497+0000", 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4499999999999997e-07 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656518+0000", 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1294e-05 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656526+0000", 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7570000000000004e-06 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656532+0000", 2024-09-21T11:57:08.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9689999999999999e-06 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656639+0000", 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010705999999999999 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656753+0000", 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011366099999999999 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657088+0000", 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00033480000000000001 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.860812+0000", 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.203723983 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.860856+0000", 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3983999999999998e-05 2024-09-21T11:57:08.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867254+0000", 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063985939999999996 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867337+0000", 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2649000000000001e-05 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867379+0000", 2024-09-21T11:57:08.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1854000000000003e-05 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:119715 9.b e277/247 ondisk, result = 0)", 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 391.75164480199999, 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19469081599999999, 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656562+0000", 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656563+0000", 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2699999999999998e-07 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656565+0000", 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5900000000000002e-06 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656572+0000", 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8730000000000001e-06 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.656577+0000", 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3020000000000002e-06 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851228+0000", 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19465021800000001 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851237+0000", 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2639999999999997e-06 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851253+0000", 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5741999999999999e-05 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:119715 9.b e277/247 ondisk, result = 0)", 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 391.75084571999997, 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20004322699999999, 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.450 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657361+0000", 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657362+0000", 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2799999999999995e-07 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.657398+0000", 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6180000000000003e-05 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851962+0000", 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19456380700000001 2024-09-21T11:57:08.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851969+0000", 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2470000000000002e-06 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857236+0000", 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052674760000000001 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857245+0000", 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3350000000000007e-06 2024-09-21T11:57:08.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.857404+0000", 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015935400000000001 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:119726 9.9 9:9360eb09:::benchmark_data_smithi043_126002_object119725:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:57:08.454 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 391.74904439699998, 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20832704299999999, 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 119726 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:57:08.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659162+0000", 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659164+0000", 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2450000000000001e-06 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.659190+0000", 2024-09-21T11:57:08.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5653999999999999e-05 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851168+0000", 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.191977544 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.851185+0000", 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7118000000000001e-05 2024-09-21T11:57:08.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854231+0000", 2024-09-21T11:57:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003045889 2024-09-21T11:57:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854382+0000", 2024-09-21T11:57:08.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015179700000000001 2024-09-21T11:57:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,7", 2024-09-21T11:57:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.854513+0000", 2024-09-21T11:57:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013041499999999999 2024-09-21T11:57:08.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867241+0000", 2024-09-21T11:57:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012728013 2024-09-21T11:57:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867388+0000", 2024-09-21T11:57:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147128 2024-09-21T11:57:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867435+0000", 2024-09-21T11:57:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6724000000000002e-05 2024-09-21T11:57:08.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867467+0000", 2024-09-21T11:57:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1986000000000002e-05 2024-09-21T11:57:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:50:36.867489+0000", 2024-09-21T11:57:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2529999999999999e-05 2024-09-21T11:57:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.463 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47521 10.4 10:207ae421:::benchmark_data_smithi043_150678_object47520:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:57:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.835329+0000", 2024-09-21T11:57:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 234.57287787199999, 2024-09-21T11:57:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.30159633899999999, 2024-09-21T11:57:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.464 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47521 2024-09-21T11:57:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835329+0000", 2024-09-21T11:57:08.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835329+0000", 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835330+0000", 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-06 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835511+0000", 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000180896 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835570+0000", 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.914e-05 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835577+0000", 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4549999999999998e-06 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.839940+0000", 2024-09-21T11:57:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043626489999999997 2024-09-21T11:57:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.840064+0000", 2024-09-21T11:57:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012452599999999999 2024-09-21T11:57:08.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.840193+0000", 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012866900000000001 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843992+0000", 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037985509999999998 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.845585+0000", 2024-09-21T11:57:08.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015931949999999999 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136797+0000", 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29121228599999999 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136857+0000", 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0177999999999999e-05 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136925+0000", 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7770999999999999e-05 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47525 10.4 10:29bd7892:::benchmark_data_smithi043_150678_object47524:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.841626+0000", 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 234.566580155, 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29545084599999999, 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.472 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47525 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841626+0000", 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841626+0000", 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841628+0000", 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4610000000000001e-06 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841655+0000", 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6658000000000001e-05 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841682+0000", 2024-09-21T11:57:08.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7674999999999998e-05 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841691+0000", 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5520000000000005e-06 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841931+0000", 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023983500000000001 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.475 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842058+0000", 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012696499999999999 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842192+0000", 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013453200000000001 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.845735+0000", 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035430750000000001 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.847002+0000", 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012665899999999999 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136972+0000", 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28997065100000002 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137022+0000", 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9577999999999999e-05 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137077+0000", 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5274e-05 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47558 10.4 10:2e79000f:::benchmark_data_smithi043_150678_object47557:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.855768+0000", 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 234.55243886100001, 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28151058899999998, 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47558 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855768+0000", 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855768+0000", 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855769+0000", 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.33e-06 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855788+0000", 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9289000000000001e-05 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855811+0000", 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3139e-05 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855819+0000", 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2819999999999999e-06 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855885+0000", 2024-09-21T11:57:08.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5813000000000002e-05 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.856043+0000", 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015814 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.856194+0000", 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015090899999999999 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.859384+0000", 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031903259999999998 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860384+0000", 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00099984199999999992 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137145+0000", 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27676123800000002 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137217+0000", 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2330999999999996e-05 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137278+0000", 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0949999999999998e-05 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47561 10.4 10:2c90ec34:::benchmark_data_smithi043_150678_object47560:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.857677+0000", 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 234.55052956200001, 2024-09-21T11:57:08.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27975456399999998, 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47561 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857677+0000", 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857677+0000", 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857678+0000", 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.297e-06 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857699+0000", 2024-09-21T11:57:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1055000000000001e-05 2024-09-21T11:57:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857714+0000", 2024-09-21T11:57:08.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4229e-05 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857721+0000", 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8399999999999995e-06 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857847+0000", 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125618 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857956+0000", 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109215 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.858055+0000", 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8906999999999993e-05 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.859507+0000", 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0014520920000000001 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860490+0000", 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098292600000000007 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137335+0000", 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27684498899999999 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137393+0000", 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8255999999999999e-05 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137432+0000", 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8139999999999997e-05 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:67192 10.4 10:20483ca3:::benchmark_data_smithi043_150678_object67191:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:28.342058+0000", 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 220.06614851500001, 2024-09-21T11:57:08.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19868539599999999, 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 67192 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342058+0000", 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342058+0000", 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342060+0000", 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6700000000000001e-06 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342080+0000", 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0582e-05 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342095+0000", 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4853e-05 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342102+0000", 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2069999999999998e-06 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342158+0000", 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5599000000000001e-05 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342286+0000", 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012840200000000001 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.342431+0000", 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014455499999999999 2024-09-21T11:57:08.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.345136+0000", 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027053469999999999 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.345250+0000", 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114228 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.540632+0000", 2024-09-21T11:57:08.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.195381519 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.540694+0000", 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1975000000000002e-05 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:28.540743+0000", 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9459000000000002e-05 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.498 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:85498 10.8 10:1841726b:::benchmark_data_smithi043_150678_object85497:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:44.793462+0000", 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 203.61474426699999, 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28091800300000003, 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 85498 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793462+0000", 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793462+0000", 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793464+0000", 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.282e-06 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793488+0000", 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4277000000000002e-05 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793498+0000", 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0564000000000001e-05 2024-09-21T11:57:08.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793512+0000", 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3858e-05 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793594+0000", 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1499e-05 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793742+0000", 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014823599999999999 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793877+0000", 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000135227 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.797141+0000", 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032633900000000001 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.798058+0000", 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00091767599999999997 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074222+0000", 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27616407700000001 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074310+0000", 2024-09-21T11:57:08.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7566999999999997e-05 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074380+0000", 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0350000000000002e-05 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:85533 10.8 10:1dd7dbdf:::benchmark_data_smithi043_150678_object85532:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:44.822341+0000", 2024-09-21T11:57:08.505 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 203.58586522300001, 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.25225375999999999, 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 85533 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822341+0000", 2024-09-21T11:57:08.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822341+0000", 2024-09-21T11:57:08.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822342+0000", 2024-09-21T11:57:08.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.164e-06 2024-09-21T11:57:08.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822370+0000", 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7681000000000001e-05 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822381+0000", 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1032999999999999e-05 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822388+0000", 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0400000000000004e-06 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822464+0000", 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5916999999999998e-05 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822690+0000", 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000225388 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.822866+0000", 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017633899999999999 2024-09-21T11:57:08.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.826112+0000", 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032457430000000002 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.826939+0000", 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00082743300000000003 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074462+0000", 2024-09-21T11:57:08.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.24752277 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074555+0000", 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2961000000000007e-05 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074595+0000", 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0290999999999998e-05 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126842 10.4 10:2c1c1280:::benchmark_data_smithi043_150678_object4091:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.884029+0000", 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 162.52417763700001, 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.23360112199999999, 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126842 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.513 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884029+0000", 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884029+0000", 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884030+0000", 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4499999999999997e-07 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884032+0000", 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8200000000000001e-06 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884039+0000", 2024-09-21T11:57:08.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5910000000000001e-06 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.884044+0000", 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3539999999999999e-06 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.887155+0000", 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031106319999999999 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.887284+0000", 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000128634 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.887406+0000", 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012195 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.890430+0000", 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030242569999999998 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895372+0000", 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049420849999999997 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117520+0000", 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.22214824699999999 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117582+0000", 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1618999999999994e-05 2024-09-21T11:57:08.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117630+0000", 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8288000000000003e-05 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126858 10.4 10:2178d15c:::benchmark_data_smithi043_150678_object4107:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.895928+0000", 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 162.51227832800001, 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.22202257, 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.519 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126858 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895928+0000", 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895928+0000", 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895929+0000", 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4700000000000001e-07 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895932+0000", 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4089999999999999e-06 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895941+0000", 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2240000000000001e-06 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895948+0000", 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2639999999999997e-06 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.896006+0000", 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7969e-05 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.896110+0000", 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010406 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.896213+0000", 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010307899999999999 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.899350+0000", 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031370899999999999 2024-09-21T11:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.901791+0000", 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024410310000000002 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117743+0000", 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.215951699 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117903+0000", 2024-09-21T11:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016066600000000001 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.117951+0000", 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7432000000000003e-05 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126885 10.4 10:2892f7f4:::benchmark_data_smithi043_150678_object4134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.918908+0000", 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 162.489299004, 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.20485952199999999, 2024-09-21T11:57:08.525 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126885 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918908+0000", 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918908+0000", 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918909+0000", 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2750000000000001e-06 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918915+0000", 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7660000000000004e-06 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918928+0000", 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3444e-05 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918937+0000", 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8040000000000007e-06 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.918996+0000", 2024-09-21T11:57:08.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9425000000000001e-05 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.919146+0000", 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015016399999999999 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.919290+0000", 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000143234 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.921837+0000", 2024-09-21T11:57:08.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025476990000000001 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.922942+0000", 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0011045759999999999 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123550+0000", 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.200607601 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123718+0000", 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000168708 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123767+0000", 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8826000000000003e-05 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126898 10.4 10:2c034698:::benchmark_data_smithi043_150678_object4147:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:08.531 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.930744+0000", 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 162.47746290500001, 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19317583399999999, 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126898 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930744+0000", 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930744+0000", 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930745+0000", 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.499e-06 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930749+0000", 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2259999999999999e-06 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930763+0000", 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3512e-05 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930770+0000", 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2899999999999997e-06 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930833+0000", 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3257999999999996e-05 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930950+0000", 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011681 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.931074+0000", 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012377699999999999 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.933746+0000", 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026722899999999999 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.934205+0000", 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00045867100000000001 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123819+0000", 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.18961380999999999 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123882+0000", 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2739999999999999e-05 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123919+0000", 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7951000000000003e-05 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:168906 10.8 10:1f344508:::benchmark_data_smithi043_150678_object46155:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e358)", 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:04.074915+0000", 2024-09-21T11:57:08.537 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 124.333291567, 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.167228132, 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 168906 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074915+0000", 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074915+0000", 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074916+0000", 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4020000000000001e-06 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074921+0000", 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.865e-06 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074932+0000", 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0550000000000001e-05 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.074939+0000", 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.571e-06 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.075042+0000", 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102817 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.075156+0000", 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113479 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.075301+0000", 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014525000000000001 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.077475+0000", 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021743520000000001 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.078271+0000", 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00079528200000000002 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.242030+0000", 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.16375980200000001 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.242090+0000", 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9790000000000002e-05 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.242143+0000", 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2972e-05 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:224200 10.8 10:148c8889:::benchmark_data_smithi043_150678_object101449:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e374)", 2024-09-21T11:57:08.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:58.689418+0000", 2024-09-21T11:57:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.718788301999993, 2024-09-21T11:57:08.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.13092927800000001, 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 224200 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.689418+0000", 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.689418+0000", 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.689420+0000", 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4789999999999999e-06 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.689424+0000", 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2440000000000004e-06 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.689436+0000", 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1919000000000001e-05 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.689458+0000", 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1724000000000001e-05 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.689515+0000", 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7485000000000003e-05 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.689619+0000", 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010355 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.689784+0000", 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016496699999999999 2024-09-21T11:57:08.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.694083+0000", 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0042994280000000001 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.694203+0000", 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120328 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.820238+0000", 2024-09-21T11:57:08.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12603423499999999 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.820299+0000", 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1566e-05 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.820348+0000", 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8353000000000003e-05 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:224216 10.8 10:19664f3e:::benchmark_data_smithi043_150678_object101465:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e374)", 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:58.699638+0000", 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 69.708568069999998, 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.12967387, 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 224216 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.551 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.699638+0000", 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.699638+0000", 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.699640+0000", 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9700000000000002e-06 2024-09-21T11:57:08.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.699645+0000", 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6940000000000003e-06 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.699656+0000", 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0906e-05 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.699663+0000", 2024-09-21T11:57:08.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8109999999999997e-06 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.699793+0000", 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012980700000000001 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.699912+0000", 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119515 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:57:08.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.700018+0000", 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105399 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.820549+0000", 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.120531057 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.820590+0000", 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1684999999999998e-05 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.829215+0000", 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0086250509999999999 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.829270+0000", 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4515000000000002e-05 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:58.829312+0000", 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2459999999999997e-05 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.556 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:08.557 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:08.557 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:57:08.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:08.558 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:08.558 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:08.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:08 smithi043 ceph-mon[36454]: 4.1a scrub starts 2024-09-21T11:57:08.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:08 smithi043 ceph-mon[36454]: 4.1a scrub ok 2024-09-21T11:57:08.658 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:08 smithi043 ceph-mon[29710]: 4.1a scrub starts 2024-09-21T11:57:08.659 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:08 smithi043 ceph-mon[29710]: 4.1a scrub ok 2024-09-21T11:57:08.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:08.686 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:08.687 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 28 16 54055 54039 120.596 82.9375 0.00647981 0.00828701 2024-09-21T11:57:08.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:08.688 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:08.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:08 smithi175 ceph-mon[32613]: 4.1a scrub starts 2024-09-21T11:57:08.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:08 smithi175 ceph-mon[32613]: 4.1a scrub ok 2024-09-21T11:57:08.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:08.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:57:08.828 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:08.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:08.829 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:08.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:08.956 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:08.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:08.957 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:09.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:09.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:09.106 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:09.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:09.207 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:09.242 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:09.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:09.243 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:09.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:09.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:09.386 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:09.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:09.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:09 smithi043 ceph-mon[29710]: pgmap v831: 148 pgs: 148 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-21T11:57:09.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:09 smithi043 ceph-mon[29710]: 4.7 scrub starts 2024-09-21T11:57:09.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:09 smithi043 ceph-mon[29710]: 4.7 scrub ok 2024-09-21T11:57:09.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:09 smithi043 ceph-mon[36454]: pgmap v831: 148 pgs: 148 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-21T11:57:09.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:09 smithi043 ceph-mon[36454]: 4.7 scrub starts 2024-09-21T11:57:09.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:09 smithi043 ceph-mon[36454]: 4.7 scrub ok 2024-09-21T11:57:09.517 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:09.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:09.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:09.619 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:09.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:09.657 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:09.687 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 29 15 55487 55472 119.526 89.5625 0.0118009 0.00836122 2024-09-21T11:57:09.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:09.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:09 smithi175 ceph-mon[32613]: pgmap v831: 148 pgs: 148 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-21T11:57:09.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:09 smithi175 ceph-mon[32613]: 4.7 scrub starts 2024-09-21T11:57:09.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:09 smithi175 ceph-mon[32613]: 4.7 scrub ok 2024-09-21T11:57:09.801 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:09.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:09.802 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:09.903 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:09.945 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:09.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:09.946 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:10.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:10.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:10.092 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:10.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:10.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:10.221 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:10.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:10.222 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:10.322 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:10.345 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:10.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:10.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:10.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:10.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:10.481 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:10.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:10.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:10.618 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:10.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:10.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:10.687 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 30 16 56917 56901 118.519 89.3125 0.00605777 0.00843208 2024-09-21T11:57:10.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:10.745 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:10.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:10.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:10.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:10.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:10.887 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:10.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:11.019 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:11.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:11.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:11.121 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:11.155 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:11.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:11.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:11.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:11.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:11.297 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:11.397 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:11.420 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:11.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:11.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:11.522 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:11.554 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:11.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:11.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:11.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:11.688 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 31 16 58098 58082 117.075 73.8125 0.00781402 0.00853613 2024-09-21T11:57:11.699 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:11.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:11.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:11 smithi043 ceph-mon[29710]: pgmap v832: 148 pgs: 148 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 722 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:57:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:11 smithi043 ceph-mon[29710]: 5.9 scrub starts 2024-09-21T11:57:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:11 smithi043 ceph-mon[29710]: 5.9 scrub ok 2024-09-21T11:57:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:11 smithi043 ceph-mon[29710]: 3.19 scrub starts 2024-09-21T11:57:11.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:11 smithi043 ceph-mon[29710]: 3.19 scrub ok 2024-09-21T11:57:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:11 smithi043 ceph-mon[36454]: pgmap v832: 148 pgs: 148 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 722 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:57:11.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:11 smithi043 ceph-mon[36454]: 5.9 scrub starts 2024-09-21T11:57:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:11 smithi043 ceph-mon[36454]: 5.9 scrub ok 2024-09-21T11:57:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:11 smithi043 ceph-mon[36454]: 3.19 scrub starts 2024-09-21T11:57:11.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:11 smithi043 ceph-mon[36454]: 3.19 scrub ok 2024-09-21T11:57:11.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:11 smithi175 ceph-mon[32613]: pgmap v832: 148 pgs: 148 active+clean; 3.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 722 B/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-21T11:57:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:11 smithi175 ceph-mon[32613]: 5.9 scrub starts 2024-09-21T11:57:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:11 smithi175 ceph-mon[32613]: 5.9 scrub ok 2024-09-21T11:57:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:11 smithi175 ceph-mon[32613]: 3.19 scrub starts 2024-09-21T11:57:11.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:11 smithi175 ceph-mon[32613]: 3.19 scrub ok 2024-09-21T11:57:11.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:11.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:11.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:11.947 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:11.974 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:11.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:11.976 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:12.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:12.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:12.099 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:12.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:12.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:12.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:12.238 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:12.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:57:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:57:12.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:12.375 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:12.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:12.376 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:12.476 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:12.502 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:12.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:12.503 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:12.604 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:12.625 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:12.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:12.626 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:12.687 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 32 16 59276 59260 115.717 73.625 0.00324832 0.00861841 2024-09-21T11:57:12.727 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:12.757 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:12.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:12.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:12.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:12.900 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:12.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:12.901 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:13.002 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:13.023 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:13.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:13.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:13.125 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:13.151 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:13.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:13.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:13.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:13.281 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:13.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:13.282 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:13.383 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:13.416 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:13.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:13.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:13.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:13.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:13.540 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:13.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:13.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:13.669 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:13.669 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:13.687 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 33 16 60120 60104 113.809 52.75 0.00309711 0.00875741 2024-09-21T11:57:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:13 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:13 smithi043 ceph-mon[29710]: pgmap v833: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 113 MiB/s wr, 1.82k op/s 2024-09-21T11:57:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:13 smithi043 ceph-mon[29710]: 6.14 scrub starts 2024-09-21T11:57:13.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:13 smithi043 ceph-mon[29710]: 6.14 scrub ok 2024-09-21T11:57:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:13 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:13 smithi043 ceph-mon[36454]: pgmap v833: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 113 MiB/s wr, 1.82k op/s 2024-09-21T11:57:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:13 smithi043 ceph-mon[36454]: 6.14 scrub starts 2024-09-21T11:57:13.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:13 smithi043 ceph-mon[36454]: 6.14 scrub ok 2024-09-21T11:57:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:13 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:13 smithi175 ceph-mon[32613]: pgmap v833: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 113 MiB/s wr, 1.82k op/s 2024-09-21T11:57:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:13 smithi175 ceph-mon[32613]: 6.14 scrub starts 2024-09-21T11:57:13.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:13 smithi175 ceph-mon[32613]: 6.14 scrub ok 2024-09-21T11:57:13.770 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:13.799 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:13.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:13.800 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:13.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:13.939 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:13.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:13.941 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:14.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:14.071 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:14.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:14.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:14.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:14.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:14.215 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:14.216 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:14.317 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:14.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:14.343 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:14.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:14.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:57:14.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:14.469 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:14.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:14.470 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:14.493 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:57:14.494 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:57:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:57:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:57:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:57:14.495 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:57:14.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:14.612 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:14.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:14.613 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:14.688 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 34 16 61314 61298 112.657 74.625 0.00468805 0.00887082 2024-09-21T11:57:14.714 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:14.745 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:14.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:14.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:14.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:14 smithi175 ceph-mon[32613]: 5.1a scrub starts 2024-09-21T11:57:14.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:14 smithi175 ceph-mon[32613]: 5.1a scrub ok 2024-09-21T11:57:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:14 smithi175 ceph-mon[32613]: 3.8 scrub starts 2024-09-21T11:57:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:14 smithi175 ceph-mon[32613]: 3.8 scrub ok 2024-09-21T11:57:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:14 smithi175 ceph-mon[32613]: 5.18 scrub starts 2024-09-21T11:57:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:14 smithi175 ceph-mon[32613]: 5.18 scrub ok 2024-09-21T11:57:14.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2893924649' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:14.847 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:14.875 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:14.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:14.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:14.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:14.979 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[36454]: 5.1a scrub starts 2024-09-21T11:57:14.979 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[36454]: 5.1a scrub ok 2024-09-21T11:57:14.979 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[36454]: 3.8 scrub starts 2024-09-21T11:57:14.979 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[36454]: 3.8 scrub ok 2024-09-21T11:57:14.979 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[36454]: 5.18 scrub starts 2024-09-21T11:57:14.979 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[36454]: 5.18 scrub ok 2024-09-21T11:57:14.979 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2893924649' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:14.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[29710]: 5.1a scrub starts 2024-09-21T11:57:14.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[29710]: 5.1a scrub ok 2024-09-21T11:57:14.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[29710]: 3.8 scrub starts 2024-09-21T11:57:14.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[29710]: 3.8 scrub ok 2024-09-21T11:57:14.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[29710]: 5.18 scrub starts 2024-09-21T11:57:14.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[29710]: 5.18 scrub ok 2024-09-21T11:57:14.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2893924649' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:15.010 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:15.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:15.011 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:15.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:15.154 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:15.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:15.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:15.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:15.289 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:15.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:15.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:15.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:15.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:15.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:15.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:57:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:57:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:15.532 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:15.559 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:15.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:15.560 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:15.604 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:15.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:15.688 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:15.688 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 35 16 62666 62650 111.851 84.5 0.0196618 0.00893512 2024-09-21T11:57:15.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:15.689 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:15.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:15.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:15 smithi043 ceph-mon[29710]: pgmap v834: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-21T11:57:15.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:15 smithi043 ceph-mon[29710]: 3.2 scrub starts 2024-09-21T11:57:15.794 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:15 smithi043 ceph-mon[29710]: 3.2 scrub ok 2024-09-21T11:57:15.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:15 smithi043 ceph-mon[36454]: pgmap v834: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-21T11:57:15.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:15 smithi043 ceph-mon[36454]: 3.2 scrub starts 2024-09-21T11:57:15.794 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:15 smithi043 ceph-mon[36454]: 3.2 scrub ok 2024-09-21T11:57:15.819 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:15.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:15.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:15.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:15.926 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:15.930 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:15.950 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:15.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:15.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:16.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:15 smithi175 ceph-mon[32613]: pgmap v834: 148 pgs: 148 active+clean; 3.5 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-21T11:57:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:15 smithi175 ceph-mon[32613]: 3.2 scrub starts 2024-09-21T11:57:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:15 smithi175 ceph-mon[32613]: 3.2 scrub ok 2024-09-21T11:57:16.052 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:16.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:16.088 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:16.088 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:16.188 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:16.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:16.222 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:16.222 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:16.323 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:16.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:16.354 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:16.354 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:16.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:16.494 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:16.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:16.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:16.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:16.623 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:16.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:16.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:16.688 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 36 16 63829 63813 110.763 72.6875 0.00267094 0.00899919 2024-09-21T11:57:16.725 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:16 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T11:57:16.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:16 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T11:57:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:16 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T11:57:16.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:16 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T11:57:16.755 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:16.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:16.756 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:16.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:16.894 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:16.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:16.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:16.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:17.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:16 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T11:57:17.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:16 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T11:57:17.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:17.032 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:17.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:17.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:17.157 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:17.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:17.159 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:17.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:17.299 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:17.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:17.300 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:17.401 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:17.433 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:17.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:17.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:17.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:17.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:17.558 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:17.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:57:17.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:17.682 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:17.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:17.686 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:17.688 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 37 16 64735 64719 109.3 56.625 0.260254 0.00912033 2024-09-21T11:57:17.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:17.829 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:17.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:17.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:17.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[29710]: pgmap v835: 148 pgs: 148 active+clean; 3.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 924 B/s rd, 95 MiB/s wr, 1.53k op/s 2024-09-21T11:57:17.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[29710]: from='client.26390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:57:17.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[29710]: 4.14 scrub starts 2024-09-21T11:57:17.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[29710]: 4.14 scrub ok 2024-09-21T11:57:17.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:17.859 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:17.859 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[36454]: pgmap v835: 148 pgs: 148 active+clean; 3.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 924 B/s rd, 95 MiB/s wr, 1.53k op/s 2024-09-21T11:57:17.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[36454]: from='client.26390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:57:17.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[36454]: 4.14 scrub starts 2024-09-21T11:57:17.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[36454]: 4.14 scrub ok 2024-09-21T11:57:17.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:17.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:17 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:17.931 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:17.956 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:17.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:17.957 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:17.970 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:57:17.976 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:57:18.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:17 smithi175 ceph-mon[32613]: pgmap v835: 148 pgs: 148 active+clean; 3.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 924 B/s rd, 95 MiB/s wr, 1.53k op/s 2024-09-21T11:57:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:17 smithi175 ceph-mon[32613]: from='client.26390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:57:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:17 smithi175 ceph-mon[32613]: 4.14 scrub starts 2024-09-21T11:57:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:17 smithi175 ceph-mon[32613]: 4.14 scrub ok 2024-09-21T11:57:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:17 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:17 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:18.058 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:18.110 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:18.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:18.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:18.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:18.258 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:18.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:18.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:18.360 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:18.391 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:18.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:18.392 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:18.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:18.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:18.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:18.633 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:18.658 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:18.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:18.659 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:18.688 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 38 16 65817 65801 108.203 67.625 0.0138158 0.00923646 2024-09-21T11:57:18.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:18 smithi043 ceph-mon[36454]: 3.15 scrub starts 2024-09-21T11:57:18.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:18 smithi043 ceph-mon[36454]: 3.15 scrub ok 2024-09-21T11:57:18.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:18 smithi043 ceph-mon[29710]: 3.15 scrub starts 2024-09-21T11:57:18.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:18 smithi043 ceph-mon[29710]: 3.15 scrub ok 2024-09-21T11:57:18.760 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:18.787 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:18.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:18.788 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:18.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:18.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:18.928 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:18.928 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:19.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:18 smithi175 ceph-mon[32613]: 3.15 scrub starts 2024-09-21T11:57:19.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:18 smithi175 ceph-mon[32613]: 3.15 scrub ok 2024-09-21T11:57:19.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:19.077 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:19.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:19.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:19.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:19.206 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:19.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:19.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:19.307 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:19.335 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:19.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:19.336 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:19.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:19.460 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:19.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:19.461 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:57:19.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:19.588 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:19.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:19.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:19.688 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 39 16 67020 67004 107.356 75.1875 0.0180772 0.0093094 2024-09-21T11:57:19.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:19.732 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:19.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:19.733 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:19.834 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:19.863 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:19.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:19.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:19.883 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:19 smithi043 ceph-mon[36454]: pgmap v836: 148 pgs: 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-21T11:57:19.883 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:19 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3905396115' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:57:19.883 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:19 smithi043 ceph-mon[36454]: 4.11 scrub starts 2024-09-21T11:57:19.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:19 smithi043 ceph-mon[36454]: 4.11 scrub ok 2024-09-21T11:57:19.884 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:19 smithi043 ceph-mon[29710]: pgmap v836: 148 pgs: 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-21T11:57:19.884 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:19 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3905396115' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:57:19.884 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:19 smithi043 ceph-mon[29710]: 4.11 scrub starts 2024-09-21T11:57:19.884 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:19 smithi043 ceph-mon[29710]: 4.11 scrub ok 2024-09-21T11:57:19.939 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":835,"stamp":"2024-09-21T11:57:16.136137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3922467355,"num_objects":60042,"num_object_clones":0,"num_object_copies":180126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60042,"num_whiteouts":0,"num_read":2269,"num_read_kb":2126,"num_write":60678,"num_write_kb":3831625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":1188763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60691,"ondisk_log_size":60691,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":16465500,"kb_used_data":11418076,"kb_used_omap":12,"kb_used_meta":5047283,"kb_avail":733495716,"statfs":{"total":767960285184,"available":751099613184,"internally_reserved":0,"allocated":11692109824,"data_stored":11688723081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":5168418354},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,6,13,4,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1107361792,"num_objects":16897,"num_object_clones":0,"num_object_copies":50691,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16897,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16897,"num_write_kb":1081408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.079176"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":88,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067563+0000","last_change":"2024-09-21T11:55:28.565270+0000","last_active":"2024-09-21T11:57:05.067563+0000","last_peered":"2024-09-21T11:57:05.067563+0000","last_clean":"2024-09-21T11:57:05.067563+0000","last_became_active":"2024-09-21T11:55:28.564399+0000","last_became_peered":"2024-09-21T11:55:28.564399+0000","last_unstale":"2024-09-21T11:57:05.067563+0000","last_undegraded":"2024-09-21T11:57:05.067563+0000","last_fullsized":"2024-09-21T11:57:05.067563+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.303555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.303555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.026551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":768,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067337+0000","last_change":"2024-09-21T11:54:27.283019+0000","last_active":"2024-09-21T11:57:05.067337+0000","last_peered":"2024-09-21T11:57:05.067337+0000","last_clean":"2024-09-21T11:57:05.067337+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:57:05.067337+0000","last_undegraded":"2024-09-21T11:57:05.067337+0000","last_fullsized":"2024-09-21T11:57:05.067337+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:27.282826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:27.282826+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:26.053550+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":454,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068177+0000","last_change":"2024-09-21T11:56:08.429983+0000","last_active":"2024-09-21T11:57:05.068177+0000","last_peered":"2024-09-21T11:57:05.068177+0000","last_clean":"2024-09-21T11:57:05.068177+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:57:05.068177+0000","last_undegraded":"2024-09-21T11:57:05.068177+0000","last_fullsized":"2024-09-21T11:57:05.068177+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:08.429912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.429912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:17.067807+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":541,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066006+0000","last_change":"2024-09-21T11:51:43.528594+0000","last_active":"2024-09-21T11:57:05.066006+0000","last_peered":"2024-09-21T11:57:05.066006+0000","last_clean":"2024-09-21T11:57:05.066006+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:57:05.066006+0000","last_undegraded":"2024-09-21T11:57:05.066006+0000","last_fullsized":"2024-09-21T11:57:05.066006+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:17.102412+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":748,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062081+0000","last_change":"2024-09-21T11:56:49.693769+0000","last_active":"2024-09-21T11:57:05.062081+0000","last_peered":"2024-09-21T11:57:05.062081+0000","last_clean":"2024-09-21T11:57:05.062081+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:57:05.062081+0000","last_undegraded":"2024-09-21T11:57:05.062081+0000","last_fullsized":"2024-09-21T11:57:05.062081+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:49.693594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:49.693594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:03.909258+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":730,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062171+0000","last_change":"2024-09-21T11:55:25.429625+0000","last_active":"2024-09-21T11:57:05.062171+0000","last_peered":"2024-09-21T11:57:05.062171+0000","last_clean":"2024-09-21T11:57:05.062171+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:57:05.062171+0000","last_undegraded":"2024-09-21T11:57:05.062171+0000","last_fullsized":"2024-09-21T11:57:05.062171+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:25.429559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:25.429559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":784,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.063389+0000","last_change":"2024-09-21T11:56:31.572626+0000","last_active":"2024-09-21T11:57:05.063389+0000","last_peered":"2024-09-21T11:57:05.063389+0000","last_clean":"2024-09-21T11:57:05.063389+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:57:05.063389+0000","last_undegraded":"2024-09-21T11:57:05.063389+0000","last_fullsized":"2024-09-21T11:57:05.063389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:31.572525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:31.572525+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:41.538532+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":584,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:13.961006+0000","last_change":"2024-09-21T11:57:13.961006+0000","last_active":"2024-09-21T11:57:13.961006+0000","last_peered":"2024-09-21T11:57:13.961006+0000","last_clean":"2024-09-21T11:57:13.961006+0000","last_became_active":"2024-09-21T11:55:28.561338+0000","last_became_peered":"2024-09-21T11:55:28.561338+0000","last_unstale":"2024-09-21T11:57:13.961006+0000","last_undegraded":"2024-09-21T11:57:13.961006+0000","last_fullsized":"2024-09-21T11:57:13.961006+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:13.960961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:13.960961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:34.935232+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":727,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067290+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:57:05.067290+0000","last_peered":"2024-09-21T11:57:05.067290+0000","last_clean":"2024-09-21T11:57:05.067290+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:57:05.067290+0000","last_undegraded":"2024-09-21T11:57:05.067290+0000","last_fullsized":"2024-09-21T11:57:05.067290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:22.116605+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":759,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:07.774990+0000","last_change":"2024-09-21T11:57:07.774990+0000","last_active":"2024-09-21T11:57:07.774990+0000","last_peered":"2024-09-21T11:57:07.774990+0000","last_clean":"2024-09-21T11:57:07.774990+0000","last_became_active":"2024-09-21T11:55:28.575920+0000","last_became_peered":"2024-09-21T11:55:28.575920+0000","last_unstale":"2024-09-21T11:57:07.774990+0000","last_undegraded":"2024-09-21T11:57:07.774990+0000","last_fullsized":"2024-09-21T11:57:07.774990+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:07.774892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:07.774892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:14.619408+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":727,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066688+0000","last_change":"2024-09-21T11:55:05.096606+0000","last_active":"2024-09-21T11:57:05.066688+0000","last_peered":"2024-09-21T11:57:05.066688+0000","last_clean":"2024-09-21T11:57:05.066688+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:57:05.066688+0000","last_undegraded":"2024-09-21T11:57:05.066688+0000","last_fullsized":"2024-09-21T11:57:05.066688+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:05.096542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:05.096542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:42.070654+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":404,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.072531+0000","last_change":"2024-09-21T11:56:36.422776+0000","last_active":"2024-09-21T11:57:05.072531+0000","last_peered":"2024-09-21T11:57:05.072531+0000","last_clean":"2024-09-21T11:57:05.072531+0000","last_became_active":"2024-09-21T11:55:28.566921+0000","last_became_peered":"2024-09-21T11:55:28.566921+0000","last_unstale":"2024-09-21T11:57:05.072531+0000","last_undegraded":"2024-09-21T11:57:05.072531+0000","last_fullsized":"2024-09-21T11:57:05.072531+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:36.422726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:36.422726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.958604+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":773,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066415+0000","last_change":"2024-09-21T11:56:02.994041+0000","last_active":"2024-09-21T11:57:05.066415+0000","last_peered":"2024-09-21T11:57:05.066415+0000","last_clean":"2024-09-21T11:57:05.066415+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:57:05.066415+0000","last_undegraded":"2024-09-21T11:57:05.066415+0000","last_fullsized":"2024-09-21T11:57:05.066415+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.993966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.993966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:09.442277+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":761,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062480+0000","last_change":"2024-09-21T11:56:59.659683+0000","last_active":"2024-09-21T11:57:05.062480+0000","last_peered":"2024-09-21T11:57:05.062480+0000","last_clean":"2024-09-21T11:57:05.062480+0000","last_became_active":"2024-09-21T11:56:16.667750+0000","last_became_peered":"2024-09-21T11:56:16.667750+0000","last_unstale":"2024-09-21T11:57:05.062480+0000","last_undegraded":"2024-09-21T11:57:05.062480+0000","last_fullsized":"2024-09-21T11:57:05.062480+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'7","last_scrub_stamp":"2024-09-21T11:56:59.659576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.659576+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:01.258966+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":733,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.064195+0000","last_change":"2024-09-21T11:49:23.452007+0000","last_active":"2024-09-21T11:57:05.064195+0000","last_peered":"2024-09-21T11:57:05.064195+0000","last_clean":"2024-09-21T11:57:05.064195+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:57:05.064195+0000","last_undegraded":"2024-09-21T11:57:05.064195+0000","last_fullsized":"2024-09-21T11:57:05.064195+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:23.451910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:23.451910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:12.606428+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":738,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066718+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:57:05.066718+0000","last_peered":"2024-09-21T11:57:05.066718+0000","last_clean":"2024-09-21T11:57:05.066718+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:57:05.066718+0000","last_undegraded":"2024-09-21T11:57:05.066718+0000","last_fullsized":"2024-09-21T11:57:05.066718+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:19.025012+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":247,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067518+0000","last_change":"2024-09-21T11:56:34.073296+0000","last_active":"2024-09-21T11:57:05.067518+0000","last_peered":"2024-09-21T11:57:05.067518+0000","last_clean":"2024-09-21T11:57:05.067518+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:57:05.067518+0000","last_undegraded":"2024-09-21T11:57:05.067518+0000","last_fullsized":"2024-09-21T11:57:05.067518+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:34.073249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:34.073249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:46.860083+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":88,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.064901+0000","last_change":"2024-09-21T11:55:28.562731+0000","last_active":"2024-09-21T11:57:05.064901+0000","last_peered":"2024-09-21T11:57:05.064901+0000","last_clean":"2024-09-21T11:57:05.064901+0000","last_became_active":"2024-09-21T11:55:28.562057+0000","last_became_peered":"2024-09-21T11:55:28.562057+0000","last_unstale":"2024-09-21T11:57:05.064901+0000","last_undegraded":"2024-09-21T11:57:05.064901+0000","last_fullsized":"2024-09-21T11:57:05.064901+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:27.176320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:27.176320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:43.343459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":883,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066312+0000","last_change":"2024-09-21T11:56:01.647675+0000","last_active":"2024-09-21T11:57:05.066312+0000","last_peered":"2024-09-21T11:57:05.066312+0000","last_clean":"2024-09-21T11:57:05.066312+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T11:57:05.066312+0000","last_undegraded":"2024-09-21T11:57:05.066312+0000","last_fullsized":"2024-09-21T11:57:05.066312+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:01.647580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:01.647580+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:23.282522+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":453,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068034+0000","last_change":"2024-09-21T11:56:16.669857+0000","last_active":"2024-09-21T11:57:05.068034+0000","last_peered":"2024-09-21T11:57:05.068034+0000","last_clean":"2024-09-21T11:57:05.068034+0000","last_became_active":"2024-09-21T11:56:16.669551+0000","last_became_peered":"2024-09-21T11:56:16.669551+0000","last_unstale":"2024-09-21T11:57:05.068034+0000","last_undegraded":"2024-09-21T11:57:05.068034+0000","last_fullsized":"2024-09-21T11:57:05.068034+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:56.501356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:56.501356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:14.605337+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":742,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062349+0000","last_change":"2024-09-21T11:57:04.647951+0000","last_active":"2024-09-21T11:57:05.062349+0000","last_peered":"2024-09-21T11:57:05.062349+0000","last_clean":"2024-09-21T11:57:05.062349+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:57:05.062349+0000","last_undegraded":"2024-09-21T11:57:05.062349+0000","last_fullsized":"2024-09-21T11:57:05.062349+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:04.647749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:04.647749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:13.218840+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":561,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067427+0000","last_change":"2024-09-21T11:55:57.442188+0000","last_active":"2024-09-21T11:57:05.067427+0000","last_peered":"2024-09-21T11:57:05.067427+0000","last_clean":"2024-09-21T11:57:05.067427+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:57:05.067427+0000","last_undegraded":"2024-09-21T11:57:05.067427+0000","last_fullsized":"2024-09-21T11:57:05.067427+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:57.441965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.441965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:22.964661+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":802,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.061684+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:57:05.061684+0000","last_peered":"2024-09-21T11:57:05.061684+0000","last_clean":"2024-09-21T11:57:05.061684+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:57:05.061684+0000","last_undegraded":"2024-09-21T11:57:05.061684+0000","last_fullsized":"2024-09-21T11:57:05.061684+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:47.692625+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":254,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.061570+0000","last_change":"2024-09-21T11:56:33.487190+0000","last_active":"2024-09-21T11:57:05.061570+0000","last_peered":"2024-09-21T11:57:05.061570+0000","last_clean":"2024-09-21T11:57:05.061570+0000","last_became_active":"2024-09-21T11:56:16.666765+0000","last_became_peered":"2024-09-21T11:56:16.666765+0000","last_unstale":"2024-09-21T11:57:05.061570+0000","last_undegraded":"2024-09-21T11:57:05.061570+0000","last_fullsized":"2024-09-21T11:57:05.061570+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.487132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.487132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:03.363944+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":242,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.069179+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:57:05.069179+0000","last_peered":"2024-09-21T11:57:05.069179+0000","last_clean":"2024-09-21T11:57:05.069179+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:57:05.069179+0000","last_undegraded":"2024-09-21T11:57:05.069179+0000","last_fullsized":"2024-09-21T11:57:05.069179+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:23.147359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":469,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:10.530989+0000","last_change":"2024-09-21T11:57:10.530989+0000","last_active":"2024-09-21T11:57:10.530989+0000","last_peered":"2024-09-21T11:57:10.530989+0000","last_clean":"2024-09-21T11:57:10.530989+0000","last_became_active":"2024-09-21T11:55:28.568598+0000","last_became_peered":"2024-09-21T11:55:28.568598+0000","last_unstale":"2024-09-21T11:57:10.530989+0000","last_undegraded":"2024-09-21T11:57:10.530989+0000","last_fullsized":"2024-09-21T11:57:10.530989+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:10.530913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.530913+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:13.614764+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":771,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.070323+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:57:05.070323+0000","last_peered":"2024-09-21T11:57:05.070323+0000","last_clean":"2024-09-21T11:57:05.070323+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:57:05.070323+0000","last_undegraded":"2024-09-21T11:57:05.070323+0000","last_fullsized":"2024-09-21T11:57:05.070323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:13.139725+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"383'9","reported_seq":984,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:10.577733+0000","last_change":"2024-09-21T11:56:16.677508+0000","last_active":"2024-09-21T11:57:10.577733+0000","last_peered":"2024-09-21T11:57:10.577733+0000","last_clean":"2024-09-21T11:57:10.577733+0000","last_became_active":"2024-09-21T11:56:16.673446+0000","last_became_peered":"2024-09-21T11:56:16.673446+0000","last_unstale":"2024-09-21T11:57:10.577733+0000","last_undegraded":"2024-09-21T11:57:10.577733+0000","last_fullsized":"2024-09-21T11:57:10.577733+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:57.232943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.232943+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:20.776566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"402'3704","reported_seq":3735,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:11.591442+0000","last_change":"2024-09-21T11:56:36.507880+0000","last_active":"2024-09-21T11:57:11.591442+0000","last_peered":"2024-09-21T11:57:11.591442+0000","last_clean":"2024-09-21T11:57:11.591442+0000","last_became_active":"2024-09-21T11:56:36.507606+0000","last_became_peered":"2024-09-21T11:56:36.507606+0000","last_unstale":"2024-09-21T11:57:11.591442+0000","last_undegraded":"2024-09-21T11:57:11.591442+0000","last_fullsized":"2024-09-21T11:57:11.591442+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3704,"log_dups_size":0,"ondisk_log_size":3704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:44.536721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242745344,"num_objects":3704,"num_object_clones":0,"num_object_copies":11112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3704,"num_write_kb":237056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":480,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068400+0000","last_change":"2024-09-21T11:55:52.562992+0000","last_active":"2024-09-21T11:57:05.068400+0000","last_peered":"2024-09-21T11:57:05.068400+0000","last_clean":"2024-09-21T11:57:05.068400+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:57:05.068400+0000","last_undegraded":"2024-09-21T11:57:05.068400+0000","last_fullsized":"2024-09-21T11:57:05.068400+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:55:52.562928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.562928+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:22.482120+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":767,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:13.738771+0000","last_change":"2024-09-21T11:57:13.738771+0000","last_active":"2024-09-21T11:57:13.738771+0000","last_peered":"2024-09-21T11:57:13.738771+0000","last_clean":"2024-09-21T11:57:13.738771+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:57:13.738771+0000","last_undegraded":"2024-09-21T11:57:13.738771+0000","last_fullsized":"2024-09-21T11:57:13.738771+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:13.738727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:13.738727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:34.168236+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"366'10","reported_seq":351,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.577035+0000","last_change":"2024-09-21T11:56:48.961086+0000","last_active":"2024-09-21T11:57:05.577035+0000","last_peered":"2024-09-21T11:57:05.577035+0000","last_clean":"2024-09-21T11:57:05.577035+0000","last_became_active":"2024-09-21T11:55:28.575209+0000","last_became_peered":"2024-09-21T11:55:28.575209+0000","last_unstale":"2024-09-21T11:57:05.577035+0000","last_undegraded":"2024-09-21T11:57:05.577035+0000","last_fullsized":"2024-09-21T11:57:05.577035+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"366'10","last_scrub_stamp":"2024-09-21T11:56:48.961017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.961017+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:14.388477+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":777,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066906+0000","last_change":"2024-09-21T11:56:57.993660+0000","last_active":"2024-09-21T11:57:05.066906+0000","last_peered":"2024-09-21T11:57:05.066906+0000","last_clean":"2024-09-21T11:57:05.066906+0000","last_became_active":"2024-09-21T11:55:28.564378+0000","last_became_peered":"2024-09-21T11:55:28.564378+0000","last_unstale":"2024-09-21T11:57:05.066906+0000","last_undegraded":"2024-09-21T11:57:05.066906+0000","last_fullsized":"2024-09-21T11:57:05.066906+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:57.993615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:57.993615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:10.442348+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"402'3723","reported_seq":3754,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:11.266058+0000","last_change":"2024-09-21T11:56:36.511425+0000","last_active":"2024-09-21T11:57:11.266058+0000","last_peered":"2024-09-21T11:57:11.266058+0000","last_clean":"2024-09-21T11:57:11.266058+0000","last_became_active":"2024-09-21T11:56:36.511191+0000","last_became_peered":"2024-09-21T11:56:36.511191+0000","last_unstale":"2024-09-21T11:57:11.266058+0000","last_undegraded":"2024-09-21T11:57:11.266058+0000","last_fullsized":"2024-09-21T11:57:11.266058+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3723,"log_dups_size":0,"ondisk_log_size":3723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:04.769503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243990528,"num_objects":3723,"num_object_clones":0,"num_object_copies":11169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3723,"num_write_kb":238272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":833,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.064950+0000","last_change":"2024-09-21T11:57:00.632121+0000","last_active":"2024-09-21T11:57:05.064950+0000","last_peered":"2024-09-21T11:57:05.064950+0000","last_clean":"2024-09-21T11:57:05.064950+0000","last_became_active":"2024-09-21T11:55:28.563496+0000","last_became_peered":"2024-09-21T11:55:28.563496+0000","last_unstale":"2024-09-21T11:57:05.064950+0000","last_undegraded":"2024-09-21T11:57:05.064950+0000","last_fullsized":"2024-09-21T11:57:05.064950+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:57:00.631769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:00.631769+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:05.924170+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":761,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.063948+0000","last_change":"2024-09-21T11:56:53.701703+0000","last_active":"2024-09-21T11:57:05.063948+0000","last_peered":"2024-09-21T11:57:05.063948+0000","last_clean":"2024-09-21T11:57:05.063948+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:57:05.063948+0000","last_undegraded":"2024-09-21T11:57:05.063948+0000","last_fullsized":"2024-09-21T11:57:05.063948+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.701639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.701639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:10.075609+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":239,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062735+0000","last_change":"2024-09-21T11:56:06.506517+0000","last_active":"2024-09-21T11:57:05.062735+0000","last_peered":"2024-09-21T11:57:05.062735+0000","last_clean":"2024-09-21T11:57:05.062735+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:57:05.062735+0000","last_undegraded":"2024-09-21T11:57:05.062735+0000","last_fullsized":"2024-09-21T11:57:05.062735+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:06.506449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:06.506449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:31.003771+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":241,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065578+0000","last_change":"2024-09-21T11:54:53.844954+0000","last_active":"2024-09-21T11:57:05.065578+0000","last_peered":"2024-09-21T11:57:05.065578+0000","last_clean":"2024-09-21T11:57:05.065578+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:57:05.065578+0000","last_undegraded":"2024-09-21T11:57:05.065578+0000","last_fullsized":"2024-09-21T11:57:05.065578+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.844876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.844876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.755477+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"402'3538","reported_seq":3568,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:11.600595+0000","last_change":"2024-09-21T11:56:36.508433+0000","last_active":"2024-09-21T11:57:11.600595+0000","last_peered":"2024-09-21T11:57:11.600595+0000","last_clean":"2024-09-21T11:57:11.600595+0000","last_became_active":"2024-09-21T11:56:36.507746+0000","last_became_peered":"2024-09-21T11:56:36.507746+0000","last_unstale":"2024-09-21T11:57:11.600595+0000","last_undegraded":"2024-09-21T11:57:11.600595+0000","last_fullsized":"2024-09-21T11:57:11.600595+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3538,"log_dups_size":0,"ondisk_log_size":3538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:54.005355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231866368,"num_objects":3538,"num_object_clones":0,"num_object_copies":10614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3538,"num_write_kb":226432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":766,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066764+0000","last_change":"2024-09-21T11:56:16.682561+0000","last_active":"2024-09-21T11:57:05.066764+0000","last_peered":"2024-09-21T11:57:05.066764+0000","last_clean":"2024-09-21T11:57:05.066764+0000","last_became_active":"2024-09-21T11:56:16.670451+0000","last_became_peered":"2024-09-21T11:56:16.670451+0000","last_unstale":"2024-09-21T11:57:05.066764+0000","last_undegraded":"2024-09-21T11:57:05.066764+0000","last_fullsized":"2024-09-21T11:57:05.066764+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:21.067144+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":434,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.071083+0000","last_change":"2024-09-21T11:56:48.446718+0000","last_active":"2024-09-21T11:57:05.071083+0000","last_peered":"2024-09-21T11:57:05.071083+0000","last_clean":"2024-09-21T11:57:05.071083+0000","last_became_active":"2024-09-21T11:55:28.567133+0000","last_became_peered":"2024-09-21T11:55:28.567133+0000","last_unstale":"2024-09-21T11:57:05.071083+0000","last_undegraded":"2024-09-21T11:57:05.071083+0000","last_fullsized":"2024-09-21T11:57:05.071083+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:48.446635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.446635+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:16.560147+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":977,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:10.580390+0000","last_change":"2024-09-21T11:56:05.567407+0000","last_active":"2024-09-21T11:57:10.580390+0000","last_peered":"2024-09-21T11:57:10.580390+0000","last_clean":"2024-09-21T11:57:10.580390+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:57:10.580390+0000","last_undegraded":"2024-09-21T11:57:10.580390+0000","last_fullsized":"2024-09-21T11:57:10.580390+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:05.567332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.567332+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.902664+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":751,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.064772+0000","last_change":"2024-09-21T11:56:34.581966+0000","last_active":"2024-09-21T11:57:05.064772+0000","last_peered":"2024-09-21T11:57:05.064772+0000","last_clean":"2024-09-21T11:57:05.064772+0000","last_became_active":"2024-09-21T11:56:34.581626+0000","last_became_peered":"2024-09-21T11:56:34.581626+0000","last_unstale":"2024-09-21T11:57:05.064772+0000","last_undegraded":"2024-09-21T11:57:05.064772+0000","last_fullsized":"2024-09-21T11:57:05.064772+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.643323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.643323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:21.060965+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"402'3630","reported_seq":3661,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:11.757714+0000","last_change":"2024-09-21T11:56:36.623897+0000","last_active":"2024-09-21T11:57:11.757714+0000","last_peered":"2024-09-21T11:57:11.757714+0000","last_clean":"2024-09-21T11:57:11.757714+0000","last_became_active":"2024-09-21T11:56:36.623581+0000","last_became_peered":"2024-09-21T11:56:36.623581+0000","last_unstale":"2024-09-21T11:57:11.757714+0000","last_undegraded":"2024-09-21T11:57:11.757714+0000","last_fullsized":"2024-09-21T11:57:11.757714+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3630,"log_dups_size":0,"ondisk_log_size":3630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:46.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237895680,"num_objects":3630,"num_object_clones":0,"num_object_copies":10890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3630,"num_write_kb":232320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":492,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067714+0000","last_change":"2024-09-21T11:56:16.679360+0000","last_active":"2024-09-21T11:57:05.067714+0000","last_peered":"2024-09-21T11:57:05.067714+0000","last_clean":"2024-09-21T11:57:05.067714+0000","last_became_active":"2024-09-21T11:56:16.668951+0000","last_became_peered":"2024-09-21T11:56:16.668951+0000","last_unstale":"2024-09-21T11:57:05.067714+0000","last_undegraded":"2024-09-21T11:57:05.067714+0000","last_fullsized":"2024-09-21T11:57:05.067714+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:56:09.421193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.421193+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:32.891630+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":777,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067370+0000","last_change":"2024-09-21T11:56:51.036609+0000","last_active":"2024-09-21T11:57:05.067370+0000","last_peered":"2024-09-21T11:57:05.067370+0000","last_clean":"2024-09-21T11:57:05.067370+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:57:05.067370+0000","last_undegraded":"2024-09-21T11:57:05.067370+0000","last_fullsized":"2024-09-21T11:57:05.067370+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:51.036535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:51.036535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:11.806537+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":955,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.578055+0000","last_change":"2024-09-21T11:55:26.222904+0000","last_active":"2024-09-21T11:57:05.578055+0000","last_peered":"2024-09-21T11:57:05.578055+0000","last_clean":"2024-09-21T11:57:05.578055+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:57:05.578055+0000","last_undegraded":"2024-09-21T11:57:05.578055+0000","last_fullsized":"2024-09-21T11:57:05.578055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:26.222823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:26.222823+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.191943+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":741,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.069009+0000","last_change":"2024-09-21T11:53:48.256119+0000","last_active":"2024-09-21T11:57:05.069009+0000","last_peered":"2024-09-21T11:57:05.069009+0000","last_clean":"2024-09-21T11:57:05.069009+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:57:05.069009+0000","last_undegraded":"2024-09-21T11:57:05.069009+0000","last_fullsized":"2024-09-21T11:57:05.069009+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:48.255951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:48.255951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:25.212487+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"402'3657","reported_seq":3688,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:11.590002+0000","last_change":"2024-09-21T11:56:36.508895+0000","last_active":"2024-09-21T11:57:11.590002+0000","last_peered":"2024-09-21T11:57:11.590002+0000","last_clean":"2024-09-21T11:57:11.590002+0000","last_became_active":"2024-09-21T11:56:36.508626+0000","last_became_peered":"2024-09-21T11:56:36.508626+0000","last_unstale":"2024-09-21T11:57:11.590002+0000","last_undegraded":"2024-09-21T11:57:11.590002+0000","last_fullsized":"2024-09-21T11:57:11.590002+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3657,"log_dups_size":0,"ondisk_log_size":3657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:50.842122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239665152,"num_objects":3657,"num_object_clones":0,"num_object_copies":10971,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3657,"num_write_kb":234048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":808,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.069597+0000","last_change":"2024-09-21T11:56:35.177700+0000","last_active":"2024-09-21T11:57:05.069597+0000","last_peered":"2024-09-21T11:57:05.069597+0000","last_clean":"2024-09-21T11:57:05.069597+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:57:05.069597+0000","last_undegraded":"2024-09-21T11:57:05.069597+0000","last_fullsized":"2024-09-21T11:57:05.069597+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:56:35.177639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.177639+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:45.157349+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":446,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.069213+0000","last_change":"2024-09-21T11:56:58.434536+0000","last_active":"2024-09-21T11:57:05.069213+0000","last_peered":"2024-09-21T11:57:05.069213+0000","last_clean":"2024-09-21T11:57:05.069213+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:57:05.069213+0000","last_undegraded":"2024-09-21T11:57:05.069213+0000","last_fullsized":"2024-09-21T11:57:05.069213+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.434464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.434464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:21.450424+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":729,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067452+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:57:05.067452+0000","last_peered":"2024-09-21T11:57:05.067452+0000","last_clean":"2024-09-21T11:57:05.067452+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:57:05.067452+0000","last_undegraded":"2024-09-21T11:57:05.067452+0000","last_fullsized":"2024-09-21T11:57:05.067452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:13.302731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":438,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068030+0000","last_change":"2024-09-21T11:55:51.602665+0000","last_active":"2024-09-21T11:57:05.068030+0000","last_peered":"2024-09-21T11:57:05.068030+0000","last_clean":"2024-09-21T11:57:05.068030+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:57:05.068030+0000","last_undegraded":"2024-09-21T11:57:05.068030+0000","last_fullsized":"2024-09-21T11:57:05.068030+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.602587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.602587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.913702+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"402'3924","reported_seq":3953,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:15.816653+0000","last_change":"2024-09-21T11:56:36.510297+0000","last_active":"2024-09-21T11:57:15.816653+0000","last_peered":"2024-09-21T11:57:15.816653+0000","last_clean":"2024-09-21T11:57:15.816653+0000","last_became_active":"2024-09-21T11:56:36.510018+0000","last_became_peered":"2024-09-21T11:56:36.510018+0000","last_unstale":"2024-09-21T11:57:15.816653+0000","last_undegraded":"2024-09-21T11:57:15.816653+0000","last_fullsized":"2024-09-21T11:57:15.816653+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3924,"log_dups_size":0,"ondisk_log_size":3924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:04.522929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":257163264,"num_objects":3924,"num_object_clones":0,"num_object_copies":11772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3924,"num_write_kb":251136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":584,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.577304+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:57:05.577304+0000","last_peered":"2024-09-21T11:57:05.577304+0000","last_clean":"2024-09-21T11:57:05.577304+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:57:05.577304+0000","last_undegraded":"2024-09-21T11:57:05.577304+0000","last_fullsized":"2024-09-21T11:57:05.577304+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:19.535620+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":264,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.069755+0000","last_change":"2024-09-21T11:56:29.248925+0000","last_active":"2024-09-21T11:57:05.069755+0000","last_peered":"2024-09-21T11:57:05.069755+0000","last_clean":"2024-09-21T11:57:05.069755+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:57:05.069755+0000","last_undegraded":"2024-09-21T11:57:05.069755+0000","last_fullsized":"2024-09-21T11:57:05.069755+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:29.248875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.248875+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:51.923814+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":769,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066456+0000","last_change":"2024-09-21T11:56:28.083398+0000","last_active":"2024-09-21T11:57:05.066456+0000","last_peered":"2024-09-21T11:57:05.066456+0000","last_clean":"2024-09-21T11:57:05.066456+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:57:05.066456+0000","last_undegraded":"2024-09-21T11:57:05.066456+0000","last_fullsized":"2024-09-21T11:57:05.066456+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:56:28.083324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.083324+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.861453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":408,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067623+0000","last_change":"2024-09-21T11:56:16.667228+0000","last_active":"2024-09-21T11:57:05.067623+0000","last_peered":"2024-09-21T11:57:05.067623+0000","last_clean":"2024-09-21T11:57:05.067623+0000","last_became_active":"2024-09-21T11:56:16.666883+0000","last_became_peered":"2024-09-21T11:56:16.666883+0000","last_unstale":"2024-09-21T11:57:05.067623+0000","last_undegraded":"2024-09-21T11:57:05.067623+0000","last_fullsized":"2024-09-21T11:57:05.067623+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:18.570168+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":717,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.061957+0000","last_change":"2024-09-21T11:54:34.452033+0000","last_active":"2024-09-21T11:57:05.061957+0000","last_peered":"2024-09-21T11:57:05.061957+0000","last_clean":"2024-09-21T11:57:05.061957+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:57:05.061957+0000","last_undegraded":"2024-09-21T11:57:05.061957+0000","last_fullsized":"2024-09-21T11:57:05.061957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:34.451939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:34.451939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:40.548652+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"402'3936","reported_seq":3967,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:15.812062+0000","last_change":"2024-09-21T11:56:36.509094+0000","last_active":"2024-09-21T11:57:15.812062+0000","last_peered":"2024-09-21T11:57:15.812062+0000","last_clean":"2024-09-21T11:57:15.812062+0000","last_became_active":"2024-09-21T11:56:36.508840+0000","last_became_peered":"2024-09-21T11:56:36.508840+0000","last_unstale":"2024-09-21T11:57:15.812062+0000","last_undegraded":"2024-09-21T11:57:15.812062+0000","last_fullsized":"2024-09-21T11:57:15.812062+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3936,"log_dups_size":0,"ondisk_log_size":3936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:01.292057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":257949696,"num_objects":3936,"num_object_clones":0,"num_object_copies":11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3936,"num_write_kb":251904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":408,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:08.488118+0000","last_change":"2024-09-21T11:57:08.488118+0000","last_active":"2024-09-21T11:57:08.488118+0000","last_peered":"2024-09-21T11:57:08.488118+0000","last_clean":"2024-09-21T11:57:08.488118+0000","last_became_active":"2024-09-21T11:55:28.569095+0000","last_became_peered":"2024-09-21T11:55:28.569095+0000","last_unstale":"2024-09-21T11:57:08.488118+0000","last_undegraded":"2024-09-21T11:57:08.488118+0000","last_fullsized":"2024-09-21T11:57:08.488118+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:57:08.488050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:08.488050+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:38.067318+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":443,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.071304+0000","last_change":"2024-09-21T11:55:06.628254+0000","last_active":"2024-09-21T11:57:05.071304+0000","last_peered":"2024-09-21T11:57:05.071304+0000","last_clean":"2024-09-21T11:57:05.071304+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:57:05.071304+0000","last_undegraded":"2024-09-21T11:57:05.071304+0000","last_fullsized":"2024-09-21T11:57:05.071304+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:55:06.628029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:06.628029+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:40.416603+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":771,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065402+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:57:05.065402+0000","last_peered":"2024-09-21T11:57:05.065402+0000","last_clean":"2024-09-21T11:57:05.065402+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:57:05.065402+0000","last_undegraded":"2024-09-21T11:57:05.065402+0000","last_fullsized":"2024-09-21T11:57:05.065402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:10.681786+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":737,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.069119+0000","last_change":"2024-09-21T11:55:06.871048+0000","last_active":"2024-09-21T11:57:05.069119+0000","last_peered":"2024-09-21T11:57:05.069119+0000","last_clean":"2024-09-21T11:57:05.069119+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:57:05.069119+0000","last_undegraded":"2024-09-21T11:57:05.069119+0000","last_fullsized":"2024-09-21T11:57:05.069119+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:06.870710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:06.870710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.731490+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":741,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065163+0000","last_change":"2024-09-21T11:56:10.595430+0000","last_active":"2024-09-21T11:57:05.065163+0000","last_peered":"2024-09-21T11:57:05.065163+0000","last_clean":"2024-09-21T11:57:05.065163+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:57:05.065163+0000","last_undegraded":"2024-09-21T11:57:05.065163+0000","last_fullsized":"2024-09-21T11:57:05.065163+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:10.595373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:10.595373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:19.061656+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"402'3951","reported_seq":3982,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:15.808117+0000","last_change":"2024-09-21T11:56:36.509416+0000","last_active":"2024-09-21T11:57:15.808117+0000","last_peered":"2024-09-21T11:57:15.808117+0000","last_clean":"2024-09-21T11:57:15.808117+0000","last_became_active":"2024-09-21T11:56:36.509115+0000","last_became_peered":"2024-09-21T11:56:36.509115+0000","last_unstale":"2024-09-21T11:57:15.808117+0000","last_undegraded":"2024-09-21T11:57:15.808117+0000","last_fullsized":"2024-09-21T11:57:15.808117+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:39.857708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258932736,"num_objects":3951,"num_object_clones":0,"num_object_copies":11853,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3951,"num_write_kb":252864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":624,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.071177+0000","last_change":"2024-09-21T11:55:28.581087+0000","last_active":"2024-09-21T11:57:05.071177+0000","last_peered":"2024-09-21T11:57:05.071177+0000","last_clean":"2024-09-21T11:57:05.071177+0000","last_became_active":"2024-09-21T11:55:28.566613+0000","last_became_peered":"2024-09-21T11:55:28.566613+0000","last_unstale":"2024-09-21T11:57:05.071177+0000","last_undegraded":"2024-09-21T11:57:05.071177+0000","last_fullsized":"2024-09-21T11:57:05.071177+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:55:10.349240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:10.349240+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:44.472182+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":776,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068898+0000","last_change":"2024-09-21T11:57:01.083611+0000","last_active":"2024-09-21T11:57:05.068898+0000","last_peered":"2024-09-21T11:57:05.068898+0000","last_clean":"2024-09-21T11:57:05.068898+0000","last_became_active":"2024-09-21T11:55:28.572037+0000","last_became_peered":"2024-09-21T11:55:28.572037+0000","last_unstale":"2024-09-21T11:57:05.068898+0000","last_undegraded":"2024-09-21T11:57:05.068898+0000","last_fullsized":"2024-09-21T11:57:05.068898+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:01.083437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.083437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:09.754225+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1817,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:09.519313+0000","last_change":"2024-09-21T11:56:09.604887+0000","last_active":"2024-09-21T11:57:09.519313+0000","last_peered":"2024-09-21T11:57:09.519313+0000","last_clean":"2024-09-21T11:57:09.519313+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:57:09.519313+0000","last_undegraded":"2024-09-21T11:57:09.519313+0000","last_fullsized":"2024-09-21T11:57:09.519313+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:56:09.604808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.604808+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:21.526417+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1021,"num_read_kb":1016,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":758,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067483+0000","last_change":"2024-09-21T11:56:57.019113+0000","last_active":"2024-09-21T11:57:05.067483+0000","last_peered":"2024-09-21T11:57:05.067483+0000","last_clean":"2024-09-21T11:57:05.067483+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:57:05.067483+0000","last_undegraded":"2024-09-21T11:57:05.067483+0000","last_fullsized":"2024-09-21T11:57:05.067483+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:57.019066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:57.019066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:23.845888+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":438,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068941+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:57:05.068941+0000","last_peered":"2024-09-21T11:57:05.068941+0000","last_clean":"2024-09-21T11:57:05.068941+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:57:05.068941+0000","last_undegraded":"2024-09-21T11:57:05.068941+0000","last_fullsized":"2024-09-21T11:57:05.068941+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:25.527742+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"402'3955","reported_seq":3986,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:15.811778+0000","last_change":"2024-09-21T11:56:36.509270+0000","last_active":"2024-09-21T11:57:15.811778+0000","last_peered":"2024-09-21T11:57:15.811778+0000","last_clean":"2024-09-21T11:57:15.811778+0000","last_became_active":"2024-09-21T11:56:36.508533+0000","last_became_peered":"2024-09-21T11:56:36.508533+0000","last_unstale":"2024-09-21T11:57:15.811778+0000","last_undegraded":"2024-09-21T11:57:15.811778+0000","last_fullsized":"2024-09-21T11:57:15.811778+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3955,"log_dups_size":0,"ondisk_log_size":3955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.121205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259194880,"num_objects":3955,"num_object_clones":0,"num_object_copies":11865,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3955,"num_write_kb":253120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":997,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:10.577753+0000","last_change":"2024-09-21T11:56:29.095770+0000","last_active":"2024-09-21T11:57:10.577753+0000","last_peered":"2024-09-21T11:57:10.577753+0000","last_clean":"2024-09-21T11:57:10.577753+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:57:10.577753+0000","last_undegraded":"2024-09-21T11:57:10.577753+0000","last_fullsized":"2024-09-21T11:57:10.577753+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:56:29.095713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.095713+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.293114+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":754,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:14.756298+0000","last_change":"2024-09-21T11:57:14.756298+0000","last_active":"2024-09-21T11:57:14.756298+0000","last_peered":"2024-09-21T11:57:14.756298+0000","last_clean":"2024-09-21T11:57:14.756298+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:57:14.756298+0000","last_undegraded":"2024-09-21T11:57:14.756298+0000","last_fullsized":"2024-09-21T11:57:14.756298+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:14.756227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:14.756227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:28.632117+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":831,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065793+0000","last_change":"2024-09-21T11:56:07.609933+0000","last_active":"2024-09-21T11:57:05.065793+0000","last_peered":"2024-09-21T11:57:05.065793+0000","last_clean":"2024-09-21T11:57:05.065793+0000","last_became_active":"2024-09-21T11:55:28.570670+0000","last_became_peered":"2024-09-21T11:55:28.570670+0000","last_unstale":"2024-09-21T11:57:05.065793+0000","last_undegraded":"2024-09-21T11:57:05.065793+0000","last_fullsized":"2024-09-21T11:57:05.065793+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:56:07.609858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:56:07.609858+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:35.001556+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":737,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065800+0000","last_change":"2024-09-21T11:51:18.318095+0000","last_active":"2024-09-21T11:57:05.065800+0000","last_peered":"2024-09-21T11:57:05.065800+0000","last_clean":"2024-09-21T11:57:05.065800+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:57:05.065800+0000","last_undegraded":"2024-09-21T11:57:05.065800+0000","last_fullsized":"2024-09-21T11:57:05.065800+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.317950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.317950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:22.592787+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":756,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.904684+0000","last_change":"2024-09-21T11:57:05.904684+0000","last_active":"2024-09-21T11:57:05.904684+0000","last_peered":"2024-09-21T11:57:05.904684+0000","last_clean":"2024-09-21T11:57:05.904684+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:57:05.904684+0000","last_undegraded":"2024-09-21T11:57:05.904684+0000","last_fullsized":"2024-09-21T11:57:05.904684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:05.904621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:05.904621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:19.946798+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"402'4013","reported_seq":4044,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:15.812074+0000","last_change":"2024-09-21T11:56:36.508185+0000","last_active":"2024-09-21T11:57:15.812074+0000","last_peered":"2024-09-21T11:57:15.812074+0000","last_clean":"2024-09-21T11:57:15.812074+0000","last_became_active":"2024-09-21T11:56:36.507849+0000","last_became_peered":"2024-09-21T11:56:36.507849+0000","last_unstale":"2024-09-21T11:57:15.812074+0000","last_undegraded":"2024-09-21T11:57:15.812074+0000","last_fullsized":"2024-09-21T11:57:15.812074+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.525964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262995968,"num_objects":4013,"num_object_clones":0,"num_object_copies":12039,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4013,"num_write_kb":256832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":762,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067619+0000","last_change":"2024-09-21T11:56:09.204336+0000","last_active":"2024-09-21T11:57:05.067619+0000","last_peered":"2024-09-21T11:57:05.067619+0000","last_clean":"2024-09-21T11:57:05.067619+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:57:05.067619+0000","last_undegraded":"2024-09-21T11:57:05.067619+0000","last_fullsized":"2024-09-21T11:57:05.067619+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:56:09.204270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.204270+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:17.886802+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":254,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065263+0000","last_change":"2024-09-21T11:56:16.671887+0000","last_active":"2024-09-21T11:57:05.065263+0000","last_peered":"2024-09-21T11:57:05.065263+0000","last_clean":"2024-09-21T11:57:05.065263+0000","last_became_active":"2024-09-21T11:56:16.671572+0000","last_became_peered":"2024-09-21T11:56:16.671572+0000","last_unstale":"2024-09-21T11:57:05.065263+0000","last_undegraded":"2024-09-21T11:57:05.065263+0000","last_fullsized":"2024-09-21T11:57:05.065263+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:05.663102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.663102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:27.961529+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":743,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068259+0000","last_change":"2024-09-21T11:56:30.997926+0000","last_active":"2024-09-21T11:57:05.068259+0000","last_peered":"2024-09-21T11:57:05.068259+0000","last_clean":"2024-09-21T11:57:05.068259+0000","last_became_active":"2024-09-21T11:56:16.671360+0000","last_became_peered":"2024-09-21T11:56:16.671360+0000","last_unstale":"2024-09-21T11:57:05.068259+0000","last_undegraded":"2024-09-21T11:57:05.068259+0000","last_fullsized":"2024-09-21T11:57:05.068259+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:30.997880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:30.997880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:34.105058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":747,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062097+0000","last_change":"2024-09-21T11:56:28.556475+0000","last_active":"2024-09-21T11:57:05.062097+0000","last_peered":"2024-09-21T11:57:05.062097+0000","last_clean":"2024-09-21T11:57:05.062097+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:57:05.062097+0000","last_undegraded":"2024-09-21T11:57:05.062097+0000","last_fullsized":"2024-09-21T11:57:05.062097+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.556423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.556423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:33.767058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"402'3658","reported_seq":3689,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:11.725646+0000","last_change":"2024-09-21T11:56:36.511935+0000","last_active":"2024-09-21T11:57:11.725646+0000","last_peered":"2024-09-21T11:57:11.725646+0000","last_clean":"2024-09-21T11:57:11.725646+0000","last_became_active":"2024-09-21T11:56:36.511642+0000","last_became_peered":"2024-09-21T11:56:36.511642+0000","last_unstale":"2024-09-21T11:57:11.725646+0000","last_undegraded":"2024-09-21T11:57:11.725646+0000","last_fullsized":"2024-09-21T11:57:11.725646+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3658,"log_dups_size":0,"ondisk_log_size":3658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:43.996894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239730688,"num_objects":3658,"num_object_clones":0,"num_object_copies":10974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3658,"num_write_kb":234112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":850,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067561+0000","last_change":"2024-09-21T11:56:03.668325+0000","last_active":"2024-09-21T11:57:05.067561+0000","last_peered":"2024-09-21T11:57:05.067561+0000","last_clean":"2024-09-21T11:57:05.067561+0000","last_became_active":"2024-09-21T11:55:28.561575+0000","last_became_peered":"2024-09-21T11:55:28.561575+0000","last_unstale":"2024-09-21T11:57:05.067561+0000","last_undegraded":"2024-09-21T11:57:05.067561+0000","last_fullsized":"2024-09-21T11:57:05.067561+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:56:03.667750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:03.667750+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:16.766955+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":774,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067805+0000","last_change":"2024-09-21T11:56:28.045735+0000","last_active":"2024-09-21T11:57:05.067805+0000","last_peered":"2024-09-21T11:57:05.067805+0000","last_clean":"2024-09-21T11:57:05.067805+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:57:05.067805+0000","last_undegraded":"2024-09-21T11:57:05.067805+0000","last_fullsized":"2024-09-21T11:57:05.067805+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.045670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.045670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:35.259699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":438,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068856+0000","last_change":"2024-09-21T11:55:10.605948+0000","last_active":"2024-09-21T11:57:05.068856+0000","last_peered":"2024-09-21T11:57:05.068856+0000","last_clean":"2024-09-21T11:57:05.068856+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:57:05.068856+0000","last_undegraded":"2024-09-21T11:57:05.068856+0000","last_fullsized":"2024-09-21T11:57:05.068856+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:10.605869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:10.605869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.452804+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":741,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068113+0000","last_change":"2024-09-21T11:55:24.583429+0000","last_active":"2024-09-21T11:57:05.068113+0000","last_peered":"2024-09-21T11:57:05.068113+0000","last_clean":"2024-09-21T11:57:05.068113+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:57:05.068113+0000","last_undegraded":"2024-09-21T11:57:05.068113+0000","last_fullsized":"2024-09-21T11:57:05.068113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.583208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.583208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:41.672664+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"402'3867","reported_seq":3896,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:15.755463+0000","last_change":"2024-09-21T11:56:36.504822+0000","last_active":"2024-09-21T11:57:15.755463+0000","last_peered":"2024-09-21T11:57:15.755463+0000","last_clean":"2024-09-21T11:57:15.755463+0000","last_became_active":"2024-09-21T11:56:36.504516+0000","last_became_peered":"2024-09-21T11:56:36.504516+0000","last_unstale":"2024-09-21T11:57:15.755463+0000","last_undegraded":"2024-09-21T11:57:15.755463+0000","last_fullsized":"2024-09-21T11:57:15.755463+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3867,"log_dups_size":0,"ondisk_log_size":3867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:54.965928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253427712,"num_objects":3867,"num_object_clones":0,"num_object_copies":11601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3867,"num_write_kb":247488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":731,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.061822+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:57:05.061822+0000","last_peered":"2024-09-21T11:57:05.061822+0000","last_clean":"2024-09-21T11:57:05.061822+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:57:05.061822+0000","last_undegraded":"2024-09-21T11:57:05.061822+0000","last_fullsized":"2024-09-21T11:57:05.061822+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:22.507278+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":744,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062875+0000","last_change":"2024-09-21T11:56:09.486510+0000","last_active":"2024-09-21T11:57:05.062875+0000","last_peered":"2024-09-21T11:57:05.062875+0000","last_clean":"2024-09-21T11:57:05.062875+0000","last_became_active":"2024-09-21T11:55:28.568312+0000","last_became_peered":"2024-09-21T11:55:28.568312+0000","last_unstale":"2024-09-21T11:57:05.062875+0000","last_undegraded":"2024-09-21T11:57:05.062875+0000","last_fullsized":"2024-09-21T11:57:05.062875+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:09.486386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.486386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:29.172746+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":968,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.576668+0000","last_change":"2024-09-21T11:56:08.680971+0000","last_active":"2024-09-21T11:57:05.576668+0000","last_peered":"2024-09-21T11:57:05.576668+0000","last_clean":"2024-09-21T11:57:05.576668+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:57:05.576668+0000","last_undegraded":"2024-09-21T11:57:05.576668+0000","last_fullsized":"2024-09-21T11:57:05.576668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:56:08.680766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.680766+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:16.508041+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":565,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.071557+0000","last_change":"2024-09-21T11:56:16.669124+0000","last_active":"2024-09-21T11:57:05.071557+0000","last_peered":"2024-09-21T11:57:05.071557+0000","last_clean":"2024-09-21T11:57:05.071557+0000","last_became_active":"2024-09-21T11:56:16.668785+0000","last_became_peered":"2024-09-21T11:56:16.668785+0000","last_unstale":"2024-09-21T11:57:05.071557+0000","last_undegraded":"2024-09-21T11:57:05.071557+0000","last_fullsized":"2024-09-21T11:57:05.071557+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:07.658022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:07.658022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:42.380428+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"402'3585","reported_seq":3616,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:11.220216+0000","last_change":"2024-09-21T11:56:36.508655+0000","last_active":"2024-09-21T11:57:11.220216+0000","last_peered":"2024-09-21T11:57:11.220216+0000","last_clean":"2024-09-21T11:57:11.220216+0000","last_became_active":"2024-09-21T11:56:36.508299+0000","last_became_peered":"2024-09-21T11:56:36.508299+0000","last_unstale":"2024-09-21T11:57:11.220216+0000","last_undegraded":"2024-09-21T11:57:11.220216+0000","last_fullsized":"2024-09-21T11:57:11.220216+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3585,"log_dups_size":0,"ondisk_log_size":3585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:05.125216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234946560,"num_objects":3585,"num_object_clones":0,"num_object_copies":10755,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3585,"num_write_kb":229440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":791,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.070092+0000","last_change":"2024-09-21T11:56:48.169371+0000","last_active":"2024-09-21T11:57:05.070092+0000","last_peered":"2024-09-21T11:57:05.070092+0000","last_clean":"2024-09-21T11:57:05.070092+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:57:05.070092+0000","last_undegraded":"2024-09-21T11:57:05.070092+0000","last_fullsized":"2024-09-21T11:57:05.070092+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:48.168795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.168795+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:09.022696+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":776,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066557+0000","last_change":"2024-09-21T11:56:59.989715+0000","last_active":"2024-09-21T11:57:05.066557+0000","last_peered":"2024-09-21T11:57:05.066557+0000","last_clean":"2024-09-21T11:57:05.066557+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:57:05.066557+0000","last_undegraded":"2024-09-21T11:57:05.066557+0000","last_fullsized":"2024-09-21T11:57:05.066557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:59.989636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.989636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:25.546942+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":736,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065850+0000","last_change":"2024-09-21T11:56:16.668904+0000","last_active":"2024-09-21T11:57:05.065850+0000","last_peered":"2024-09-21T11:57:05.065850+0000","last_clean":"2024-09-21T11:57:05.065850+0000","last_became_active":"2024-09-21T11:56:16.668588+0000","last_became_peered":"2024-09-21T11:56:16.668588+0000","last_unstale":"2024-09-21T11:57:05.065850+0000","last_undegraded":"2024-09-21T11:57:05.065850+0000","last_fullsized":"2024-09-21T11:57:05.065850+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.990626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.990626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:18.813963+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":95,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.063911+0000","last_change":"2024-09-21T11:55:35.532779+0000","last_active":"2024-09-21T11:57:05.063911+0000","last_peered":"2024-09-21T11:57:05.063911+0000","last_clean":"2024-09-21T11:57:05.063911+0000","last_became_active":"2024-09-21T11:55:28.566108+0000","last_became_peered":"2024-09-21T11:55:28.566108+0000","last_unstale":"2024-09-21T11:57:05.063911+0000","last_undegraded":"2024-09-21T11:57:05.063911+0000","last_fullsized":"2024-09-21T11:57:05.063911+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:35.532577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:35.532577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:42.362223+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"402'3526","reported_seq":3557,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:11.261525+0000","last_change":"2024-09-21T11:56:36.506979+0000","last_active":"2024-09-21T11:57:11.261525+0000","last_peered":"2024-09-21T11:57:11.261525+0000","last_clean":"2024-09-21T11:57:11.261525+0000","last_became_active":"2024-09-21T11:56:36.506694+0000","last_became_peered":"2024-09-21T11:56:36.506694+0000","last_unstale":"2024-09-21T11:57:11.261525+0000","last_undegraded":"2024-09-21T11:57:11.261525+0000","last_fullsized":"2024-09-21T11:57:11.261525+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3526,"log_dups_size":0,"ondisk_log_size":3526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.639288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231079936,"num_objects":3526,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3526,"num_write_kb":225664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":849,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067577+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:57:05.067577+0000","last_peered":"2024-09-21T11:57:05.067577+0000","last_clean":"2024-09-21T11:57:05.067577+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:57:05.067577+0000","last_undegraded":"2024-09-21T11:57:05.067577+0000","last_fullsized":"2024-09-21T11:57:05.067577+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:18.201979+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":744,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065761+0000","last_change":"2024-09-21T11:56:50.313230+0000","last_active":"2024-09-21T11:57:05.065761+0000","last_peered":"2024-09-21T11:57:05.065761+0000","last_clean":"2024-09-21T11:57:05.065761+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:57:05.065761+0000","last_undegraded":"2024-09-21T11:57:05.065761+0000","last_fullsized":"2024-09-21T11:57:05.065761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.313165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.313165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:09.383551+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":745,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.064727+0000","last_change":"2024-09-21T11:51:48.941202+0000","last_active":"2024-09-21T11:57:05.064727+0000","last_peered":"2024-09-21T11:57:05.064727+0000","last_clean":"2024-09-21T11:57:05.064727+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:57:05.064727+0000","last_undegraded":"2024-09-21T11:57:05.064727+0000","last_fullsized":"2024-09-21T11:57:05.064727+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:48.941128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:48.941128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:24.866192+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":748,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065705+0000","last_change":"2024-09-21T11:56:55.414105+0000","last_active":"2024-09-21T11:57:05.065705+0000","last_peered":"2024-09-21T11:57:05.065705+0000","last_clean":"2024-09-21T11:57:05.065705+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:57:05.065705+0000","last_undegraded":"2024-09-21T11:57:05.065705+0000","last_fullsized":"2024-09-21T11:57:05.065705+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:55.414046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:55.414046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:01.463812+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"402'3647","reported_seq":3675,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:11.780999+0000","last_change":"2024-09-21T11:56:36.623904+0000","last_active":"2024-09-21T11:57:11.780999+0000","last_peered":"2024-09-21T11:57:11.780999+0000","last_clean":"2024-09-21T11:57:11.780999+0000","last_became_active":"2024-09-21T11:56:36.623611+0000","last_became_peered":"2024-09-21T11:56:36.623611+0000","last_unstale":"2024-09-21T11:57:11.780999+0000","last_undegraded":"2024-09-21T11:57:11.780999+0000","last_fullsized":"2024-09-21T11:57:11.780999+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3647,"log_dups_size":0,"ondisk_log_size":3647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.371771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239009792,"num_objects":3647,"num_object_clones":0,"num_object_copies":10941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3647,"num_write_kb":233408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":771,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068518+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:57:05.068518+0000","last_peered":"2024-09-21T11:57:05.068518+0000","last_clean":"2024-09-21T11:57:05.068518+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:57:05.068518+0000","last_undegraded":"2024-09-21T11:57:05.068518+0000","last_fullsized":"2024-09-21T11:57:05.068518+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:16.543488+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":789,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065430+0000","last_change":"2024-09-21T11:56:29.486116+0000","last_active":"2024-09-21T11:57:05.065430+0000","last_peered":"2024-09-21T11:57:05.065430+0000","last_clean":"2024-09-21T11:57:05.065430+0000","last_became_active":"2024-09-21T11:56:16.670969+0000","last_became_peered":"2024-09-21T11:56:16.670969+0000","last_unstale":"2024-09-21T11:57:05.065430+0000","last_undegraded":"2024-09-21T11:57:05.065430+0000","last_fullsized":"2024-09-21T11:57:05.065430+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:29.486067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.486067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:47.036894+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":759,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.064849+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:57:05.064849+0000","last_peered":"2024-09-21T11:57:05.064849+0000","last_clean":"2024-09-21T11:57:05.064849+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:57:05.064849+0000","last_undegraded":"2024-09-21T11:57:05.064849+0000","last_fullsized":"2024-09-21T11:57:05.064849+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:13.728141+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":747,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062312+0000","last_change":"2024-09-21T11:56:02.541452+0000","last_active":"2024-09-21T11:57:05.062312+0000","last_peered":"2024-09-21T11:57:05.062312+0000","last_clean":"2024-09-21T11:57:05.062312+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:57:05.062312+0000","last_undegraded":"2024-09-21T11:57:05.062312+0000","last_fullsized":"2024-09-21T11:57:05.062312+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.541232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.541232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:16.599825+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"402'3529","reported_seq":3560,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:11.438469+0000","last_change":"2024-09-21T11:56:36.624091+0000","last_active":"2024-09-21T11:57:11.438469+0000","last_peered":"2024-09-21T11:57:11.438469+0000","last_clean":"2024-09-21T11:57:11.438469+0000","last_became_active":"2024-09-21T11:56:36.623765+0000","last_became_peered":"2024-09-21T11:56:36.623765+0000","last_unstale":"2024-09-21T11:57:11.438469+0000","last_undegraded":"2024-09-21T11:57:11.438469+0000","last_fullsized":"2024-09-21T11:57:11.438469+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":3529,"log_dups_size":0,"ondisk_log_size":3529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:59.412761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231276544,"num_objects":3529,"num_object_clones":0,"num_object_copies":10587,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3529,"num_write_kb":225856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":791,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066177+0000","last_change":"2024-09-21T11:56:29.063211+0000","last_active":"2024-09-21T11:57:05.066177+0000","last_peered":"2024-09-21T11:57:05.066177+0000","last_clean":"2024-09-21T11:57:05.066177+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:57:05.066177+0000","last_undegraded":"2024-09-21T11:57:05.066177+0000","last_fullsized":"2024-09-21T11:57:05.066177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:29.063150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.063150+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:45.931544+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":793,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065911+0000","last_change":"2024-09-21T11:57:01.222171+0000","last_active":"2024-09-21T11:57:05.065911+0000","last_peered":"2024-09-21T11:57:05.065911+0000","last_clean":"2024-09-21T11:57:05.065911+0000","last_became_active":"2024-09-21T11:55:28.561948+0000","last_became_peered":"2024-09-21T11:55:28.561948+0000","last_unstale":"2024-09-21T11:57:05.065911+0000","last_undegraded":"2024-09-21T11:57:05.065911+0000","last_fullsized":"2024-09-21T11:57:05.065911+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:57:01.221960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.221960+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:23.325733+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":967,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:10.282224+0000","last_change":"2024-09-21T11:57:10.282180+0000","last_active":"2024-09-21T11:57:10.282224+0000","last_peered":"2024-09-21T11:57:10.282224+0000","last_clean":"2024-09-21T11:57:10.282224+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:57:10.282224+0000","last_undegraded":"2024-09-21T11:57:10.282224+0000","last_fullsized":"2024-09-21T11:57:10.282224+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:10.282134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.282134+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:12.540343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":724,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066108+0000","last_change":"2024-09-21T11:53:27.992553+0000","last_active":"2024-09-21T11:57:05.066108+0000","last_peered":"2024-09-21T11:57:05.066108+0000","last_clean":"2024-09-21T11:57:05.066108+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:57:05.066108+0000","last_undegraded":"2024-09-21T11:57:05.066108+0000","last_fullsized":"2024-09-21T11:57:05.066108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.992489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.992489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:41.096226+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":767,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068570+0000","last_change":"2024-09-21T11:55:28.557481+0000","last_active":"2024-09-21T11:57:05.068570+0000","last_peered":"2024-09-21T11:57:05.068570+0000","last_clean":"2024-09-21T11:57:05.068570+0000","last_became_active":"2024-09-21T11:55:28.557190+0000","last_became_peered":"2024-09-21T11:55:28.557190+0000","last_unstale":"2024-09-21T11:57:05.068570+0000","last_undegraded":"2024-09-21T11:57:05.068570+0000","last_fullsized":"2024-09-21T11:57:05.068570+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:38.208809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.208809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.969866+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":781,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.063806+0000","last_change":"2024-09-21T11:56:29.528662+0000","last_active":"2024-09-21T11:57:05.063806+0000","last_peered":"2024-09-21T11:57:05.063806+0000","last_clean":"2024-09-21T11:57:05.063806+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:57:05.063806+0000","last_undegraded":"2024-09-21T11:57:05.063806+0000","last_fullsized":"2024-09-21T11:57:05.063806+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:56:29.528598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.528598+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:47.850219+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":774,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066491+0000","last_change":"2024-09-21T11:55:54.003787+0000","last_active":"2024-09-21T11:57:05.066491+0000","last_peered":"2024-09-21T11:57:05.066491+0000","last_clean":"2024-09-21T11:57:05.066491+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:57:05.066491+0000","last_undegraded":"2024-09-21T11:57:05.066491+0000","last_fullsized":"2024-09-21T11:57:05.066491+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:54.003722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:54.003722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:06.305761+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":763,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065073+0000","last_change":"2024-09-21T11:57:04.198898+0000","last_active":"2024-09-21T11:57:05.065073+0000","last_peered":"2024-09-21T11:57:05.065073+0000","last_clean":"2024-09-21T11:57:05.065073+0000","last_became_active":"2024-09-21T11:56:16.670467+0000","last_became_peered":"2024-09-21T11:56:16.670467+0000","last_unstale":"2024-09-21T11:57:05.065073+0000","last_undegraded":"2024-09-21T11:57:05.065073+0000","last_fullsized":"2024-09-21T11:57:05.065073+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:04.198818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:04.198818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:16.880979+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":774,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.070491+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:57:05.070491+0000","last_peered":"2024-09-21T11:57:05.070491+0000","last_clean":"2024-09-21T11:57:05.070491+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:57:05.070491+0000","last_undegraded":"2024-09-21T11:57:05.070491+0000","last_fullsized":"2024-09-21T11:57:05.070491+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:18.135115+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":736,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.064663+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:57:05.064663+0000","last_peered":"2024-09-21T11:57:05.064663+0000","last_clean":"2024-09-21T11:57:05.064663+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:57:05.064663+0000","last_undegraded":"2024-09-21T11:57:05.064663+0000","last_fullsized":"2024-09-21T11:57:05.064663+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:25.568680+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":767,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.063647+0000","last_change":"2024-09-21T11:56:50.696344+0000","last_active":"2024-09-21T11:57:05.063647+0000","last_peered":"2024-09-21T11:57:05.063647+0000","last_clean":"2024-09-21T11:57:05.063647+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:57:05.063647+0000","last_undegraded":"2024-09-21T11:57:05.063647+0000","last_fullsized":"2024-09-21T11:57:05.063647+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.696253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.696253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:00.421085+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":733,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065953+0000","last_change":"2024-09-21T11:49:48.001924+0000","last_active":"2024-09-21T11:57:05.065953+0000","last_peered":"2024-09-21T11:57:05.065953+0000","last_clean":"2024-09-21T11:57:05.065953+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:57:05.065953+0000","last_undegraded":"2024-09-21T11:57:05.065953+0000","last_fullsized":"2024-09-21T11:57:05.065953+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:48.001860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:48.001860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:11.666188+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":773,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067196+0000","last_change":"2024-09-21T11:56:34.119200+0000","last_active":"2024-09-21T11:57:05.067196+0000","last_peered":"2024-09-21T11:57:05.067196+0000","last_clean":"2024-09-21T11:57:05.067196+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:57:05.067196+0000","last_undegraded":"2024-09-21T11:57:05.067196+0000","last_fullsized":"2024-09-21T11:57:05.067196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:34.119128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:34.119128+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.898211+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":760,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.072310+0000","last_change":"2024-09-21T11:55:04.304207+0000","last_active":"2024-09-21T11:57:05.072310+0000","last_peered":"2024-09-21T11:57:05.072310+0000","last_clean":"2024-09-21T11:57:05.072310+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:57:05.072310+0000","last_undegraded":"2024-09-21T11:57:05.072310+0000","last_fullsized":"2024-09-21T11:57:05.072310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:04.304136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:04.304136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:45.199932+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":937,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:10.579775+0000","last_change":"2024-09-21T11:55:51.317564+0000","last_active":"2024-09-21T11:57:10.579775+0000","last_peered":"2024-09-21T11:57:10.579775+0000","last_clean":"2024-09-21T11:57:10.579775+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:57:10.579775+0000","last_undegraded":"2024-09-21T11:57:10.579775+0000","last_fullsized":"2024-09-21T11:57:10.579775+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:51.317473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.317473+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":231,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068029+0000","last_change":"2024-09-21T11:51:43.531318+0000","last_active":"2024-09-21T11:57:05.068029+0000","last_peered":"2024-09-21T11:57:05.068029+0000","last_clean":"2024-09-21T11:57:05.068029+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:57:05.068029+0000","last_undegraded":"2024-09-21T11:57:05.068029+0000","last_fullsized":"2024-09-21T11:57:05.068029+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.826834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.826834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.778367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":793,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.061908+0000","last_change":"2024-09-21T11:53:25.264037+0000","last_active":"2024-09-21T11:57:05.061908+0000","last_peered":"2024-09-21T11:57:05.061908+0000","last_clean":"2024-09-21T11:57:05.061908+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:57:05.061908+0000","last_undegraded":"2024-09-21T11:57:05.061908+0000","last_fullsized":"2024-09-21T11:57:05.061908+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:25.263955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:25.263955+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.464657+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":744,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.064995+0000","last_change":"2024-09-21T11:56:33.473303+0000","last_active":"2024-09-21T11:57:05.064995+0000","last_peered":"2024-09-21T11:57:05.064995+0000","last_clean":"2024-09-21T11:57:05.064995+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:57:05.064995+0000","last_undegraded":"2024-09-21T11:57:05.064995+0000","last_fullsized":"2024-09-21T11:57:05.064995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.473240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.473240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.182237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":990,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.576666+0000","last_change":"2024-09-21T11:56:56.051487+0000","last_active":"2024-09-21T11:57:05.576666+0000","last_peered":"2024-09-21T11:57:05.576666+0000","last_clean":"2024-09-21T11:57:05.576666+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:57:05.576666+0000","last_undegraded":"2024-09-21T11:57:05.576666+0000","last_fullsized":"2024-09-21T11:57:05.576666+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:56.051300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:56.051300+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:11.899165+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":103,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065167+0000","last_change":"2024-09-21T11:56:58.280761+0000","last_active":"2024-09-21T11:57:05.065167+0000","last_peered":"2024-09-21T11:57:05.065167+0000","last_clean":"2024-09-21T11:57:05.065167+0000","last_became_active":"2024-09-21T11:55:28.565700+0000","last_became_peered":"2024-09-21T11:55:28.565700+0000","last_unstale":"2024-09-21T11:57:05.065167+0000","last_undegraded":"2024-09-21T11:57:05.065167+0000","last_fullsized":"2024-09-21T11:57:05.065167+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.280698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.280698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:01.265389+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":781,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067798+0000","last_change":"2024-09-21T11:51:43.533694+0000","last_active":"2024-09-21T11:57:05.067798+0000","last_peered":"2024-09-21T11:57:05.067798+0000","last_clean":"2024-09-21T11:57:05.067798+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:57:05.067798+0000","last_undegraded":"2024-09-21T11:57:05.067798+0000","last_fullsized":"2024-09-21T11:57:05.067798+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:17.242122+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":247,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066469+0000","last_change":"2024-09-21T11:55:24.538954+0000","last_active":"2024-09-21T11:57:05.066469+0000","last_peered":"2024-09-21T11:57:05.066469+0000","last_clean":"2024-09-21T11:57:05.066469+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:57:05.066469+0000","last_undegraded":"2024-09-21T11:57:05.066469+0000","last_fullsized":"2024-09-21T11:57:05.066469+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.538880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.538880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:40.390211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":446,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067636+0000","last_change":"2024-09-21T11:56:00.441644+0000","last_active":"2024-09-21T11:57:05.067636+0000","last_peered":"2024-09-21T11:57:05.067636+0000","last_clean":"2024-09-21T11:57:05.067636+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:57:05.067636+0000","last_undegraded":"2024-09-21T11:57:05.067636+0000","last_fullsized":"2024-09-21T11:57:05.067636+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:00.441565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:00.441565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:27.350768+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":732,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062635+0000","last_change":"2024-09-21T11:55:28.563670+0000","last_active":"2024-09-21T11:57:05.062635+0000","last_peered":"2024-09-21T11:57:05.062635+0000","last_clean":"2024-09-21T11:57:05.062635+0000","last_became_active":"2024-09-21T11:55:28.563340+0000","last_became_peered":"2024-09-21T11:55:28.563340+0000","last_unstale":"2024-09-21T11:57:05.062635+0000","last_undegraded":"2024-09-21T11:57:05.062635+0000","last_fullsized":"2024-09-21T11:57:05.062635+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:01.434123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.434123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:43.602222+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":457,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067863+0000","last_change":"2024-09-21T11:55:59.419239+0000","last_active":"2024-09-21T11:57:05.067863+0000","last_peered":"2024-09-21T11:57:05.067863+0000","last_clean":"2024-09-21T11:57:05.067863+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:57:05.067863+0000","last_undegraded":"2024-09-21T11:57:05.067863+0000","last_fullsized":"2024-09-21T11:57:05.067863+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:59.419148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.419148+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:24.349135+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":736,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065362+0000","last_change":"2024-09-21T11:49:46.502816+0000","last_active":"2024-09-21T11:57:05.065362+0000","last_peered":"2024-09-21T11:57:05.065362+0000","last_clean":"2024-09-21T11:57:05.065362+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:57:05.065362+0000","last_undegraded":"2024-09-21T11:57:05.065362+0000","last_fullsized":"2024-09-21T11:57:05.065362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:46.502776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:49:46.502776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.630324+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":743,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062400+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:57:05.062400+0000","last_peered":"2024-09-21T11:57:05.062400+0000","last_clean":"2024-09-21T11:57:05.062400+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:57:05.062400+0000","last_undegraded":"2024-09-21T11:57:05.062400+0000","last_fullsized":"2024-09-21T11:57:05.062400+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:19.549269+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":771,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068851+0000","last_change":"2024-09-21T11:55:59.288321+0000","last_active":"2024-09-21T11:57:05.068851+0000","last_peered":"2024-09-21T11:57:05.068851+0000","last_clean":"2024-09-21T11:57:05.068851+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:57:05.068851+0000","last_undegraded":"2024-09-21T11:57:05.068851+0000","last_fullsized":"2024-09-21T11:57:05.068851+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:55:59.288240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.288240+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:13.275297+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":772,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062827+0000","last_change":"2024-09-21T11:53:32.308905+0000","last_active":"2024-09-21T11:57:05.062827+0000","last_peered":"2024-09-21T11:57:05.062827+0000","last_clean":"2024-09-21T11:57:05.062827+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:57:05.062827+0000","last_undegraded":"2024-09-21T11:57:05.062827+0000","last_fullsized":"2024-09-21T11:57:05.062827+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:53:32.308827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:32.308827+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:16.498027+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":766,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.066623+0000","last_change":"2024-09-21T11:56:47.967479+0000","last_active":"2024-09-21T11:57:05.066623+0000","last_peered":"2024-09-21T11:57:05.066623+0000","last_clean":"2024-09-21T11:57:05.066623+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:57:05.066623+0000","last_undegraded":"2024-09-21T11:57:05.066623+0000","last_fullsized":"2024-09-21T11:57:05.066623+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:47.967392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:47.967392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:10.636665+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":720,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.063793+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:57:05.063793+0000","last_peered":"2024-09-21T11:57:05.063793+0000","last_clean":"2024-09-21T11:57:05.063793+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:57:05.063793+0000","last_undegraded":"2024-09-21T11:57:05.063793+0000","last_fullsized":"2024-09-21T11:57:05.063793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:19.625346+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":740,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.062772+0000","last_change":"2024-09-21T11:55:28.564578+0000","last_active":"2024-09-21T11:57:05.062772+0000","last_peered":"2024-09-21T11:57:05.062772+0000","last_clean":"2024-09-21T11:57:05.062772+0000","last_became_active":"2024-09-21T11:55:28.564261+0000","last_became_peered":"2024-09-21T11:55:28.564261+0000","last_unstale":"2024-09-21T11:57:05.062772+0000","last_undegraded":"2024-09-21T11:57:05.062772+0000","last_fullsized":"2024-09-21T11:57:05.062772+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.551852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.551852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:39.540862+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":784,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.064867+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:57:05.064867+0000","last_peered":"2024-09-21T11:57:05.064867+0000","last_clean":"2024-09-21T11:57:05.064867+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:57:05.064867+0000","last_undegraded":"2024-09-21T11:57:05.064867+0000","last_fullsized":"2024-09-21T11:57:05.064867+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:25.518080+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":736,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.069650+0000","last_change":"2024-09-21T11:56:53.232038+0000","last_active":"2024-09-21T11:57:05.069650+0000","last_peered":"2024-09-21T11:57:05.069650+0000","last_clean":"2024-09-21T11:57:05.069650+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:57:05.069650+0000","last_undegraded":"2024-09-21T11:57:05.069650+0000","last_fullsized":"2024-09-21T11:57:05.069650+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.231963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.231963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:57.228542+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":743,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.068235+0000","last_change":"2024-09-21T11:55:52.217954+0000","last_active":"2024-09-21T11:57:05.068235+0000","last_peered":"2024-09-21T11:57:05.068235+0000","last_clean":"2024-09-21T11:57:05.068235+0000","last_became_active":"2024-09-21T11:55:28.575256+0000","last_became_peered":"2024-09-21T11:55:28.575256+0000","last_unstale":"2024-09-21T11:57:05.068235+0000","last_undegraded":"2024-09-21T11:57:05.068235+0000","last_fullsized":"2024-09-21T11:57:05.068235+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:52.217898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.217898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:15.160221+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":740,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.064116+0000","last_change":"2024-09-21T11:54:57.381670+0000","last_active":"2024-09-21T11:57:05.064116+0000","last_peered":"2024-09-21T11:57:05.064116+0000","last_clean":"2024-09-21T11:57:05.064116+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:57:05.064116+0000","last_undegraded":"2024-09-21T11:57:05.064116+0000","last_fullsized":"2024-09-21T11:57:05.064116+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:57.381605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:57.381605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.460530+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":445,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067895+0000","last_change":"2024-09-21T11:56:16.669730+0000","last_active":"2024-09-21T11:57:05.067895+0000","last_peered":"2024-09-21T11:57:05.067895+0000","last_clean":"2024-09-21T11:57:05.067895+0000","last_became_active":"2024-09-21T11:56:16.669407+0000","last_became_peered":"2024-09-21T11:56:16.669407+0000","last_unstale":"2024-09-21T11:57:05.067895+0000","last_undegraded":"2024-09-21T11:57:05.067895+0000","last_fullsized":"2024-09-21T11:57:05.067895+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:24.528780+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":766,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.065091+0000","last_change":"2024-09-21T11:55:28.563845+0000","last_active":"2024-09-21T11:57:05.065091+0000","last_peered":"2024-09-21T11:57:05.065091+0000","last_clean":"2024-09-21T11:57:05.065091+0000","last_became_active":"2024-09-21T11:55:28.563540+0000","last_became_peered":"2024-09-21T11:55:28.563540+0000","last_unstale":"2024-09-21T11:57:05.065091+0000","last_undegraded":"2024-09-21T11:57:05.065091+0000","last_fullsized":"2024-09-21T11:57:05.065091+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:23.601318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.601318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.497705+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":842,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.067948+0000","last_change":"2024-09-21T11:56:33.048379+0000","last_active":"2024-09-21T11:57:05.067948+0000","last_peered":"2024-09-21T11:57:05.067948+0000","last_clean":"2024-09-21T11:57:05.067948+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:57:05.067948+0000","last_undegraded":"2024-09-21T11:57:05.067948+0000","last_fullsized":"2024-09-21T11:57:05.067948+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:56:33.048316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.048316+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.955454+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":737,"reported_epoch":402,"state":"active+clean","last_fresh":"2024-09-21T11:57:05.070210+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:57:05.070210+0000","last_peered":"2024-09-21T11:57:05.070210+0000","last_clean":"2024-09-21T11:57:05.070210+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:57:05.070210+0000","last_undegraded":"2024-09-21T11:57:05.070210+0000","last_fullsized":"2024-09-21T11:57:05.070210+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:47.211178+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":3921870848,"num_objects":59843,"num_object_clones":0,"num_object_copies":179529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59843,"num_write_kb":3829952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11773542400,"data_stored":11773542400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59843,"ondisk_log_size":59843,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":6024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":758,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1021,"num_read_kb":1016,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234195,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2268280,"kb_used_data":1615144,"kb_used_omap":1,"kb_used_meta":653118,"kb_avail":91476872,"statfs":{"total":95995035648,"available":93672316928,"internally_reserved":0,"allocated":1653907456,"data_stored":1653471865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":668793290},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463129,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2527460,"kb_used_data":1968724,"kb_used_omap":1,"kb_used_meta":558718,"kb_avail":91217692,"statfs":{"total":95995035648,"available":93406916608,"internally_reserved":0,"allocated":2015973376,"data_stored":2015558389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":572127686},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724765,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1521132,"kb_used_data":908252,"kb_used_omap":1,"kb_used_meta":612862,"kb_avail":92224020,"statfs":{"total":95995035648,"available":94437396480,"internally_reserved":0,"allocated":930050048,"data_stored":929625287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":627571142},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920996,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1269280,"kb_used_data":736912,"kb_used_omap":1,"kb_used_meta":532350,"kb_avail":92475872,"statfs":{"total":95995035648,"available":94695292928,"internally_reserved":0,"allocated":754597888,"data_stored":754186090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":545126853},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117226,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2513504,"kb_used_data":1774288,"kb_used_omap":1,"kb_used_meta":739198,"kb_avail":91231648,"statfs":{"total":95995035648,"available":93421207552,"internally_reserved":0,"allocated":1816870912,"data_stored":1816458315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":756939207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,5,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411567,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2063468,"kb_used_data":1396444,"kb_used_omap":1,"kb_used_meta":667006,"kb_avail":91681684,"statfs":{"total":95995035648,"available":93882044416,"internally_reserved":0,"allocated":1429958656,"data_stored":1429534702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":683014597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575094,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2748564,"kb_used_data":2081540,"kb_used_omap":1,"kb_used_meta":667006,"kb_avail":90996588,"statfs":{"total":95995035648,"available":93180506112,"internally_reserved":0,"allocated":2131496960,"data_stored":2131033122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":683014598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,4,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738620,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1553812,"kb_used_data":936772,"kb_used_omap":1,"kb_used_meta":617022,"kb_avail":92191340,"statfs":{"total":95995035648,"available":94403932160,"internally_reserved":0,"allocated":959254528,"data_stored":958855311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":631830981},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":968032256,"data_stored":968032256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2135425024,"data_stored":2135425024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1440743424,"data_stored":1440743424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1819410432,"data_stored":1819410432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":763428864,"data_stored":763428864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":938803200,"data_stored":938803200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2053832704,"data_stored":2053832704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1653866496,"data_stored":1653866496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:57:19.945 INFO:tasks.thrashosds.thrasher:Setting 4.8 to [1, 0, 2, 5, 3, 7] 2024-09-21T11:57:19.945 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '4.8', '1', '0', '2', '5', '3', '7'] 2024-09-21T11:57:19.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pg-upmap-items 4.8 1 0 2 5 3 7 2024-09-21T11:57:19.964 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:19.990 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:19.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:19.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:20.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:19 smithi175 ceph-mon[32613]: pgmap v836: 148 pgs: 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 91 MiB/s wr, 1.46k op/s 2024-09-21T11:57:20.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:19 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3905396115' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:57:20.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:19 smithi175 ceph-mon[32613]: 4.11 scrub starts 2024-09-21T11:57:20.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:19 smithi175 ceph-mon[32613]: 4.11 scrub ok 2024-09-21T11:57:20.091 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:20.118 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:20.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:20.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:20.220 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:20.247 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:20.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:20.248 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:20.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:20.396 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:20.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:20.397 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:20.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:20.540 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:20.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:20.541 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:20.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:20.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:20.684 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:20.688 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:57:20.688092+0000 min lat: 0.00120284 max lat: 0.292846 avg lat: 0.00936969 2024-09-21T11:57:20.689 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:57:20.689 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 40 16 68293 68277 106.661 79.5625 0.0163261 0.00936969 2024-09-21T11:57:20.784 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:20.815 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:20.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:20.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:20.883 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:57:20.883 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:57:20.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:20 smithi043 ceph-mon[29710]: 6.5 scrub starts 2024-09-21T11:57:20.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:20 smithi043 ceph-mon[29710]: 6.5 scrub ok 2024-09-21T11:57:20.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:20 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:20.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3905396115' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:57:20.884 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:20 smithi043 ceph-mon[29710]: osdmap e403: 8 total, 8 up, 8 in 2024-09-21T11:57:20.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:20 smithi043 ceph-mon[36454]: 6.5 scrub starts 2024-09-21T11:57:20.885 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:20 smithi043 ceph-mon[36454]: 6.5 scrub ok 2024-09-21T11:57:20.885 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:20 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:20.885 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3905396115' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:57:20.885 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:20 smithi043 ceph-mon[36454]: osdmap e403: 8 total, 8 up, 8 in 2024-09-21T11:57:20.917 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:20.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:20.958 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:20.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:21.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:20 smithi175 ceph-mon[32613]: 6.5 scrub starts 2024-09-21T11:57:21.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:20 smithi175 ceph-mon[32613]: 6.5 scrub ok 2024-09-21T11:57:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:20 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3905396115' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:57:21.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:20 smithi175 ceph-mon[32613]: osdmap e403: 8 total, 8 up, 8 in 2024-09-21T11:57:21.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:21.095 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:21.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:21.096 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:21.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:21.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:21.234 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:21.335 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:21.362 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:21.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:21.363 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:21.464 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:21.493 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:21.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:21.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:21.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:21.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:57:21.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:21.656 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:21.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:21.692 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 41 16 69586 69570 106.023 80.8125 0.0250556 0.00942614 2024-09-21T11:57:21.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:21.786 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:21.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:21 smithi043 ceph-mon[36454]: pgmap v838: 148 pgs: 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-21T11:57:21.817 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:21 smithi043 ceph-mon[29710]: pgmap v838: 148 pgs: 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-21T11:57:21.835 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:21.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:21.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:21.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:21.986 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:21.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:21.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:22.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:21 smithi175 ceph-mon[32613]: pgmap v838: 148 pgs: 148 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-21T11:57:22.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:57:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:57:22.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:22.134 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:22.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:22.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:22.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:22.263 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:22.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:22.264 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:22.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:22.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:22.407 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:22.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:22.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:22.553 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:22.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:22.634 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:57:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:57:22.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:22.678 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:57:22.678 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:57:22.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:22.684 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:22.684 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:22.692 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 42 16 70700 70684 105.156 69.625 0.0107583 0.00950415 2024-09-21T11:57:22.712 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:57:22.712 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:57:22.753 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:57:22.753 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:57:22.784 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:22.790 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:57:22.792 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:57:22.792 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:57:22.813 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:22.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:22.814 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:22.843 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:57:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:57:22.876 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:57:22.876 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:57:22.915 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:22.916 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:57:22.917 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:57:22.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:22.966 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:22.966 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:23.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:23.091 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:23.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:23.092 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:23.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:23.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:23.228 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:23.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:23.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:23.372 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:23.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:23.373 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:23.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:23.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:23.511 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:23.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:23.612 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:23.640 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:23.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:23.641 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:23.692 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 43 16 71757 71741 104.247 66.0625 0.0315402 0.00958363 2024-09-21T11:57:23.742 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:23.767 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:23.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:23.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:23.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:23.902 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:23.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:23.903 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[36454]: pgmap v839: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-21T11:57:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2727144122' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:57:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:57:23.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1682423763' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "4.8", "id": [1, 0, 2, 5, 3, 7]} : dispatch 2024-09-21T11:57:23.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "4.8", "id": [1, 0, 2, 5, 3, 7]} : dispatch 2024-09-21T11:57:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[29710]: pgmap v839: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-21T11:57:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2727144122' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:57:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:57:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1682423763' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "4.8", "id": [1, 0, 2, 5, 3, 7]} : dispatch 2024-09-21T11:57:23.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:23 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "4.8", "id": [1, 0, 2, 5, 3, 7]} : dispatch 2024-09-21T11:57:24.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:24.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:23 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:24.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:23 smithi175 ceph-mon[32613]: pgmap v839: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-21T11:57:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2727144122' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:23 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:57:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:23 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:57:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1682423763' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "4.8", "id": [1, 0, 2, 5, 3, 7]} : dispatch 2024-09-21T11:57:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:23 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "4.8", "id": [1, 0, 2, 5, 3, 7]} : dispatch 2024-09-21T11:57:24.031 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:24.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:24.032 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:24.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:24.171 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:24.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:24.172 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:24.272 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:24.302 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:24.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:24.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:24.405 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:24.434 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:24.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:24.435 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:24.444 INFO:teuthology.orchestra.run.smithi043.stderr:set 4.8 pg_upmap_items mapping to [1->0,2->5,3->7] 2024-09-21T11:57:24.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:24.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:24.558 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:24.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:24.694 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 44 16 72889 72873 103.481 70.75 0.0158504 0.00965545 2024-09-21T11:57:24.704 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:24.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:24.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:24 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.8", "id": [1, 0, 2, 5, 3, 7]}]': finished 2024-09-21T11:57:24.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:24 smithi043 ceph-mon[29710]: osdmap e404: 8 total, 8 up, 8 in 2024-09-21T11:57:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:24 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.8", "id": [1, 0, 2, 5, 3, 7]}]': finished 2024-09-21T11:57:24.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:24 smithi043 ceph-mon[36454]: osdmap e404: 8 total, 8 up, 8 in 2024-09-21T11:57:24.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:24.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:24.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:24.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:24.973 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:24.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:24.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:24 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "4.8", "id": [1, 0, 2, 5, 3, 7]}]': finished 2024-09-21T11:57:25.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:24 smithi175 ceph-mon[32613]: osdmap e404: 8 total, 8 up, 8 in 2024-09-21T11:57:25.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:25.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:25.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:25.200 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:25.226 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:25.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:25.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:25.328 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:25.355 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:25.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:25.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:25.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:25.485 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:57:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:57:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:25.496 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:25.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:25.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:25.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:25.629 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:25.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:25.630 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:25.694 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 45 16 74007 73991 102.735 69.875 0.0152015 0.00972863 2024-09-21T11:57:25.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:25.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:25 smithi043 ceph-mon[36454]: pgmap v841: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:57:25.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:25 smithi043 ceph-mon[29710]: pgmap v841: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:57:25.777 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:25.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:25.778 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:25.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:25.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:25.929 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:25 smithi175 ceph-mon[32613]: pgmap v841: 148 pgs: 148 active+clean; 4.2 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-21T11:57:26.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:26.059 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:26.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:26.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:26.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:26.216 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:26.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:26.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:26.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:26.364 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:26.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:26.365 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:26.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:26.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:26.511 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:26.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:26.612 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:26.637 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:26.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:26.638 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:26.671 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:26.694 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 46 16 75069 75053 101.944 66.375 0.0273793 0.00980302 2024-09-21T11:57:26.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:26.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:26.786 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:26.868 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:26 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:57:26.868 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:26 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:57:26.868 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:26 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:57:26.868 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:26 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:26.868 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:26 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:57:26.868 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:26 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:57:26.868 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:26 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:57:26.869 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:26 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:26.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:26.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:26 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:57:26.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:26 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:57:26.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:26 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:57:26.898 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:26 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:26.929 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:26.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:26.930 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:27.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:27.056 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:27.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:27.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:27.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:27.204 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:27.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:27.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:27.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:27.336 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:27.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:27.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:27.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:27.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:27.481 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:27.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:27.544 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:57:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:57:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:57:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:57:27.545 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:57:27.582 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:27.607 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:27.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:27.608 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:27.694 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 47 16 76177 76161 101.248 69.25 0.0198808 0.00987138 2024-09-21T11:57:27.709 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:27.740 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:27.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:27.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:27.842 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:27.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:27.873 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:27.874 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:27.974 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:27 smithi043 ceph-mon[29710]: pgmap v842: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 61 MiB/s wr, 977 op/s 2024-09-21T11:57:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:27 smithi043 ceph-mon[29710]: osdmap e405: 8 total, 8 up, 8 in 2024-09-21T11:57:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:27 smithi043 ceph-mon[36454]: pgmap v842: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 61 MiB/s wr, 977 op/s 2024-09-21T11:57:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:27 smithi043 ceph-mon[36454]: osdmap e405: 8 total, 8 up, 8 in 2024-09-21T11:57:28.001 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:28.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:28.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:28.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:27 smithi175 ceph-mon[32613]: pgmap v842: 148 pgs: 148 active+clean; 4.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 61 MiB/s wr, 977 op/s 2024-09-21T11:57:28.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:27 smithi175 ceph-mon[32613]: osdmap e405: 8 total, 8 up, 8 in 2024-09-21T11:57:28.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:28.142 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:28.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:28.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:28.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:28.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:28.286 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:28.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:28.431 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:28.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:28.432 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:28.533 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:28.562 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:28.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:28.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:57:28.625 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:28 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1637038651' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:28.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:57:28.627 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:28 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1637038651' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:28.631 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:57:28.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:28.697 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 48 16 77324 77308 100.628 71.6875 0.0231592 0.00993136 2024-09-21T11:57:28.725 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:28.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:28.727 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:28.827 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:28.850 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:28.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:28.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:28.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:29.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:28 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1637038651' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:29.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:29.046 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:29.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:29.146 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:29.181 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:29.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:29.182 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:29.283 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:29.315 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:29.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:29.316 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:29.417 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:29.440 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:29.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:29.441 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:29.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:29.588 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:29.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:29.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:29.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:29.698 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 49 15 78430 78415 99.9841 69.1875 0.02562 0.00999643 2024-09-21T11:57:29.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:29 smithi043 ceph-mon[29710]: pgmap v844: 148 pgs: 148 active+clean; 4.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-21T11:57:29.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:29 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:29.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1637038651' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:57:29.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:29 smithi043 ceph-mon[29710]: osdmap e406: 8 total, 8 up, 8 in 2024-09-21T11:57:29.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:29 smithi043 ceph-mon[36454]: pgmap v844: 148 pgs: 148 active+clean; 4.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-21T11:57:29.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:29 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:29.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1637038651' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:57:29.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:29 smithi043 ceph-mon[36454]: osdmap e406: 8 total, 8 up, 8 in 2024-09-21T11:57:29.735 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:29.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:29.736 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:29.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:29.864 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:29.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:29.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:29.966 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:30.000 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:30.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:30.001 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:30.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:29 smithi175 ceph-mon[32613]: pgmap v844: 148 pgs: 148 active+clean; 4.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-21T11:57:30.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:29 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1637038651' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:57:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:29 smithi175 ceph-mon[32613]: osdmap e406: 8 total, 8 up, 8 in 2024-09-21T11:57:30.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:30.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:30.142 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:30.143 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:30.243 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:30.269 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:30.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:30.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:30.333 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:57:30.333 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:57:30.333 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:57:30.333 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:57:30.333 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.333 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:166714 9.0 9:0db06241:::benchmark_data_smithi043_126002_object39648:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:18.113330+0000", 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 372.21366367299999, 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.182493339, 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 166714 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.334 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113330+0000", 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113330+0000", 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113331+0000", 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.068e-06 2024-09-21T11:57:30.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113335+0000", 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6059999999999996e-06 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113349+0000", 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.323e-05 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113356+0000", 2024-09-21T11:57:30.336 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9920000000000001e-06 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113412+0000", 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6793999999999999e-05 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113528+0000", 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115573 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113645+0000", 2024-09-21T11:57:30.337 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116731 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.116389+0000", 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027440640000000001 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.116875+0000", 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00048631400000000001 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.338 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295714+0000", 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17883933099999999 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295772+0000", 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7215e-05 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295823+0000", 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1421000000000001e-05 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:166730 9.0 9:0f1e7941:::benchmark_data_smithi043_126002_object39664:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:18.126334+0000", 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 372.20065961099999, 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17205351699999999, 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 166730 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.340 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126334+0000", 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126334+0000", 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126335+0000", 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9199999999999999e-07 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126340+0000", 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9749999999999998e-06 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126369+0000", 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.955e-05 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.126379+0000", 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4700000000000008e-06 2024-09-21T11:57:30.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.128585+0000", 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022058529999999998 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.128691+0000", 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106638 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.128800+0000", 2024-09-21T11:57:30.343 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108323 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.130823+0000", 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020232050000000001 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.132530+0000", 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0017074970000000001 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.298292+0000", 2024-09-21T11:57:30.344 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.16576127099999999 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.298345+0000", 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3437999999999999e-05 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.298387+0000", 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2404999999999998e-05 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:211057 9.e e296/247 9:722bc66e:::benchmark_data_smithi043_126002_object83991:head v 296'13171, mlcod=296'13169)", 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409320+0000", 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 334.91767321499998, 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29737601699999999, 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409320+0000", 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409320+0000", 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409321+0000", 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0800000000000004e-07 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409325+0000", 2024-09-21T11:57:30.347 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-06 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409333+0000", 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4669999999999998e-06 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409338+0000", 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.609e-06 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.348 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409445+0000", 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106464 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409463+0000", 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8719000000000002e-05 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706603+0000", 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.297139561 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.349 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706696+0000", 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3189000000000005e-05 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:211059 9.0 9:08679dd9:::benchmark_data_smithi043_126002_object83993:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409680+0000", 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 334.91731310400002, 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29635377000000002, 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.350 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 211059 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409680+0000", 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409680+0000", 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409682+0000", 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3379999999999999e-06 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409687+0000", 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4929999999999998e-06 2024-09-21T11:57:30.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409698+0000", 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409706+0000", 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9680000000000006e-06 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.410134+0000", 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000427494 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.410245+0000", 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111241 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.410365+0000", 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120077 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705854+0000", 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29548879099999997 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705865+0000", 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1168999999999999e-05 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705894+0000", 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9088999999999999e-05 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705988+0000", 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4019999999999998e-05 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706034+0000", 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6109000000000002e-05 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:211058 9.d e296/247 9:b1ac4faf:::benchmark_data_smithi043_126002_object83992:head v 296'13146, mlcod=296'13144)", 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409691+0000", 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 334.917302474, 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29170147000000002, 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.356 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409691+0000", 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409691+0000", 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.357 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409692+0000", 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0200000000000001e-07 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409698+0000", 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3860000000000004e-06 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409708+0000", 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3540000000000006e-06 2024-09-21T11:57:30.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409714+0000", 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8120000000000002e-06 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409791+0000", 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6365999999999994e-05 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409815+0000", 2024-09-21T11:57:30.359 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4087000000000002e-05 2024-09-21T11:57:30.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413064+0000", 2024-09-21T11:57:30.360 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032496109999999999 2024-09-21T11:57:30.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.360 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.360 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701392+0000", 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.288328052 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:211060 9.0 9:0749ae3e:::benchmark_data_smithi043_126002_object83994:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409751+0000", 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 334.91724217299998, 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.296470603, 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.361 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 211060 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409751+0000", 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.362 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409751+0000", 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409752+0000", 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9299999999999997e-07 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409756+0000", 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8349999999999997e-06 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409764+0000", 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9580000000000007e-06 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409771+0000", 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6819999999999997e-06 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701791+0000", 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29201998200000001 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.364 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701945+0000", 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000154518 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.702140+0000", 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019451399999999999 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706071+0000", 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039315360000000002 2024-09-21T11:57:30.365 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706081+0000", 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6900000000000004e-06 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706111+0000", 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9796999999999999e-05 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706188+0000", 2024-09-21T11:57:30.366 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7074000000000001e-05 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706222+0000", 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4224000000000003e-05 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.367 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:211067 9.4 9:2a2e5d4d:::benchmark_data_smithi043_126002_object84001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.412385+0000", 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 334.914608751, 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.30543494500000001, 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 211067 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.368 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412385+0000", 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412385+0000", 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412386+0000", 2024-09-21T11:57:30.369 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8029999999999999e-06 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412912+0000", 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052593400000000004 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701342+0000", 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28842914800000002 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.370 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701361+0000", 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9381999999999998e-05 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701424+0000", 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2634999999999999e-05 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701566+0000", 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014276 2024-09-21T11:57:30.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701694+0000", 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012741900000000001 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.707583+0000", 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058892190000000002 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.707636+0000", 2024-09-21T11:57:30.372 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3359999999999997e-05 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.717703+0000", 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010067066 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.717763+0000", 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9849999999999998e-05 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.373 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.717820+0000", 2024-09-21T11:57:30.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6369000000000002e-05 2024-09-21T11:57:30.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.374 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.374 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.374 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.374 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:211066 9.a e296/247 9:51f636b8:::benchmark_data_smithi043_126002_object84000:head v 296'13283, mlcod=296'13281)", 2024-09-21T11:57:30.374 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.412517+0000", 2024-09-21T11:57:30.374 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 334.91447616800002, 2024-09-21T11:57:30.374 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29723240400000001, 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412517+0000", 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412517+0000", 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412518+0000", 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412621+0000", 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102394 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412632+0000", 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0849000000000001e-05 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412639+0000", 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7769999999999997e-06 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.377 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701367+0000", 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28872841500000002 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701394+0000", 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7345999999999999e-05 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.709721+0000", 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083270189999999997 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.378 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.709750+0000", 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8390000000000001e-05 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:211059 9.0 e296/247 ondisk, result = 0)", 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 334.91370076200002, 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.292746124, 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:57:30.379 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:57:30.380 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1399999999999996e-07 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413297+0000", 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8070000000000001e-06 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701343+0000", 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28804594700000002 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.381 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701357+0000", 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3959e-05 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705885+0000", 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0045275940000000002 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705890+0000", 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.382 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706039+0000", 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014890199999999999 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:211059 9.0 e296/247 ondisk, result = 0)", 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.414118+0000", 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 334.91287547100001, 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29174787600000002, 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414118+0000", 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414118+0000", 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414119+0000", 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5899999999999995e-07 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414123+0000", 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5170000000000001e-06 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701349+0000", 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28722547300000001 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701359+0000", 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0664e-05 2024-09-21T11:57:30.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705840+0000", 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044810070000000004 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705848+0000", 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3900000000000004e-06 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705866+0000", 2024-09-21T11:57:30.386 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7966e-05 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:126300 10.b e348/313 10:d6278167:::benchmark_data_smithi043_150678_object3549:head v 348'7832, mlcod=348'7829)", 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 185.093922781, 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.23630744300000001, 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.388 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.388 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:57:30.388 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.389 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.389 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:57:30.389 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1999999999999999e-07 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233078+0000", 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6150000000000002e-06 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233088+0000", 2024-09-21T11:57:30.390 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7499999999999998e-06 2024-09-21T11:57:30.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.391 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:30.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.392 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233094+0000", 2024-09-21T11:57:30.392 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-21T11:57:30.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.392 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233148+0000", 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3943000000000002e-05 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233168+0000", 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9822000000000001e-05 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.469306+0000", 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.236138347 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.393 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.469378+0000", 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2000000000000002e-05 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126310 10.2 10:429ff067:::benchmark_data_smithi043_150678_object3559:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.234619+0000", 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 185.09237432099999, 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.23258517300000001, 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126310 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234619+0000", 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234619+0000", 2024-09-21T11:57:30.395 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234621+0000", 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0660000000000002e-06 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234627+0000", 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3090000000000001e-06 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.396 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456221+0000", 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.22159378199999999 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456236+0000", 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304999999999999e-05 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456300+0000", 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4294999999999993e-05 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456588+0000", 2024-09-21T11:57:30.397 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028769499999999998 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456701+0000", 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113275 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.464708+0000", 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080072679999999997 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.398 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.464728+0000", 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9700000000000001e-05 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467080+0000", 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00235214 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467137+0000", 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7194999999999997e-05 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467204+0000", 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7143000000000004e-05 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:126311 10.0 e348/313 10:03ca3f46:::benchmark_data_smithi043_150678_object3560:head v 348'7980, mlcod=348'7978)", 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.238509+0000", 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 185.08848422200001, 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.22861142000000001, 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.400 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238509+0000", 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238509+0000", 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.401 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238511+0000", 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4559999999999999e-06 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238524+0000", 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2875e-05 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.455912+0000", 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.21738855400000001 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.455934+0000", 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1517000000000001e-05 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456941+0000", 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010073829999999999 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456966+0000", 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4556e-05 2024-09-21T11:57:30.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467084+0000", 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010118240000000001 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467121+0000", 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6838999999999997e-05 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:130557 10.2 10:4741c7dc:::benchmark_data_smithi043_150678_object7806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.493746+0000", 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 180.83324728400001, 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.18472519600000001, 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 130557 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493746+0000", 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493746+0000", 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493748+0000", 2024-09-21T11:57:30.406 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0480000000000001e-06 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493754+0000", 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0769999999999996e-06 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493795+0000", 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0853999999999999e-05 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493804+0000", 2024-09-21T11:57:30.407 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4950000000000008e-06 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493862+0000", 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8749999999999998e-05 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493968+0000", 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105327 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.494070+0000", 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010192 2024-09-21T11:57:30.408 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497356+0000", 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003286027 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678126+0000", 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.18076989199999999 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678302+0000", 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017640099999999999 2024-09-21T11:57:30.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678415+0000", 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112965 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678471+0000", 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6440000000000002e-05 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.410 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:130563 10.2 10:4b5202a2:::benchmark_data_smithi043_150678_object7812:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.497727+0000", 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 180.829266789, 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.183693038, 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 130563 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.411 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497727+0000", 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497727+0000", 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497728+0000", 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.807e-06 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.412 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497735+0000", 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497748+0000", 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2532e-05 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497756+0000", 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1100000000000003e-06 2024-09-21T11:57:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.500514+0000", 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027577190000000001 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675393+0000", 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17487897299999999 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675484+0000", 2024-09-21T11:57:30.414 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1860999999999993e-05 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678546+0000", 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030618709999999999 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681236+0000", 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026896289999999998 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.415 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681299+0000", 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2710000000000001e-05 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681371+0000", 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2080000000000001e-05 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681420+0000", 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8996999999999999e-05 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.416 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:130569 10.2 10:46c1164d:::benchmark_data_smithi043_150678_object7818:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.502522+0000", 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 180.82447182499999, 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17911437499999999, 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 130569 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.417 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502522+0000", 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502522+0000", 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502524+0000", 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1940000000000001e-06 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.418 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502531+0000", 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0960000000000003e-06 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675246+0000", 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.172715127 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675264+0000", 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8306999999999999e-05 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.419 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678690+0000", 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034253310000000002 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678797+0000", 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107679 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678873+0000", 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5402999999999996e-05 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.420 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681486+0000", 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026133100000000002 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681534+0000", 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8352000000000001e-05 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681547+0000", 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2784e-05 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.421 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681595+0000", 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7403e-05 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681636+0000", 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1389e-05 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:130568 10.0 e348/313 10:0d347089:::benchmark_data_smithi043_150678_object7817:head v 348'8247, mlcod=348'8245)", 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.502849+0000", 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 180.824144469, 2024-09-21T11:57:30.422 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.180754058, 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502849+0000", 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502849+0000", 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.423 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502850+0000", 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-06 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502860+0000", 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0000000000000001e-05 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675242+0000", 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17238205500000001 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675254+0000", 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1874e-05 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675376+0000", 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012161 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675398+0000", 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2949999999999999e-05 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.683567+0000", 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0081684180000000002 2024-09-21T11:57:30.425 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.683603+0000", 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6127999999999997e-05 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25910.0:130557 10.2 e348/313 ondisk, result = 0)", 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.504828+0000", 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 180.82216498599999, 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17365343999999999, 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.426 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.504828+0000", 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.504828+0000", 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.427 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.504829+0000", 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.012e-06 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.504836+0000", 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1859999999999997e-06 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675240+0000", 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17040451500000001 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.428 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675265+0000", 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4658999999999999e-05 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678286+0000", 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003021087 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678294+0000", 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6350000000000006e-06 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.429 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678482+0000", 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018834599999999999 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:60092 11.c 11:39d222d2:::benchmark_data_smithi043_178089_object60091:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:57:13.474688+0000", 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 16.852305694000002, 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27958532800000002, 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.430 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 60092 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474688+0000", 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.431 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474688+0000", 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474689+0000", 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.201e-06 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.432 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474709+0000", 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0089999999999999e-05 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474719+0000", 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0302000000000001e-05 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474725+0000", 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3759999999999999e-06 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.433 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474776+0000", 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1663000000000003e-05 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474920+0000", 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014358400000000001 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.475107+0000", 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000186949 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.434 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.477348+0000", 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022414790000000002 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.477634+0000", 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028583999999999999 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.754107+0000", 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27647271099999998 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.435 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.754213+0000", 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010594000000000001 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.754273+0000", 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0192999999999998e-05 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:72802 11.c 11:384e41fb:::benchmark_data_smithi043_178089_object72801:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e404)", 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:57:24.480989+0000", 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 5.846004905, 2024-09-21T11:57:30.436 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17437597299999999, 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 72802 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.480989+0000", 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.437 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.480989+0000", 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.480989+0000", 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3399999999999997e-07 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.481013+0000", 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3054999999999999e-05 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.438 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.481023+0000", 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0861e-05 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.481030+0000", 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8959999999999997e-06 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.483675+0000", 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026445919999999999 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:30.439 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.483786+0000", 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011119400000000001 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.484081+0000", 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00029542799999999998 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.501048+0000", 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.016966324000000001 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.505496+0000", 2024-09-21T11:57:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044478340000000003 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.655054+0000", 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.14955822199999999 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.655161+0000", 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107459 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:24.655365+0000", 2024-09-21T11:57:30.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000203174 2024-09-21T11:57:30.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.442 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.442 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:30.442 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:30.442 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:57:30.470 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:30.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:30.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:30.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:30.616 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:30.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:30.617 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:30.698 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 50 16 79579 79563 99.4194 71.75 0.0257748 0.0100536 2024-09-21T11:57:30.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:30.768 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:30.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:30.769 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:30.869 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:30.901 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:30.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:30.902 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:31.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:31.008 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:57:31.008 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:57:31.008 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-21T11:57:31.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:57:31.031 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:31.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:31.032 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:31.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:31.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:31.226 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:31.254 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:57:31.255 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:57:31.303 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:57:31.303 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:57:31.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:31.339 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:57:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:57:31.369 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:31.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:31.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:31.381 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:57:31.381 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:57:31.416 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:57:31.416 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:57:31.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:57:31.448 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:57:31.448 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:57:31.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:31.488 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:57:31.488 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:57:31.498 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:31.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:31.499 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:31.524 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:57:31.524 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:57:31.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:31.619 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:31 smithi043 ceph-mon[29710]: pgmap v846: 148 pgs: 148 active+clean; 4.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 778 B/s rd, 53 MiB/s wr, 845 op/s 2024-09-21T11:57:31.673 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:31.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:31.674 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:31.701 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 51 16 80642 80626 98.771 66.4375 0.00656168 0.0101184 2024-09-21T11:57:31.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:31.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:31.826 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:31.826 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:31.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:57:31.890 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1861483814' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:31.891 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:31 smithi043 ceph-mon[36454]: pgmap v846: 148 pgs: 148 active+clean; 4.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 778 B/s rd, 53 MiB/s wr, 845 op/s 2024-09-21T11:57:31.891 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1861483814' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:31.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:31.951 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:31.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:31.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:31 smithi175 ceph-mon[32613]: pgmap v846: 148 pgs: 148 active+clean; 4.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 778 B/s rd, 53 MiB/s wr, 845 op/s 2024-09-21T11:57:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1861483814' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:32.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:57:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:57:32.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:32.102 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:32.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:32.103 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:32.203 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:32.238 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:32.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:32.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:32.340 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:32.370 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:32.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:32.371 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:32.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:32.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:32.519 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:32.620 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:32.646 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:32.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:32.647 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:32.699 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 52 16 81778 81762 98.237 71 0.0239364 0.0101733 2024-09-21T11:57:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:32.714 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:32.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:32.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:32.804 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:32.905 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:32.933 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:32.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:32.934 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:33.035 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:33.064 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:33.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:33.065 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:33.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:33.189 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:33.189 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:32 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:33.189 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:33.189 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:32 smithi043 ceph-mon[29710]: pgmap v847: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-21T11:57:33.189 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:32 smithi043 ceph-mon[29710]: osdmap e407: 8 total, 8 up, 8 in 2024-09-21T11:57:33.190 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:33.190 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:32 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:33.190 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:33.190 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:32 smithi043 ceph-mon[36454]: pgmap v847: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-21T11:57:33.190 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:32 smithi043 ceph-mon[36454]: osdmap e407: 8 total, 8 up, 8 in 2024-09-21T11:57:33.208 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:33.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:33.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:33.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:33.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:32 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:32 smithi175 ceph-mon[32613]: pgmap v847: 148 pgs: 148 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 87 MiB/s wr, 1.39k op/s 2024-09-21T11:57:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:32 smithi175 ceph-mon[32613]: osdmap e407: 8 total, 8 up, 8 in 2024-09-21T11:57:33.311 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:33.343 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:33.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:33.344 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:33.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:33.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:33.471 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:33.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:33.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:33.598 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:33.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:33.599 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:33.699 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 53 16 82900 82884 97.7066 70.125 0.00355419 0.0102286 2024-09-21T11:57:33.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:33.744 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:33.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:33.745 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:33.834 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:33.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:33.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:33.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:33.989 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:57:33.992 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:57:34.007 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:34.035 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:34.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:34.036 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:34.136 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:34.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:33 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:57:34.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:33 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:57:34.148 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:33 smithi043 ceph-mon[29710]: osdmap e408: 8 total, 8 up, 8 in 2024-09-21T11:57:34.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:33 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:57:34.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:33 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:57:34.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:33 smithi043 ceph-mon[36454]: osdmap e408: 8 total, 8 up, 8 in 2024-09-21T11:57:34.161 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:34.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:34.162 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:34.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:33 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:57:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:33 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:57:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:33 smithi175 ceph-mon[32613]: osdmap e408: 8 total, 8 up, 8 in 2024-09-21T11:57:34.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:34.295 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:34.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:34.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:34.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:34.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:34.427 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:34.427 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:34.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:34.569 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:34.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:34.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:34.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:57:34.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:34.701 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 54 16 83991 83975 97.1568 68.1875 0.0309394 0.0102857 2024-09-21T11:57:34.725 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:34.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:34.726 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:34.752 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:57:34.753 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:57:34.827 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:34.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:34.862 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:34.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:34.959 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:57:34.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:35.019 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:35.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:35.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:35.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:35.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:35.163 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:35.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:34 smithi043 ceph-mon[29710]: from='client.26077 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:57:35.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:34 smithi043 ceph-mon[29710]: pgmap v850: 148 pgs: 148 active+clean; 4.9 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 58 MiB/s wr, 931 op/s 2024-09-21T11:57:35.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:34 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3368764312' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:57:35.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:34 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:57:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:34 smithi043 ceph-mon[36454]: from='client.26077 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:57:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:34 smithi043 ceph-mon[36454]: pgmap v850: 148 pgs: 148 active+clean; 4.9 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 58 MiB/s wr, 931 op/s 2024-09-21T11:57:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:34 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3368764312' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:57:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:34 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:57:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:34 smithi175 ceph-mon[32613]: from='client.26077 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:57:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:34 smithi175 ceph-mon[32613]: pgmap v850: 148 pgs: 148 active+clean; 4.9 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 58 MiB/s wr, 931 op/s 2024-09-21T11:57:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:34 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3368764312' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:57:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:34 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:57:35.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:35.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:35.314 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:35.314 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:35.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:35.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:35.458 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:57:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:57:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:35.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:35.592 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:35.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:35.593 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:35.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:35.701 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 55 16 85156 85140 96.7142 72.8125 0.0104144 0.0103347 2024-09-21T11:57:35.727 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:35.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:35.728 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:35.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:35.841 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:57:35.875 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:35.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:35.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:35.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:36.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:36.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:36.120 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:36.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:36.155 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:36.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:36.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:35 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:36.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:35 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:57:36.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:35 smithi043 ceph-mon[29710]: osdmap e409: 8 total, 8 up, 8 in 2024-09-21T11:57:36.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:35 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:36.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:35 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:57:36.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:35 smithi043 ceph-mon[36454]: osdmap e409: 8 total, 8 up, 8 in 2024-09-21T11:57:36.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:36.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:35 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:36.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:35 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:57:36.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:35 smithi175 ceph-mon[32613]: osdmap e409: 8 total, 8 up, 8 in 2024-09-21T11:57:36.309 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:36.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:36.311 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:36.411 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:36.444 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:36.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:36.445 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:36.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:36.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:36.583 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:36.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:36.680 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":849,"stamp":"2024-09-21T11:57:33.926007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5257173531,"num_objects":80408,"num_object_clones":0,"num_object_copies":241224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80408,"num_whiteouts":0,"num_read":2289,"num_read_kb":2146,"num_write":81044,"num_write_kb":5135049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":1188763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77857,"ondisk_log_size":77857,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20481836,"kb_used_data":15355228,"kb_used_omap":12,"kb_used_meta":5126451,"kb_avail":729479380,"statfs":{"total":767960285184,"available":746986885120,"internally_reserved":0,"allocated":15723753472,"data_stored":15720199265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":5249486386},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,9,6,8,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":366280704,"num_objects":5589,"num_object_clones":0,"num_object_copies":16767,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5589,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5589,"num_write_kb":357696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.444754"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":96,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.618529+0000","last_change":"2024-09-21T11:55:28.565270+0000","last_active":"2024-09-21T11:57:28.618529+0000","last_peered":"2024-09-21T11:57:28.618529+0000","last_clean":"2024-09-21T11:57:28.618529+0000","last_became_active":"2024-09-21T11:55:28.564399+0000","last_became_peered":"2024-09-21T11:55:28.564399+0000","last_unstale":"2024-09-21T11:57:28.618529+0000","last_undegraded":"2024-09-21T11:57:28.618529+0000","last_fullsized":"2024-09-21T11:57:28.618529+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.303555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.303555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.026551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":776,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622167+0000","last_change":"2024-09-21T11:54:27.283019+0000","last_active":"2024-09-21T11:57:28.622167+0000","last_peered":"2024-09-21T11:57:28.622167+0000","last_clean":"2024-09-21T11:57:28.622167+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:57:28.622167+0000","last_undegraded":"2024-09-21T11:57:28.622167+0000","last_fullsized":"2024-09-21T11:57:28.622167+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:54:27.282826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:27.282826+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":462,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626323+0000","last_change":"2024-09-21T11:56:08.429983+0000","last_active":"2024-09-21T11:57:28.626323+0000","last_peered":"2024-09-21T11:57:28.626323+0000","last_clean":"2024-09-21T11:57:28.626323+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:57:28.626323+0000","last_undegraded":"2024-09-21T11:57:28.626323+0000","last_fullsized":"2024-09-21T11:57:28.626323+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:08.429912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.429912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":549,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.614993+0000","last_change":"2024-09-21T11:51:43.528594+0000","last_active":"2024-09-21T11:57:28.614993+0000","last_peered":"2024-09-21T11:57:28.614993+0000","last_clean":"2024-09-21T11:57:28.614993+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:57:28.614993+0000","last_undegraded":"2024-09-21T11:57:28.614993+0000","last_fullsized":"2024-09-21T11:57:28.614993+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:08.053151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:49:08.053151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:21.767545+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":756,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628525+0000","last_change":"2024-09-21T11:56:49.693769+0000","last_active":"2024-09-21T11:57:28.628525+0000","last_peered":"2024-09-21T11:57:28.628525+0000","last_clean":"2024-09-21T11:57:28.628525+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:57:28.628525+0000","last_undegraded":"2024-09-21T11:57:28.628525+0000","last_fullsized":"2024-09-21T11:57:28.628525+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:49.693594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:49.693594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:03.909258+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":738,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.635473+0000","last_change":"2024-09-21T11:55:25.429625+0000","last_active":"2024-09-21T11:57:28.635473+0000","last_peered":"2024-09-21T11:57:28.635473+0000","last_clean":"2024-09-21T11:57:28.635473+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:57:28.635473+0000","last_undegraded":"2024-09-21T11:57:28.635473+0000","last_fullsized":"2024-09-21T11:57:28.635473+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:25.429559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:25.429559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":792,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.630089+0000","last_change":"2024-09-21T11:56:31.572626+0000","last_active":"2024-09-21T11:57:28.630089+0000","last_peered":"2024-09-21T11:57:28.630089+0000","last_clean":"2024-09-21T11:57:28.630089+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:57:28.630089+0000","last_undegraded":"2024-09-21T11:57:28.630089+0000","last_fullsized":"2024-09-21T11:57:28.630089+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:31.572525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:31.572525+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:41.538532+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":592,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622275+0000","last_change":"2024-09-21T11:57:13.961006+0000","last_active":"2024-09-21T11:57:28.622275+0000","last_peered":"2024-09-21T11:57:28.622275+0000","last_clean":"2024-09-21T11:57:28.622275+0000","last_became_active":"2024-09-21T11:55:28.561338+0000","last_became_peered":"2024-09-21T11:55:28.561338+0000","last_unstale":"2024-09-21T11:57:28.622275+0000","last_undegraded":"2024-09-21T11:57:28.622275+0000","last_fullsized":"2024-09-21T11:57:28.622275+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:13.960961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:13.960961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:34.935232+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":736,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.616817+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:57:28.616817+0000","last_peered":"2024-09-21T11:57:28.616817+0000","last_clean":"2024-09-21T11:57:28.616817+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:57:28.616817+0000","last_undegraded":"2024-09-21T11:57:28.616817+0000","last_fullsized":"2024-09-21T11:57:28.616817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:30.933888+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":767,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.635733+0000","last_change":"2024-09-21T11:57:07.774990+0000","last_active":"2024-09-21T11:57:28.635733+0000","last_peered":"2024-09-21T11:57:28.635733+0000","last_clean":"2024-09-21T11:57:28.635733+0000","last_became_active":"2024-09-21T11:55:28.575920+0000","last_became_peered":"2024-09-21T11:55:28.575920+0000","last_unstale":"2024-09-21T11:57:28.635733+0000","last_undegraded":"2024-09-21T11:57:28.635733+0000","last_fullsized":"2024-09-21T11:57:28.635733+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:07.774892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:07.774892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:14.619408+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":735,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.615451+0000","last_change":"2024-09-21T11:55:05.096606+0000","last_active":"2024-09-21T11:57:28.615451+0000","last_peered":"2024-09-21T11:57:28.615451+0000","last_clean":"2024-09-21T11:57:28.615451+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:57:28.615451+0000","last_undegraded":"2024-09-21T11:57:28.615451+0000","last_fullsized":"2024-09-21T11:57:28.615451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:05.096542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:05.096542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:42.070654+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":412,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627272+0000","last_change":"2024-09-21T11:56:36.422776+0000","last_active":"2024-09-21T11:57:28.627272+0000","last_peered":"2024-09-21T11:57:28.627272+0000","last_clean":"2024-09-21T11:57:28.627272+0000","last_became_active":"2024-09-21T11:55:28.566921+0000","last_became_peered":"2024-09-21T11:55:28.566921+0000","last_unstale":"2024-09-21T11:57:28.627272+0000","last_undegraded":"2024-09-21T11:57:28.627272+0000","last_fullsized":"2024-09-21T11:57:28.627272+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:36.422726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:36.422726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.958604+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":782,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.615940+0000","last_change":"2024-09-21T11:56:02.994041+0000","last_active":"2024-09-21T11:57:28.615940+0000","last_peered":"2024-09-21T11:57:28.615940+0000","last_clean":"2024-09-21T11:57:28.615940+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:57:28.615940+0000","last_undegraded":"2024-09-21T11:57:28.615940+0000","last_fullsized":"2024-09-21T11:57:28.615940+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.993966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.993966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":769,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628911+0000","last_change":"2024-09-21T11:56:59.659683+0000","last_active":"2024-09-21T11:57:28.628911+0000","last_peered":"2024-09-21T11:57:28.628911+0000","last_clean":"2024-09-21T11:57:28.628911+0000","last_became_active":"2024-09-21T11:56:16.667750+0000","last_became_peered":"2024-09-21T11:56:16.667750+0000","last_unstale":"2024-09-21T11:57:28.628911+0000","last_undegraded":"2024-09-21T11:57:28.628911+0000","last_fullsized":"2024-09-21T11:57:28.628911+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'7","last_scrub_stamp":"2024-09-21T11:56:59.659576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.659576+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:01.258966+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":750,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627056+0000","last_change":"2024-09-21T11:57:13.323002+0000","last_active":"2024-09-21T11:57:28.627056+0000","last_peered":"2024-09-21T11:57:28.627056+0000","last_clean":"2024-09-21T11:57:28.627056+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:57:28.627056+0000","last_undegraded":"2024-09-21T11:57:28.627056+0000","last_fullsized":"2024-09-21T11:57:28.627056+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:13.322953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:13.322953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:18.735588+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":746,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.615067+0000","last_change":"2024-09-21T11:50:36.629891+0000","last_active":"2024-09-21T11:57:28.615067+0000","last_peered":"2024-09-21T11:57:28.615067+0000","last_clean":"2024-09-21T11:57:28.615067+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:57:28.615067+0000","last_undegraded":"2024-09-21T11:57:28.615067+0000","last_fullsized":"2024-09-21T11:57:28.615067+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:36.629801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:36.629801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:24.795838+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":255,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622535+0000","last_change":"2024-09-21T11:56:34.073296+0000","last_active":"2024-09-21T11:57:28.622535+0000","last_peered":"2024-09-21T11:57:28.622535+0000","last_clean":"2024-09-21T11:57:28.622535+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:57:28.622535+0000","last_undegraded":"2024-09-21T11:57:28.622535+0000","last_fullsized":"2024-09-21T11:57:28.622535+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:34.073249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:34.073249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:46.860083+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":96,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628390+0000","last_change":"2024-09-21T11:55:28.562731+0000","last_active":"2024-09-21T11:57:28.628390+0000","last_peered":"2024-09-21T11:57:28.628390+0000","last_clean":"2024-09-21T11:57:28.628390+0000","last_became_active":"2024-09-21T11:55:28.562057+0000","last_became_peered":"2024-09-21T11:55:28.562057+0000","last_unstale":"2024-09-21T11:57:28.628390+0000","last_undegraded":"2024-09-21T11:57:28.628390+0000","last_fullsized":"2024-09-21T11:57:28.628390+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:27.176320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:27.176320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:43.343459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":891,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.619475+0000","last_change":"2024-09-21T11:56:01.647675+0000","last_active":"2024-09-21T11:57:28.619475+0000","last_peered":"2024-09-21T11:57:28.619475+0000","last_clean":"2024-09-21T11:57:28.619475+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T11:57:28.619475+0000","last_undegraded":"2024-09-21T11:57:28.619475+0000","last_fullsized":"2024-09-21T11:57:28.619475+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:01.647580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:01.647580+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":461,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627503+0000","last_change":"2024-09-21T11:56:16.669857+0000","last_active":"2024-09-21T11:57:28.627503+0000","last_peered":"2024-09-21T11:57:28.627503+0000","last_clean":"2024-09-21T11:57:28.627503+0000","last_became_active":"2024-09-21T11:56:16.669551+0000","last_became_peered":"2024-09-21T11:56:16.669551+0000","last_unstale":"2024-09-21T11:57:28.627503+0000","last_undegraded":"2024-09-21T11:57:28.627503+0000","last_fullsized":"2024-09-21T11:57:28.627503+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:56.501356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:56.501356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":750,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.634703+0000","last_change":"2024-09-21T11:57:04.647951+0000","last_active":"2024-09-21T11:57:28.634703+0000","last_peered":"2024-09-21T11:57:28.634703+0000","last_clean":"2024-09-21T11:57:28.634703+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:57:28.634703+0000","last_undegraded":"2024-09-21T11:57:28.634703+0000","last_fullsized":"2024-09-21T11:57:28.634703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:04.647749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:04.647749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:13.218840+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":569,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626093+0000","last_change":"2024-09-21T11:55:57.442188+0000","last_active":"2024-09-21T11:57:28.626093+0000","last_peered":"2024-09-21T11:57:28.626093+0000","last_clean":"2024-09-21T11:57:28.626093+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:57:28.626093+0000","last_undegraded":"2024-09-21T11:57:28.626093+0000","last_fullsized":"2024-09-21T11:57:28.626093+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:57.441965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.441965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":810,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.634414+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:57:28.634414+0000","last_peered":"2024-09-21T11:57:28.634414+0000","last_clean":"2024-09-21T11:57:28.634414+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:57:28.634414+0000","last_undegraded":"2024-09-21T11:57:28.634414+0000","last_fullsized":"2024-09-21T11:57:28.634414+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:47.692625+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":262,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628504+0000","last_change":"2024-09-21T11:56:33.487190+0000","last_active":"2024-09-21T11:57:28.628504+0000","last_peered":"2024-09-21T11:57:28.628504+0000","last_clean":"2024-09-21T11:57:28.628504+0000","last_became_active":"2024-09-21T11:56:16.666765+0000","last_became_peered":"2024-09-21T11:56:16.666765+0000","last_unstale":"2024-09-21T11:57:28.628504+0000","last_undegraded":"2024-09-21T11:57:28.628504+0000","last_fullsized":"2024-09-21T11:57:28.628504+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.487132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.487132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:03.363944+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":252,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.483881+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:57:32.483881+0000","last_peered":"2024-09-21T11:57:32.483881+0000","last_clean":"2024-09-21T11:57:32.483881+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:57:32.483881+0000","last_undegraded":"2024-09-21T11:57:32.483881+0000","last_fullsized":"2024-09-21T11:57:32.483881+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:23.154080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":477,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626528+0000","last_change":"2024-09-21T11:57:10.530989+0000","last_active":"2024-09-21T11:57:28.626528+0000","last_peered":"2024-09-21T11:57:28.626528+0000","last_clean":"2024-09-21T11:57:28.626528+0000","last_became_active":"2024-09-21T11:55:28.568598+0000","last_became_peered":"2024-09-21T11:55:28.568598+0000","last_unstale":"2024-09-21T11:57:28.626528+0000","last_undegraded":"2024-09-21T11:57:28.626528+0000","last_fullsized":"2024-09-21T11:57:28.626528+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:10.530913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.530913+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:13.614764+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":781,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.483827+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:57:32.483827+0000","last_peered":"2024-09-21T11:57:32.483827+0000","last_clean":"2024-09-21T11:57:32.483827+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:57:32.483827+0000","last_undegraded":"2024-09-21T11:57:32.483827+0000","last_fullsized":"2024-09-21T11:57:32.483827+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:28.176930+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"383'9","reported_seq":995,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.623037+0000","last_change":"2024-09-21T11:56:16.677508+0000","last_active":"2024-09-21T11:57:28.623037+0000","last_peered":"2024-09-21T11:57:28.623037+0000","last_clean":"2024-09-21T11:57:28.623037+0000","last_became_active":"2024-09-21T11:56:16.673446+0000","last_became_peered":"2024-09-21T11:56:16.673446+0000","last_unstale":"2024-09-21T11:57:28.623037+0000","last_undegraded":"2024-09-21T11:57:28.623037+0000","last_fullsized":"2024-09-21T11:57:28.623037+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:57.232943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.232943+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"406'5141","reported_seq":5180,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:31.573605+0000","last_change":"2024-09-21T11:56:36.507880+0000","last_active":"2024-09-21T11:57:31.573605+0000","last_peered":"2024-09-21T11:57:31.573605+0000","last_clean":"2024-09-21T11:57:31.573605+0000","last_became_active":"2024-09-21T11:56:36.507606+0000","last_became_peered":"2024-09-21T11:56:36.507606+0000","last_unstale":"2024-09-21T11:57:31.573605+0000","last_undegraded":"2024-09-21T11:57:31.573605+0000","last_fullsized":"2024-09-21T11:57:31.573605+0000","mapping_epoch":392,"log_start":"396'700","ondisk_log_start":"396'700","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4441,"log_dups_size":0,"ondisk_log_size":4441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:44.536721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":336920576,"num_objects":5141,"num_object_clones":0,"num_object_copies":15423,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5141,"num_write_kb":329024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":488,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.634315+0000","last_change":"2024-09-21T11:55:52.562992+0000","last_active":"2024-09-21T11:57:28.634315+0000","last_peered":"2024-09-21T11:57:28.634315+0000","last_clean":"2024-09-21T11:57:28.634315+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:57:28.634315+0000","last_undegraded":"2024-09-21T11:57:28.634315+0000","last_fullsized":"2024-09-21T11:57:28.634315+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:55:52.562928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.562928+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":776,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.629439+0000","last_change":"2024-09-21T11:57:13.738771+0000","last_active":"2024-09-21T11:57:28.629439+0000","last_peered":"2024-09-21T11:57:28.629439+0000","last_clean":"2024-09-21T11:57:28.629439+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:57:28.629439+0000","last_undegraded":"2024-09-21T11:57:28.629439+0000","last_fullsized":"2024-09-21T11:57:28.629439+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:13.738727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:13.738727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:34.168236+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"366'10","reported_seq":363,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.616317+0000","last_change":"2024-09-21T11:56:48.961086+0000","last_active":"2024-09-21T11:57:28.616317+0000","last_peered":"2024-09-21T11:57:28.616317+0000","last_clean":"2024-09-21T11:57:28.616317+0000","last_became_active":"2024-09-21T11:55:28.575209+0000","last_became_peered":"2024-09-21T11:55:28.575209+0000","last_unstale":"2024-09-21T11:57:28.616317+0000","last_undegraded":"2024-09-21T11:57:28.616317+0000","last_fullsized":"2024-09-21T11:57:28.616317+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"366'10","last_scrub_stamp":"2024-09-21T11:56:48.961017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.961017+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:14.388477+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":785,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.616368+0000","last_change":"2024-09-21T11:56:57.993660+0000","last_active":"2024-09-21T11:57:28.616368+0000","last_peered":"2024-09-21T11:57:28.616368+0000","last_clean":"2024-09-21T11:57:28.616368+0000","last_became_active":"2024-09-21T11:55:28.564378+0000","last_became_peered":"2024-09-21T11:55:28.564378+0000","last_unstale":"2024-09-21T11:57:28.616368+0000","last_undegraded":"2024-09-21T11:57:28.616368+0000","last_fullsized":"2024-09-21T11:57:28.616368+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:57.993615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:57.993615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:10.442348+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"406'5140","reported_seq":5179,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:31.274476+0000","last_change":"2024-09-21T11:56:36.511425+0000","last_active":"2024-09-21T11:57:31.274476+0000","last_peered":"2024-09-21T11:57:31.274476+0000","last_clean":"2024-09-21T11:57:31.274476+0000","last_became_active":"2024-09-21T11:56:36.511191+0000","last_became_peered":"2024-09-21T11:56:36.511191+0000","last_unstale":"2024-09-21T11:57:31.274476+0000","last_undegraded":"2024-09-21T11:57:31.274476+0000","last_fullsized":"2024-09-21T11:57:31.274476+0000","mapping_epoch":392,"log_start":"396'500","ondisk_log_start":"396'500","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:04.769503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":336855040,"num_objects":5140,"num_object_clones":0,"num_object_copies":15420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5140,"num_write_kb":328960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":841,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628761+0000","last_change":"2024-09-21T11:57:00.632121+0000","last_active":"2024-09-21T11:57:28.628761+0000","last_peered":"2024-09-21T11:57:28.628761+0000","last_clean":"2024-09-21T11:57:28.628761+0000","last_became_active":"2024-09-21T11:55:28.563496+0000","last_became_peered":"2024-09-21T11:55:28.563496+0000","last_unstale":"2024-09-21T11:57:28.628761+0000","last_undegraded":"2024-09-21T11:57:28.628761+0000","last_fullsized":"2024-09-21T11:57:28.628761+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:57:00.631769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:00.631769+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:05.924170+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":770,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.635028+0000","last_change":"2024-09-21T11:56:53.701703+0000","last_active":"2024-09-21T11:57:28.635028+0000","last_peered":"2024-09-21T11:57:28.635028+0000","last_clean":"2024-09-21T11:57:28.635028+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:57:28.635028+0000","last_undegraded":"2024-09-21T11:57:28.635028+0000","last_fullsized":"2024-09-21T11:57:28.635028+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.701639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.701639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:10.075609+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":247,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.629582+0000","last_change":"2024-09-21T11:56:06.506517+0000","last_active":"2024-09-21T11:57:28.629582+0000","last_peered":"2024-09-21T11:57:28.629582+0000","last_clean":"2024-09-21T11:57:28.629582+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:57:28.629582+0000","last_undegraded":"2024-09-21T11:57:28.629582+0000","last_fullsized":"2024-09-21T11:57:28.629582+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:06.506449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:06.506449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:31.003771+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":249,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.618505+0000","last_change":"2024-09-21T11:54:53.844954+0000","last_active":"2024-09-21T11:57:28.618505+0000","last_peered":"2024-09-21T11:57:28.618505+0000","last_clean":"2024-09-21T11:57:28.618505+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:57:28.618505+0000","last_undegraded":"2024-09-21T11:57:28.618505+0000","last_fullsized":"2024-09-21T11:57:28.618505+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:53.844876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:53.844876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.755477+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"406'4929","reported_seq":4968,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:31.622180+0000","last_change":"2024-09-21T11:56:36.508433+0000","last_active":"2024-09-21T11:57:31.622180+0000","last_peered":"2024-09-21T11:57:31.622180+0000","last_clean":"2024-09-21T11:57:31.622180+0000","last_became_active":"2024-09-21T11:56:36.507746+0000","last_became_peered":"2024-09-21T11:56:36.507746+0000","last_unstale":"2024-09-21T11:57:31.622180+0000","last_undegraded":"2024-09-21T11:57:31.622180+0000","last_fullsized":"2024-09-21T11:57:31.622180+0000","mapping_epoch":392,"log_start":"396'500","ondisk_log_start":"396'500","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4429,"log_dups_size":0,"ondisk_log_size":4429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:54.005355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":323026944,"num_objects":4929,"num_object_clones":0,"num_object_copies":14787,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4929,"num_write_kb":315456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":774,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.615153+0000","last_change":"2024-09-21T11:56:16.682561+0000","last_active":"2024-09-21T11:57:28.615153+0000","last_peered":"2024-09-21T11:57:28.615153+0000","last_clean":"2024-09-21T11:57:28.615153+0000","last_became_active":"2024-09-21T11:56:16.670451+0000","last_became_peered":"2024-09-21T11:56:16.670451+0000","last_unstale":"2024-09-21T11:57:28.615153+0000","last_undegraded":"2024-09-21T11:57:28.615153+0000","last_fullsized":"2024-09-21T11:57:28.615153+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:50:38.339941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:38.339941+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:25.781476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":442,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628054+0000","last_change":"2024-09-21T11:56:48.446718+0000","last_active":"2024-09-21T11:57:28.628054+0000","last_peered":"2024-09-21T11:57:28.628054+0000","last_clean":"2024-09-21T11:57:28.628054+0000","last_became_active":"2024-09-21T11:55:28.567133+0000","last_became_peered":"2024-09-21T11:55:28.567133+0000","last_unstale":"2024-09-21T11:57:28.628054+0000","last_undegraded":"2024-09-21T11:57:28.628054+0000","last_fullsized":"2024-09-21T11:57:28.628054+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:48.446635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.446635+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:16.560147+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":988,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628599+0000","last_change":"2024-09-21T11:56:05.567407+0000","last_active":"2024-09-21T11:57:28.628599+0000","last_peered":"2024-09-21T11:57:28.628599+0000","last_clean":"2024-09-21T11:57:28.628599+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:57:28.628599+0000","last_undegraded":"2024-09-21T11:57:28.628599+0000","last_fullsized":"2024-09-21T11:57:28.628599+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:05.567332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.567332+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":759,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628333+0000","last_change":"2024-09-21T11:56:34.581966+0000","last_active":"2024-09-21T11:57:28.628333+0000","last_peered":"2024-09-21T11:57:28.628333+0000","last_clean":"2024-09-21T11:57:28.628333+0000","last_became_active":"2024-09-21T11:56:34.581626+0000","last_became_peered":"2024-09-21T11:56:34.581626+0000","last_unstale":"2024-09-21T11:57:28.628333+0000","last_undegraded":"2024-09-21T11:57:28.628333+0000","last_fullsized":"2024-09-21T11:57:28.628333+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.643323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.643323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"406'5013","reported_seq":5052,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:31.740469+0000","last_change":"2024-09-21T11:56:36.623897+0000","last_active":"2024-09-21T11:57:31.740469+0000","last_peered":"2024-09-21T11:57:31.740469+0000","last_clean":"2024-09-21T11:57:31.740469+0000","last_became_active":"2024-09-21T11:56:36.623581+0000","last_became_peered":"2024-09-21T11:56:36.623581+0000","last_unstale":"2024-09-21T11:57:31.740469+0000","last_undegraded":"2024-09-21T11:57:31.740469+0000","last_fullsized":"2024-09-21T11:57:31.740469+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:46.606322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328531968,"num_objects":5013,"num_object_clones":0,"num_object_copies":15039,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5013,"num_write_kb":320832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":501,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626806+0000","last_change":"2024-09-21T11:56:16.679360+0000","last_active":"2024-09-21T11:57:28.626806+0000","last_peered":"2024-09-21T11:57:28.626806+0000","last_clean":"2024-09-21T11:57:28.626806+0000","last_became_active":"2024-09-21T11:56:16.668951+0000","last_became_peered":"2024-09-21T11:56:16.668951+0000","last_unstale":"2024-09-21T11:57:28.626806+0000","last_undegraded":"2024-09-21T11:57:28.626806+0000","last_fullsized":"2024-09-21T11:57:28.626806+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:56:09.421193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.421193+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:32.891630+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":785,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.616883+0000","last_change":"2024-09-21T11:56:51.036609+0000","last_active":"2024-09-21T11:57:28.616883+0000","last_peered":"2024-09-21T11:57:28.616883+0000","last_clean":"2024-09-21T11:57:28.616883+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:57:28.616883+0000","last_undegraded":"2024-09-21T11:57:28.616883+0000","last_fullsized":"2024-09-21T11:57:28.616883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:51.036535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:51.036535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:11.806537+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":969,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.484370+0000","last_change":"2024-09-21T11:55:26.222904+0000","last_active":"2024-09-21T11:57:32.484370+0000","last_peered":"2024-09-21T11:57:32.484370+0000","last_clean":"2024-09-21T11:57:32.484370+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:57:32.484370+0000","last_undegraded":"2024-09-21T11:57:32.484370+0000","last_fullsized":"2024-09-21T11:57:32.484370+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:26.222823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:26.222823+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.191943+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":751,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.483677+0000","last_change":"2024-09-21T11:53:48.256119+0000","last_active":"2024-09-21T11:57:32.483677+0000","last_peered":"2024-09-21T11:57:32.483677+0000","last_clean":"2024-09-21T11:57:32.483677+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:57:32.483677+0000","last_undegraded":"2024-09-21T11:57:32.483677+0000","last_fullsized":"2024-09-21T11:57:32.483677+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:48.255951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:48.255951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"406'5085","reported_seq":5124,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:31.584626+0000","last_change":"2024-09-21T11:56:36.508895+0000","last_active":"2024-09-21T11:57:31.584626+0000","last_peered":"2024-09-21T11:57:31.584626+0000","last_clean":"2024-09-21T11:57:31.584626+0000","last_became_active":"2024-09-21T11:56:36.508626+0000","last_became_peered":"2024-09-21T11:56:36.508626+0000","last_unstale":"2024-09-21T11:57:31.584626+0000","last_undegraded":"2024-09-21T11:57:31.584626+0000","last_fullsized":"2024-09-21T11:57:31.584626+0000","mapping_epoch":392,"log_start":"396'600","ondisk_log_start":"396'600","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4485,"log_dups_size":0,"ondisk_log_size":4485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:50.842122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":333250560,"num_objects":5085,"num_object_clones":0,"num_object_copies":15255,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5085,"num_write_kb":325440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":818,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.484553+0000","last_change":"2024-09-21T11:56:35.177700+0000","last_active":"2024-09-21T11:57:32.484553+0000","last_peered":"2024-09-21T11:57:32.484553+0000","last_clean":"2024-09-21T11:57:32.484553+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:57:32.484553+0000","last_undegraded":"2024-09-21T11:57:32.484553+0000","last_fullsized":"2024-09-21T11:57:32.484553+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:56:35.177639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.177639+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:45.157349+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":455,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.629644+0000","last_change":"2024-09-21T11:56:58.434536+0000","last_active":"2024-09-21T11:57:28.629644+0000","last_peered":"2024-09-21T11:57:28.629644+0000","last_clean":"2024-09-21T11:57:28.629644+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:57:28.629644+0000","last_undegraded":"2024-09-21T11:57:28.629644+0000","last_fullsized":"2024-09-21T11:57:28.629644+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.434464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.434464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:21.450424+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":737,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622294+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:57:28.622294+0000","last_peered":"2024-09-21T11:57:28.622294+0000","last_clean":"2024-09-21T11:57:28.622294+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:57:28.622294+0000","last_undegraded":"2024-09-21T11:57:28.622294+0000","last_fullsized":"2024-09-21T11:57:28.622294+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":447,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626108+0000","last_change":"2024-09-21T11:55:51.602665+0000","last_active":"2024-09-21T11:57:28.626108+0000","last_peered":"2024-09-21T11:57:28.626108+0000","last_clean":"2024-09-21T11:57:28.626108+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:57:28.626108+0000","last_undegraded":"2024-09-21T11:57:28.626108+0000","last_fullsized":"2024-09-21T11:57:28.626108+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.602587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.602587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:31.507832+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"406'5038","reported_seq":5077,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:30.832730+0000","last_change":"2024-09-21T11:56:36.510297+0000","last_active":"2024-09-21T11:57:30.832730+0000","last_peered":"2024-09-21T11:57:30.832730+0000","last_clean":"2024-09-21T11:57:30.832730+0000","last_became_active":"2024-09-21T11:56:36.510018+0000","last_became_peered":"2024-09-21T11:56:36.510018+0000","last_unstale":"2024-09-21T11:57:30.832730+0000","last_undegraded":"2024-09-21T11:57:30.832730+0000","last_fullsized":"2024-09-21T11:57:30.832730+0000","mapping_epoch":392,"log_start":"395'100","ondisk_log_start":"395'100","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4938,"log_dups_size":0,"ondisk_log_size":4938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:04.522929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330170368,"num_objects":5038,"num_object_clones":0,"num_object_copies":15114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5038,"num_write_kb":322432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":597,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:30.582305+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:57:30.582305+0000","last_peered":"2024-09-21T11:57:30.582305+0000","last_clean":"2024-09-21T11:57:30.582305+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:57:30.582305+0000","last_undegraded":"2024-09-21T11:57:30.582305+0000","last_fullsized":"2024-09-21T11:57:30.582305+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:24.504573+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":274,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.484804+0000","last_change":"2024-09-21T11:56:29.248925+0000","last_active":"2024-09-21T11:57:32.484804+0000","last_peered":"2024-09-21T11:57:32.484804+0000","last_clean":"2024-09-21T11:57:32.484804+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:57:32.484804+0000","last_undegraded":"2024-09-21T11:57:32.484804+0000","last_fullsized":"2024-09-21T11:57:32.484804+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:29.248875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.248875+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:51.923814+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":777,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.615613+0000","last_change":"2024-09-21T11:56:28.083398+0000","last_active":"2024-09-21T11:57:28.615613+0000","last_peered":"2024-09-21T11:57:28.615613+0000","last_clean":"2024-09-21T11:57:28.615613+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:57:28.615613+0000","last_undegraded":"2024-09-21T11:57:28.615613+0000","last_fullsized":"2024-09-21T11:57:28.615613+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:56:28.083324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.083324+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.861453+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":416,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627171+0000","last_change":"2024-09-21T11:56:16.667228+0000","last_active":"2024-09-21T11:57:28.627171+0000","last_peered":"2024-09-21T11:57:28.627171+0000","last_clean":"2024-09-21T11:57:28.627171+0000","last_became_active":"2024-09-21T11:56:16.666883+0000","last_became_peered":"2024-09-21T11:56:16.666883+0000","last_unstale":"2024-09-21T11:57:28.627171+0000","last_undegraded":"2024-09-21T11:57:28.627171+0000","last_fullsized":"2024-09-21T11:57:28.627171+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:22.506463+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,7],"acting":[1,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":726,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.635266+0000","last_change":"2024-09-21T11:54:34.452033+0000","last_active":"2024-09-21T11:57:28.635266+0000","last_peered":"2024-09-21T11:57:28.635266+0000","last_clean":"2024-09-21T11:57:28.635266+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:57:28.635266+0000","last_undegraded":"2024-09-21T11:57:28.635266+0000","last_fullsized":"2024-09-21T11:57:28.635266+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:34.451939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:34.451939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:40.548652+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"406'5004","reported_seq":5043,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:30.836790+0000","last_change":"2024-09-21T11:56:36.509094+0000","last_active":"2024-09-21T11:57:30.836790+0000","last_peered":"2024-09-21T11:57:30.836790+0000","last_clean":"2024-09-21T11:57:30.836790+0000","last_became_active":"2024-09-21T11:56:36.508840+0000","last_became_peered":"2024-09-21T11:56:36.508840+0000","last_unstale":"2024-09-21T11:57:30.836790+0000","last_undegraded":"2024-09-21T11:57:30.836790+0000","last_fullsized":"2024-09-21T11:57:30.836790+0000","mapping_epoch":392,"log_start":"395'100","ondisk_log_start":"395'100","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4904,"log_dups_size":0,"ondisk_log_size":4904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:01.292057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327942144,"num_objects":5004,"num_object_clones":0,"num_object_copies":15012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5004,"num_write_kb":320256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":417,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.619414+0000","last_change":"2024-09-21T11:57:08.488118+0000","last_active":"2024-09-21T11:57:28.619414+0000","last_peered":"2024-09-21T11:57:28.619414+0000","last_clean":"2024-09-21T11:57:28.619414+0000","last_became_active":"2024-09-21T11:55:28.569095+0000","last_became_peered":"2024-09-21T11:55:28.569095+0000","last_unstale":"2024-09-21T11:57:28.619414+0000","last_undegraded":"2024-09-21T11:57:28.619414+0000","last_fullsized":"2024-09-21T11:57:28.619414+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:57:08.488050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:08.488050+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:38.067318+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":451,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626131+0000","last_change":"2024-09-21T11:55:06.628254+0000","last_active":"2024-09-21T11:57:28.626131+0000","last_peered":"2024-09-21T11:57:28.626131+0000","last_clean":"2024-09-21T11:57:28.626131+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:57:28.626131+0000","last_undegraded":"2024-09-21T11:57:28.626131+0000","last_fullsized":"2024-09-21T11:57:28.626131+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:55:06.628029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:06.628029+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:40.416603+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":779,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.620073+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:57:28.620073+0000","last_peered":"2024-09-21T11:57:28.620073+0000","last_clean":"2024-09-21T11:57:28.620073+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:57:28.620073+0000","last_undegraded":"2024-09-21T11:57:28.620073+0000","last_fullsized":"2024-09-21T11:57:28.620073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":746,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628761+0000","last_change":"2024-09-21T11:55:06.871048+0000","last_active":"2024-09-21T11:57:28.628761+0000","last_peered":"2024-09-21T11:57:28.628761+0000","last_clean":"2024-09-21T11:57:28.628761+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:57:28.628761+0000","last_undegraded":"2024-09-21T11:57:28.628761+0000","last_fullsized":"2024-09-21T11:57:28.628761+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:06.870710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:06.870710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.731490+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":757,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627287+0000","last_change":"2024-09-21T11:57:19.418167+0000","last_active":"2024-09-21T11:57:28.627287+0000","last_peered":"2024-09-21T11:57:28.627287+0000","last_clean":"2024-09-21T11:57:28.627287+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:57:28.627287+0000","last_undegraded":"2024-09-21T11:57:28.627287+0000","last_fullsized":"2024-09-21T11:57:28.627287+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:19.418116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:19.418116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:20.299285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"406'5035","reported_seq":5074,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:30.788014+0000","last_change":"2024-09-21T11:56:36.509416+0000","last_active":"2024-09-21T11:57:30.788014+0000","last_peered":"2024-09-21T11:57:30.788014+0000","last_clean":"2024-09-21T11:57:30.788014+0000","last_became_active":"2024-09-21T11:56:36.509115+0000","last_became_peered":"2024-09-21T11:56:36.509115+0000","last_unstale":"2024-09-21T11:57:30.788014+0000","last_undegraded":"2024-09-21T11:57:30.788014+0000","last_fullsized":"2024-09-21T11:57:30.788014+0000","mapping_epoch":392,"log_start":"395'100","ondisk_log_start":"395'100","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4935,"log_dups_size":0,"ondisk_log_size":4935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:39.857708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329973760,"num_objects":5035,"num_object_clones":0,"num_object_copies":15105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5035,"num_write_kb":322240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":632,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628141+0000","last_change":"2024-09-21T11:55:28.581087+0000","last_active":"2024-09-21T11:57:28.628141+0000","last_peered":"2024-09-21T11:57:28.628141+0000","last_clean":"2024-09-21T11:57:28.628141+0000","last_became_active":"2024-09-21T11:55:28.566613+0000","last_became_peered":"2024-09-21T11:55:28.566613+0000","last_unstale":"2024-09-21T11:57:28.628141+0000","last_undegraded":"2024-09-21T11:57:28.628141+0000","last_fullsized":"2024-09-21T11:57:28.628141+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:55:10.349240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:10.349240+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:44.472182+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,7],"acting":[1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":786,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.485107+0000","last_change":"2024-09-21T11:57:01.083611+0000","last_active":"2024-09-21T11:57:32.485107+0000","last_peered":"2024-09-21T11:57:32.485107+0000","last_clean":"2024-09-21T11:57:32.485107+0000","last_became_active":"2024-09-21T11:55:28.572037+0000","last_became_peered":"2024-09-21T11:55:28.572037+0000","last_unstale":"2024-09-21T11:57:32.485107+0000","last_undegraded":"2024-09-21T11:57:32.485107+0000","last_fullsized":"2024-09-21T11:57:32.485107+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:01.083437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.083437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:09.754225+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1845,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:29.545275+0000","last_change":"2024-09-21T11:56:09.604887+0000","last_active":"2024-09-21T11:57:29.545275+0000","last_peered":"2024-09-21T11:57:29.545275+0000","last_clean":"2024-09-21T11:57:29.545275+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:57:29.545275+0000","last_undegraded":"2024-09-21T11:57:29.545275+0000","last_fullsized":"2024-09-21T11:57:29.545275+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:56:09.604808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.604808+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1041,"num_read_kb":1036,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":767,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.618520+0000","last_change":"2024-09-21T11:56:57.019113+0000","last_active":"2024-09-21T11:57:28.618520+0000","last_peered":"2024-09-21T11:57:28.618520+0000","last_clean":"2024-09-21T11:57:28.618520+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:57:28.618520+0000","last_undegraded":"2024-09-21T11:57:28.618520+0000","last_fullsized":"2024-09-21T11:57:28.618520+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:57.019066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:57.019066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:23.845888+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":447,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628226+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:57:28.628226+0000","last_peered":"2024-09-21T11:57:28.628226+0000","last_clean":"2024-09-21T11:57:28.628226+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:57:28.628226+0000","last_undegraded":"2024-09-21T11:57:28.628226+0000","last_fullsized":"2024-09-21T11:57:28.628226+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:29.466749+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"406'4967","reported_seq":5006,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:30.836905+0000","last_change":"2024-09-21T11:56:36.509270+0000","last_active":"2024-09-21T11:57:30.836905+0000","last_peered":"2024-09-21T11:57:30.836905+0000","last_clean":"2024-09-21T11:57:30.836905+0000","last_became_active":"2024-09-21T11:56:36.508533+0000","last_became_peered":"2024-09-21T11:56:36.508533+0000","last_unstale":"2024-09-21T11:57:30.836905+0000","last_undegraded":"2024-09-21T11:57:30.836905+0000","last_fullsized":"2024-09-21T11:57:30.836905+0000","mapping_epoch":392,"log_start":"395'100","ondisk_log_start":"395'100","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4867,"log_dups_size":0,"ondisk_log_size":4867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.121205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":325517312,"num_objects":4967,"num_object_clones":0,"num_object_copies":14901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4967,"num_write_kb":317888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":1008,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622941+0000","last_change":"2024-09-21T11:56:29.095770+0000","last_active":"2024-09-21T11:57:28.622941+0000","last_peered":"2024-09-21T11:57:28.622941+0000","last_clean":"2024-09-21T11:57:28.622941+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:57:28.622941+0000","last_undegraded":"2024-09-21T11:57:28.622941+0000","last_fullsized":"2024-09-21T11:57:28.622941+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:56:29.095713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.095713+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.293114+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":762,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.635955+0000","last_change":"2024-09-21T11:57:14.756298+0000","last_active":"2024-09-21T11:57:28.635955+0000","last_peered":"2024-09-21T11:57:28.635955+0000","last_clean":"2024-09-21T11:57:28.635955+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:57:28.635955+0000","last_undegraded":"2024-09-21T11:57:28.635955+0000","last_fullsized":"2024-09-21T11:57:28.635955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:14.756227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:14.756227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:28.632117+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":839,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627835+0000","last_change":"2024-09-21T11:56:07.609933+0000","last_active":"2024-09-21T11:57:28.627835+0000","last_peered":"2024-09-21T11:57:28.627835+0000","last_clean":"2024-09-21T11:57:28.627835+0000","last_became_active":"2024-09-21T11:55:28.570670+0000","last_became_peered":"2024-09-21T11:55:28.570670+0000","last_unstale":"2024-09-21T11:57:28.627835+0000","last_undegraded":"2024-09-21T11:57:28.627835+0000","last_fullsized":"2024-09-21T11:57:28.627835+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:56:07.609858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:56:07.609858+0000","objects_scrubbed":2,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:35.001556+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":745,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627404+0000","last_change":"2024-09-21T11:51:18.318095+0000","last_active":"2024-09-21T11:57:28.627404+0000","last_peered":"2024-09-21T11:57:28.627404+0000","last_clean":"2024-09-21T11:57:28.627404+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:57:28.627404+0000","last_undegraded":"2024-09-21T11:57:28.627404+0000","last_fullsized":"2024-09-21T11:57:28.627404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:18.317950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:18.317950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:24.370371+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":765,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.618493+0000","last_change":"2024-09-21T11:57:05.904684+0000","last_active":"2024-09-21T11:57:28.618493+0000","last_peered":"2024-09-21T11:57:28.618493+0000","last_clean":"2024-09-21T11:57:28.618493+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:57:28.618493+0000","last_undegraded":"2024-09-21T11:57:28.618493+0000","last_fullsized":"2024-09-21T11:57:28.618493+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:05.904621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:05.904621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:19.946798+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"406'5003","reported_seq":5042,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:30.761492+0000","last_change":"2024-09-21T11:56:36.508185+0000","last_active":"2024-09-21T11:57:30.761492+0000","last_peered":"2024-09-21T11:57:30.761492+0000","last_clean":"2024-09-21T11:57:30.761492+0000","last_became_active":"2024-09-21T11:56:36.507849+0000","last_became_peered":"2024-09-21T11:56:36.507849+0000","last_unstale":"2024-09-21T11:57:30.761492+0000","last_undegraded":"2024-09-21T11:57:30.761492+0000","last_fullsized":"2024-09-21T11:57:30.761492+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5003,"log_dups_size":0,"ondisk_log_size":5003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.525964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327876608,"num_objects":5003,"num_object_clones":0,"num_object_copies":15009,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5003,"num_write_kb":320192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":770,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622640+0000","last_change":"2024-09-21T11:56:09.204336+0000","last_active":"2024-09-21T11:57:28.622640+0000","last_peered":"2024-09-21T11:57:28.622640+0000","last_clean":"2024-09-21T11:57:28.622640+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:57:28.622640+0000","last_undegraded":"2024-09-21T11:57:28.622640+0000","last_fullsized":"2024-09-21T11:57:28.622640+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:56:09.204270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.204270+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":262,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.619974+0000","last_change":"2024-09-21T11:56:16.671887+0000","last_active":"2024-09-21T11:57:28.619974+0000","last_peered":"2024-09-21T11:57:28.619974+0000","last_clean":"2024-09-21T11:57:28.619974+0000","last_became_active":"2024-09-21T11:56:16.671572+0000","last_became_peered":"2024-09-21T11:56:16.671572+0000","last_unstale":"2024-09-21T11:57:28.619974+0000","last_undegraded":"2024-09-21T11:57:28.619974+0000","last_fullsized":"2024-09-21T11:57:28.619974+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:05.663102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.663102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,1],"acting":[2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":751,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.618403+0000","last_change":"2024-09-21T11:56:30.997926+0000","last_active":"2024-09-21T11:57:28.618403+0000","last_peered":"2024-09-21T11:57:28.618403+0000","last_clean":"2024-09-21T11:57:28.618403+0000","last_became_active":"2024-09-21T11:56:16.671360+0000","last_became_peered":"2024-09-21T11:56:16.671360+0000","last_unstale":"2024-09-21T11:57:28.618403+0000","last_undegraded":"2024-09-21T11:57:28.618403+0000","last_fullsized":"2024-09-21T11:57:28.618403+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:30.997880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:30.997880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:34.105058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":755,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628395+0000","last_change":"2024-09-21T11:56:28.556475+0000","last_active":"2024-09-21T11:57:28.628395+0000","last_peered":"2024-09-21T11:57:28.628395+0000","last_clean":"2024-09-21T11:57:28.628395+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:57:28.628395+0000","last_undegraded":"2024-09-21T11:57:28.628395+0000","last_fullsized":"2024-09-21T11:57:28.628395+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.556423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.556423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:33.767058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2","version":"406'5094","reported_seq":5132,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:31.800984+0000","last_change":"2024-09-21T11:56:36.511935+0000","last_active":"2024-09-21T11:57:31.800984+0000","last_peered":"2024-09-21T11:57:31.800984+0000","last_clean":"2024-09-21T11:57:31.800984+0000","last_became_active":"2024-09-21T11:56:36.511642+0000","last_became_peered":"2024-09-21T11:56:36.511642+0000","last_unstale":"2024-09-21T11:57:31.800984+0000","last_undegraded":"2024-09-21T11:57:31.800984+0000","last_fullsized":"2024-09-21T11:57:31.800984+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5094,"log_dups_size":0,"ondisk_log_size":5094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:43.996894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":333840384,"num_objects":5094,"num_object_clones":0,"num_object_copies":15282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5094,"num_write_kb":326016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":858,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.619187+0000","last_change":"2024-09-21T11:56:03.668325+0000","last_active":"2024-09-21T11:57:28.619187+0000","last_peered":"2024-09-21T11:57:28.619187+0000","last_clean":"2024-09-21T11:57:28.619187+0000","last_became_active":"2024-09-21T11:55:28.561575+0000","last_became_peered":"2024-09-21T11:55:28.561575+0000","last_unstale":"2024-09-21T11:57:28.619187+0000","last_undegraded":"2024-09-21T11:57:28.619187+0000","last_fullsized":"2024-09-21T11:57:28.619187+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:56:03.667750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:03.667750+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":783,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622811+0000","last_change":"2024-09-21T11:56:28.045735+0000","last_active":"2024-09-21T11:57:28.622811+0000","last_peered":"2024-09-21T11:57:28.622811+0000","last_clean":"2024-09-21T11:57:28.622811+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:57:28.622811+0000","last_undegraded":"2024-09-21T11:57:28.622811+0000","last_fullsized":"2024-09-21T11:57:28.622811+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.045670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.045670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:35.259699+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":446,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628097+0000","last_change":"2024-09-21T11:55:10.605948+0000","last_active":"2024-09-21T11:57:28.628097+0000","last_peered":"2024-09-21T11:57:28.628097+0000","last_clean":"2024-09-21T11:57:28.628097+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:57:28.628097+0000","last_undegraded":"2024-09-21T11:57:28.628097+0000","last_fullsized":"2024-09-21T11:57:28.628097+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:10.605869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:10.605869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.452804+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":749,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.619657+0000","last_change":"2024-09-21T11:55:24.583429+0000","last_active":"2024-09-21T11:57:28.619657+0000","last_peered":"2024-09-21T11:57:28.619657+0000","last_clean":"2024-09-21T11:57:28.619657+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:57:28.619657+0000","last_undegraded":"2024-09-21T11:57:28.619657+0000","last_fullsized":"2024-09-21T11:57:28.619657+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.583208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.583208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:41.672664+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"406'4895","reported_seq":4934,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:30.819972+0000","last_change":"2024-09-21T11:56:36.504822+0000","last_active":"2024-09-21T11:57:30.819972+0000","last_peered":"2024-09-21T11:57:30.819972+0000","last_clean":"2024-09-21T11:57:30.819972+0000","last_became_active":"2024-09-21T11:56:36.504516+0000","last_became_peered":"2024-09-21T11:56:36.504516+0000","last_unstale":"2024-09-21T11:57:30.819972+0000","last_undegraded":"2024-09-21T11:57:30.819972+0000","last_fullsized":"2024-09-21T11:57:30.819972+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4895,"log_dups_size":0,"ondisk_log_size":4895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:54.965928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320798720,"num_objects":4895,"num_object_clones":0,"num_object_copies":14685,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4895,"num_write_kb":313280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":739,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.635181+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:57:28.635181+0000","last_peered":"2024-09-21T11:57:28.635181+0000","last_clean":"2024-09-21T11:57:28.635181+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:57:28.635181+0000","last_undegraded":"2024-09-21T11:57:28.635181+0000","last_fullsized":"2024-09-21T11:57:28.635181+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:23.696543+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":752,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.629125+0000","last_change":"2024-09-21T11:56:09.486510+0000","last_active":"2024-09-21T11:57:28.629125+0000","last_peered":"2024-09-21T11:57:28.629125+0000","last_clean":"2024-09-21T11:57:28.629125+0000","last_became_active":"2024-09-21T11:55:28.568312+0000","last_became_peered":"2024-09-21T11:55:28.568312+0000","last_unstale":"2024-09-21T11:57:28.629125+0000","last_undegraded":"2024-09-21T11:57:28.629125+0000","last_fullsized":"2024-09-21T11:57:28.629125+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:09.486386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.486386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:29.172746+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":980,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.619134+0000","last_change":"2024-09-21T11:56:08.680971+0000","last_active":"2024-09-21T11:57:28.619134+0000","last_peered":"2024-09-21T11:57:28.619134+0000","last_clean":"2024-09-21T11:57:28.619134+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:57:28.619134+0000","last_undegraded":"2024-09-21T11:57:28.619134+0000","last_fullsized":"2024-09-21T11:57:28.619134+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:56:08.680766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.680766+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":573,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628547+0000","last_change":"2024-09-21T11:56:16.669124+0000","last_active":"2024-09-21T11:57:28.628547+0000","last_peered":"2024-09-21T11:57:28.628547+0000","last_clean":"2024-09-21T11:57:28.628547+0000","last_became_active":"2024-09-21T11:56:16.668785+0000","last_became_peered":"2024-09-21T11:56:16.668785+0000","last_unstale":"2024-09-21T11:57:28.628547+0000","last_undegraded":"2024-09-21T11:57:28.628547+0000","last_fullsized":"2024-09-21T11:57:28.628547+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:07.658022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:07.658022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:42.380428+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"406'4983","reported_seq":5022,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:31.254887+0000","last_change":"2024-09-21T11:56:36.508655+0000","last_active":"2024-09-21T11:57:31.254887+0000","last_peered":"2024-09-21T11:57:31.254887+0000","last_clean":"2024-09-21T11:57:31.254887+0000","last_became_active":"2024-09-21T11:56:36.508299+0000","last_became_peered":"2024-09-21T11:56:36.508299+0000","last_unstale":"2024-09-21T11:57:31.254887+0000","last_undegraded":"2024-09-21T11:57:31.254887+0000","last_fullsized":"2024-09-21T11:57:31.254887+0000","mapping_epoch":392,"log_start":"396'300","ondisk_log_start":"396'300","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4683,"log_dups_size":0,"ondisk_log_size":4683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:05.125216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":326565888,"num_objects":4983,"num_object_clones":0,"num_object_copies":14949,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4983,"num_write_kb":318912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":801,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.484465+0000","last_change":"2024-09-21T11:56:48.169371+0000","last_active":"2024-09-21T11:57:32.484465+0000","last_peered":"2024-09-21T11:57:32.484465+0000","last_clean":"2024-09-21T11:57:32.484465+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:57:32.484465+0000","last_undegraded":"2024-09-21T11:57:32.484465+0000","last_fullsized":"2024-09-21T11:57:32.484465+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:48.168795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.168795+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:09.022696+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":784,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.616037+0000","last_change":"2024-09-21T11:56:59.989715+0000","last_active":"2024-09-21T11:57:28.616037+0000","last_peered":"2024-09-21T11:57:28.616037+0000","last_clean":"2024-09-21T11:57:28.616037+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:57:28.616037+0000","last_undegraded":"2024-09-21T11:57:28.616037+0000","last_fullsized":"2024-09-21T11:57:28.616037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:59.989636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.989636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:25.546942+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":745,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.618657+0000","last_change":"2024-09-21T11:56:16.668904+0000","last_active":"2024-09-21T11:57:28.618657+0000","last_peered":"2024-09-21T11:57:28.618657+0000","last_clean":"2024-09-21T11:57:28.618657+0000","last_became_active":"2024-09-21T11:56:16.668588+0000","last_became_peered":"2024-09-21T11:56:16.668588+0000","last_unstale":"2024-09-21T11:57:28.618657+0000","last_undegraded":"2024-09-21T11:57:28.618657+0000","last_fullsized":"2024-09-21T11:57:28.618657+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:26.990626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:53:26.990626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":103,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626694+0000","last_change":"2024-09-21T11:55:35.532779+0000","last_active":"2024-09-21T11:57:28.626694+0000","last_peered":"2024-09-21T11:57:28.626694+0000","last_clean":"2024-09-21T11:57:28.626694+0000","last_became_active":"2024-09-21T11:55:28.566108+0000","last_became_peered":"2024-09-21T11:55:28.566108+0000","last_unstale":"2024-09-21T11:57:28.626694+0000","last_undegraded":"2024-09-21T11:57:28.626694+0000","last_fullsized":"2024-09-21T11:57:28.626694+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:35.532577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:35.532577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:42.362223+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.5","version":"406'4896","reported_seq":4935,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:31.279431+0000","last_change":"2024-09-21T11:56:36.506979+0000","last_active":"2024-09-21T11:57:31.279431+0000","last_peered":"2024-09-21T11:57:31.279431+0000","last_clean":"2024-09-21T11:57:31.279431+0000","last_became_active":"2024-09-21T11:56:36.506694+0000","last_became_peered":"2024-09-21T11:56:36.506694+0000","last_unstale":"2024-09-21T11:57:31.279431+0000","last_undegraded":"2024-09-21T11:57:31.279431+0000","last_fullsized":"2024-09-21T11:57:31.279431+0000","mapping_epoch":392,"log_start":"396'200","ondisk_log_start":"396'200","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.639288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320864256,"num_objects":4896,"num_object_clones":0,"num_object_copies":14688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4896,"num_write_kb":313344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":858,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622571+0000","last_change":"2024-09-21T11:50:24.359357+0000","last_active":"2024-09-21T11:57:28.622571+0000","last_peered":"2024-09-21T11:57:28.622571+0000","last_clean":"2024-09-21T11:57:28.622571+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:57:28.622571+0000","last_undegraded":"2024-09-21T11:57:28.622571+0000","last_fullsized":"2024-09-21T11:57:28.622571+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:50:24.359263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:24.359263+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:28.881029+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":752,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627636+0000","last_change":"2024-09-21T11:56:50.313230+0000","last_active":"2024-09-21T11:57:28.627636+0000","last_peered":"2024-09-21T11:57:28.627636+0000","last_clean":"2024-09-21T11:57:28.627636+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:57:28.627636+0000","last_undegraded":"2024-09-21T11:57:28.627636+0000","last_fullsized":"2024-09-21T11:57:28.627636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.313165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.313165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:09.383551+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":753,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.618900+0000","last_change":"2024-09-21T11:51:48.941202+0000","last_active":"2024-09-21T11:57:28.618900+0000","last_peered":"2024-09-21T11:57:28.618900+0000","last_clean":"2024-09-21T11:57:28.618900+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:57:28.618900+0000","last_undegraded":"2024-09-21T11:57:28.618900+0000","last_fullsized":"2024-09-21T11:57:28.618900+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:48.941128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:48.941128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:25.848816+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":757,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627791+0000","last_change":"2024-09-21T11:56:55.414105+0000","last_active":"2024-09-21T11:57:28.627791+0000","last_peered":"2024-09-21T11:57:28.627791+0000","last_clean":"2024-09-21T11:57:28.627791+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:57:28.627791+0000","last_undegraded":"2024-09-21T11:57:28.627791+0000","last_fullsized":"2024-09-21T11:57:28.627791+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:55.414046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:55.414046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:01.463812+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"406'5070","reported_seq":5109,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:31.726309+0000","last_change":"2024-09-21T11:56:36.623904+0000","last_active":"2024-09-21T11:57:31.726309+0000","last_peered":"2024-09-21T11:57:31.726309+0000","last_clean":"2024-09-21T11:57:31.726309+0000","last_became_active":"2024-09-21T11:56:36.623611+0000","last_became_peered":"2024-09-21T11:56:36.623611+0000","last_unstale":"2024-09-21T11:57:31.726309+0000","last_undegraded":"2024-09-21T11:57:31.726309+0000","last_fullsized":"2024-09-21T11:57:31.726309+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5070,"log_dups_size":0,"ondisk_log_size":5070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.371771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":332267520,"num_objects":5070,"num_object_clones":0,"num_object_copies":15210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5070,"num_write_kb":324480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":779,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626433+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:57:28.626433+0000","last_peered":"2024-09-21T11:57:28.626433+0000","last_clean":"2024-09-21T11:57:28.626433+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:57:28.626433+0000","last_undegraded":"2024-09-21T11:57:28.626433+0000","last_fullsized":"2024-09-21T11:57:28.626433+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:21.513710+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":797,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627554+0000","last_change":"2024-09-21T11:56:29.486116+0000","last_active":"2024-09-21T11:57:28.627554+0000","last_peered":"2024-09-21T11:57:28.627554+0000","last_clean":"2024-09-21T11:57:28.627554+0000","last_became_active":"2024-09-21T11:56:16.670969+0000","last_became_peered":"2024-09-21T11:56:16.670969+0000","last_unstale":"2024-09-21T11:57:28.627554+0000","last_undegraded":"2024-09-21T11:57:28.627554+0000","last_fullsized":"2024-09-21T11:57:28.627554+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:29.486067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.486067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:47.036894+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":768,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.619736+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:57:28.619736+0000","last_peered":"2024-09-21T11:57:28.619736+0000","last_clean":"2024-09-21T11:57:28.619736+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:57:28.619736+0000","last_undegraded":"2024-09-21T11:57:28.619736+0000","last_fullsized":"2024-09-21T11:57:28.619736+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":755,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628739+0000","last_change":"2024-09-21T11:56:02.541452+0000","last_active":"2024-09-21T11:57:28.628739+0000","last_peered":"2024-09-21T11:57:28.628739+0000","last_clean":"2024-09-21T11:57:28.628739+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:57:28.628739+0000","last_undegraded":"2024-09-21T11:57:28.628739+0000","last_fullsized":"2024-09-21T11:57:28.628739+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.541232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.541232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"406'4916","reported_seq":4955,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:31.418624+0000","last_change":"2024-09-21T11:56:36.624091+0000","last_active":"2024-09-21T11:57:31.418624+0000","last_peered":"2024-09-21T11:57:31.418624+0000","last_clean":"2024-09-21T11:57:31.418624+0000","last_became_active":"2024-09-21T11:56:36.623765+0000","last_became_peered":"2024-09-21T11:56:36.623765+0000","last_unstale":"2024-09-21T11:57:31.418624+0000","last_undegraded":"2024-09-21T11:57:31.418624+0000","last_fullsized":"2024-09-21T11:57:31.418624+0000","mapping_epoch":392,"log_start":"0'0","ondisk_log_start":"0'0","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4916,"log_dups_size":0,"ondisk_log_size":4916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:59.412761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322174976,"num_objects":4916,"num_object_clones":0,"num_object_copies":14748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4916,"num_write_kb":314624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":800,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.615162+0000","last_change":"2024-09-21T11:56:29.063211+0000","last_active":"2024-09-21T11:57:28.615162+0000","last_peered":"2024-09-21T11:57:28.615162+0000","last_clean":"2024-09-21T11:57:28.615162+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:57:28.615162+0000","last_undegraded":"2024-09-21T11:57:28.615162+0000","last_fullsized":"2024-09-21T11:57:28.615162+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:29.063150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.063150+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:45.931544+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":801,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627297+0000","last_change":"2024-09-21T11:57:01.222171+0000","last_active":"2024-09-21T11:57:28.627297+0000","last_peered":"2024-09-21T11:57:28.627297+0000","last_clean":"2024-09-21T11:57:28.627297+0000","last_became_active":"2024-09-21T11:55:28.561948+0000","last_became_peered":"2024-09-21T11:55:28.561948+0000","last_unstale":"2024-09-21T11:57:28.627297+0000","last_undegraded":"2024-09-21T11:57:28.627297+0000","last_fullsized":"2024-09-21T11:57:28.627297+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:57:01.221960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.221960+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:23.325733+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":980,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626877+0000","last_change":"2024-09-21T11:57:10.282180+0000","last_active":"2024-09-21T11:57:28.626877+0000","last_peered":"2024-09-21T11:57:28.626877+0000","last_clean":"2024-09-21T11:57:28.626877+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:57:28.626877+0000","last_undegraded":"2024-09-21T11:57:28.626877+0000","last_fullsized":"2024-09-21T11:57:28.626877+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:10.282134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.282134+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:12.540343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":733,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.615309+0000","last_change":"2024-09-21T11:53:27.992553+0000","last_active":"2024-09-21T11:57:28.615309+0000","last_peered":"2024-09-21T11:57:28.615309+0000","last_clean":"2024-09-21T11:57:28.615309+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:57:28.615309+0000","last_undegraded":"2024-09-21T11:57:28.615309+0000","last_fullsized":"2024-09-21T11:57:28.615309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:53:27.992489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.992489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:41.096226+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":775,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622688+0000","last_change":"2024-09-21T11:55:28.557481+0000","last_active":"2024-09-21T11:57:28.622688+0000","last_peered":"2024-09-21T11:57:28.622688+0000","last_clean":"2024-09-21T11:57:28.622688+0000","last_became_active":"2024-09-21T11:55:28.557190+0000","last_became_peered":"2024-09-21T11:55:28.557190+0000","last_unstale":"2024-09-21T11:57:28.622688+0000","last_undegraded":"2024-09-21T11:57:28.622688+0000","last_fullsized":"2024-09-21T11:57:28.622688+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:38.208809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:38.208809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.969866+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":789,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.634904+0000","last_change":"2024-09-21T11:56:29.528662+0000","last_active":"2024-09-21T11:57:28.634904+0000","last_peered":"2024-09-21T11:57:28.634904+0000","last_clean":"2024-09-21T11:57:28.634904+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:57:28.634904+0000","last_undegraded":"2024-09-21T11:57:28.634904+0000","last_fullsized":"2024-09-21T11:57:28.634904+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:56:29.528598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.528598+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:47.850219+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":790,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.616367+0000","last_change":"2024-09-21T11:57:15.919990+0000","last_active":"2024-09-21T11:57:28.616367+0000","last_peered":"2024-09-21T11:57:28.616367+0000","last_clean":"2024-09-21T11:57:28.616367+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:57:28.616367+0000","last_undegraded":"2024-09-21T11:57:28.616367+0000","last_fullsized":"2024-09-21T11:57:28.616367+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:15.919902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:15.919902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:20.920551+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":771,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627195+0000","last_change":"2024-09-21T11:57:04.198898+0000","last_active":"2024-09-21T11:57:28.627195+0000","last_peered":"2024-09-21T11:57:28.627195+0000","last_clean":"2024-09-21T11:57:28.627195+0000","last_became_active":"2024-09-21T11:56:16.670467+0000","last_became_peered":"2024-09-21T11:56:16.670467+0000","last_unstale":"2024-09-21T11:57:28.627195+0000","last_undegraded":"2024-09-21T11:57:28.627195+0000","last_fullsized":"2024-09-21T11:57:28.627195+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:04.198818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:04.198818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:16.880979+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":784,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.484015+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:57:32.484015+0000","last_peered":"2024-09-21T11:57:32.484015+0000","last_clean":"2024-09-21T11:57:32.484015+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:57:32.484015+0000","last_undegraded":"2024-09-21T11:57:32.484015+0000","last_fullsized":"2024-09-21T11:57:32.484015+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":744,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627037+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:57:28.627037+0000","last_peered":"2024-09-21T11:57:28.627037+0000","last_clean":"2024-09-21T11:57:28.627037+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:57:28.627037+0000","last_undegraded":"2024-09-21T11:57:28.627037+0000","last_fullsized":"2024-09-21T11:57:28.627037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":775,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.634779+0000","last_change":"2024-09-21T11:56:50.696344+0000","last_active":"2024-09-21T11:57:28.634779+0000","last_peered":"2024-09-21T11:57:28.634779+0000","last_clean":"2024-09-21T11:57:28.634779+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:57:28.634779+0000","last_undegraded":"2024-09-21T11:57:28.634779+0000","last_fullsized":"2024-09-21T11:57:28.634779+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.696253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.696253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:00.421085+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":749,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.618756+0000","last_change":"2024-09-21T11:57:12.518014+0000","last_active":"2024-09-21T11:57:28.618756+0000","last_peered":"2024-09-21T11:57:28.618756+0000","last_clean":"2024-09-21T11:57:28.618756+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:57:28.618756+0000","last_undegraded":"2024-09-21T11:57:28.618756+0000","last_fullsized":"2024-09-21T11:57:28.618756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:12.517953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:12.517953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:19.717876+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":781,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.616721+0000","last_change":"2024-09-21T11:56:34.119200+0000","last_active":"2024-09-21T11:57:28.616721+0000","last_peered":"2024-09-21T11:57:28.616721+0000","last_clean":"2024-09-21T11:57:28.616721+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:57:28.616721+0000","last_undegraded":"2024-09-21T11:57:28.616721+0000","last_fullsized":"2024-09-21T11:57:28.616721+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:34.119128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:34.119128+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.898211+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":770,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.484550+0000","last_change":"2024-09-21T11:55:04.304207+0000","last_active":"2024-09-21T11:57:32.484550+0000","last_peered":"2024-09-21T11:57:32.484550+0000","last_clean":"2024-09-21T11:57:32.484550+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:57:32.484550+0000","last_undegraded":"2024-09-21T11:57:32.484550+0000","last_fullsized":"2024-09-21T11:57:32.484550+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:04.304136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:04.304136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:45.199932+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":948,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628651+0000","last_change":"2024-09-21T11:55:51.317564+0000","last_active":"2024-09-21T11:57:28.628651+0000","last_peered":"2024-09-21T11:57:28.628651+0000","last_clean":"2024-09-21T11:57:28.628651+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:57:28.628651+0000","last_undegraded":"2024-09-21T11:57:28.628651+0000","last_fullsized":"2024-09-21T11:57:28.628651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:51.317473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.317473+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":239,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.619569+0000","last_change":"2024-09-21T11:51:43.531318+0000","last_active":"2024-09-21T11:57:28.619569+0000","last_peered":"2024-09-21T11:57:28.619569+0000","last_clean":"2024-09-21T11:57:28.619569+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:57:28.619569+0000","last_undegraded":"2024-09-21T11:57:28.619569+0000","last_fullsized":"2024-09-21T11:57:28.619569+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:20.826834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:51:20.826834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:19.797406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":809,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.635320+0000","last_change":"2024-09-21T11:57:16.786845+0000","last_active":"2024-09-21T11:57:28.635320+0000","last_peered":"2024-09-21T11:57:28.635320+0000","last_clean":"2024-09-21T11:57:28.635320+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:57:28.635320+0000","last_undegraded":"2024-09-21T11:57:28.635320+0000","last_fullsized":"2024-09-21T11:57:28.635320+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:57:16.786756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:16.786756+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:26.307010+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":752,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628486+0000","last_change":"2024-09-21T11:56:33.473303+0000","last_active":"2024-09-21T11:57:28.628486+0000","last_peered":"2024-09-21T11:57:28.628486+0000","last_clean":"2024-09-21T11:57:28.628486+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:57:28.628486+0000","last_undegraded":"2024-09-21T11:57:28.628486+0000","last_fullsized":"2024-09-21T11:57:28.628486+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.473240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.473240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:36.182237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":1002,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.616543+0000","last_change":"2024-09-21T11:56:56.051487+0000","last_active":"2024-09-21T11:57:28.616543+0000","last_peered":"2024-09-21T11:57:28.616543+0000","last_clean":"2024-09-21T11:57:28.616543+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:57:28.616543+0000","last_undegraded":"2024-09-21T11:57:28.616543+0000","last_fullsized":"2024-09-21T11:57:28.616543+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:56.051300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:56.051300+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:11.899165+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":111,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627089+0000","last_change":"2024-09-21T11:56:58.280761+0000","last_active":"2024-09-21T11:57:28.627089+0000","last_peered":"2024-09-21T11:57:28.627089+0000","last_clean":"2024-09-21T11:57:28.627089+0000","last_became_active":"2024-09-21T11:55:28.565700+0000","last_became_peered":"2024-09-21T11:55:28.565700+0000","last_unstale":"2024-09-21T11:57:28.627089+0000","last_undegraded":"2024-09-21T11:57:28.627089+0000","last_fullsized":"2024-09-21T11:57:28.627089+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.280698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.280698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:01.265389+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":789,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622508+0000","last_change":"2024-09-21T11:51:43.533694+0000","last_active":"2024-09-21T11:57:28.622508+0000","last_peered":"2024-09-21T11:57:28.622508+0000","last_clean":"2024-09-21T11:57:28.622508+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:57:28.622508+0000","last_undegraded":"2024-09-21T11:57:28.622508+0000","last_fullsized":"2024-09-21T11:57:28.622508+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:50:35.363840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:35.363840+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:24.915640+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":255,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627672+0000","last_change":"2024-09-21T11:55:24.538954+0000","last_active":"2024-09-21T11:57:28.627672+0000","last_peered":"2024-09-21T11:57:28.627672+0000","last_clean":"2024-09-21T11:57:28.627672+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:57:28.627672+0000","last_undegraded":"2024-09-21T11:57:28.627672+0000","last_fullsized":"2024-09-21T11:57:28.627672+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.538880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.538880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:40.390211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":454,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626677+0000","last_change":"2024-09-21T11:56:00.441644+0000","last_active":"2024-09-21T11:57:28.626677+0000","last_peered":"2024-09-21T11:57:28.626677+0000","last_clean":"2024-09-21T11:57:28.626677+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:57:28.626677+0000","last_undegraded":"2024-09-21T11:57:28.626677+0000","last_fullsized":"2024-09-21T11:57:28.626677+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:00.441565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:00.441565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":740,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.629440+0000","last_change":"2024-09-21T11:55:28.563670+0000","last_active":"2024-09-21T11:57:28.629440+0000","last_peered":"2024-09-21T11:57:28.629440+0000","last_clean":"2024-09-21T11:57:28.629440+0000","last_became_active":"2024-09-21T11:55:28.563340+0000","last_became_peered":"2024-09-21T11:55:28.563340+0000","last_unstale":"2024-09-21T11:57:28.629440+0000","last_undegraded":"2024-09-21T11:57:28.629440+0000","last_fullsized":"2024-09-21T11:57:28.629440+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:01.434123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.434123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:43.602222+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":465,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626894+0000","last_change":"2024-09-21T11:55:59.419239+0000","last_active":"2024-09-21T11:57:28.626894+0000","last_peered":"2024-09-21T11:57:28.626894+0000","last_clean":"2024-09-21T11:57:28.626894+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:57:28.626894+0000","last_undegraded":"2024-09-21T11:57:28.626894+0000","last_fullsized":"2024-09-21T11:57:28.626894+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:59.419148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.419148+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":752,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627474+0000","last_change":"2024-09-21T11:57:17.492529+0000","last_active":"2024-09-21T11:57:28.627474+0000","last_peered":"2024-09-21T11:57:28.627474+0000","last_clean":"2024-09-21T11:57:28.627474+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:57:28.627474+0000","last_undegraded":"2024-09-21T11:57:28.627474+0000","last_fullsized":"2024-09-21T11:57:28.627474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:17.492466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:17.492466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:31.386547+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":751,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628824+0000","last_change":"2024-09-21T11:49:49.210005+0000","last_active":"2024-09-21T11:57:28.628824+0000","last_peered":"2024-09-21T11:57:28.628824+0000","last_clean":"2024-09-21T11:57:28.628824+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:57:28.628824+0000","last_undegraded":"2024-09-21T11:57:28.628824+0000","last_fullsized":"2024-09-21T11:57:28.628824+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:49:49.209938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:49:49.209938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:20.749247+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":781,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.483481+0000","last_change":"2024-09-21T11:55:59.288321+0000","last_active":"2024-09-21T11:57:32.483481+0000","last_peered":"2024-09-21T11:57:32.483481+0000","last_clean":"2024-09-21T11:57:32.483481+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:57:32.483481+0000","last_undegraded":"2024-09-21T11:57:32.483481+0000","last_fullsized":"2024-09-21T11:57:32.483481+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:55:59.288240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.288240+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":788,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.629671+0000","last_change":"2024-09-21T11:57:18.759800+0000","last_active":"2024-09-21T11:57:28.629671+0000","last_peered":"2024-09-21T11:57:28.629671+0000","last_clean":"2024-09-21T11:57:28.629671+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:57:28.629671+0000","last_undegraded":"2024-09-21T11:57:28.629671+0000","last_fullsized":"2024-09-21T11:57:28.629671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:57:18.759710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:18.759710+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:21.149967+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":774,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.616488+0000","last_change":"2024-09-21T11:56:47.967479+0000","last_active":"2024-09-21T11:57:28.616488+0000","last_peered":"2024-09-21T11:57:28.616488+0000","last_clean":"2024-09-21T11:57:28.616488+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:57:28.616488+0000","last_undegraded":"2024-09-21T11:57:28.616488+0000","last_fullsized":"2024-09-21T11:57:28.616488+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:47.967392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:47.967392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:10.636665+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":728,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626577+0000","last_change":"2024-09-21T11:51:10.352053+0000","last_active":"2024-09-21T11:57:28.626577+0000","last_peered":"2024-09-21T11:57:28.626577+0000","last_clean":"2024-09-21T11:57:28.626577+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:57:28.626577+0000","last_undegraded":"2024-09-21T11:57:28.626577+0000","last_fullsized":"2024-09-21T11:57:28.626577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:10.351997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:51:10.351997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:22.372804+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":748,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.629000+0000","last_change":"2024-09-21T11:55:28.564578+0000","last_active":"2024-09-21T11:57:28.629000+0000","last_peered":"2024-09-21T11:57:28.629000+0000","last_clean":"2024-09-21T11:57:28.629000+0000","last_became_active":"2024-09-21T11:55:28.564261+0000","last_became_peered":"2024-09-21T11:55:28.564261+0000","last_unstale":"2024-09-21T11:57:28.629000+0000","last_undegraded":"2024-09-21T11:57:28.629000+0000","last_fullsized":"2024-09-21T11:57:28.629000+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.551852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.551852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:39.540862+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":792,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628657+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:57:28.628657+0000","last_peered":"2024-09-21T11:57:28.628657+0000","last_clean":"2024-09-21T11:57:28.628657+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:57:28.628657+0000","last_undegraded":"2024-09-21T11:57:28.628657+0000","last_fullsized":"2024-09-21T11:57:28.628657+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":746,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.484293+0000","last_change":"2024-09-21T11:56:53.232038+0000","last_active":"2024-09-21T11:57:32.484293+0000","last_peered":"2024-09-21T11:57:32.484293+0000","last_clean":"2024-09-21T11:57:32.484293+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:57:32.484293+0000","last_undegraded":"2024-09-21T11:57:32.484293+0000","last_fullsized":"2024-09-21T11:57:32.484293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.231963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.231963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:57.228542+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":751,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.622742+0000","last_change":"2024-09-21T11:55:52.217954+0000","last_active":"2024-09-21T11:57:28.622742+0000","last_peered":"2024-09-21T11:57:28.622742+0000","last_clean":"2024-09-21T11:57:28.622742+0000","last_became_active":"2024-09-21T11:55:28.575256+0000","last_became_peered":"2024-09-21T11:55:28.575256+0000","last_unstale":"2024-09-21T11:57:28.622742+0000","last_undegraded":"2024-09-21T11:57:28.622742+0000","last_fullsized":"2024-09-21T11:57:28.622742+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:52.217898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.217898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":748,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.626800+0000","last_change":"2024-09-21T11:54:57.381670+0000","last_active":"2024-09-21T11:57:28.626800+0000","last_peered":"2024-09-21T11:57:28.626800+0000","last_clean":"2024-09-21T11:57:28.626800+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:57:28.626800+0000","last_undegraded":"2024-09-21T11:57:28.626800+0000","last_fullsized":"2024-09-21T11:57:28.626800+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:57.381605+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:54:57.381605+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:37.460530+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":453,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.627371+0000","last_change":"2024-09-21T11:56:16.669730+0000","last_active":"2024-09-21T11:57:28.627371+0000","last_peered":"2024-09-21T11:57:28.627371+0000","last_clean":"2024-09-21T11:57:28.627371+0000","last_became_active":"2024-09-21T11:56:16.669407+0000","last_became_peered":"2024-09-21T11:56:16.669407+0000","last_unstale":"2024-09-21T11:57:28.627371+0000","last_undegraded":"2024-09-21T11:57:28.627371+0000","last_fullsized":"2024-09-21T11:57:28.627371+0000","mapping_epoch":381,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":382,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:26.462785+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,2],"acting":[1,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":774,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.628915+0000","last_change":"2024-09-21T11:55:28.563845+0000","last_active":"2024-09-21T11:57:28.628915+0000","last_peered":"2024-09-21T11:57:28.628915+0000","last_clean":"2024-09-21T11:57:28.628915+0000","last_became_active":"2024-09-21T11:55:28.563540+0000","last_became_peered":"2024-09-21T11:55:28.563540+0000","last_unstale":"2024-09-21T11:57:28.628915+0000","last_undegraded":"2024-09-21T11:57:28.628915+0000","last_fullsized":"2024-09-21T11:57:28.628915+0000","mapping_epoch":364,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":365,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:23.601318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:54:23.601318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:38.497705+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":850,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-21T11:57:28.623042+0000","last_change":"2024-09-21T11:56:33.048379+0000","last_active":"2024-09-21T11:57:28.623042+0000","last_peered":"2024-09-21T11:57:28.623042+0000","last_clean":"2024-09-21T11:57:28.623042+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:57:28.623042+0000","last_undegraded":"2024-09-21T11:57:28.623042+0000","last_fullsized":"2024-09-21T11:57:28.623042+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:56:33.048316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.048316+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:48.955454+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":747,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-21T11:57:32.483717+0000","last_change":"2024-09-21T11:50:05.730519+0000","last_active":"2024-09-21T11:57:32.483717+0000","last_peered":"2024-09-21T11:57:32.483717+0000","last_clean":"2024-09-21T11:57:32.483717+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:57:32.483717+0000","last_undegraded":"2024-09-21T11:57:32.483717+0000","last_fullsized":"2024-09-21T11:57:32.483717+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:05.730468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:05.730468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:57:47.211178+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":5256577024,"num_objects":80209,"num_object_clones":0,"num_object_copies":240627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80209,"num_write_kb":5133376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15785590784,"data_stored":15785590784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77009,"ondisk_log_size":77009,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71,"ondisk_log_size":71,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":6024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":758,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1041,"num_read_kb":1036,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234199,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2902504,"kb_used_data":2243800,"kb_used_omap":1,"kb_used_meta":658686,"kb_avail":90842648,"statfs":{"total":95995035648,"available":93022871552,"internally_reserved":0,"allocated":2297651200,"data_stored":2297194676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":674494922},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463132,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3078040,"kb_used_data":2505412,"kb_used_omap":1,"kb_used_meta":572606,"kb_avail":90667112,"statfs":{"total":95995035648,"available":92843122688,"internally_reserved":0,"allocated":2565541888,"data_stored":2565105968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":586348998},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724769,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1881372,"kb_used_data":1264332,"kb_used_omap":1,"kb_used_meta":617022,"kb_avail":91863780,"statfs":{"total":95995035648,"available":94068510720,"internally_reserved":0,"allocated":1294675968,"data_stored":1294230274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":631830982},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920999,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1485720,"kb_used_data":940864,"kb_used_omap":1,"kb_used_meta":544830,"kb_avail":92259432,"statfs":{"total":95995035648,"available":94473658368,"internally_reserved":0,"allocated":963444736,"data_stored":963012005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":557906373},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117229,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2978516,"kb_used_data":2247616,"kb_used_omap":1,"kb_used_meta":730878,"kb_avail":90766636,"statfs":{"total":95995035648,"available":92945035264,"internally_reserved":0,"allocated":2301558784,"data_stored":2301125254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":748419527},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411571,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2630428,"kb_used_data":1949516,"kb_used_omap":1,"kb_used_meta":680894,"kb_avail":91114724,"statfs":{"total":95995035648,"available":93301477376,"internally_reserved":0,"allocated":1996304384,"data_stored":1995859497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":697235909},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575098,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3585476,"kb_used_data":2904564,"kb_used_omap":1,"kb_used_meta":680894,"kb_avail":90159676,"statfs":{"total":95995035648,"available":92323508224,"internally_reserved":0,"allocated":2974273536,"data_stored":2973788765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":697235910},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738624,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1939780,"kb_used_data":1299124,"kb_used_omap":1,"kb_used_meta":640638,"kb_avail":91805372,"statfs":{"total":95995035648,"available":94008700928,"internally_reserved":0,"allocated":1330302976,"data_stored":1329882826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":656013765},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1336803328,"data_stored":1336803328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2977300480,"data_stored":2977300480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1996423168,"data_stored":1996423168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2303983616,"data_stored":2303983616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":971309056,"data_stored":971309056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1303379968,"data_stored":1303379968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2599288832,"data_stored":2599288832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2297102336,"data_stored":2297102336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:57:36.686 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-21T11:57:36.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool set .mgr pg_num 11 2024-09-21T11:57:36.687 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:36.701 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 56 16 86325 86309 96.2919 73.0625 0.00488644 0.0103785 2024-09-21T11:57:36.714 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:36.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:36.716 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:36.732 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:36.744 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:36.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:36.843 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:36.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:36.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:36.946 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:36.970 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:36.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:36.971 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:37.072 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:37.095 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:37.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:37.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:37.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:37.217 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:36 smithi043 ceph-mon[29710]: 6.0 scrub starts 2024-09-21T11:57:37.217 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:36 smithi043 ceph-mon[29710]: 6.0 scrub ok 2024-09-21T11:57:37.217 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:36 smithi043 ceph-mon[29710]: pgmap v852: 148 pgs: 148 active+clean; 5.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-21T11:57:37.217 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:36 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:57:37.217 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:36 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:57:37.217 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:36 smithi043 ceph-mon[36454]: 6.0 scrub starts 2024-09-21T11:57:37.217 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:36 smithi043 ceph-mon[36454]: 6.0 scrub ok 2024-09-21T11:57:37.217 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:36 smithi043 ceph-mon[36454]: pgmap v852: 148 pgs: 148 active+clean; 5.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-21T11:57:37.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:36 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:57:37.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:36 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:57:37.253 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:37.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:37.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:37.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:36 smithi175 ceph-mon[32613]: 6.0 scrub starts 2024-09-21T11:57:37.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:36 smithi175 ceph-mon[32613]: 6.0 scrub ok 2024-09-21T11:57:37.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:36 smithi175 ceph-mon[32613]: pgmap v852: 148 pgs: 148 active+clean; 5.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-21T11:57:37.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:36 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:57:37.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:36 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:57:37.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:37.390 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:37.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:37.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:37.491 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:37.522 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:37.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:37.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:37.624 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:37.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:37.661 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:37.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:37.701 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 57 16 87368 87352 95.7462 65.1875 0.050402 0.0104377 2024-09-21T11:57:37.762 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:37.776 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:57:37.777 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:57:37.794 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:37.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:37.795 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:37.813 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:57:37.813 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:57:37.814 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:57:37.814 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:57:37.855 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:57:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:57:37.895 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:57:37.895 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:57:37.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:37.932 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:57:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:57:37.937 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:37.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:37.938 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:37.965 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:57:37.965 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:57:38.001 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:57:38.001 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:57:38.036 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[29710]: 6.10 scrub starts 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[29710]: 6.10 scrub ok 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[29710]: osdmap e410: 8 total, 8 up, 8 in 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[29710]: 3.1e scrub starts 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[29710]: 3.1e scrub ok 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3479746328' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[36454]: 6.10 scrub starts 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[36454]: 6.10 scrub ok 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[36454]: osdmap e410: 8 total, 8 up, 8 in 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[36454]: 3.1e scrub starts 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[36454]: 3.1e scrub ok 2024-09-21T11:57:38.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:37 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3479746328' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:38.038 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:57:38.038 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:57:38.039 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:38.065 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:38.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:38.066 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:38.166 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:38.197 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:38.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:38.198 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:37 smithi175 ceph-mon[32613]: 6.10 scrub starts 2024-09-21T11:57:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:37 smithi175 ceph-mon[32613]: 6.10 scrub ok 2024-09-21T11:57:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:37 smithi175 ceph-mon[32613]: osdmap e410: 8 total, 8 up, 8 in 2024-09-21T11:57:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:37 smithi175 ceph-mon[32613]: 3.1e scrub starts 2024-09-21T11:57:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:37 smithi175 ceph-mon[32613]: 3.1e scrub ok 2024-09-21T11:57:38.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:37 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3479746328' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:38.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:57:38.299 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:38.327 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:38.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:38.328 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:38.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:38.455 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:38.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:38.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:38.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:38.584 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:38.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:38.585 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:38.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:38.703 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 58 16 88532 88516 95.3486 72.75 0.0157071 0.0104828 2024-09-21T11:57:38.730 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:38.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:38.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:38.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:38.860 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:38.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:38.861 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:38.961 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:38.992 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:38.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:38.993 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:39.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:39.119 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:38 smithi043 ceph-mon[36454]: 4.18 scrub starts 2024-09-21T11:57:39.119 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:38 smithi043 ceph-mon[36454]: 4.18 scrub ok 2024-09-21T11:57:39.119 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:38 smithi043 ceph-mon[36454]: pgmap v854: 148 pgs: 148 active+clean; 5.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 903 B/s rd, 62 MiB/s wr, 991 op/s 2024-09-21T11:57:39.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:38 smithi043 ceph-mon[29710]: 4.18 scrub starts 2024-09-21T11:57:39.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:38 smithi043 ceph-mon[29710]: 4.18 scrub ok 2024-09-21T11:57:39.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:38 smithi043 ceph-mon[29710]: pgmap v854: 148 pgs: 148 active+clean; 5.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 903 B/s rd, 62 MiB/s wr, 991 op/s 2024-09-21T11:57:39.123 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:39.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:39.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:39.225 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:39.257 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:39.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:39.258 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:39.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:38 smithi175 ceph-mon[32613]: 4.18 scrub starts 2024-09-21T11:57:39.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:38 smithi175 ceph-mon[32613]: 4.18 scrub ok 2024-09-21T11:57:39.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:38 smithi175 ceph-mon[32613]: pgmap v854: 148 pgs: 148 active+clean; 5.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 903 B/s rd, 62 MiB/s wr, 991 op/s 2024-09-21T11:57:39.359 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:39.396 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:39.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:39.397 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:39.498 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:39.530 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:39.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:39.531 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:39.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:57:39.629 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:39.631 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:39.657 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:39.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:39.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:39.710 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 59 16 89727 89711 94.9853 74.6875 0.0215488 0.010522 2024-09-21T11:57:39.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:39.814 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:39.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:39.815 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:39.916 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:39.945 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:39.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:39.946 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:39.970 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:40.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:40.070 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:40.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:40.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:40.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[29710]: 3.10 scrub starts 2024-09-21T11:57:40.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[29710]: 3.10 scrub ok 2024-09-21T11:57:40.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[29710]: 6.12 scrub starts 2024-09-21T11:57:40.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[29710]: 6.12 scrub ok 2024-09-21T11:57:40.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[29710]: 3.18 scrub starts 2024-09-21T11:57:40.114 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[29710]: 3.18 scrub ok 2024-09-21T11:57:40.115 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[36454]: 3.10 scrub starts 2024-09-21T11:57:40.115 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[36454]: 3.10 scrub ok 2024-09-21T11:57:40.115 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[36454]: 6.12 scrub starts 2024-09-21T11:57:40.115 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[36454]: 6.12 scrub ok 2024-09-21T11:57:40.115 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[36454]: 3.18 scrub starts 2024-09-21T11:57:40.115 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:40 smithi043 ceph-mon[36454]: 3.18 scrub ok 2024-09-21T11:57:40.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:40.201 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:40.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:40.202 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:40.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:40 smithi175 ceph-mon[32613]: 3.10 scrub starts 2024-09-21T11:57:40.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:40 smithi175 ceph-mon[32613]: 3.10 scrub ok 2024-09-21T11:57:40.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:40 smithi175 ceph-mon[32613]: 6.12 scrub starts 2024-09-21T11:57:40.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:40 smithi175 ceph-mon[32613]: 6.12 scrub ok 2024-09-21T11:57:40.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:40 smithi175 ceph-mon[32613]: 3.18 scrub starts 2024-09-21T11:57:40.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:40 smithi175 ceph-mon[32613]: 3.18 scrub ok 2024-09-21T11:57:40.302 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:40.329 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:40.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:40.330 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:40.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:40.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:40.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:40.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:40.612 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:40.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:40.613 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:40.710 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:57:40.709684+0000 min lat: 0.00120284 max lat: 0.292846 avg lat: 0.0105734 2024-09-21T11:57:40.710 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:57:40.710 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 60 16 90809 90793 94.5294 67.625 0.0154044 0.0105734 2024-09-21T11:57:40.715 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:40.745 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:40.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:40.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:40.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:40.882 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:40.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:40.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:40.885 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:57:40.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:41.035 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 1 pg_num to 11 2024-09-21T11:57:41.039 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:41.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:41.040 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:41.141 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:41.168 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:41.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:41.169 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:57:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:57:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[29710]: osdmap e411: 8 total, 8 up, 8 in 2024-09-21T11:57:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[29710]: 5.3 scrub starts 2024-09-21T11:57:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[29710]: 5.3 scrub ok 2024-09-21T11:57:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[29710]: pgmap v856: 148 pgs: 148 active+clean; 5.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 936 op/s 2024-09-21T11:57:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[29710]: 3.13 scrub starts 2024-09-21T11:57:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[29710]: 3.13 scrub ok 2024-09-21T11:57:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2464863487' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-21T11:57:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-21T11:57:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:57:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:57:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[36454]: osdmap e411: 8 total, 8 up, 8 in 2024-09-21T11:57:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[36454]: 5.3 scrub starts 2024-09-21T11:57:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[36454]: 5.3 scrub ok 2024-09-21T11:57:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[36454]: pgmap v856: 148 pgs: 148 active+clean; 5.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 936 op/s 2024-09-21T11:57:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[36454]: 3.13 scrub starts 2024-09-21T11:57:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[36454]: 3.13 scrub ok 2024-09-21T11:57:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2464863487' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-21T11:57:41.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:41 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-21T11:57:41.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:57:41.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:57:41.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-mon[32613]: osdmap e411: 8 total, 8 up, 8 in 2024-09-21T11:57:41.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-mon[32613]: 5.3 scrub starts 2024-09-21T11:57:41.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-mon[32613]: 5.3 scrub ok 2024-09-21T11:57:41.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-mon[32613]: pgmap v856: 148 pgs: 148 active+clean; 5.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 936 op/s 2024-09-21T11:57:41.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-mon[32613]: 3.13 scrub starts 2024-09-21T11:57:41.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-mon[32613]: 3.13 scrub ok 2024-09-21T11:57:41.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2464863487' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-21T11:57:41.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-21T11:57:41.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:41.321 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:41.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:41.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:41.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:41.476 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:41.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:41.477 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:41.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:41.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:41.624 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:41.625 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:41.710 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 61 16 91825 91809 94.0208 63.5 0.0238949 0.0106311 2024-09-21T11:57:41.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:41.753 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:41.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:41.755 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:41.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:41.908 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:41.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:41.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:42.009 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:42.043 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:42.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:42.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:57:42.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:42.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:42.173 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:42.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:42 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-21T11:57:42.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:42 smithi175 ceph-mon[32613]: osdmap e412: 8 total, 8 up, 8 in 2024-09-21T11:57:42.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/724080113' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:42.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:42 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:42.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:57:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:57:42.273 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:42.306 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:42.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:42.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:42.327 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:42.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:42.453 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:42.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:42.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:42.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:42 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-21T11:57:42.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:42 smithi043 ceph-mon[29710]: osdmap e412: 8 total, 8 up, 8 in 2024-09-21T11:57:42.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/724080113' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:42.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:42 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:42.481 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:42 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-21T11:57:42.481 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:42 smithi043 ceph-mon[36454]: osdmap e412: 8 total, 8 up, 8 in 2024-09-21T11:57:42.482 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/724080113' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:42.482 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:42 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:57:42.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:42.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:42.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:42.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:42.697 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:42.710 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 62 16 92948 92932 93.6364 70.1875 0.00964875 0.0106747 2024-09-21T11:57:42.728 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:42.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:42.729 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:42.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:42.870 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:42.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:42.871 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:42.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:43.000 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:43.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:43.001 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:43.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:43.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:43.152 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:43.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: 6.a scrub starts 2024-09-21T11:57:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: 6.a scrub ok 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: 5.11 scrub starts 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: 5.11 scrub ok 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: osdmap e413: 8 total, 8 up, 8 in 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: pgmap v859: 148 pgs: 148 active+clean; 5.6 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: osdmap e414: 8 total, 8 up, 8 in 2024-09-21T11:57:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: 5.c scrub starts 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: 5.c scrub ok 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: 6.2 scrub starts 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[29710]: 6.2 scrub ok 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: 6.a scrub starts 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: 6.a scrub ok 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: 5.11 scrub starts 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: 5.11 scrub ok 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:57:43.243 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: osdmap e413: 8 total, 8 up, 8 in 2024-09-21T11:57:43.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: pgmap v859: 148 pgs: 148 active+clean; 5.6 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:57:43.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:43.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:43.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-21T11:57:43.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: osdmap e414: 8 total, 8 up, 8 in 2024-09-21T11:57:43.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: 5.c scrub starts 2024-09-21T11:57:43.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: 5.c scrub ok 2024-09-21T11:57:43.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: 6.2 scrub starts 2024-09-21T11:57:43.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:43 smithi043 ceph-mon[36454]: 6.2 scrub ok 2024-09-21T11:57:43.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:43.303 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:43.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:43.304 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:43.405 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:43.435 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:43.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:43.436 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: 6.a scrub starts 2024-09-21T11:57:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: 6.a scrub ok 2024-09-21T11:57:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: 5.11 scrub starts 2024-09-21T11:57:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: 5.11 scrub ok 2024-09-21T11:57:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:57:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T11:57:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:57:43.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: osdmap e413: 8 total, 8 up, 8 in 2024-09-21T11:57:43.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: pgmap v859: 148 pgs: 148 active+clean; 5.6 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-21T11:57:43.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:43.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:43.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-21T11:57:43.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: osdmap e414: 8 total, 8 up, 8 in 2024-09-21T11:57:43.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: 5.c scrub starts 2024-09-21T11:57:43.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: 5.c scrub ok 2024-09-21T11:57:43.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: 6.2 scrub starts 2024-09-21T11:57:43.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:43 smithi175 ceph-mon[32613]: 6.2 scrub ok 2024-09-21T11:57:43.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:43.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:43.584 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:43.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:43.712 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 63 15 94124 94109 93.3153 73.5625 0.0307275 0.0107111 2024-09-21T11:57:43.731 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:43.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:43.732 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:43.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:43.881 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:43.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:43.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:43.983 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:43.989 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:57:43.989 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:57:44.009 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:44.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:44.010 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:44.026 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:57:44.026 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:57:44.066 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:57:44.066 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:57:44.103 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:57:44.104 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:57:44.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:44.149 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:57:44.149 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:57:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:44.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:44.153 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:44.181 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:57:44.182 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:57:44.218 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:57:44.219 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:57:44.253 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:44.255 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:57:44.255 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:57:44.282 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:44.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:44.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:44.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:44.424 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:44.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:44.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47529 10.3 10:ce2591d9:::benchmark_data_smithi043_150678_object47528:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.839060+0000", 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 270.046185632, 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29800015000000002, 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.440 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47529 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.839060+0000", 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.839060+0000", 2024-09-21T11:57:44.441 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.839061+0000", 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.048e-06 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.839082+0000", 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1562000000000001e-05 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.442 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.839093+0000", 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0761000000000001e-05 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.839100+0000", 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0790000000000003e-06 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.839990+0000", 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00088966399999999997 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.840104+0000", 2024-09-21T11:57:44.443 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113977 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.840226+0000", 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012253500000000001 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842492+0000", 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002265329 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.844911+0000", 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0024194289999999999 2024-09-21T11:57:44.444 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136953+0000", 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29204144500000001 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137014+0000", 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1759000000000003e-05 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137060+0000", 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5562000000000002e-05 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.445 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47537 10.5 10:a3aa179a:::benchmark_data_smithi043_150678_object47536:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.843647+0000", 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 270.04159819199998, 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29115557600000003, 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.446 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47537 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843647+0000", 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843647+0000", 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.447 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843649+0000", 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6139999999999999e-06 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843673+0000", 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4348000000000001e-05 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843688+0000", 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5173e-05 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.448 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843697+0000", 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4810000000000006e-06 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843749+0000", 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2049999999999998e-05 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843888+0000", 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013882200000000001 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.449 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.844070+0000", 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018208499999999999 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.847691+0000", 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0036210249999999999 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.848719+0000", 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001027732 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.134666+0000", 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28594777799999999 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.134741+0000", 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4778999999999997e-05 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.134803+0000", 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1688999999999998e-05 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47546 10.3 10:c26054b3:::benchmark_data_smithi043_150678_object47545:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.848778+0000", 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 270.03646766399999, 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28854101599999998, 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47546 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.848778+0000", 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.848778+0000", 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.848779+0000", 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.31e-06 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.453 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.848801+0000", 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2376000000000001e-05 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.848812+0000", 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0225e-05 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.848818+0000", 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7279999999999998e-06 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.454 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.848879+0000", 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1041000000000001e-05 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.849023+0000", 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014393300000000001 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.849175+0000", 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000151853 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.455 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.851097+0000", 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0019219389999999999 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.853127+0000", 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020296889999999999 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137176+0000", 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28404868599999999 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137230+0000", 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4651999999999998e-05 2024-09-21T11:57:44.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137319+0000", 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8584000000000004e-05 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47556 10.3 10:ca52aeba:::benchmark_data_smithi043_150678_object47555:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.855428+0000", 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 270.02981777899998, 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28449940400000001, 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.457 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47556 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855428+0000", 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855428+0000", 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.458 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855429+0000", 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5379999999999999e-06 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855453+0000", 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3816999999999999e-05 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855467+0000", 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3523000000000001e-05 2024-09-21T11:57:44.459 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855474+0000", 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7800000000000001e-06 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855543+0000", 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9111999999999998e-05 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855701+0000", 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000157987 2024-09-21T11:57:44.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855868+0000", 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016678400000000001 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.857462+0000", 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001593765 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.858145+0000", 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00068322299999999995 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.139821+0000", 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28167551699999999 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.139879+0000", 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8224999999999999e-05 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.462 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.139927+0000", 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8133000000000001e-05 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47563 10.3 10:cccd708c:::benchmark_data_smithi043_150678_object47562:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.859990+0000", 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 270.025255937, 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28011235099999998, 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.463 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47563 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.859990+0000", 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.464 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.859990+0000", 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.859991+0000", 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3e-06 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860008+0000", 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6911000000000001e-05 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860019+0000", 2024-09-21T11:57:44.465 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1484999999999999e-05 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860027+0000", 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9810000000000003e-06 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860076+0000", 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9215000000000002e-05 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.466 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860173+0000", 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6428999999999994e-05 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860304+0000", 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000131385 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.861693+0000", 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001388448 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.467 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.862187+0000", 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00049417899999999997 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.140018+0000", 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27783157200000003 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.140067+0000", 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8505999999999999e-05 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.140102+0000", 2024-09-21T11:57:44.468 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4940000000000001e-05 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:67844 10.3 10:c750a974:::benchmark_data_smithi043_150678_object67843:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:29.047276+0000", 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 254.83796943499999, 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29365253200000002, 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.469 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 67844 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.047276+0000", 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.047276+0000", 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.047278+0000", 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.638e-06 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.047303+0000", 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5401999999999998e-05 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.047315+0000", 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2187e-05 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.471 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.047321+0000", 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2669999999999997e-06 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.047391+0000", 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0494999999999995e-05 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.047533+0000", 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014215899999999999 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.472 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.047668+0000", 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013486300000000001 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.050440+0000", 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027715349999999999 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.052023+0000", 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015839339999999999 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.473 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.340832+0000", 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28880818200000002 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.340883+0000", 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1811000000000003e-05 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.340929+0000", 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5058999999999997e-05 2024-09-21T11:57:44.474 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:67850 10.3 10:c78a6c91:::benchmark_data_smithi043_150678_object67849:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:29.052423+0000", 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 254.832822601, 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28863115299999997, 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 67850 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.475 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.052423+0000", 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.052423+0000", 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.052425+0000", 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.813e-06 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.476 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.052445+0000", 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0418e-05 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.052462+0000", 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6946999999999999e-05 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.052470+0000", 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8080000000000005e-06 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.477 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.052520+0000", 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0274e-05 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.052656+0000", 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013613699999999999 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.052796+0000", 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013962800000000001 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.478 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.057564+0000", 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047683279999999996 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.058553+0000", 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098854299999999993 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.340971+0000", 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.282418157 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.341018+0000", 2024-09-21T11:57:44.479 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7352999999999999e-05 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.341054+0000", 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5747000000000002e-05 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:67851 10.3 10:cfcee09a:::benchmark_data_smithi043_150678_object67850:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:29.053449+0000", 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 254.83179691800001, 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28772920899999999, 2024-09-21T11:57:44.480 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 67851 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.053449+0000", 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.481 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.053449+0000", 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.053450+0000", 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0699999999999999e-06 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.053471+0000", 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0996999999999999e-05 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.482 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.053484+0000", 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3550000000000001e-05 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.053491+0000", 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.055113+0000", 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0016214549999999999 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.483 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.055277+0000", 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016370800000000001 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.055429+0000", 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015232399999999999 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.058794+0000", 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033649790000000001 2024-09-21T11:57:44.484 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.059161+0000", 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000367408 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.341097+0000", 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28193584799999999 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.341143+0000", 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5732000000000002e-05 2024-09-21T11:57:44.485 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.341178+0000", 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4919000000000002e-05 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:67889 10.3 10:ceca0a77:::benchmark_data_smithi043_150678_object67888:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:29.095238+0000", 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 254.790007092, 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.24922365699999999, 2024-09-21T11:57:44.486 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 67889 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.095238+0000", 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.487 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.095238+0000", 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.095240+0000", 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.398e-06 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.095275+0000", 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5571999999999998e-05 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.488 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.095290+0000", 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4946000000000001e-05 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.095297+0000", 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.81e-06 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.095353+0000", 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5852e-05 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.095465+0000", 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011251099999999999 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.095604+0000", 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000138493 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.097750+0000", 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0021460070000000001 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.490 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.098022+0000", 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00027223299999999998 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.344364+0000", 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.24634225700000001 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.344414+0000", 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0049999999999997e-05 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.491 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:29.344462+0000", 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7528000000000003e-05 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126839 10.5 10:abc75aad:::benchmark_data_smithi043_150678_object4088:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.883659+0000", 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 198.00158641499999, 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.23927644200000001, 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.492 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126839 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.883659+0000", 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.883659+0000", 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.883660+0000", 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4410000000000001e-06 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.883665+0000", 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6360000000000002e-06 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.883679+0000", 2024-09-21T11:57:44.494 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3932e-05 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.883687+0000", 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.807e-06 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.883743+0000", 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6412999999999997e-05 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.495 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.883863+0000", 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119658 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.883984+0000", 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121147 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.887763+0000", 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00377918 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.496 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.891891+0000", 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041279309999999996 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.122837+0000", 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.23094589200000001 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.122893+0000", 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6236999999999997e-05 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.497 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.122935+0000", 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2168e-05 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126855 10.3 10:ced7525c:::benchmark_data_smithi043_150678_object4104:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.895241+0000", 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 197.99000482599999, 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.225180402, 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.498 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126855 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895241+0000", 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.499 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895241+0000", 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895242+0000", 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8029999999999999e-06 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895248+0000", 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3009999999999997e-06 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.500 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895269+0000", 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1078e-05 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895277+0000", 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3909999999999998e-06 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895345+0000", 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7960000000000007e-05 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.501 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895561+0000", 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000215885 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.895743+0000", 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018232099999999999 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.899337+0000", 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035934199999999999 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.502 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.901722+0000", 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0023852489999999999 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.120263+0000", 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.218541228 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.120367+0000", 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103337 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.503 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.120421+0000", 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4428999999999997e-05 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126871 10.5 10:a1ae4880:::benchmark_data_smithi043_150678_object4120:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.906687+0000", 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 197.97855861100001, 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.21657839700000001, 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.504 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126871 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.906687+0000", 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.906687+0000", 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.505 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.906688+0000", 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6589999999999999e-06 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.906694+0000", 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3589999999999999e-06 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.906707+0000", 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2944e-05 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.906714+0000", 2024-09-21T11:57:44.506 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4250000000000001e-06 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.906785+0000", 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1179000000000004e-05 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.906911+0000", 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012546000000000001 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.907026+0000", 2024-09-21T11:57:44.507 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114641 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.910138+0000", 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031123779999999998 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.910620+0000", 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00048239799999999998 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.508 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.122998+0000", 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.21237774600000001 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123222+0000", 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022395100000000001 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123265+0000", 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3257000000000003e-05 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.509 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126893 10.5 10:af251bb8:::benchmark_data_smithi043_150678_object4142:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.926835+0000", 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 197.95841083100001, 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19664079900000001, 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126893 2024-09-21T11:57:44.510 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.926835+0000", 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.926835+0000", 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.926836+0000", 2024-09-21T11:57:44.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5400000000000001e-06 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.926843+0000", 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.533e-06 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.926854+0000", 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1032e-05 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.926860+0000", 2024-09-21T11:57:44.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9839999999999998e-06 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.926921+0000", 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0865000000000001e-05 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.927055+0000", 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000134814 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.927194+0000", 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000138651 2024-09-21T11:57:44.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930396+0000", 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032018039999999999 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.930453+0000", 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6725999999999999e-05 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123304+0000", 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19285105499999999 2024-09-21T11:57:44.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123439+0000", 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013523300000000001 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.123475+0000", 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6562e-05 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126896 10.3 10:cd3750cf:::benchmark_data_smithi043_150678_object4145:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.929717+0000", 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 197.955528117, 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.19104075200000001, 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126896 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.516 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.929717+0000", 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.929717+0000", 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.929719+0000", 2024-09-21T11:57:44.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.376e-06 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.929724+0000", 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4060000000000004e-06 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.929738+0000", 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3393999999999999e-05 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.518 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.929745+0000", 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4800000000000004e-06 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.929794+0000", 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9484999999999998e-05 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.929874+0000", 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9772999999999997e-05 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,5", 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.929942+0000", 2024-09-21T11:57:44.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7323999999999994e-05 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.933311+0000", 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033696529999999998 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.933854+0000", 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000542637 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.120468+0000", 2024-09-21T11:57:44.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.186613905 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.120717+0000", 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000249628 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:26.120758+0000", 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0691000000000001e-05 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:168887 10.5 10:ae684a96:::benchmark_data_smithi043_150678_object46136:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e358)", 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:04.060938+0000", 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 159.82430726800001, 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.188797464, 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 168887 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.522 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.060938+0000", 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.060938+0000", 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.060939+0000", 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.186e-06 2024-09-21T11:57:44.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.060943+0000", 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0289999999999999e-06 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.060952+0000", 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0059999999999998e-06 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.060959+0000", 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6479999999999997e-06 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.067154+0000", 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061951080000000004 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.067275+0000", 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120497 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.067398+0000", 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123533 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.234931+0000", 2024-09-21T11:57:44.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.167532495 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.234996+0000", 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5764000000000003e-05 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.249604+0000", 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014607439 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.249687+0000", 2024-09-21T11:57:44.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3065000000000005e-05 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:04.249736+0000", 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8693999999999998e-05 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:60070 11.2 11:4bb55b2a:::benchmark_data_smithi043_178089_object60069:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:57:13.459078+0000", 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.426167152000001, 2024-09-21T11:57:44.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.292482884, 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 60070 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.459078+0000", 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.459078+0000", 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.459079+0000", 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4300000000000001e-07 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.459132+0000", 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2707000000000002e-05 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.459141+0000", 2024-09-21T11:57:44.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9919999999999996e-06 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.459148+0000", 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8059999999999997e-06 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.459204+0000", 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6162000000000002e-05 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.459340+0000", 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013651200000000001 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.459468+0000", 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012766700000000001 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.463523+0000", 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0040552699999999997 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.464781+0000", 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001257656 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.751464+0000", 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28668335700000003 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.751517+0000", 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2846000000000002e-05 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.751561+0000", 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3965999999999999e-05 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:60078 11.2 11:48ee6289:::benchmark_data_smithi043_178089_object60077:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:57:13.465350+0000", 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.419895316000002, 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28642408499999999, 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.533 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 60078 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.465350+0000", 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.465350+0000", 2024-09-21T11:57:44.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.465351+0000", 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.294e-06 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.465398+0000", 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7015000000000003e-05 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.465409+0000", 2024-09-21T11:57:44.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0738e-05 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.465416+0000", 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8249999999999999e-06 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.465469+0000", 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3384999999999998e-05 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.465583+0000", 2024-09-21T11:57:44.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113268 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.465700+0000", 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117619 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.469155+0000", 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034549870000000001 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.469224+0000", 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8528999999999994e-05 2024-09-21T11:57:44.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.751622+0000", 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28239825699999999 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.751706+0000", 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4288000000000003e-05 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.751774+0000", 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7879999999999994e-05 2024-09-21T11:57:44.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.26339.0:60092 11.c e402/392 11:39d222d2:::benchmark_data_smithi043_178089_object60091:head v 402'3791, mlcod=402'3789)", 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:57:13.475192+0000", 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.410053333, 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27824842399999999, 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.475192+0000", 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.475192+0000", 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.475193+0000", 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6799999999999999e-07 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.475238+0000", 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5117999999999997e-05 2024-09-21T11:57:44.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.475245+0000", 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9129999999999997e-06 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.475251+0000", 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6339999999999998e-06 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.748897+0000", 2024-09-21T11:57:44.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27364680200000002 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.748921+0000", 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3575e-05 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.753396+0000", 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044749539999999997 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.753441+0000", 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4660000000000003e-05 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:60096 11.2 11:40f6c44e:::benchmark_data_smithi043_178089_object60095:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:57:13.477984+0000", 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.407261394999999, 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27751082399999999, 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:57:44.543 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 60096 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.477984+0000", 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.477984+0000", 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.477985+0000", 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.204e-06 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.478031+0000", 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5791999999999998e-05 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.748873+0000", 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27084175900000002 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.748886+0000", 2024-09-21T11:57:44.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2772000000000001e-05 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.748947+0000", 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1606000000000006e-05 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.749343+0000", 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00039617500000000002 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,3", 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.749484+0000", 2024-09-21T11:57:44.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014102200000000001 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.751577+0000", 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0020927150000000002 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.752441+0000", 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000863596 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.755332+0000", 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028916409999999999 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.755421+0000", 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8609000000000005e-05 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.755495+0000", 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3932999999999998e-05 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.26339.0:60097 11.4 e402/392 11:2b28348e:::benchmark_data_smithi043_178089_object60096:head v 402'3722, mlcod=402'3720)", 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:57:13.478464+0000", 2024-09-21T11:57:44.548 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 30.406781282000001, 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27492829499999999, 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.478464+0000", 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.478464+0000", 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:57:44.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.478465+0000", 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7899999999999997e-07 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.478532+0000", 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6965999999999995e-05 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.748853+0000", 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27032119900000001 2024-09-21T11:57:44.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.748870+0000", 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6296000000000001e-05 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.748926+0000", 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6082000000000003e-05 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.748945+0000", 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9531e-05 2024-09-21T11:57:44.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.753311+0000", 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043655660000000004 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.753392+0000", 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1775999999999998e-05 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:57:44.552 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:57:44.553 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:44.580 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:44.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:44.581 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:44.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:44.712 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 64 16 95245 95229 92.9511 70 0.00727078 0.010754 2024-09-21T11:57:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:44.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:44.722 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:44.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: osdmap e415: 8 total, 8 up, 8 in 2024-09-21T11:57:44.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: 3.0 scrub starts 2024-09-21T11:57:44.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: 3.0 scrub ok 2024-09-21T11:57:44.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: 3.1f scrub starts 2024-09-21T11:57:44.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: 3.1f scrub ok 2024-09-21T11:57:44.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: 6.13 scrub starts 2024-09-21T11:57:44.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: 6.13 scrub ok 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: 4.e scrub starts 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: 4.e scrub ok 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2183690779' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: osdmap e415: 8 total, 8 up, 8 in 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: 3.0 scrub starts 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: 3.0 scrub ok 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: 3.1f scrub starts 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: 3.1f scrub ok 2024-09-21T11:57:44.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: 6.13 scrub starts 2024-09-21T11:57:44.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: 6.13 scrub ok 2024-09-21T11:57:44.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: 4.e scrub starts 2024-09-21T11:57:44.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: 4.e scrub ok 2024-09-21T11:57:44.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2183690779' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:44.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:44.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:44 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:44.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: osdmap e415: 8 total, 8 up, 8 in 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: 3.0 scrub starts 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: 3.0 scrub ok 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: 3.1f scrub starts 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: 3.1f scrub ok 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: 6.13 scrub starts 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: 6.13 scrub ok 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: 4.e scrub starts 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: 4.e scrub ok 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2183690779' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:44 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-21T11:57:44.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:44.856 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:44.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:44.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:44.958 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:44.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:57:44.988 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:44.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:44.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:45.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:45.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:45.132 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:45.132 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:45.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:45.259 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:45.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:45.260 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:45.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:45.403 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:45.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:45.404 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:45.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:57:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:57:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:45.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:45.555 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:45.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:45.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:45.657 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:45.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:45.700 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:45.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:45.712 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 65 16 96431 96415 92.6615 74.125 0.0142659 0.0107871 2024-09-21T11:57:45.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:45 smithi175 ceph-mon[32613]: pgmap v862: 158 pgs: 10 unknown, 148 active+clean; 5.6 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-21T11:57:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:45 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-21T11:57:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:45 smithi175 ceph-mon[32613]: osdmap e416: 8 total, 8 up, 8 in 2024-09-21T11:57:45.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:45 smithi175 ceph-mon[32613]: osdmap e417: 8 total, 8 up, 8 in 2024-09-21T11:57:45.801 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:45.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:45.837 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:45.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:45.938 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:45.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:45.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:45.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:45.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:45 smithi043 ceph-mon[29710]: pgmap v862: 158 pgs: 10 unknown, 148 active+clean; 5.6 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-21T11:57:45.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:45 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-21T11:57:45.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:45 smithi043 ceph-mon[29710]: osdmap e416: 8 total, 8 up, 8 in 2024-09-21T11:57:45.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:45 smithi043 ceph-mon[29710]: osdmap e417: 8 total, 8 up, 8 in 2024-09-21T11:57:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:45 smithi043 ceph-mon[36454]: pgmap v862: 158 pgs: 10 unknown, 148 active+clean; 5.6 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-21T11:57:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:45 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-21T11:57:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:45 smithi043 ceph-mon[36454]: osdmap e416: 8 total, 8 up, 8 in 2024-09-21T11:57:45.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:45 smithi043 ceph-mon[36454]: osdmap e417: 8 total, 8 up, 8 in 2024-09-21T11:57:46.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:46.111 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:46.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:46.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:46.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:46.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:46.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:46.348 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:46.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:46.385 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:46.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:46.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:46.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:46.523 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:46.624 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:46.654 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:46.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:46.655 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:46.675 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:46 smithi175 ceph-mon[32613]: 5.6 scrub starts 2024-09-21T11:57:46.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:46 smithi175 ceph-mon[32613]: 5.6 scrub ok 2024-09-21T11:57:46.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:46 smithi175 ceph-mon[32613]: 1.7 scrub starts 2024-09-21T11:57:46.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:46 smithi175 ceph-mon[32613]: 1.7 scrub ok 2024-09-21T11:57:46.712 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 66 16 97633 97617 92.3957 75.125 0.012058 0.0108182 2024-09-21T11:57:46.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:46.793 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:46.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:46.794 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:46.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:46.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:46 smithi043 ceph-mon[29710]: 5.6 scrub starts 2024-09-21T11:57:46.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:46 smithi043 ceph-mon[29710]: 5.6 scrub ok 2024-09-21T11:57:46.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:46 smithi043 ceph-mon[29710]: 1.7 scrub starts 2024-09-21T11:57:46.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:46 smithi043 ceph-mon[29710]: 1.7 scrub ok 2024-09-21T11:57:46.924 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:46 smithi043 ceph-mon[36454]: 5.6 scrub starts 2024-09-21T11:57:46.924 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:46 smithi043 ceph-mon[36454]: 5.6 scrub ok 2024-09-21T11:57:46.924 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:46 smithi043 ceph-mon[36454]: 1.7 scrub starts 2024-09-21T11:57:46.924 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:46 smithi043 ceph-mon[36454]: 1.7 scrub ok 2024-09-21T11:57:46.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:46.935 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:46.935 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:46.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:57:47.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:47.067 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:47.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:47.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:47.159 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:57:47.160 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:57:47.160 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-21T11:57:47.160 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-21T11:57:47.160 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-21T11:57:47.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-09-21T11:57:47.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:47.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:47.242 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:47.243 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:47.343 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:47.391 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:47.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:47.392 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:47.493 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:47.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:47.534 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:47.534 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:47.561 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:57:47.635 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:47.664 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:47.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:47.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:47.712 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 67 16 98799 98783 92.1044 72.875 0.0065392 0.0108522 2024-09-21T11:57:47.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:47.798 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:47.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:47.799 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:47.902 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:47.934 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:47.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:47.935 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:47.968 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:47 smithi175 ceph-mon[32613]: pgmap v865: 158 pgs: 7 unknown, 151 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 15 KiB/s rd, 33 MiB/s wr, 538 op/s; 119 KiB/s, 0 objects/s recovering 2024-09-21T11:57:47.968 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:47 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:47.968 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:47 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:47.969 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:47 smithi175 ceph-mon[32613]: osdmap e418: 8 total, 8 up, 8 in 2024-09-21T11:57:47.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:47 smithi043 ceph-mon[36454]: pgmap v865: 158 pgs: 7 unknown, 151 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 15 KiB/s rd, 33 MiB/s wr, 538 op/s; 119 KiB/s, 0 objects/s recovering 2024-09-21T11:57:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:47 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:47 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:47.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:47 smithi043 ceph-mon[36454]: osdmap e418: 8 total, 8 up, 8 in 2024-09-21T11:57:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:47 smithi043 ceph-mon[29710]: pgmap v865: 158 pgs: 7 unknown, 151 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 15 KiB/s rd, 33 MiB/s wr, 538 op/s; 119 KiB/s, 0 objects/s recovering 2024-09-21T11:57:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:47 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:57:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:47 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:57:47.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:47 smithi043 ceph-mon[29710]: osdmap e418: 8 total, 8 up, 8 in 2024-09-21T11:57:48.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:48.066 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:48.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:48.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:48.168 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:48.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:48.210 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:48.211 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:48.287 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:57:48.287 INFO:teuthology.orchestra.run.smithi175.stdout: "success": "" 2024-09-21T11:57:48.287 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:57:48.312 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:48.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:48.335 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:48.335 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:48.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:48.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:48.479 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:48.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:48.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:48 smithi043 ceph-mon[36454]: 1.4 scrub starts 2024-09-21T11:57:48.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:48 smithi043 ceph-mon[36454]: 1.4 scrub ok 2024-09-21T11:57:48.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:48 smithi043 ceph-mon[36454]: 5.1e scrub starts 2024-09-21T11:57:48.537 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:48 smithi043 ceph-mon[36454]: 5.1e scrub ok 2024-09-21T11:57:48.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:48.604 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:48.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:48.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:48.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:48.713 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 68 16 99943 99927 91.8014 71.5 0.0230178 0.0108874 2024-09-21T11:57:48.745 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:48.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:48.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:48.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:48.875 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:48 smithi175 ceph-mon[32613]: 1.4 scrub starts 2024-09-21T11:57:48.875 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:48 smithi175 ceph-mon[32613]: 1.4 scrub ok 2024-09-21T11:57:48.875 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:48 smithi175 ceph-mon[32613]: 5.1e scrub starts 2024-09-21T11:57:48.875 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:48 smithi175 ceph-mon[32613]: 5.1e scrub ok 2024-09-21T11:57:48.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:48.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:48.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:48 smithi043 ceph-mon[29710]: 1.4 scrub starts 2024-09-21T11:57:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:48 smithi043 ceph-mon[29710]: 1.4 scrub ok 2024-09-21T11:57:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:48 smithi043 ceph-mon[29710]: 5.1e scrub starts 2024-09-21T11:57:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:48 smithi043 ceph-mon[29710]: 5.1e scrub ok 2024-09-21T11:57:49.025 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:49.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:49.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:49.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:49.171 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:49.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:49.172 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:49.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:49.310 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:49.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:49.312 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:49.413 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:49.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:49.472 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:49.473 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:49.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:49.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:49.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:49.595 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:49.696 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:49.713 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 69 15 101063 101048 91.4864 70.0625 0.00543169 0.0109244 2024-09-21T11:57:49.726 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:49.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:49.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:49.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:49 smithi175 ceph-mon[32613]: 1.5 scrub starts 2024-09-21T11:57:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:49 smithi175 ceph-mon[32613]: 1.5 scrub ok 2024-09-21T11:57:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:49 smithi175 ceph-mon[32613]: pgmap v867: 158 pgs: 158 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 659 B/s rd, 77 MiB/s wr, 1.22k op/s; 34 KiB/s, 0 objects/s recovering 2024-09-21T11:57:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:49 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:57:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:49 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:57:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:49 smithi175 ceph-mon[32613]: 1.3 scrub starts 2024-09-21T11:57:49.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:49 smithi175 ceph-mon[32613]: 1.3 scrub ok 2024-09-21T11:57:49.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:49.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:49.865 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:49.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[29710]: 1.5 scrub starts 2024-09-21T11:57:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[29710]: 1.5 scrub ok 2024-09-21T11:57:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[29710]: pgmap v867: 158 pgs: 158 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 659 B/s rd, 77 MiB/s wr, 1.22k op/s; 34 KiB/s, 0 objects/s recovering 2024-09-21T11:57:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:57:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:57:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[29710]: 1.3 scrub starts 2024-09-21T11:57:49.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[29710]: 1.3 scrub ok 2024-09-21T11:57:49.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[36454]: 1.5 scrub starts 2024-09-21T11:57:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[36454]: 1.5 scrub ok 2024-09-21T11:57:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[36454]: pgmap v867: 158 pgs: 158 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 659 B/s rd, 77 MiB/s wr, 1.22k op/s; 34 KiB/s, 0 objects/s recovering 2024-09-21T11:57:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:57:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:57:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[36454]: 1.3 scrub starts 2024-09-21T11:57:49.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:49 smithi043 ceph-mon[36454]: 1.3 scrub ok 2024-09-21T11:57:50.002 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:50.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:50.003 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:50.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:50.124 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:50.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:50.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:50.226 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:50.254 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:50.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:50.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:50.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:50.386 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:50.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:50.387 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:50.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:50.511 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:50.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:50.512 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:50.613 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:50.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:50.640 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:50.640 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:50.713 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 70 16 102180 102164 91.1759 69.75 0.0136968 0.0109623 2024-09-21T11:57:50.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:50.766 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:50.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:50.767 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:50.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:50.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:50.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:50.912 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[29710]: 6.f scrub starts 2024-09-21T11:57:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[29710]: 6.f scrub ok 2024-09-21T11:57:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[29710]: 1.a scrub starts 2024-09-21T11:57:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[29710]: 1.a scrub ok 2024-09-21T11:57:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[29710]: 3.12 scrub starts 2024-09-21T11:57:50.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[29710]: 3.12 scrub ok 2024-09-21T11:57:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[36454]: 6.f scrub starts 2024-09-21T11:57:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[36454]: 6.f scrub ok 2024-09-21T11:57:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[36454]: 1.a scrub starts 2024-09-21T11:57:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[36454]: 1.a scrub ok 2024-09-21T11:57:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[36454]: 3.12 scrub starts 2024-09-21T11:57:50.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:50 smithi043 ceph-mon[36454]: 3.12 scrub ok 2024-09-21T11:57:51.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:50 smithi175 ceph-mon[32613]: 6.f scrub starts 2024-09-21T11:57:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:50 smithi175 ceph-mon[32613]: 6.f scrub ok 2024-09-21T11:57:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:50 smithi175 ceph-mon[32613]: 1.a scrub starts 2024-09-21T11:57:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:50 smithi175 ceph-mon[32613]: 1.a scrub ok 2024-09-21T11:57:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:50 smithi175 ceph-mon[32613]: 3.12 scrub starts 2024-09-21T11:57:51.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:50 smithi175 ceph-mon[32613]: 3.12 scrub ok 2024-09-21T11:57:51.013 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:51.046 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:51.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:51.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:51.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:51.190 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:51.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:51.191 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:51.292 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:51.313 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:51.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:51.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:51.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:51.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:51.453 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:51.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:51.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:51.590 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:51.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:51.713 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 71 16 103234 103218 90.8196 65.875 0.0094265 0.0110046 2024-09-21T11:57:51.715 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:51.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:51.716 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:51.817 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:51.838 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:51 smithi175 ceph-mon[32613]: pgmap v868: 158 pgs: 158 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 511 B/s rd, 59 MiB/s wr, 950 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-21T11:57:51.838 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:51 smithi175 ceph-mon[32613]: 1.2 scrub starts 2024-09-21T11:57:51.838 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:51 smithi175 ceph-mon[32613]: 1.2 scrub ok 2024-09-21T11:57:51.838 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:51 smithi175 ceph-mon[32613]: 4.3 scrub starts 2024-09-21T11:57:51.838 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:51 smithi175 ceph-mon[32613]: 4.3 scrub ok 2024-09-21T11:57:51.838 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:51 smithi175 ceph-mon[32613]: 1.6 scrub starts 2024-09-21T11:57:51.838 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:51 smithi175 ceph-mon[32613]: 1.6 scrub ok 2024-09-21T11:57:51.857 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:51.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:51.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:51.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[29710]: pgmap v868: 158 pgs: 158 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 511 B/s rd, 59 MiB/s wr, 950 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-21T11:57:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[29710]: 1.2 scrub starts 2024-09-21T11:57:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[29710]: 1.2 scrub ok 2024-09-21T11:57:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[29710]: 4.3 scrub starts 2024-09-21T11:57:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[29710]: 4.3 scrub ok 2024-09-21T11:57:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[29710]: 1.6 scrub starts 2024-09-21T11:57:51.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[29710]: 1.6 scrub ok 2024-09-21T11:57:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[36454]: pgmap v868: 158 pgs: 158 active+clean; 5.9 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 511 B/s rd, 59 MiB/s wr, 950 op/s; 26 KiB/s, 0 objects/s recovering 2024-09-21T11:57:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[36454]: 1.2 scrub starts 2024-09-21T11:57:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[36454]: 1.2 scrub ok 2024-09-21T11:57:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[36454]: 4.3 scrub starts 2024-09-21T11:57:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[36454]: 4.3 scrub ok 2024-09-21T11:57:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[36454]: 1.6 scrub starts 2024-09-21T11:57:51.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:51 smithi043 ceph-mon[36454]: 1.6 scrub ok 2024-09-21T11:57:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:51.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:51.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:52.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:52.120 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:52.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:52.121 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:52.222 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:52.260 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:57:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:57:52.263 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:52.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:52.264 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:52.365 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:52.400 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:52.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:52.401 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:52.501 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:52.532 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:52.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:52.533 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:52.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:52.680 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:52.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:52.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:52.713 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 72 16 104359 104343 90.5348 70.3125 0.0249069 0.0110404 2024-09-21T11:57:52.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:57:52.810 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:52.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:52.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:57:52.880 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:52 smithi043 ceph-mon[29710]: 6.e scrub starts 2024-09-21T11:57:52.880 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:52 smithi043 ceph-mon[29710]: 6.e scrub ok 2024-09-21T11:57:52.881 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:52 smithi043 ceph-mon[29710]: 3.3 scrub starts 2024-09-21T11:57:52.881 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:52 smithi043 ceph-mon[29710]: 3.3 scrub ok 2024-09-21T11:57:52.881 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:52 smithi043 ceph-mon[36454]: 6.e scrub starts 2024-09-21T11:57:52.881 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:52 smithi043 ceph-mon[36454]: 6.e scrub ok 2024-09-21T11:57:52.881 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:52 smithi043 ceph-mon[36454]: 3.3 scrub starts 2024-09-21T11:57:52.881 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:52 smithi043 ceph-mon[36454]: 3.3 scrub ok 2024-09-21T11:57:52.913 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:52.964 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:52.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:52.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:53.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:52 smithi175 ceph-mon[32613]: 6.e scrub starts 2024-09-21T11:57:53.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:52 smithi175 ceph-mon[32613]: 6.e scrub ok 2024-09-21T11:57:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:52 smithi175 ceph-mon[32613]: 3.3 scrub starts 2024-09-21T11:57:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:52 smithi175 ceph-mon[32613]: 3.3 scrub ok 2024-09-21T11:57:53.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:53.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:53.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:53.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:53.272 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:53.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:53.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:53.374 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:53.407 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:53.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:53.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:53.452 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:53.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:53.552 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:53.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:53.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:53.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:53.691 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:53.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:53.692 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:53.713 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 73 16 105554 105538 90.3177 74.6875 0.00950932 0.0110671 2024-09-21T11:57:53.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:53.830 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:53.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:53.831 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:53.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:53.960 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:53.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:53.961 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[36454]: 4.1c scrub starts 2024-09-21T11:57:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[36454]: 4.1c scrub ok 2024-09-21T11:57:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[36454]: pgmap v869: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.3 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 93 MiB/s wr, 1.48k op/s 2024-09-21T11:57:53.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[36454]: 1.9 scrub starts 2024-09-21T11:57:53.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[36454]: 1.9 scrub ok 2024-09-21T11:57:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[29710]: 4.1c scrub starts 2024-09-21T11:57:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[29710]: 4.1c scrub ok 2024-09-21T11:57:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[29710]: pgmap v869: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.3 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 93 MiB/s wr, 1.48k op/s 2024-09-21T11:57:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[29710]: 1.9 scrub starts 2024-09-21T11:57:53.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:53 smithi043 ceph-mon[29710]: 1.9 scrub ok 2024-09-21T11:57:54.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:53 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:57:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:53 smithi175 ceph-mon[32613]: 4.1c scrub starts 2024-09-21T11:57:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:53 smithi175 ceph-mon[32613]: 4.1c scrub ok 2024-09-21T11:57:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:53 smithi175 ceph-mon[32613]: pgmap v869: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.3 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 93 MiB/s wr, 1.48k op/s 2024-09-21T11:57:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:53 smithi175 ceph-mon[32613]: 1.9 scrub starts 2024-09-21T11:57:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:53 smithi175 ceph-mon[32613]: 1.9 scrub ok 2024-09-21T11:57:54.063 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:54.088 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:54.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:54.089 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:54.190 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:54.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:54.225 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:54.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:54.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:54.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:54.365 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:54.366 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:57:54.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 0 1 2024-09-21T11:57:54.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:54.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:54.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:54.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:54.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:54.632 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:54.713 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 74 16 106747 106731 90.1044 74.5625 0.00650315 0.0110933 2024-09-21T11:57:54.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:54 smithi043 ceph-mon[29710]: 1.1 scrub starts 2024-09-21T11:57:54.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:54 smithi043 ceph-mon[29710]: 1.1 scrub ok 2024-09-21T11:57:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:54 smithi043 ceph-mon[36454]: 1.1 scrub starts 2024-09-21T11:57:54.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:54 smithi043 ceph-mon[36454]: 1.1 scrub ok 2024-09-21T11:57:54.787 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:54.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:54.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:54.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:54.938 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:54.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:54.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:54.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:57:55.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:54 smithi175 ceph-mon[32613]: 1.1 scrub starts 2024-09-21T11:57:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:54 smithi175 ceph-mon[32613]: 1.1 scrub ok 2024-09-21T11:57:55.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:55.143 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:55.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:55.144 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:55.153 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:55.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:55.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:55.295 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:55.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:55.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:55.412 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:57:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:57:55.422 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:55.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:55.423 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:55.523 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:55.552 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:55.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:55.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:55.590 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:57:55.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:55.668 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[36454]: pgmap v870: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 82 MiB/s wr, 1.30k op/s 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[36454]: 4.6 scrub starts 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[36454]: 4.6 scrub ok 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[36454]: 6.6 scrub starts 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[36454]: 6.6 scrub ok 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[36454]: 1.8 scrub starts 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[36454]: 1.8 scrub ok 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2206597840' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[29710]: pgmap v870: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 82 MiB/s wr, 1.30k op/s 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[29710]: 4.6 scrub starts 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[29710]: 4.6 scrub ok 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[29710]: 6.6 scrub starts 2024-09-21T11:57:55.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[29710]: 6.6 scrub ok 2024-09-21T11:57:55.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[29710]: 1.8 scrub starts 2024-09-21T11:57:55.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[29710]: 1.8 scrub ok 2024-09-21T11:57:55.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:55 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2206597840' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:57:55.682 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:55.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:55.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:55.713 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 75 16 107772 107756 89.7573 64.0625 0.0122645 0.0111363 2024-09-21T11:57:55.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:55.857 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:55.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:55.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:55.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:55.988 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:57:55.988 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:57:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:55.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:56.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:56.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:55 smithi175 ceph-mon[32613]: pgmap v870: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 82 MiB/s wr, 1.30k op/s 2024-09-21T11:57:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:55 smithi175 ceph-mon[32613]: 4.6 scrub starts 2024-09-21T11:57:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:55 smithi175 ceph-mon[32613]: 4.6 scrub ok 2024-09-21T11:57:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:55 smithi175 ceph-mon[32613]: 6.6 scrub starts 2024-09-21T11:57:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:55 smithi175 ceph-mon[32613]: 6.6 scrub ok 2024-09-21T11:57:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:55 smithi175 ceph-mon[32613]: 1.8 scrub starts 2024-09-21T11:57:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:55 smithi175 ceph-mon[32613]: 1.8 scrub ok 2024-09-21T11:57:56.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:55 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2206597840' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:57:56.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:56.137 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:56.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:56.138 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:56.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:56.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:56.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:56.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:56.419 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:56.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:56.421 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:56.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:56.560 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:56.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:56.561 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:56.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:56.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:56.710 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:56.710 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:56.713 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 76 16 108982 108966 89.5713 75.625 0.00989677 0.0111593 2024-09-21T11:57:56.811 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:56.836 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:56.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:56.837 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:56.938 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:56.969 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:56.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:56.969 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:56.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:56 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:56.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:56 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2206597840' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:57:56.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:56 smithi043 ceph-mon[36454]: osdmap e419: 8 total, 8 up, 8 in 2024-09-21T11:57:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:56 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:56 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2206597840' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:57:56.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:56 smithi043 ceph-mon[29710]: osdmap e419: 8 total, 8 up, 8 in 2024-09-21T11:57:57.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:56 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:57:57.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:56 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2206597840' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:57:57.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:56 smithi175 ceph-mon[32613]: osdmap e419: 8 total, 8 up, 8 in 2024-09-21T11:57:57.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:57.110 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:57.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:57.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:57.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:57.250 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:57.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:57.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:57.352 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:57.383 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:57.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:57.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:57.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:57.503 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T11:57:57.527 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:57.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:57.528 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:57.629 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:57.651 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:57.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:57.652 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:57.714 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 77 16 110035 110019 89.2627 65.8125 0.00285333 0.0111787 2024-09-21T11:57:57.752 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:57.774 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:57.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:57.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:57.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:57 smithi043 ceph-mon[36454]: pgmap v872: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 56 MiB/s wr, 887 op/s 2024-09-21T11:57:57.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:57 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4214686165' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:57:57.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:57 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:57:57.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:57 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:57:57.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:57 smithi043 ceph-mon[36454]: osdmap e420: 8 total, 8 up, 8 in 2024-09-21T11:57:57.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:57 smithi043 ceph-mon[29710]: pgmap v872: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 56 MiB/s wr, 887 op/s 2024-09-21T11:57:57.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4214686165' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:57:57.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:57 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:57:57.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:57 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:57:57.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:57 smithi043 ceph-mon[29710]: osdmap e420: 8 total, 8 up, 8 in 2024-09-21T11:57:57.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:57:57.876 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:57:57.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:57.907 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:57.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:57:58.009 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:58.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:57 smithi175 ceph-mon[32613]: pgmap v872: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 56 MiB/s wr, 887 op/s 2024-09-21T11:57:58.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:57 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4214686165' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:57:58.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:57 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T11:57:58.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:57 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T11:57:58.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:57 smithi175 ceph-mon[32613]: osdmap e420: 8 total, 8 up, 8 in 2024-09-21T11:57:58.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:58.030 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:58.031 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:58.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:58.155 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:58.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:58.156 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:58.257 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:58.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:58.279 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:58.280 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:58.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:58.402 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:58.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:58.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:58.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:57:58.539 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:58.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:58.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:57:58.641 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:58.662 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:58.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:58.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:58.712 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 2 1 2024-09-21T11:57:58.715 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 78 16 110035 110019 88.1187 0 - 0.0111787 2024-09-21T11:57:58.722 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:57:58.763 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:58.785 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:58.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:58.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:58.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:57:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:58.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:58.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:57:59.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:59.033 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:59.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:59.034 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:59.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:59.156 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:59.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:59.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:59.258 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:57:59.279 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:59.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:59.280 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:57:59.381 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:59.403 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:59.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:59.404 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:59.504 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:59.526 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:59.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:59.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:59.583 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:57:59.583 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:57:59.600 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:57:59.601 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:57:59.617 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:57:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:57:59.628 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:57:59.634 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:57:59.634 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:57:59.650 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:59.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:59.651 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:57:59.651 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:57:59.651 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:57:59.668 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:57:59.697 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:57:59.698 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:57:59.698 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:57:59.702 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:57:59.702 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:57:59.714 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 79 16 110035 110019 87.0037 0 - 0.0111787 2024-09-21T11:57:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:59 smithi043 ceph-mon[36454]: pgmap v874: 158 pgs: 1 unknown, 8 peering, 149 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 35 MiB/s wr, 550 op/s 2024-09-21T11:57:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:59 smithi043 ceph-mon[36454]: osdmap e421: 8 total, 8 up, 8 in 2024-09-21T11:57:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:59 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:57:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:59 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:57:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:57:59 smithi043 ceph-mon[36454]: Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-21T11:57:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:59 smithi043 ceph-mon[29710]: pgmap v874: 158 pgs: 1 unknown, 8 peering, 149 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 35 MiB/s wr, 550 op/s 2024-09-21T11:57:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:59 smithi043 ceph-mon[29710]: osdmap e421: 8 total, 8 up, 8 in 2024-09-21T11:57:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:59 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:57:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:59 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:57:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:57:59 smithi043 ceph-mon[29710]: Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-21T11:57:59.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:57:59.775 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:57:59.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:59.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:57:59.815 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:57:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:57:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:57:59.816 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:57:59.816 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:57:59.877 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:57:59.899 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:57:59.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:57:59.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:57:59.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:58:00.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:00.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:59 smithi175 ceph-mon[32613]: pgmap v874: 158 pgs: 1 unknown, 8 peering, 149 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 35 MiB/s wr, 550 op/s 2024-09-21T11:58:00.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:59 smithi175 ceph-mon[32613]: osdmap e421: 8 total, 8 up, 8 in 2024-09-21T11:58:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:59 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:58:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:59 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:58:00.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:57:59 smithi175 ceph-mon[32613]: Health check failed: Reduced data availability: 4 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-21T11:58:00.026 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:00.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:00.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:00.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:00.178 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:00.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:00.179 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:00.280 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:00.324 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:00.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:00.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:00.426 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:00.453 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:00.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:00.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:00.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:00.582 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:00.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:00.583 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:00.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:00.715 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:2024-09-21T11:58:00.714268+0000 min lat: 0.00120284 max lat: 2.51203 avg lat: 0.0115521 2024-09-21T11:58:00.715 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-21T11:58:00.715 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 80 16 110816 110800 86.5255 16.2708 0.00795743 0.0115521 2024-09-21T11:58:00.731 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:00.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:00.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:00.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[29710]: 11.9 starting backfill to osd.0 from (0'0,0'0] MAX to 419'6971 2024-09-21T11:58:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[29710]: osdmap e422: 8 total, 8 up, 8 in 2024-09-21T11:58:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[29710]: 11.6 starting backfill to osd.0 from (0'0,0'0] MAX to 419'6902 2024-09-21T11:58:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/81875390' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:58:00.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:58:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[36454]: 11.9 starting backfill to osd.0 from (0'0,0'0] MAX to 419'6971 2024-09-21T11:58:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[36454]: osdmap e422: 8 total, 8 up, 8 in 2024-09-21T11:58:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[36454]: 11.6 starting backfill to osd.0 from (0'0,0'0] MAX to 419'6902 2024-09-21T11:58:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/81875390' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:00.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:58:00.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:00 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:58:00.833 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:00.861 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:00.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:00.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:00.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:01.008 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:01.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:01.011 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:00 smithi175 ceph-mon[32613]: 11.9 starting backfill to osd.0 from (0'0,0'0] MAX to 419'6971 2024-09-21T11:58:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:00 smithi175 ceph-mon[32613]: osdmap e422: 8 total, 8 up, 8 in 2024-09-21T11:58:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:00 smithi175 ceph-mon[32613]: 11.6 starting backfill to osd.0 from (0'0,0'0] MAX to 419'6902 2024-09-21T11:58:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/81875390' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:00 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:58:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:00 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:58:01.112 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:01.140 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:01.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:01.141 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:01.242 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:01.267 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:01.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:01.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:01.270 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:01.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:58:01.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:01.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:01.415 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:01.415 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:01.516 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:01.540 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:01.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:01.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:01.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:01.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:01.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:01.715 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 81 16 111887 111871 86.2837 66.9375 0.0123685 0.011584 2024-09-21T11:58:01.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:01.810 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:01 smithi043 ceph-mon[29710]: pgmap v877: 158 pgs: 1 unknown, 8 peering, 149 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail 2024-09-21T11:58:01.810 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:01 smithi043 ceph-mon[29710]: osdmap e423: 8 total, 8 up, 8 in 2024-09-21T11:58:01.810 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:01 smithi043 ceph-mon[36454]: pgmap v877: 158 pgs: 1 unknown, 8 peering, 149 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail 2024-09-21T11:58:01.810 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:01 smithi043 ceph-mon[36454]: osdmap e423: 8 total, 8 up, 8 in 2024-09-21T11:58:01.823 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:01.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:01.824 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:01.851 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:01 smithi175 ceph-mon[32613]: pgmap v877: 158 pgs: 1 unknown, 8 peering, 149 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail 2024-09-21T11:58:01.851 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:01 smithi175 ceph-mon[32613]: osdmap e423: 8 total, 8 up, 8 in 2024-09-21T11:58:01.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:01.947 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:01.992 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:01.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:01.993 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:02.020 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:02.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:02.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:02.127 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:02.128 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:02.229 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:02.258 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:02.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:02.259 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:02.260 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:58:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:58:02.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:02.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:02.406 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:02.406 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:02.507 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:02.508 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T11:58:02.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:02.534 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:02.534 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:02.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:58:02.635 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:02.659 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:02.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:02.660 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:02.716 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 82 16 112879 112863 85.9867 62 0.0266119 0.0116246 2024-09-21T11:58:02.761 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:02.788 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:02.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:02.789 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:02.889 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:02.914 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:02.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:02.916 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:03.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:03.043 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:03.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:03.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:03.145 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:03.175 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:03.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:03.176 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:03 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:03 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:58:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:03 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:03 smithi175 ceph-mon[32613]: pgmap v879: 158 pgs: 1 active+remapped+backfilling, 8 peering, 1 active+remapped+backfill_toofull, 148 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 40 MiB/s wr, 636 op/s; 13897/334929 objects misplaced (4.149%); 2.0 MiB/s, 35 objects/s recovering 2024-09-21T11:58:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:03 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3858801677' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:58:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:03 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:58:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:03 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:58:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:03 smithi175 ceph-mon[32613]: osdmap e424: 8 total, 8 up, 8 in 2024-09-21T11:58:03.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:03.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:03.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:03.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:03.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:58:03.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:03.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[29710]: pgmap v879: 158 pgs: 1 active+remapped+backfilling, 8 peering, 1 active+remapped+backfill_toofull, 148 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 40 MiB/s wr, 636 op/s; 13897/334929 objects misplaced (4.149%); 2.0 MiB/s, 35 objects/s recovering 2024-09-21T11:58:03.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3858801677' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[29710]: osdmap e424: 8 total, 8 up, 8 in 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[36454]: pgmap v879: 158 pgs: 1 active+remapped+backfilling, 8 peering, 1 active+remapped+backfill_toofull, 148 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 40 MiB/s wr, 636 op/s; 13897/334929 objects misplaced (4.149%); 2.0 MiB/s, 35 objects/s recovering 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3858801677' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T11:58:03.388 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:03 smithi043 ceph-mon[36454]: osdmap e424: 8 total, 8 up, 8 in 2024-09-21T11:58:03.420 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:03.445 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:03.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:03.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:03.547 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:03.569 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:03.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:03.570 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:03.574 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245657 8.b 8:db11300a:::benchmark_data_smithi043_97042_object108563:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 580.00245203099996, 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0167927969999999, 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245657 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565746+0000", 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565747+0000", 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5340000000000001e-06 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565752+0000", 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8459999999999999e-06 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565762+0000", 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6029999999999994e-06 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565769+0000", 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3019999999999997e-06 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565833+0000", 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4139000000000004e-05 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.565842+0000", 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2550000000000003e-06 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576490+0000", 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.010647965 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576618+0000", 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000128233 2024-09-21T11:58:03.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.576763+0000", 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000144845 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582393+0000", 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056301229999999999 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.580 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582458+0000", 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4715000000000006e-05 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582472+0000", 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3985000000000001e-05 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582509+0000", 2024-09-21T11:58:03.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7063000000000001e-05 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582538+0000", 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9189000000000001e-05 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245660 8.b 8:d0b775d8:::benchmark_data_smithi043_97042_object108566:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 579.99952699400001, 2024-09-21T11:58:03.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0141818570000001, 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245660 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568671+0000", 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0200000000000003e-07 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568673+0000", 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2029999999999999e-06 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568679+0000", 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7590000000000003e-06 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568684+0000", 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1800000000000004e-06 2024-09-21T11:58:03.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568765+0000", 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0898999999999999e-05 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568774+0000", 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2e-06 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.579784+0000", 2024-09-21T11:58:03.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0110101520000001 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.579907+0000", 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000122404 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.580042+0000", 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000135205 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582646+0000", 2024-09-21T11:58:03.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026039240000000001 2024-09-21T11:58:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582746+0000", 2024-09-21T11:58:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100447 2024-09-21T11:58:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582764+0000", 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7265000000000002e-05 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582813+0000", 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9487999999999998e-05 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582852+0000", 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9329000000000002e-05 2024-09-21T11:58:03.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245661 8.b 8:d3f46cc5:::benchmark_data_smithi043_97042_object108567:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 579.99949818000005, 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.026528527, 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.590 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245661 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568699+0000", 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568700+0000", 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9200000000000001e-07 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568702+0000", 2024-09-21T11:58:03.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4049999999999998e-06 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568708+0000", 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0900000000000001e-06 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568715+0000", 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3319999999999997e-06 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568799+0000", 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4618999999999997e-05 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.568806+0000", 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3239999999999999e-06 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.582891+0000", 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0140855200000001 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.583014+0000", 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012294699999999999 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.583136+0000", 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121651 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588881+0000", 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057450959999999999 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588920+0000", 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9140999999999997e-05 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595113+0000", 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061930309999999999 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595177+0000", 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4243000000000001e-05 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595228+0000", 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0636000000000001e-05 2024-09-21T11:58:03.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245694 8.b 8:d4a2706b:::benchmark_data_smithi043_97042_object108600:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 579.970084875, 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99723669299999995, 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.598 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245694 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598113+0000", 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598114+0000", 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.203e-06 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598118+0000", 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3760000000000001e-06 2024-09-21T11:58:03.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598129+0000", 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0373999999999999e-05 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598137+0000", 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8290000000000008e-06 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598208+0000", 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1235000000000005e-05 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.598217+0000", 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0480000000000004e-06 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585700+0000", 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98748349499999999 2024-09-21T11:58:03.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585809+0000", 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010897500000000001 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.585910+0000", 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100326 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588958+0000", 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030481290000000001 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595096+0000", 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.006137829 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595241+0000", 2024-09-21T11:58:03.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014534899999999999 2024-09-21T11:58:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595298+0000", 2024-09-21T11:58:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7256999999999997e-05 2024-09-21T11:58:03.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595349+0000", 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1267999999999998e-05 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245711 8.b 8:d32e232b:::benchmark_data_smithi043_97042_object108617:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:58:03.606 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 579.95384299900002, 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98116202900000005, 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245711 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614355+0000", 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614357+0000", 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9910000000000001e-06 2024-09-21T11:58:03.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614362+0000", 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8119999999999996e-06 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614381+0000", 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8799000000000001e-05 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614390+0000", 2024-09-21T11:58:03.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6289999999999999e-06 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614452+0000", 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2620999999999996e-05 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.614462+0000", 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9869999999999994e-06 2024-09-21T11:58:03.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.588986+0000", 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.974523374 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.589275+0000", 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028897299999999999 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:58:03.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.589384+0000", 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000109497 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595361+0000", 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059765479999999999 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595400+0000", 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8847999999999998e-05 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595444+0000", 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4270999999999997e-05 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595473+0000", 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9339000000000001e-05 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595517+0000", 2024-09-21T11:58:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3340000000000002e-05 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245713 8.b 8:d46d9f23:::benchmark_data_smithi043_97042_object108619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 579.95303984400005, 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98051237099999999, 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.614 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245713 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615158+0000", 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615159+0000", 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.339e-06 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.616 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615164+0000", 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6659999999999999e-06 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615180+0000", 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5835e-05 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615188+0000", 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7600000000000008e-06 2024-09-21T11:58:03.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615244+0000", 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5172e-05 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.615252+0000", 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6969999999999999e-06 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592015+0000", 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97676251999999997 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592146+0000", 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000131094 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.592299+0000", 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015273399999999999 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595406+0000", 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031073920000000001 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595425+0000", 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9119000000000001e-05 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595582+0000", 2024-09-21T11:58:03.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001573 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595632+0000", 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9282e-05 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595670+0000", 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8461000000000003e-05 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.621 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245721 8.b 8:de2b861f:::benchmark_data_smithi043_97042_object108627:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 579.94513411499997, 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98811751400000003, 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:58:03.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:58:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245721 2024-09-21T11:58:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:58:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623063+0000", 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623065+0000", 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.553e-06 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623072+0000", 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4729999999999997e-06 2024-09-21T11:58:03.624 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623087+0000", 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5197999999999999e-05 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623095+0000", 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2369999999999992e-06 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.625 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623159+0000", 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3953000000000002e-05 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.623168+0000", 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0350000000000007e-06 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595704+0000", 2024-09-21T11:58:03.626 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97253655100000003 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595813+0000", 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108168 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.595916+0000", 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000103482 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.627 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604767+0000", 2024-09-21T11:58:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088508360000000008 2024-09-21T11:58:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604858+0000", 2024-09-21T11:58:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0768999999999998e-05 2024-09-21T11:58:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.628 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611090+0000", 2024-09-21T11:58:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0062325690000000003 2024-09-21T11:58:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611136+0000", 2024-09-21T11:58:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5197000000000001e-05 2024-09-21T11:58:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.629 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611181+0000", 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5493e-05 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245797 8.b 8:d43bc7ef:::benchmark_data_smithi043_97042_object108703:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:58:03.630 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 579.86689009899999, 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.910005379, 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245797 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.631 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701307+0000", 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.632 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701309+0000", 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1999999999999999e-06 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701315+0000", 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1090000000000003e-06 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701326+0000", 2024-09-21T11:58:03.633 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0956e-05 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701334+0000", 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7570000000000004e-06 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701426+0000", 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.2200999999999994e-05 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.634 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.701436+0000", 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0175e-05 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.598806+0000", 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.89737045500000001 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.598902+0000", 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5308000000000003e-05 2024-09-21T11:58:03.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.599142+0000", 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024078399999999999 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604823+0000", 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056808700000000002 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.636 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.610984+0000", 2024-09-21T11:58:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061603750000000001 2024-09-21T11:58:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.637 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611191+0000", 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000207612 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611273+0000", 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1924999999999999e-05 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.638 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611313+0000", 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9651999999999999e-05 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245872 8.b 8:d0b8f221:::benchmark_data_smithi043_97042_object108778:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 579.77954128099998, 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.82278842799999996, 2024-09-21T11:58:03.639 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245872 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788656+0000", 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788658+0000", 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.708e-06 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.641 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788664+0000", 2024-09-21T11:58:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5140000000000001e-06 2024-09-21T11:58:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788679+0000", 2024-09-21T11:58:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5764e-05 2024-09-21T11:58:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.642 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788691+0000", 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.173e-05 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788774+0000", 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3029999999999996e-05 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788783+0000", 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9840000000000007e-06 2024-09-21T11:58:03.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.601802+0000", 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81301899799999999 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.601917+0000", 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011464 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.644 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.602046+0000", 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000129228 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604894+0000", 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028480250000000001 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611032+0000", 2024-09-21T11:58:03.645 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061384710000000004 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611319+0000", 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028687299999999999 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611405+0000", 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5921999999999995e-05 2024-09-21T11:58:03.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611445+0000", 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9541e-05 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.15150.0:245873 8.b 8:dccc5dce:::benchmark_data_smithi043_97042_object108779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e233)", 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 579.77948442800005, 2024-09-21T11:58:03.647 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.82286793400000002, 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.15150", 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/2078210135", 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 245873 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.648 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788713+0000", 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788714+0000", 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5300000000000004e-07 2024-09-21T11:58:03.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788717+0000", 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2849999999999999e-06 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788723+0000", 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3300000000000004e-06 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.650 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788729+0000", 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3979999999999998e-06 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788805+0000", 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6068000000000004e-05 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:23.788810+0000", 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4720000000000003e-06 2024-09-21T11:58:03.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.604923+0000", 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81611315500000003 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.605021+0000", 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7503999999999993e-05 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,4", 2024-09-21T11:58:03.652 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.605122+0000", 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010059400000000001 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.610912+0000", 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057902530000000004 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611078+0000", 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016624000000000001 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.653 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611455+0000", 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037679500000000003 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611538+0000", 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3288999999999995e-05 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:48:24.611581+0000", 2024-09-21T11:58:03.654 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2997999999999997e-05 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194299 10.4 10:28bf67cd:::benchmark_data_smithi043_150678_object71548:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.581309+0000", 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 155.98688897599999, 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1019873919999998, 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.655 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194299 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581309+0000", 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581309+0000", 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581310+0000", 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3769999999999999e-06 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.657 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581319+0000", 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4210000000000001e-06 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581327+0000", 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7389999999999999e-06 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581334+0000", 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1080000000000004e-06 2024-09-21T11:58:03.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581401+0000", 2024-09-21T11:58:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6445000000000005e-05 2024-09-21T11:58:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.581411+0000", 2024-09-21T11:58:03.659 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9939999999999995e-06 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556211+0000", 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97480030200000001 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556223+0000", 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1844e-05 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.660 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.675650+0000", 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1194268890000001 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.677880+0000", 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00223026 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.661 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.678201+0000", 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032071399999999999 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.682954+0000", 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0047530539999999996 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683014+0000", 2024-09-21T11:58:03.662 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0504999999999998e-05 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683069+0000", 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4252000000000002e-05 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683086+0000", 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7492000000000001e-05 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.663 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683148+0000", 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1648000000000003e-05 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683296+0000", 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148348 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.664 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194318 10.4 10:2adde221:::benchmark_data_smithi043_150678_object71567:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.600714+0000", 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 155.96748353300001, 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0883899850000001, 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:03.665 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194318 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600714+0000", 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600714+0000", 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.666 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600716+0000", 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3549999999999999e-06 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600754+0000", 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7953000000000001e-05 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.667 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600787+0000", 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.2662e-05 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600797+0000", 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0239e-05 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600975+0000", 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017730400000000001 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.668 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600985+0000", 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0231000000000001e-05 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556251+0000", 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.95526609299999998 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.669 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556266+0000", 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4663e-05 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683516+0000", 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.127250428 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683657+0000", 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014146699999999999 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.670 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.683806+0000", 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148131 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688816+0000", 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050102640000000004 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688855+0000", 2024-09-21T11:58:03.671 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9101999999999999e-05 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688866+0000", 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1002000000000001e-05 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688937+0000", 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0576999999999999e-05 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.672 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688994+0000", 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7679999999999997e-05 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689104+0000", 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010983400000000001 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.673 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194320 10.4 10:287edeca:::benchmark_data_smithi043_150678_object71569:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.600823+0000", 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 155.96737469600001, 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.088657736, 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:03.674 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194320 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600823+0000", 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600823+0000", 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.675 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600823+0000", 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6199999999999998e-07 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600827+0000", 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1269999999999999e-06 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.676 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600833+0000", 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0599999999999996e-06 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.600838+0000", 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.601008+0000", 2024-09-21T11:58:03.677 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000169968 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.601014+0000", 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4969999999999998e-06 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556293+0000", 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.95527826599999999 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.678 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556301+0000", 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2570000000000007e-06 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.686056+0000", 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1297548079999999 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.686143+0000", 2024-09-21T11:58:03.679 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6985999999999997e-05 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.686286+0000", 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014329699999999999 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.688875+0000", 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025888059999999999 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.680 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689175+0000", 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000300167 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689228+0000", 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3185e-05 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.681 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689294+0000", 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5982000000000006e-05 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689359+0000", 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4987000000000006e-05 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689481+0000", 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012158 2024-09-21T11:58:03.682 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194340 10.4 10:236291d1:::benchmark_data_smithi043_150678_object71589:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.620209+0000", 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 155.94798905299999, 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.101492264, 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.683 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194340 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620209+0000", 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.684 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.685 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620209+0000", 2024-09-21T11:58:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.686 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620211+0000", 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.17e-06 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620217+0000", 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2060000000000004e-06 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.687 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620234+0000", 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7153999999999999e-05 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620242+0000", 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3960000000000006e-06 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620374+0000", 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013184700000000001 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.688 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.620384+0000", 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7019999999999996e-06 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556329+0000", 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.93594473600000005 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.689 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556337+0000", 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9750000000000007e-06 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689522+0000", 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1331854770000001 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689648+0000", 2024-09-21T11:58:03.690 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125826 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.689794+0000", 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000146305 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698662+0000", 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0088681130000000004 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.691 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698680+0000", 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7312e-05 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698782+0000", 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010262199999999999 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.721411+0000", 2024-09-21T11:58:03.692 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022628385000000001 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.721581+0000", 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017054099999999999 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.721701+0000", 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119497 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194360 10.4 10:22a7317d:::benchmark_data_smithi043_150678_object71609:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.638142+0000", 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 155.93005600999999, 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0838764699999999, 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.694 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194360 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638142+0000", 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.695 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638142+0000", 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638144+0000", 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.984e-06 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638150+0000", 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3969999999999999e-06 2024-09-21T11:58:03.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638170+0000", 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9559e-05 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638178+0000", 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5499999999999995e-06 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638293+0000", 2024-09-21T11:58:03.697 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114521 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.638304+0000", 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1287000000000001e-05 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556366+0000", 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.91806218399999995 2024-09-21T11:58:03.698 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556375+0000", 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7190000000000007e-06 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.692612+0000", 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.136237707 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.699 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.692769+0000", 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015609 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.692911+0000", 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014248599999999999 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698686+0000", 2024-09-21T11:58:03.700 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0057744409999999999 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698734+0000", 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8278000000000001e-05 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698864+0000", 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013069199999999999 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.701 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.721750+0000", 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.022885784999999999 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.721905+0000", 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015452599999999999 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722018+0000", 2024-09-21T11:58:03.702 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011326400000000001 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194367 10.4 10:201541d4:::benchmark_data_smithi043_150678_object71616:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.646661+0000", 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 155.92153645799999, 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0756208169999999, 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.703 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194367 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646661+0000", 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.704 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646661+0000", 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646663+0000", 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4500000000000001e-06 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646676+0000", 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2972999999999999e-05 2024-09-21T11:58:03.705 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646690+0000", 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4127999999999999e-05 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646696+0000", 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4099999999999996e-06 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.706 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646766+0000", 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0028000000000001e-05 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.646775+0000", 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7290000000000006e-06 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556406+0000", 2024-09-21T11:58:03.707 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.90963151200000003 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556416+0000", 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4350000000000003e-06 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.695728+0000", 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1393118389999999 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.708 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.695829+0000", 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000101401 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.695908+0000", 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9169e-05 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698691+0000", 2024-09-21T11:58:03.709 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027825929999999999 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706414+0000", 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077236359999999999 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706463+0000", 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8832999999999998e-05 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.710 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722087+0000", 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015623662 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722156+0000", 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9255e-05 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722282+0000", 2024-09-21T11:58:03.711 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000125764 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194376 10.4 10:26311a81:::benchmark_data_smithi043_150678_object71625:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.658363+0000", 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 155.909834824, 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.064158495, 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.712 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194376 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658363+0000", 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.713 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658363+0000", 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658364+0000", 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.655e-06 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658371+0000", 2024-09-21T11:58:03.714 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4239999999999998e-06 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658389+0000", 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8346e-05 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658397+0000", 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5070000000000003e-06 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.715 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658468+0000", 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0887000000000001e-05 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.658480+0000", 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2006e-05 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556442+0000", 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.89796246999999996 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556449+0000", 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4109999999999999e-06 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.698913+0000", 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.142463979 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.699050+0000", 2024-09-21T11:58:03.717 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000136286 2024-09-21T11:58:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:58:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.699189+0000", 2024-09-21T11:58:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013936299999999999 2024-09-21T11:58:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.718 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706428+0000", 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0072391399999999998 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706501+0000", 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3170999999999995e-05 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706541+0000", 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.9657000000000003e-05 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722333+0000", 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.015791667999999998 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722392+0000", 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9348000000000002e-05 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.720 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722521+0000", 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012917699999999999 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194389 10.4 10:2e49ec61:::benchmark_data_smithi043_150678_object71638:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.674458+0000", 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 155.89373983999999, 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0482976590000002, 2024-09-21T11:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194389 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.722 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674458+0000", 2024-09-21T11:58:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674458+0000", 2024-09-21T11:58:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.723 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674460+0000", 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9829999999999999e-06 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674467+0000", 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0389999999999999e-06 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674491+0000", 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3977000000000001e-05 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674500+0000", 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.9409999999999996e-06 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674578+0000", 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8226999999999995e-05 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.674591+0000", 2024-09-21T11:58:03.725 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3202e-05 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556476+0000", 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.88188519700000001 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556484+0000", 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.396e-06 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.726 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.703276+0000", 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.146791801 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.703405+0000", 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000129615 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:58:03.727 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.703540+0000", 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000134778 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.712886+0000", 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0093464740000000004 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.712929+0000", 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2154999999999999e-05 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.728 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.720772+0000", 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007843104 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722572+0000", 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001800711 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722634+0000", 2024-09-21T11:58:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.1736000000000007e-05 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722755+0000", 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121323 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194396 10.4 10:2cc12a85:::benchmark_data_smithi043_150678_object71645:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.689435+0000", 2024-09-21T11:58:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 155.87876296100001, 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0335722509999998, 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194396 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689435+0000", 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689435+0000", 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689436+0000", 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4890000000000001e-06 2024-09-21T11:58:03.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689442+0000", 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.818e-06 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689455+0000", 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2819999999999999e-05 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.733 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689462+0000", 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4580000000000004e-06 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689528+0000", 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6221000000000002e-05 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.689536+0000", 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9929999999999995e-06 2024-09-21T11:58:03.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556510+0000", 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.86697341500000003 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.556518+0000", 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0239999999999997e-06 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.735 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706571+0000", 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.150052691 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706680+0000", 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010967999999999999 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.706801+0000", 2024-09-21T11:58:03.736 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120394 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.712969+0000", 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0061681189999999997 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.713014+0000", 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4926999999999999e-05 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.737 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.720775+0000", 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0077615119999999999 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722800+0000", 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002024311 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.722857+0000", 2024-09-21T11:58:03.738 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7161999999999999e-05 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.723007+0000", 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015021699999999999 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:194404 10.4 10:2c63f935:::benchmark_data_smithi043_150678_object71653:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e364)", 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:55:27.718920+0000", 2024-09-21T11:58:03.739 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 155.84927744800001, 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0043187869999999, 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 194404 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:03.740 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718920+0000", 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718920+0000", 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718922+0000", 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4100000000000001e-06 2024-09-21T11:58:03.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718927+0000", 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7749999999999998e-06 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718939+0000", 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2038000000000001e-05 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:03.742 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.718948+0000", 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4500000000000004e-06 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.719014+0000", 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6309000000000005e-05 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:27.719021+0000", 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0700000000000001e-06 2024-09-21T11:58:03.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.558746+0000", 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.839724688 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:28.558767+0000", 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0749000000000001e-05 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.744 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.709717+0000", 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1509502739999999 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.709865+0000", 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000148589 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1,4", 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.710005+0000", 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0001392 2024-09-21T11:58:03.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.720727+0000", 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010722687999999999 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.720764+0000", 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6353000000000002e-05 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.746 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.720780+0000", 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5795e-05 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.723059+0000", 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022793800000000001 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.723117+0000", 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8557000000000002e-05 2024-09-21T11:58:03.747 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:58:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:55:29.723239+0000", 2024-09-21T11:58:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121462 2024-09-21T11:58:03.748 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:03.749 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:03.749 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:58:03.749 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 83 16 113882 113866 85.706 62.6875 0.0231004 0.0116625 2024-09-21T11:58:03.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:03.750 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:03.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:03.851 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:03.879 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:03.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:03.880 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:03.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:04.010 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:04.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:04.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:04.112 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:04.143 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:04.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:04.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:04.244 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:04.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:04.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:04.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:04.433 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:04.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:04.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:04.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:04 smithi043 ceph-mon[29710]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:04.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:04 smithi043 ceph-mon[36454]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:04.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:04 smithi175 ceph-mon[32613]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:04.535 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:04.569 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:04.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:04.570 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:04.671 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:04.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 3 1 2024-09-21T11:58:04.716 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 84 16 115022 115006 85.5339 71.25 0.0116557 0.0116864 2024-09-21T11:58:04.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:04.724 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:04.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:04.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:04.871 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:04.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:04.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:04.997 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:04.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:04.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:05.099 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:05.170 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:05.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:05.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:05.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:05.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:05.317 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:05.317 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:05.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:05.456 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:05.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:05.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:05.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:05 smithi043 ceph-mon[29710]: pgmap v881: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 907 B/s rd, 109 MiB/s wr, 1.74k op/s; 13897/334929 objects misplaced (4.149%); 1.7 MiB/s, 30 objects/s recovering 2024-09-21T11:58:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:58:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:58:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:05.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:05 smithi043 ceph-mon[36454]: pgmap v881: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 907 B/s rd, 109 MiB/s wr, 1.74k op/s; 13897/334929 objects misplaced (4.149%); 1.7 MiB/s, 30 objects/s recovering 2024-09-21T11:58:05.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:05 smithi175 ceph-mon[32613]: pgmap v881: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 907 B/s rd, 109 MiB/s wr, 1.74k op/s; 13897/334929 objects misplaced (4.149%); 1.7 MiB/s, 30 objects/s recovering 2024-09-21T11:58:05.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:05.596 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:05.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:05.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:05.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:05.716 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 85 16 115913 115897 85.1828 55.6875 0.0127866 0.0117341 2024-09-21T11:58:05.736 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:05.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:05.738 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:05.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:05.878 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:05.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:05.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:05.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:06.001 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:06.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:06.002 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:06.059 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:58:06.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:06.145 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:06.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:06.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:06.246 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:06.274 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:06.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:06.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:06.376 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:06.406 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:06.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:06.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:06.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:06 smithi043 ceph-mon[29710]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 4 pgs peering) 2024-09-21T11:58:06.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:06 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1434151638' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:06.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:06 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:06.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:06 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:58:06.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:06 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:58:06.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:06 smithi043 ceph-mon[36454]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 4 pgs peering) 2024-09-21T11:58:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:06 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1434151638' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:06 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:06 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:58:06.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:06 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:58:06.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:06.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:06 smithi175 ceph-mon[32613]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs inactive, 4 pgs peering) 2024-09-21T11:58:06.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:06 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1434151638' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:06.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:06 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:06.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:06 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:58:06.512 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:06 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:58:06.537 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:06.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:06.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:06.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:06.677 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:06.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:06.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:06.716 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 86 16 116926 116910 84.9285 63.3125 0.00439445 0.0117696 2024-09-21T11:58:06.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:06.808 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:06.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:06.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:06.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:06.936 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:06.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:06.937 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:07.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:07.064 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:07.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:07.065 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:07.165 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:07.193 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:07.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:07.194 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:07.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:07.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:07.334 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:07.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:07.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:07.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:07.474 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:58:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[29710]: osdmap e425: 8 total, 8 up, 8 in 2024-09-21T11:58:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[29710]: pgmap v883: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 122 MiB/s wr, 1.95k op/s; 14237/340683 objects misplaced (4.179%); 1.6 MiB/s, 28 objects/s recovering 2024-09-21T11:58:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:58:07.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:58:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:58:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[36454]: osdmap e425: 8 total, 8 up, 8 in 2024-09-21T11:58:07.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[36454]: pgmap v883: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 122 MiB/s wr, 1.95k op/s; 14237/340683 objects misplaced (4.179%); 1.6 MiB/s, 28 objects/s recovering 2024-09-21T11:58:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:58:07.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:07 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:58:07.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:07 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:07.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:07 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:58:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:07 smithi175 ceph-mon[32613]: osdmap e425: 8 total, 8 up, 8 in 2024-09-21T11:58:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:07 smithi175 ceph-mon[32613]: pgmap v883: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 122 MiB/s wr, 1.95k op/s; 14237/340683 objects misplaced (4.179%); 1.6 MiB/s, 28 objects/s recovering 2024-09-21T11:58:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:07 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:58:07.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:07 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:58:07.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:07.606 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:07.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:07.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:07.708 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:07.716 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 87 16 117991 117975 84.7174 66.5625 0.0131169 0.0117985 2024-09-21T11:58:07.736 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:07.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:07.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:07.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:07.874 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:07.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:07.875 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:07.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:08.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:08.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:08.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:08.153 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:08.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:08.154 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:08.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:08.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:08.286 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:08.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:08.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:08.419 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:08.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:08.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:08.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:08 smithi043 ceph-mon[29710]: osdmap e426: 8 total, 8 up, 8 in 2024-09-21T11:58:08.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:08 smithi043 ceph-mon[36454]: osdmap e426: 8 total, 8 up, 8 in 2024-09-21T11:58:08.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:08 smithi175 ceph-mon[32613]: osdmap e426: 8 total, 8 up, 8 in 2024-09-21T11:58:08.522 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:08.548 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:08.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:08.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:08.650 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:08.677 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:08.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:08.678 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:08.716 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 88 16 119123 119107 84.5587 70.75 0.0176051 0.0118212 2024-09-21T11:58:08.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:08.826 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:08.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:08.827 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:08.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:08.966 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:08.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:08.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:09.068 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:09.113 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:09.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:09.114 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:09.204 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T11:58:09.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:09.269 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:09.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:09.271 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:09.372 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:09.411 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:09.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:09.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:09.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:09 smithi043 ceph-mon[29710]: pgmap v885: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 842 op/s; 14128/350079 objects misplaced (4.036%); 4.3 MiB/s, 69 objects/s recovering 2024-09-21T11:58:09.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:09 smithi043 ceph-mon[36454]: pgmap v885: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 842 op/s; 14128/350079 objects misplaced (4.036%); 4.3 MiB/s, 69 objects/s recovering 2024-09-21T11:58:09.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:09 smithi175 ceph-mon[32613]: pgmap v885: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 53 MiB/s wr, 842 op/s; 14128/350079 objects misplaced (4.036%); 4.3 MiB/s, 69 objects/s recovering 2024-09-21T11:58:09.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:09.551 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:09.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:09.552 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:09.604 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:09.653 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:09.688 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:09.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:09.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:09.716 INFO:tasks.radosbench.radosbench.0.smithi043.stdout: 89 16 120269 120253 84.4133 71.625 0.0094996 0.0118413 2024-09-21T11:58:09.772 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:09.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:09.864 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:09.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:09.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:09.966 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:09.994 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:09.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:09.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:10.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:10.148 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:10.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:10.149 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:10.250 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:10.294 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:10.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:10.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:10.396 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:10.397 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:10.422 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:10.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:10.423 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:10.479 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:58:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:58:10.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:10.525 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:58:10.525 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:58:10.561 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:58:10.562 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:58:10.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:10.571 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:10.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:10.601 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:58:10.601 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:58:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:58:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:58:10.623 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T11:58:10.623 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:58:10.638 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:58:10.639 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:58:10.671 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:58:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:58:10.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:10.695 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:10.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:10.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:10.698 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:58:10.698 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:58:10.716 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Total time run: 90.01 2024-09-21T11:58:10.716 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Total writes made: 121232 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Write size: 65536 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Object size: 65536 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Bandwidth (MB/sec): 84.1795 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev Bandwidth: 32.7488 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max bandwidth (MB/sec): 192 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min bandwidth (MB/sec): 0 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Average IOPS: 1346 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev IOPS: 523.989 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max IOPS: 3072 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min IOPS: 0 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Average Latency(s): 0.0118748 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Stddev Latency(s): 0.0298217 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Max latency(s): 2.51203 2024-09-21T11:58:10.717 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Min latency(s): 0.00120284 2024-09-21T11:58:10.719 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:58:10.720 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:58:10.721 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Cleaning up (deleting benchmark objects) 2024-09-21T11:58:10.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:10.838 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:10.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:10.839 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:10.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:10.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:10.968 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:11.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:11.117 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:11.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:11.118 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:11.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:11.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:11.260 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:11.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:11.361 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:11.392 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:11.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:11.393 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:11.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:11 smithi043 ceph-mon[36454]: pgmap v886: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 668 B/s rd, 41 MiB/s wr, 659 op/s; 14128/350079 objects misplaced (4.036%); 3.3 MiB/s, 54 objects/s recovering 2024-09-21T11:58:11.427 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:11 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3998238361' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:11.427 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:11 smithi043 ceph-mon[29710]: pgmap v886: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 668 B/s rd, 41 MiB/s wr, 659 op/s; 14128/350079 objects misplaced (4.036%); 3.3 MiB/s, 54 objects/s recovering 2024-09-21T11:58:11.427 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:11 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3998238361' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:11.494 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:11.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:11 smithi175 ceph-mon[32613]: pgmap v886: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 668 B/s rd, 41 MiB/s wr, 659 op/s; 14128/350079 objects misplaced (4.036%); 3.3 MiB/s, 54 objects/s recovering 2024-09-21T11:58:11.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:11 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3998238361' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:11.519 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:11.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:11.520 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:11.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:11.645 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:11.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:11.646 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:11.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:11.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:11.790 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:11.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:11.791 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:58:11.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:11.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:11.929 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:12.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:12.061 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:12.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:12.062 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:12.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:12.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:12.203 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:12.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T11:58:12.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:58:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:58:12.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:12.350 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:12.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:12.351 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:12.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:12.491 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:12.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:12 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4079198887' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:58:12.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:12 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:58:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:12 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4079198887' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:58:12.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:12 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:58:12.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:12.492 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:12.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:12 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4079198887' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:58:12.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:12 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T11:58:12.593 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:12.622 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:12.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:12.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:12.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:12.763 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:12.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:12.764 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:12.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:12.908 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:12.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:12.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:13.009 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:13.034 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:13.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:13.035 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:13.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:13.165 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:13.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:13.166 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:13.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:13.294 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:13.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:13.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:13.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:13.440 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:13.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:13.441 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:13.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:13 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:13.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:13 smithi043 ceph-mon[36454]: pgmap v887: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 76 MiB/s wr, 1.31k op/s; 14096/362226 objects misplaced (3.891%); 7.1 MiB/s, 112 objects/s recovering 2024-09-21T11:58:13.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:13 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:58:13.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:13 smithi043 ceph-mon[36454]: osdmap e427: 8 total, 8 up, 8 in 2024-09-21T11:58:13.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:13 smithi043 ceph-mon[36454]: osdmap e428: 8 total, 8 up, 8 in 2024-09-21T11:58:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:13 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:13.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:13 smithi043 ceph-mon[29710]: pgmap v887: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 76 MiB/s wr, 1.31k op/s; 14096/362226 objects misplaced (3.891%); 7.1 MiB/s, 112 objects/s recovering 2024-09-21T11:58:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:13 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:58:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:13 smithi043 ceph-mon[29710]: osdmap e427: 8 total, 8 up, 8 in 2024-09-21T11:58:13.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:13 smithi043 ceph-mon[29710]: osdmap e428: 8 total, 8 up, 8 in 2024-09-21T11:58:13.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:13 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:13.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:13 smithi175 ceph-mon[32613]: pgmap v887: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 76 MiB/s wr, 1.31k op/s; 14096/362226 objects misplaced (3.891%); 7.1 MiB/s, 112 objects/s recovering 2024-09-21T11:58:13.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:13 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T11:58:13.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:13 smithi175 ceph-mon[32613]: osdmap e427: 8 total, 8 up, 8 in 2024-09-21T11:58:13.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:13 smithi175 ceph-mon[32613]: osdmap e428: 8 total, 8 up, 8 in 2024-09-21T11:58:13.541 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:13.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:13.572 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:13.572 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:13.673 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:13.694 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:13.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:13.695 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:13.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T11:58:13.796 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:13.827 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:13.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:13.828 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:13.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:13.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:13.970 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:13.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:14.069 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:14.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:14.106 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:14.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:14.107 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:14.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:14.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:14.255 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:14.255 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:14.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:14.398 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:14.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:14.399 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:14.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:14.524 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:14.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:14.525 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:14.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:14.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:14.667 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:14.717 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:58:14.717 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:58:14.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:14.805 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:14.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:14.806 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:14.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:14.933 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:14.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:14.934 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:15.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 4 1 2024-09-21T11:58:15.079 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:15.080 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:15.105 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:15.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:15.106 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:15.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:15.236 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:58:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:58:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:15.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:15.254 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:15.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:15.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:15.376 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:15.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:15.377 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:15.478 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:15.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:15 smithi043 ceph-mon[29710]: pgmap v890: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 866 B/s rd, 42 MiB/s wr, 766 op/s; 14096/362226 objects misplaced (3.891%); 4.4 MiB/s, 69 objects/s recovering 2024-09-21T11:58:15.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:15 smithi043 ceph-mon[36454]: pgmap v890: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 866 B/s rd, 42 MiB/s wr, 766 op/s; 14096/362226 objects misplaced (3.891%); 4.4 MiB/s, 69 objects/s recovering 2024-09-21T11:58:15.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:15.500 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:15.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:15.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:15 smithi175 ceph-mon[32613]: pgmap v890: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.4 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 866 B/s rd, 42 MiB/s wr, 766 op/s; 14096/362226 objects misplaced (3.891%); 4.4 MiB/s, 69 objects/s recovering 2024-09-21T11:58:15.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:15.652 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:15.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:15.653 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:15.748 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:15.755 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:15.780 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:15.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:15.781 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:15.882 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:15.908 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:15.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:15.909 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:16.009 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:16.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:16.039 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:16.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:16.140 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:16.165 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:16.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:16.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:16.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:58:16.253 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:58:16.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:16.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:16.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:16.422 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:16.447 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:16.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:16.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:16.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:16 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1109010556' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:16.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:16 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:16 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1109010556' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:16.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:16 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:16.501 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:16 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1109010556' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:16.501 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:16 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:16.540 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T11:58:16.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:16.612 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:16.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:16.613 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:16.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:16.751 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:16.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:16.752 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:16.853 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:16.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:16.900 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:16.928 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:17.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:17.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:17.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:17.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:17.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:17.194 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:17.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:17.275 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T11:58:17.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:17.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:17.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: pgmap v891: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 767 B/s rd, 37 MiB/s wr, 903 op/s; 13799/356844 objects misplaced (3.867%); 3.9 MiB/s, 61 objects/s recovering 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: osdmap e429: 8 total, 8 up, 8 in 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4025535593' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:58:17.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: pgmap v891: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 767 B/s rd, 37 MiB/s wr, 903 op/s; 13799/356844 objects misplaced (3.867%); 3.9 MiB/s, 61 objects/s recovering 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: osdmap e429: 8 total, 8 up, 8 in 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4025535593' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:58:17.352 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:17 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: pgmap v891: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 767 B/s rd, 37 MiB/s wr, 903 op/s; 13799/356844 objects misplaced (3.867%); 3.9 MiB/s, 61 objects/s recovering 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: osdmap e429: 8 total, 8 up, 8 in 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4025535593' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:58:17.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:17 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:17.436 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:17.458 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:17.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:17.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:17.542 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:58:17.543 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:58:17.560 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:17.582 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:17.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:17.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:17.684 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:17.705 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:17.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:17.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:17.806 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:17.828 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:17.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:17.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:17.929 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:17.930 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:58:17.931 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:58:17.950 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:17.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:17.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:17.952 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:58:17.952 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:58:17.974 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:58:17.974 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:58:17.996 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:58:17.996 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:58:18.017 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:58:18.017 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:58:18.035 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:58:18.036 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:58:18.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:18.055 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:58:18.055 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:58:18.079 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:18.080 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:58:18.080 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:58:18.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:18.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:18.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:18.206 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:18.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:18.207 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:18.212 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:58:18.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:18.329 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:18.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:18.330 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:18.376 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:58:18.376 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[29710]: osdmap e430: 8 total, 8 up, 8 in 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[29710]: osdmap e431: 8 total, 8 up, 8 in 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[29710]: 4.1 scrub starts 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[29710]: 4.1 scrub ok 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/826605392' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[36454]: osdmap e430: 8 total, 8 up, 8 in 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[36454]: osdmap e431: 8 total, 8 up, 8 in 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[36454]: 4.1 scrub starts 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[36454]: 4.1 scrub ok 2024-09-21T11:58:18.377 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:18 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/826605392' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:18.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 5 1 2024-09-21T11:58:18.430 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:18.453 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:18.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:18.454 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:18.477 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:18.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:18 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:58:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:18 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:58:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:18 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T11:58:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:18 smithi175 ceph-mon[32613]: osdmap e430: 8 total, 8 up, 8 in 2024-09-21T11:58:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:18 smithi175 ceph-mon[32613]: osdmap e431: 8 total, 8 up, 8 in 2024-09-21T11:58:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:18 smithi175 ceph-mon[32613]: 4.1 scrub starts 2024-09-21T11:58:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:18 smithi175 ceph-mon[32613]: 4.1 scrub ok 2024-09-21T11:58:18.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:18 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/826605392' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:18.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:18.578 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:18.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:18.579 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:18.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:18.731 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:18.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:18.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:18.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:18.874 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:18.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:18.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:18.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:19.015 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:19.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:19.016 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:19.072 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:58:19.072 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:58:19.072 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.072 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.26339.0:128102 11.5 e432/392 11:aa12e934:::benchmark_data_smithi043_178089_object6867:head v 432'7880, mlcod=432'7878)", 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:58:19.069467+0000", 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.00066296900000000001, 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00068894400000000004, 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069467+0000", 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.073 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069467+0000", 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069468+0000", 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3000000000000001e-07 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069473+0000", 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1030000000000001e-06 2024-09-21T11:58:19.074 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069480+0000", 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4359999999999996e-06 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069486+0000", 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4929999999999998e-06 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069537+0000", 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1221000000000003e-05 2024-09-21T11:58:19.075 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069556+0000", 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8518000000000001e-05 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.26339.0:128104 11.8 e432/392 11:1d446d45:::benchmark_data_smithi043_178089_object6869:head v 432'8043, mlcod=432'8041)", 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:58:19.069368+0000", 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.00076253699999999998, 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00084029700000000005, 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:19.076 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069368+0000", 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069368+0000", 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.077 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069369+0000", 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1710000000000001e-06 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069375+0000", 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.9970000000000004e-06 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069385+0000", 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8549999999999997e-06 2024-09-21T11:58:19.078 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069392+0000", 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8650000000000003e-06 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069449+0000", 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7039000000000001e-05 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:19.069470+0000", 2024-09-21T11:58:19.079 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1229999999999998e-05 2024-09-21T11:58:19.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:19.080 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:19.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:19.080 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:19.080 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:58:19.080 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 2 2024-09-21T11:58:19.080 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:58:19.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:19.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:58:19.158 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:19.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:19.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:19.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:19.321 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:19.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:19.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:19.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:19.454 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:19.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:19.455 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:19.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:19.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:19.601 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:19.702 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:19.741 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:19.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:19.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:19.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:19 smithi175 ceph-mon[32613]: pgmap v895: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 815 op/s; 12929/348399 objects misplaced (3.711%); 7.2 MiB/s, 115 objects/s recovering 2024-09-21T11:58:19.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:19 smithi175 ceph-mon[32613]: 3.e scrub starts 2024-09-21T11:58:19.820 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:19 smithi175 ceph-mon[32613]: 3.e scrub ok 2024-09-21T11:58:19.821 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:19 smithi175 ceph-mon[32613]: osdmap e432: 8 total, 8 up, 8 in 2024-09-21T11:58:19.821 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:19 smithi175 ceph-mon[32613]: 11.e starting backfill to osd.4 from (0'0,0'0] MAX to 429'7913 2024-09-21T11:58:19.821 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:19 smithi175 ceph-mon[32613]: 11.c starting backfill to osd.4 from (0'0,0'0] MAX to 429'7968 2024-09-21T11:58:19.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:19.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:58:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:19.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:19.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:19.979 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[29710]: pgmap v895: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 815 op/s; 12929/348399 objects misplaced (3.711%); 7.2 MiB/s, 115 objects/s recovering 2024-09-21T11:58:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[29710]: 3.e scrub starts 2024-09-21T11:58:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[29710]: 3.e scrub ok 2024-09-21T11:58:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[29710]: osdmap e432: 8 total, 8 up, 8 in 2024-09-21T11:58:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[29710]: 11.e starting backfill to osd.4 from (0'0,0'0] MAX to 429'7913 2024-09-21T11:58:19.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[29710]: 11.c starting backfill to osd.4 from (0'0,0'0] MAX to 429'7968 2024-09-21T11:58:19.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[36454]: pgmap v895: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 815 op/s; 12929/348399 objects misplaced (3.711%); 7.2 MiB/s, 115 objects/s recovering 2024-09-21T11:58:19.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[36454]: 3.e scrub starts 2024-09-21T11:58:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[36454]: 3.e scrub ok 2024-09-21T11:58:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[36454]: osdmap e432: 8 total, 8 up, 8 in 2024-09-21T11:58:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[36454]: 11.e starting backfill to osd.4 from (0'0,0'0] MAX to 429'7913 2024-09-21T11:58:19.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:19 smithi043 ceph-mon[36454]: 11.c starting backfill to osd.4 from (0'0,0'0] MAX to 429'7968 2024-09-21T11:58:20.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:20.018 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:20.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:20.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:20.164 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:20.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:20.165 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:20.225 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:20.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:20.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:20.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:20.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:20.436 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:20.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:20.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:20.538 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:20.577 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:20.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:20.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:20.688 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:20.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:20.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:20.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:20 smithi175 ceph-mon[32613]: 5.1f scrub starts 2024-09-21T11:58:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:20 smithi175 ceph-mon[32613]: 5.1f scrub ok 2024-09-21T11:58:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:20 smithi175 ceph-mon[32613]: 5.1 scrub starts 2024-09-21T11:58:20.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:20 smithi175 ceph-mon[32613]: 5.1 scrub ok 2024-09-21T11:58:20.830 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:20.867 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:20.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:20.868 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:20.962 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:58:20.962 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:58:20.963 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:58:20.963 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:58:20.963 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:58:20.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:20 smithi043 ceph-mon[29710]: 5.1f scrub starts 2024-09-21T11:58:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:20 smithi043 ceph-mon[29710]: 5.1f scrub ok 2024-09-21T11:58:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:20 smithi043 ceph-mon[29710]: 5.1 scrub starts 2024-09-21T11:58:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:20 smithi043 ceph-mon[29710]: 5.1 scrub ok 2024-09-21T11:58:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:20 smithi043 ceph-mon[36454]: 5.1f scrub starts 2024-09-21T11:58:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:20 smithi043 ceph-mon[36454]: 5.1f scrub ok 2024-09-21T11:58:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:20 smithi043 ceph-mon[36454]: 5.1 scrub starts 2024-09-21T11:58:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:20 smithi043 ceph-mon[36454]: 5.1 scrub ok 2024-09-21T11:58:21.005 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:21.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:21.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:21.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:21.132 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:21.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:21.133 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:21.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:21.269 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:21.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:21.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:21.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:21.392 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:21.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:21.393 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:21.494 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:21.526 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:21.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:21.527 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:21.604 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:21.610 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:21.628 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:21.654 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:21.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:21.655 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[36454]: 11.3 scrub starts 2024-09-21T11:58:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[36454]: 6.17 scrub starts 2024-09-21T11:58:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[36454]: 6.17 scrub ok 2024-09-21T11:58:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[36454]: pgmap v897: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 704 op/s; 12929/348399 objects misplaced (3.711%); 10 MiB/s, 162 objects/s recovering 2024-09-21T11:58:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[36454]: 6.15 scrub starts 2024-09-21T11:58:21.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[36454]: 6.15 scrub ok 2024-09-21T11:58:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[29710]: 11.3 scrub starts 2024-09-21T11:58:21.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[29710]: 6.17 scrub starts 2024-09-21T11:58:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[29710]: 6.17 scrub ok 2024-09-21T11:58:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[29710]: pgmap v897: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 704 op/s; 12929/348399 objects misplaced (3.711%); 10 MiB/s, 162 objects/s recovering 2024-09-21T11:58:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[29710]: 6.15 scrub starts 2024-09-21T11:58:21.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:21 smithi043 ceph-mon[29710]: 6.15 scrub ok 2024-09-21T11:58:21.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:21.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:21.808 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:21.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:21.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:21 smithi175 ceph-mon[32613]: 11.3 scrub starts 2024-09-21T11:58:21.873 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:21 smithi175 ceph-mon[32613]: 6.17 scrub starts 2024-09-21T11:58:21.873 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:21 smithi175 ceph-mon[32613]: 6.17 scrub ok 2024-09-21T11:58:21.873 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:21 smithi175 ceph-mon[32613]: pgmap v897: 158 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 156 active+clean; 7.1 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 704 op/s; 12929/348399 objects misplaced (3.711%); 10 MiB/s, 162 objects/s recovering 2024-09-21T11:58:21.873 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:21 smithi175 ceph-mon[32613]: 6.15 scrub starts 2024-09-21T11:58:21.873 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:21 smithi175 ceph-mon[32613]: 6.15 scrub ok 2024-09-21T11:58:21.909 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:21.932 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:21.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:21.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:22.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:22.074 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:22.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:22.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:22.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:22.207 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:22.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:22.208 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:22.260 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:58:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:58:22.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:22.335 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:22.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:22.336 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:22.437 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:22.464 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:22.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:22.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:22.566 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:22.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:22.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:22.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:22.696 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:22.720 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:22.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:22.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:22 smithi043 ceph-mon[29710]: 5.a scrub starts 2024-09-21T11:58:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:22 smithi043 ceph-mon[29710]: 5.a scrub ok 2024-09-21T11:58:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:22 smithi043 ceph-mon[29710]: 6.1a scrub starts 2024-09-21T11:58:22.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:22 smithi043 ceph-mon[29710]: 6.1a scrub ok 2024-09-21T11:58:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:22 smithi043 ceph-mon[36454]: 5.a scrub starts 2024-09-21T11:58:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:22 smithi043 ceph-mon[36454]: 5.a scrub ok 2024-09-21T11:58:22.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:22 smithi043 ceph-mon[36454]: 6.1a scrub starts 2024-09-21T11:58:22.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:22 smithi043 ceph-mon[36454]: 6.1a scrub ok 2024-09-21T11:58:22.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:22.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:22.868 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:22.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:22.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:23.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:22 smithi175 ceph-mon[32613]: 5.a scrub starts 2024-09-21T11:58:23.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:22 smithi175 ceph-mon[32613]: 5.a scrub ok 2024-09-21T11:58:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:22 smithi175 ceph-mon[32613]: 6.1a scrub starts 2024-09-21T11:58:23.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:22 smithi175 ceph-mon[32613]: 6.1a scrub ok 2024-09-21T11:58:23.014 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:23.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:23.018 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:23.118 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:23.130 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:58:23.146 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:23.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:23.147 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:23.248 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:23.285 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:23.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:23.286 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:23.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:23.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:23.420 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:23.420 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:23.513 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:23.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:23.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:58:23.558 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:23.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:23.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:23.580 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:58:23.580 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T11:58:23.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:23.690 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:23.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:23.691 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:23.791 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:23.837 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[29710]: 11.3 scrub ok 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[29710]: pgmap v898: 158 pgs: 1 active+remapped+backfilling, 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 153 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.12k op/s; 25057/336969 objects misplaced (7.436%); 22 MiB/s, 353 objects/s recovering 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[29710]: 5.1a scrub starts 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[29710]: 5.1a scrub ok 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[29710]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/859289443' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2788532692' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:23.838 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[36454]: 11.3 scrub ok 2024-09-21T11:58:23.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[36454]: pgmap v898: 158 pgs: 1 active+remapped+backfilling, 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 153 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.12k op/s; 25057/336969 objects misplaced (7.436%); 22 MiB/s, 353 objects/s recovering 2024-09-21T11:58:23.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[36454]: 5.1a scrub starts 2024-09-21T11:58:23.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[36454]: 5.1a scrub ok 2024-09-21T11:58:23.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[36454]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:23.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/859289443' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:23.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:23.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2788532692' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:58:23.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:23 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:58:23.842 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:23 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:23.842 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:23 smithi175 ceph-mon[32613]: 11.3 scrub ok 2024-09-21T11:58:23.842 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:23 smithi175 ceph-mon[32613]: pgmap v898: 158 pgs: 1 active+remapped+backfilling, 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 153 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.12k op/s; 25057/336969 objects misplaced (7.436%); 22 MiB/s, 353 objects/s recovering 2024-09-21T11:58:23.842 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:23 smithi175 ceph-mon[32613]: 5.1a scrub starts 2024-09-21T11:58:23.842 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:23 smithi175 ceph-mon[32613]: 5.1a scrub ok 2024-09-21T11:58:23.842 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:23 smithi175 ceph-mon[32613]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:23.842 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/859289443' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:23.842 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:23 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:23.843 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:23 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2788532692' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:58:23.843 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:23 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T11:58:23.843 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:23.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:23.844 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:23.945 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:23.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:23.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:24.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:24.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:24.109 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:24.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:24.122 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:58:24.122 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:58:24.123 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:58:24.123 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:58:24.123 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.123 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:166707 9.6 9:627f9972:::benchmark_data_smithi043_126002_object39641:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:58:24.123 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:18.109124+0000", 2024-09-21T11:58:24.123 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 426.00950510899997, 2024-09-21T11:58:24.124 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18514305, 2024-09-21T11:58:24.124 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.124 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.124 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.124 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:58:24.125 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:58:24.125 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 166707 2024-09-21T11:58:24.125 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.125 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.125 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.125 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.125 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109124+0000", 2024-09-21T11:58:24.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.126 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.126 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.126 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.126 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109124+0000", 2024-09-21T11:58:24.126 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.127 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.127 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109125+0000", 2024-09-21T11:58:24.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.362e-06 2024-09-21T11:58:24.127 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109130+0000", 2024-09-21T11:58:24.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3789999999999999e-06 2024-09-21T11:58:24.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109140+0000", 2024-09-21T11:58:24.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9739999999999997e-06 2024-09-21T11:58:24.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109147+0000", 2024-09-21T11:58:24.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2289999999999997e-06 2024-09-21T11:58:24.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109198+0000", 2024-09-21T11:58:24.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1131999999999998e-05 2024-09-21T11:58:24.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109305+0000", 2024-09-21T11:58:24.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010662099999999999 2024-09-21T11:58:24.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,2", 2024-09-21T11:58:24.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.109527+0000", 2024-09-21T11:58:24.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00022260199999999999 2024-09-21T11:58:24.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.112844+0000", 2024-09-21T11:58:24.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.003316608 2024-09-21T11:58:24.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.113270+0000", 2024-09-21T11:58:24.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00042589800000000002 2024-09-21T11:58:24.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.134 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.134 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.294193+0000", 2024-09-21T11:58:24.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.180923054 2024-09-21T11:58:24.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.294233+0000", 2024-09-21T11:58:24.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9733000000000001e-05 2024-09-21T11:58:24.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:18.294267+0000", 2024-09-21T11:58:24.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4458000000000001e-05 2024-09-21T11:58:24.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.136 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.136 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.137 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211066 9.a 9:51f636b8:::benchmark_data_smithi043_126002_object84000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:58:24.137 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:58:24.137 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 388.70665738700001, 2024-09-21T11:58:24.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.30144848499999999, 2024-09-21T11:58:24.137 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.138 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.138 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.138 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.138 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:58:24.138 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:58:24.139 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211066 2024-09-21T11:58:24.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.139 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:58:24.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.140 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.140 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:58:24.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:58:24.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T11:58:24.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411975+0000", 2024-09-21T11:58:24.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.401e-06 2024-09-21T11:58:24.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411979+0000", 2024-09-21T11:58:24.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4889999999999997e-06 2024-09-21T11:58:24.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411984+0000", 2024-09-21T11:58:24.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2530000000000004e-06 2024-09-21T11:58:24.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412132+0000", 2024-09-21T11:58:24.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014723399999999999 2024-09-21T11:58:24.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412228+0000", 2024-09-21T11:58:24.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6713999999999996e-05 2024-09-21T11:58:24.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,7", 2024-09-21T11:58:24.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412367+0000", 2024-09-21T11:58:24.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013867400000000001 2024-09-21T11:58:24.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415401+0000", 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030337630000000001 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.421153+0000", 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057522909999999997 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.713334+0000", 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29218145600000001 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.713381+0000", 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6492000000000001e-05 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.713420+0000", 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9387000000000001e-05 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.147 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67189 10.6 10:65d25444:::benchmark_data_smithi043_150678_object67188:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.338348+0000", 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.78028073799999, 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.210603178, 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.148 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.149 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.149 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67189 2024-09-21T11:58:24.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.149 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338348+0000", 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338348+0000", 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338350+0000", 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4670000000000001e-06 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338456+0000", 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010598399999999999 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338466+0000", 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0482999999999999e-05 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338473+0000", 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6810000000000001e-06 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338544+0000", 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0992000000000001e-05 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338681+0000", 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000137059 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.338828+0000", 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146582 2024-09-21T11:58:24.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.340772+0000", 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0019443959999999999 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.341306+0000", 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00053431300000000002 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.548824+0000", 2024-09-21T11:58:24.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.207518006 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.548901+0000", 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6970000000000003e-05 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.548952+0000", 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0244999999999998e-05 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.155 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67201 10.6 10:64739ad8:::benchmark_data_smithi043_150678_object67200:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:58:24.155 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.347253+0000", 2024-09-21T11:58:24.155 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.77137626799998, 2024-09-21T11:58:24.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.20183933500000001, 2024-09-21T11:58:24.155 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.155 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.155 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.155 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.155 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67201 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347253+0000", 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347253+0000", 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347255+0000", 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.835e-06 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347359+0000", 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104629 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347374+0000", 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4418e-05 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347385+0000", 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0807e-05 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347457+0000", 2024-09-21T11:58:24.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2028999999999998e-05 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347597+0000", 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013995899999999999 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.347765+0000", 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016854899999999999 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.350194+0000", 2024-09-21T11:58:24.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002429198 2024-09-21T11:58:24.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.350953+0000", 2024-09-21T11:58:24.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00075869499999999999 2024-09-21T11:58:24.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.548992+0000", 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.198038465 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549053+0000", 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1884999999999994e-05 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549092+0000", 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8865999999999997e-05 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67203 10.6 10:6478c686:::benchmark_data_smithi043_150678_object67202:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.351169+0000", 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.767460578, 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19805109200000001, 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.162 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67203 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351169+0000", 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351169+0000", 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351170+0000", 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5770000000000001e-06 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351238+0000", 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7508999999999993e-05 2024-09-21T11:58:24.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351249+0000", 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1793e-05 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351257+0000", 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3409999999999997e-06 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351300+0000", 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3575000000000002e-05 2024-09-21T11:58:24.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351396+0000", 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5711000000000006e-05 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.351484+0000", 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8200000000000003e-05 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.353164+0000", 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001679435 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.353614+0000", 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00044980699999999998 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549142+0000", 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19552818499999999 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.167 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549185+0000", 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3698000000000001e-05 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549220+0000", 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4261000000000003e-05 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.168 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67206 10.6 10:650072e7:::benchmark_data_smithi043_150678_object67205:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:28.356129+0000", 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.762500106, 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19332545400000001, 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67206 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356129+0000", 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356129+0000", 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356130+0000", 2024-09-21T11:58:24.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2440000000000001e-06 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356159+0000", 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.8768e-05 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356170+0000", 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.111e-05 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356176+0000", 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.305e-06 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356217+0000", 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1125999999999999e-05 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356308+0000", 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.1793999999999996e-05 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.356392+0000", 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3438999999999999e-05 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.357662+0000", 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012698469999999999 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.358436+0000", 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00077422400000000001 2024-09-21T11:58:24.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549312+0000", 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.19087628200000001 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549413+0000", 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101264 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:28.549455+0000", 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1050999999999997e-05 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67834 10.6 10:6b4bd2cf:::benchmark_data_smithi043_150678_object67833:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.042578+0000", 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.07605104499999, 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28913464900000002, 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.175 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67834 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042578+0000", 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042578+0000", 2024-09-21T11:58:24.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042580+0000", 2024-09-21T11:58:24.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.443e-06 2024-09-21T11:58:24.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042613+0000", 2024-09-21T11:58:24.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3355000000000002e-05 2024-09-21T11:58:24.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.178 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042623+0000", 2024-09-21T11:58:24.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8290000000000003e-06 2024-09-21T11:58:24.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042630+0000", 2024-09-21T11:58:24.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7519999999999999e-06 2024-09-21T11:58:24.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042846+0000", 2024-09-21T11:58:24.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021617899999999999 2024-09-21T11:58:24.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042970+0000", 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123807 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043305+0000", 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033573499999999998 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331521+0000", 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28821593200000001 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331551+0000", 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9621999999999999e-05 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331587+0000", 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6513000000000001e-05 2024-09-21T11:58:24.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331664+0000", 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6893999999999999e-05 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331713+0000", 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8588000000000003e-05 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.183 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67836 10.6 10:654cbc8c:::benchmark_data_smithi043_150678_object67835:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.043478+0000", 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.07515155599998, 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28847367400000001, 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.184 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67836 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043478+0000", 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043478+0000", 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043479+0000", 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0780000000000001e-06 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043547+0000", 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7952000000000003e-05 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043557+0000", 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0547000000000001e-05 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043564+0000", 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0550000000000002e-06 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046156+0000", 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002592123 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046282+0000", 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125747 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046402+0000", 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012027999999999999 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331787+0000", 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28538471399999998 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331797+0000", 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6050000000000004e-06 2024-09-21T11:58:24.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331831+0000", 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4458000000000001e-05 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331911+0000", 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9338999999999994e-05 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331951+0000", 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0775999999999997e-05 2024-09-21T11:58:24.189 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:67834 10.6 e331/313 ondisk, result = 0)", 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046066+0000", 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.07256298999999, 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28548575300000001, 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046066+0000", 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046066+0000", 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046067+0000", 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0210000000000001e-06 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046072+0000", 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046082+0000", 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3449999999999995e-06 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046089+0000", 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0740000000000004e-06 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331506+0000", 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28541750199999999 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331515+0000", 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8950000000000004e-06 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331552+0000", 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6764000000000002e-05 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:67834 10.6 e331/313 ondisk, result = 0)", 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046496+0000", 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.07213277699998, 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.285227234, 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:58:24.194 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046496+0000", 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046496+0000", 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046497+0000", 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0299999999999999e-07 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046501+0000", 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2359999999999998e-06 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046509+0000", 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9719999999999992e-06 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046516+0000", 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3679999999999998e-06 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331575+0000", 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28505966199999999 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331582+0000", 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2790000000000004e-06 2024-09-21T11:58:24.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331724+0000", 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014211399999999999 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67837 10.a e331/313 10:5163063c:::benchmark_data_smithi043_150678_object67836:head v 331'4187, mlcod=331'4185)", 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046734+0000", 2024-09-21T11:58:24.198 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.07189469899998, 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28267978599999999, 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046734+0000", 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046734+0000", 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046735+0000", 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7300000000000005e-07 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046777+0000", 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1236999999999998e-05 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046786+0000", 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6069999999999997e-06 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046793+0000", 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3990000000000001e-06 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046836+0000", 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2938999999999997e-05 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046853+0000", 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7424000000000001e-05 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329365+0000", 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28251246899999999 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329414+0000", 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8838000000000002e-05 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.202 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67842 10.7 e331/313 10:e64204f5:::benchmark_data_smithi043_150678_object67841:head v 331'4261, mlcod=331'4259)", 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.071180459, 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28956349100000001, 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:58:24.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047488+0000", 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8732000000000001e-05 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047497+0000", 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8599999999999999e-06 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047503+0000", 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7289999999999997e-06 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047550+0000", 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7571000000000002e-05 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329323+0000", 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28177280300000002 2024-09-21T11:58:24.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.336914+0000", 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0075908069999999998 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337012+0000", 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8308999999999997e-05 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67844 10.3 e331/313 10:c750a974:::benchmark_data_smithi043_150678_object67843:head v 331'4259, mlcod=331'4257)", 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.047834+0000", 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.07079514499998, 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29078393699999999, 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047834+0000", 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.208 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.209 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.209 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047834+0000", 2024-09-21T11:58:24.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047835+0000", 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9999999999999996e-07 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047887+0000", 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1690999999999997e-05 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329326+0000", 2024-09-21T11:58:24.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28143986999999998 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329343+0000", 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6688999999999999e-05 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329400+0000", 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6820999999999997e-05 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329424+0000", 2024-09-21T11:58:24.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3693000000000002e-05 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.338574+0000", 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091507940000000003 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.338618+0000", 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3578999999999997e-05 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67847 10.6 10:65456b12:::benchmark_data_smithi043_150678_object67846:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.052219+0000", 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.06641021500002, 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28538154100000002, 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.213 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67847 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052219+0000", 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052219+0000", 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052221+0000", 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.108e-06 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052286+0000", 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4874000000000003e-05 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329342+0000", 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27705603200000001 2024-09-21T11:58:24.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329356+0000", 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4419e-05 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331988+0000", 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026312319999999998 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.332118+0000", 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012998 2024-09-21T11:58:24.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.332241+0000", 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012306900000000001 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337426+0000", 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005185449 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337464+0000", 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7957999999999998e-05 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337515+0000", 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1017000000000003e-05 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337562+0000", 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7200999999999998e-05 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337601+0000", 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8201999999999998e-05 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:67836 10.6 e331/313 ondisk, result = 0)", 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.053932+0000", 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 295.06469728799999, 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27786533400000002, 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:58:24.219 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053932+0000", 2024-09-21T11:58:24.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053932+0000", 2024-09-21T11:58:24.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053933+0000", 2024-09-21T11:58:24.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.381e-06 2024-09-21T11:58:24.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053940+0000", 2024-09-21T11:58:24.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1810000000000004e-06 2024-09-21T11:58:24.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329343+0000", 2024-09-21T11:58:24.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27540220300000001 2024-09-21T11:58:24.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329355+0000", 2024-09-21T11:58:24.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2237e-05 2024-09-21T11:58:24.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331776+0000", 2024-09-21T11:58:24.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024212000000000001 2024-09-21T11:58:24.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331783+0000", 2024-09-21T11:58:24.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5429999999999999e-06 2024-09-21T11:58:24.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331797+0000", 2024-09-21T11:58:24.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4589e-05 2024-09-21T11:58:24.226 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.226 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.226 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.227 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72213 10.6 10:6e0a855e:::benchmark_data_smithi043_150678_object72212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:58:24.227 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.969718+0000", 2024-09-21T11:58:24.227 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 291.14891095799999, 2024-09-21T11:58:24.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23677020700000001, 2024-09-21T11:58:24.227 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.227 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.228 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.228 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72213 2024-09-21T11:58:24.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.229 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969718+0000", 2024-09-21T11:58:24.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969718+0000", 2024-09-21T11:58:24.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969720+0000", 2024-09-21T11:58:24.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.522e-06 2024-09-21T11:58:24.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969793+0000", 2024-09-21T11:58:24.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3422000000000003e-05 2024-09-21T11:58:24.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969810+0000", 2024-09-21T11:58:24.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7014e-05 2024-09-21T11:58:24.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969819+0000", 2024-09-21T11:58:24.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.653e-06 2024-09-21T11:58:24.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969874+0000", 2024-09-21T11:58:24.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5568000000000001e-05 2024-09-21T11:58:24.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970035+0000", 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016056600000000001 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970175+0000", 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014015300000000001 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.972425+0000", 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022500749999999998 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.973522+0000", 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010972670000000001 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206358+0000", 2024-09-21T11:58:24.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23283527300000001 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206438+0000", 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9804999999999999e-05 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206488+0000", 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0889e-05 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:85555 10.9 10:9e997645:::benchmark_data_smithi043_150678_object85554:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.837759+0000", 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 279.28086991100002, 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24446227800000001, 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 85555 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837759+0000", 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837759+0000", 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837761+0000", 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1500000000000002e-06 2024-09-21T11:58:24.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837839+0000", 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7088999999999993e-05 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837852+0000", 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3409e-05 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837859+0000", 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.635e-06 2024-09-21T11:58:24.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837907+0000", 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8232000000000002e-05 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.838017+0000", 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011044000000000001 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.838100+0000", 2024-09-21T11:58:24.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3128999999999996e-05 2024-09-21T11:58:24.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.840553+0000", 2024-09-21T11:58:24.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024521650000000001 2024-09-21T11:58:24.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.840749+0000", 2024-09-21T11:58:24.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019673600000000001 2024-09-21T11:58:24.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082065+0000", 2024-09-21T11:58:24.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.241316108 2024-09-21T11:58:24.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082158+0000", 2024-09-21T11:58:24.244 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2916000000000003e-05 2024-09-21T11:58:24.244 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.245 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082222+0000", 2024-09-21T11:58:24.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3268999999999993e-05 2024-09-21T11:58:24.246 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.246 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.246 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.247 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:85563 10.9 10:97d65cde:::benchmark_data_smithi043_150678_object85562:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:58:24.247 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.847840+0000", 2024-09-21T11:58:24.247 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 279.27078883299998, 2024-09-21T11:58:24.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.234536209, 2024-09-21T11:58:24.248 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.248 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.248 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.248 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.248 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 85563 2024-09-21T11:58:24.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.249 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847840+0000", 2024-09-21T11:58:24.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847840+0000", 2024-09-21T11:58:24.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847842+0000", 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.482e-06 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847894+0000", 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2577999999999997e-05 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847905+0000", 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0764000000000001e-05 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847911+0000", 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7960000000000001e-06 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847964+0000", 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3365000000000002e-05 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848068+0000", 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103872 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848154+0000", 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6026000000000006e-05 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.851858+0000", 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0037039949999999999 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.851882+0000", 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3261999999999999e-05 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082261+0000", 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23037903200000001 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082340+0000", 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9379e-05 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082377+0000", 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6658e-05 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:85564 10.9 10:98f1ca55:::benchmark_data_smithi043_150678_object85563:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.848489+0000", 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 279.27014031599998, 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24391052699999999, 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.255 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 85564 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848489+0000", 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848489+0000", 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848490+0000", 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8100000000000002e-07 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848561+0000", 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1595999999999997e-05 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848569+0000", 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6760000000000007e-06 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848573+0000", 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4850000000000003e-06 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.850273+0000", 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001699952 2024-09-21T11:58:24.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.850369+0000", 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5712999999999997e-05 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.850456+0000", 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7298000000000003e-05 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.851904+0000", 2024-09-21T11:58:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001447439 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.852025+0000", 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000121432 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.092269+0000", 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24024353500000001 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.092357+0000", 2024-09-21T11:58:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8367000000000003e-05 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.092399+0000", 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2252999999999997e-05 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.26339.0:59244 11.a e402/392 11:559102be:::benchmark_data_smithi043_178089_object59243:head v 402'3744, mlcod=402'3739)", 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:57:12.554380+0000", 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 71.564249372999996, 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.18227127100000001, 2024-09-21T11:58:24.261 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:57:12.554380+0000", 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:57:12.554380+0000", 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:57:12.554381+0000", 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.2600000000000002e-07 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:57:12.554430+0000", 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9767e-05 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.263 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:24.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:57:12.727515+0000", 2024-09-21T11:58:24.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.173085032 2024-09-21T11:58:24.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:24.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:57:12.727525+0000", 2024-09-21T11:58:24.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0054999999999999e-05 2024-09-21T11:58:24.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:24.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:57:12.728260+0000", 2024-09-21T11:58:24.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00073509499999999995 2024-09-21T11:58:24.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:24.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:57:12.728278+0000", 2024-09-21T11:58:24.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7295999999999998e-05 2024-09-21T11:58:24.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:58:24.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:57:12.736605+0000", 2024-09-21T11:58:24.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.008327648 2024-09-21T11:58:24.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:24.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:24.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:58:24.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:57:12.736651+0000", 2024-09-21T11:58:24.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.5652000000000003e-05 2024-09-21T11:58:24.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.267 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.267 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:24.267 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:24.267 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:58:24.267 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:24.297 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:24.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:24.298 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:24.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:24.440 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:24.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:24.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:24.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:24.584 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:24.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:24.585 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:24.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:24.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:24.717 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:24.717 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:24.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:24 smithi175 ceph-mon[32613]: 5.10 scrub starts 2024-09-21T11:58:24.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:24 smithi175 ceph-mon[32613]: 5.10 scrub ok 2024-09-21T11:58:24.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:24 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:58:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:24 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:58:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:24 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:58:24.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:24 smithi175 ceph-mon[32613]: osdmap e433: 8 total, 8 up, 8 in 2024-09-21T11:58:24.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:24.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:24.860 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:24.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:24.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[36454]: 5.10 scrub starts 2024-09-21T11:58:24.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[36454]: 5.10 scrub ok 2024-09-21T11:58:24.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:58:24.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:58:24.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:58:24.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[36454]: osdmap e433: 8 total, 8 up, 8 in 2024-09-21T11:58:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[29710]: 5.10 scrub starts 2024-09-21T11:58:24.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[29710]: 5.10 scrub ok 2024-09-21T11:58:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:58:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:58:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T11:58:24.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:24 smithi043 ceph-mon[29710]: osdmap e433: 8 total, 8 up, 8 in 2024-09-21T11:58:25.004 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:25.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:25.005 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:25.106 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:25.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:25.146 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:25.147 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:25.153 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:58:25.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 6 1 2024-09-21T11:58:25.239 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:25.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:25.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:25.278 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:25.279 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:25.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:25.407 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:25.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:25.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:25.434 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:58:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:25.509 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:25.549 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:25.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:25.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:25.551 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:25.651 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:25.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:25.682 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:25.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:25.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[36454]: pgmap v900: 158 pgs: 1 active+remapped+backfilling, 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 153 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 572 op/s; 25057/336969 objects misplaced (7.436%); 13 MiB/s, 215 objects/s recovering 2024-09-21T11:58:25.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[36454]: 6.19 scrub starts 2024-09-21T11:58:25.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[36454]: 6.19 scrub ok 2024-09-21T11:58:25.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[36454]: 5.13 scrub starts 2024-09-21T11:58:25.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[36454]: 5.13 scrub ok 2024-09-21T11:58:25.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:58:25.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:58:25.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[29710]: pgmap v900: 158 pgs: 1 active+remapped+backfilling, 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 153 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 572 op/s; 25057/336969 objects misplaced (7.436%); 13 MiB/s, 215 objects/s recovering 2024-09-21T11:58:25.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[29710]: 6.19 scrub starts 2024-09-21T11:58:25.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[29710]: 6.19 scrub ok 2024-09-21T11:58:25.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[29710]: 5.13 scrub starts 2024-09-21T11:58:25.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[29710]: 5.13 scrub ok 2024-09-21T11:58:25.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:58:25.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:25 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:58:25.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:25.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:25.833 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:25.833 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:25.899 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:25 smithi175 ceph-mon[32613]: pgmap v900: 158 pgs: 1 active+remapped+backfilling, 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 153 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 572 op/s; 25057/336969 objects misplaced (7.436%); 13 MiB/s, 215 objects/s recovering 2024-09-21T11:58:25.899 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:25 smithi175 ceph-mon[32613]: 6.19 scrub starts 2024-09-21T11:58:25.899 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:25 smithi175 ceph-mon[32613]: 6.19 scrub ok 2024-09-21T11:58:25.899 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:25 smithi175 ceph-mon[32613]: 5.13 scrub starts 2024-09-21T11:58:25.899 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:25 smithi175 ceph-mon[32613]: 5.13 scrub ok 2024-09-21T11:58:25.899 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:25 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:58:25.899 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:25 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:58:25.934 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:25.945 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:25.957 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:58:25.957 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:58:25.961 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:25.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:25.962 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:26.001 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:58:26.001 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:58:26.045 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:58:26.045 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:58:26.063 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:26.085 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:58:26.085 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:58:26.091 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:26.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:26.094 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:26.118 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:58:26.118 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:58:26.153 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:58:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:58:26.184 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:58:26.184 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:58:26.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.184 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.26339.0:135348 11.3 11:c4f52020:::benchmark_data_smithi043_178089_object14113:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e434)", 2024-09-21T11:58:26.184 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:58:26.170143+0000", 2024-09-21T11:58:26.184 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.012178392, 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.012199514999999999, 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.26339", 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 135348 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.170143+0000", 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:26.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.170143+0000", 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.170144+0000", 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.316e-06 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.170148+0000", 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.968e-06 2024-09-21T11:58:26.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.170158+0000", 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9299999999999998e-06 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.170164+0000", 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2720000000000003e-06 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:58:26.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.170213+0000", 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9175999999999998e-05 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.170288+0000", 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.4988999999999996e-05 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,7", 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.170353+0000", 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5079999999999994e-05 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:58:26.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.172835+0000", 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024814479999999998 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:58:26.173597+0000", 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00076230999999999998 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 1 2024-09-21T11:58:26.189 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:58:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:58:26.192 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:58:26.195 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:26.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:26.226 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:26.226 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:26.227 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:58:26.227 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:58:26.327 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:26.352 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:26.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:26.353 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:26.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:26.491 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:26.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:26.492 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:26.592 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:26.618 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:26.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:26.619 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:26.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:26.749 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:26.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:26.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:26.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:58:26.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:26.917 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:26.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:26.918 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:26.968 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:26 smithi175 ceph-mon[32613]: osdmap e434: 8 total, 8 up, 8 in 2024-09-21T11:58:26.968 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:26 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2807479073' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:26.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:26 smithi043 ceph-mon[29710]: osdmap e434: 8 total, 8 up, 8 in 2024-09-21T11:58:26.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:26 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2807479073' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:26.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:26 smithi043 ceph-mon[36454]: osdmap e434: 8 total, 8 up, 8 in 2024-09-21T11:58:26.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:26 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2807479073' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:27.004 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T11:58:27.020 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:27.080 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:27.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:27.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:27.182 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:27.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:27.219 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:27.220 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:27.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:27.375 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:27.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:27.376 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:27.387 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:27.477 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:27.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:27.534 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:27.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:27.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:27.677 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:27.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:27.678 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:27 smithi175 ceph-mon[32613]: pgmap v902: 158 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 154 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 669 B/s rd, 0 B/s wr, 758 op/s; 24738/331023 objects misplaced (7.473%); 12 MiB/s, 188 objects/s recovering 2024-09-21T11:58:27.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:27 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:58:27.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:27.808 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:27.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:27.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:27.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:27.950 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:27.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:27.951 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:27 smithi043 ceph-mon[36454]: pgmap v902: 158 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 154 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 669 B/s rd, 0 B/s wr, 758 op/s; 24738/331023 objects misplaced (7.473%); 12 MiB/s, 188 objects/s recovering 2024-09-21T11:58:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:27 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:58:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:27 smithi043 ceph-mon[29710]: pgmap v902: 158 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 154 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 669 B/s rd, 0 B/s wr, 758 op/s; 24738/331023 objects misplaced (7.473%); 12 MiB/s, 188 objects/s recovering 2024-09-21T11:58:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:27 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:58:27.992 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:58:27.992 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:58:28.052 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:28.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:28.081 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:28.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:28.182 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:28.212 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:28.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:28.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:28.314 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:28.345 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:28.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:28.346 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:28.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:28.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:28.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:28.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:28.624 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:28.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:28.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:28.685 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T11:58:28.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:28.770 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:28.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:28.771 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:28.867 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:28.872 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:28.898 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:28.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:28.899 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:29.000 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:29.006 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:29.033 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:29.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:29.034 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:29.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:29.161 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:29.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:29.162 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:29.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:29.290 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:29.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:29.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:29.392 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:29.418 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:29.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:29.419 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:29.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:29.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:29.559 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:29.594 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:58:29.594 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:58:29.660 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:29.686 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:29.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:29.687 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: 5.b scrub starts 2024-09-21T11:58:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: 5.b scrub ok 2024-09-21T11:58:29.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: pgmap v903: 158 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 154 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.12k op/s; 23480/321516 objects misplaced (7.303%); 14 MiB/s, 231 objects/s recovering 2024-09-21T11:58:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: 5.4 scrub starts 2024-09-21T11:58:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: 5.4 scrub ok 2024-09-21T11:58:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: 4.13 scrub starts 2024-09-21T11:58:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: 4.13 scrub ok 2024-09-21T11:58:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:58:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:58:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:58:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1363818049' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:58:29.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:58:29.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: 5.b scrub starts 2024-09-21T11:58:29.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: 5.b scrub ok 2024-09-21T11:58:29.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: pgmap v903: 158 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 154 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.12k op/s; 23480/321516 objects misplaced (7.303%); 14 MiB/s, 231 objects/s recovering 2024-09-21T11:58:29.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: 5.4 scrub starts 2024-09-21T11:58:29.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: 5.4 scrub ok 2024-09-21T11:58:29.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: 4.13 scrub starts 2024-09-21T11:58:29.743 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: 4.13 scrub ok 2024-09-21T11:58:29.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:58:29.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:58:29.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:58:29.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1363818049' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:58:29.744 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:29 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:58:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: 5.b scrub starts 2024-09-21T11:58:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: 5.b scrub ok 2024-09-21T11:58:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: pgmap v903: 158 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 154 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.12k op/s; 23480/321516 objects misplaced (7.303%); 14 MiB/s, 231 objects/s recovering 2024-09-21T11:58:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: 5.4 scrub starts 2024-09-21T11:58:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: 5.4 scrub ok 2024-09-21T11:58:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: 4.13 scrub starts 2024-09-21T11:58:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: 4.13 scrub ok 2024-09-21T11:58:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:58:29.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:58:29.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:58:29.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1363818049' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:58:29.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:29 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T11:58:29.763 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T11:58:29.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:29.833 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:29.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:29.834 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:29.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:29.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:29.966 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:29.966 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:30.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:30.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:30.101 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:30.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:30.202 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:30.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:30.246 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:30.347 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:30.379 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:30.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:30.380 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:30.432 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T11:58:30.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:30.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:58:30.520 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:30.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:30.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:30.622 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:30.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 7 1 2024-09-21T11:58:30.653 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:30.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:30.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:30.754 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:30.764 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:58:30.784 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:30.787 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:30.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:30.788 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:30.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:30.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:30.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[36454]: 4.a scrub starts 2024-09-21T11:58:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[36454]: 4.a scrub ok 2024-09-21T11:58:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:58:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[36454]: osdmap e435: 8 total, 8 up, 8 in 2024-09-21T11:58:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[36454]: 5.f scrub starts 2024-09-21T11:58:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[36454]: 5.f scrub ok 2024-09-21T11:58:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2807809865' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:58:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[29710]: 4.a scrub starts 2024-09-21T11:58:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[29710]: 4.a scrub ok 2024-09-21T11:58:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:58:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[29710]: osdmap e435: 8 total, 8 up, 8 in 2024-09-21T11:58:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[29710]: 5.f scrub starts 2024-09-21T11:58:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[29710]: 5.f scrub ok 2024-09-21T11:58:30.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:30 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2807809865' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:58:31.052 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:31.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:30 smithi175 ceph-mon[32613]: 4.a scrub starts 2024-09-21T11:58:31.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:30 smithi175 ceph-mon[32613]: 4.a scrub ok 2024-09-21T11:58:31.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:30 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T11:58:31.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:30 smithi175 ceph-mon[32613]: osdmap e435: 8 total, 8 up, 8 in 2024-09-21T11:58:31.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:30 smithi175 ceph-mon[32613]: 5.f scrub starts 2024-09-21T11:58:31.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:30 smithi175 ceph-mon[32613]: 5.f scrub ok 2024-09-21T11:58:31.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:30 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2807809865' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:58:31.082 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:31.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:31.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:31.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:31.228 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:31.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:31.229 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:31.330 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:31.360 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:31.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:31.361 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:31.462 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:31.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:31.489 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:31.489 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:31.590 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:31.613 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:31.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:31.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:31.625 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:31.692 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:58:31.693 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:58:31.716 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:31.742 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:31.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:31.743 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:31.830 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[36454]: pgmap v905: 158 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 154 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 776 B/s rd, 0 B/s wr, 781 op/s; 23480/321516 objects misplaced (7.303%); 4.0 MiB/s, 63 objects/s recovering 2024-09-21T11:58:31.830 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[36454]: 6.5 scrub starts 2024-09-21T11:58:31.830 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[36454]: 6.5 scrub ok 2024-09-21T11:58:31.831 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[36454]: 5.18 scrub starts 2024-09-21T11:58:31.831 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[36454]: 5.18 scrub ok 2024-09-21T11:58:31.831 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:31.831 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2807809865' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:58:31.831 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[36454]: osdmap e436: 8 total, 8 up, 8 in 2024-09-21T11:58:31.831 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[29710]: pgmap v905: 158 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 154 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 776 B/s rd, 0 B/s wr, 781 op/s; 23480/321516 objects misplaced (7.303%); 4.0 MiB/s, 63 objects/s recovering 2024-09-21T11:58:31.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[29710]: 6.5 scrub starts 2024-09-21T11:58:31.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[29710]: 6.5 scrub ok 2024-09-21T11:58:31.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[29710]: 5.18 scrub starts 2024-09-21T11:58:31.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[29710]: 5.18 scrub ok 2024-09-21T11:58:31.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:31.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2807809865' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:58:31.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:31 smithi043 ceph-mon[29710]: osdmap e436: 8 total, 8 up, 8 in 2024-09-21T11:58:31.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:31.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:31.886 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:31.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:31.987 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:32.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:31 smithi175 ceph-mon[32613]: pgmap v905: 158 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 154 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 776 B/s rd, 0 B/s wr, 781 op/s; 23480/321516 objects misplaced (7.303%); 4.0 MiB/s, 63 objects/s recovering 2024-09-21T11:58:32.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:31 smithi175 ceph-mon[32613]: 6.5 scrub starts 2024-09-21T11:58:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:31 smithi175 ceph-mon[32613]: 6.5 scrub ok 2024-09-21T11:58:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:31 smithi175 ceph-mon[32613]: 5.18 scrub starts 2024-09-21T11:58:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:31 smithi175 ceph-mon[32613]: 5.18 scrub ok 2024-09-21T11:58:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:31 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:31 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2807809865' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:58:32.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:31 smithi175 ceph-mon[32613]: osdmap e436: 8 total, 8 up, 8 in 2024-09-21T11:58:32.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:58:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:58:32.016 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:32.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:32.017 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:32.118 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:32.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:32.156 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:32.256 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:32.280 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:32.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:32.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:32.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:32.421 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:32.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:32.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:32.425 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T11:58:32.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:32.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:32.561 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:32.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:32.702 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:32.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:32.703 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:32.742 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:32.804 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:32.830 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:32.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:32.831 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:32.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:32.953 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:32.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:32.954 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:33.032 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:33.032 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:32 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1639984302' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:58:33.032 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:32 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:58:33.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:33.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:33.092 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:33.092 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:33.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:33.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:33.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:33.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:33.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:32 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1639984302' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:58:33.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:32 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:58:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:32 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1639984302' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:58:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:32 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T11:58:33.298 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:58:33.298 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:58:33.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:33.373 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:33.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:33.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:33.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:33.512 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:33.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:33.513 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:33.585 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T11:58:33.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:58:33.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:33.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:33.684 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:33.684 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:33.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:33.839 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:33.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:33.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:33.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:33.982 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:33.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:33.983 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:33.984 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T11:58:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:33 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:33 smithi043 ceph-mon[29710]: pgmap v907: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.53k op/s; 21979/306735 objects misplaced (7.165%); 8.1 MiB/s, 129 objects/s recovering 2024-09-21T11:58:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:33 smithi043 ceph-mon[29710]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:33 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:58:33.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:33 smithi043 ceph-mon[29710]: osdmap e437: 8 total, 8 up, 8 in 2024-09-21T11:58:33.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:33 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:33.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:33 smithi043 ceph-mon[36454]: pgmap v907: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.53k op/s; 21979/306735 objects misplaced (7.165%); 8.1 MiB/s, 129 objects/s recovering 2024-09-21T11:58:33.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:33 smithi043 ceph-mon[36454]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:33.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:33 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:58:33.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:33 smithi043 ceph-mon[36454]: osdmap e437: 8 total, 8 up, 8 in 2024-09-21T11:58:34.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:34.109 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:34.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:34.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:34.210 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:34.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:34.247 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:34.247 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:34.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:33 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:34.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:33 smithi175 ceph-mon[32613]: pgmap v907: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.53k op/s; 21979/306735 objects misplaced (7.165%); 8.1 MiB/s, 129 objects/s recovering 2024-09-21T11:58:34.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:33 smithi175 ceph-mon[32613]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:33 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T11:58:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:33 smithi175 ceph-mon[32613]: osdmap e437: 8 total, 8 up, 8 in 2024-09-21T11:58:34.331 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:58:34.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:34.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:34.388 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:34.489 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:34.520 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:34.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:34.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:34.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:34.661 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:34.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:34.662 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:34.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:34.790 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:34.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:34.791 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:34.892 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:34.923 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:34.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:34.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:35.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:35.056 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:35.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:35.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:35.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:35.186 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:35.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:35.187 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:35.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:35.326 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:35.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:35.327 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:35.335 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:58:35.336 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:58:35.428 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:35.455 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:35.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:35.456 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:35.490 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:58:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:58:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:35.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:35.584 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:35.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:35.585 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:35.646 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:35.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:35.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:35.735 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:35.735 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:35.836 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:35.861 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:35.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:35.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:35.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:36.001 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:36.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:36.002 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:36.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:36.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:36.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:36.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:35 smithi043 ceph-mon[36454]: pgmap v909: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 821 op/s; 21979/306735 objects misplaced (7.165%); 4.5 MiB/s, 71 objects/s recovering 2024-09-21T11:58:36.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:35 smithi043 ceph-mon[29710]: pgmap v909: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 821 op/s; 21979/306735 objects misplaced (7.165%); 4.5 MiB/s, 71 objects/s recovering 2024-09-21T11:58:36.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:36.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:35 smithi175 ceph-mon[32613]: pgmap v909: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 821 op/s; 21979/306735 objects misplaced (7.165%); 4.5 MiB/s, 71 objects/s recovering 2024-09-21T11:58:36.271 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:36.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:36.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:36.327 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 1 1 2024-09-21T11:58:36.373 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:36.403 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:36.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:36.404 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:36.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T11:58:36.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:36.532 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:36.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:36.532 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:36.633 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:36.659 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:36.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:36.660 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:36.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:36.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:36.819 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:36.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:36.920 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:36.946 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:36.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:36.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:36.982 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:58:36.982 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:58:37.023 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:58:37.023 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:58:37.051 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:37.065 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:58:37.065 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:58:37.077 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:37.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:37.078 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:37.102 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:58:37.102 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:58:37.134 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:58:37.135 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:58:37.171 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:58:37.171 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:58:37.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:37.203 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:58:37.203 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:58:37.236 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:37.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:37.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:37.239 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:58:37.240 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:58:37.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:37.387 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:37.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:37.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:37.489 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:37.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:37.524 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:37.524 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:37.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:37.665 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:37.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:37.666 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:37.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:37.767 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:37.791 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:37.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:37.792 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:37.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:37.941 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:37.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:37.943 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:37.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:37 smithi043 ceph-mon[29710]: pgmap v910: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 797 B/s rd, 0 B/s wr, 1.04k op/s; 21128/301386 objects misplaced (7.010%); 9.9 MiB/s, 159 objects/s recovering 2024-09-21T11:58:37.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:37 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/196691176' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:37.972 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:37 smithi043 ceph-mon[29710]: osdmap e438: 8 total, 8 up, 8 in 2024-09-21T11:58:37.972 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:37 smithi043 ceph-mon[36454]: pgmap v910: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 797 B/s rd, 0 B/s wr, 1.04k op/s; 21128/301386 objects misplaced (7.010%); 9.9 MiB/s, 159 objects/s recovering 2024-09-21T11:58:37.972 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:37 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/196691176' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:37.972 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:37 smithi043 ceph-mon[36454]: osdmap e438: 8 total, 8 up, 8 in 2024-09-21T11:58:38.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:38.086 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:38.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:38.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:38.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:38.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:38.246 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:38.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:37 smithi175 ceph-mon[32613]: pgmap v910: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 797 B/s rd, 0 B/s wr, 1.04k op/s; 21128/301386 objects misplaced (7.010%); 9.9 MiB/s, 159 objects/s recovering 2024-09-21T11:58:38.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:37 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/196691176' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:38.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:37 smithi175 ceph-mon[32613]: osdmap e438: 8 total, 8 up, 8 in 2024-09-21T11:58:38.347 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:38.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:38.379 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:38.379 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:38.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:38.506 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:38.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:38.507 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:38.530 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:58:38.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:38.636 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:38.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:38.637 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:38.737 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:38.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:38.783 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:38.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:38.879 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:38.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:38.937 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:38.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:38.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:58:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:58:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[29710]: pgmap v912: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.32k op/s; 19874/292263 objects misplaced (6.800%); 13 MiB/s, 206 objects/s recovering 2024-09-21T11:58:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2980740955' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:38.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:58:38.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[29710]: osdmap e439: 8 total, 8 up, 8 in 2024-09-21T11:58:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:58:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:58:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[36454]: pgmap v912: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.32k op/s; 19874/292263 objects misplaced (6.800%); 13 MiB/s, 206 objects/s recovering 2024-09-21T11:58:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2980740955' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:58:38.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:38 smithi043 ceph-mon[36454]: osdmap e439: 8 total, 8 up, 8 in 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.26339.0:147644 11.6 e439/421 11:690843aa:::benchmark_data_smithi043_178089_object26409:head v 439'9280, mlcod=439'9278)", 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:58:38.825417+0000", 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0044862169999999998, 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0045125080000000001, 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.825417+0000", 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.825417+0000", 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.825418+0000", 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3430000000000001e-06 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.825426+0000", 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5140000000000004e-06 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.825443+0000", 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6671999999999999e-05 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.825451+0000", 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4270000000000008e-06 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.825688+0000", 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023651499999999999 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.825762+0000", 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4805e-05 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:39.006 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.26339.0:147637 11.9 e439/421 11:955b5020:::benchmark_data_smithi043_178089_object26402:head v 439'9338, mlcod=439'9336)", 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:58:38.826911+0000", 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0029924140000000001, 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030740939999999999, 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:39.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.826911+0000", 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.826911+0000", 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.826912+0000", 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.209e-06 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.826920+0000", 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1689999999999992e-06 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.826936+0000", 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6121999999999999e-05 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.826945+0000", 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1799999999999996e-06 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.827173+0000", 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00022815199999999999 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.827210+0000", 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7311000000000002e-05 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.26339.0:147653 11.6 e439/421 11:69b7fe7b:::benchmark_data_smithi043_178089_object26418:head v 439'9281, mlcod=439'9278)", 2024-09-21T11:58:39.010 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:58:38.828555+0000", 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0013478559999999999, 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001478095, 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.828555+0000", 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.828555+0000", 2024-09-21T11:58:39.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.828556+0000", 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4600000000000004e-07 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.828563+0000", 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6259999999999997e-06 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.828578+0000", 2024-09-21T11:58:39.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5273000000000001e-05 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.828585+0000", 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1960000000000002e-06 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.828798+0000", 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00021273200000000001 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:58:39.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:38.828823+0000", 2024-09-21T11:58:39.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.5474e-05 2024-09-21T11:58:39.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:39.014 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:58:39.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:39.014 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:58:39.014 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:58:39.014 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 3 2024-09-21T11:58:39.014 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:58:39.040 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:39.067 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:39.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:39.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:39.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:39.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:39.208 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:39.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:38 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:58:39.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:38 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:58:39.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:38 smithi175 ceph-mon[32613]: pgmap v912: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.32k op/s; 19874/292263 objects misplaced (6.800%); 13 MiB/s, 206 objects/s recovering 2024-09-21T11:58:39.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:38 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2980740955' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:39.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:38 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:58:39.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:38 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:39.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:38 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:58:39.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:38 smithi175 ceph-mon[32613]: osdmap e439: 8 total, 8 up, 8 in 2024-09-21T11:58:39.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:39.336 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:39.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:39.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:39.438 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:39.465 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:39.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:39.466 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:39.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:39.607 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:39.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:39.608 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:39.709 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:39.740 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:39.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:39.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:39.842 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:39.870 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:39.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:39.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:39.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:40.002 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:40.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:40.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:40.103 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:40.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T11:58:40.131 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:40.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:40.132 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:40.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:58:40.234 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:40.259 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:40.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:40.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:40.361 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:40.387 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:40.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:40.388 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:40.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:40.552 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:40.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:40.553 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:40.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:40.715 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:40.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:40.716 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:40.817 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:40.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:40.845 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:40.846 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:40.946 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:40.972 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:40.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:40.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:41.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:41.102 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:41.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:41.104 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:41.204 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:41.233 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:41.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:41.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:41.334 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:41.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:41.366 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:41.366 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:41.467 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:41 smithi043 ceph-mon[36454]: pgmap v914: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 691 B/s rd, 0 B/s wr, 652 op/s; 19874/292263 objects misplaced (6.800%); 9.3 MiB/s, 148 objects/s recovering 2024-09-21T11:58:41.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:41 smithi043 ceph-mon[29710]: pgmap v914: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 691 B/s rd, 0 B/s wr, 652 op/s; 19874/292263 objects misplaced (6.800%); 9.3 MiB/s, 148 objects/s recovering 2024-09-21T11:58:41.498 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:41.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:41.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:41.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:41 smithi175 ceph-mon[32613]: pgmap v914: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.9 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 691 B/s rd, 0 B/s wr, 652 op/s; 19874/292263 objects misplaced (6.800%); 9.3 MiB/s, 148 objects/s recovering 2024-09-21T11:58:41.600 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:41.628 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:41.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:41.629 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:41.730 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:41.756 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:41.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:41.758 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:41.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:41.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:41.901 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:42.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:42.041 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:42.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:42.042 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:42.143 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:42.164 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:42.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:42.165 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:42.260 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:58:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:58:42.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:42.292 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:42.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:42.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:42.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:42.419 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:42.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:42.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:42.521 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:42.550 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:42.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:42.552 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:42.652 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:42.677 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:42.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:42.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:42.778 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:42.813 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:42.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:42.815 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:42.915 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:42.938 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:42.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:42.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:43.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:43.083 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:43.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:43.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:43.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:43.216 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:43.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:43.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:43.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:43.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:43.358 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:43.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:43.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:43.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:43.504 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:43.505 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:43.605 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:43.632 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:43.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:43.633 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:43.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:43.765 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:43.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:43.766 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:43.867 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:43.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:43.905 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:43.906 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:43.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:43 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:43.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:43 smithi043 ceph-mon[29710]: pgmap v915: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s; 18128/277185 objects misplaced (6.540%); 15 MiB/s, 232 objects/s recovering 2024-09-21T11:58:43.994 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:43 smithi043 ceph-mon[29710]: osdmap e440: 8 total, 8 up, 8 in 2024-09-21T11:58:43.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:43 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:43.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:43 smithi043 ceph-mon[36454]: pgmap v915: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s; 18128/277185 objects misplaced (6.540%); 15 MiB/s, 232 objects/s recovering 2024-09-21T11:58:43.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:43 smithi043 ceph-mon[36454]: osdmap e440: 8 total, 8 up, 8 in 2024-09-21T11:58:44.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:44.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:43 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:44.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:43 smithi175 ceph-mon[32613]: pgmap v915: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s; 18128/277185 objects misplaced (6.540%); 15 MiB/s, 232 objects/s recovering 2024-09-21T11:58:44.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:43 smithi175 ceph-mon[32613]: osdmap e440: 8 total, 8 up, 8 in 2024-09-21T11:58:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:44.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:44.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:44.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:44.207 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:44.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:44.208 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:44.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:44.332 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:44.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:44.333 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:44.434 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:44.459 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:44.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:44.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:44.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:44.604 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:44.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:44.605 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:44.705 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:44.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:44.728 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:44.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:44.729 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:44.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:44.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:44.885 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:44.885 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:44.986 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:45.011 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:45.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:45.012 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:45.113 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:45.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:45.140 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:45.140 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:45.241 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:45.271 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:45.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:45.272 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:45.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:45.413 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:45.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:45.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:45.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:58:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:58:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:45.515 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:45.518 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T11:58:45.539 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:45.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:45.540 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:45.641 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:45.669 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:45.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:45.671 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:45.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:45.797 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:45.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:45.798 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:45.812 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:45 smithi175 ceph-mon[32613]: pgmap v917: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 755 op/s; 18128/277185 objects misplaced (6.540%); 7.1 MiB/s, 114 objects/s recovering 2024-09-21T11:58:45.812 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2155200099' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:58:45.812 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:45 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:58:45.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:45 smithi043 ceph-mon[29710]: pgmap v917: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 755 op/s; 18128/277185 objects misplaced (6.540%); 7.1 MiB/s, 114 objects/s recovering 2024-09-21T11:58:45.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2155200099' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:58:45.865 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:45 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:58:45.865 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:45 smithi043 ceph-mon[36454]: pgmap v917: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.6 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 755 op/s; 18128/277185 objects misplaced (6.540%); 7.1 MiB/s, 114 objects/s recovering 2024-09-21T11:58:45.866 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2155200099' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:58:45.866 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:45 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T11:58:45.899 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:45.927 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:45.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:45.928 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:46.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:46.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:46.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:46.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:46.199 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:46.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:46.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:46.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:58:46.301 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:46.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:46.331 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:46.331 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:46.432 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:46.474 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:46.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:46.475 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:46.543 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:46 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:58:46.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:46 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:58:46.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:46.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:46.622 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:46.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:46.723 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:46.752 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:46.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:46.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:46.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:46 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T11:58:46.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:46 smithi175 ceph-mon[32613]: osdmap e441: 8 total, 8 up, 8 in 2024-09-21T11:58:46.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:46.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:46.893 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:46.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:46.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:46 smithi043 ceph-mon[36454]: osdmap e441: 8 total, 8 up, 8 in 2024-09-21T11:58:46.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:46 smithi043 ceph-mon[29710]: osdmap e441: 8 total, 8 up, 8 in 2024-09-21T11:58:46.995 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:47.029 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:47.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:47.030 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:47.131 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:47.152 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:47.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:47.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:47.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:47.278 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:47.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:47.279 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:47.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:47.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:47.412 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:47.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:47.557 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:47.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:47.558 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:47.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:47.686 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:47.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:47.687 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:47 smithi175 ceph-mon[32613]: pgmap v919: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 669 B/s rd, 0 B/s wr, 890 op/s; 17376/271848 objects misplaced (6.392%); 11 MiB/s, 168 objects/s recovering 2024-09-21T11:58:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:47 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:58:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:47 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:58:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:47 smithi175 ceph-mon[32613]: osdmap e442: 8 total, 8 up, 8 in 2024-09-21T11:58:47.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:47 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:47.788 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:47.815 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:47.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:47.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:47.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:47 smithi043 ceph-mon[29710]: pgmap v919: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 669 B/s rd, 0 B/s wr, 890 op/s; 17376/271848 objects misplaced (6.392%); 11 MiB/s, 168 objects/s recovering 2024-09-21T11:58:47.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:47 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:58:47.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:47 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:58:47.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:47 smithi043 ceph-mon[29710]: osdmap e442: 8 total, 8 up, 8 in 2024-09-21T11:58:47.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:47 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:47.862 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:58:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:58:47.862 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:47 smithi043 ceph-mon[36454]: pgmap v919: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 669 B/s rd, 0 B/s wr, 890 op/s; 17376/271848 objects misplaced (6.392%); 11 MiB/s, 168 objects/s recovering 2024-09-21T11:58:47.862 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:47 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:58:47.862 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:47 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:58:47.862 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:47 smithi043 ceph-mon[36454]: osdmap e442: 8 total, 8 up, 8 in 2024-09-21T11:58:47.863 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:47 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:58:47.909 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:58:47.909 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:58:47.917 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:47.950 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:58:47.950 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:58:47.965 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:47.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:47.966 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:47.987 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:58:47.988 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:58:48.025 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:58:48.026 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:58:48.060 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:58:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:58:48.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:48.097 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:58:48.097 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:58:48.112 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:48.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:48.113 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:48.130 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:58:48.131 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:58:48.214 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:48.243 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:48.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:48.244 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:48.344 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:48.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:48.377 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:48.377 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:48.477 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:48.507 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:48.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:48.508 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:48.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:48.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:48.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:48.752 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:48.795 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:48.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:48.796 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:48.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:48.914 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:48 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1259664283' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:48.914 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:48 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1259664283' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:48.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:48.958 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:48.958 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:48.969 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T11:58:48.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:58:49.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:48 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1259664283' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:49.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:49.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:49.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:49.211 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:49.236 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:49.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:49.237 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:49.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:49.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:49.367 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:49.367 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:49.468 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:49.493 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:49.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:49.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:49.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:49.633 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:49.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:49.634 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:49.732 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:49 smithi043 ceph-mon[29710]: pgmap v921: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 777 op/s; 16250/263202 objects misplaced (6.174%); 10 MiB/s, 165 objects/s recovering 2024-09-21T11:58:49.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:49 smithi043 ceph-mon[36454]: pgmap v921: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 777 op/s; 16250/263202 objects misplaced (6.174%); 10 MiB/s, 165 objects/s recovering 2024-09-21T11:58:49.735 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:49.800 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:58:49.801 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:58:49.813 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:49.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:49.814 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:49.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:49.944 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:49.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:49.946 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:50.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:49 smithi175 ceph-mon[32613]: pgmap v921: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 777 op/s; 16250/263202 objects misplaced (6.174%); 10 MiB/s, 165 objects/s recovering 2024-09-21T11:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:50.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:50.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:50.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:50.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:50.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:50.245 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:50.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:50.388 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:50.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:50.389 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:50.490 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:50.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:50.523 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:50.524 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:50.593 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T11:58:50.624 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:50.651 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:50.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:50.652 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:50.753 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:50.782 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:50.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:50.783 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:50.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:50.921 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:50.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:50.922 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:51.023 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:51.050 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:51.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:51.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:51.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:51.206 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:51.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:51.209 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:51.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:51.337 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:51.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:51.338 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:51.438 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:51.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:51.464 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:51.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:51.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:58:51.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:51.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:51.594 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:51.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:51.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:51.718 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:51.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:51.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:51.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:51 smithi043 ceph-mon[29710]: pgmap v922: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 667 B/s rd, 0 B/s wr, 608 op/s; 16250/263202 objects misplaced (6.174%); 8.1 MiB/s, 129 objects/s recovering 2024-09-21T11:58:51.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:51 smithi043 ceph-mon[36454]: pgmap v922: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 667 B/s rd, 0 B/s wr, 608 op/s; 16250/263202 objects misplaced (6.174%); 8.1 MiB/s, 129 objects/s recovering 2024-09-21T11:58:51.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:51.829 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:58:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:58:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:51.858 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:51.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:51.859 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:51.903 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:51 smithi175 ceph-mon[32613]: pgmap v922: 158 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 154 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 667 B/s rd, 0 B/s wr, 608 op/s; 16250/263202 objects misplaced (6.174%); 8.1 MiB/s, 129 objects/s recovering 2024-09-21T11:58:51.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:52.021 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:52.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:52.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:52.123 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:52.152 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:52.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:52.153 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:52.254 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:52.260 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:58:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:58:52.280 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:52.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:52.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:52.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:52.420 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:52.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:52.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:52.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:52.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:52.566 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:52.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:58:52.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:52.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:52.703 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:52 smithi043 ceph-mon[36454]: from='client.16737 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:58:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:52 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2676123717' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:52 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:52 smithi043 ceph-mon[29710]: from='client.16737 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:58:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:52 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2676123717' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:52.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:52 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:52.804 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:52.836 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:52.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:52.837 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:52.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:52.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:52.975 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:53.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:52 smithi175 ceph-mon[32613]: from='client.16737 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:58:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:52 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2676123717' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:53.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:52 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:58:53.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:53.101 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:53.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:53.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:53.203 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:53.233 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:53.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:53.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:53.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:53.375 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:53.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:53.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:53.477 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:53.516 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:53.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:53.517 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:53.617 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:53.643 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:53.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:53.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:53.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:53.788 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:53.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:53.789 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:53.890 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:53.891 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:53.891 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[29710]: pgmap v923: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 14078/248979 objects misplaced (5.654%); 18 MiB/s, 293 objects/s recovering 2024-09-21T11:58:53.891 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[29710]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:53.891 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:53.891 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:58:53.891 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[29710]: osdmap e443: 8 total, 8 up, 8 in 2024-09-21T11:58:53.892 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:53.892 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[36454]: pgmap v923: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 14078/248979 objects misplaced (5.654%); 18 MiB/s, 293 objects/s recovering 2024-09-21T11:58:53.892 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[36454]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:53.892 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:53.892 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:58:53.892 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:53 smithi043 ceph-mon[36454]: osdmap e443: 8 total, 8 up, 8 in 2024-09-21T11:58:53.916 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:53.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:53.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:53.940 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":922,"stamp":"2024-09-21T11:58:50.177532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5737224755,"num_objects":87734,"num_object_clones":0,"num_object_copies":263202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16250,"num_objects_unfound":0,"num_objects_dirty":87734,"num_whiteouts":0,"num_read":2239,"num_read_kb":1991,"num_write":155653,"num_write_kb":7758894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7180,"num_bytes_recovered":447030147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82118,"ondisk_log_size":82118,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":478,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":22927080,"kb_used_data":17246024,"kb_used_omap":12,"kb_used_meta":5680371,"kb_avail":727034136,"statfs":{"total":767960285184,"available":744482955264,"internally_reserved":0,"allocated":17659928576,"data_stored":17654932909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":5816700466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,3,8,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-305463296,"num_objects":-4661,"num_object_clones":0,"num_object_copies":-13983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1878,"num_objects_unfound":0,"num_objects_dirty":-4661,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4661,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":996,"num_bytes_recovered":65273856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.666835"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":54,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.566695+0000","last_change":"2024-09-21T11:57:58.529638+0000","last_active":"2024-09-21T11:58:46.566695+0000","last_peered":"2024-09-21T11:58:46.566695+0000","last_clean":"2024-09-21T11:58:46.566695+0000","last_became_active":"2024-09-21T11:57:58.529245+0000","last_became_peered":"2024-09-21T11:57:58.529245+0000","last_unstale":"2024-09-21T11:58:46.566695+0000","last_undegraded":"2024-09-21T11:58:46.566695+0000","last_fullsized":"2024-09-21T11:58:46.566695+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:43.783030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.783030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:01.460062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":854,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532363+0000","last_change":"2024-09-21T11:57:37.936348+0000","last_active":"2024-09-21T11:58:45.532363+0000","last_peered":"2024-09-21T11:58:45.532363+0000","last_clean":"2024-09-21T11:58:45.532363+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:58:45.532363+0000","last_undegraded":"2024-09-21T11:58:45.532363+0000","last_fullsized":"2024-09-21T11:58:45.532363+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:37.936279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.936279+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:39.639770+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":534,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562240+0000","last_change":"2024-09-21T11:56:08.429983+0000","last_active":"2024-09-21T11:58:46.562240+0000","last_peered":"2024-09-21T11:58:46.562240+0000","last_clean":"2024-09-21T11:58:46.562240+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:58:46.562240+0000","last_undegraded":"2024-09-21T11:58:46.562240+0000","last_fullsized":"2024-09-21T11:58:46.562240+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:08.429912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.429912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":627,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515145+0000","last_change":"2024-09-21T11:58:21.827334+0000","last_active":"2024-09-21T11:58:45.515145+0000","last_peered":"2024-09-21T11:58:45.515145+0000","last_clean":"2024-09-21T11:58:45.515145+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:58:45.515145+0000","last_undegraded":"2024-09-21T11:58:45.515145+0000","last_fullsized":"2024-09-21T11:58:45.515145+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:21.827047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:21.827047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:43.889402+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":827,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531281+0000","last_change":"2024-09-21T11:56:49.693769+0000","last_active":"2024-09-21T11:58:45.531281+0000","last_peered":"2024-09-21T11:58:45.531281+0000","last_clean":"2024-09-21T11:58:45.531281+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:58:45.531281+0000","last_undegraded":"2024-09-21T11:58:45.531281+0000","last_fullsized":"2024-09-21T11:58:45.531281+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:49.693594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:49.693594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":817,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532062+0000","last_change":"2024-09-21T11:57:37.711838+0000","last_active":"2024-09-21T11:58:45.532062+0000","last_peered":"2024-09-21T11:58:45.532062+0000","last_clean":"2024-09-21T11:58:45.532062+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:58:45.532062+0000","last_undegraded":"2024-09-21T11:58:45.532062+0000","last_fullsized":"2024-09-21T11:58:45.532062+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:37.711758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.711758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:04.785508+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":862,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.536733+0000","last_change":"2024-09-21T11:56:31.572626+0000","last_active":"2024-09-21T11:58:45.536733+0000","last_peered":"2024-09-21T11:58:45.536733+0000","last_clean":"2024-09-21T11:58:45.536733+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:58:45.536733+0000","last_undegraded":"2024-09-21T11:58:45.536733+0000","last_fullsized":"2024-09-21T11:58:45.536733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:31.572525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:31.572525+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":677,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.534045+0000","last_change":"2024-09-21T11:58:30.584828+0000","last_active":"2024-09-21T11:58:45.534045+0000","last_peered":"2024-09-21T11:58:45.534045+0000","last_clean":"2024-09-21T11:58:45.534045+0000","last_became_active":"2024-09-21T11:58:17.527501+0000","last_became_peered":"2024-09-21T11:58:17.527501+0000","last_unstale":"2024-09-21T11:58:45.534045+0000","last_undegraded":"2024-09-21T11:58:45.534045+0000","last_fullsized":"2024-09-21T11:58:45.534045+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:30.584774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:30.584774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:50.421881+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":806,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515725+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:58:45.515725+0000","last_peered":"2024-09-21T11:58:45.515725+0000","last_clean":"2024-09-21T11:58:45.515725+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:58:45.515725+0000","last_undegraded":"2024-09-21T11:58:45.515725+0000","last_fullsized":"2024-09-21T11:58:45.515725+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.819967+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":854,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531632+0000","last_change":"2024-09-21T11:58:17.547443+0000","last_active":"2024-09-21T11:58:45.531632+0000","last_peered":"2024-09-21T11:58:45.531632+0000","last_clean":"2024-09-21T11:58:45.531632+0000","last_became_active":"2024-09-21T11:58:17.525792+0000","last_became_peered":"2024-09-21T11:58:17.525792+0000","last_unstale":"2024-09-21T11:58:45.531632+0000","last_undegraded":"2024-09-21T11:58:45.531632+0000","last_fullsized":"2024-09-21T11:58:45.531632+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:07.774892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:07.774892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":806,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515319+0000","last_change":"2024-09-21T11:55:05.096606+0000","last_active":"2024-09-21T11:58:45.515319+0000","last_peered":"2024-09-21T11:58:45.515319+0000","last_clean":"2024-09-21T11:58:45.515319+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:58:45.515319+0000","last_undegraded":"2024-09-21T11:58:45.515319+0000","last_fullsized":"2024-09-21T11:58:45.515319+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:05.096542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:05.096542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:57.793506+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":54,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.564817+0000","last_change":"2024-09-21T11:57:58.527130+0000","last_active":"2024-09-21T11:58:46.564817+0000","last_peered":"2024-09-21T11:58:46.564817+0000","last_clean":"2024-09-21T11:58:46.564817+0000","last_became_active":"2024-09-21T11:57:58.526820+0000","last_became_peered":"2024-09-21T11:57:58.526820+0000","last_unstale":"2024-09-21T11:58:46.564817+0000","last_undegraded":"2024-09-21T11:58:46.564817+0000","last_fullsized":"2024-09-21T11:58:46.564817+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:36.422726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:36.422726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":852,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515435+0000","last_change":"2024-09-21T11:56:02.994041+0000","last_active":"2024-09-21T11:58:45.515435+0000","last_peered":"2024-09-21T11:58:45.515435+0000","last_clean":"2024-09-21T11:58:45.515435+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:58:45.515435+0000","last_undegraded":"2024-09-21T11:58:45.515435+0000","last_fullsized":"2024-09-21T11:58:45.515435+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.993966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.993966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:35.823113+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":852,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531642+0000","last_change":"2024-09-21T11:58:17.549945+0000","last_active":"2024-09-21T11:58:45.531642+0000","last_peered":"2024-09-21T11:58:45.531642+0000","last_clean":"2024-09-21T11:58:45.531642+0000","last_became_active":"2024-09-21T11:58:17.525789+0000","last_became_peered":"2024-09-21T11:58:17.525789+0000","last_unstale":"2024-09-21T11:58:45.531642+0000","last_undegraded":"2024-09-21T11:58:45.531642+0000","last_fullsized":"2024-09-21T11:58:45.531642+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'7","last_scrub_stamp":"2024-09-21T11:56:59.659576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.659576+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":828,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.530117+0000","last_change":"2024-09-21T11:58:22.381598+0000","last_active":"2024-09-21T11:58:45.530117+0000","last_peered":"2024-09-21T11:58:45.530117+0000","last_clean":"2024-09-21T11:58:45.530117+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:58:45.530117+0000","last_undegraded":"2024-09-21T11:58:45.530117+0000","last_fullsized":"2024-09-21T11:58:45.530117+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:22.381531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:22.381531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.811109+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":824,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515115+0000","last_change":"2024-09-21T11:58:24.861423+0000","last_active":"2024-09-21T11:58:45.515115+0000","last_peered":"2024-09-21T11:58:45.515115+0000","last_clean":"2024-09-21T11:58:45.515115+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:58:45.515115+0000","last_undegraded":"2024-09-21T11:58:45.515115+0000","last_fullsized":"2024-09-21T11:58:45.515115+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:24.861354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:24.861354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.549127+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":326,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532609+0000","last_change":"2024-09-21T11:56:34.073296+0000","last_active":"2024-09-21T11:58:45.532609+0000","last_peered":"2024-09-21T11:58:45.532609+0000","last_clean":"2024-09-21T11:58:45.532609+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:58:45.532609+0000","last_undegraded":"2024-09-21T11:58:45.532609+0000","last_fullsized":"2024-09-21T11:58:45.532609+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:34.073249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:34.073249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:15.736984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":54,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.566832+0000","last_change":"2024-09-21T11:57:58.534353+0000","last_active":"2024-09-21T11:58:46.566832+0000","last_peered":"2024-09-21T11:58:46.566832+0000","last_clean":"2024-09-21T11:58:46.566832+0000","last_became_active":"2024-09-21T11:57:58.530327+0000","last_became_peered":"2024-09-21T11:57:58.530327+0000","last_unstale":"2024-09-21T11:58:46.566832+0000","last_undegraded":"2024-09-21T11:58:46.566832+0000","last_fullsized":"2024-09-21T11:58:46.566832+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:27.176320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:27.176320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":972,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.560596+0000","last_change":"2024-09-21T11:57:51.913043+0000","last_active":"2024-09-21T11:58:46.560596+0000","last_peered":"2024-09-21T11:58:46.560596+0000","last_clean":"2024-09-21T11:58:46.560596+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T11:58:46.560596+0000","last_undegraded":"2024-09-21T11:58:46.560596+0000","last_fullsized":"2024-09-21T11:58:46.560596+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:57:51.912800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.912800+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:55.941418+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":540,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.549420+0000","last_change":"2024-09-21T11:58:17.533215+0000","last_active":"2024-09-21T11:58:46.549420+0000","last_peered":"2024-09-21T11:58:46.549420+0000","last_clean":"2024-09-21T11:58:46.549420+0000","last_became_active":"2024-09-21T11:58:17.532890+0000","last_became_peered":"2024-09-21T11:58:17.532890+0000","last_unstale":"2024-09-21T11:58:46.549420+0000","last_undegraded":"2024-09-21T11:58:46.549420+0000","last_fullsized":"2024-09-21T11:58:46.549420+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:56.501356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:56.501356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":820,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532061+0000","last_change":"2024-09-21T11:57:04.647951+0000","last_active":"2024-09-21T11:58:45.532061+0000","last_peered":"2024-09-21T11:58:45.532061+0000","last_clean":"2024-09-21T11:58:45.532061+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:58:45.532061+0000","last_undegraded":"2024-09-21T11:58:45.532061+0000","last_fullsized":"2024-09-21T11:58:45.532061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:04.647749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:04.647749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":641,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.567912+0000","last_change":"2024-09-21T11:55:57.442188+0000","last_active":"2024-09-21T11:58:46.567912+0000","last_peered":"2024-09-21T11:58:46.567912+0000","last_clean":"2024-09-21T11:58:46.567912+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:58:46.567912+0000","last_undegraded":"2024-09-21T11:58:46.567912+0000","last_fullsized":"2024-09-21T11:58:46.567912+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:57.441965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.441965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":881,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531592+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:58:45.531592+0000","last_peered":"2024-09-21T11:58:45.531592+0000","last_clean":"2024-09-21T11:58:45.531592+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:58:45.531592+0000","last_undegraded":"2024-09-21T11:58:45.531592+0000","last_fullsized":"2024-09-21T11:58:45.531592+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:56.834377+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":339,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531553+0000","last_change":"2024-09-21T11:58:17.529121+0000","last_active":"2024-09-21T11:58:45.531553+0000","last_peered":"2024-09-21T11:58:45.531553+0000","last_clean":"2024-09-21T11:58:45.531553+0000","last_became_active":"2024-09-21T11:58:17.528785+0000","last_became_peered":"2024-09-21T11:58:17.528785+0000","last_unstale":"2024-09-21T11:58:45.531553+0000","last_undegraded":"2024-09-21T11:58:45.531553+0000","last_fullsized":"2024-09-21T11:58:45.531553+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.487132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.487132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":322,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.566729+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:58:46.566729+0000","last_peered":"2024-09-21T11:58:46.566729+0000","last_clean":"2024-09-21T11:58:46.566729+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:58:46.566729+0000","last_undegraded":"2024-09-21T11:58:46.566729+0000","last_fullsized":"2024-09-21T11:58:46.566729+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":560,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562426+0000","last_change":"2024-09-21T11:58:17.554468+0000","last_active":"2024-09-21T11:58:46.562426+0000","last_peered":"2024-09-21T11:58:46.562426+0000","last_clean":"2024-09-21T11:58:46.562426+0000","last_became_active":"2024-09-21T11:58:17.529593+0000","last_became_peered":"2024-09-21T11:58:17.529593+0000","last_unstale":"2024-09-21T11:58:46.562426+0000","last_undegraded":"2024-09-21T11:58:46.562426+0000","last_fullsized":"2024-09-21T11:58:46.562426+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:10.530913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.530913+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":852,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.569080+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:58:46.569080+0000","last_peered":"2024-09-21T11:58:46.569080+0000","last_clean":"2024-09-21T11:58:46.569080+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:58:46.569080+0000","last_undegraded":"2024-09-21T11:58:46.569080+0000","last_fullsized":"2024-09-21T11:58:46.569080+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"432'10","reported_seq":1101,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.596995+0000","last_change":"2024-09-21T11:58:18.736596+0000","last_active":"2024-09-21T11:58:45.596995+0000","last_peered":"2024-09-21T11:58:45.596995+0000","last_clean":"2024-09-21T11:58:45.596995+0000","last_became_active":"2024-09-21T11:58:17.529830+0000","last_became_peered":"2024-09-21T11:58:17.529830+0000","last_unstale":"2024-09-21T11:58:45.596995+0000","last_undegraded":"2024-09-21T11:58:45.596995+0000","last_fullsized":"2024-09-21T11:58:45.596995+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"383'9","last_scrub_stamp":"2024-09-21T11:58:18.736520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:18.736520+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:40.102209+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":68,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515347+0000","last_change":"2024-09-21T11:57:49.896884+0000","last_active":"2024-09-21T11:58:45.515347+0000","last_peered":"2024-09-21T11:58:45.515347+0000","last_clean":"2024-09-21T11:58:45.515347+0000","last_became_active":"2024-09-21T11:57:45.504680+0000","last_became_peered":"2024-09-21T11:57:45.504680+0000","last_unstale":"2024-09-21T11:58:45.515347+0000","last_undegraded":"2024-09-21T11:58:45.515347+0000","last_fullsized":"2024-09-21T11:58:45.515347+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:49.896819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:49.896819+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:52.548939+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"442'9824","reported_seq":9936,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.700173+0000","last_change":"2024-09-21T11:56:36.507880+0000","last_active":"2024-09-21T11:58:46.700173+0000","last_peered":"2024-09-21T11:58:46.700173+0000","last_clean":"2024-09-21T11:58:46.700173+0000","last_became_active":"2024-09-21T11:56:36.507606+0000","last_became_peered":"2024-09-21T11:56:36.507606+0000","last_unstale":"2024-09-21T11:58:46.700173+0000","last_undegraded":"2024-09-21T11:58:46.700173+0000","last_fullsized":"2024-09-21T11:58:46.700173+0000","mapping_epoch":392,"log_start":"406'5022","ondisk_log_start":"406'5022","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356122624,"num_objects":5434,"num_object_clones":0,"num_object_copies":16302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9824,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":561,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.563535+0000","last_change":"2024-09-21T11:55:52.562992+0000","last_active":"2024-09-21T11:58:46.563535+0000","last_peered":"2024-09-21T11:58:46.563535+0000","last_clean":"2024-09-21T11:58:46.563535+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:58:46.563535+0000","last_undegraded":"2024-09-21T11:58:46.563535+0000","last_fullsized":"2024-09-21T11:58:46.563535+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:55:52.562928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.562928+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:55.612843+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":846,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.542866+0000","last_change":"2024-09-21T11:57:13.738771+0000","last_active":"2024-09-21T11:58:45.542866+0000","last_peered":"2024-09-21T11:58:45.542866+0000","last_clean":"2024-09-21T11:58:45.542866+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:58:45.542866+0000","last_undegraded":"2024-09-21T11:58:45.542866+0000","last_fullsized":"2024-09-21T11:58:45.542866+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:13.738727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:13.738727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"432'11","reported_seq":461,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515634+0000","last_change":"2024-09-21T11:58:17.537763+0000","last_active":"2024-09-21T11:58:45.515634+0000","last_peered":"2024-09-21T11:58:45.515634+0000","last_clean":"2024-09-21T11:58:45.515634+0000","last_became_active":"2024-09-21T11:58:17.526752+0000","last_became_peered":"2024-09-21T11:58:17.526752+0000","last_unstale":"2024-09-21T11:58:45.515634+0000","last_undegraded":"2024-09-21T11:58:45.515634+0000","last_fullsized":"2024-09-21T11:58:45.515634+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"366'10","last_scrub_stamp":"2024-09-21T11:56:48.961017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.961017+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.739596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":871,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515422+0000","last_change":"2024-09-21T11:58:16.866853+0000","last_active":"2024-09-21T11:58:45.515422+0000","last_peered":"2024-09-21T11:58:45.515422+0000","last_clean":"2024-09-21T11:58:45.515422+0000","last_became_active":"2024-09-21T11:57:58.526927+0000","last_became_peered":"2024-09-21T11:57:58.526927+0000","last_unstale":"2024-09-21T11:58:45.515422+0000","last_undegraded":"2024-09-21T11:58:45.515422+0000","last_fullsized":"2024-09-21T11:58:45.515422+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:16.866782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:16.866782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.387681+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":70,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561517+0000","last_change":"2024-09-21T11:57:47.906127+0000","last_active":"2024-09-21T11:58:46.561517+0000","last_peered":"2024-09-21T11:58:46.561517+0000","last_clean":"2024-09-21T11:58:46.561517+0000","last_became_active":"2024-09-21T11:57:45.514811+0000","last_became_peered":"2024-09-21T11:57:45.514811+0000","last_unstale":"2024-09-21T11:58:46.561517+0000","last_undegraded":"2024-09-21T11:58:46.561517+0000","last_fullsized":"2024-09-21T11:58:46.561517+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:47.905784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:47.905784+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:15.591116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"441'9959","reported_seq":10067,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.225229+0000","last_change":"2024-09-21T11:56:36.511425+0000","last_active":"2024-09-21T11:58:46.225229+0000","last_peered":"2024-09-21T11:58:46.225229+0000","last_clean":"2024-09-21T11:58:46.225229+0000","last_became_active":"2024-09-21T11:56:36.511191+0000","last_became_peered":"2024-09-21T11:56:36.511191+0000","last_unstale":"2024-09-21T11:58:46.225229+0000","last_undegraded":"2024-09-21T11:58:46.225229+0000","last_fullsized":"2024-09-21T11:58:46.225229+0000","mapping_epoch":392,"log_start":"407'5242","ondisk_log_start":"407'5242","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4717,"log_dups_size":0,"ondisk_log_size":4717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:11.336526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":372834304,"num_objects":5689,"num_object_clones":0,"num_object_copies":17067,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9959,"num_write_kb":500736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":937,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.542583+0000","last_change":"2024-09-21T11:57:58.547816+0000","last_active":"2024-09-21T11:58:45.542583+0000","last_peered":"2024-09-21T11:58:45.542583+0000","last_clean":"2024-09-21T11:58:45.542583+0000","last_became_active":"2024-09-21T11:57:58.537184+0000","last_became_peered":"2024-09-21T11:57:58.537184+0000","last_unstale":"2024-09-21T11:58:45.542583+0000","last_undegraded":"2024-09-21T11:58:45.542583+0000","last_fullsized":"2024-09-21T11:58:45.542583+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:57:00.631769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:00.631769+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":840,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532475+0000","last_change":"2024-09-21T11:56:53.701703+0000","last_active":"2024-09-21T11:58:45.532475+0000","last_peered":"2024-09-21T11:58:45.532475+0000","last_clean":"2024-09-21T11:58:45.532475+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:58:45.532475+0000","last_undegraded":"2024-09-21T11:58:45.532475+0000","last_fullsized":"2024-09-21T11:58:45.532475+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.701639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.701639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":326,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.535321+0000","last_change":"2024-09-21T11:58:19.959956+0000","last_active":"2024-09-21T11:58:45.535321+0000","last_peered":"2024-09-21T11:58:45.535321+0000","last_clean":"2024-09-21T11:58:45.535321+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:58:45.535321+0000","last_undegraded":"2024-09-21T11:58:45.535321+0000","last_fullsized":"2024-09-21T11:58:45.535321+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:19.959912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:19.959912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.737932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":330,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.539905+0000","last_change":"2024-09-21T11:57:42.760097+0000","last_active":"2024-09-21T11:58:46.539905+0000","last_peered":"2024-09-21T11:58:46.539905+0000","last_clean":"2024-09-21T11:58:46.539905+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:58:46.539905+0000","last_undegraded":"2024-09-21T11:58:46.539905+0000","last_fullsized":"2024-09-21T11:58:46.539905+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:42.755633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:42.755633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:46.597748+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":73,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531366+0000","last_change":"2024-09-21T11:57:47.804542+0000","last_active":"2024-09-21T11:58:45.531366+0000","last_peered":"2024-09-21T11:58:45.531366+0000","last_clean":"2024-09-21T11:58:45.531366+0000","last_became_active":"2024-09-21T11:57:45.514189+0000","last_became_peered":"2024-09-21T11:57:45.514189+0000","last_unstale":"2024-09-21T11:58:45.531366+0000","last_undegraded":"2024-09-21T11:58:45.531366+0000","last_fullsized":"2024-09-21T11:58:45.531366+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:47.804466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:47.804466+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:58.484095+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"442'9610","reported_seq":9733,"reported_epoch":442,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-21T11:58:46.738646+0000","last_change":"2024-09-21T11:58:18.669587+0000","last_active":"2024-09-21T11:58:46.738646+0000","last_peered":"2024-09-21T11:58:46.738646+0000","last_clean":"2024-09-21T11:58:17.235375+0000","last_became_active":"2024-09-21T11:58:18.605530+0000","last_became_peered":"2024-09-21T11:58:18.605530+0000","last_unstale":"2024-09-21T11:58:46.738646+0000","last_undegraded":"2024-09-21T11:58:46.738646+0000","last_fullsized":"2024-09-21T11:58:46.738646+0000","mapping_epoch":431,"log_start":"406'4767","ondisk_log_start":"406'4767","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4843,"log_dups_size":0,"ondisk_log_size":4843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358612992,"num_objects":5472,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5472,"num_objects_unfound":0,"num_objects_dirty":5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9610,"num_write_kb":482624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,7,0],"avail_no_missing":["1","0","7"],"object_location_counts":[{"shards":"0,1,7","objects":5472}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":876,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515140+0000","last_change":"2024-09-21T11:58:17.828372+0000","last_active":"2024-09-21T11:58:45.515140+0000","last_peered":"2024-09-21T11:58:45.515140+0000","last_clean":"2024-09-21T11:58:45.515140+0000","last_became_active":"2024-09-21T11:58:17.525491+0000","last_became_peered":"2024-09-21T11:58:17.525491+0000","last_unstale":"2024-09-21T11:58:45.515140+0000","last_undegraded":"2024-09-21T11:58:45.515140+0000","last_fullsized":"2024-09-21T11:58:45.515140+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:58:17.828305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:17.828305+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.685496+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":58,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.569178+0000","last_change":"2024-09-21T11:57:58.539709+0000","last_active":"2024-09-21T11:58:46.569178+0000","last_peered":"2024-09-21T11:58:46.569178+0000","last_clean":"2024-09-21T11:58:46.569178+0000","last_became_active":"2024-09-21T11:57:58.526938+0000","last_became_peered":"2024-09-21T11:57:58.526938+0000","last_unstale":"2024-09-21T11:58:46.569178+0000","last_undegraded":"2024-09-21T11:58:46.569178+0000","last_fullsized":"2024-09-21T11:58:46.569178+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:48.446635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.446635+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":1074,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.542433+0000","last_change":"2024-09-21T11:56:05.567407+0000","last_active":"2024-09-21T11:58:45.542433+0000","last_peered":"2024-09-21T11:58:45.542433+0000","last_clean":"2024-09-21T11:58:45.542433+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:58:45.542433+0000","last_undegraded":"2024-09-21T11:58:45.542433+0000","last_fullsized":"2024-09-21T11:58:45.542433+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:05.567332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.567332+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:05.895448+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":829,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.530309+0000","last_change":"2024-09-21T11:56:34.581966+0000","last_active":"2024-09-21T11:58:45.530309+0000","last_peered":"2024-09-21T11:58:45.530309+0000","last_clean":"2024-09-21T11:58:45.530309+0000","last_became_active":"2024-09-21T11:56:34.581626+0000","last_became_peered":"2024-09-21T11:56:34.581626+0000","last_unstale":"2024-09-21T11:58:45.530309+0000","last_undegraded":"2024-09-21T11:58:45.530309+0000","last_fullsized":"2024-09-21T11:58:45.530309+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.643323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.643323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:35.106418+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":154,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532840+0000","last_change":"2024-09-21T11:57:45.998708+0000","last_active":"2024-09-21T11:58:45.532840+0000","last_peered":"2024-09-21T11:58:45.532840+0000","last_clean":"2024-09-21T11:58:45.532840+0000","last_became_active":"2024-09-21T11:57:45.514310+0000","last_became_peered":"2024-09-21T11:57:45.514310+0000","last_unstale":"2024-09-21T11:58:45.532840+0000","last_undegraded":"2024-09-21T11:58:45.532840+0000","last_fullsized":"2024-09-21T11:58:45.532840+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:45.998644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:45.998644+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.495485+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"442'9651","reported_seq":9763,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.889069+0000","last_change":"2024-09-21T11:56:36.623897+0000","last_active":"2024-09-21T11:58:46.889069+0000","last_peered":"2024-09-21T11:58:46.889069+0000","last_clean":"2024-09-21T11:58:46.889069+0000","last_became_active":"2024-09-21T11:56:36.623581+0000","last_became_peered":"2024-09-21T11:56:36.623581+0000","last_unstale":"2024-09-21T11:58:46.889069+0000","last_undegraded":"2024-09-21T11:58:46.889069+0000","last_fullsized":"2024-09-21T11:58:46.889069+0000","mapping_epoch":392,"log_start":"402'3516","ondisk_log_start":"402'3516","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":6135,"log_dups_size":0,"ondisk_log_size":6135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:00.021032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357040152,"num_objects":5449,"num_object_clones":0,"num_object_copies":16347,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9651,"num_write_kb":483137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":592,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.575210+0000","last_change":"2024-09-21T11:58:17.557543+0000","last_active":"2024-09-21T11:58:46.575210+0000","last_peered":"2024-09-21T11:58:46.575210+0000","last_clean":"2024-09-21T11:58:46.575210+0000","last_became_active":"2024-09-21T11:58:17.531001+0000","last_became_peered":"2024-09-21T11:58:17.531001+0000","last_unstale":"2024-09-21T11:58:46.575210+0000","last_undegraded":"2024-09-21T11:58:46.575210+0000","last_fullsized":"2024-09-21T11:58:46.575210+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:56:09.421193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.421193+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":856,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515751+0000","last_change":"2024-09-21T11:56:51.036609+0000","last_active":"2024-09-21T11:58:45.515751+0000","last_peered":"2024-09-21T11:58:45.515751+0000","last_clean":"2024-09-21T11:58:45.515751+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:58:45.515751+0000","last_undegraded":"2024-09-21T11:58:45.515751+0000","last_fullsized":"2024-09-21T11:58:45.515751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:51.036535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:51.036535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:40.871456+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1062,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.567171+0000","last_change":"2024-09-21T11:57:40.102377+0000","last_active":"2024-09-21T11:58:46.567171+0000","last_peered":"2024-09-21T11:58:46.567171+0000","last_clean":"2024-09-21T11:58:46.567171+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:58:46.567171+0000","last_undegraded":"2024-09-21T11:58:46.567171+0000","last_fullsized":"2024-09-21T11:58:46.567171+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:40.102294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:40.102294+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:59.370714+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":830,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.565046+0000","last_change":"2024-09-21T11:57:36.109392+0000","last_active":"2024-09-21T11:58:46.565046+0000","last_peered":"2024-09-21T11:58:46.565046+0000","last_clean":"2024-09-21T11:58:46.565046+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:58:46.565046+0000","last_undegraded":"2024-09-21T11:58:46.565046+0000","last_fullsized":"2024-09-21T11:58:46.565046+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:36.109317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:36.109317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:01.441881+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":154,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532515+0000","last_change":"2024-09-21T11:57:51.011971+0000","last_active":"2024-09-21T11:58:45.532515+0000","last_peered":"2024-09-21T11:58:45.532515+0000","last_clean":"2024-09-21T11:58:45.532515+0000","last_became_active":"2024-09-21T11:57:45.518764+0000","last_became_peered":"2024-09-21T11:57:45.518764+0000","last_unstale":"2024-09-21T11:58:45.532515+0000","last_undegraded":"2024-09-21T11:58:45.532515+0000","last_fullsized":"2024-09-21T11:58:45.532515+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:51.011892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.011892+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:03.731641+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.c","version":"442'9763","reported_seq":9879,"reported_epoch":442,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-21T11:58:46.756805+0000","last_change":"2024-09-21T11:58:18.622714+0000","last_active":"2024-09-21T11:58:46.756805+0000","last_peered":"2024-09-21T11:58:46.756805+0000","last_clean":"2024-09-21T11:58:17.235882+0000","last_became_active":"2024-09-21T11:58:18.606350+0000","last_became_peered":"2024-09-21T11:58:18.606350+0000","last_unstale":"2024-09-21T11:58:46.756805+0000","last_undegraded":"2024-09-21T11:58:46.756805+0000","last_fullsized":"2024-09-21T11:58:46.756805+0000","mapping_epoch":431,"log_start":"406'4979","ondisk_log_start":"406'4979","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4784,"log_dups_size":0,"ondisk_log_size":4784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354877440,"num_objects":5415,"num_object_clones":0,"num_object_copies":16245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5415,"num_objects_unfound":0,"num_objects_dirty":5415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9763,"num_write_kb":485696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,7],"avail_no_missing":["1","2","7"],"object_location_counts":[{"shards":"1,2,7","objects":5415}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":896,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.567292+0000","last_change":"2024-09-21T11:57:50.976268+0000","last_active":"2024-09-21T11:58:46.567292+0000","last_peered":"2024-09-21T11:58:46.567292+0000","last_clean":"2024-09-21T11:58:46.567292+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:58:46.567292+0000","last_undegraded":"2024-09-21T11:58:46.567292+0000","last_fullsized":"2024-09-21T11:58:46.567292+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:57:50.976029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:50.976029+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:00.941833+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":527,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562685+0000","last_change":"2024-09-21T11:56:58.434536+0000","last_active":"2024-09-21T11:58:46.562685+0000","last_peered":"2024-09-21T11:58:46.562685+0000","last_clean":"2024-09-21T11:58:46.562685+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:58:46.562685+0000","last_undegraded":"2024-09-21T11:58:46.562685+0000","last_fullsized":"2024-09-21T11:58:46.562685+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.434464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.434464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":807,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532408+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:58:45.532408+0000","last_peered":"2024-09-21T11:58:45.532408+0000","last_clean":"2024-09-21T11:58:45.532408+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:58:45.532408+0000","last_undegraded":"2024-09-21T11:58:45.532408+0000","last_fullsized":"2024-09-21T11:58:45.532408+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:34.633118+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":519,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562094+0000","last_change":"2024-09-21T11:55:51.602665+0000","last_active":"2024-09-21T11:58:46.562094+0000","last_peered":"2024-09-21T11:58:46.562094+0000","last_clean":"2024-09-21T11:58:46.562094+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:58:46.562094+0000","last_undegraded":"2024-09-21T11:58:46.562094+0000","last_fullsized":"2024-09-21T11:58:46.562094+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.602587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.602587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":68,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515007+0000","last_change":"2024-09-21T11:57:53.922876+0000","last_active":"2024-09-21T11:58:45.515007+0000","last_peered":"2024-09-21T11:58:45.515007+0000","last_clean":"2024-09-21T11:58:45.515007+0000","last_became_active":"2024-09-21T11:57:45.515242+0000","last_became_peered":"2024-09-21T11:57:45.515242+0000","last_unstale":"2024-09-21T11:58:45.515007+0000","last_undegraded":"2024-09-21T11:58:45.515007+0000","last_fullsized":"2024-09-21T11:58:45.515007+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:53.922795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:53.922795+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:20.497165+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.b","version":"441'9629","reported_seq":9737,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.954737+0000","last_change":"2024-09-21T11:56:36.510297+0000","last_active":"2024-09-21T11:58:45.954737+0000","last_peered":"2024-09-21T11:58:45.954737+0000","last_clean":"2024-09-21T11:58:45.954737+0000","last_became_active":"2024-09-21T11:56:36.510018+0000","last_became_peered":"2024-09-21T11:56:36.510018+0000","last_unstale":"2024-09-21T11:58:45.954737+0000","last_undegraded":"2024-09-21T11:58:45.954737+0000","last_fullsized":"2024-09-21T11:58:45.954737+0000","mapping_epoch":392,"log_start":"406'4921","ondisk_log_start":"406'4921","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4708,"log_dups_size":0,"ondisk_log_size":4708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359464960,"num_objects":5485,"num_object_clones":0,"num_object_copies":16455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9629,"num_write_kb":483648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":684,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562348+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:58:46.562348+0000","last_peered":"2024-09-21T11:58:46.562348+0000","last_clean":"2024-09-21T11:58:46.562348+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:58:46.562348+0000","last_undegraded":"2024-09-21T11:58:46.562348+0000","last_fullsized":"2024-09-21T11:58:46.562348+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":353,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.567385+0000","last_change":"2024-09-21T11:57:51.934799+0000","last_active":"2024-09-21T11:58:46.567385+0000","last_peered":"2024-09-21T11:58:46.567385+0000","last_clean":"2024-09-21T11:58:46.567385+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:58:46.567385+0000","last_undegraded":"2024-09-21T11:58:46.567385+0000","last_fullsized":"2024-09-21T11:58:46.567385+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:51.934563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.934563+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:17.585958+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":848,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515598+0000","last_change":"2024-09-21T11:56:28.083398+0000","last_active":"2024-09-21T11:58:45.515598+0000","last_peered":"2024-09-21T11:58:45.515598+0000","last_clean":"2024-09-21T11:58:45.515598+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:58:45.515598+0000","last_undegraded":"2024-09-21T11:58:45.515598+0000","last_fullsized":"2024-09-21T11:58:45.515598+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:56:28.083324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.083324+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:59.743090+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":61,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.566736+0000","last_change":"2024-09-21T11:58:17.527987+0000","last_active":"2024-09-21T11:58:46.566736+0000","last_peered":"2024-09-21T11:58:46.566736+0000","last_clean":"2024-09-21T11:58:46.566736+0000","last_became_active":"2024-09-21T11:58:17.527653+0000","last_became_peered":"2024-09-21T11:58:17.527653+0000","last_unstale":"2024-09-21T11:58:46.566736+0000","last_undegraded":"2024-09-21T11:58:46.566736+0000","last_fullsized":"2024-09-21T11:58:46.566736+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":804,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532564+0000","last_change":"2024-09-21T11:57:54.859726+0000","last_active":"2024-09-21T11:58:45.532564+0000","last_peered":"2024-09-21T11:58:45.532564+0000","last_clean":"2024-09-21T11:58:45.532564+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:58:45.532564+0000","last_undegraded":"2024-09-21T11:58:45.532564+0000","last_fullsized":"2024-09-21T11:58:45.532564+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:54.859668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.859668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.673552+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":948,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.529887+0000","last_change":"2024-09-21T11:58:17.529673+0000","last_active":"2024-09-21T11:58:45.529887+0000","last_peered":"2024-09-21T11:58:45.529887+0000","last_clean":"2024-09-21T11:58:45.529887+0000","last_became_active":"2024-09-21T11:58:17.529304+0000","last_became_peered":"2024-09-21T11:58:17.529304+0000","last_unstale":"2024-09-21T11:58:45.529887+0000","last_undegraded":"2024-09-21T11:58:45.529887+0000","last_fullsized":"2024-09-21T11:58:45.529887+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:50.548003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:50.548003+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:04.592556+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"441'9678","reported_seq":9786,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.859439+0000","last_change":"2024-09-21T11:56:36.509094+0000","last_active":"2024-09-21T11:58:45.859439+0000","last_peered":"2024-09-21T11:58:45.859439+0000","last_clean":"2024-09-21T11:58:45.859439+0000","last_became_active":"2024-09-21T11:56:36.508840+0000","last_became_peered":"2024-09-21T11:56:36.508840+0000","last_unstale":"2024-09-21T11:58:45.859439+0000","last_undegraded":"2024-09-21T11:58:45.859439+0000","last_fullsized":"2024-09-21T11:58:45.859439+0000","mapping_epoch":392,"log_start":"406'4995","ondisk_log_start":"406'4995","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4683,"log_dups_size":0,"ondisk_log_size":4683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363593728,"num_objects":5548,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9678,"num_write_kb":487232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":508,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561111+0000","last_change":"2024-09-21T11:57:58.543832+0000","last_active":"2024-09-21T11:58:46.561111+0000","last_peered":"2024-09-21T11:58:46.561111+0000","last_clean":"2024-09-21T11:58:46.561111+0000","last_became_active":"2024-09-21T11:57:58.530507+0000","last_became_peered":"2024-09-21T11:57:58.530507+0000","last_unstale":"2024-09-21T11:58:46.561111+0000","last_undegraded":"2024-09-21T11:58:46.561111+0000","last_fullsized":"2024-09-21T11:58:46.561111+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:57:08.488050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:08.488050+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":531,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.568031+0000","last_change":"2024-09-21T11:57:43.583962+0000","last_active":"2024-09-21T11:58:46.568031+0000","last_peered":"2024-09-21T11:58:46.568031+0000","last_clean":"2024-09-21T11:58:46.568031+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:58:46.568031+0000","last_undegraded":"2024-09-21T11:58:46.568031+0000","last_fullsized":"2024-09-21T11:58:46.568031+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:43.583736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.583736+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:11.292974+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":852,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.539931+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:58:46.539931+0000","last_peered":"2024-09-21T11:58:46.539931+0000","last_clean":"2024-09-21T11:58:46.539931+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:58:46.539931+0000","last_undegraded":"2024-09-21T11:58:46.539931+0000","last_fullsized":"2024-09-21T11:58:46.539931+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.044139+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":826,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.560948+0000","last_change":"2024-09-21T11:57:44.790249+0000","last_active":"2024-09-21T11:58:46.560948+0000","last_peered":"2024-09-21T11:58:46.560948+0000","last_clean":"2024-09-21T11:58:46.560948+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:58:46.560948+0000","last_undegraded":"2024-09-21T11:58:46.560948+0000","last_fullsized":"2024-09-21T11:58:46.560948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:44.790174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:44.790174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:49.311243+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":835,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.530378+0000","last_change":"2024-09-21T11:58:30.226389+0000","last_active":"2024-09-21T11:58:45.530378+0000","last_peered":"2024-09-21T11:58:45.530378+0000","last_clean":"2024-09-21T11:58:45.530378+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:58:45.530378+0000","last_undegraded":"2024-09-21T11:58:45.530378+0000","last_fullsized":"2024-09-21T11:58:45.530378+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:30.226345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:30.226345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:50.129480+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":822,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.565642+0000","last_change":"2024-09-21T11:58:30.794254+0000","last_active":"2024-09-21T11:58:46.565642+0000","last_peered":"2024-09-21T11:58:46.565642+0000","last_clean":"2024-09-21T11:58:46.565642+0000","last_became_active":"2024-09-21T11:58:30.793717+0000","last_became_peered":"2024-09-21T11:58:30.793717+0000","last_unstale":"2024-09-21T11:58:46.565642+0000","last_undegraded":"2024-09-21T11:58:46.565642+0000","last_fullsized":"2024-09-21T11:58:46.565642+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":436,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:48.627711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.627711+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:14.347769+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"441'9754","reported_seq":12983,"reported_epoch":441,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:58:45.951736+0000","last_change":"2024-09-21T11:58:30.022462+0000","last_active":"2024-09-21T11:58:45.951736+0000","last_peered":"2024-09-21T11:58:45.951736+0000","last_clean":"2024-09-21T11:57:57.490959+0000","last_became_active":"2024-09-21T11:57:59.990813+0000","last_became_peered":"2024-09-21T11:57:59.990813+0000","last_unstale":"2024-09-21T11:58:45.951736+0000","last_undegraded":"2024-09-21T11:58:45.951736+0000","last_fullsized":"2024-09-21T11:58:45.951736+0000","mapping_epoch":421,"log_start":"406'5045","ondisk_log_start":"406'5045","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":366477312,"num_objects":5592,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4154,"num_objects_unfound":0,"num_objects_dirty":5592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9754,"num_write_kb":491072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1555,"num_bytes_recovered":101908480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,1],"avail_no_missing":["3","1","6"],"object_location_counts":[{"shards":"1,3,6","objects":5592}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":68,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.568699+0000","last_change":"2024-09-21T11:57:58.545640+0000","last_active":"2024-09-21T11:58:46.568699+0000","last_peered":"2024-09-21T11:58:46.568699+0000","last_clean":"2024-09-21T11:58:46.568699+0000","last_became_active":"2024-09-21T11:57:58.527381+0000","last_became_peered":"2024-09-21T11:57:58.527381+0000","last_unstale":"2024-09-21T11:58:46.568699+0000","last_undegraded":"2024-09-21T11:58:46.568699+0000","last_fullsized":"2024-09-21T11:58:46.568699+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:57:54.679272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.679272+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:09.756620+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":864,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.567778+0000","last_change":"2024-09-21T11:58:17.528448+0000","last_active":"2024-09-21T11:58:46.567778+0000","last_peered":"2024-09-21T11:58:46.567778+0000","last_clean":"2024-09-21T11:58:46.567778+0000","last_became_active":"2024-09-21T11:58:17.528154+0000","last_became_peered":"2024-09-21T11:58:17.528154+0000","last_unstale":"2024-09-21T11:58:46.567778+0000","last_undegraded":"2024-09-21T11:58:46.567778+0000","last_fullsized":"2024-09-21T11:58:46.567778+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:01.083437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.083437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1991,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.529605+0000","last_change":"2024-09-21T11:56:09.604887+0000","last_active":"2024-09-21T11:58:45.529605+0000","last_peered":"2024-09-21T11:58:45.529605+0000","last_clean":"2024-09-21T11:58:45.529605+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:58:45.529605+0000","last_undegraded":"2024-09-21T11:58:45.529605+0000","last_fullsized":"2024-09-21T11:58:45.529605+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:56:09.604808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.604808+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.102017+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1116,"num_read_kb":1111,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":837,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.516026+0000","last_change":"2024-09-21T11:56:57.019113+0000","last_active":"2024-09-21T11:58:45.516026+0000","last_peered":"2024-09-21T11:58:45.516026+0000","last_clean":"2024-09-21T11:58:45.516026+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:58:45.516026+0000","last_undegraded":"2024-09-21T11:58:45.516026+0000","last_fullsized":"2024-09-21T11:58:45.516026+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:57.019066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:57.019066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":519,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562252+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:58:46.562252+0000","last_peered":"2024-09-21T11:58:46.562252+0000","last_clean":"2024-09-21T11:58:46.562252+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:58:46.562252+0000","last_undegraded":"2024-09-21T11:58:46.562252+0000","last_fullsized":"2024-09-21T11:58:46.562252+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"441'9689","reported_seq":9799,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.955134+0000","last_change":"2024-09-21T11:56:36.509270+0000","last_active":"2024-09-21T11:58:45.955134+0000","last_peered":"2024-09-21T11:58:45.955134+0000","last_clean":"2024-09-21T11:58:45.955134+0000","last_became_active":"2024-09-21T11:56:36.508533+0000","last_became_peered":"2024-09-21T11:56:36.508533+0000","last_unstale":"2024-09-21T11:58:45.955134+0000","last_undegraded":"2024-09-21T11:58:45.955134+0000","last_fullsized":"2024-09-21T11:58:45.955134+0000","mapping_epoch":392,"log_start":"406'5022","ondisk_log_start":"406'5022","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4667,"log_dups_size":0,"ondisk_log_size":4667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353828864,"num_objects":5399,"num_object_clones":0,"num_object_copies":16197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9689,"num_write_kb":482816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":1093,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.533864+0000","last_change":"2024-09-21T11:56:29.095770+0000","last_active":"2024-09-21T11:58:45.533864+0000","last_peered":"2024-09-21T11:58:45.533864+0000","last_clean":"2024-09-21T11:58:45.533864+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:58:45.533864+0000","last_undegraded":"2024-09-21T11:58:45.533864+0000","last_fullsized":"2024-09-21T11:58:45.533864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:56:29.095713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.095713+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:04.820902+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":833,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.533318+0000","last_change":"2024-09-21T11:57:14.756298+0000","last_active":"2024-09-21T11:58:45.533318+0000","last_peered":"2024-09-21T11:58:45.533318+0000","last_clean":"2024-09-21T11:58:45.533318+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:58:45.533318+0000","last_undegraded":"2024-09-21T11:58:45.533318+0000","last_fullsized":"2024-09-21T11:58:45.533318+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:14.756227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:14.756227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":943,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.529707+0000","last_change":"2024-09-21T11:57:58.529309+0000","last_active":"2024-09-21T11:58:45.529707+0000","last_peered":"2024-09-21T11:58:45.529707+0000","last_clean":"2024-09-21T11:58:45.529707+0000","last_became_active":"2024-09-21T11:57:58.528825+0000","last_became_peered":"2024-09-21T11:57:58.528825+0000","last_unstale":"2024-09-21T11:58:45.529707+0000","last_undegraded":"2024-09-21T11:58:45.529707+0000","last_fullsized":"2024-09-21T11:58:45.529707+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:48.495794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.495794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:54.548925+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":823,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531246+0000","last_change":"2024-09-21T11:58:28.274182+0000","last_active":"2024-09-21T11:58:45.531246+0000","last_peered":"2024-09-21T11:58:45.531246+0000","last_clean":"2024-09-21T11:58:45.531246+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:58:45.531246+0000","last_undegraded":"2024-09-21T11:58:45.531246+0000","last_fullsized":"2024-09-21T11:58:45.531246+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:28.274120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.274120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.924347+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":835,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515992+0000","last_change":"2024-09-21T11:57:05.904684+0000","last_active":"2024-09-21T11:58:45.515992+0000","last_peered":"2024-09-21T11:58:45.515992+0000","last_clean":"2024-09-21T11:58:45.515992+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:58:45.515992+0000","last_undegraded":"2024-09-21T11:58:45.515992+0000","last_fullsized":"2024-09-21T11:58:45.515992+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:05.904621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:05.904621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"441'9681","reported_seq":9791,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.876560+0000","last_change":"2024-09-21T11:56:36.508185+0000","last_active":"2024-09-21T11:58:45.876560+0000","last_peered":"2024-09-21T11:58:45.876560+0000","last_clean":"2024-09-21T11:58:45.876560+0000","last_became_active":"2024-09-21T11:56:36.507849+0000","last_became_peered":"2024-09-21T11:56:36.507849+0000","last_unstale":"2024-09-21T11:58:45.876560+0000","last_undegraded":"2024-09-21T11:58:45.876560+0000","last_fullsized":"2024-09-21T11:58:45.876560+0000","mapping_epoch":392,"log_start":"403'4304","ondisk_log_start":"403'4304","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5377,"log_dups_size":0,"ondisk_log_size":5377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.720402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356843520,"num_objects":5445,"num_object_clones":0,"num_object_copies":16335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9681,"num_write_kb":484032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":849,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532691+0000","last_change":"2024-09-21T11:57:43.928736+0000","last_active":"2024-09-21T11:58:45.532691+0000","last_peered":"2024-09-21T11:58:45.532691+0000","last_clean":"2024-09-21T11:58:45.532691+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:58:45.532691+0000","last_undegraded":"2024-09-21T11:58:45.532691+0000","last_fullsized":"2024-09-21T11:58:45.532691+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:57:43.928681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.928681+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:55.242338+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":342,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.539809+0000","last_change":"2024-09-21T11:58:17.529749+0000","last_active":"2024-09-21T11:58:46.539809+0000","last_peered":"2024-09-21T11:58:46.539809+0000","last_clean":"2024-09-21T11:58:46.539809+0000","last_became_active":"2024-09-21T11:58:17.529395+0000","last_became_peered":"2024-09-21T11:58:17.529395+0000","last_unstale":"2024-09-21T11:58:46.539809+0000","last_undegraded":"2024-09-21T11:58:46.539809+0000","last_fullsized":"2024-09-21T11:58:46.539809+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:05.663102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.663102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":837,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515940+0000","last_change":"2024-09-21T11:58:29.807560+0000","last_active":"2024-09-21T11:58:45.515940+0000","last_peered":"2024-09-21T11:58:45.515940+0000","last_clean":"2024-09-21T11:58:45.515940+0000","last_became_active":"2024-09-21T11:58:17.527945+0000","last_became_peered":"2024-09-21T11:58:17.527945+0000","last_unstale":"2024-09-21T11:58:45.515940+0000","last_undegraded":"2024-09-21T11:58:45.515940+0000","last_fullsized":"2024-09-21T11:58:45.515940+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:29.807504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:29.807504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:58.143758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":826,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531463+0000","last_change":"2024-09-21T11:56:28.556475+0000","last_active":"2024-09-21T11:58:45.531463+0000","last_peered":"2024-09-21T11:58:45.531463+0000","last_clean":"2024-09-21T11:58:45.531463+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:58:45.531463+0000","last_undegraded":"2024-09-21T11:58:45.531463+0000","last_fullsized":"2024-09-21T11:58:45.531463+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.556423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.556423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":75,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.514965+0000","last_change":"2024-09-21T11:57:54.931806+0000","last_active":"2024-09-21T11:58:45.514965+0000","last_peered":"2024-09-21T11:58:45.514965+0000","last_clean":"2024-09-21T11:58:45.514965+0000","last_became_active":"2024-09-21T11:57:45.534494+0000","last_became_peered":"2024-09-21T11:57:45.534494+0000","last_unstale":"2024-09-21T11:58:45.514965+0000","last_undegraded":"2024-09-21T11:58:45.514965+0000","last_fullsized":"2024-09-21T11:58:45.514965+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:54.931704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.931704+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:23.002342+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2","version":"442'9705","reported_seq":9817,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.906249+0000","last_change":"2024-09-21T11:56:36.511935+0000","last_active":"2024-09-21T11:58:46.906249+0000","last_peered":"2024-09-21T11:58:46.906249+0000","last_clean":"2024-09-21T11:58:46.906249+0000","last_became_active":"2024-09-21T11:56:36.511642+0000","last_became_peered":"2024-09-21T11:56:36.511642+0000","last_unstale":"2024-09-21T11:58:46.906249+0000","last_undegraded":"2024-09-21T11:58:46.906249+0000","last_fullsized":"2024-09-21T11:58:46.906249+0000","mapping_epoch":392,"log_start":"402'3497","ondisk_log_start":"402'3497","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:58.985148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":348979200,"num_objects":5325,"num_object_clones":0,"num_object_copies":15975,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9705,"num_write_kb":480960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":956,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561707+0000","last_change":"2024-09-21T11:58:17.555274+0000","last_active":"2024-09-21T11:58:46.561707+0000","last_peered":"2024-09-21T11:58:46.561707+0000","last_clean":"2024-09-21T11:58:46.561707+0000","last_became_active":"2024-09-21T11:58:17.527029+0000","last_became_peered":"2024-09-21T11:58:17.527029+0000","last_unstale":"2024-09-21T11:58:46.561707+0000","last_undegraded":"2024-09-21T11:58:46.561707+0000","last_fullsized":"2024-09-21T11:58:46.561707+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:56:03.667750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:03.667750+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:37.084768+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":853,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.534321+0000","last_change":"2024-09-21T11:56:28.045735+0000","last_active":"2024-09-21T11:58:45.534321+0000","last_peered":"2024-09-21T11:58:45.534321+0000","last_clean":"2024-09-21T11:58:45.534321+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:58:45.534321+0000","last_undegraded":"2024-09-21T11:58:45.534321+0000","last_fullsized":"2024-09-21T11:58:45.534321+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.045670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.045670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:01.924205+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":527,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562172+0000","last_change":"2024-09-21T11:57:42.599025+0000","last_active":"2024-09-21T11:58:46.562172+0000","last_peered":"2024-09-21T11:58:46.562172+0000","last_clean":"2024-09-21T11:58:46.562172+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:58:46.562172+0000","last_undegraded":"2024-09-21T11:58:46.562172+0000","last_fullsized":"2024-09-21T11:58:46.562172+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:42.598710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:42.598710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:59.154856+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":830,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561385+0000","last_change":"2024-09-21T11:57:48.898803+0000","last_active":"2024-09-21T11:58:46.561385+0000","last_peered":"2024-09-21T11:58:46.561385+0000","last_clean":"2024-09-21T11:58:46.561385+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:58:46.561385+0000","last_undegraded":"2024-09-21T11:58:46.561385+0000","last_fullsized":"2024-09-21T11:58:46.561385+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:48.898731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.898731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:52.841115+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":941,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.529716+0000","last_change":"2024-09-21T11:57:52.493051+0000","last_active":"2024-09-21T11:58:45.529716+0000","last_peered":"2024-09-21T11:58:45.529716+0000","last_clean":"2024-09-21T11:58:45.529716+0000","last_became_active":"2024-09-21T11:57:45.516610+0000","last_became_peered":"2024-09-21T11:57:45.516610+0000","last_unstale":"2024-09-21T11:58:45.529716+0000","last_undegraded":"2024-09-21T11:58:45.529716+0000","last_fullsized":"2024-09-21T11:58:45.529716+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:52.492971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:52.492971+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:03.920474+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3","version":"441'9679","reported_seq":9869,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.857354+0000","last_change":"2024-09-21T11:58:22.114392+0000","last_active":"2024-09-21T11:58:45.857354+0000","last_peered":"2024-09-21T11:58:45.857354+0000","last_clean":"2024-09-21T11:58:45.857354+0000","last_became_active":"2024-09-21T11:56:36.504516+0000","last_became_peered":"2024-09-21T11:56:36.504516+0000","last_unstale":"2024-09-21T11:58:45.857354+0000","last_undegraded":"2024-09-21T11:58:45.857354+0000","last_fullsized":"2024-09-21T11:58:45.857354+0000","mapping_epoch":392,"log_start":"403'4295","ondisk_log_start":"403'4295","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"432'8216","last_scrub_stamp":"2024-09-21T11:58:22.114283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:58:22.114283+0000","objects_scrubbed":7071,"log_size":5384,"log_dups_size":0,"ondisk_log_size":5384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:43.111443+0000","scrub_duration":2344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363528192,"num_objects":5547,"num_object_clones":0,"num_object_copies":16641,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9679,"num_write_kb":487232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":810,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531810+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:58:45.531810+0000","last_peered":"2024-09-21T11:58:45.531810+0000","last_clean":"2024-09-21T11:58:45.531810+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:58:45.531810+0000","last_undegraded":"2024-09-21T11:58:45.531810+0000","last_fullsized":"2024-09-21T11:58:45.531810+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":830,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531831+0000","last_change":"2024-09-21T11:57:58.535943+0000","last_active":"2024-09-21T11:58:45.531831+0000","last_peered":"2024-09-21T11:58:45.531831+0000","last_clean":"2024-09-21T11:58:45.531831+0000","last_became_active":"2024-09-21T11:57:58.535642+0000","last_became_peered":"2024-09-21T11:57:58.535642+0000","last_unstale":"2024-09-21T11:58:45.531831+0000","last_undegraded":"2024-09-21T11:58:45.531831+0000","last_fullsized":"2024-09-21T11:58:45.531831+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:09.486386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.486386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.896343+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1068,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561618+0000","last_change":"2024-09-21T11:56:08.680971+0000","last_active":"2024-09-21T11:58:46.561618+0000","last_peered":"2024-09-21T11:58:46.561618+0000","last_clean":"2024-09-21T11:58:46.561618+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:58:46.561618+0000","last_undegraded":"2024-09-21T11:58:46.561618+0000","last_fullsized":"2024-09-21T11:58:46.561618+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:56:08.680766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.680766+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:46.070618+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":661,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.563591+0000","last_change":"2024-09-21T11:58:17.532859+0000","last_active":"2024-09-21T11:58:46.563591+0000","last_peered":"2024-09-21T11:58:46.563591+0000","last_clean":"2024-09-21T11:58:46.563591+0000","last_became_active":"2024-09-21T11:58:17.531268+0000","last_became_peered":"2024-09-21T11:58:17.531268+0000","last_unstale":"2024-09-21T11:58:46.563591+0000","last_undegraded":"2024-09-21T11:58:46.563591+0000","last_fullsized":"2024-09-21T11:58:46.563591+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:51.665137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.665137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:57.566087+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"441'9612","reported_seq":9722,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.269940+0000","last_change":"2024-09-21T11:56:36.508655+0000","last_active":"2024-09-21T11:58:46.269940+0000","last_peered":"2024-09-21T11:58:46.269940+0000","last_clean":"2024-09-21T11:58:46.269940+0000","last_became_active":"2024-09-21T11:56:36.508299+0000","last_became_peered":"2024-09-21T11:56:36.508299+0000","last_unstale":"2024-09-21T11:58:46.269940+0000","last_undegraded":"2024-09-21T11:58:46.269940+0000","last_fullsized":"2024-09-21T11:58:46.269940+0000","mapping_epoch":392,"log_start":"406'4915","ondisk_log_start":"406'4915","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:12.333452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358350848,"num_objects":5468,"num_object_clones":0,"num_object_copies":16404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9612,"num_write_kb":482560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":872,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.567281+0000","last_change":"2024-09-21T11:56:48.169371+0000","last_active":"2024-09-21T11:58:46.567281+0000","last_peered":"2024-09-21T11:58:46.567281+0000","last_clean":"2024-09-21T11:58:46.567281+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:58:46.567281+0000","last_undegraded":"2024-09-21T11:58:46.567281+0000","last_fullsized":"2024-09-21T11:58:46.567281+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:48.168795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.168795+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":855,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515528+0000","last_change":"2024-09-21T11:56:59.989715+0000","last_active":"2024-09-21T11:58:45.515528+0000","last_peered":"2024-09-21T11:58:45.515528+0000","last_clean":"2024-09-21T11:58:45.515528+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:58:45.515528+0000","last_undegraded":"2024-09-21T11:58:45.515528+0000","last_fullsized":"2024-09-21T11:58:45.515528+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:59.989636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.989636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":832,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.560870+0000","last_change":"2024-09-21T11:58:21.147985+0000","last_active":"2024-09-21T11:58:46.560870+0000","last_peered":"2024-09-21T11:58:46.560870+0000","last_clean":"2024-09-21T11:58:46.560870+0000","last_became_active":"2024-09-21T11:58:17.525791+0000","last_became_peered":"2024-09-21T11:58:17.525791+0000","last_unstale":"2024-09-21T11:58:46.560870+0000","last_undegraded":"2024-09-21T11:58:46.560870+0000","last_fullsized":"2024-09-21T11:58:46.560870+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:21.147942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:21.147942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:41.780615+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":54,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.567681+0000","last_change":"2024-09-21T11:57:58.529616+0000","last_active":"2024-09-21T11:58:46.567681+0000","last_peered":"2024-09-21T11:58:46.567681+0000","last_clean":"2024-09-21T11:58:46.567681+0000","last_became_active":"2024-09-21T11:57:58.529278+0000","last_became_peered":"2024-09-21T11:57:58.529278+0000","last_unstale":"2024-09-21T11:58:46.567681+0000","last_undegraded":"2024-09-21T11:58:46.567681+0000","last_fullsized":"2024-09-21T11:58:46.567681+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:35.532577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:35.532577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"441'9519","reported_seq":9629,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.267220+0000","last_change":"2024-09-21T11:56:36.506979+0000","last_active":"2024-09-21T11:58:46.267220+0000","last_peered":"2024-09-21T11:58:46.267220+0000","last_clean":"2024-09-21T11:58:46.267220+0000","last_became_active":"2024-09-21T11:56:36.506694+0000","last_became_peered":"2024-09-21T11:56:36.506694+0000","last_unstale":"2024-09-21T11:58:46.267220+0000","last_undegraded":"2024-09-21T11:58:46.267220+0000","last_fullsized":"2024-09-21T11:58:46.267220+0000","mapping_epoch":392,"log_start":"406'4819","ondisk_log_start":"406'4819","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4700,"log_dups_size":0,"ondisk_log_size":4700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:03.357476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":355401728,"num_objects":5423,"num_object_clones":0,"num_object_copies":16269,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9519,"num_write_kb":478144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":936,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.534203+0000","last_change":"2024-09-21T11:58:29.624654+0000","last_active":"2024-09-21T11:58:45.534203+0000","last_peered":"2024-09-21T11:58:45.534203+0000","last_clean":"2024-09-21T11:58:45.534203+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:58:45.534203+0000","last_undegraded":"2024-09-21T11:58:45.534203+0000","last_fullsized":"2024-09-21T11:58:45.534203+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:58:29.624549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:29.624549+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:48.198468+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":823,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531104+0000","last_change":"2024-09-21T11:56:50.313230+0000","last_active":"2024-09-21T11:58:45.531104+0000","last_peered":"2024-09-21T11:58:45.531104+0000","last_clean":"2024-09-21T11:58:45.531104+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:58:45.531104+0000","last_undegraded":"2024-09-21T11:58:45.531104+0000","last_fullsized":"2024-09-21T11:58:45.531104+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.313165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.313165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.295853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":834,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.560346+0000","last_change":"2024-09-21T11:58:28.041091+0000","last_active":"2024-09-21T11:58:46.560346+0000","last_peered":"2024-09-21T11:58:46.560346+0000","last_clean":"2024-09-21T11:58:46.560346+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:58:46.560346+0000","last_undegraded":"2024-09-21T11:58:46.560346+0000","last_fullsized":"2024-09-21T11:58:46.560346+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:28.041026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.041026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:52.616083+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":827,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.529583+0000","last_change":"2024-09-21T11:56:55.414105+0000","last_active":"2024-09-21T11:58:45.529583+0000","last_peered":"2024-09-21T11:58:45.529583+0000","last_clean":"2024-09-21T11:58:45.529583+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:58:45.529583+0000","last_undegraded":"2024-09-21T11:58:45.529583+0000","last_fullsized":"2024-09-21T11:58:45.529583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:55.414046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:55.414046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:05.330948+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"442'9746","reported_seq":20396,"reported_epoch":442,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:58:46.774691+0000","last_change":"2024-09-21T11:58:17.536432+0000","last_active":"2024-09-21T11:58:46.774691+0000","last_peered":"2024-09-21T11:58:46.774691+0000","last_clean":"2024-09-21T11:57:57.476661+0000","last_became_active":"2024-09-21T11:57:59.988777+0000","last_became_peered":"2024-09-21T11:57:59.988777+0000","last_unstale":"2024-09-21T11:58:46.774691+0000","last_undegraded":"2024-09-21T11:58:46.774691+0000","last_fullsized":"2024-09-21T11:58:46.774691+0000","mapping_epoch":421,"log_start":"406'4915","ondisk_log_start":"406'4915","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4831,"log_dups_size":0,"ondisk_log_size":4831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356515840,"num_objects":5440,"num_object_clones":0,"num_object_copies":16320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1209,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9746,"num_write_kb":485952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5266,"num_bytes_recovered":345112576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[1,2,4],"avail_no_missing":["1","2","4"],"object_location_counts":[{"shards":"1,2,4","objects":5440}],"blocked_by":[],"up_primary":0,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":852,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562342+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:58:46.562342+0000","last_peered":"2024-09-21T11:58:46.562342+0000","last_clean":"2024-09-21T11:58:46.562342+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:58:46.562342+0000","last_undegraded":"2024-09-21T11:58:46.562342+0000","last_fullsized":"2024-09-21T11:58:46.562342+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":890,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531057+0000","last_change":"2024-09-21T11:58:18.420036+0000","last_active":"2024-09-21T11:58:45.531057+0000","last_peered":"2024-09-21T11:58:45.531057+0000","last_clean":"2024-09-21T11:58:45.531057+0000","last_became_active":"2024-09-21T11:58:17.530676+0000","last_became_peered":"2024-09-21T11:58:17.530676+0000","last_unstale":"2024-09-21T11:58:45.531057+0000","last_undegraded":"2024-09-21T11:58:45.531057+0000","last_fullsized":"2024-09-21T11:58:45.531057+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:18.419990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:58:18.419990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:25.445566+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":840,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.539663+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:58:46.539663+0000","last_peered":"2024-09-21T11:58:46.539663+0000","last_clean":"2024-09-21T11:58:46.539663+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:58:46.539663+0000","last_undegraded":"2024-09-21T11:58:46.539663+0000","last_fullsized":"2024-09-21T11:58:46.539663+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.052833+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":826,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531498+0000","last_change":"2024-09-21T11:56:02.541452+0000","last_active":"2024-09-21T11:58:45.531498+0000","last_peered":"2024-09-21T11:58:45.531498+0000","last_clean":"2024-09-21T11:58:45.531498+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:58:45.531498+0000","last_undegraded":"2024-09-21T11:58:45.531498+0000","last_fullsized":"2024-09-21T11:58:45.531498+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.541232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.541232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:09.854321+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"441'9432","reported_seq":9541,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.446150+0000","last_change":"2024-09-21T11:56:36.624091+0000","last_active":"2024-09-21T11:58:46.446150+0000","last_peered":"2024-09-21T11:58:46.446150+0000","last_clean":"2024-09-21T11:58:46.446150+0000","last_became_active":"2024-09-21T11:56:36.623765+0000","last_became_peered":"2024-09-21T11:56:36.623765+0000","last_unstale":"2024-09-21T11:58:46.446150+0000","last_undegraded":"2024-09-21T11:58:46.446150+0000","last_fullsized":"2024-09-21T11:58:46.446150+0000","mapping_epoch":392,"log_start":"403'4199","ondisk_log_start":"403'4199","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5233,"log_dups_size":0,"ondisk_log_size":5233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:03.772665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":354156544,"num_objects":5404,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9432,"num_write_kb":474752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":870,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515088+0000","last_change":"2024-09-21T11:56:29.063211+0000","last_active":"2024-09-21T11:58:45.515088+0000","last_peered":"2024-09-21T11:58:45.515088+0000","last_clean":"2024-09-21T11:58:45.515088+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:58:45.515088+0000","last_undegraded":"2024-09-21T11:58:45.515088+0000","last_fullsized":"2024-09-21T11:58:45.515088+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:29.063150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.063150+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:02.807388+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":885,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531122+0000","last_change":"2024-09-21T11:57:58.539544+0000","last_active":"2024-09-21T11:58:45.531122+0000","last_peered":"2024-09-21T11:58:45.531122+0000","last_clean":"2024-09-21T11:58:45.531122+0000","last_became_active":"2024-09-21T11:57:58.528814+0000","last_became_peered":"2024-09-21T11:57:58.528814+0000","last_unstale":"2024-09-21T11:58:45.531122+0000","last_undegraded":"2024-09-21T11:58:45.531122+0000","last_fullsized":"2024-09-21T11:58:45.531122+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:57:01.221960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.221960+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.207576+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1065,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.529617+0000","last_change":"2024-09-21T11:57:10.282180+0000","last_active":"2024-09-21T11:58:45.529617+0000","last_peered":"2024-09-21T11:58:45.529617+0000","last_clean":"2024-09-21T11:58:45.529617+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:58:45.529617+0000","last_undegraded":"2024-09-21T11:58:45.529617+0000","last_fullsized":"2024-09-21T11:58:45.529617+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:10.282134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.282134+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:14.298512+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":811,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515234+0000","last_change":"2024-09-21T11:57:41.791340+0000","last_active":"2024-09-21T11:58:45.515234+0000","last_peered":"2024-09-21T11:58:45.515234+0000","last_clean":"2024-09-21T11:58:45.515234+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:58:45.515234+0000","last_undegraded":"2024-09-21T11:58:45.515234+0000","last_fullsized":"2024-09-21T11:58:45.515234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:41.791261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:41.791261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:46.401133+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":861,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.534134+0000","last_change":"2024-09-21T11:57:58.527427+0000","last_active":"2024-09-21T11:58:45.534134+0000","last_peered":"2024-09-21T11:58:45.534134+0000","last_clean":"2024-09-21T11:58:45.534134+0000","last_became_active":"2024-09-21T11:57:58.527156+0000","last_became_peered":"2024-09-21T11:57:58.527156+0000","last_unstale":"2024-09-21T11:58:45.534134+0000","last_undegraded":"2024-09-21T11:58:45.534134+0000","last_fullsized":"2024-09-21T11:58:45.534134+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:38.938666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:38.938666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:49.086819+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":859,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532351+0000","last_change":"2024-09-21T11:56:29.528662+0000","last_active":"2024-09-21T11:58:45.532351+0000","last_peered":"2024-09-21T11:58:45.532351+0000","last_clean":"2024-09-21T11:58:45.532351+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:58:45.532351+0000","last_undegraded":"2024-09-21T11:58:45.532351+0000","last_fullsized":"2024-09-21T11:58:45.532351+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:56:29.528598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.528598+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":860,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515687+0000","last_change":"2024-09-21T11:57:15.919990+0000","last_active":"2024-09-21T11:58:45.515687+0000","last_peered":"2024-09-21T11:58:45.515687+0000","last_clean":"2024-09-21T11:58:45.515687+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:58:45.515687+0000","last_undegraded":"2024-09-21T11:58:45.515687+0000","last_fullsized":"2024-09-21T11:58:45.515687+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:15.919902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:15.919902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":856,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.530281+0000","last_change":"2024-09-21T11:58:20.347779+0000","last_active":"2024-09-21T11:58:45.530281+0000","last_peered":"2024-09-21T11:58:45.530281+0000","last_clean":"2024-09-21T11:58:45.530281+0000","last_became_active":"2024-09-21T11:58:17.529420+0000","last_became_peered":"2024-09-21T11:58:17.529420+0000","last_unstale":"2024-09-21T11:58:45.530281+0000","last_undegraded":"2024-09-21T11:58:45.530281+0000","last_fullsized":"2024-09-21T11:58:45.530281+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:20.347736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:20.347736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:47.925674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":854,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.569330+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:58:46.569330+0000","last_peered":"2024-09-21T11:58:46.569330+0000","last_clean":"2024-09-21T11:58:46.569330+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:58:46.569330+0000","last_undegraded":"2024-09-21T11:58:46.569330+0000","last_fullsized":"2024-09-21T11:58:46.569330+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":815,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.529947+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:58:45.529947+0000","last_peered":"2024-09-21T11:58:45.529947+0000","last_clean":"2024-09-21T11:58:45.529947+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:58:45.529947+0000","last_undegraded":"2024-09-21T11:58:45.529947+0000","last_fullsized":"2024-09-21T11:58:45.529947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.174525+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":845,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532158+0000","last_change":"2024-09-21T11:56:50.696344+0000","last_active":"2024-09-21T11:58:45.532158+0000","last_peered":"2024-09-21T11:58:45.532158+0000","last_clean":"2024-09-21T11:58:45.532158+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:58:45.532158+0000","last_undegraded":"2024-09-21T11:58:45.532158+0000","last_fullsized":"2024-09-21T11:58:45.532158+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.696253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.696253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":822,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.560225+0000","last_change":"2024-09-21T11:57:12.518014+0000","last_active":"2024-09-21T11:58:46.560225+0000","last_peered":"2024-09-21T11:58:46.560225+0000","last_clean":"2024-09-21T11:58:46.560225+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:58:46.560225+0000","last_undegraded":"2024-09-21T11:58:46.560225+0000","last_fullsized":"2024-09-21T11:58:46.560225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:12.517953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:12.517953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":851,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515682+0000","last_change":"2024-09-21T11:56:34.119200+0000","last_active":"2024-09-21T11:58:45.515682+0000","last_peered":"2024-09-21T11:58:45.515682+0000","last_clean":"2024-09-21T11:58:45.515682+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:58:45.515682+0000","last_undegraded":"2024-09-21T11:58:45.515682+0000","last_fullsized":"2024-09-21T11:58:45.515682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:34.119128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:34.119128+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:00.786471+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":849,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.563979+0000","last_change":"2024-09-21T11:57:49.993421+0000","last_active":"2024-09-21T11:58:46.563979+0000","last_peered":"2024-09-21T11:58:46.563979+0000","last_clean":"2024-09-21T11:58:46.563979+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:58:46.563979+0000","last_undegraded":"2024-09-21T11:58:46.563979+0000","last_fullsized":"2024-09-21T11:58:46.563979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:49.993339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:49.993339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:52.674944+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1033,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531679+0000","last_change":"2024-09-21T11:55:51.317564+0000","last_active":"2024-09-21T11:58:45.531679+0000","last_peered":"2024-09-21T11:58:45.531679+0000","last_clean":"2024-09-21T11:58:45.531679+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:58:45.531679+0000","last_undegraded":"2024-09-21T11:58:45.531679+0000","last_fullsized":"2024-09-21T11:58:45.531679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:51.317473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.317473+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:04.938497+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":319,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561296+0000","last_change":"2024-09-21T11:58:20.129924+0000","last_active":"2024-09-21T11:58:46.561296+0000","last_peered":"2024-09-21T11:58:46.561296+0000","last_clean":"2024-09-21T11:58:46.561296+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:58:46.561296+0000","last_undegraded":"2024-09-21T11:58:46.561296+0000","last_fullsized":"2024-09-21T11:58:46.561296+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:20.129854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:20.129854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:38.523985+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":879,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531907+0000","last_change":"2024-09-21T11:57:16.786845+0000","last_active":"2024-09-21T11:58:45.531907+0000","last_peered":"2024-09-21T11:58:45.531907+0000","last_clean":"2024-09-21T11:58:45.531907+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:58:45.531907+0000","last_undegraded":"2024-09-21T11:58:45.531907+0000","last_fullsized":"2024-09-21T11:58:45.531907+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:57:16.786756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:16.786756+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":831,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531297+0000","last_change":"2024-09-21T11:57:40.391494+0000","last_active":"2024-09-21T11:58:45.531297+0000","last_peered":"2024-09-21T11:58:45.531297+0000","last_clean":"2024-09-21T11:58:45.531297+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:58:45.531297+0000","last_undegraded":"2024-09-21T11:58:45.531297+0000","last_fullsized":"2024-09-21T11:58:45.531297+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:40.391409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:40.391409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:08.001729+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":1087,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515633+0000","last_change":"2024-09-21T11:56:56.051487+0000","last_active":"2024-09-21T11:58:45.515633+0000","last_peered":"2024-09-21T11:58:45.515633+0000","last_clean":"2024-09-21T11:58:45.515633+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:58:45.515633+0000","last_undegraded":"2024-09-21T11:58:45.515633+0000","last_fullsized":"2024-09-21T11:58:45.515633+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:56.051300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:56.051300+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.828079+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":54,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.568596+0000","last_change":"2024-09-21T11:57:58.534490+0000","last_active":"2024-09-21T11:58:46.568596+0000","last_peered":"2024-09-21T11:58:46.568596+0000","last_clean":"2024-09-21T11:58:46.568596+0000","last_became_active":"2024-09-21T11:57:58.533810+0000","last_became_peered":"2024-09-21T11:57:58.533810+0000","last_unstale":"2024-09-21T11:58:46.568596+0000","last_undegraded":"2024-09-21T11:58:46.568596+0000","last_fullsized":"2024-09-21T11:58:46.568596+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.280698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.280698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":868,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.532842+0000","last_change":"2024-09-21T11:58:28.658668+0000","last_active":"2024-09-21T11:58:45.532842+0000","last_peered":"2024-09-21T11:58:45.532842+0000","last_clean":"2024-09-21T11:58:45.532842+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:58:45.532842+0000","last_undegraded":"2024-09-21T11:58:45.532842+0000","last_fullsized":"2024-09-21T11:58:45.532842+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:28.658574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.658574+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:54.766743+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":326,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531370+0000","last_change":"2024-09-21T11:55:24.538954+0000","last_active":"2024-09-21T11:58:45.531370+0000","last_peered":"2024-09-21T11:58:45.531370+0000","last_clean":"2024-09-21T11:58:45.531370+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:58:45.531370+0000","last_undegraded":"2024-09-21T11:58:45.531370+0000","last_fullsized":"2024-09-21T11:58:45.531370+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.538880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.538880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:56.460358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":527,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.575155+0000","last_change":"2024-09-21T11:56:00.441644+0000","last_active":"2024-09-21T11:58:46.575155+0000","last_peered":"2024-09-21T11:58:46.575155+0000","last_clean":"2024-09-21T11:58:46.575155+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:58:46.575155+0000","last_undegraded":"2024-09-21T11:58:46.575155+0000","last_fullsized":"2024-09-21T11:58:46.575155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:00.441565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:00.441565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":818,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.535274+0000","last_change":"2024-09-21T11:57:58.531539+0000","last_active":"2024-09-21T11:58:45.535274+0000","last_peered":"2024-09-21T11:58:45.535274+0000","last_clean":"2024-09-21T11:58:45.535274+0000","last_became_active":"2024-09-21T11:57:58.531213+0000","last_became_peered":"2024-09-21T11:57:58.531213+0000","last_unstale":"2024-09-21T11:58:45.535274+0000","last_undegraded":"2024-09-21T11:58:45.535274+0000","last_fullsized":"2024-09-21T11:58:45.535274+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:01.434123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.434123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:02.942881+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":538,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.575419+0000","last_change":"2024-09-21T11:55:59.419239+0000","last_active":"2024-09-21T11:58:46.575419+0000","last_peered":"2024-09-21T11:58:46.575419+0000","last_clean":"2024-09-21T11:58:46.575419+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:58:46.575419+0000","last_undegraded":"2024-09-21T11:58:46.575419+0000","last_fullsized":"2024-09-21T11:58:46.575419+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:59.419148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.419148+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":822,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.530959+0000","last_change":"2024-09-21T11:57:17.492529+0000","last_active":"2024-09-21T11:58:45.530959+0000","last_peered":"2024-09-21T11:58:45.530959+0000","last_clean":"2024-09-21T11:58:45.530959+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:58:45.530959+0000","last_undegraded":"2024-09-21T11:58:45.530959+0000","last_fullsized":"2024-09-21T11:58:45.530959+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:17.492466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:17.492466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:37.151802+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":829,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531592+0000","last_change":"2024-09-21T11:58:24.939301+0000","last_active":"2024-09-21T11:58:45.531592+0000","last_peered":"2024-09-21T11:58:45.531592+0000","last_clean":"2024-09-21T11:58:45.531592+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:58:45.531592+0000","last_undegraded":"2024-09-21T11:58:45.531592+0000","last_fullsized":"2024-09-21T11:58:45.531592+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:24.939230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:24.939230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.727906+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":860,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.564938+0000","last_change":"2024-09-21T11:57:37.071994+0000","last_active":"2024-09-21T11:58:46.564938+0000","last_peered":"2024-09-21T11:58:46.564938+0000","last_clean":"2024-09-21T11:58:46.564938+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:58:46.564938+0000","last_undegraded":"2024-09-21T11:58:46.564938+0000","last_fullsized":"2024-09-21T11:58:46.564938+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:57:37.071908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.071908+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":859,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.536283+0000","last_change":"2024-09-21T11:57:18.759800+0000","last_active":"2024-09-21T11:58:45.536283+0000","last_peered":"2024-09-21T11:58:45.536283+0000","last_clean":"2024-09-21T11:58:45.536283+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:58:45.536283+0000","last_undegraded":"2024-09-21T11:58:45.536283+0000","last_fullsized":"2024-09-21T11:58:45.536283+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:57:18.759710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:18.759710+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":844,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.515766+0000","last_change":"2024-09-21T11:56:47.967479+0000","last_active":"2024-09-21T11:58:45.515766+0000","last_peered":"2024-09-21T11:58:45.515766+0000","last_clean":"2024-09-21T11:58:45.515766+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:58:45.515766+0000","last_undegraded":"2024-09-21T11:58:45.515766+0000","last_fullsized":"2024-09-21T11:58:45.515766+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:47.967392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:47.967392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:15.865477+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":807,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.529340+0000","last_change":"2024-09-21T11:58:23.430651+0000","last_active":"2024-09-21T11:58:45.529340+0000","last_peered":"2024-09-21T11:58:45.529340+0000","last_clean":"2024-09-21T11:58:45.529340+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:58:45.529340+0000","last_undegraded":"2024-09-21T11:58:45.529340+0000","last_fullsized":"2024-09-21T11:58:45.529340+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:23.430517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:23.430517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:25.920377+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":833,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.531745+0000","last_change":"2024-09-21T11:57:58.537818+0000","last_active":"2024-09-21T11:58:45.531745+0000","last_peered":"2024-09-21T11:58:45.531745+0000","last_clean":"2024-09-21T11:58:45.531745+0000","last_became_active":"2024-09-21T11:57:58.537121+0000","last_became_peered":"2024-09-21T11:57:58.537121+0000","last_unstale":"2024-09-21T11:58:45.531745+0000","last_undegraded":"2024-09-21T11:58:45.531745+0000","last_fullsized":"2024-09-21T11:58:45.531745+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:43.803926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.803926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:02.513607+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":863,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.542531+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:58:45.542531+0000","last_peered":"2024-09-21T11:58:45.542531+0000","last_clean":"2024-09-21T11:58:45.542531+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:58:45.542531+0000","last_undegraded":"2024-09-21T11:58:45.542531+0000","last_fullsized":"2024-09-21T11:58:45.542531+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":816,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.567147+0000","last_change":"2024-09-21T11:56:53.232038+0000","last_active":"2024-09-21T11:58:46.567147+0000","last_peered":"2024-09-21T11:58:46.567147+0000","last_clean":"2024-09-21T11:58:46.567147+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:58:46.567147+0000","last_undegraded":"2024-09-21T11:58:46.567147+0000","last_fullsized":"2024-09-21T11:58:46.567147+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.231963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.231963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":837,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.535559+0000","last_change":"2024-09-21T11:58:17.528224+0000","last_active":"2024-09-21T11:58:45.535559+0000","last_peered":"2024-09-21T11:58:45.535559+0000","last_clean":"2024-09-21T11:58:45.535559+0000","last_became_active":"2024-09-21T11:58:17.527743+0000","last_became_peered":"2024-09-21T11:58:17.527743+0000","last_unstale":"2024-09-21T11:58:45.535559+0000","last_undegraded":"2024-09-21T11:58:45.535559+0000","last_fullsized":"2024-09-21T11:58:45.535559+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:41.911361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:41.911361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:04.639774+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":826,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.529989+0000","last_change":"2024-09-21T11:57:39.362771+0000","last_active":"2024-09-21T11:58:45.529989+0000","last_peered":"2024-09-21T11:58:45.529989+0000","last_clean":"2024-09-21T11:58:45.529989+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:58:45.529989+0000","last_undegraded":"2024-09-21T11:58:45.529989+0000","last_fullsized":"2024-09-21T11:58:45.529989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:39.362713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:39.362713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:53.366779+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":532,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.549382+0000","last_change":"2024-09-21T11:58:17.532076+0000","last_active":"2024-09-21T11:58:46.549382+0000","last_peered":"2024-09-21T11:58:46.549382+0000","last_clean":"2024-09-21T11:58:46.549382+0000","last_became_active":"2024-09-21T11:58:17.531567+0000","last_became_peered":"2024-09-21T11:58:17.531567+0000","last_unstale":"2024-09-21T11:58:46.549382+0000","last_undegraded":"2024-09-21T11:58:46.549382+0000","last_fullsized":"2024-09-21T11:58:46.549382+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":859,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.542734+0000","last_change":"2024-09-21T11:57:58.536812+0000","last_active":"2024-09-21T11:58:45.542734+0000","last_peered":"2024-09-21T11:58:45.542734+0000","last_clean":"2024-09-21T11:58:45.542734+0000","last_became_active":"2024-09-21T11:57:58.535015+0000","last_became_peered":"2024-09-21T11:57:58.535015+0000","last_unstale":"2024-09-21T11:58:45.542734+0000","last_undegraded":"2024-09-21T11:58:45.542734+0000","last_fullsized":"2024-09-21T11:58:45.542734+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:39.735027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:39.735027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":929,"reported_epoch":441,"state":"active+clean","last_fresh":"2024-09-21T11:58:45.533117+0000","last_change":"2024-09-21T11:58:16.743891+0000","last_active":"2024-09-21T11:58:45.533117+0000","last_peered":"2024-09-21T11:58:45.533117+0000","last_clean":"2024-09-21T11:58:45.533117+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:58:45.533117+0000","last_undegraded":"2024-09-21T11:58:45.533117+0000","last_fullsized":"2024-09-21T11:58:45.533117+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:16.743830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:16.743830+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.512604+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":826,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.568919+0000","last_change":"2024-09-21T11:57:48.045901+0000","last_active":"2024-09-21T11:58:46.568919+0000","last_peered":"2024-09-21T11:58:46.568919+0000","last_clean":"2024-09-21T11:58:46.568919+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:58:46.568919+0000","last_undegraded":"2024-09-21T11:58:46.568919+0000","last_fullsized":"2024-09-21T11:58:46.568919+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:48.045820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.045820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:49.527330+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":5736628248,"num_objects":87535,"num_object_clones":0,"num_object_copies":262605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":16250,"num_objects_unfound":0,"num_objects_dirty":87535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154931,"num_write_kb":7758849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6822,"num_bytes_recovered":447021056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17584173056,"data_stored":17584160840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80478,"ondisk_log_size":80478,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":6904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1116,"num_read_kb":1111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234214,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3233272,"kb_used_data":2471848,"kb_used_omap":1,"kb_used_meta":761406,"kb_avail":90511880,"statfs":{"total":95995035648,"available":92684165120,"internally_reserved":0,"allocated":2531172352,"data_stored":2530556722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":779680202},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463147,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3446132,"kb_used_data":2819296,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":90299020,"statfs":{"total":95995035648,"available":92466196480,"internally_reserved":0,"allocated":2886959104,"data_stored":2886352034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":641792454},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724784,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2062472,"kb_used_data":1399604,"kb_used_omap":1,"kb_used_meta":662846,"kb_avail":91682680,"statfs":{"total":95995035648,"available":93883064320,"internally_reserved":0,"allocated":1433194496,"data_stored":1432566584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":678754758},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733921014,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1629376,"kb_used_data":1060844,"kb_used_omap":1,"kb_used_meta":568446,"kb_avail":92115776,"statfs":{"total":95995035648,"available":94326554624,"internally_reserved":0,"allocated":1086304256,"data_stored":1085677889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":582089157},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117244,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3271020,"kb_used_data":2473308,"kb_used_omap":1,"kb_used_meta":797502,"kb_avail":90474132,"statfs":{"total":95995035648,"available":92645511168,"internally_reserved":0,"allocated":2532667392,"data_stored":2532056754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":816642503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411586,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2882372,"kb_used_data":2111152,"kb_used_omap":1,"kb_used_meta":771134,"kb_avail":90862780,"statfs":{"total":95995035648,"available":93043486720,"internally_reserved":0,"allocated":2161819648,"data_stored":2161196443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":789641669},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575113,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3899048,"kb_used_data":3159696,"kb_used_omap":1,"kb_used_meta":739198,"kb_avail":89846104,"statfs":{"total":95995035648,"available":92002410496,"internally_reserved":0,"allocated":3235528704,"data_stored":3234879823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":756939206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738639,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2503388,"kb_used_data":1750276,"kb_used_omap":1,"kb_used_meta":753086,"kb_avail":91241764,"statfs":{"total":95995035648,"available":93431566336,"internally_reserved":0,"allocated":1792282624,"data_stored":1791646660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":771160517},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1797652480,"data_stored":1797652480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3230007296,"data_stored":3230007296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2142769152,"data_stored":2142765080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2510815232,"data_stored":2510815232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1076695040,"data_stored":1076690968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1422852096,"data_stored":1422852096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2881818624,"data_stored":2881814552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2521563136,"data_stored":2521563136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:58:53.947 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:58:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:53 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:58:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:53 smithi175 ceph-mon[32613]: pgmap v923: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 14078/248979 objects misplaced (5.654%); 18 MiB/s, 293 objects/s recovering 2024-09-21T11:58:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:53 smithi175 ceph-mon[32613]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-21T11:58:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:53 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:58:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:53 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:58:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:53 smithi175 ceph-mon[32613]: osdmap e443: 8 total, 8 up, 8 in 2024-09-21T11:58:54.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:54.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:54.057 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:54.057 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:54.158 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:54.183 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:54.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:54.185 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:54.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:54.338 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:54.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:54.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:54.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:54.482 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:54.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:54.483 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:54.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:54.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:54.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:54.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:54.761 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:54.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:54.762 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:54.852 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:54 smithi043 ceph-mon[36454]: 6.12 scrub starts 2024-09-21T11:58:54.853 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:54 smithi043 ceph-mon[36454]: 6.12 scrub ok 2024-09-21T11:58:54.853 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:54 smithi043 ceph-mon[29710]: 6.12 scrub starts 2024-09-21T11:58:54.853 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:54 smithi043 ceph-mon[29710]: 6.12 scrub ok 2024-09-21T11:58:54.863 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:54.889 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:54.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:54.890 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:54.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:55.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:54 smithi175 ceph-mon[32613]: 6.12 scrub starts 2024-09-21T11:58:55.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:54 smithi175 ceph-mon[32613]: 6.12 scrub ok 2024-09-21T11:58:55.015 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:55.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:55.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:55.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:55.162 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:55.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:55.163 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:55.263 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:55.286 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:55.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:55.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:55.288 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:58:55.288 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:58:55.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:55.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:55.438 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:55.438 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:55.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:58:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:58:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:58:55.539 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:55.560 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:55.568 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:55.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:55.569 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:55.670 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:55.699 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:55.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:55.700 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:55.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:55.826 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:55.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:55.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:55.927 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:55.958 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:55.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:55.959 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:55.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:55 smithi043 ceph-mon[36454]: pgmap v925: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 953 op/s; 14078/248979 objects misplaced (5.654%); 14 MiB/s, 227 objects/s recovering 2024-09-21T11:58:55.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:55 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T11:58:55.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:55 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T11:58:55.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:55 smithi043 ceph-mon[29710]: pgmap v925: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 953 op/s; 14078/248979 objects misplaced (5.654%); 14 MiB/s, 227 objects/s recovering 2024-09-21T11:58:55.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:55 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T11:58:55.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:55 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T11:58:56.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:55 smithi175 ceph-mon[32613]: pgmap v925: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 5.1 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 953 op/s; 14078/248979 objects misplaced (5.654%); 14 MiB/s, 227 objects/s recovering 2024-09-21T11:58:56.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:55 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T11:58:56.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:55 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T11:58:56.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:56.093 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:56.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:56.093 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:58:56.113 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:56.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T11:58:56.194 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:56.225 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:56.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:56.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:56.326 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:56.356 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:56.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:56.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:56.458 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:56.484 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:56.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:56.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:56.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:56.615 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:56.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:56.616 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:56.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:56.764 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:56.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:56.766 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:56.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:58:56.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:56.967 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:56.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:56.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:57.069 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:57.093 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:57.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:57.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:57.194 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:57.221 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:57.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:57.222 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:57.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:57.387 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:58:57.387 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:58:57.413 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:58:57.416 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:58:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:57.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:57.437 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:57.452 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:58:57.452 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:58:57.484 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:58:57.484 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:58:57.507 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:58:57.507 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:58:57.536 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:58:57.536 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:58:57.537 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:58:57.538 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:57.566 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:57.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:57.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:57.577 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:58:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:58:57.617 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:58:57.617 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:58:57.653 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:58:57.653 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:58:57.668 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:57.695 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:57.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:57.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:57.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:57.839 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:57.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:57.840 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:57.940 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[36454]: pgmap v926: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 4.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 530 B/s rd, 0 B/s wr, 854 op/s; 13481/243585 objects misplaced (5.534%); 14 MiB/s, 230 objects/s recovering 2024-09-21T11:58:57.940 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[36454]: 3.14 scrub starts 2024-09-21T11:58:57.940 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[36454]: 3.14 scrub ok 2024-09-21T11:58:57.940 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[36454]: 4.e scrub starts 2024-09-21T11:58:57.940 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[36454]: 4.e scrub ok 2024-09-21T11:58:57.940 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T11:58:57.940 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T11:58:57.940 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1930070608' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:57.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[29710]: pgmap v926: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 4.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 530 B/s rd, 0 B/s wr, 854 op/s; 13481/243585 objects misplaced (5.534%); 14 MiB/s, 230 objects/s recovering 2024-09-21T11:58:57.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[29710]: 3.14 scrub starts 2024-09-21T11:58:57.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[29710]: 3.14 scrub ok 2024-09-21T11:58:57.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[29710]: 4.e scrub starts 2024-09-21T11:58:57.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[29710]: 4.e scrub ok 2024-09-21T11:58:57.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T11:58:57.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T11:58:57.941 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1930070608' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:57.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:57.967 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:57.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:57.968 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:57 smithi175 ceph-mon[32613]: pgmap v926: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 4.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 530 B/s rd, 0 B/s wr, 854 op/s; 13481/243585 objects misplaced (5.534%); 14 MiB/s, 230 objects/s recovering 2024-09-21T11:58:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:57 smithi175 ceph-mon[32613]: 3.14 scrub starts 2024-09-21T11:58:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:57 smithi175 ceph-mon[32613]: 3.14 scrub ok 2024-09-21T11:58:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:57 smithi175 ceph-mon[32613]: 4.e scrub starts 2024-09-21T11:58:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:57 smithi175 ceph-mon[32613]: 4.e scrub ok 2024-09-21T11:58:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:57 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T11:58:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:57 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T11:58:58.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:57 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1930070608' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:58:58.070 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:58.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:58.099 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:58.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:58.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:58.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:58.242 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:58.342 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:58.368 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:58.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:58.369 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:58.470 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:58.502 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:58.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:58.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:58.604 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:58.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:58.645 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:58.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:58.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:58.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:58.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:58.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:58:58.915 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:58.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:58.916 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:58:58.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:58 smithi043 ceph-mon[36454]: from='client.26185 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:58:58.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:58 smithi043 ceph-mon[29710]: from='client.26185 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:58:59.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:58 smithi175 ceph-mon[32613]: from='client.26185 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:58:59.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:58:59.046 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:59.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:59.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:58:59.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:59.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:59.195 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:59.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:59.296 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:58:59.340 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:59.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:59.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:58:59.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:58:59.475 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:59.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:59.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:58:59.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:58:59.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:59.619 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:59.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:58:59.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:58:59.743 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:58:59.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:59.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:58:59.844 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:58:59.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:58:59.880 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:58:59.880 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:58:59.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:58:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:59 smithi043 ceph-mon[36454]: pgmap v927: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 969 op/s; 11783/234135 objects misplaced (5.033%); 18 MiB/s, 282 objects/s recovering 2024-09-21T11:58:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:59 smithi043 ceph-mon[36454]: 5.1b scrub starts 2024-09-21T11:58:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:58:59 smithi043 ceph-mon[36454]: 5.1b scrub ok 2024-09-21T11:58:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:59 smithi043 ceph-mon[29710]: pgmap v927: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 969 op/s; 11783/234135 objects misplaced (5.033%); 18 MiB/s, 282 objects/s recovering 2024-09-21T11:58:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:59 smithi043 ceph-mon[29710]: 5.1b scrub starts 2024-09-21T11:58:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:58:59 smithi043 ceph-mon[29710]: 5.1b scrub ok 2024-09-21T11:59:00.005 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:00.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:00.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:00.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:59 smithi175 ceph-mon[32613]: pgmap v927: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 969 op/s; 11783/234135 objects misplaced (5.033%); 18 MiB/s, 282 objects/s recovering 2024-09-21T11:59:00.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:59 smithi175 ceph-mon[32613]: 5.1b scrub starts 2024-09-21T11:59:00.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:58:59 smithi175 ceph-mon[32613]: 5.1b scrub ok 2024-09-21T11:59:00.107 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:00.134 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:00.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:00.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:00.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:00.281 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:00.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:00.282 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:00.382 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:00.406 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:00.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:00.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:00.508 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:00.534 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:00.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:00.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:00.635 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:00.661 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:00.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:00.662 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:00.763 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:00.794 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:00.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:00.795 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:00.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:00.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:00.933 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:00.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:00.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:00 smithi043 ceph-mon[36454]: 11.2 scrub starts 2024-09-21T11:59:00.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:00 smithi043 ceph-mon[36454]: 2.2 scrub starts 2024-09-21T11:59:00.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:00 smithi043 ceph-mon[36454]: 2.2 scrub ok 2024-09-21T11:59:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:00 smithi043 ceph-mon[29710]: 11.2 scrub starts 2024-09-21T11:59:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:00 smithi043 ceph-mon[29710]: 2.2 scrub starts 2024-09-21T11:59:00.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:00 smithi043 ceph-mon[29710]: 2.2 scrub ok 2024-09-21T11:59:01.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:00 smithi175 ceph-mon[32613]: 11.2 scrub starts 2024-09-21T11:59:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:00 smithi175 ceph-mon[32613]: 2.2 scrub starts 2024-09-21T11:59:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:00 smithi175 ceph-mon[32613]: 2.2 scrub ok 2024-09-21T11:59:01.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:01.056 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:01.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:01.057 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:01.158 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:01.186 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:01.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:01.187 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:01.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:01.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:01.328 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:01.428 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:01.473 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:01.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:01.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:01.574 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:01.607 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:01.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:01.608 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:01.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:01.747 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:01.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:01.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:01.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:01.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:01.881 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:01.882 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:01.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:01 smithi175 ceph-mon[32613]: 11.d scrub starts 2024-09-21T11:59:01.912 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:01 smithi175 ceph-mon[32613]: pgmap v928: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 969 op/s; 11783/234135 objects misplaced (5.033%); 18 MiB/s, 282 objects/s recovering 2024-09-21T11:59:01.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:01.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:01 smithi043 ceph-mon[36454]: 11.d scrub starts 2024-09-21T11:59:01.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:01 smithi043 ceph-mon[36454]: pgmap v928: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 969 op/s; 11783/234135 objects misplaced (5.033%); 18 MiB/s, 282 objects/s recovering 2024-09-21T11:59:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:01 smithi043 ceph-mon[29710]: 11.d scrub starts 2024-09-21T11:59:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:01 smithi043 ceph-mon[29710]: pgmap v928: 158 pgs: 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 969 op/s; 11783/234135 objects misplaced (5.033%); 18 MiB/s, 282 objects/s recovering 2024-09-21T11:59:02.013 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:02.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:02.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:02.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:02.136 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:02.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:02.137 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:02.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:02.260 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:59:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:59:02.265 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:02.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:02.266 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:02.367 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:02.389 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:02.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:02.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:02.491 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:02.513 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:02.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:02.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:02.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:02.641 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:02.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:02.642 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:02.743 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:02.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:02.775 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:02.776 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:02.876 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:02.909 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:02.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:02.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:02.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:02 smithi043 ceph-mon[29710]: 11.2 scrub ok 2024-09-21T11:59:02.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:02 smithi043 ceph-mon[29710]: osdmap e444: 8 total, 8 up, 8 in 2024-09-21T11:59:02.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:02 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:02 smithi043 ceph-mon[36454]: 11.2 scrub ok 2024-09-21T11:59:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:02 smithi043 ceph-mon[36454]: osdmap e444: 8 total, 8 up, 8 in 2024-09-21T11:59:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:02 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:03.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:02 smithi175 ceph-mon[32613]: 11.2 scrub ok 2024-09-21T11:59:03.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:02 smithi175 ceph-mon[32613]: osdmap e444: 8 total, 8 up, 8 in 2024-09-21T11:59:03.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:02 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:03.011 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:03.037 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:03.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:03.038 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:03.139 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:03.167 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:03.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:03.168 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:03.269 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:03.297 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:03.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:03.298 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:03.399 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:03.426 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:03.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:03.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:03.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:03.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:03.580 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:03.580 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:03.680 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:03.718 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:03.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:03.719 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:03.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:03.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:03.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[29710]: 11.d scrub ok 2024-09-21T11:59:03.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[29710]: pgmap v930: 158 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 153 active+clean; 4.5 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 949 op/s; 10020/221616 objects misplaced (4.521%); 17 MiB/s, 276 objects/s recovering 2024-09-21T11:59:03.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[29710]: osdmap e445: 8 total, 8 up, 8 in 2024-09-21T11:59:03.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[29710]: 3.a scrub starts 2024-09-21T11:59:03.834 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[29710]: 3.a scrub ok 2024-09-21T11:59:03.834 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:03.834 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[36454]: 11.d scrub ok 2024-09-21T11:59:03.834 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[36454]: pgmap v930: 158 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 153 active+clean; 4.5 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 949 op/s; 10020/221616 objects misplaced (4.521%); 17 MiB/s, 276 objects/s recovering 2024-09-21T11:59:03.835 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[36454]: osdmap e445: 8 total, 8 up, 8 in 2024-09-21T11:59:03.835 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[36454]: 3.a scrub starts 2024-09-21T11:59:03.835 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:03 smithi043 ceph-mon[36454]: 3.a scrub ok 2024-09-21T11:59:03.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:03.867 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:03.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:04.010 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:03 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:03 smithi175 ceph-mon[32613]: 11.d scrub ok 2024-09-21T11:59:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:03 smithi175 ceph-mon[32613]: pgmap v930: 158 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 153 active+clean; 4.5 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 949 op/s; 10020/221616 objects misplaced (4.521%); 17 MiB/s, 276 objects/s recovering 2024-09-21T11:59:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:03 smithi175 ceph-mon[32613]: osdmap e445: 8 total, 8 up, 8 in 2024-09-21T11:59:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:03 smithi175 ceph-mon[32613]: 3.a scrub starts 2024-09-21T11:59:04.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:03 smithi175 ceph-mon[32613]: 3.a scrub ok 2024-09-21T11:59:04.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:04.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:04.113 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:04.138 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:04.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:04.139 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:04.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:04.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:04.285 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:04.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:04.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:04.412 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:04.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:04.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:04.459 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":926,"stamp":"2024-09-21T11:58:56.180537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5308684851,"num_objects":81195,"num_object_clones":0,"num_object_copies":243585,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13481,"num_objects_unfound":0,"num_objects_dirty":81195,"num_whiteouts":0,"num_read":2244,"num_read_kb":1996,"num_write":162192,"num_write_kb":7758894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8944,"num_bytes_recovered":562635651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82457,"ondisk_log_size":82457,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":478,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":21786672,"kb_used_data":16093488,"kb_used_omap":12,"kb_used_meta":5692851,"kb_avail":728174544,"statfs":{"total":767960285184,"available":745650733056,"internally_reserved":0,"allocated":16479731712,"data_stored":16474918155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":5829479986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-540737536,"num_objects":-8251,"num_object_clones":0,"num_object_copies":-24753,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3895,"num_objects_unfound":0,"num_objects_dirty":-8251,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":8251,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2229,"num_bytes_recovered":146079744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.657420"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":56,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.587917+0000","last_change":"2024-09-21T11:57:58.529638+0000","last_active":"2024-09-21T11:58:52.587917+0000","last_peered":"2024-09-21T11:58:52.587917+0000","last_clean":"2024-09-21T11:58:52.587917+0000","last_became_active":"2024-09-21T11:57:58.529245+0000","last_became_peered":"2024-09-21T11:57:58.529245+0000","last_unstale":"2024-09-21T11:58:52.587917+0000","last_undegraded":"2024-09-21T11:58:52.587917+0000","last_fullsized":"2024-09-21T11:58:52.587917+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:43.783030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.783030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:01.460062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":858,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.580256+0000","last_change":"2024-09-21T11:57:37.936348+0000","last_active":"2024-09-21T11:58:52.580256+0000","last_peered":"2024-09-21T11:58:52.580256+0000","last_clean":"2024-09-21T11:58:52.580256+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:58:52.580256+0000","last_undegraded":"2024-09-21T11:58:52.580256+0000","last_fullsized":"2024-09-21T11:58:52.580256+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:37.936279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.936279+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:39.639770+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":534,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562240+0000","last_change":"2024-09-21T11:56:08.429983+0000","last_active":"2024-09-21T11:58:46.562240+0000","last_peered":"2024-09-21T11:58:46.562240+0000","last_clean":"2024-09-21T11:58:46.562240+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:58:46.562240+0000","last_undegraded":"2024-09-21T11:58:46.562240+0000","last_fullsized":"2024-09-21T11:58:46.562240+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:08.429912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.429912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":629,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524179+0000","last_change":"2024-09-21T11:58:21.827334+0000","last_active":"2024-09-21T11:58:46.524179+0000","last_peered":"2024-09-21T11:58:46.524179+0000","last_clean":"2024-09-21T11:58:46.524179+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:58:46.524179+0000","last_undegraded":"2024-09-21T11:58:46.524179+0000","last_fullsized":"2024-09-21T11:58:46.524179+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:21.827047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:21.827047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:43.889402+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":831,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579904+0000","last_change":"2024-09-21T11:56:49.693769+0000","last_active":"2024-09-21T11:58:52.579904+0000","last_peered":"2024-09-21T11:58:52.579904+0000","last_clean":"2024-09-21T11:58:52.579904+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:58:52.579904+0000","last_undegraded":"2024-09-21T11:58:52.579904+0000","last_fullsized":"2024-09-21T11:58:52.579904+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:49.693594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:49.693594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":821,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.578690+0000","last_change":"2024-09-21T11:57:37.711838+0000","last_active":"2024-09-21T11:58:52.578690+0000","last_peered":"2024-09-21T11:58:52.578690+0000","last_clean":"2024-09-21T11:58:52.578690+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:58:52.578690+0000","last_undegraded":"2024-09-21T11:58:52.578690+0000","last_fullsized":"2024-09-21T11:58:52.578690+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:37.711758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.711758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:04.785508+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":866,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.578788+0000","last_change":"2024-09-21T11:56:31.572626+0000","last_active":"2024-09-21T11:58:52.578788+0000","last_peered":"2024-09-21T11:58:52.578788+0000","last_clean":"2024-09-21T11:58:52.578788+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:58:52.578788+0000","last_undegraded":"2024-09-21T11:58:52.578788+0000","last_fullsized":"2024-09-21T11:58:52.578788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:31.572525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:31.572525+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":681,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.582538+0000","last_change":"2024-09-21T11:58:30.584828+0000","last_active":"2024-09-21T11:58:52.582538+0000","last_peered":"2024-09-21T11:58:52.582538+0000","last_clean":"2024-09-21T11:58:52.582538+0000","last_became_active":"2024-09-21T11:58:17.527501+0000","last_became_peered":"2024-09-21T11:58:17.527501+0000","last_unstale":"2024-09-21T11:58:52.582538+0000","last_undegraded":"2024-09-21T11:58:52.582538+0000","last_fullsized":"2024-09-21T11:58:52.582538+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:30.584774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:30.584774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:50.421881+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":808,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.525025+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:58:46.525025+0000","last_peered":"2024-09-21T11:58:46.525025+0000","last_clean":"2024-09-21T11:58:46.525025+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:58:46.525025+0000","last_undegraded":"2024-09-21T11:58:46.525025+0000","last_fullsized":"2024-09-21T11:58:46.525025+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.819967+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":858,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.576799+0000","last_change":"2024-09-21T11:58:17.547443+0000","last_active":"2024-09-21T11:58:52.576799+0000","last_peered":"2024-09-21T11:58:52.576799+0000","last_clean":"2024-09-21T11:58:52.576799+0000","last_became_active":"2024-09-21T11:58:17.525792+0000","last_became_peered":"2024-09-21T11:58:17.525792+0000","last_unstale":"2024-09-21T11:58:52.576799+0000","last_undegraded":"2024-09-21T11:58:52.576799+0000","last_fullsized":"2024-09-21T11:58:52.576799+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:07.774892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:07.774892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":808,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524283+0000","last_change":"2024-09-21T11:55:05.096606+0000","last_active":"2024-09-21T11:58:46.524283+0000","last_peered":"2024-09-21T11:58:46.524283+0000","last_clean":"2024-09-21T11:58:46.524283+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:58:46.524283+0000","last_undegraded":"2024-09-21T11:58:46.524283+0000","last_fullsized":"2024-09-21T11:58:46.524283+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:05.096542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:05.096542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:57.793506+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":56,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.586947+0000","last_change":"2024-09-21T11:57:58.527130+0000","last_active":"2024-09-21T11:58:52.586947+0000","last_peered":"2024-09-21T11:58:52.586947+0000","last_clean":"2024-09-21T11:58:52.586947+0000","last_became_active":"2024-09-21T11:57:58.526820+0000","last_became_peered":"2024-09-21T11:57:58.526820+0000","last_unstale":"2024-09-21T11:58:52.586947+0000","last_undegraded":"2024-09-21T11:58:52.586947+0000","last_fullsized":"2024-09-21T11:58:52.586947+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:36.422726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:36.422726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":854,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524720+0000","last_change":"2024-09-21T11:56:02.994041+0000","last_active":"2024-09-21T11:58:46.524720+0000","last_peered":"2024-09-21T11:58:46.524720+0000","last_clean":"2024-09-21T11:58:46.524720+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:58:46.524720+0000","last_undegraded":"2024-09-21T11:58:46.524720+0000","last_fullsized":"2024-09-21T11:58:46.524720+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.993966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.993966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:35.823113+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":856,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.580182+0000","last_change":"2024-09-21T11:58:17.549945+0000","last_active":"2024-09-21T11:58:52.580182+0000","last_peered":"2024-09-21T11:58:52.580182+0000","last_clean":"2024-09-21T11:58:52.580182+0000","last_became_active":"2024-09-21T11:58:17.525789+0000","last_became_peered":"2024-09-21T11:58:17.525789+0000","last_unstale":"2024-09-21T11:58:52.580182+0000","last_undegraded":"2024-09-21T11:58:52.580182+0000","last_fullsized":"2024-09-21T11:58:52.580182+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'7","last_scrub_stamp":"2024-09-21T11:56:59.659576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.659576+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":830,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562357+0000","last_change":"2024-09-21T11:58:22.381598+0000","last_active":"2024-09-21T11:58:46.562357+0000","last_peered":"2024-09-21T11:58:46.562357+0000","last_clean":"2024-09-21T11:58:46.562357+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:58:46.562357+0000","last_undegraded":"2024-09-21T11:58:46.562357+0000","last_fullsized":"2024-09-21T11:58:46.562357+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:22.381531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:22.381531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.811109+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":826,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524113+0000","last_change":"2024-09-21T11:58:24.861423+0000","last_active":"2024-09-21T11:58:46.524113+0000","last_peered":"2024-09-21T11:58:46.524113+0000","last_clean":"2024-09-21T11:58:46.524113+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:58:46.524113+0000","last_undegraded":"2024-09-21T11:58:46.524113+0000","last_fullsized":"2024-09-21T11:58:46.524113+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:24.861354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:24.861354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.549127+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":330,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579374+0000","last_change":"2024-09-21T11:56:34.073296+0000","last_active":"2024-09-21T11:58:52.579374+0000","last_peered":"2024-09-21T11:58:52.579374+0000","last_clean":"2024-09-21T11:58:52.579374+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:58:52.579374+0000","last_undegraded":"2024-09-21T11:58:52.579374+0000","last_fullsized":"2024-09-21T11:58:52.579374+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:34.073249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:34.073249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:15.736984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":56,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.589893+0000","last_change":"2024-09-21T11:57:58.534353+0000","last_active":"2024-09-21T11:58:52.589893+0000","last_peered":"2024-09-21T11:58:52.589893+0000","last_clean":"2024-09-21T11:58:52.589893+0000","last_became_active":"2024-09-21T11:57:58.530327+0000","last_became_peered":"2024-09-21T11:57:58.530327+0000","last_unstale":"2024-09-21T11:58:52.589893+0000","last_undegraded":"2024-09-21T11:58:52.589893+0000","last_fullsized":"2024-09-21T11:58:52.589893+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:27.176320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:27.176320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":972,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.560596+0000","last_change":"2024-09-21T11:57:51.913043+0000","last_active":"2024-09-21T11:58:46.560596+0000","last_peered":"2024-09-21T11:58:46.560596+0000","last_clean":"2024-09-21T11:58:46.560596+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T11:58:46.560596+0000","last_undegraded":"2024-09-21T11:58:46.560596+0000","last_fullsized":"2024-09-21T11:58:46.560596+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:57:51.912800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.912800+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:55.941418+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":540,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.549420+0000","last_change":"2024-09-21T11:58:17.533215+0000","last_active":"2024-09-21T11:58:46.549420+0000","last_peered":"2024-09-21T11:58:46.549420+0000","last_clean":"2024-09-21T11:58:46.549420+0000","last_became_active":"2024-09-21T11:58:17.532890+0000","last_became_peered":"2024-09-21T11:58:17.532890+0000","last_unstale":"2024-09-21T11:58:46.549420+0000","last_undegraded":"2024-09-21T11:58:46.549420+0000","last_fullsized":"2024-09-21T11:58:46.549420+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:56.501356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:56.501356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":825,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579036+0000","last_change":"2024-09-21T11:57:04.647951+0000","last_active":"2024-09-21T11:58:52.579036+0000","last_peered":"2024-09-21T11:58:52.579036+0000","last_clean":"2024-09-21T11:58:52.579036+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:58:52.579036+0000","last_undegraded":"2024-09-21T11:58:52.579036+0000","last_fullsized":"2024-09-21T11:58:52.579036+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:04.647749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:04.647749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":641,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.567912+0000","last_change":"2024-09-21T11:55:57.442188+0000","last_active":"2024-09-21T11:58:46.567912+0000","last_peered":"2024-09-21T11:58:46.567912+0000","last_clean":"2024-09-21T11:58:46.567912+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:58:46.567912+0000","last_undegraded":"2024-09-21T11:58:46.567912+0000","last_fullsized":"2024-09-21T11:58:46.567912+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:57.441965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.441965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":885,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.577958+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:58:52.577958+0000","last_peered":"2024-09-21T11:58:52.577958+0000","last_clean":"2024-09-21T11:58:52.577958+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:58:52.577958+0000","last_undegraded":"2024-09-21T11:58:52.577958+0000","last_fullsized":"2024-09-21T11:58:52.577958+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:56.834377+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":343,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.576896+0000","last_change":"2024-09-21T11:58:17.529121+0000","last_active":"2024-09-21T11:58:52.576896+0000","last_peered":"2024-09-21T11:58:52.576896+0000","last_clean":"2024-09-21T11:58:52.576896+0000","last_became_active":"2024-09-21T11:58:17.528785+0000","last_became_peered":"2024-09-21T11:58:17.528785+0000","last_unstale":"2024-09-21T11:58:52.576896+0000","last_undegraded":"2024-09-21T11:58:52.576896+0000","last_fullsized":"2024-09-21T11:58:52.576896+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.487132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.487132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":324,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.587576+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:58:52.587576+0000","last_peered":"2024-09-21T11:58:52.587576+0000","last_clean":"2024-09-21T11:58:52.587576+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:58:52.587576+0000","last_undegraded":"2024-09-21T11:58:52.587576+0000","last_fullsized":"2024-09-21T11:58:52.587576+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":560,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562426+0000","last_change":"2024-09-21T11:58:17.554468+0000","last_active":"2024-09-21T11:58:46.562426+0000","last_peered":"2024-09-21T11:58:46.562426+0000","last_clean":"2024-09-21T11:58:46.562426+0000","last_became_active":"2024-09-21T11:58:17.529593+0000","last_became_peered":"2024-09-21T11:58:17.529593+0000","last_unstale":"2024-09-21T11:58:46.562426+0000","last_undegraded":"2024-09-21T11:58:46.562426+0000","last_fullsized":"2024-09-21T11:58:46.562426+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:10.530913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.530913+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":854,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.589105+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:58:52.589105+0000","last_peered":"2024-09-21T11:58:52.589105+0000","last_clean":"2024-09-21T11:58:52.589105+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:58:52.589105+0000","last_undegraded":"2024-09-21T11:58:52.589105+0000","last_fullsized":"2024-09-21T11:58:52.589105+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"432'10","reported_seq":1107,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579539+0000","last_change":"2024-09-21T11:58:18.736596+0000","last_active":"2024-09-21T11:58:52.579539+0000","last_peered":"2024-09-21T11:58:52.579539+0000","last_clean":"2024-09-21T11:58:52.579539+0000","last_became_active":"2024-09-21T11:58:17.529830+0000","last_became_peered":"2024-09-21T11:58:17.529830+0000","last_unstale":"2024-09-21T11:58:52.579539+0000","last_undegraded":"2024-09-21T11:58:52.579539+0000","last_fullsized":"2024-09-21T11:58:52.579539+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"383'9","last_scrub_stamp":"2024-09-21T11:58:18.736520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:18.736520+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:40.102209+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":70,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524124+0000","last_change":"2024-09-21T11:57:49.896884+0000","last_active":"2024-09-21T11:58:46.524124+0000","last_peered":"2024-09-21T11:58:46.524124+0000","last_clean":"2024-09-21T11:58:46.524124+0000","last_became_active":"2024-09-21T11:57:45.504680+0000","last_became_peered":"2024-09-21T11:57:45.504680+0000","last_unstale":"2024-09-21T11:58:46.524124+0000","last_undegraded":"2024-09-21T11:58:46.524124+0000","last_fullsized":"2024-09-21T11:58:46.524124+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:49.896819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:49.896819+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:52.548939+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"442'10095","reported_seq":10207,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:51.789660+0000","last_change":"2024-09-21T11:56:36.507880+0000","last_active":"2024-09-21T11:58:51.789660+0000","last_peered":"2024-09-21T11:58:51.789660+0000","last_clean":"2024-09-21T11:58:51.789660+0000","last_became_active":"2024-09-21T11:56:36.507606+0000","last_became_peered":"2024-09-21T11:56:36.507606+0000","last_unstale":"2024-09-21T11:58:51.789660+0000","last_undegraded":"2024-09-21T11:58:51.789660+0000","last_fullsized":"2024-09-21T11:58:51.789660+0000","mapping_epoch":392,"log_start":"408'5322","ondisk_log_start":"408'5322","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4773,"log_dups_size":0,"ondisk_log_size":4773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":338362368,"num_objects":5163,"num_object_clones":0,"num_object_copies":15489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10095,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":561,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.563535+0000","last_change":"2024-09-21T11:55:52.562992+0000","last_active":"2024-09-21T11:58:46.563535+0000","last_peered":"2024-09-21T11:58:46.563535+0000","last_clean":"2024-09-21T11:58:46.563535+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:58:46.563535+0000","last_undegraded":"2024-09-21T11:58:46.563535+0000","last_fullsized":"2024-09-21T11:58:46.563535+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:55:52.562928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.562928+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:55.612843+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":850,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.583842+0000","last_change":"2024-09-21T11:57:13.738771+0000","last_active":"2024-09-21T11:58:52.583842+0000","last_peered":"2024-09-21T11:58:52.583842+0000","last_clean":"2024-09-21T11:58:52.583842+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:58:52.583842+0000","last_undegraded":"2024-09-21T11:58:52.583842+0000","last_fullsized":"2024-09-21T11:58:52.583842+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:13.738727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:13.738727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"432'11","reported_seq":464,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524635+0000","last_change":"2024-09-21T11:58:17.537763+0000","last_active":"2024-09-21T11:58:46.524635+0000","last_peered":"2024-09-21T11:58:46.524635+0000","last_clean":"2024-09-21T11:58:46.524635+0000","last_became_active":"2024-09-21T11:58:17.526752+0000","last_became_peered":"2024-09-21T11:58:17.526752+0000","last_unstale":"2024-09-21T11:58:46.524635+0000","last_undegraded":"2024-09-21T11:58:46.524635+0000","last_fullsized":"2024-09-21T11:58:46.524635+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"366'10","last_scrub_stamp":"2024-09-21T11:56:48.961017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.961017+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.739596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":873,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524668+0000","last_change":"2024-09-21T11:58:16.866853+0000","last_active":"2024-09-21T11:58:46.524668+0000","last_peered":"2024-09-21T11:58:46.524668+0000","last_clean":"2024-09-21T11:58:46.524668+0000","last_became_active":"2024-09-21T11:57:58.526927+0000","last_became_peered":"2024-09-21T11:57:58.526927+0000","last_unstale":"2024-09-21T11:58:46.524668+0000","last_undegraded":"2024-09-21T11:58:46.524668+0000","last_fullsized":"2024-09-21T11:58:46.524668+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:16.866782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:16.866782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.387681+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":70,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561517+0000","last_change":"2024-09-21T11:57:47.906127+0000","last_active":"2024-09-21T11:58:46.561517+0000","last_peered":"2024-09-21T11:58:46.561517+0000","last_clean":"2024-09-21T11:58:46.561517+0000","last_became_active":"2024-09-21T11:57:45.514811+0000","last_became_peered":"2024-09-21T11:57:45.514811+0000","last_unstale":"2024-09-21T11:58:46.561517+0000","last_undegraded":"2024-09-21T11:58:46.561517+0000","last_fullsized":"2024-09-21T11:58:46.561517+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:47.905784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:47.905784+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:15.591116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"442'10300","reported_seq":10411,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:51.312031+0000","last_change":"2024-09-21T11:56:36.511425+0000","last_active":"2024-09-21T11:58:51.312031+0000","last_peered":"2024-09-21T11:58:51.312031+0000","last_clean":"2024-09-21T11:58:51.312031+0000","last_became_active":"2024-09-21T11:56:36.511191+0000","last_became_peered":"2024-09-21T11:56:36.511191+0000","last_unstale":"2024-09-21T11:58:51.312031+0000","last_undegraded":"2024-09-21T11:58:51.312031+0000","last_fullsized":"2024-09-21T11:58:51.312031+0000","mapping_epoch":392,"log_start":"409'5542","ondisk_log_start":"409'5542","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4758,"log_dups_size":0,"ondisk_log_size":4758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:11.336526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":350486528,"num_objects":5348,"num_object_clones":0,"num_object_copies":16044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10300,"num_write_kb":500736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":941,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.581744+0000","last_change":"2024-09-21T11:57:58.547816+0000","last_active":"2024-09-21T11:58:52.581744+0000","last_peered":"2024-09-21T11:58:52.581744+0000","last_clean":"2024-09-21T11:58:52.581744+0000","last_became_active":"2024-09-21T11:57:58.537184+0000","last_became_peered":"2024-09-21T11:57:58.537184+0000","last_unstale":"2024-09-21T11:58:52.581744+0000","last_undegraded":"2024-09-21T11:58:52.581744+0000","last_fullsized":"2024-09-21T11:58:52.581744+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:57:00.631769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:00.631769+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":844,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579477+0000","last_change":"2024-09-21T11:56:53.701703+0000","last_active":"2024-09-21T11:58:52.579477+0000","last_peered":"2024-09-21T11:58:52.579477+0000","last_clean":"2024-09-21T11:58:52.579477+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:58:52.579477+0000","last_undegraded":"2024-09-21T11:58:52.579477+0000","last_fullsized":"2024-09-21T11:58:52.579477+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.701639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.701639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":330,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.576494+0000","last_change":"2024-09-21T11:58:19.959956+0000","last_active":"2024-09-21T11:58:52.576494+0000","last_peered":"2024-09-21T11:58:52.576494+0000","last_clean":"2024-09-21T11:58:52.576494+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:58:52.576494+0000","last_undegraded":"2024-09-21T11:58:52.576494+0000","last_fullsized":"2024-09-21T11:58:52.576494+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:19.959912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:19.959912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.737932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":330,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.539905+0000","last_change":"2024-09-21T11:57:42.760097+0000","last_active":"2024-09-21T11:58:46.539905+0000","last_peered":"2024-09-21T11:58:46.539905+0000","last_clean":"2024-09-21T11:58:46.539905+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:58:46.539905+0000","last_undegraded":"2024-09-21T11:58:46.539905+0000","last_fullsized":"2024-09-21T11:58:46.539905+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:42.755633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:42.755633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:46.597748+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":77,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.576714+0000","last_change":"2024-09-21T11:57:47.804542+0000","last_active":"2024-09-21T11:58:52.576714+0000","last_peered":"2024-09-21T11:58:52.576714+0000","last_clean":"2024-09-21T11:58:52.576714+0000","last_became_active":"2024-09-21T11:57:45.514189+0000","last_became_peered":"2024-09-21T11:57:45.514189+0000","last_unstale":"2024-09-21T11:58:52.576714+0000","last_undegraded":"2024-09-21T11:58:52.576714+0000","last_fullsized":"2024-09-21T11:58:52.576714+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:47.804466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:47.804466+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:58.484095+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"442'9900","reported_seq":10028,"reported_epoch":442,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-21T11:58:51.800074+0000","last_change":"2024-09-21T11:58:48.686396+0000","last_active":"2024-09-21T11:58:51.800074+0000","last_peered":"2024-09-21T11:58:51.800074+0000","last_clean":"2024-09-21T11:58:17.235375+0000","last_became_active":"2024-09-21T11:58:18.605530+0000","last_became_peered":"2024-09-21T11:58:18.605530+0000","last_unstale":"2024-09-21T11:58:51.800074+0000","last_undegraded":"2024-09-21T11:58:51.800074+0000","last_fullsized":"2024-09-21T11:58:51.800074+0000","mapping_epoch":431,"log_start":"407'5067","ondisk_log_start":"407'5067","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4833,"log_dups_size":0,"ondisk_log_size":4833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":339607552,"num_objects":5182,"num_object_clones":0,"num_object_copies":15546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5182,"num_objects_unfound":0,"num_objects_dirty":5182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9900,"num_write_kb":482624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,7,0],"avail_no_missing":["1","0","7"],"object_location_counts":[{"shards":"0,1,7","objects":5182}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":878,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524166+0000","last_change":"2024-09-21T11:58:17.828372+0000","last_active":"2024-09-21T11:58:46.524166+0000","last_peered":"2024-09-21T11:58:46.524166+0000","last_clean":"2024-09-21T11:58:46.524166+0000","last_became_active":"2024-09-21T11:58:17.525491+0000","last_became_peered":"2024-09-21T11:58:17.525491+0000","last_unstale":"2024-09-21T11:58:46.524166+0000","last_undegraded":"2024-09-21T11:58:46.524166+0000","last_fullsized":"2024-09-21T11:58:46.524166+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:58:17.828305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:17.828305+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.685496+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":60,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.589192+0000","last_change":"2024-09-21T11:57:58.539709+0000","last_active":"2024-09-21T11:58:52.589192+0000","last_peered":"2024-09-21T11:58:52.589192+0000","last_clean":"2024-09-21T11:58:52.589192+0000","last_became_active":"2024-09-21T11:57:58.526938+0000","last_became_peered":"2024-09-21T11:57:58.526938+0000","last_unstale":"2024-09-21T11:58:52.589192+0000","last_undegraded":"2024-09-21T11:58:52.589192+0000","last_fullsized":"2024-09-21T11:58:52.589192+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:48.446635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.446635+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":1080,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.581603+0000","last_change":"2024-09-21T11:56:05.567407+0000","last_active":"2024-09-21T11:58:52.581603+0000","last_peered":"2024-09-21T11:58:52.581603+0000","last_clean":"2024-09-21T11:58:52.581603+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:58:52.581603+0000","last_undegraded":"2024-09-21T11:58:52.581603+0000","last_fullsized":"2024-09-21T11:58:52.581603+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:05.567332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.567332+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:05.895448+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":831,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562398+0000","last_change":"2024-09-21T11:56:34.581966+0000","last_active":"2024-09-21T11:58:46.562398+0000","last_peered":"2024-09-21T11:58:46.562398+0000","last_clean":"2024-09-21T11:58:46.562398+0000","last_became_active":"2024-09-21T11:56:34.581626+0000","last_became_peered":"2024-09-21T11:56:34.581626+0000","last_unstale":"2024-09-21T11:58:46.562398+0000","last_undegraded":"2024-09-21T11:58:46.562398+0000","last_fullsized":"2024-09-21T11:58:46.562398+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.643323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.643323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:35.106418+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":158,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579351+0000","last_change":"2024-09-21T11:57:45.998708+0000","last_active":"2024-09-21T11:58:52.579351+0000","last_peered":"2024-09-21T11:58:52.579351+0000","last_clean":"2024-09-21T11:58:52.579351+0000","last_became_active":"2024-09-21T11:57:45.514310+0000","last_became_peered":"2024-09-21T11:57:45.514310+0000","last_unstale":"2024-09-21T11:58:52.579351+0000","last_undegraded":"2024-09-21T11:58:52.579351+0000","last_fullsized":"2024-09-21T11:58:52.579351+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:45.998644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:45.998644+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.495485+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"442'9937","reported_seq":10050,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:51.907976+0000","last_change":"2024-09-21T11:56:36.623897+0000","last_active":"2024-09-21T11:58:51.907976+0000","last_peered":"2024-09-21T11:58:51.907976+0000","last_clean":"2024-09-21T11:58:51.907976+0000","last_became_active":"2024-09-21T11:56:36.623581+0000","last_became_peered":"2024-09-21T11:56:36.623581+0000","last_unstale":"2024-09-21T11:58:51.907976+0000","last_undegraded":"2024-09-21T11:58:51.907976+0000","last_fullsized":"2024-09-21T11:58:51.907976+0000","mapping_epoch":392,"log_start":"402'3716","ondisk_log_start":"402'3716","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":6221,"log_dups_size":0,"ondisk_log_size":6221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:00.021032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":338296856,"num_objects":5163,"num_object_clones":0,"num_object_copies":15489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9937,"num_write_kb":483137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":592,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.575210+0000","last_change":"2024-09-21T11:58:17.557543+0000","last_active":"2024-09-21T11:58:46.575210+0000","last_peered":"2024-09-21T11:58:46.575210+0000","last_clean":"2024-09-21T11:58:46.575210+0000","last_became_active":"2024-09-21T11:58:17.531001+0000","last_became_peered":"2024-09-21T11:58:17.531001+0000","last_unstale":"2024-09-21T11:58:46.575210+0000","last_undegraded":"2024-09-21T11:58:46.575210+0000","last_fullsized":"2024-09-21T11:58:46.575210+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:56:09.421193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.421193+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":858,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.525068+0000","last_change":"2024-09-21T11:56:51.036609+0000","last_active":"2024-09-21T11:58:46.525068+0000","last_peered":"2024-09-21T11:58:46.525068+0000","last_clean":"2024-09-21T11:58:46.525068+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:58:46.525068+0000","last_undegraded":"2024-09-21T11:58:46.525068+0000","last_fullsized":"2024-09-21T11:58:46.525068+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:51.036535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:51.036535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:40.871456+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1066,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.590185+0000","last_change":"2024-09-21T11:57:40.102377+0000","last_active":"2024-09-21T11:58:52.590185+0000","last_peered":"2024-09-21T11:58:52.590185+0000","last_clean":"2024-09-21T11:58:52.590185+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:58:52.590185+0000","last_undegraded":"2024-09-21T11:58:52.590185+0000","last_fullsized":"2024-09-21T11:58:52.590185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:40.102294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:40.102294+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:59.370714+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":832,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.589607+0000","last_change":"2024-09-21T11:57:36.109392+0000","last_active":"2024-09-21T11:58:52.589607+0000","last_peered":"2024-09-21T11:58:52.589607+0000","last_clean":"2024-09-21T11:58:52.589607+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:58:52.589607+0000","last_undegraded":"2024-09-21T11:58:52.589607+0000","last_fullsized":"2024-09-21T11:58:52.589607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:36.109317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:36.109317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:01.441881+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":158,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579283+0000","last_change":"2024-09-21T11:57:51.011971+0000","last_active":"2024-09-21T11:58:52.579283+0000","last_peered":"2024-09-21T11:58:52.579283+0000","last_clean":"2024-09-21T11:58:52.579283+0000","last_became_active":"2024-09-21T11:57:45.518764+0000","last_became_peered":"2024-09-21T11:57:45.518764+0000","last_unstale":"2024-09-21T11:58:52.579283+0000","last_undegraded":"2024-09-21T11:58:52.579283+0000","last_fullsized":"2024-09-21T11:58:52.579283+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:51.011892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.011892+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:03.731641+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.c","version":"442'10056","reported_seq":11084,"reported_epoch":442,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:58:51.799378+0000","last_change":"2024-09-21T11:58:48.624484+0000","last_active":"2024-09-21T11:58:51.799378+0000","last_peered":"2024-09-21T11:58:51.799378+0000","last_clean":"2024-09-21T11:58:17.235882+0000","last_became_active":"2024-09-21T11:58:18.606350+0000","last_became_peered":"2024-09-21T11:58:18.606350+0000","last_unstale":"2024-09-21T11:58:51.799378+0000","last_undegraded":"2024-09-21T11:58:51.799378+0000","last_fullsized":"2024-09-21T11:58:51.799378+0000","mapping_epoch":431,"log_start":"408'5279","ondisk_log_start":"408'5279","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4777,"log_dups_size":0,"ondisk_log_size":4777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335675392,"num_objects":5122,"num_object_clones":0,"num_object_copies":15366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4677,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10056,"num_write_kb":485696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":454,"num_bytes_recovered":29753344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,7],"avail_no_missing":["1","2","7"],"object_location_counts":[{"shards":"1,2,7","objects":5122}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":899,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.591635+0000","last_change":"2024-09-21T11:57:50.976268+0000","last_active":"2024-09-21T11:58:52.591635+0000","last_peered":"2024-09-21T11:58:52.591635+0000","last_clean":"2024-09-21T11:58:52.591635+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:58:52.591635+0000","last_undegraded":"2024-09-21T11:58:52.591635+0000","last_fullsized":"2024-09-21T11:58:52.591635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:57:50.976029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:50.976029+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:00.941833+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":527,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562685+0000","last_change":"2024-09-21T11:56:58.434536+0000","last_active":"2024-09-21T11:58:46.562685+0000","last_peered":"2024-09-21T11:58:46.562685+0000","last_clean":"2024-09-21T11:58:46.562685+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:58:46.562685+0000","last_undegraded":"2024-09-21T11:58:46.562685+0000","last_fullsized":"2024-09-21T11:58:46.562685+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.434464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.434464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":812,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579007+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:58:52.579007+0000","last_peered":"2024-09-21T11:58:52.579007+0000","last_clean":"2024-09-21T11:58:52.579007+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:58:52.579007+0000","last_undegraded":"2024-09-21T11:58:52.579007+0000","last_fullsized":"2024-09-21T11:58:52.579007+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:34.633118+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":519,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562094+0000","last_change":"2024-09-21T11:55:51.602665+0000","last_active":"2024-09-21T11:58:46.562094+0000","last_peered":"2024-09-21T11:58:46.562094+0000","last_clean":"2024-09-21T11:58:46.562094+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:58:46.562094+0000","last_undegraded":"2024-09-21T11:58:46.562094+0000","last_fullsized":"2024-09-21T11:58:46.562094+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.602587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.602587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":70,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.523989+0000","last_change":"2024-09-21T11:57:53.922876+0000","last_active":"2024-09-21T11:58:46.523989+0000","last_peered":"2024-09-21T11:58:46.523989+0000","last_clean":"2024-09-21T11:58:46.523989+0000","last_became_active":"2024-09-21T11:57:45.515242+0000","last_became_peered":"2024-09-21T11:57:45.515242+0000","last_unstale":"2024-09-21T11:58:46.523989+0000","last_undegraded":"2024-09-21T11:58:46.523989+0000","last_fullsized":"2024-09-21T11:58:46.523989+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:53.922795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:53.922795+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:20.497165+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.b","version":"443'10236","reported_seq":10350,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:55.986398+0000","last_change":"2024-09-21T11:56:36.510297+0000","last_active":"2024-09-21T11:58:55.986398+0000","last_peered":"2024-09-21T11:58:55.986398+0000","last_clean":"2024-09-21T11:58:55.986398+0000","last_became_active":"2024-09-21T11:56:36.510018+0000","last_became_peered":"2024-09-21T11:56:36.510018+0000","last_unstale":"2024-09-21T11:58:55.986398+0000","last_undegraded":"2024-09-21T11:58:55.986398+0000","last_fullsized":"2024-09-21T11:58:55.986398+0000","mapping_epoch":392,"log_start":"410'5521","ondisk_log_start":"410'5521","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319684608,"num_objects":4878,"num_object_clones":0,"num_object_copies":14634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10236,"num_write_kb":483648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":685,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562386+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:58:46.562386+0000","last_peered":"2024-09-21T11:58:46.562386+0000","last_clean":"2024-09-21T11:58:46.562386+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:58:46.562386+0000","last_undegraded":"2024-09-21T11:58:46.562386+0000","last_fullsized":"2024-09-21T11:58:46.562386+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":355,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.590598+0000","last_change":"2024-09-21T11:57:51.934799+0000","last_active":"2024-09-21T11:58:52.590598+0000","last_peered":"2024-09-21T11:58:52.590598+0000","last_clean":"2024-09-21T11:58:52.590598+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:58:52.590598+0000","last_undegraded":"2024-09-21T11:58:52.590598+0000","last_fullsized":"2024-09-21T11:58:52.590598+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:51.934563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.934563+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:17.585958+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":850,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524458+0000","last_change":"2024-09-21T11:56:28.083398+0000","last_active":"2024-09-21T11:58:46.524458+0000","last_peered":"2024-09-21T11:58:46.524458+0000","last_clean":"2024-09-21T11:58:46.524458+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:58:46.524458+0000","last_undegraded":"2024-09-21T11:58:46.524458+0000","last_fullsized":"2024-09-21T11:58:46.524458+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:56:28.083324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.083324+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:59.743090+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":63,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.587786+0000","last_change":"2024-09-21T11:58:17.527987+0000","last_active":"2024-09-21T11:58:52.587786+0000","last_peered":"2024-09-21T11:58:52.587786+0000","last_clean":"2024-09-21T11:58:52.587786+0000","last_became_active":"2024-09-21T11:58:17.527653+0000","last_became_peered":"2024-09-21T11:58:17.527653+0000","last_unstale":"2024-09-21T11:58:52.587786+0000","last_undegraded":"2024-09-21T11:58:52.587786+0000","last_fullsized":"2024-09-21T11:58:52.587786+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":808,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.578491+0000","last_change":"2024-09-21T11:57:54.859726+0000","last_active":"2024-09-21T11:58:52.578491+0000","last_peered":"2024-09-21T11:58:52.578491+0000","last_clean":"2024-09-21T11:58:52.578491+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:58:52.578491+0000","last_undegraded":"2024-09-21T11:58:52.578491+0000","last_fullsized":"2024-09-21T11:58:52.578491+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:54.859668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.859668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.673552+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":950,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562142+0000","last_change":"2024-09-21T11:58:17.529673+0000","last_active":"2024-09-21T11:58:46.562142+0000","last_peered":"2024-09-21T11:58:46.562142+0000","last_clean":"2024-09-21T11:58:46.562142+0000","last_became_active":"2024-09-21T11:58:17.529304+0000","last_became_peered":"2024-09-21T11:58:17.529304+0000","last_unstale":"2024-09-21T11:58:46.562142+0000","last_undegraded":"2024-09-21T11:58:46.562142+0000","last_fullsized":"2024-09-21T11:58:46.562142+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:50.548003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:50.548003+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:04.592556+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"443'10310","reported_seq":10424,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:55.955336+0000","last_change":"2024-09-21T11:56:36.509094+0000","last_active":"2024-09-21T11:58:55.955336+0000","last_peered":"2024-09-21T11:58:55.955336+0000","last_clean":"2024-09-21T11:58:55.955336+0000","last_became_active":"2024-09-21T11:56:36.508840+0000","last_became_peered":"2024-09-21T11:56:36.508840+0000","last_unstale":"2024-09-21T11:58:55.955336+0000","last_undegraded":"2024-09-21T11:58:55.955336+0000","last_fullsized":"2024-09-21T11:58:55.955336+0000","mapping_epoch":392,"log_start":"410'5595","ondisk_log_start":"410'5595","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4715,"log_dups_size":0,"ondisk_log_size":4715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322174976,"num_objects":4916,"num_object_clones":0,"num_object_copies":14748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10310,"num_write_kb":487232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":508,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561111+0000","last_change":"2024-09-21T11:57:58.543832+0000","last_active":"2024-09-21T11:58:46.561111+0000","last_peered":"2024-09-21T11:58:46.561111+0000","last_clean":"2024-09-21T11:58:46.561111+0000","last_became_active":"2024-09-21T11:57:58.530507+0000","last_became_peered":"2024-09-21T11:57:58.530507+0000","last_unstale":"2024-09-21T11:58:46.561111+0000","last_undegraded":"2024-09-21T11:58:46.561111+0000","last_fullsized":"2024-09-21T11:58:46.561111+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:57:08.488050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:08.488050+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":531,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.568031+0000","last_change":"2024-09-21T11:57:43.583962+0000","last_active":"2024-09-21T11:58:46.568031+0000","last_peered":"2024-09-21T11:58:46.568031+0000","last_clean":"2024-09-21T11:58:46.568031+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:58:46.568031+0000","last_undegraded":"2024-09-21T11:58:46.568031+0000","last_fullsized":"2024-09-21T11:58:46.568031+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:43.583736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.583736+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:11.292974+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":852,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.539931+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:58:46.539931+0000","last_peered":"2024-09-21T11:58:46.539931+0000","last_clean":"2024-09-21T11:58:46.539931+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:58:46.539931+0000","last_undegraded":"2024-09-21T11:58:46.539931+0000","last_fullsized":"2024-09-21T11:58:46.539931+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.044139+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":826,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.560948+0000","last_change":"2024-09-21T11:57:44.790249+0000","last_active":"2024-09-21T11:58:46.560948+0000","last_peered":"2024-09-21T11:58:46.560948+0000","last_clean":"2024-09-21T11:58:46.560948+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:58:46.560948+0000","last_undegraded":"2024-09-21T11:58:46.560948+0000","last_fullsized":"2024-09-21T11:58:46.560948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:44.790174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:44.790174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:49.311243+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":837,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562464+0000","last_change":"2024-09-21T11:58:30.226389+0000","last_active":"2024-09-21T11:58:46.562464+0000","last_peered":"2024-09-21T11:58:46.562464+0000","last_clean":"2024-09-21T11:58:46.562464+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:58:46.562464+0000","last_undegraded":"2024-09-21T11:58:46.562464+0000","last_fullsized":"2024-09-21T11:58:46.562464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:30.226345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:30.226345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:50.129480+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":822,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.565642+0000","last_change":"2024-09-21T11:58:30.794254+0000","last_active":"2024-09-21T11:58:46.565642+0000","last_peered":"2024-09-21T11:58:46.565642+0000","last_clean":"2024-09-21T11:58:46.565642+0000","last_became_active":"2024-09-21T11:58:30.793717+0000","last_became_peered":"2024-09-21T11:58:30.793717+0000","last_unstale":"2024-09-21T11:58:46.565642+0000","last_undegraded":"2024-09-21T11:58:46.565642+0000","last_fullsized":"2024-09-21T11:58:46.565642+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":436,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:48.627711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.627711+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:14.347769+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"443'10356","reported_seq":15312,"reported_epoch":443,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:58:55.981951+0000","last_change":"2024-09-21T11:58:30.022462+0000","last_active":"2024-09-21T11:58:55.981951+0000","last_peered":"2024-09-21T11:58:55.981951+0000","last_clean":"2024-09-21T11:57:57.490959+0000","last_became_active":"2024-09-21T11:57:59.990813+0000","last_became_peered":"2024-09-21T11:57:59.990813+0000","last_unstale":"2024-09-21T11:58:55.981951+0000","last_undegraded":"2024-09-21T11:58:55.981951+0000","last_fullsized":"2024-09-21T11:58:55.981951+0000","mapping_epoch":421,"log_start":"410'5645","ondisk_log_start":"410'5645","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4711,"log_dups_size":0,"ondisk_log_size":4711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327024640,"num_objects":4990,"num_object_clones":0,"num_object_copies":14970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2909,"num_objects_unfound":0,"num_objects_dirty":4990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10356,"num_write_kb":491072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2416,"num_bytes_recovered":158334976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,1],"avail_no_missing":["3","1","6"],"object_location_counts":[{"shards":"1,3,6","objects":4990}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":70,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.587699+0000","last_change":"2024-09-21T11:57:58.545640+0000","last_active":"2024-09-21T11:58:52.587699+0000","last_peered":"2024-09-21T11:58:52.587699+0000","last_clean":"2024-09-21T11:58:52.587699+0000","last_became_active":"2024-09-21T11:57:58.527381+0000","last_became_peered":"2024-09-21T11:57:58.527381+0000","last_unstale":"2024-09-21T11:58:52.587699+0000","last_undegraded":"2024-09-21T11:58:52.587699+0000","last_fullsized":"2024-09-21T11:58:52.587699+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:57:54.679272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.679272+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:09.756620+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":866,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.588405+0000","last_change":"2024-09-21T11:58:17.528448+0000","last_active":"2024-09-21T11:58:52.588405+0000","last_peered":"2024-09-21T11:58:52.588405+0000","last_clean":"2024-09-21T11:58:52.588405+0000","last_became_active":"2024-09-21T11:58:17.528154+0000","last_became_peered":"2024-09-21T11:58:17.528154+0000","last_unstale":"2024-09-21T11:58:52.588405+0000","last_undegraded":"2024-09-21T11:58:52.588405+0000","last_fullsized":"2024-09-21T11:58:52.588405+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:01.083437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.083437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":1998,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:49.686308+0000","last_change":"2024-09-21T11:56:09.604887+0000","last_active":"2024-09-21T11:58:49.686308+0000","last_peered":"2024-09-21T11:58:49.686308+0000","last_clean":"2024-09-21T11:58:49.686308+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:58:49.686308+0000","last_undegraded":"2024-09-21T11:58:49.686308+0000","last_fullsized":"2024-09-21T11:58:49.686308+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:56:09.604808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.604808+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.102017+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1121,"num_read_kb":1116,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":839,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.525078+0000","last_change":"2024-09-21T11:56:57.019113+0000","last_active":"2024-09-21T11:58:46.525078+0000","last_peered":"2024-09-21T11:58:46.525078+0000","last_clean":"2024-09-21T11:58:46.525078+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:58:46.525078+0000","last_undegraded":"2024-09-21T11:58:46.525078+0000","last_fullsized":"2024-09-21T11:58:46.525078+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:57.019066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:57.019066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":519,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562252+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:58:46.562252+0000","last_peered":"2024-09-21T11:58:46.562252+0000","last_clean":"2024-09-21T11:58:46.562252+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:58:46.562252+0000","last_undegraded":"2024-09-21T11:58:46.562252+0000","last_fullsized":"2024-09-21T11:58:46.562252+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"443'10233","reported_seq":10347,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:55.957493+0000","last_change":"2024-09-21T11:56:36.509270+0000","last_active":"2024-09-21T11:58:55.957493+0000","last_peered":"2024-09-21T11:58:55.957493+0000","last_clean":"2024-09-21T11:58:55.957493+0000","last_became_active":"2024-09-21T11:56:36.508533+0000","last_became_peered":"2024-09-21T11:56:36.508533+0000","last_unstale":"2024-09-21T11:58:55.957493+0000","last_undegraded":"2024-09-21T11:58:55.957493+0000","last_fullsized":"2024-09-21T11:58:55.957493+0000","mapping_epoch":392,"log_start":"410'5522","ondisk_log_start":"410'5522","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4711,"log_dups_size":0,"ondisk_log_size":4711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318177280,"num_objects":4855,"num_object_clones":0,"num_object_copies":14565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10233,"num_write_kb":482816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":1100,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:55.598990+0000","last_change":"2024-09-21T11:56:29.095770+0000","last_active":"2024-09-21T11:58:55.598990+0000","last_peered":"2024-09-21T11:58:55.598990+0000","last_clean":"2024-09-21T11:58:55.598990+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:58:55.598990+0000","last_undegraded":"2024-09-21T11:58:55.598990+0000","last_fullsized":"2024-09-21T11:58:55.598990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:56:29.095713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.095713+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:04.820902+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":837,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.577331+0000","last_change":"2024-09-21T11:57:14.756298+0000","last_active":"2024-09-21T11:58:52.577331+0000","last_peered":"2024-09-21T11:58:52.577331+0000","last_clean":"2024-09-21T11:58:52.577331+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:58:52.577331+0000","last_undegraded":"2024-09-21T11:58:52.577331+0000","last_fullsized":"2024-09-21T11:58:52.577331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:14.756227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:14.756227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":945,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562203+0000","last_change":"2024-09-21T11:57:58.529309+0000","last_active":"2024-09-21T11:58:46.562203+0000","last_peered":"2024-09-21T11:58:46.562203+0000","last_clean":"2024-09-21T11:58:46.562203+0000","last_became_active":"2024-09-21T11:57:58.528825+0000","last_became_peered":"2024-09-21T11:57:58.528825+0000","last_unstale":"2024-09-21T11:58:46.562203+0000","last_undegraded":"2024-09-21T11:58:46.562203+0000","last_fullsized":"2024-09-21T11:58:46.562203+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:48.495794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.495794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:54.548925+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":826,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562658+0000","last_change":"2024-09-21T11:58:28.274182+0000","last_active":"2024-09-21T11:58:46.562658+0000","last_peered":"2024-09-21T11:58:46.562658+0000","last_clean":"2024-09-21T11:58:46.562658+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:58:46.562658+0000","last_undegraded":"2024-09-21T11:58:46.562658+0000","last_fullsized":"2024-09-21T11:58:46.562658+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:28.274120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.274120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.924347+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":837,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524991+0000","last_change":"2024-09-21T11:57:05.904684+0000","last_active":"2024-09-21T11:58:46.524991+0000","last_peered":"2024-09-21T11:58:46.524991+0000","last_clean":"2024-09-21T11:58:46.524991+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:58:46.524991+0000","last_undegraded":"2024-09-21T11:58:46.524991+0000","last_fullsized":"2024-09-21T11:58:46.524991+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:05.904621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:05.904621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"443'10305","reported_seq":10419,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:55.823501+0000","last_change":"2024-09-21T11:56:36.508185+0000","last_active":"2024-09-21T11:58:55.823501+0000","last_peered":"2024-09-21T11:58:55.823501+0000","last_clean":"2024-09-21T11:58:55.823501+0000","last_became_active":"2024-09-21T11:56:36.507849+0000","last_became_peered":"2024-09-21T11:56:36.507849+0000","last_unstale":"2024-09-21T11:58:55.823501+0000","last_undegraded":"2024-09-21T11:58:55.823501+0000","last_fullsized":"2024-09-21T11:58:55.823501+0000","mapping_epoch":392,"log_start":"406'4904","ondisk_log_start":"406'4904","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5401,"log_dups_size":0,"ondisk_log_size":5401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.720402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315949056,"num_objects":4821,"num_object_clones":0,"num_object_copies":14463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10305,"num_write_kb":484032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":853,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579495+0000","last_change":"2024-09-21T11:57:43.928736+0000","last_active":"2024-09-21T11:58:52.579495+0000","last_peered":"2024-09-21T11:58:52.579495+0000","last_clean":"2024-09-21T11:58:52.579495+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:58:52.579495+0000","last_undegraded":"2024-09-21T11:58:52.579495+0000","last_fullsized":"2024-09-21T11:58:52.579495+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:57:43.928681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.928681+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:55.242338+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":342,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.539809+0000","last_change":"2024-09-21T11:58:17.529749+0000","last_active":"2024-09-21T11:58:46.539809+0000","last_peered":"2024-09-21T11:58:46.539809+0000","last_clean":"2024-09-21T11:58:46.539809+0000","last_became_active":"2024-09-21T11:58:17.529395+0000","last_became_peered":"2024-09-21T11:58:17.529395+0000","last_unstale":"2024-09-21T11:58:46.539809+0000","last_undegraded":"2024-09-21T11:58:46.539809+0000","last_fullsized":"2024-09-21T11:58:46.539809+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:05.663102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.663102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":839,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524926+0000","last_change":"2024-09-21T11:58:29.807560+0000","last_active":"2024-09-21T11:58:46.524926+0000","last_peered":"2024-09-21T11:58:46.524926+0000","last_clean":"2024-09-21T11:58:46.524926+0000","last_became_active":"2024-09-21T11:58:17.527945+0000","last_became_peered":"2024-09-21T11:58:17.527945+0000","last_unstale":"2024-09-21T11:58:46.524926+0000","last_undegraded":"2024-09-21T11:58:46.524926+0000","last_fullsized":"2024-09-21T11:58:46.524926+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:29.807504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:29.807504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:58.143758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":830,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.576811+0000","last_change":"2024-09-21T11:56:28.556475+0000","last_active":"2024-09-21T11:58:52.576811+0000","last_peered":"2024-09-21T11:58:52.576811+0000","last_clean":"2024-09-21T11:58:52.576811+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:58:52.576811+0000","last_undegraded":"2024-09-21T11:58:52.576811+0000","last_fullsized":"2024-09-21T11:58:52.576811+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.556423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.556423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":77,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.523947+0000","last_change":"2024-09-21T11:57:54.931806+0000","last_active":"2024-09-21T11:58:46.523947+0000","last_peered":"2024-09-21T11:58:46.523947+0000","last_clean":"2024-09-21T11:58:46.523947+0000","last_became_active":"2024-09-21T11:57:45.534494+0000","last_became_peered":"2024-09-21T11:57:45.534494+0000","last_unstale":"2024-09-21T11:58:46.523947+0000","last_undegraded":"2024-09-21T11:58:46.523947+0000","last_fullsized":"2024-09-21T11:58:46.523947+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:54.931704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.931704+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:23.002342+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2","version":"442'9985","reported_seq":10097,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:51.874015+0000","last_change":"2024-09-21T11:56:36.511935+0000","last_active":"2024-09-21T11:58:51.874015+0000","last_peered":"2024-09-21T11:58:51.874015+0000","last_clean":"2024-09-21T11:58:51.874015+0000","last_became_active":"2024-09-21T11:56:36.511642+0000","last_became_peered":"2024-09-21T11:56:36.511642+0000","last_unstale":"2024-09-21T11:58:51.874015+0000","last_undegraded":"2024-09-21T11:58:51.874015+0000","last_fullsized":"2024-09-21T11:58:51.874015+0000","mapping_epoch":392,"log_start":"402'3797","ondisk_log_start":"402'3797","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":6188,"log_dups_size":0,"ondisk_log_size":6188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:58.985148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330629120,"num_objects":5045,"num_object_clones":0,"num_object_copies":15135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9985,"num_write_kb":480960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":956,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561707+0000","last_change":"2024-09-21T11:58:17.555274+0000","last_active":"2024-09-21T11:58:46.561707+0000","last_peered":"2024-09-21T11:58:46.561707+0000","last_clean":"2024-09-21T11:58:46.561707+0000","last_became_active":"2024-09-21T11:58:17.527029+0000","last_became_peered":"2024-09-21T11:58:17.527029+0000","last_unstale":"2024-09-21T11:58:46.561707+0000","last_undegraded":"2024-09-21T11:58:46.561707+0000","last_fullsized":"2024-09-21T11:58:46.561707+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:56:03.667750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:03.667750+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:37.084768+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":857,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579416+0000","last_change":"2024-09-21T11:56:28.045735+0000","last_active":"2024-09-21T11:58:52.579416+0000","last_peered":"2024-09-21T11:58:52.579416+0000","last_clean":"2024-09-21T11:58:52.579416+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:58:52.579416+0000","last_undegraded":"2024-09-21T11:58:52.579416+0000","last_fullsized":"2024-09-21T11:58:52.579416+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.045670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.045670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:01.924205+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":527,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562172+0000","last_change":"2024-09-21T11:57:42.599025+0000","last_active":"2024-09-21T11:58:46.562172+0000","last_peered":"2024-09-21T11:58:46.562172+0000","last_clean":"2024-09-21T11:58:46.562172+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:58:46.562172+0000","last_undegraded":"2024-09-21T11:58:46.562172+0000","last_fullsized":"2024-09-21T11:58:46.562172+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:42.598710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:42.598710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:59.154856+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":830,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561385+0000","last_change":"2024-09-21T11:57:48.898803+0000","last_active":"2024-09-21T11:58:46.561385+0000","last_peered":"2024-09-21T11:58:46.561385+0000","last_clean":"2024-09-21T11:58:46.561385+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:58:46.561385+0000","last_undegraded":"2024-09-21T11:58:46.561385+0000","last_fullsized":"2024-09-21T11:58:46.561385+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:48.898731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.898731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:52.841115+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":943,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562140+0000","last_change":"2024-09-21T11:57:52.493051+0000","last_active":"2024-09-21T11:58:46.562140+0000","last_peered":"2024-09-21T11:58:46.562140+0000","last_clean":"2024-09-21T11:58:46.562140+0000","last_became_active":"2024-09-21T11:57:45.516610+0000","last_became_peered":"2024-09-21T11:57:45.516610+0000","last_unstale":"2024-09-21T11:58:46.562140+0000","last_undegraded":"2024-09-21T11:58:46.562140+0000","last_fullsized":"2024-09-21T11:58:46.562140+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:52.492971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:52.492971+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:03.920474+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3","version":"443'10237","reported_seq":10431,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:55.884125+0000","last_change":"2024-09-21T11:58:22.114392+0000","last_active":"2024-09-21T11:58:55.884125+0000","last_peered":"2024-09-21T11:58:55.884125+0000","last_clean":"2024-09-21T11:58:55.884125+0000","last_became_active":"2024-09-21T11:56:36.504516+0000","last_became_peered":"2024-09-21T11:56:36.504516+0000","last_unstale":"2024-09-21T11:58:55.884125+0000","last_undegraded":"2024-09-21T11:58:55.884125+0000","last_fullsized":"2024-09-21T11:58:55.884125+0000","mapping_epoch":392,"log_start":"406'4795","ondisk_log_start":"406'4795","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"432'8216","last_scrub_stamp":"2024-09-21T11:58:22.114283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:58:22.114283+0000","objects_scrubbed":7071,"log_size":5442,"log_dups_size":0,"ondisk_log_size":5442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:43.111443+0000","scrub_duration":2344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":326959104,"num_objects":4989,"num_object_clones":0,"num_object_copies":14967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10237,"num_write_kb":487232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":814,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.577958+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:58:52.577958+0000","last_peered":"2024-09-21T11:58:52.577958+0000","last_clean":"2024-09-21T11:58:52.577958+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:58:52.577958+0000","last_undegraded":"2024-09-21T11:58:52.577958+0000","last_fullsized":"2024-09-21T11:58:52.577958+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":834,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.580479+0000","last_change":"2024-09-21T11:57:58.535943+0000","last_active":"2024-09-21T11:58:52.580479+0000","last_peered":"2024-09-21T11:58:52.580479+0000","last_clean":"2024-09-21T11:58:52.580479+0000","last_became_active":"2024-09-21T11:57:58.535642+0000","last_became_peered":"2024-09-21T11:57:58.535642+0000","last_unstale":"2024-09-21T11:58:52.580479+0000","last_undegraded":"2024-09-21T11:58:52.580479+0000","last_fullsized":"2024-09-21T11:58:52.580479+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:09.486386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.486386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.896343+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1069,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561659+0000","last_change":"2024-09-21T11:56:08.680971+0000","last_active":"2024-09-21T11:58:46.561659+0000","last_peered":"2024-09-21T11:58:46.561659+0000","last_clean":"2024-09-21T11:58:46.561659+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:58:46.561659+0000","last_undegraded":"2024-09-21T11:58:46.561659+0000","last_fullsized":"2024-09-21T11:58:46.561659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:56:08.680766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.680766+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:46.070618+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":661,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.563591+0000","last_change":"2024-09-21T11:58:17.532859+0000","last_active":"2024-09-21T11:58:46.563591+0000","last_peered":"2024-09-21T11:58:46.563591+0000","last_clean":"2024-09-21T11:58:46.563591+0000","last_became_active":"2024-09-21T11:58:17.531268+0000","last_became_peered":"2024-09-21T11:58:17.531268+0000","last_unstale":"2024-09-21T11:58:46.563591+0000","last_undegraded":"2024-09-21T11:58:46.563591+0000","last_fullsized":"2024-09-21T11:58:46.563591+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:51.665137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.665137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:57.566087+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"442'9916","reported_seq":10028,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:51.277800+0000","last_change":"2024-09-21T11:56:36.508655+0000","last_active":"2024-09-21T11:58:51.277800+0000","last_peered":"2024-09-21T11:58:51.277800+0000","last_clean":"2024-09-21T11:58:51.277800+0000","last_became_active":"2024-09-21T11:56:36.508299+0000","last_became_peered":"2024-09-21T11:56:36.508299+0000","last_unstale":"2024-09-21T11:58:51.277800+0000","last_undegraded":"2024-09-21T11:58:51.277800+0000","last_fullsized":"2024-09-21T11:58:51.277800+0000","mapping_epoch":392,"log_start":"408'5215","ondisk_log_start":"408'5215","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4701,"log_dups_size":0,"ondisk_log_size":4701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:12.333452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":338427904,"num_objects":5164,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9916,"num_write_kb":482560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":874,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.590240+0000","last_change":"2024-09-21T11:56:48.169371+0000","last_active":"2024-09-21T11:58:52.590240+0000","last_peered":"2024-09-21T11:58:52.590240+0000","last_clean":"2024-09-21T11:58:52.590240+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:58:52.590240+0000","last_undegraded":"2024-09-21T11:58:52.590240+0000","last_fullsized":"2024-09-21T11:58:52.590240+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:48.168795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.168795+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":857,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524652+0000","last_change":"2024-09-21T11:56:59.989715+0000","last_active":"2024-09-21T11:58:46.524652+0000","last_peered":"2024-09-21T11:58:46.524652+0000","last_clean":"2024-09-21T11:58:46.524652+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:58:46.524652+0000","last_undegraded":"2024-09-21T11:58:46.524652+0000","last_fullsized":"2024-09-21T11:58:46.524652+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:59.989636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.989636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":832,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.560870+0000","last_change":"2024-09-21T11:58:21.147985+0000","last_active":"2024-09-21T11:58:46.560870+0000","last_peered":"2024-09-21T11:58:46.560870+0000","last_clean":"2024-09-21T11:58:46.560870+0000","last_became_active":"2024-09-21T11:58:17.525791+0000","last_became_peered":"2024-09-21T11:58:17.525791+0000","last_unstale":"2024-09-21T11:58:46.560870+0000","last_undegraded":"2024-09-21T11:58:46.560870+0000","last_fullsized":"2024-09-21T11:58:46.560870+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:21.147942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:21.147942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:41.780615+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":56,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.587180+0000","last_change":"2024-09-21T11:57:58.529616+0000","last_active":"2024-09-21T11:58:52.587180+0000","last_peered":"2024-09-21T11:58:52.587180+0000","last_clean":"2024-09-21T11:58:52.587180+0000","last_became_active":"2024-09-21T11:57:58.529278+0000","last_became_peered":"2024-09-21T11:57:58.529278+0000","last_unstale":"2024-09-21T11:58:52.587180+0000","last_undegraded":"2024-09-21T11:58:52.587180+0000","last_fullsized":"2024-09-21T11:58:52.587180+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:35.532577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:35.532577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"442'9805","reported_seq":9917,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:51.277598+0000","last_change":"2024-09-21T11:56:36.506979+0000","last_active":"2024-09-21T11:58:51.277598+0000","last_peered":"2024-09-21T11:58:51.277598+0000","last_clean":"2024-09-21T11:58:51.277598+0000","last_became_active":"2024-09-21T11:56:36.506694+0000","last_became_peered":"2024-09-21T11:56:36.506694+0000","last_unstale":"2024-09-21T11:58:51.277598+0000","last_undegraded":"2024-09-21T11:58:51.277598+0000","last_fullsized":"2024-09-21T11:58:51.277598+0000","mapping_epoch":392,"log_start":"407'5019","ondisk_log_start":"407'5019","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4786,"log_dups_size":0,"ondisk_log_size":4786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:03.357476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":336658432,"num_objects":5137,"num_object_clones":0,"num_object_copies":15411,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9805,"num_write_kb":478144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":940,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579114+0000","last_change":"2024-09-21T11:58:29.624654+0000","last_active":"2024-09-21T11:58:52.579114+0000","last_peered":"2024-09-21T11:58:52.579114+0000","last_clean":"2024-09-21T11:58:52.579114+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:58:52.579114+0000","last_undegraded":"2024-09-21T11:58:52.579114+0000","last_fullsized":"2024-09-21T11:58:52.579114+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:58:29.624549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:29.624549+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:48.198468+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":825,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562746+0000","last_change":"2024-09-21T11:56:50.313230+0000","last_active":"2024-09-21T11:58:46.562746+0000","last_peered":"2024-09-21T11:58:46.562746+0000","last_clean":"2024-09-21T11:58:46.562746+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:58:46.562746+0000","last_undegraded":"2024-09-21T11:58:46.562746+0000","last_fullsized":"2024-09-21T11:58:46.562746+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.313165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.313165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.295853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":834,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.560346+0000","last_change":"2024-09-21T11:58:28.041091+0000","last_active":"2024-09-21T11:58:46.560346+0000","last_peered":"2024-09-21T11:58:46.560346+0000","last_clean":"2024-09-21T11:58:46.560346+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:58:46.560346+0000","last_undegraded":"2024-09-21T11:58:46.560346+0000","last_fullsized":"2024-09-21T11:58:46.560346+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:28.041026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.041026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:52.616083+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":829,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562109+0000","last_change":"2024-09-21T11:56:55.414105+0000","last_active":"2024-09-21T11:58:46.562109+0000","last_peered":"2024-09-21T11:58:46.562109+0000","last_clean":"2024-09-21T11:58:46.562109+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:58:46.562109+0000","last_undegraded":"2024-09-21T11:58:46.562109+0000","last_fullsized":"2024-09-21T11:58:46.562109+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:55.414046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:55.414046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:05.330948+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"442'10037","reported_seq":21587,"reported_epoch":442,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:58:51.788314+0000","last_change":"2024-09-21T11:58:17.536432+0000","last_active":"2024-09-21T11:58:51.788314+0000","last_peered":"2024-09-21T11:58:51.788314+0000","last_clean":"2024-09-21T11:57:57.476661+0000","last_became_active":"2024-09-21T11:57:59.988777+0000","last_became_peered":"2024-09-21T11:57:59.988777+0000","last_unstale":"2024-09-21T11:58:51.788314+0000","last_undegraded":"2024-09-21T11:58:51.788314+0000","last_fullsized":"2024-09-21T11:58:51.788314+0000","mapping_epoch":421,"log_start":"408'5215","ondisk_log_start":"408'5215","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4822,"log_dups_size":0,"ondisk_log_size":4822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":337444864,"num_objects":5149,"num_object_clones":0,"num_object_copies":15447,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":713,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10037,"num_write_kb":485952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5715,"num_bytes_recovered":374538240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[1,2,4],"avail_no_missing":["1","2","4"],"object_location_counts":[{"shards":"1,2,4","objects":5149}],"blocked_by":[],"up_primary":0,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":852,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562342+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:58:46.562342+0000","last_peered":"2024-09-21T11:58:46.562342+0000","last_clean":"2024-09-21T11:58:46.562342+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:58:46.562342+0000","last_undegraded":"2024-09-21T11:58:46.562342+0000","last_fullsized":"2024-09-21T11:58:46.562342+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":892,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562992+0000","last_change":"2024-09-21T11:58:18.420036+0000","last_active":"2024-09-21T11:58:46.562992+0000","last_peered":"2024-09-21T11:58:46.562992+0000","last_clean":"2024-09-21T11:58:46.562992+0000","last_became_active":"2024-09-21T11:58:17.530676+0000","last_became_peered":"2024-09-21T11:58:17.530676+0000","last_unstale":"2024-09-21T11:58:46.562992+0000","last_undegraded":"2024-09-21T11:58:46.562992+0000","last_fullsized":"2024-09-21T11:58:46.562992+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:18.419990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:58:18.419990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:25.445566+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":840,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.539663+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:58:46.539663+0000","last_peered":"2024-09-21T11:58:46.539663+0000","last_clean":"2024-09-21T11:58:46.539663+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:58:46.539663+0000","last_undegraded":"2024-09-21T11:58:46.539663+0000","last_fullsized":"2024-09-21T11:58:46.539663+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.052833+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":830,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.580006+0000","last_change":"2024-09-21T11:56:02.541452+0000","last_active":"2024-09-21T11:58:52.580006+0000","last_peered":"2024-09-21T11:58:52.580006+0000","last_clean":"2024-09-21T11:58:52.580006+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:58:52.580006+0000","last_undegraded":"2024-09-21T11:58:52.580006+0000","last_fullsized":"2024-09-21T11:58:52.580006+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.541232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.541232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:09.854321+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"442'9762","reported_seq":9874,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:51.446543+0000","last_change":"2024-09-21T11:56:36.624091+0000","last_active":"2024-09-21T11:58:51.446543+0000","last_peered":"2024-09-21T11:58:51.446543+0000","last_clean":"2024-09-21T11:58:51.446543+0000","last_became_active":"2024-09-21T11:56:36.623765+0000","last_became_peered":"2024-09-21T11:56:36.623765+0000","last_unstale":"2024-09-21T11:58:51.446543+0000","last_undegraded":"2024-09-21T11:58:51.446543+0000","last_fullsized":"2024-09-21T11:58:51.446543+0000","mapping_epoch":392,"log_start":"404'4499","ondisk_log_start":"404'4499","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5263,"log_dups_size":0,"ondisk_log_size":5263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:03.772665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":332529664,"num_objects":5074,"num_object_clones":0,"num_object_copies":15222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9762,"num_write_kb":474752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":872,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524023+0000","last_change":"2024-09-21T11:56:29.063211+0000","last_active":"2024-09-21T11:58:46.524023+0000","last_peered":"2024-09-21T11:58:46.524023+0000","last_clean":"2024-09-21T11:58:46.524023+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:58:46.524023+0000","last_undegraded":"2024-09-21T11:58:46.524023+0000","last_fullsized":"2024-09-21T11:58:46.524023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:29.063150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.063150+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:02.807388+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":887,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562868+0000","last_change":"2024-09-21T11:57:58.539544+0000","last_active":"2024-09-21T11:58:46.562868+0000","last_peered":"2024-09-21T11:58:46.562868+0000","last_clean":"2024-09-21T11:58:46.562868+0000","last_became_active":"2024-09-21T11:57:58.528814+0000","last_became_peered":"2024-09-21T11:57:58.528814+0000","last_unstale":"2024-09-21T11:58:46.562868+0000","last_undegraded":"2024-09-21T11:58:46.562868+0000","last_fullsized":"2024-09-21T11:58:46.562868+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:57:01.221960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.221960+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.207576+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1068,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562178+0000","last_change":"2024-09-21T11:57:10.282180+0000","last_active":"2024-09-21T11:58:46.562178+0000","last_peered":"2024-09-21T11:58:46.562178+0000","last_clean":"2024-09-21T11:58:46.562178+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:58:46.562178+0000","last_undegraded":"2024-09-21T11:58:46.562178+0000","last_fullsized":"2024-09-21T11:58:46.562178+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:10.282134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.282134+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:14.298512+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":813,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524262+0000","last_change":"2024-09-21T11:57:41.791340+0000","last_active":"2024-09-21T11:58:46.524262+0000","last_peered":"2024-09-21T11:58:46.524262+0000","last_clean":"2024-09-21T11:58:46.524262+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:58:46.524262+0000","last_undegraded":"2024-09-21T11:58:46.524262+0000","last_fullsized":"2024-09-21T11:58:46.524262+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:41.791261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:41.791261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":865,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.582609+0000","last_change":"2024-09-21T11:57:58.527427+0000","last_active":"2024-09-21T11:58:52.582609+0000","last_peered":"2024-09-21T11:58:52.582609+0000","last_clean":"2024-09-21T11:58:52.582609+0000","last_became_active":"2024-09-21T11:57:58.527156+0000","last_became_peered":"2024-09-21T11:57:58.527156+0000","last_unstale":"2024-09-21T11:58:52.582609+0000","last_undegraded":"2024-09-21T11:58:52.582609+0000","last_fullsized":"2024-09-21T11:58:52.582609+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:38.938666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:38.938666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:49.629819+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":863,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579339+0000","last_change":"2024-09-21T11:56:29.528662+0000","last_active":"2024-09-21T11:58:52.579339+0000","last_peered":"2024-09-21T11:58:52.579339+0000","last_clean":"2024-09-21T11:58:52.579339+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:58:52.579339+0000","last_undegraded":"2024-09-21T11:58:52.579339+0000","last_fullsized":"2024-09-21T11:58:52.579339+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:56:29.528598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.528598+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":862,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524642+0000","last_change":"2024-09-21T11:57:15.919990+0000","last_active":"2024-09-21T11:58:46.524642+0000","last_peered":"2024-09-21T11:58:46.524642+0000","last_clean":"2024-09-21T11:58:46.524642+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:58:46.524642+0000","last_undegraded":"2024-09-21T11:58:46.524642+0000","last_fullsized":"2024-09-21T11:58:46.524642+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:15.919902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:15.919902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":858,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562424+0000","last_change":"2024-09-21T11:58:20.347779+0000","last_active":"2024-09-21T11:58:46.562424+0000","last_peered":"2024-09-21T11:58:46.562424+0000","last_clean":"2024-09-21T11:58:46.562424+0000","last_became_active":"2024-09-21T11:58:17.529420+0000","last_became_peered":"2024-09-21T11:58:17.529420+0000","last_unstale":"2024-09-21T11:58:46.562424+0000","last_undegraded":"2024-09-21T11:58:46.562424+0000","last_fullsized":"2024-09-21T11:58:46.562424+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:20.347736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:20.347736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:47.925674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":856,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.589347+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:58:52.589347+0000","last_peered":"2024-09-21T11:58:52.589347+0000","last_clean":"2024-09-21T11:58:52.589347+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:58:52.589347+0000","last_undegraded":"2024-09-21T11:58:52.589347+0000","last_fullsized":"2024-09-21T11:58:52.589347+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":817,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562299+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:58:46.562299+0000","last_peered":"2024-09-21T11:58:46.562299+0000","last_clean":"2024-09-21T11:58:46.562299+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:58:46.562299+0000","last_undegraded":"2024-09-21T11:58:46.562299+0000","last_fullsized":"2024-09-21T11:58:46.562299+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.174525+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":849,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579123+0000","last_change":"2024-09-21T11:56:50.696344+0000","last_active":"2024-09-21T11:58:52.579123+0000","last_peered":"2024-09-21T11:58:52.579123+0000","last_clean":"2024-09-21T11:58:52.579123+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:58:52.579123+0000","last_undegraded":"2024-09-21T11:58:52.579123+0000","last_fullsized":"2024-09-21T11:58:52.579123+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.696253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.696253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":822,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.560225+0000","last_change":"2024-09-21T11:57:12.518014+0000","last_active":"2024-09-21T11:58:46.560225+0000","last_peered":"2024-09-21T11:58:46.560225+0000","last_clean":"2024-09-21T11:58:46.560225+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:58:46.560225+0000","last_undegraded":"2024-09-21T11:58:46.560225+0000","last_fullsized":"2024-09-21T11:58:46.560225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:12.517953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:12.517953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":853,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524983+0000","last_change":"2024-09-21T11:56:34.119200+0000","last_active":"2024-09-21T11:58:46.524983+0000","last_peered":"2024-09-21T11:58:46.524983+0000","last_clean":"2024-09-21T11:58:46.524983+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:58:46.524983+0000","last_undegraded":"2024-09-21T11:58:46.524983+0000","last_fullsized":"2024-09-21T11:58:46.524983+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:34.119128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:34.119128+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:00.786471+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":851,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.587543+0000","last_change":"2024-09-21T11:57:49.993421+0000","last_active":"2024-09-21T11:58:52.587543+0000","last_peered":"2024-09-21T11:58:52.587543+0000","last_clean":"2024-09-21T11:58:52.587543+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:58:52.587543+0000","last_undegraded":"2024-09-21T11:58:52.587543+0000","last_fullsized":"2024-09-21T11:58:52.587543+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:49.993339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:49.993339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:52.674944+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1040,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:55.599547+0000","last_change":"2024-09-21T11:55:51.317564+0000","last_active":"2024-09-21T11:58:55.599547+0000","last_peered":"2024-09-21T11:58:55.599547+0000","last_clean":"2024-09-21T11:58:55.599547+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:58:55.599547+0000","last_undegraded":"2024-09-21T11:58:55.599547+0000","last_fullsized":"2024-09-21T11:58:55.599547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:51.317473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.317473+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:04.938497+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":319,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561296+0000","last_change":"2024-09-21T11:58:20.129924+0000","last_active":"2024-09-21T11:58:46.561296+0000","last_peered":"2024-09-21T11:58:46.561296+0000","last_clean":"2024-09-21T11:58:46.561296+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:58:46.561296+0000","last_undegraded":"2024-09-21T11:58:46.561296+0000","last_fullsized":"2024-09-21T11:58:46.561296+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:20.129854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:20.129854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:38.523985+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":884,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.578050+0000","last_change":"2024-09-21T11:57:16.786845+0000","last_active":"2024-09-21T11:58:52.578050+0000","last_peered":"2024-09-21T11:58:52.578050+0000","last_clean":"2024-09-21T11:58:52.578050+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:58:52.578050+0000","last_undegraded":"2024-09-21T11:58:52.578050+0000","last_fullsized":"2024-09-21T11:58:52.578050+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:57:16.786756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:16.786756+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":833,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562613+0000","last_change":"2024-09-21T11:57:40.391494+0000","last_active":"2024-09-21T11:58:46.562613+0000","last_peered":"2024-09-21T11:58:46.562613+0000","last_clean":"2024-09-21T11:58:46.562613+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:58:46.562613+0000","last_undegraded":"2024-09-21T11:58:46.562613+0000","last_fullsized":"2024-09-21T11:58:46.562613+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:40.391409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:40.391409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:08.001729+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":1090,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524852+0000","last_change":"2024-09-21T11:56:56.051487+0000","last_active":"2024-09-21T11:58:46.524852+0000","last_peered":"2024-09-21T11:58:46.524852+0000","last_clean":"2024-09-21T11:58:46.524852+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:58:46.524852+0000","last_undegraded":"2024-09-21T11:58:46.524852+0000","last_fullsized":"2024-09-21T11:58:46.524852+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:56.051300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:56.051300+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.828079+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":56,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.587619+0000","last_change":"2024-09-21T11:57:58.534490+0000","last_active":"2024-09-21T11:58:52.587619+0000","last_peered":"2024-09-21T11:58:52.587619+0000","last_clean":"2024-09-21T11:58:52.587619+0000","last_became_active":"2024-09-21T11:57:58.533810+0000","last_became_peered":"2024-09-21T11:57:58.533810+0000","last_unstale":"2024-09-21T11:58:52.587619+0000","last_undegraded":"2024-09-21T11:58:52.587619+0000","last_fullsized":"2024-09-21T11:58:52.587619+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.280698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.280698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":872,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579690+0000","last_change":"2024-09-21T11:58:28.658668+0000","last_active":"2024-09-21T11:58:52.579690+0000","last_peered":"2024-09-21T11:58:52.579690+0000","last_clean":"2024-09-21T11:58:52.579690+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:58:52.579690+0000","last_undegraded":"2024-09-21T11:58:52.579690+0000","last_fullsized":"2024-09-21T11:58:52.579690+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:28.658574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.658574+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:54.766743+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":328,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562763+0000","last_change":"2024-09-21T11:55:24.538954+0000","last_active":"2024-09-21T11:58:46.562763+0000","last_peered":"2024-09-21T11:58:46.562763+0000","last_clean":"2024-09-21T11:58:46.562763+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:58:46.562763+0000","last_undegraded":"2024-09-21T11:58:46.562763+0000","last_fullsized":"2024-09-21T11:58:46.562763+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:24.538880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:24.538880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:56.460358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":527,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.575155+0000","last_change":"2024-09-21T11:56:00.441644+0000","last_active":"2024-09-21T11:58:46.575155+0000","last_peered":"2024-09-21T11:58:46.575155+0000","last_clean":"2024-09-21T11:58:46.575155+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:58:46.575155+0000","last_undegraded":"2024-09-21T11:58:46.575155+0000","last_fullsized":"2024-09-21T11:58:46.575155+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:00.441565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:00.441565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":822,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.576405+0000","last_change":"2024-09-21T11:57:58.531539+0000","last_active":"2024-09-21T11:58:52.576405+0000","last_peered":"2024-09-21T11:58:52.576405+0000","last_clean":"2024-09-21T11:58:52.576405+0000","last_became_active":"2024-09-21T11:57:58.531213+0000","last_became_peered":"2024-09-21T11:57:58.531213+0000","last_unstale":"2024-09-21T11:58:52.576405+0000","last_undegraded":"2024-09-21T11:58:52.576405+0000","last_fullsized":"2024-09-21T11:58:52.576405+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:01.434123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.434123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:02.942881+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":538,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.575419+0000","last_change":"2024-09-21T11:55:59.419239+0000","last_active":"2024-09-21T11:58:46.575419+0000","last_peered":"2024-09-21T11:58:46.575419+0000","last_clean":"2024-09-21T11:58:46.575419+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:58:46.575419+0000","last_undegraded":"2024-09-21T11:58:46.575419+0000","last_fullsized":"2024-09-21T11:58:46.575419+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:59.419148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.419148+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":824,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562617+0000","last_change":"2024-09-21T11:57:17.492529+0000","last_active":"2024-09-21T11:58:46.562617+0000","last_peered":"2024-09-21T11:58:46.562617+0000","last_clean":"2024-09-21T11:58:46.562617+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:58:46.562617+0000","last_undegraded":"2024-09-21T11:58:46.562617+0000","last_fullsized":"2024-09-21T11:58:46.562617+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:17.492466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:17.492466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:37.151802+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":834,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.580098+0000","last_change":"2024-09-21T11:58:24.939301+0000","last_active":"2024-09-21T11:58:52.580098+0000","last_peered":"2024-09-21T11:58:52.580098+0000","last_clean":"2024-09-21T11:58:52.580098+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:58:52.580098+0000","last_undegraded":"2024-09-21T11:58:52.580098+0000","last_fullsized":"2024-09-21T11:58:52.580098+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:24.939230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:24.939230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.727906+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":862,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.587069+0000","last_change":"2024-09-21T11:57:37.071994+0000","last_active":"2024-09-21T11:58:52.587069+0000","last_peered":"2024-09-21T11:58:52.587069+0000","last_clean":"2024-09-21T11:58:52.587069+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:58:52.587069+0000","last_undegraded":"2024-09-21T11:58:52.587069+0000","last_fullsized":"2024-09-21T11:58:52.587069+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:57:37.071908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.071908+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":863,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.576636+0000","last_change":"2024-09-21T11:57:18.759800+0000","last_active":"2024-09-21T11:58:52.576636+0000","last_peered":"2024-09-21T11:58:52.576636+0000","last_clean":"2024-09-21T11:58:52.576636+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:58:52.576636+0000","last_undegraded":"2024-09-21T11:58:52.576636+0000","last_fullsized":"2024-09-21T11:58:52.576636+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:57:18.759710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:18.759710+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":846,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.524687+0000","last_change":"2024-09-21T11:56:47.967479+0000","last_active":"2024-09-21T11:58:46.524687+0000","last_peered":"2024-09-21T11:58:46.524687+0000","last_clean":"2024-09-21T11:58:46.524687+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:58:46.524687+0000","last_undegraded":"2024-09-21T11:58:46.524687+0000","last_fullsized":"2024-09-21T11:58:46.524687+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:47.967392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:47.967392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:15.865477+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":809,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.561899+0000","last_change":"2024-09-21T11:58:23.430651+0000","last_active":"2024-09-21T11:58:46.561899+0000","last_peered":"2024-09-21T11:58:46.561899+0000","last_clean":"2024-09-21T11:58:46.561899+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:58:46.561899+0000","last_undegraded":"2024-09-21T11:58:46.561899+0000","last_fullsized":"2024-09-21T11:58:46.561899+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:23.430517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:23.430517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:25.920377+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":838,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.580391+0000","last_change":"2024-09-21T11:57:58.537818+0000","last_active":"2024-09-21T11:58:52.580391+0000","last_peered":"2024-09-21T11:58:52.580391+0000","last_clean":"2024-09-21T11:58:52.580391+0000","last_became_active":"2024-09-21T11:57:58.537121+0000","last_became_peered":"2024-09-21T11:57:58.537121+0000","last_unstale":"2024-09-21T11:58:52.580391+0000","last_undegraded":"2024-09-21T11:58:52.580391+0000","last_fullsized":"2024-09-21T11:58:52.580391+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:43.803926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.803926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:02.513607+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":867,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.581653+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:58:52.581653+0000","last_peered":"2024-09-21T11:58:52.581653+0000","last_clean":"2024-09-21T11:58:52.581653+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:58:52.581653+0000","last_undegraded":"2024-09-21T11:58:52.581653+0000","last_fullsized":"2024-09-21T11:58:52.581653+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":818,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.589533+0000","last_change":"2024-09-21T11:56:53.232038+0000","last_active":"2024-09-21T11:58:52.589533+0000","last_peered":"2024-09-21T11:58:52.589533+0000","last_clean":"2024-09-21T11:58:52.589533+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:58:52.589533+0000","last_undegraded":"2024-09-21T11:58:52.589533+0000","last_fullsized":"2024-09-21T11:58:52.589533+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.231963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.231963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":841,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579060+0000","last_change":"2024-09-21T11:58:17.528224+0000","last_active":"2024-09-21T11:58:52.579060+0000","last_peered":"2024-09-21T11:58:52.579060+0000","last_clean":"2024-09-21T11:58:52.579060+0000","last_became_active":"2024-09-21T11:58:17.527743+0000","last_became_peered":"2024-09-21T11:58:17.527743+0000","last_unstale":"2024-09-21T11:58:52.579060+0000","last_undegraded":"2024-09-21T11:58:52.579060+0000","last_fullsized":"2024-09-21T11:58:52.579060+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:41.911361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:41.911361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:04.639774+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":829,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.562308+0000","last_change":"2024-09-21T11:57:39.362771+0000","last_active":"2024-09-21T11:58:46.562308+0000","last_peered":"2024-09-21T11:58:46.562308+0000","last_clean":"2024-09-21T11:58:46.562308+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:58:46.562308+0000","last_undegraded":"2024-09-21T11:58:46.562308+0000","last_fullsized":"2024-09-21T11:58:46.562308+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:39.362713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:39.362713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:58:53.366779+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":532,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-21T11:58:46.549382+0000","last_change":"2024-09-21T11:58:17.532076+0000","last_active":"2024-09-21T11:58:46.549382+0000","last_peered":"2024-09-21T11:58:46.549382+0000","last_clean":"2024-09-21T11:58:46.549382+0000","last_became_active":"2024-09-21T11:58:17.531567+0000","last_became_peered":"2024-09-21T11:58:17.531567+0000","last_unstale":"2024-09-21T11:58:46.549382+0000","last_undegraded":"2024-09-21T11:58:46.549382+0000","last_fullsized":"2024-09-21T11:58:46.549382+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":863,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.581889+0000","last_change":"2024-09-21T11:57:58.536812+0000","last_active":"2024-09-21T11:58:52.581889+0000","last_peered":"2024-09-21T11:58:52.581889+0000","last_clean":"2024-09-21T11:58:52.581889+0000","last_became_active":"2024-09-21T11:57:58.535015+0000","last_became_peered":"2024-09-21T11:57:58.535015+0000","last_unstale":"2024-09-21T11:58:52.581889+0000","last_undegraded":"2024-09-21T11:58:52.581889+0000","last_fullsized":"2024-09-21T11:58:52.581889+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:39.735027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:39.735027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":933,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.579515+0000","last_change":"2024-09-21T11:58:16.743891+0000","last_active":"2024-09-21T11:58:52.579515+0000","last_peered":"2024-09-21T11:58:52.579515+0000","last_clean":"2024-09-21T11:58:52.579515+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:58:52.579515+0000","last_undegraded":"2024-09-21T11:58:52.579515+0000","last_fullsized":"2024-09-21T11:58:52.579515+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:16.743830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:16.743830+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.512604+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":828,"reported_epoch":443,"state":"active+clean","last_fresh":"2024-09-21T11:58:52.587907+0000","last_change":"2024-09-21T11:57:48.045901+0000","last_active":"2024-09-21T11:58:52.587907+0000","last_peered":"2024-09-21T11:58:52.587907+0000","last_clean":"2024-09-21T11:58:52.587907+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:58:52.587907+0000","last_undegraded":"2024-09-21T11:58:52.587907+0000","last_fullsized":"2024-09-21T11:58:52.587907+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:48.045820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.045820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":5308088344,"num_objects":80996,"num_object_clones":0,"num_object_copies":242988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13481,"num_objects_unfound":0,"num_objects_dirty":80996,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161470,"num_write_kb":7758849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8586,"num_bytes_recovered":562626560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384077824,"data_stored":16384065608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80817,"ondisk_log_size":80817,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":6904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1121,"num_read_kb":1116,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234215,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3098500,"kb_used_data":2345332,"kb_used_omap":1,"kb_used_meta":753086,"kb_avail":90646652,"statfs":{"total":95995035648,"available":92822171648,"internally_reserved":0,"allocated":2401619968,"data_stored":2400999270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":771160522},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463149,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3145172,"kb_used_data":2518404,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":90599980,"statfs":{"total":95995035648,"available":92774379520,"internally_reserved":0,"allocated":2578845696,"data_stored":2578307804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":641792454},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724785,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1993680,"kb_used_data":1326656,"kb_used_omap":1,"kb_used_meta":667006,"kb_avail":91751472,"statfs":{"total":95995035648,"available":93953507328,"internally_reserved":0,"allocated":1358495744,"data_stored":1357862764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":683014598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733921016,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1570988,"kb_used_data":998364,"kb_used_omap":1,"kb_used_meta":572606,"kb_avail":92174164,"statfs":{"total":95995035648,"available":94386343936,"internally_reserved":0,"allocated":1022324736,"data_stored":1021762437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":586348997},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117246,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3009244,"kb_used_data":2211724,"kb_used_omap":1,"kb_used_meta":797502,"kb_avail":90735908,"statfs":{"total":95995035648,"available":92913569792,"internally_reserved":0,"allocated":2264805376,"data_stored":2264258806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":816642503},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411587,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2776652,"kb_used_data":1997180,"kb_used_omap":1,"kb_used_meta":779454,"kb_avail":90968500,"statfs":{"total":95995035648,"available":93151744000,"internally_reserved":0,"allocated":2045112320,"data_stored":2044484047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":798161349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575114,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3716064,"kb_used_data":2985168,"kb_used_omap":1,"kb_used_meta":730878,"kb_avail":90029088,"statfs":{"total":95995035648,"available":92189786112,"internally_reserved":0,"allocated":3056812032,"data_stored":3056163151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":748419526},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738640,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2476372,"kb_used_data":1710660,"kb_used_omap":1,"kb_used_meta":765566,"kb_avail":91268780,"statfs":{"total":95995035648,"available":93459230720,"internally_reserved":0,"allocated":1751715840,"data_stored":1751079876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":783940037},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1741553664,"data_stored":1741553664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3054960640,"data_stored":3054960640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2026967040,"data_stored":2026962968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2236481536,"data_stored":2236481536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1019678720,"data_stored":1019674648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1344864256,"data_stored":1344864256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2573012992,"data_stored":2573008920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2386558976,"data_stored":2386558976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:59:04.467 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph status --format=json 2024-09-21T11:59:04.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:04.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:04.571 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:04.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:04.656 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:59:04.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:04.696 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:04.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:04.697 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:04.793 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.26339.0:171576 11.e e446/431)", 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:04.782657+0000", 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0058212239999999998, 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058437519999999998, 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:04.794 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782657+0000", 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782657+0000", 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782657+0000", 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3099999999999998e-07 2024-09-21T11:59:04.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782659+0000", 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.514e-06 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782668+0000", 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4549999999999995e-06 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782673+0000", 2024-09-21T11:59:04.796 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.344e-06 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:171585 11.c 11:3e9c8802:::benchmark_data_smithi043_178089_object50350:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:04.783539+0000", 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0049382879999999999, 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050133570000000004, 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 171585 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.783539+0000", 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.783539+0000", 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.783541+0000", 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2780000000000001e-06 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.783545+0000", 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6700000000000002e-06 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.783555+0000", 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0027000000000001e-05 2024-09-21T11:59:04.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.783562+0000", 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.905e-06 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.786049+0000", 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002486149 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.786584+0000", 2024-09-21T11:59:04.800 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00053502599999999997 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4,7", 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.786731+0000", 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000147682 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:171572 11.0 11:0b0afd95:::benchmark_data_smithi043_178089_object50337:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:04.773626+0000", 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.014851405999999999, 2024-09-21T11:59:04.801 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.014991384, 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 171572 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.773626+0000", 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.773626+0000", 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.773628+0000", 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.44e-06 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.807 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.773634+0000", 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.767e-06 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.773730+0000", 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6502000000000005e-05 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.773750+0000", 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9590999999999999e-05 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.808 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.773810+0000", 2024-09-21T11:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9994000000000002e-05 2024-09-21T11:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.773920+0000", 2024-09-21T11:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010990299999999999 2024-09-21T11:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.809 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.774034+0000", 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113977 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.784134+0000", 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010100952999999999 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:171574 11.0 11:0243a2ad:::benchmark_data_smithi043_178089_object50339:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:04.777573+0000", 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010904787000000001, 2024-09-21T11:59:04.810 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011110360999999999, 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 171574 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:04.811 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.777573+0000", 2024-09-21T11:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.777573+0000", 2024-09-21T11:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:04.812 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.777575+0000", 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.556e-06 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.777580+0000", 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6099999999999997e-06 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.777593+0000", 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2821e-05 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.777600+0000", 2024-09-21T11:59:04.813 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2389999999999996e-06 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.784169+0000", 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0065684719999999997 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.786971+0000", 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.002801871 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.814 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,7", 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.787085+0000", 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114059 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:171575 11.c 11:34852424:::benchmark_data_smithi043_178089_object50340:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:04.778318+0000", 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010159488, 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010430073999999999, 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:04.815 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 171575 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778318+0000", 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778318+0000", 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.816 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778319+0000", 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1299999999999999e-07 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778323+0000", 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0609999999999997e-06 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778334+0000", 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0988000000000001e-05 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.817 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778341+0000", 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2300000000000002e-06 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778578+0000", 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023649899999999999 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778709+0000", 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000131037 2024-09-21T11:59:04.818 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4,7", 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778848+0000", 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000138787 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782974+0000", 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0041266269999999999 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.786016+0000", 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030420069999999998 2024-09-21T11:59:04.819 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:171576 11.e 11:71f12961:::benchmark_data_smithi043_178089_object50341:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:04.778360+0000", 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.010117584000000001, 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010464787999999999, 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:04.820 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 171576 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778360+0000", 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778360+0000", 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.821 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778361+0000", 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6800000000000001e-07 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778364+0000", 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.016e-06 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:04.822 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778370+0000", 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4860000000000002e-06 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778376+0000", 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9359999999999996e-06 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778519+0000", 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014246300000000001 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.823 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778653+0000", 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013463400000000001 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4,7", 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.778786+0000", 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013298300000000001 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:171579 11.e 11:76b7112c:::benchmark_data_smithi043_178089_object50344:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:04.779677+0000", 2024-09-21T11:59:04.824 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0088008140000000006, 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0092116190000000008, 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 171579 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.779677+0000", 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.779677+0000", 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.779678+0000", 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4e-07 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.779683+0000", 2024-09-21T11:59:04.826 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1760000000000001e-06 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.779694+0000", 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0838e-05 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.779702+0000", 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8310000000000001e-06 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782645+0000", 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029427849999999998 2024-09-21T11:59:04.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782795+0000", 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000150474 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4,7", 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.783542+0000", 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00074719599999999999 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.828 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.26339.0:171576 11.e e446/431)", 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:04.782550+0000", 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0059276930000000004, 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063997949999999998, 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782550+0000", 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:04.829 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782550+0000", 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782551+0000", 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0400000000000005e-07 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782555+0000", 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7220000000000001e-06 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:04.830 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782564+0000", 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.5189999999999998e-06 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782571+0000", 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1640000000000004e-06 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.26339.0:171576 11.e e446/431)", 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:04.782628+0000", 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0058499329999999999, 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063596499999999997, 2024-09-21T11:59:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "queued for pg", 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782628+0000", 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782628+0000", 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:04.832 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782628+0000", 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.3099999999999999e-07 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782630+0000", 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9810000000000002e-06 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782635+0000", 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9819999999999999e-06 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:04.833 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:04.782640+0000", 2024-09-21T11:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.391e-06 2024-09-21T11:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 9 2024-09-21T11:59:04.834 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:59:04.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:04.880 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:04.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:04.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:04.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:04 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1495262472' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:04.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:04 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:04 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1495262472' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:04.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:04 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:05.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:04 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1495262472' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:05.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:04 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:05.029 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:05.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:05.030 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:05.130 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:05.156 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:05.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:05.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:05.258 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:05.286 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:05.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:05.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:05.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:59:05.388 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:05.417 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:05.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:05.418 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:59:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:05.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:05.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:05.563 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:05.563 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:05.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:05.686 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:05.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:05.687 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:05.788 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:05.819 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:05.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:05.820 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:05.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:05.947 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:05.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:05.948 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[36454]: 4.7 scrub starts 2024-09-21T11:59:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[36454]: 4.7 scrub ok 2024-09-21T11:59:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[36454]: pgmap v932: 158 pgs: 1 peering, 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 153 active+clean; 4.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 947 op/s; 9718/220860 objects misplaced (4.400%); 20 MiB/s, 317 objects/s recovering 2024-09-21T11:59:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:59:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:05.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[36454]: osdmap e446: 8 total, 8 up, 8 in 2024-09-21T11:59:05.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[29710]: 4.7 scrub starts 2024-09-21T11:59:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[29710]: 4.7 scrub ok 2024-09-21T11:59:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[29710]: pgmap v932: 158 pgs: 1 peering, 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 153 active+clean; 4.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 947 op/s; 9718/220860 objects misplaced (4.400%); 20 MiB/s, 317 objects/s recovering 2024-09-21T11:59:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:59:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:05.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:05 smithi043 ceph-mon[29710]: osdmap e446: 8 total, 8 up, 8 in 2024-09-21T11:59:06.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:05 smithi175 ceph-mon[32613]: 4.7 scrub starts 2024-09-21T11:59:06.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:05 smithi175 ceph-mon[32613]: 4.7 scrub ok 2024-09-21T11:59:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:05 smithi175 ceph-mon[32613]: pgmap v932: 158 pgs: 1 peering, 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 153 active+clean; 4.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 947 op/s; 9718/220860 objects misplaced (4.400%); 20 MiB/s, 317 objects/s recovering 2024-09-21T11:59:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:05 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:59:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:05 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:06.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:05 smithi175 ceph-mon[32613]: osdmap e446: 8 total, 8 up, 8 in 2024-09-21T11:59:06.048 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:06.075 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:06.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:06.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:06.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:06.220 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:06.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:06.221 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:06.322 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:06.349 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:06.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:06.350 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:06.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:06.493 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:06.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:06.494 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:06.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:06.617 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:06.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:06.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:06.719 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:06.758 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:06.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:06.759 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:06.860 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:06.890 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:06.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:06.891 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:06 smithi043 ceph-mon[36454]: 6.2 scrub starts 2024-09-21T11:59:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:06 smithi043 ceph-mon[36454]: 6.2 scrub ok 2024-09-21T11:59:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:06 smithi043 ceph-mon[36454]: 4.5 scrub starts 2024-09-21T11:59:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:06 smithi043 ceph-mon[36454]: 4.5 scrub ok 2024-09-21T11:59:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:06 smithi043 ceph-mon[29710]: 6.2 scrub starts 2024-09-21T11:59:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:06 smithi043 ceph-mon[29710]: 6.2 scrub ok 2024-09-21T11:59:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:06 smithi043 ceph-mon[29710]: 4.5 scrub starts 2024-09-21T11:59:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:06 smithi043 ceph-mon[29710]: 4.5 scrub ok 2024-09-21T11:59:06.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:07.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:06 smithi175 ceph-mon[32613]: 6.2 scrub starts 2024-09-21T11:59:07.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:06 smithi175 ceph-mon[32613]: 6.2 scrub ok 2024-09-21T11:59:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:06 smithi175 ceph-mon[32613]: 4.5 scrub starts 2024-09-21T11:59:07.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:06 smithi175 ceph-mon[32613]: 4.5 scrub ok 2024-09-21T11:59:07.016 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:07.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:07.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:07.118 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:07.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:07.154 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:07.154 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:07.255 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:07.285 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:07.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:07.286 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:07.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:07.425 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:07.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:07.426 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:07.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:07.567 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:07.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:07.568 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:07.669 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:07.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:07.701 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:07.702 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:07.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:07.839 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:07.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:07.841 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:07.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:07.976 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:07.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:07.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:07.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:07 smithi043 ceph-mon[36454]: pgmap v934: 158 pgs: 1 peering, 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 153 active+clean; 4.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 991 op/s; 8814/216288 objects misplaced (4.075%); 24 MiB/s, 381 objects/s recovering 2024-09-21T11:59:07.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:07 smithi043 ceph-mon[29710]: pgmap v934: 158 pgs: 1 peering, 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 153 active+clean; 4.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 991 op/s; 8814/216288 objects misplaced (4.075%); 24 MiB/s, 381 objects/s recovering 2024-09-21T11:59:08.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:07 smithi175 ceph-mon[32613]: pgmap v934: 158 pgs: 1 peering, 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 153 active+clean; 4.4 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 991 op/s; 8814/216288 objects misplaced (4.075%); 24 MiB/s, 381 objects/s recovering 2024-09-21T11:59:08.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:08.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:08.113 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:08.113 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:08.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:08.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:08.256 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:08.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:08.356 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:08.383 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:08.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:08.384 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:08.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:08.524 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:08.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:08.526 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:08.627 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:08.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:08.659 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:08.659 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:08.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:08.800 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:08.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:08.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:08.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:08.946 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:08.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:08.947 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:59:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:59:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[36454]: 11.5 scrub starts 2024-09-21T11:59:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[36454]: osdmap e447: 8 total, 8 up, 8 in 2024-09-21T11:59:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[36454]: 1.6 scrub starts 2024-09-21T11:59:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[36454]: 1.6 scrub ok 2024-09-21T11:59:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[36454]: 4.15 scrub starts 2024-09-21T11:59:08.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[36454]: 4.15 scrub ok 2024-09-21T11:59:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:59:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:59:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[29710]: 11.5 scrub starts 2024-09-21T11:59:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[29710]: osdmap e447: 8 total, 8 up, 8 in 2024-09-21T11:59:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[29710]: 1.6 scrub starts 2024-09-21T11:59:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[29710]: 1.6 scrub ok 2024-09-21T11:59:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[29710]: 4.15 scrub starts 2024-09-21T11:59:08.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:08 smithi043 ceph-mon[29710]: 4.15 scrub ok 2024-09-21T11:59:09.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:08 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:59:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:08 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:59:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:08 smithi175 ceph-mon[32613]: 11.5 scrub starts 2024-09-21T11:59:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:08 smithi175 ceph-mon[32613]: osdmap e447: 8 total, 8 up, 8 in 2024-09-21T11:59:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:08 smithi175 ceph-mon[32613]: 1.6 scrub starts 2024-09-21T11:59:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:08 smithi175 ceph-mon[32613]: 1.6 scrub ok 2024-09-21T11:59:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:08 smithi175 ceph-mon[32613]: 4.15 scrub starts 2024-09-21T11:59:09.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:08 smithi175 ceph-mon[32613]: 4.15 scrub ok 2024-09-21T11:59:09.048 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:09.077 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:09.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:09.078 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:09.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:09.203 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:09.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:09.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:09.305 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:09.334 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:09.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:09.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:09.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:09.479 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:09.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:09.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:09.580 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:09.611 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:09.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:09.612 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:09.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[36454]: pgmap v936: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 4.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 747 op/s; 7345/208179 objects misplaced (3.528%); 23 MiB/s, 366 objects/s recovering 2024-09-21T11:59:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[36454]: 11.5 scrub ok 2024-09-21T11:59:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[36454]: 6.8 scrub starts 2024-09-21T11:59:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[36454]: 6.8 scrub ok 2024-09-21T11:59:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[36454]: 5.11 scrub starts 2024-09-21T11:59:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[36454]: 5.11 scrub ok 2024-09-21T11:59:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[36454]: 4.8 scrub starts 2024-09-21T11:59:09.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[36454]: 4.8 scrub ok 2024-09-21T11:59:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[29710]: pgmap v936: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 4.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 747 op/s; 7345/208179 objects misplaced (3.528%); 23 MiB/s, 366 objects/s recovering 2024-09-21T11:59:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[29710]: 11.5 scrub ok 2024-09-21T11:59:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[29710]: 6.8 scrub starts 2024-09-21T11:59:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[29710]: 6.8 scrub ok 2024-09-21T11:59:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[29710]: 5.11 scrub starts 2024-09-21T11:59:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[29710]: 5.11 scrub ok 2024-09-21T11:59:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[29710]: 4.8 scrub starts 2024-09-21T11:59:09.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:09 smithi043 ceph-mon[29710]: 4.8 scrub ok 2024-09-21T11:59:09.745 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:09.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:09.746 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:09.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:09.889 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:09.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:09.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:09.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:10.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:09 smithi175 ceph-mon[32613]: pgmap v936: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 4.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 747 op/s; 7345/208179 objects misplaced (3.528%); 23 MiB/s, 366 objects/s recovering 2024-09-21T11:59:10.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:09 smithi175 ceph-mon[32613]: 11.5 scrub ok 2024-09-21T11:59:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:09 smithi175 ceph-mon[32613]: 6.8 scrub starts 2024-09-21T11:59:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:09 smithi175 ceph-mon[32613]: 6.8 scrub ok 2024-09-21T11:59:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:09 smithi175 ceph-mon[32613]: 5.11 scrub starts 2024-09-21T11:59:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:09 smithi175 ceph-mon[32613]: 5.11 scrub ok 2024-09-21T11:59:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:09 smithi175 ceph-mon[32613]: 4.8 scrub starts 2024-09-21T11:59:10.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:09 smithi175 ceph-mon[32613]: 4.8 scrub ok 2024-09-21T11:59:10.018 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:10.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:10.019 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:10.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T11:59:10.057 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:10.068 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:10.120 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:10.145 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:10.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:10.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:10.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:10.274 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:10.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:10.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:10.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:10.443 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:10.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:10.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:10.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:10.585 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:10.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:10.586 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:10.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:10.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:10.725 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:10.726 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:10.785 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:59:10.785 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:59:10.802 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:59:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:59:10.826 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:10.840 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:59:10.840 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:59:10.866 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:10.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:10.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:10.883 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:59:10.883 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:59:10.919 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:59:10.919 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:59:10.952 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:59:10.952 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:59:10.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:10.974 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:59:10.974 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:59:10.974 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[36454]: 5.6 scrub starts 2024-09-21T11:59:10.974 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[36454]: 5.6 scrub ok 2024-09-21T11:59:10.974 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[36454]: osdmap e448: 8 total, 8 up, 8 in 2024-09-21T11:59:10.975 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[36454]: 11.7 scrub starts 2024-09-21T11:59:10.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:59:10.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:59:10.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[29710]: 5.6 scrub starts 2024-09-21T11:59:10.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[29710]: 5.6 scrub ok 2024-09-21T11:59:10.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[29710]: osdmap e448: 8 total, 8 up, 8 in 2024-09-21T11:59:10.975 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:10 smithi043 ceph-mon[29710]: 11.7 scrub starts 2024-09-21T11:59:10.989 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:59:10.990 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:59:10.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:10.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:10.994 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:11.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:10 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:59:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:10 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:59:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:10 smithi175 ceph-mon[32613]: 5.6 scrub starts 2024-09-21T11:59:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:10 smithi175 ceph-mon[32613]: 5.6 scrub ok 2024-09-21T11:59:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:10 smithi175 ceph-mon[32613]: osdmap e448: 8 total, 8 up, 8 in 2024-09-21T11:59:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:10 smithi175 ceph-mon[32613]: 11.7 scrub starts 2024-09-21T11:59:11.026 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:59:11.026 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:59:11.095 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:11.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:11.135 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:11.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:59:11.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:11.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:11.318 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:11.419 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:11.443 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:11.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:11.444 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:11.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:11.569 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:11.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:11.569 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:11.670 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:11.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:11.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:11.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:11.828 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:59:11.828 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"4f2c5252-780d-11ef-baf6-efdc52797490","health":{"status":"HEALTH_WARN","checks":{"PG_BACKFILL_FULL":{"severity":"HEALTH_WARN","summary":{"message":"Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1404,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":448,"num_osds":8,"num_up_osds":8,"osd_up_since":1726918789,"num_in_osds":8,"osd_in_since":1726919222,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":155},{"state_name":"active+remapped+backfilling","count":2},{"state_name":"active+remapped+backfill_toofull","count":1}],"num_pgs":158,"num_pools":7,"num_objects":69393,"data_bytes":4535228979,"bytes_used":19972812800,"bytes_avail":747987472384,"bytes_total":767960285184,"misplaced_objects":7345,"misplaced_total":208179,"misplaced_ratio":0.035282137007094858,"recovering_objects_per_sec":318,"recovering_bytes_per_sec":20843610,"recovering_keys_per_sec":0,"num_objects_recovered":1909,"num_bytes_recovered":125108224,"num_keys_recovered":0,"read_bytes_sec":853,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":704},"fsmap":{"epoch":1,"btime":"2024-09-21T11:34:07:249039+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.43:8443/","prometheus":"http://172.21.15.43:9283/"}},"servicemap":{"epoch":15,"modified":"2024-09-21T11:56:32.116130+0000","services":{"rgw":{"daemons":{"summary":"","14445":{"start_epoch":12,"start_stamp":"2024-09-21T11:40:07.452481+0000","gid":14445,"addr":"172.21.15.43:0/2311158182","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-5103-g5c1cdc3c (5c1cdc3c470dae78e91a0ed2bcf13509a046d301) squid (dev)","ceph_version_short":"19.3.0-5103-g5c1cdc3c","container_hostname":"smithi043","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi043","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024","kernel_version":"5.14.0-508.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32493484","num_handles":"1","os":"Linux","pid":"2","realm_id":"","realm_name":"","zone_id":"0b57ffae-5838-4433-ac11-de7e749d627a","zone_name":"default","zonegroup_id":"3832d8a7-33d0-4a2e-9811-afaccd40f4de","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T11:59:11.828 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:11 smithi043 ceph-mon[36454]: pgmap v938: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 4.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 705 op/s; 7345/208179 objects misplaced (3.528%); 20 MiB/s, 318 objects/s recovering 2024-09-21T11:59:11.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:11 smithi043 ceph-mon[36454]: 1.9 scrub starts 2024-09-21T11:59:11.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:11 smithi043 ceph-mon[36454]: 1.9 scrub ok 2024-09-21T11:59:11.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:11 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2335309462' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:11.829 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:11 smithi043 ceph-mon[36454]: 11.7 scrub ok 2024-09-21T11:59:11.829 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:11 smithi043 ceph-mon[29710]: pgmap v938: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 4.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 705 op/s; 7345/208179 objects misplaced (3.528%); 20 MiB/s, 318 objects/s recovering 2024-09-21T11:59:11.829 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:11 smithi043 ceph-mon[29710]: 1.9 scrub starts 2024-09-21T11:59:11.829 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:11 smithi043 ceph-mon[29710]: 1.9 scrub ok 2024-09-21T11:59:11.829 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:11 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2335309462' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:11.830 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:11 smithi043 ceph-mon[29710]: 11.7 scrub ok 2024-09-21T11:59:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:11.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:11.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:11.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:11.988 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:11.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:11.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:12.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:11 smithi175 ceph-mon[32613]: pgmap v938: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 4.2 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 705 op/s; 7345/208179 objects misplaced (3.528%); 20 MiB/s, 318 objects/s recovering 2024-09-21T11:59:12.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:11 smithi175 ceph-mon[32613]: 1.9 scrub starts 2024-09-21T11:59:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:11 smithi175 ceph-mon[32613]: 1.9 scrub ok 2024-09-21T11:59:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:11 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2335309462' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:12.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:11 smithi175 ceph-mon[32613]: 11.7 scrub ok 2024-09-21T11:59:12.011 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:59:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:59:12.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:12.134 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:12.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:12.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:12.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:12.277 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:12.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:12.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:12.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:12.415 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:12.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:12.417 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:12.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:12.546 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:12.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:12.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:12.648 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:12.676 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:12.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:12.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:12.777 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:12.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:12.819 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:12.819 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:12.909 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:12.920 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:12.942 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:12.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:12.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:12.961 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-21T11:59:12.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:59:12.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:12 smithi043 ceph-mon[36454]: 11.f scrub starts 2024-09-21T11:59:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:12 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2115439327' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T11:59:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:12 smithi043 ceph-mon[36454]: 5.7 scrub starts 2024-09-21T11:59:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:12 smithi043 ceph-mon[36454]: 5.7 scrub ok 2024-09-21T11:59:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:12 smithi043 ceph-mon[29710]: 11.f scrub starts 2024-09-21T11:59:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:12 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2115439327' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T11:59:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:12 smithi043 ceph-mon[29710]: 5.7 scrub starts 2024-09-21T11:59:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:12 smithi043 ceph-mon[29710]: 5.7 scrub ok 2024-09-21T11:59:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:12 smithi175 ceph-mon[32613]: 11.f scrub starts 2024-09-21T11:59:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:12 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2115439327' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-21T11:59:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:12 smithi175 ceph-mon[32613]: 5.7 scrub starts 2024-09-21T11:59:13.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:12 smithi175 ceph-mon[32613]: 5.7 scrub ok 2024-09-21T11:59:13.045 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:13.072 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:13.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:13.073 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:13.173 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:13.204 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:13.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:13.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:13.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:13.361 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:13.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:13.362 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:13.463 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:13.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:13.507 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:13.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:13.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:13.639 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:13.640 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:13.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:13.768 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:13.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:13.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:13.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:13.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:13.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:13.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:13.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:13.943 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[36454]: pgmap v939: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 3.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.18k op/s; 5705/194109 objects misplaced (2.939%); 25 MiB/s, 406 objects/s recovering 2024-09-21T11:59:13.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[36454]: 11.4 scrub starts 2024-09-21T11:59:13.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[36454]: 11.f scrub ok 2024-09-21T11:59:13.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T11:59:13.944 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T11:59:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[29710]: pgmap v939: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 3.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.18k op/s; 5705/194109 objects misplaced (2.939%); 25 MiB/s, 406 objects/s recovering 2024-09-21T11:59:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[29710]: 11.4 scrub starts 2024-09-21T11:59:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[29710]: 11.f scrub ok 2024-09-21T11:59:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T11:59:13.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:13 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T11:59:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:13 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:13 smithi175 ceph-mon[32613]: pgmap v939: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 3.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.18k op/s; 5705/194109 objects misplaced (2.939%); 25 MiB/s, 406 objects/s recovering 2024-09-21T11:59:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:13 smithi175 ceph-mon[32613]: 11.4 scrub starts 2024-09-21T11:59:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:13 smithi175 ceph-mon[32613]: 11.f scrub ok 2024-09-21T11:59:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:13 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T11:59:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:13 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T11:59:14.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:14.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:14.068 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:14.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:14.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:59:14.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:14.198 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:14.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:14.199 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:14.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:14.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:14.351 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:14.351 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:14.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:14.503 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:14.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:14.504 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:14.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:14.649 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:14.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:14.650 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:14.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:14.786 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:14.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:14.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:14.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:14.927 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:14.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:14.928 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:15.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:14 smithi175 ceph-mon[32613]: 11.4 scrub ok 2024-09-21T11:59:15.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:14 smithi175 ceph-mon[32613]: 6.a scrub starts 2024-09-21T11:59:15.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:14 smithi175 ceph-mon[32613]: 6.a scrub ok 2024-09-21T11:59:15.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:15.063 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:15.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:15.064 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:15.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:14 smithi043 ceph-mon[36454]: 11.4 scrub ok 2024-09-21T11:59:15.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:14 smithi043 ceph-mon[36454]: 6.a scrub starts 2024-09-21T11:59:15.145 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:14 smithi043 ceph-mon[36454]: 6.a scrub ok 2024-09-21T11:59:15.145 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:14 smithi043 ceph-mon[29710]: 11.4 scrub ok 2024-09-21T11:59:15.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:14 smithi043 ceph-mon[29710]: 6.a scrub starts 2024-09-21T11:59:15.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:14 smithi043 ceph-mon[29710]: 6.a scrub ok 2024-09-21T11:59:15.165 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:15.190 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:15.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:15.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:15.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:15.337 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:15.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:15.338 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:15.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:15.457 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:59:15.458 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:59:15.458 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:59:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:59:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:15.481 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:15.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:15.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:15.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:15.611 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:15.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:15.612 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:15.710 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:15.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:15.735 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:15.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:15.737 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:15.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:15.887 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:15.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:15.887 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:15.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:15 smithi175 ceph-mon[32613]: pgmap v940: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 3.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 962 op/s; 5705/193206 objects misplaced (2.953%); 18 MiB/s, 281 objects/s recovering 2024-09-21T11:59:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:15 smithi175 ceph-mon[32613]: 11.1 scrub starts 2024-09-21T11:59:16.017 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:16.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:16.018 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:16.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:16.153 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:16.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:16.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:16.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:15 smithi043 ceph-mon[36454]: pgmap v940: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 3.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 962 op/s; 5705/193206 objects misplaced (2.953%); 18 MiB/s, 281 objects/s recovering 2024-09-21T11:59:16.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:15 smithi043 ceph-mon[36454]: 11.1 scrub starts 2024-09-21T11:59:16.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:15 smithi043 ceph-mon[29710]: pgmap v940: 158 pgs: 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 155 active+clean; 3.9 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 962 op/s; 5705/193206 objects misplaced (2.953%); 18 MiB/s, 281 objects/s recovering 2024-09-21T11:59:16.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:15 smithi043 ceph-mon[29710]: 11.1 scrub starts 2024-09-21T11:59:16.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:16.292 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:16.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:16.293 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:16.392 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:16.394 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:16.416 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:16.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:16.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:16.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T11:59:16.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:16.580 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:16.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:16.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:16.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:16.722 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:16.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:16.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:16.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:16.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:16.852 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:16.852 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:16.953 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:16.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:16.977 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:16.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:16 smithi043 ceph-mon[36454]: 6.1e scrub starts 2024-09-21T11:59:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:16 smithi043 ceph-mon[36454]: 6.1e scrub ok 2024-09-21T11:59:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:16 smithi043 ceph-mon[36454]: 11.1 scrub ok 2024-09-21T11:59:16.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:16 smithi043 ceph-mon[29710]: 6.1e scrub starts 2024-09-21T11:59:16.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:16 smithi043 ceph-mon[29710]: 6.1e scrub ok 2024-09-21T11:59:16.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:16 smithi043 ceph-mon[29710]: 11.1 scrub ok 2024-09-21T11:59:17.063 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:16 smithi175 ceph-mon[32613]: 6.1e scrub starts 2024-09-21T11:59:17.063 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:16 smithi175 ceph-mon[32613]: 6.1e scrub ok 2024-09-21T11:59:17.063 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:16 smithi175 ceph-mon[32613]: 11.1 scrub ok 2024-09-21T11:59:17.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:17.123 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:17.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:17.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:17.225 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:17.258 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:17.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:17.259 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:17.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:17.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:17.416 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:17.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:17.514 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:17.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:17.539 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:17.584 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:17.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:17.585 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:17.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:17.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:17.720 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:17.720 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:17.821 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:17.845 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:17.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:17.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:17.861 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:59:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:59:17.947 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:17.968 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:17.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:17.969 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[29710]: 6.f scrub starts 2024-09-21T11:59:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[29710]: 6.f scrub ok 2024-09-21T11:59:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[29710]: pgmap v941: 158 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 3.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 602 B/s rd, 0 B/s wr, 800 op/s; 5309/187767 objects misplaced (2.827%); 11 MiB/s, 178 objects/s recovering 2024-09-21T11:59:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[29710]: osdmap e449: 8 total, 8 up, 8 in 2024-09-21T11:59:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2575521143' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:17.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[36454]: 6.f scrub starts 2024-09-21T11:59:17.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[36454]: 6.f scrub ok 2024-09-21T11:59:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[36454]: pgmap v941: 158 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 3.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 602 B/s rd, 0 B/s wr, 800 op/s; 5309/187767 objects misplaced (2.827%); 11 MiB/s, 178 objects/s recovering 2024-09-21T11:59:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[36454]: osdmap e449: 8 total, 8 up, 8 in 2024-09-21T11:59:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2575521143' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:17.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:17 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:18.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:17 smithi175 ceph-mon[32613]: 6.f scrub starts 2024-09-21T11:59:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:17 smithi175 ceph-mon[32613]: 6.f scrub ok 2024-09-21T11:59:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:17 smithi175 ceph-mon[32613]: pgmap v941: 158 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 3.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 602 B/s rd, 0 B/s wr, 800 op/s; 5309/187767 objects misplaced (2.827%); 11 MiB/s, 178 objects/s recovering 2024-09-21T11:59:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:17 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:17 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:17 smithi175 ceph-mon[32613]: osdmap e449: 8 total, 8 up, 8 in 2024-09-21T11:59:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:17 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2575521143' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:18.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:17 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:18.070 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:18.091 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:18.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:18.093 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:18.193 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:18.214 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:18.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:18.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:18.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:18.338 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:18.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:18.339 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:18.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:18.462 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:18.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:18.463 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:18.537 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:59:18.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:18.591 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:18.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:18.593 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:18.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:18.725 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:18.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:18.726 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:18.827 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:18.851 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:18.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:18.852 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:18.953 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:18.975 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:18.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:18.976 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:19.077 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:19.078 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[36454]: 4.1c scrub starts 2024-09-21T11:59:19.078 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[36454]: 4.1c scrub ok 2024-09-21T11:59:19.078 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[36454]: 3.d scrub starts 2024-09-21T11:59:19.078 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[36454]: 3.d scrub ok 2024-09-21T11:59:19.078 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[36454]: 3.16 scrub starts 2024-09-21T11:59:19.078 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[36454]: 3.16 scrub ok 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[36454]: osdmap e450: 8 total, 8 up, 8 in 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[29710]: 4.1c scrub starts 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[29710]: 4.1c scrub ok 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[29710]: 3.d scrub starts 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[29710]: 3.d scrub ok 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[29710]: 3.16 scrub starts 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[29710]: 3.16 scrub ok 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:59:19.079 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:18 smithi043 ceph-mon[29710]: osdmap e450: 8 total, 8 up, 8 in 2024-09-21T11:59:19.098 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:19.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:19.099 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:19.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:19.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:19.224 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:19.224 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:18 smithi175 ceph-mon[32613]: 4.1c scrub starts 2024-09-21T11:59:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:18 smithi175 ceph-mon[32613]: 4.1c scrub ok 2024-09-21T11:59:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:18 smithi175 ceph-mon[32613]: 3.d scrub starts 2024-09-21T11:59:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:18 smithi175 ceph-mon[32613]: 3.d scrub ok 2024-09-21T11:59:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:18 smithi175 ceph-mon[32613]: 3.16 scrub starts 2024-09-21T11:59:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:18 smithi175 ceph-mon[32613]: 3.16 scrub ok 2024-09-21T11:59:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:18 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:18 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:59:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:18 smithi175 ceph-mon[32613]: osdmap e450: 8 total, 8 up, 8 in 2024-09-21T11:59:19.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:19.348 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:19.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:19.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:19.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:19.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:19.472 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:19.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:19.601 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:19.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:19.603 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:19.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:19.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:19.738 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:19.738 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:19.839 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:19.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:19.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:19.972 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:20.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:20.001 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:20.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:20.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:20.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:20.141 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:20.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:20.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:19 smithi043 ceph-mon[36454]: from='client.26212 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:59:20.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:19 smithi043 ceph-mon[36454]: 1.5 scrub starts 2024-09-21T11:59:20.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:19 smithi043 ceph-mon[36454]: 1.5 scrub ok 2024-09-21T11:59:20.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:19 smithi043 ceph-mon[36454]: pgmap v943: 158 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s; 4212/179013 objects misplaced (2.353%); 17 MiB/s, 268 objects/s recovering 2024-09-21T11:59:20.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:19 smithi043 ceph-mon[36454]: osdmap e451: 8 total, 8 up, 8 in 2024-09-21T11:59:20.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:19 smithi043 ceph-mon[29710]: from='client.26212 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:59:20.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:19 smithi043 ceph-mon[29710]: 1.5 scrub starts 2024-09-21T11:59:20.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:19 smithi043 ceph-mon[29710]: 1.5 scrub ok 2024-09-21T11:59:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:19 smithi043 ceph-mon[29710]: pgmap v943: 158 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s; 4212/179013 objects misplaced (2.353%); 17 MiB/s, 268 objects/s recovering 2024-09-21T11:59:20.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:19 smithi043 ceph-mon[29710]: osdmap e451: 8 total, 8 up, 8 in 2024-09-21T11:59:20.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:20.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:19 smithi175 ceph-mon[32613]: from='client.26212 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:59:20.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:19 smithi175 ceph-mon[32613]: 1.5 scrub starts 2024-09-21T11:59:20.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:19 smithi175 ceph-mon[32613]: 1.5 scrub ok 2024-09-21T11:59:20.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:19 smithi175 ceph-mon[32613]: pgmap v943: 158 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.15k op/s; 4212/179013 objects misplaced (2.353%); 17 MiB/s, 268 objects/s recovering 2024-09-21T11:59:20.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:19 smithi175 ceph-mon[32613]: osdmap e451: 8 total, 8 up, 8 in 2024-09-21T11:59:20.279 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:20.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:20.280 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:20.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:20.421 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:20.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:20.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:20.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:20.562 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:20.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:20.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:20.563 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:59:20.564 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:59:20.605 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:59:20.605 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:59:20.638 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:59:20.638 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:59:20.664 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:20.682 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:59:20.682 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:59:20.688 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:20.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:20.689 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:20.715 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:59:20.715 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:59:20.748 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:59:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:59:20.786 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:59:20.786 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:59:20.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:20.820 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:20.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:20.821 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:20.822 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:59:20.822 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:59:20.921 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:20.946 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:20.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:20.947 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:20.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2023848829' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:20.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2023848829' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:21.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:21.072 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:21.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:21.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:21.174 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:21.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:21.201 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:21.201 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:21.214 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":942,"stamp":"2024-09-21T11:59:17.541414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3917290035,"num_objects":59964,"num_object_clones":0,"num_object_copies":179892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4212,"num_objects_unfound":0,"num_objects_dirty":59964,"num_whiteouts":0,"num_read":2269,"num_read_kb":2021,"num_write":183423,"num_write_kb":7758894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15666,"num_bytes_recovered":1003168643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82688,"ondisk_log_size":82688,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":477,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":17871500,"kb_used_data":12069960,"kb_used_omap":12,"kb_used_meta":5801139,"kb_avail":732089716,"statfs":{"total":767960285184,"available":749659869184,"internally_reserved":0,"allocated":12359639040,"data_stored":12356582273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":5940366898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-617938944,"num_objects":-9429,"num_object_clones":0,"num_object_copies":-28287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-3133,"num_objects_unfound":0,"num_objects_dirty":-9429,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9429,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2273,"num_bytes_recovered":148963328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.356080"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":66,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.721430+0000","last_change":"2024-09-21T11:57:58.529638+0000","last_active":"2024-09-21T11:59:09.721430+0000","last_peered":"2024-09-21T11:59:09.721430+0000","last_clean":"2024-09-21T11:59:09.721430+0000","last_became_active":"2024-09-21T11:57:58.529245+0000","last_became_peered":"2024-09-21T11:57:58.529245+0000","last_unstale":"2024-09-21T11:59:09.721430+0000","last_undegraded":"2024-09-21T11:59:09.721430+0000","last_fullsized":"2024-09-21T11:59:09.721430+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:43.783030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.783030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":869,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722079+0000","last_change":"2024-09-21T11:57:37.936348+0000","last_active":"2024-09-21T11:59:09.722079+0000","last_peered":"2024-09-21T11:59:09.722079+0000","last_clean":"2024-09-21T11:59:09.722079+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:59:09.722079+0000","last_undegraded":"2024-09-21T11:59:09.722079+0000","last_fullsized":"2024-09-21T11:59:09.722079+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:37.936279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.936279+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:39.639770+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":547,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723936+0000","last_change":"2024-09-21T11:56:08.429983+0000","last_active":"2024-09-21T11:59:09.723936+0000","last_peered":"2024-09-21T11:59:09.723936+0000","last_clean":"2024-09-21T11:59:09.723936+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:59:09.723936+0000","last_undegraded":"2024-09-21T11:59:09.723936+0000","last_fullsized":"2024-09-21T11:59:09.723936+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:08.429912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.429912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":642,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.710573+0000","last_change":"2024-09-21T11:58:21.827334+0000","last_active":"2024-09-21T11:59:09.710573+0000","last_peered":"2024-09-21T11:59:09.710573+0000","last_clean":"2024-09-21T11:59:09.710573+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:59:09.710573+0000","last_undegraded":"2024-09-21T11:59:09.710573+0000","last_fullsized":"2024-09-21T11:59:09.710573+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:21.827047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:21.827047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:43.889402+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":841,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.719958+0000","last_change":"2024-09-21T11:56:49.693769+0000","last_active":"2024-09-21T11:59:09.719958+0000","last_peered":"2024-09-21T11:59:09.719958+0000","last_clean":"2024-09-21T11:59:09.719958+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:59:09.719958+0000","last_undegraded":"2024-09-21T11:59:09.719958+0000","last_fullsized":"2024-09-21T11:59:09.719958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:49.693594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:49.693594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":831,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720366+0000","last_change":"2024-09-21T11:57:37.711838+0000","last_active":"2024-09-21T11:59:09.720366+0000","last_peered":"2024-09-21T11:59:09.720366+0000","last_clean":"2024-09-21T11:59:09.720366+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:59:09.720366+0000","last_undegraded":"2024-09-21T11:59:09.720366+0000","last_fullsized":"2024-09-21T11:59:09.720366+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:37.711758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.711758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":876,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.725113+0000","last_change":"2024-09-21T11:56:31.572626+0000","last_active":"2024-09-21T11:59:09.725113+0000","last_peered":"2024-09-21T11:59:09.725113+0000","last_clean":"2024-09-21T11:59:09.725113+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:59:09.725113+0000","last_undegraded":"2024-09-21T11:59:09.725113+0000","last_fullsized":"2024-09-21T11:59:09.725113+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:31.572525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:31.572525+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":692,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722204+0000","last_change":"2024-09-21T11:58:30.584828+0000","last_active":"2024-09-21T11:59:09.722204+0000","last_peered":"2024-09-21T11:59:09.722204+0000","last_clean":"2024-09-21T11:59:09.722204+0000","last_became_active":"2024-09-21T11:58:17.527501+0000","last_became_peered":"2024-09-21T11:58:17.527501+0000","last_unstale":"2024-09-21T11:59:09.722204+0000","last_undegraded":"2024-09-21T11:59:09.722204+0000","last_fullsized":"2024-09-21T11:59:09.722204+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:30.584774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:30.584774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:50.421881+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":820,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.712472+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:59:09.712472+0000","last_peered":"2024-09-21T11:59:09.712472+0000","last_clean":"2024-09-21T11:59:09.712472+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:59:09.712472+0000","last_undegraded":"2024-09-21T11:59:09.712472+0000","last_fullsized":"2024-09-21T11:59:09.712472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.819967+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":869,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.721041+0000","last_change":"2024-09-21T11:58:17.547443+0000","last_active":"2024-09-21T11:59:09.721041+0000","last_peered":"2024-09-21T11:59:09.721041+0000","last_clean":"2024-09-21T11:59:09.721041+0000","last_became_active":"2024-09-21T11:58:17.525792+0000","last_became_peered":"2024-09-21T11:58:17.525792+0000","last_unstale":"2024-09-21T11:59:09.721041+0000","last_undegraded":"2024-09-21T11:59:09.721041+0000","last_fullsized":"2024-09-21T11:59:09.721041+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:07.774892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:07.774892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":828,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.710514+0000","last_change":"2024-09-21T11:58:58.842083+0000","last_active":"2024-09-21T11:59:09.710514+0000","last_peered":"2024-09-21T11:59:09.710514+0000","last_clean":"2024-09-21T11:59:09.710514+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:59:09.710514+0000","last_undegraded":"2024-09-21T11:59:09.710514+0000","last_fullsized":"2024-09-21T11:59:09.710514+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:58.842006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:58.842006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:22.061210+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":66,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.714340+0000","last_change":"2024-09-21T11:57:58.527130+0000","last_active":"2024-09-21T11:59:09.714340+0000","last_peered":"2024-09-21T11:59:09.714340+0000","last_clean":"2024-09-21T11:59:09.714340+0000","last_became_active":"2024-09-21T11:57:58.526820+0000","last_became_peered":"2024-09-21T11:57:58.526820+0000","last_unstale":"2024-09-21T11:59:09.714340+0000","last_undegraded":"2024-09-21T11:59:09.714340+0000","last_fullsized":"2024-09-21T11:59:09.714340+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:36.422726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:36.422726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":866,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.710926+0000","last_change":"2024-09-21T11:56:02.994041+0000","last_active":"2024-09-21T11:59:09.710926+0000","last_peered":"2024-09-21T11:59:09.710926+0000","last_clean":"2024-09-21T11:59:09.710926+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:59:09.710926+0000","last_undegraded":"2024-09-21T11:59:09.710926+0000","last_fullsized":"2024-09-21T11:59:09.710926+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.993966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.993966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:35.823113+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":867,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720418+0000","last_change":"2024-09-21T11:58:17.549945+0000","last_active":"2024-09-21T11:59:09.720418+0000","last_peered":"2024-09-21T11:59:09.720418+0000","last_clean":"2024-09-21T11:59:09.720418+0000","last_became_active":"2024-09-21T11:58:17.525789+0000","last_became_peered":"2024-09-21T11:58:17.525789+0000","last_unstale":"2024-09-21T11:59:09.720418+0000","last_undegraded":"2024-09-21T11:59:09.720418+0000","last_fullsized":"2024-09-21T11:59:09.720418+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'7","last_scrub_stamp":"2024-09-21T11:56:59.659576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.659576+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":842,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.715078+0000","last_change":"2024-09-21T11:58:22.381598+0000","last_active":"2024-09-21T11:59:09.715078+0000","last_peered":"2024-09-21T11:59:09.715078+0000","last_clean":"2024-09-21T11:59:09.715078+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:59:09.715078+0000","last_undegraded":"2024-09-21T11:59:09.715078+0000","last_fullsized":"2024-09-21T11:59:09.715078+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:22.381531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:22.381531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.811109+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":838,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.711967+0000","last_change":"2024-09-21T11:58:24.861423+0000","last_active":"2024-09-21T11:59:09.711967+0000","last_peered":"2024-09-21T11:59:09.711967+0000","last_clean":"2024-09-21T11:59:09.711967+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:59:09.711967+0000","last_undegraded":"2024-09-21T11:59:09.711967+0000","last_fullsized":"2024-09-21T11:59:09.711967+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:24.861354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:24.861354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.549127+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":348,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:15.886098+0000","last_change":"2024-09-21T11:59:15.886098+0000","last_active":"2024-09-21T11:59:15.886098+0000","last_peered":"2024-09-21T11:59:15.886098+0000","last_clean":"2024-09-21T11:59:15.886098+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:59:15.886098+0000","last_undegraded":"2024-09-21T11:59:15.886098+0000","last_fullsized":"2024-09-21T11:59:15.886098+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:15.886007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:15.886007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:23.801683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":66,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.719549+0000","last_change":"2024-09-21T11:57:58.534353+0000","last_active":"2024-09-21T11:59:09.719549+0000","last_peered":"2024-09-21T11:59:09.719549+0000","last_clean":"2024-09-21T11:59:09.719549+0000","last_became_active":"2024-09-21T11:57:58.530327+0000","last_became_peered":"2024-09-21T11:57:58.530327+0000","last_unstale":"2024-09-21T11:59:09.719549+0000","last_undegraded":"2024-09-21T11:59:09.719549+0000","last_fullsized":"2024-09-21T11:59:09.719549+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:27.176320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:27.176320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":984,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.715053+0000","last_change":"2024-09-21T11:57:51.913043+0000","last_active":"2024-09-21T11:59:09.715053+0000","last_peered":"2024-09-21T11:59:09.715053+0000","last_clean":"2024-09-21T11:59:09.715053+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T11:59:09.715053+0000","last_undegraded":"2024-09-21T11:59:09.715053+0000","last_fullsized":"2024-09-21T11:59:09.715053+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:57:51.912800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.912800+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":552,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723442+0000","last_change":"2024-09-21T11:58:17.533215+0000","last_active":"2024-09-21T11:59:09.723442+0000","last_peered":"2024-09-21T11:59:09.723442+0000","last_clean":"2024-09-21T11:59:09.723442+0000","last_became_active":"2024-09-21T11:58:17.532890+0000","last_became_peered":"2024-09-21T11:58:17.532890+0000","last_unstale":"2024-09-21T11:59:09.723442+0000","last_undegraded":"2024-09-21T11:59:09.723442+0000","last_fullsized":"2024-09-21T11:59:09.723442+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:56.501356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:56.501356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":835,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720551+0000","last_change":"2024-09-21T11:57:04.647951+0000","last_active":"2024-09-21T11:59:09.720551+0000","last_peered":"2024-09-21T11:59:09.720551+0000","last_clean":"2024-09-21T11:59:09.720551+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:59:09.720551+0000","last_undegraded":"2024-09-21T11:59:09.720551+0000","last_fullsized":"2024-09-21T11:59:09.720551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:04.647749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:04.647749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":654,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.724067+0000","last_change":"2024-09-21T11:55:57.442188+0000","last_active":"2024-09-21T11:59:09.724067+0000","last_peered":"2024-09-21T11:59:09.724067+0000","last_clean":"2024-09-21T11:59:09.724067+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:59:09.724067+0000","last_undegraded":"2024-09-21T11:59:09.724067+0000","last_fullsized":"2024-09-21T11:59:09.724067+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:57.441965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.441965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":895,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720242+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:59:09.720242+0000","last_peered":"2024-09-21T11:59:09.720242+0000","last_clean":"2024-09-21T11:59:09.720242+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:59:09.720242+0000","last_undegraded":"2024-09-21T11:59:09.720242+0000","last_fullsized":"2024-09-21T11:59:09.720242+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":354,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.721069+0000","last_change":"2024-09-21T11:58:17.529121+0000","last_active":"2024-09-21T11:59:09.721069+0000","last_peered":"2024-09-21T11:59:09.721069+0000","last_clean":"2024-09-21T11:59:09.721069+0000","last_became_active":"2024-09-21T11:58:17.528785+0000","last_became_peered":"2024-09-21T11:58:17.528785+0000","last_unstale":"2024-09-21T11:59:09.721069+0000","last_undegraded":"2024-09-21T11:59:09.721069+0000","last_fullsized":"2024-09-21T11:59:09.721069+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.487132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.487132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":335,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.719499+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:59:09.719499+0000","last_peered":"2024-09-21T11:59:09.719499+0000","last_clean":"2024-09-21T11:59:09.719499+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:59:09.719499+0000","last_undegraded":"2024-09-21T11:59:09.719499+0000","last_fullsized":"2024-09-21T11:59:09.719499+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":573,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.724145+0000","last_change":"2024-09-21T11:58:17.554468+0000","last_active":"2024-09-21T11:59:09.724145+0000","last_peered":"2024-09-21T11:59:09.724145+0000","last_clean":"2024-09-21T11:59:09.724145+0000","last_became_active":"2024-09-21T11:58:17.529593+0000","last_became_peered":"2024-09-21T11:58:17.529593+0000","last_unstale":"2024-09-21T11:59:09.724145+0000","last_undegraded":"2024-09-21T11:59:09.724145+0000","last_fullsized":"2024-09-21T11:59:09.724145+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:10.530913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.530913+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":864,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.725274+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:59:09.725274+0000","last_peered":"2024-09-21T11:59:09.725274+0000","last_clean":"2024-09-21T11:59:09.725274+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:59:09.725274+0000","last_undegraded":"2024-09-21T11:59:09.725274+0000","last_fullsized":"2024-09-21T11:59:09.725274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"432'10","reported_seq":1121,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:10.603053+0000","last_change":"2024-09-21T11:58:18.736596+0000","last_active":"2024-09-21T11:59:10.603053+0000","last_peered":"2024-09-21T11:59:10.603053+0000","last_clean":"2024-09-21T11:59:10.603053+0000","last_became_active":"2024-09-21T11:58:17.529830+0000","last_became_peered":"2024-09-21T11:58:17.529830+0000","last_unstale":"2024-09-21T11:59:10.603053+0000","last_undegraded":"2024-09-21T11:59:10.603053+0000","last_fullsized":"2024-09-21T11:59:10.603053+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"383'9","last_scrub_stamp":"2024-09-21T11:58:18.736520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:18.736520+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:40.102209+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":82,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.710376+0000","last_change":"2024-09-21T11:57:49.896884+0000","last_active":"2024-09-21T11:59:09.710376+0000","last_peered":"2024-09-21T11:59:09.710376+0000","last_clean":"2024-09-21T11:59:09.710376+0000","last_became_active":"2024-09-21T11:57:45.504680+0000","last_became_peered":"2024-09-21T11:57:45.504680+0000","last_unstale":"2024-09-21T11:59:09.710376+0000","last_undegraded":"2024-09-21T11:59:09.710376+0000","last_fullsized":"2024-09-21T11:59:09.710376+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:49.896819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:49.896819+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"448'11605","reported_seq":11729,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.830381+0000","last_change":"2024-09-21T11:56:36.507880+0000","last_active":"2024-09-21T11:59:16.830381+0000","last_peered":"2024-09-21T11:59:16.830381+0000","last_clean":"2024-09-21T11:59:16.830381+0000","last_became_active":"2024-09-21T11:56:36.507606+0000","last_became_peered":"2024-09-21T11:56:36.507606+0000","last_unstale":"2024-09-21T11:59:16.830381+0000","last_undegraded":"2024-09-21T11:59:16.830381+0000","last_fullsized":"2024-09-21T11:59:16.830381+0000","mapping_epoch":392,"log_start":"419'6822","ondisk_log_start":"419'6822","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4783,"log_dups_size":0,"ondisk_log_size":4783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239403008,"num_objects":3653,"num_object_clones":0,"num_object_copies":10959,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11605,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":573,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.724540+0000","last_change":"2024-09-21T11:55:52.562992+0000","last_active":"2024-09-21T11:59:09.724540+0000","last_peered":"2024-09-21T11:59:09.724540+0000","last_clean":"2024-09-21T11:59:09.724540+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:59:09.724540+0000","last_undegraded":"2024-09-21T11:59:09.724540+0000","last_fullsized":"2024-09-21T11:59:09.724540+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:55:52.562928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.562928+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":860,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723707+0000","last_change":"2024-09-21T11:57:13.738771+0000","last_active":"2024-09-21T11:59:09.723707+0000","last_peered":"2024-09-21T11:59:09.723707+0000","last_clean":"2024-09-21T11:59:09.723707+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:59:09.723707+0000","last_undegraded":"2024-09-21T11:59:09.723707+0000","last_fullsized":"2024-09-21T11:59:09.723707+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:13.738727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:13.738727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"432'11","reported_seq":481,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:10.602977+0000","last_change":"2024-09-21T11:58:17.537763+0000","last_active":"2024-09-21T11:59:10.602977+0000","last_peered":"2024-09-21T11:59:10.602977+0000","last_clean":"2024-09-21T11:59:10.602977+0000","last_became_active":"2024-09-21T11:58:17.526752+0000","last_became_peered":"2024-09-21T11:58:17.526752+0000","last_unstale":"2024-09-21T11:59:10.602977+0000","last_undegraded":"2024-09-21T11:59:10.602977+0000","last_fullsized":"2024-09-21T11:59:10.602977+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"366'10","last_scrub_stamp":"2024-09-21T11:56:48.961017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.961017+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.739596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":885,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.711348+0000","last_change":"2024-09-21T11:58:16.866853+0000","last_active":"2024-09-21T11:59:09.711348+0000","last_peered":"2024-09-21T11:59:09.711348+0000","last_clean":"2024-09-21T11:59:09.711348+0000","last_became_active":"2024-09-21T11:57:58.526927+0000","last_became_peered":"2024-09-21T11:57:58.526927+0000","last_unstale":"2024-09-21T11:59:09.711348+0000","last_undegraded":"2024-09-21T11:59:09.711348+0000","last_fullsized":"2024-09-21T11:59:09.711348+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:16.866782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:16.866782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.387681+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":82,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.713442+0000","last_change":"2024-09-21T11:57:47.906127+0000","last_active":"2024-09-21T11:59:09.713442+0000","last_peered":"2024-09-21T11:59:09.713442+0000","last_clean":"2024-09-21T11:59:09.713442+0000","last_became_active":"2024-09-21T11:57:45.514811+0000","last_became_peered":"2024-09-21T11:57:45.514811+0000","last_unstale":"2024-09-21T11:59:09.713442+0000","last_undegraded":"2024-09-21T11:59:09.713442+0000","last_fullsized":"2024-09-21T11:59:09.713442+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:47.905784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:47.905784+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:15.591116+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"448'11781","reported_seq":11954,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.297144+0000","last_change":"2024-09-21T11:59:12.760513+0000","last_active":"2024-09-21T11:59:16.297144+0000","last_peered":"2024-09-21T11:59:16.297144+0000","last_clean":"2024-09-21T11:59:16.297144+0000","last_became_active":"2024-09-21T11:56:36.511191+0000","last_became_peered":"2024-09-21T11:56:36.511191+0000","last_unstale":"2024-09-21T11:59:16.297144+0000","last_undegraded":"2024-09-21T11:59:16.297144+0000","last_fullsized":"2024-09-21T11:59:16.297144+0000","mapping_epoch":392,"log_start":"419'7042","ondisk_log_start":"419'7042","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'11577","last_scrub_stamp":"2024-09-21T11:59:12.760365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:12.760365+0000","objects_scrubbed":4105,"log_size":4739,"log_dups_size":0,"ondisk_log_size":4739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:27.399210+0000","scrub_duration":1126,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253427712,"num_objects":3867,"num_object_clones":0,"num_object_copies":11601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11781,"num_write_kb":500736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":951,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723308+0000","last_change":"2024-09-21T11:57:58.547816+0000","last_active":"2024-09-21T11:59:09.723308+0000","last_peered":"2024-09-21T11:59:09.723308+0000","last_clean":"2024-09-21T11:59:09.723308+0000","last_became_active":"2024-09-21T11:57:58.537184+0000","last_became_peered":"2024-09-21T11:57:58.537184+0000","last_unstale":"2024-09-21T11:59:09.723308+0000","last_undegraded":"2024-09-21T11:59:09.723308+0000","last_fullsized":"2024-09-21T11:59:09.723308+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:57:00.631769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:00.631769+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":854,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720935+0000","last_change":"2024-09-21T11:56:53.701703+0000","last_active":"2024-09-21T11:59:09.720935+0000","last_peered":"2024-09-21T11:59:09.720935+0000","last_clean":"2024-09-21T11:59:09.720935+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:59:09.720935+0000","last_undegraded":"2024-09-21T11:59:09.720935+0000","last_fullsized":"2024-09-21T11:59:09.720935+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.701639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.701639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":340,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722376+0000","last_change":"2024-09-21T11:58:19.959956+0000","last_active":"2024-09-21T11:59:09.722376+0000","last_peered":"2024-09-21T11:59:09.722376+0000","last_clean":"2024-09-21T11:59:09.722376+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:59:09.722376+0000","last_undegraded":"2024-09-21T11:59:09.722376+0000","last_fullsized":"2024-09-21T11:59:09.722376+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:19.959912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:19.959912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.737932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":350,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.712454+0000","last_change":"2024-09-21T11:59:05.095763+0000","last_active":"2024-09-21T11:59:09.712454+0000","last_peered":"2024-09-21T11:59:09.712454+0000","last_clean":"2024-09-21T11:59:09.712454+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:59:09.712454+0000","last_undegraded":"2024-09-21T11:59:09.712454+0000","last_fullsized":"2024-09-21T11:59:09.712454+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:05.095722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:05.095722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:12.462739+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":87,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.719941+0000","last_change":"2024-09-21T11:57:47.804542+0000","last_active":"2024-09-21T11:59:09.719941+0000","last_peered":"2024-09-21T11:59:09.719941+0000","last_clean":"2024-09-21T11:59:09.719941+0000","last_became_active":"2024-09-21T11:57:45.514189+0000","last_became_peered":"2024-09-21T11:57:45.514189+0000","last_unstale":"2024-09-21T11:59:09.719941+0000","last_undegraded":"2024-09-21T11:59:09.719941+0000","last_fullsized":"2024-09-21T11:59:09.719941+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:47.804466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:47.804466+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"448'11331","reported_seq":11471,"reported_epoch":448,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-21T11:59:16.851644+0000","last_change":"2024-09-21T11:58:48.686396+0000","last_active":"2024-09-21T11:59:16.851644+0000","last_peered":"2024-09-21T11:59:16.851644+0000","last_clean":"2024-09-21T11:58:17.235375+0000","last_became_active":"2024-09-21T11:58:18.605530+0000","last_became_peered":"2024-09-21T11:58:18.605530+0000","last_unstale":"2024-09-21T11:59:16.851644+0000","last_undegraded":"2024-09-21T11:59:16.851644+0000","last_fullsized":"2024-09-21T11:59:16.851644+0000","mapping_epoch":431,"log_start":"418'6567","ondisk_log_start":"418'6567","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4764,"log_dups_size":0,"ondisk_log_size":4764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245825536,"num_objects":3751,"num_object_clones":0,"num_object_copies":11253,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3751,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11331,"num_write_kb":482624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,7,0],"avail_no_missing":["1","0","7"],"object_location_counts":[{"shards":"0,1,7","objects":3751}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":890,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.712056+0000","last_change":"2024-09-21T11:58:17.828372+0000","last_active":"2024-09-21T11:59:09.712056+0000","last_peered":"2024-09-21T11:59:09.712056+0000","last_clean":"2024-09-21T11:59:09.712056+0000","last_became_active":"2024-09-21T11:58:17.525491+0000","last_became_peered":"2024-09-21T11:58:17.525491+0000","last_unstale":"2024-09-21T11:59:09.712056+0000","last_undegraded":"2024-09-21T11:59:09.712056+0000","last_fullsized":"2024-09-21T11:59:09.712056+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:58:17.828305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:17.828305+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.685496+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":70,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.725370+0000","last_change":"2024-09-21T11:57:58.539709+0000","last_active":"2024-09-21T11:59:09.725370+0000","last_peered":"2024-09-21T11:59:09.725370+0000","last_clean":"2024-09-21T11:59:09.725370+0000","last_became_active":"2024-09-21T11:57:58.526938+0000","last_became_peered":"2024-09-21T11:57:58.526938+0000","last_unstale":"2024-09-21T11:59:09.725370+0000","last_undegraded":"2024-09-21T11:59:09.725370+0000","last_fullsized":"2024-09-21T11:59:09.725370+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:48.446635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.446635+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":1094,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:10.603957+0000","last_change":"2024-09-21T11:56:05.567407+0000","last_active":"2024-09-21T11:59:10.603957+0000","last_peered":"2024-09-21T11:59:10.603957+0000","last_clean":"2024-09-21T11:59:10.603957+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:59:10.603957+0000","last_undegraded":"2024-09-21T11:59:10.603957+0000","last_fullsized":"2024-09-21T11:59:10.603957+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:05.567332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.567332+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":844,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.715388+0000","last_change":"2024-09-21T11:56:34.581966+0000","last_active":"2024-09-21T11:59:09.715388+0000","last_peered":"2024-09-21T11:59:09.715388+0000","last_clean":"2024-09-21T11:59:09.715388+0000","last_became_active":"2024-09-21T11:56:34.581626+0000","last_became_peered":"2024-09-21T11:56:34.581626+0000","last_unstale":"2024-09-21T11:59:09.715388+0000","last_undegraded":"2024-09-21T11:59:09.715388+0000","last_fullsized":"2024-09-21T11:59:09.715388+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.643323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.643323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:35.106418+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":168,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723290+0000","last_change":"2024-09-21T11:57:45.998708+0000","last_active":"2024-09-21T11:59:09.723290+0000","last_peered":"2024-09-21T11:59:09.723290+0000","last_clean":"2024-09-21T11:59:09.723290+0000","last_became_active":"2024-09-21T11:57:45.514310+0000","last_became_peered":"2024-09-21T11:57:45.514310+0000","last_unstale":"2024-09-21T11:59:09.723290+0000","last_undegraded":"2024-09-21T11:59:09.723290+0000","last_fullsized":"2024-09-21T11:59:09.723290+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:45.998644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:45.998644+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.495485+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"448'11417","reported_seq":11596,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.951420+0000","last_change":"2024-09-21T11:59:01.985776+0000","last_active":"2024-09-21T11:59:16.951420+0000","last_peered":"2024-09-21T11:59:16.951420+0000","last_clean":"2024-09-21T11:59:16.951420+0000","last_became_active":"2024-09-21T11:56:36.623581+0000","last_became_peered":"2024-09-21T11:56:36.623581+0000","last_unstale":"2024-09-21T11:59:16.951420+0000","last_undegraded":"2024-09-21T11:59:16.951420+0000","last_fullsized":"2024-09-21T11:59:16.951420+0000","mapping_epoch":392,"log_start":"408'5216","ondisk_log_start":"408'5216","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"444'10565","last_scrub_stamp":"2024-09-21T11:59:01.985630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:01.985630+0000","objects_scrubbed":4571,"log_size":6201,"log_dups_size":0,"ondisk_log_size":6201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:14.582998+0000","scrub_duration":1821,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241303576,"num_objects":3683,"num_object_clones":0,"num_object_copies":11049,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3683,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11417,"num_write_kb":483137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":604,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723819+0000","last_change":"2024-09-21T11:58:17.557543+0000","last_active":"2024-09-21T11:59:09.723819+0000","last_peered":"2024-09-21T11:59:09.723819+0000","last_clean":"2024-09-21T11:59:09.723819+0000","last_became_active":"2024-09-21T11:58:17.531001+0000","last_became_peered":"2024-09-21T11:58:17.531001+0000","last_unstale":"2024-09-21T11:59:09.723819+0000","last_undegraded":"2024-09-21T11:59:09.723819+0000","last_fullsized":"2024-09-21T11:59:09.723819+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:56:09.421193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.421193+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":870,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.712552+0000","last_change":"2024-09-21T11:56:51.036609+0000","last_active":"2024-09-21T11:59:09.712552+0000","last_peered":"2024-09-21T11:59:09.712552+0000","last_clean":"2024-09-21T11:59:09.712552+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:59:09.712552+0000","last_undegraded":"2024-09-21T11:59:09.712552+0000","last_fullsized":"2024-09-21T11:59:09.712552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:51.036535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:51.036535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:40.871456+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1080,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.725925+0000","last_change":"2024-09-21T11:57:40.102377+0000","last_active":"2024-09-21T11:59:09.725925+0000","last_peered":"2024-09-21T11:59:09.725925+0000","last_clean":"2024-09-21T11:59:09.725925+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:59:09.725925+0000","last_undegraded":"2024-09-21T11:59:09.725925+0000","last_fullsized":"2024-09-21T11:59:09.725925+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:40.102294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:40.102294+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":842,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.714615+0000","last_change":"2024-09-21T11:57:36.109392+0000","last_active":"2024-09-21T11:59:09.714615+0000","last_peered":"2024-09-21T11:59:09.714615+0000","last_clean":"2024-09-21T11:59:09.714615+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:59:09.714615+0000","last_undegraded":"2024-09-21T11:59:09.714615+0000","last_fullsized":"2024-09-21T11:59:09.714615+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:36.109317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:36.109317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":176,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722999+0000","last_change":"2024-09-21T11:59:07.885015+0000","last_active":"2024-09-21T11:59:09.722999+0000","last_peered":"2024-09-21T11:59:09.722999+0000","last_clean":"2024-09-21T11:59:09.722999+0000","last_became_active":"2024-09-21T11:57:45.518764+0000","last_became_peered":"2024-09-21T11:57:45.518764+0000","last_unstale":"2024-09-21T11:59:09.722999+0000","last_undegraded":"2024-09-21T11:59:09.722999+0000","last_fullsized":"2024-09-21T11:59:09.722999+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:59:07.884939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:59:07.884939+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:13.711897+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.c","version":"448'11509","reported_seq":20436,"reported_epoch":448,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:59:16.859007+0000","last_change":"2024-09-21T11:58:48.624484+0000","last_active":"2024-09-21T11:59:16.859007+0000","last_peered":"2024-09-21T11:59:16.859007+0000","last_clean":"2024-09-21T11:58:17.235882+0000","last_became_active":"2024-09-21T11:58:18.606350+0000","last_became_peered":"2024-09-21T11:58:18.606350+0000","last_unstale":"2024-09-21T11:59:16.859007+0000","last_undegraded":"2024-09-21T11:59:16.859007+0000","last_fullsized":"2024-09-21T11:59:16.859007+0000","mapping_epoch":431,"log_start":"418'6679","ondisk_log_start":"418'6679","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4830,"log_dups_size":0,"ondisk_log_size":4830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240451584,"num_objects":3669,"num_object_clones":0,"num_object_copies":11007,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":30,"num_objects_unfound":0,"num_objects_dirty":3669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11509,"num_write_kb":485696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4397,"num_bytes_recovered":288161792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,7],"avail_no_missing":["1","2","7"],"object_location_counts":[{"shards":"1,2,7","objects":3669}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":909,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.724921+0000","last_change":"2024-09-21T11:57:50.976268+0000","last_active":"2024-09-21T11:59:09.724921+0000","last_peered":"2024-09-21T11:59:09.724921+0000","last_clean":"2024-09-21T11:59:09.724921+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:59:09.724921+0000","last_undegraded":"2024-09-21T11:59:09.724921+0000","last_fullsized":"2024-09-21T11:59:09.724921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:57:50.976029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:50.976029+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":539,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.726954+0000","last_change":"2024-09-21T11:56:58.434536+0000","last_active":"2024-09-21T11:59:09.726954+0000","last_peered":"2024-09-21T11:59:09.726954+0000","last_clean":"2024-09-21T11:59:09.726954+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:59:09.726954+0000","last_undegraded":"2024-09-21T11:59:09.726954+0000","last_fullsized":"2024-09-21T11:59:09.726954+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.434464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.434464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":822,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723261+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:59:09.723261+0000","last_peered":"2024-09-21T11:59:09.723261+0000","last_clean":"2024-09-21T11:59:09.723261+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:59:09.723261+0000","last_undegraded":"2024-09-21T11:59:09.723261+0000","last_fullsized":"2024-09-21T11:59:09.723261+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:34.633118+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":531,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723317+0000","last_change":"2024-09-21T11:55:51.602665+0000","last_active":"2024-09-21T11:59:09.723317+0000","last_peered":"2024-09-21T11:59:09.723317+0000","last_clean":"2024-09-21T11:59:09.723317+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:59:09.723317+0000","last_undegraded":"2024-09-21T11:59:09.723317+0000","last_fullsized":"2024-09-21T11:59:09.723317+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.602587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.602587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":82,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.711762+0000","last_change":"2024-09-21T11:57:53.922876+0000","last_active":"2024-09-21T11:59:09.711762+0000","last_peered":"2024-09-21T11:59:09.711762+0000","last_clean":"2024-09-21T11:59:09.711762+0000","last_became_active":"2024-09-21T11:57:45.515242+0000","last_became_peered":"2024-09-21T11:57:45.515242+0000","last_unstale":"2024-09-21T11:59:09.711762+0000","last_undegraded":"2024-09-21T11:59:09.711762+0000","last_fullsized":"2024-09-21T11:59:09.711762+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:53.922795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:53.922795+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:20.497165+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.b","version":"448'11394","reported_seq":11518,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.118472+0000","last_change":"2024-09-21T11:56:36.510297+0000","last_active":"2024-09-21T11:59:16.118472+0000","last_peered":"2024-09-21T11:59:16.118472+0000","last_clean":"2024-09-21T11:59:16.118472+0000","last_became_active":"2024-09-21T11:56:36.510018+0000","last_became_peered":"2024-09-21T11:56:36.510018+0000","last_unstale":"2024-09-21T11:59:16.118472+0000","last_undegraded":"2024-09-21T11:59:16.118472+0000","last_fullsized":"2024-09-21T11:59:16.118472+0000","mapping_epoch":392,"log_start":"418'6721","ondisk_log_start":"418'6721","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4673,"log_dups_size":0,"ondisk_log_size":4673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243793920,"num_objects":3720,"num_object_clones":0,"num_object_copies":11160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11394,"num_write_kb":483648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":702,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:10.608723+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:59:10.608723+0000","last_peered":"2024-09-21T11:59:10.608723+0000","last_clean":"2024-09-21T11:59:10.608723+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:59:10.608723+0000","last_undegraded":"2024-09-21T11:59:10.608723+0000","last_fullsized":"2024-09-21T11:59:10.608723+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":365,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.725083+0000","last_change":"2024-09-21T11:57:51.934799+0000","last_active":"2024-09-21T11:59:09.725083+0000","last_peered":"2024-09-21T11:59:09.725083+0000","last_clean":"2024-09-21T11:59:09.725083+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:59:09.725083+0000","last_undegraded":"2024-09-21T11:59:09.725083+0000","last_fullsized":"2024-09-21T11:59:09.725083+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:51.934563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.934563+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:17.585958+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":870,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.710949+0000","last_change":"2024-09-21T11:58:59.855226+0000","last_active":"2024-09-21T11:59:09.710949+0000","last_peered":"2024-09-21T11:59:09.710949+0000","last_clean":"2024-09-21T11:59:09.710949+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:59:09.710949+0000","last_undegraded":"2024-09-21T11:59:09.710949+0000","last_fullsized":"2024-09-21T11:59:09.710949+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:58:59.855148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:58:59.855148+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:09.486850+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":73,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.721027+0000","last_change":"2024-09-21T11:58:17.527987+0000","last_active":"2024-09-21T11:59:09.721027+0000","last_peered":"2024-09-21T11:59:09.721027+0000","last_clean":"2024-09-21T11:59:09.721027+0000","last_became_active":"2024-09-21T11:58:17.527653+0000","last_became_peered":"2024-09-21T11:58:17.527653+0000","last_unstale":"2024-09-21T11:59:09.721027+0000","last_undegraded":"2024-09-21T11:59:09.721027+0000","last_fullsized":"2024-09-21T11:59:09.721027+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":818,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720089+0000","last_change":"2024-09-21T11:57:54.859726+0000","last_active":"2024-09-21T11:59:09.720089+0000","last_peered":"2024-09-21T11:59:09.720089+0000","last_clean":"2024-09-21T11:59:09.720089+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:59:09.720089+0000","last_undegraded":"2024-09-21T11:59:09.720089+0000","last_fullsized":"2024-09-21T11:59:09.720089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:54.859668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.859668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.673552+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":962,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.714909+0000","last_change":"2024-09-21T11:58:17.529673+0000","last_active":"2024-09-21T11:59:09.714909+0000","last_peered":"2024-09-21T11:59:09.714909+0000","last_clean":"2024-09-21T11:59:09.714909+0000","last_became_active":"2024-09-21T11:58:17.529304+0000","last_became_peered":"2024-09-21T11:58:17.529304+0000","last_unstale":"2024-09-21T11:59:09.714909+0000","last_undegraded":"2024-09-21T11:59:09.714909+0000","last_fullsized":"2024-09-21T11:59:09.714909+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:50.548003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:50.548003+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"448'11472","reported_seq":11596,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.153849+0000","last_change":"2024-09-21T11:56:36.509094+0000","last_active":"2024-09-21T11:59:16.153849+0000","last_peered":"2024-09-21T11:59:16.153849+0000","last_clean":"2024-09-21T11:59:16.153849+0000","last_became_active":"2024-09-21T11:56:36.508840+0000","last_became_peered":"2024-09-21T11:56:36.508840+0000","last_unstale":"2024-09-21T11:59:16.153849+0000","last_undegraded":"2024-09-21T11:59:16.153849+0000","last_fullsized":"2024-09-21T11:59:16.153849+0000","mapping_epoch":392,"log_start":"419'6795","ondisk_log_start":"419'6795","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4677,"log_dups_size":0,"ondisk_log_size":4677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246022144,"num_objects":3754,"num_object_clones":0,"num_object_copies":11262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11472,"num_write_kb":487232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":528,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722366+0000","last_change":"2024-09-21T11:59:04.130363+0000","last_active":"2024-09-21T11:59:09.722366+0000","last_peered":"2024-09-21T11:59:09.722366+0000","last_clean":"2024-09-21T11:59:09.722366+0000","last_became_active":"2024-09-21T11:57:58.530507+0000","last_became_peered":"2024-09-21T11:57:58.530507+0000","last_unstale":"2024-09-21T11:59:09.722366+0000","last_undegraded":"2024-09-21T11:59:09.722366+0000","last_fullsized":"2024-09-21T11:59:09.722366+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:59:04.130113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:04.130113+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:17.734192+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":544,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722320+0000","last_change":"2024-09-21T11:57:43.583962+0000","last_active":"2024-09-21T11:59:09.722320+0000","last_peered":"2024-09-21T11:59:09.722320+0000","last_clean":"2024-09-21T11:59:09.722320+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:59:09.722320+0000","last_undegraded":"2024-09-21T11:59:09.722320+0000","last_fullsized":"2024-09-21T11:59:09.722320+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:43.583736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.583736+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:11.292974+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":864,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.713174+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:59:09.713174+0000","last_peered":"2024-09-21T11:59:09.713174+0000","last_clean":"2024-09-21T11:59:09.713174+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:59:09.713174+0000","last_undegraded":"2024-09-21T11:59:09.713174+0000","last_fullsized":"2024-09-21T11:59:09.713174+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.044139+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":846,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.717838+0000","last_change":"2024-09-21T11:59:09.091725+0000","last_active":"2024-09-21T11:59:09.717838+0000","last_peered":"2024-09-21T11:59:09.717838+0000","last_clean":"2024-09-21T11:59:09.717838+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:59:09.717838+0000","last_undegraded":"2024-09-21T11:59:09.717838+0000","last_fullsized":"2024-09-21T11:59:09.717838+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:09.091520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:59:09.091520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:10.832593+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":850,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.715595+0000","last_change":"2024-09-21T11:58:30.226389+0000","last_active":"2024-09-21T11:59:09.715595+0000","last_peered":"2024-09-21T11:59:09.715595+0000","last_clean":"2024-09-21T11:59:09.715595+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:59:09.715595+0000","last_undegraded":"2024-09-21T11:59:09.715595+0000","last_fullsized":"2024-09-21T11:59:09.715595+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:30.226345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:30.226345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:50.129480+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":834,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.724226+0000","last_change":"2024-09-21T11:58:30.794254+0000","last_active":"2024-09-21T11:59:09.724226+0000","last_peered":"2024-09-21T11:59:09.724226+0000","last_clean":"2024-09-21T11:59:09.724226+0000","last_became_active":"2024-09-21T11:58:30.793717+0000","last_became_peered":"2024-09-21T11:58:30.793717+0000","last_unstale":"2024-09-21T11:59:09.724226+0000","last_undegraded":"2024-09-21T11:59:09.724226+0000","last_fullsized":"2024-09-21T11:59:09.724226+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":436,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:48.627711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.627711+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:14.347769+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"448'11520","reported_seq":20715,"reported_epoch":448,"state":"active+remapped+backfilling","last_fresh":"2024-09-21T11:59:16.149768+0000","last_change":"2024-09-21T11:58:30.022462+0000","last_active":"2024-09-21T11:59:16.149768+0000","last_peered":"2024-09-21T11:59:16.149768+0000","last_clean":"2024-09-21T11:57:57.490959+0000","last_became_active":"2024-09-21T11:57:59.990813+0000","last_became_peered":"2024-09-21T11:57:59.990813+0000","last_unstale":"2024-09-21T11:59:16.149768+0000","last_undegraded":"2024-09-21T11:59:16.149768+0000","last_fullsized":"2024-09-21T11:59:16.149768+0000","mapping_epoch":421,"log_start":"419'6845","ondisk_log_start":"419'6845","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4675,"log_dups_size":0,"ondisk_log_size":4675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":250740736,"num_objects":3826,"num_object_clones":0,"num_object_copies":11478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":431,"num_objects_unfound":0,"num_objects_dirty":3826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11520,"num_write_kb":491072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4532,"num_bytes_recovered":297009152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,1],"avail_no_missing":["3","1","6"],"object_location_counts":[{"shards":"1,3,6","objects":3826}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":80,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723281+0000","last_change":"2024-09-21T11:57:58.545640+0000","last_active":"2024-09-21T11:59:09.723281+0000","last_peered":"2024-09-21T11:59:09.723281+0000","last_clean":"2024-09-21T11:59:09.723281+0000","last_became_active":"2024-09-21T11:57:58.527381+0000","last_became_peered":"2024-09-21T11:57:58.527381+0000","last_unstale":"2024-09-21T11:59:09.723281+0000","last_undegraded":"2024-09-21T11:59:09.723281+0000","last_fullsized":"2024-09-21T11:59:09.723281+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:57:54.679272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.679272+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:09.756620+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":876,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.714532+0000","last_change":"2024-09-21T11:58:17.528448+0000","last_active":"2024-09-21T11:59:09.714532+0000","last_peered":"2024-09-21T11:59:09.714532+0000","last_clean":"2024-09-21T11:59:09.714532+0000","last_became_active":"2024-09-21T11:58:17.528154+0000","last_became_peered":"2024-09-21T11:58:17.528154+0000","last_unstale":"2024-09-21T11:59:09.714532+0000","last_undegraded":"2024-09-21T11:59:09.714532+0000","last_fullsized":"2024-09-21T11:59:09.714532+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:01.083437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.083437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":2035,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:14.719284+0000","last_change":"2024-09-21T11:56:09.604887+0000","last_active":"2024-09-21T11:59:14.719284+0000","last_peered":"2024-09-21T11:59:14.719284+0000","last_clean":"2024-09-21T11:59:14.719284+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:59:14.719284+0000","last_undegraded":"2024-09-21T11:59:14.719284+0000","last_fullsized":"2024-09-21T11:59:14.719284+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:56:09.604808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.604808+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.102017+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1146,"num_read_kb":1141,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":859,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:11.972760+0000","last_change":"2024-09-21T11:59:11.972760+0000","last_active":"2024-09-21T11:59:11.972760+0000","last_peered":"2024-09-21T11:59:11.972760+0000","last_clean":"2024-09-21T11:59:11.972760+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:59:11.972760+0000","last_undegraded":"2024-09-21T11:59:11.972760+0000","last_fullsized":"2024-09-21T11:59:11.972760+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:11.972680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:59:11.972680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:33.037868+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":531,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.726822+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:59:09.726822+0000","last_peered":"2024-09-21T11:59:09.726822+0000","last_clean":"2024-09-21T11:59:09.726822+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:59:09.726822+0000","last_undegraded":"2024-09-21T11:59:09.726822+0000","last_fullsized":"2024-09-21T11:59:09.726822+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"448'11428","reported_seq":11552,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.135362+0000","last_change":"2024-09-21T11:56:36.509270+0000","last_active":"2024-09-21T11:59:16.135362+0000","last_peered":"2024-09-21T11:59:16.135362+0000","last_clean":"2024-09-21T11:59:16.135362+0000","last_became_active":"2024-09-21T11:56:36.508533+0000","last_became_peered":"2024-09-21T11:56:36.508533+0000","last_unstale":"2024-09-21T11:59:16.135362+0000","last_undegraded":"2024-09-21T11:59:16.135362+0000","last_fullsized":"2024-09-21T11:59:16.135362+0000","mapping_epoch":392,"log_start":"419'6722","ondisk_log_start":"419'6722","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4706,"log_dups_size":0,"ondisk_log_size":4706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239861760,"num_objects":3660,"num_object_clones":0,"num_object_copies":10980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11428,"num_write_kb":482816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":1122,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:10.603041+0000","last_change":"2024-09-21T11:59:05.905840+0000","last_active":"2024-09-21T11:59:10.603041+0000","last_peered":"2024-09-21T11:59:10.603041+0000","last_clean":"2024-09-21T11:59:10.603041+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:59:10.603041+0000","last_undegraded":"2024-09-21T11:59:10.603041+0000","last_fullsized":"2024-09-21T11:59:10.603041+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:59:05.905765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:05.905765+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:34.307948+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":847,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722525+0000","last_change":"2024-09-21T11:57:14.756298+0000","last_active":"2024-09-21T11:59:09.722525+0000","last_peered":"2024-09-21T11:59:09.722525+0000","last_clean":"2024-09-21T11:59:09.722525+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:59:09.722525+0000","last_undegraded":"2024-09-21T11:59:09.722525+0000","last_fullsized":"2024-09-21T11:59:09.722525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:14.756227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:14.756227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":965,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.716536+0000","last_change":"2024-09-21T11:58:54.745261+0000","last_active":"2024-09-21T11:59:09.716536+0000","last_peered":"2024-09-21T11:59:09.716536+0000","last_clean":"2024-09-21T11:59:09.716536+0000","last_became_active":"2024-09-21T11:57:58.528825+0000","last_became_peered":"2024-09-21T11:57:58.528825+0000","last_unstale":"2024-09-21T11:59:09.716536+0000","last_undegraded":"2024-09-21T11:59:09.716536+0000","last_fullsized":"2024-09-21T11:59:09.716536+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:58:54.745202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:58:54.745202+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:08.610072+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":838,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.719786+0000","last_change":"2024-09-21T11:58:28.274182+0000","last_active":"2024-09-21T11:59:09.719786+0000","last_peered":"2024-09-21T11:59:09.719786+0000","last_clean":"2024-09-21T11:59:09.719786+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:59:09.719786+0000","last_undegraded":"2024-09-21T11:59:09.719786+0000","last_fullsized":"2024-09-21T11:59:09.719786+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:28.274120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.274120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.924347+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":849,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.712335+0000","last_change":"2024-09-21T11:57:05.904684+0000","last_active":"2024-09-21T11:59:09.712335+0000","last_peered":"2024-09-21T11:59:09.712335+0000","last_clean":"2024-09-21T11:59:09.712335+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:59:09.712335+0000","last_undegraded":"2024-09-21T11:59:09.712335+0000","last_fullsized":"2024-09-21T11:59:09.712335+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:05.904621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:05.904621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:46.758787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"448'11459","reported_seq":11614,"reported_epoch":448,"state":"active+clean+scrubbing","last_fresh":"2024-09-21T11:59:15.919864+0000","last_change":"2024-09-21T11:59:14.860034+0000","last_active":"2024-09-21T11:59:15.919864+0000","last_peered":"2024-09-21T11:59:15.919864+0000","last_clean":"2024-09-21T11:59:15.919864+0000","last_became_active":"2024-09-21T11:56:36.507849+0000","last_became_peered":"2024-09-21T11:56:36.507849+0000","last_unstale":"2024-09-21T11:59:15.919864+0000","last_undegraded":"2024-09-21T11:59:15.919864+0000","last_fullsized":"2024-09-21T11:59:15.919864+0000","mapping_epoch":392,"log_start":"416'6004","ondisk_log_start":"416'6004","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":2724,"log_size":5455,"log_dups_size":0,"ondisk_log_size":5455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240320512,"num_objects":3667,"num_object_clones":0,"num_object_copies":11001,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11459,"num_write_kb":484032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":871,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723104+0000","last_change":"2024-09-21T11:58:56.825792+0000","last_active":"2024-09-21T11:59:09.723104+0000","last_peered":"2024-09-21T11:59:09.723104+0000","last_clean":"2024-09-21T11:59:09.723104+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:59:09.723104+0000","last_undegraded":"2024-09-21T11:59:09.723104+0000","last_fullsized":"2024-09-21T11:59:09.723104+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:56.825707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:56.825707+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:07.271040+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":354,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.713038+0000","last_change":"2024-09-21T11:58:17.529749+0000","last_active":"2024-09-21T11:59:09.713038+0000","last_peered":"2024-09-21T11:59:09.713038+0000","last_clean":"2024-09-21T11:59:09.713038+0000","last_became_active":"2024-09-21T11:58:17.529395+0000","last_became_peered":"2024-09-21T11:58:17.529395+0000","last_unstale":"2024-09-21T11:59:09.713038+0000","last_undegraded":"2024-09-21T11:59:09.713038+0000","last_fullsized":"2024-09-21T11:59:09.713038+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:05.663102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.663102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:47.080734+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":851,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.712224+0000","last_change":"2024-09-21T11:58:29.807560+0000","last_active":"2024-09-21T11:59:09.712224+0000","last_peered":"2024-09-21T11:59:09.712224+0000","last_clean":"2024-09-21T11:59:09.712224+0000","last_became_active":"2024-09-21T11:58:17.527945+0000","last_became_peered":"2024-09-21T11:58:17.527945+0000","last_unstale":"2024-09-21T11:59:09.712224+0000","last_undegraded":"2024-09-21T11:59:09.712224+0000","last_fullsized":"2024-09-21T11:59:09.712224+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:29.807504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:29.807504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:58.143758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":840,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720051+0000","last_change":"2024-09-21T11:56:28.556475+0000","last_active":"2024-09-21T11:59:09.720051+0000","last_peered":"2024-09-21T11:59:09.720051+0000","last_clean":"2024-09-21T11:59:09.720051+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:59:09.720051+0000","last_undegraded":"2024-09-21T11:59:09.720051+0000","last_fullsized":"2024-09-21T11:59:09.720051+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.556423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.556423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":89,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.710677+0000","last_change":"2024-09-21T11:57:54.931806+0000","last_active":"2024-09-21T11:59:09.710677+0000","last_peered":"2024-09-21T11:59:09.710677+0000","last_clean":"2024-09-21T11:59:09.710677+0000","last_became_active":"2024-09-21T11:57:45.534494+0000","last_became_peered":"2024-09-21T11:57:45.534494+0000","last_unstale":"2024-09-21T11:59:09.710677+0000","last_undegraded":"2024-09-21T11:59:09.710677+0000","last_fullsized":"2024-09-21T11:59:09.710677+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:54.931704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.931704+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:23.002342+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2","version":"448'11418","reported_seq":11595,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.949565+0000","last_change":"2024-09-21T11:59:01.566946+0000","last_active":"2024-09-21T11:59:16.949565+0000","last_peered":"2024-09-21T11:59:16.949565+0000","last_clean":"2024-09-21T11:59:16.949565+0000","last_became_active":"2024-09-21T11:56:36.511642+0000","last_became_peered":"2024-09-21T11:56:36.511642+0000","last_unstale":"2024-09-21T11:59:16.949565+0000","last_undegraded":"2024-09-21T11:59:16.949565+0000","last_fullsized":"2024-09-21T11:59:16.949565+0000","mapping_epoch":392,"log_start":"407'5197","ondisk_log_start":"407'5197","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'10535","last_scrub_stamp":"2024-09-21T11:59:01.566731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:01.566731+0000","objects_scrubbed":4558,"log_size":6221,"log_dups_size":0,"ondisk_log_size":6221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:17.985934+0000","scrub_duration":2440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236716032,"num_objects":3612,"num_object_clones":0,"num_object_copies":10836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11418,"num_write_kb":480960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":968,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.718253+0000","last_change":"2024-09-21T11:58:17.555274+0000","last_active":"2024-09-21T11:59:09.718253+0000","last_peered":"2024-09-21T11:59:09.718253+0000","last_clean":"2024-09-21T11:59:09.718253+0000","last_became_active":"2024-09-21T11:58:17.527029+0000","last_became_peered":"2024-09-21T11:58:17.527029+0000","last_unstale":"2024-09-21T11:59:09.718253+0000","last_undegraded":"2024-09-21T11:59:09.718253+0000","last_fullsized":"2024-09-21T11:59:09.718253+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:56:03.667750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:03.667750+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:37.084768+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":875,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.725606+0000","last_change":"2024-09-21T11:59:02.898115+0000","last_active":"2024-09-21T11:59:09.725606+0000","last_peered":"2024-09-21T11:59:09.725606+0000","last_clean":"2024-09-21T11:59:09.725606+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:59:09.725606+0000","last_undegraded":"2024-09-21T11:59:09.725606+0000","last_fullsized":"2024-09-21T11:59:09.725606+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:02.898000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:59:02.898000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:26.416721+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":539,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.726714+0000","last_change":"2024-09-21T11:57:42.599025+0000","last_active":"2024-09-21T11:59:09.726714+0000","last_peered":"2024-09-21T11:59:09.726714+0000","last_clean":"2024-09-21T11:59:09.726714+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:59:09.726714+0000","last_undegraded":"2024-09-21T11:59:09.726714+0000","last_fullsized":"2024-09-21T11:59:09.726714+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:42.598710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:42.598710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":850,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.184608+0000","last_change":"2024-09-21T11:59:16.184608+0000","last_active":"2024-09-21T11:59:16.184608+0000","last_peered":"2024-09-21T11:59:16.184608+0000","last_clean":"2024-09-21T11:59:16.184608+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:59:16.184608+0000","last_undegraded":"2024-09-21T11:59:16.184608+0000","last_fullsized":"2024-09-21T11:59:16.184608+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:16.184540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:16.184540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:21.327851+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":963,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:10.608778+0000","last_change":"2024-09-21T11:59:10.608778+0000","last_active":"2024-09-21T11:59:10.608778+0000","last_peered":"2024-09-21T11:59:10.608778+0000","last_clean":"2024-09-21T11:59:10.608778+0000","last_became_active":"2024-09-21T11:57:45.516610+0000","last_became_peered":"2024-09-21T11:57:45.516610+0000","last_unstale":"2024-09-21T11:59:10.608778+0000","last_undegraded":"2024-09-21T11:59:10.608778+0000","last_fullsized":"2024-09-21T11:59:10.608778+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:59:10.608731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:59:10.608731+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:38.999852+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3","version":"448'11383","reported_seq":11587,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:15.898821+0000","last_change":"2024-09-21T11:58:22.114392+0000","last_active":"2024-09-21T11:59:15.898821+0000","last_peered":"2024-09-21T11:59:15.898821+0000","last_clean":"2024-09-21T11:59:15.898821+0000","last_became_active":"2024-09-21T11:56:36.504516+0000","last_became_peered":"2024-09-21T11:56:36.504516+0000","last_unstale":"2024-09-21T11:59:15.898821+0000","last_undegraded":"2024-09-21T11:59:15.898821+0000","last_fullsized":"2024-09-21T11:59:15.898821+0000","mapping_epoch":392,"log_start":"417'5995","ondisk_log_start":"417'5995","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"432'8216","last_scrub_stamp":"2024-09-21T11:58:22.114283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:58:22.114283+0000","objects_scrubbed":7071,"log_size":5388,"log_dups_size":0,"ondisk_log_size":5388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:43.111443+0000","scrub_duration":2344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251854848,"num_objects":3843,"num_object_clones":0,"num_object_copies":11529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11383,"num_write_kb":487232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":824,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.721239+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:59:09.721239+0000","last_peered":"2024-09-21T11:59:09.721239+0000","last_clean":"2024-09-21T11:59:09.721239+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:59:09.721239+0000","last_undegraded":"2024-09-21T11:59:09.721239+0000","last_fullsized":"2024-09-21T11:59:09.721239+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":844,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720545+0000","last_change":"2024-09-21T11:57:58.535943+0000","last_active":"2024-09-21T11:59:09.720545+0000","last_peered":"2024-09-21T11:59:09.720545+0000","last_clean":"2024-09-21T11:59:09.720545+0000","last_became_active":"2024-09-21T11:57:58.535642+0000","last_became_peered":"2024-09-21T11:57:58.535642+0000","last_unstale":"2024-09-21T11:59:09.720545+0000","last_undegraded":"2024-09-21T11:59:09.720545+0000","last_fullsized":"2024-09-21T11:59:09.720545+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:09.486386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.486386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.896343+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1086,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:10.603881+0000","last_change":"2024-09-21T11:56:08.680971+0000","last_active":"2024-09-21T11:59:10.603881+0000","last_peered":"2024-09-21T11:59:10.603881+0000","last_clean":"2024-09-21T11:59:10.603881+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:59:10.603881+0000","last_undegraded":"2024-09-21T11:59:10.603881+0000","last_fullsized":"2024-09-21T11:59:10.603881+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:56:08.680766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.680766+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:46.070618+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":673,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723948+0000","last_change":"2024-09-21T11:58:17.532859+0000","last_active":"2024-09-21T11:59:09.723948+0000","last_peered":"2024-09-21T11:59:09.723948+0000","last_clean":"2024-09-21T11:59:09.723948+0000","last_became_active":"2024-09-21T11:58:17.531268+0000","last_became_peered":"2024-09-21T11:58:17.531268+0000","last_unstale":"2024-09-21T11:59:09.723948+0000","last_undegraded":"2024-09-21T11:59:09.723948+0000","last_fullsized":"2024-09-21T11:59:09.723948+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:51.665137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.665137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"448'11350","reported_seq":11521,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.302311+0000","last_change":"2024-09-21T11:59:13.679316+0000","last_active":"2024-09-21T11:59:16.302311+0000","last_peered":"2024-09-21T11:59:16.302311+0000","last_clean":"2024-09-21T11:59:16.302311+0000","last_became_active":"2024-09-21T11:56:36.508299+0000","last_became_peered":"2024-09-21T11:56:36.508299+0000","last_unstale":"2024-09-21T11:59:16.302311+0000","last_undegraded":"2024-09-21T11:59:16.302311+0000","last_fullsized":"2024-09-21T11:59:16.302311+0000","mapping_epoch":392,"log_start":"418'6615","ondisk_log_start":"418'6615","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'11214","last_scrub_stamp":"2024-09-21T11:59:13.679136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:13.679136+0000","objects_scrubbed":3897,"log_size":4735,"log_dups_size":0,"ondisk_log_size":4735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:20.950833+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244449280,"num_objects":3730,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11350,"num_write_kb":482560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":884,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.724818+0000","last_change":"2024-09-21T11:56:48.169371+0000","last_active":"2024-09-21T11:59:09.724818+0000","last_peered":"2024-09-21T11:59:09.724818+0000","last_clean":"2024-09-21T11:59:09.724818+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:59:09.724818+0000","last_undegraded":"2024-09-21T11:59:09.724818+0000","last_fullsized":"2024-09-21T11:59:09.724818+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:48.168795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.168795+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":877,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:12.932516+0000","last_change":"2024-09-21T11:59:12.932516+0000","last_active":"2024-09-21T11:59:12.932516+0000","last_peered":"2024-09-21T11:59:12.932516+0000","last_clean":"2024-09-21T11:59:12.932516+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:59:12.932516+0000","last_undegraded":"2024-09-21T11:59:12.932516+0000","last_fullsized":"2024-09-21T11:59:12.932516+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:12.932440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:59:12.932440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:25.638974+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":844,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.713365+0000","last_change":"2024-09-21T11:58:21.147985+0000","last_active":"2024-09-21T11:59:09.713365+0000","last_peered":"2024-09-21T11:59:09.713365+0000","last_clean":"2024-09-21T11:59:09.713365+0000","last_became_active":"2024-09-21T11:58:17.525791+0000","last_became_peered":"2024-09-21T11:58:17.525791+0000","last_unstale":"2024-09-21T11:59:09.713365+0000","last_undegraded":"2024-09-21T11:59:09.713365+0000","last_fullsized":"2024-09-21T11:59:09.713365+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:21.147942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:21.147942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:41.780615+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":66,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.714444+0000","last_change":"2024-09-21T11:57:58.529616+0000","last_active":"2024-09-21T11:59:09.714444+0000","last_peered":"2024-09-21T11:59:09.714444+0000","last_clean":"2024-09-21T11:59:09.714444+0000","last_became_active":"2024-09-21T11:57:58.529278+0000","last_became_peered":"2024-09-21T11:57:58.529278+0000","last_unstale":"2024-09-21T11:59:09.714444+0000","last_undegraded":"2024-09-21T11:59:09.714444+0000","last_fullsized":"2024-09-21T11:59:09.714444+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:35.532577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:35.532577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"448'11228","reported_seq":11401,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.287066+0000","last_change":"2024-09-21T11:59:08.338604+0000","last_active":"2024-09-21T11:59:16.287066+0000","last_peered":"2024-09-21T11:59:16.287066+0000","last_clean":"2024-09-21T11:59:16.287066+0000","last_became_active":"2024-09-21T11:56:36.506694+0000","last_became_peered":"2024-09-21T11:56:36.506694+0000","last_unstale":"2024-09-21T11:59:16.287066+0000","last_undegraded":"2024-09-21T11:59:16.287066+0000","last_fullsized":"2024-09-21T11:59:16.287066+0000","mapping_epoch":392,"log_start":"418'6519","ondisk_log_start":"418'6519","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"447'10791","last_scrub_stamp":"2024-09-21T11:59:08.338518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.338518+0000","objects_scrubbed":4166,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:31.958551+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243400704,"num_objects":3714,"num_object_clones":0,"num_object_copies":11142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11228,"num_write_kb":478144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":950,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.725493+0000","last_change":"2024-09-21T11:58:29.624654+0000","last_active":"2024-09-21T11:59:09.725493+0000","last_peered":"2024-09-21T11:59:09.725493+0000","last_clean":"2024-09-21T11:59:09.725493+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:59:09.725493+0000","last_undegraded":"2024-09-21T11:59:09.725493+0000","last_fullsized":"2024-09-21T11:59:09.725493+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:58:29.624549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:29.624549+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:48.198468+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":837,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.716014+0000","last_change":"2024-09-21T11:56:50.313230+0000","last_active":"2024-09-21T11:59:09.716014+0000","last_peered":"2024-09-21T11:59:09.716014+0000","last_clean":"2024-09-21T11:59:09.716014+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:59:09.716014+0000","last_undegraded":"2024-09-21T11:59:09.716014+0000","last_fullsized":"2024-09-21T11:59:09.716014+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.313165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.313165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:13.295853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":846,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.713517+0000","last_change":"2024-09-21T11:58:28.041091+0000","last_active":"2024-09-21T11:59:09.713517+0000","last_peered":"2024-09-21T11:59:09.713517+0000","last_clean":"2024-09-21T11:59:09.713517+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:59:09.713517+0000","last_undegraded":"2024-09-21T11:59:09.713517+0000","last_fullsized":"2024-09-21T11:59:09.713517+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:28.041026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.041026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:52.616083+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":849,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.716438+0000","last_change":"2024-09-21T11:59:08.676283+0000","last_active":"2024-09-21T11:59:09.716438+0000","last_peered":"2024-09-21T11:59:09.716438+0000","last_clean":"2024-09-21T11:59:09.716438+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:59:09.716438+0000","last_undegraded":"2024-09-21T11:59:09.716438+0000","last_fullsized":"2024-09-21T11:59:09.716438+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:08.676235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.676235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:13.547786+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"448'11220","reported_seq":24116,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:12.615392+0000","last_change":"2024-09-21T11:59:02.700721+0000","last_active":"2024-09-21T11:59:12.615392+0000","last_peered":"2024-09-21T11:59:12.615392+0000","last_clean":"2024-09-21T11:59:12.615392+0000","last_became_active":"2024-09-21T11:59:02.673387+0000","last_became_peered":"2024-09-21T11:59:02.673387+0000","last_unstale":"2024-09-21T11:59:12.615392+0000","last_undegraded":"2024-09-21T11:59:12.615392+0000","last_fullsized":"2024-09-21T11:59:12.615392+0000","mapping_epoch":444,"log_start":"415'5915","ondisk_log_start":"415'5915","created":392,"last_epoch_clean":445,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5305,"log_dups_size":0,"ondisk_log_size":5305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259915776,"num_objects":3966,"num_object_clones":0,"num_object_copies":11898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11220,"num_write_kb":485952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6378,"num_bytes_recovered":417988608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":864,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723986+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:59:09.723986+0000","last_peered":"2024-09-21T11:59:09.723986+0000","last_clean":"2024-09-21T11:59:09.723986+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:59:09.723986+0000","last_undegraded":"2024-09-21T11:59:09.723986+0000","last_fullsized":"2024-09-21T11:59:09.723986+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":904,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.717008+0000","last_change":"2024-09-21T11:58:18.420036+0000","last_active":"2024-09-21T11:59:09.717008+0000","last_peered":"2024-09-21T11:59:09.717008+0000","last_clean":"2024-09-21T11:59:09.717008+0000","last_became_active":"2024-09-21T11:58:17.530676+0000","last_became_peered":"2024-09-21T11:58:17.530676+0000","last_unstale":"2024-09-21T11:59:09.717008+0000","last_undegraded":"2024-09-21T11:59:09.717008+0000","last_fullsized":"2024-09-21T11:59:09.717008+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:18.419990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:58:18.419990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:25.445566+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":852,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.712587+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:59:09.712587+0000","last_peered":"2024-09-21T11:59:09.712587+0000","last_clean":"2024-09-21T11:59:09.712587+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:59:09.712587+0000","last_undegraded":"2024-09-21T11:59:09.712587+0000","last_fullsized":"2024-09-21T11:59:09.712587+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.052833+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":840,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720182+0000","last_change":"2024-09-21T11:56:02.541452+0000","last_active":"2024-09-21T11:59:09.720182+0000","last_peered":"2024-09-21T11:59:09.720182+0000","last_clean":"2024-09-21T11:59:09.720182+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:59:09.720182+0000","last_undegraded":"2024-09-21T11:59:09.720182+0000","last_fullsized":"2024-09-21T11:59:09.720182+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.541232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.541232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:09.854321+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"448'11186","reported_seq":11359,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:16.496335+0000","last_change":"2024-09-21T11:59:11.044768+0000","last_active":"2024-09-21T11:59:16.496335+0000","last_peered":"2024-09-21T11:59:16.496335+0000","last_clean":"2024-09-21T11:59:16.496335+0000","last_became_active":"2024-09-21T11:56:36.623765+0000","last_became_peered":"2024-09-21T11:56:36.623765+0000","last_unstale":"2024-09-21T11:59:16.496335+0000","last_undegraded":"2024-09-21T11:59:16.496335+0000","last_fullsized":"2024-09-21T11:59:16.496335+0000","mapping_epoch":392,"log_start":"417'5999","ondisk_log_start":"417'5999","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'10886","last_scrub_stamp":"2024-09-21T11:59:11.044452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:11.044452+0000","objects_scrubbed":3980,"log_size":5187,"log_dups_size":0,"ondisk_log_size":5187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:23.581441+0000","scrub_duration":1112,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239206400,"num_objects":3650,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11186,"num_write_kb":474752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":892,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.710841+0000","last_change":"2024-09-21T11:59:08.919308+0000","last_active":"2024-09-21T11:59:09.710841+0000","last_peered":"2024-09-21T11:59:09.710841+0000","last_clean":"2024-09-21T11:59:09.710841+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:59:09.710841+0000","last_undegraded":"2024-09-21T11:59:09.710841+0000","last_fullsized":"2024-09-21T11:59:09.710841+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:59:08.919232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.919232+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:19.454285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":899,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.718330+0000","last_change":"2024-09-21T11:57:58.539544+0000","last_active":"2024-09-21T11:59:09.718330+0000","last_peered":"2024-09-21T11:59:09.718330+0000","last_clean":"2024-09-21T11:59:09.718330+0000","last_became_active":"2024-09-21T11:57:58.528814+0000","last_became_peered":"2024-09-21T11:57:58.528814+0000","last_unstale":"2024-09-21T11:59:09.718330+0000","last_undegraded":"2024-09-21T11:59:09.718330+0000","last_fullsized":"2024-09-21T11:59:09.718330+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:57:01.221960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.221960+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.207576+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1085,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:10.603082+0000","last_change":"2024-09-21T11:57:10.282180+0000","last_active":"2024-09-21T11:59:10.603082+0000","last_peered":"2024-09-21T11:59:10.603082+0000","last_clean":"2024-09-21T11:59:10.603082+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:59:10.603082+0000","last_undegraded":"2024-09-21T11:59:10.603082+0000","last_fullsized":"2024-09-21T11:59:10.603082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:10.282134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.282134+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":833,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:13.897600+0000","last_change":"2024-09-21T11:59:13.897600+0000","last_active":"2024-09-21T11:59:13.897600+0000","last_peered":"2024-09-21T11:59:13.897600+0000","last_clean":"2024-09-21T11:59:13.897600+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:59:13.897600+0000","last_undegraded":"2024-09-21T11:59:13.897600+0000","last_fullsized":"2024-09-21T11:59:13.897600+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:13.897540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:13.897540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:34.214481+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":875,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722285+0000","last_change":"2024-09-21T11:57:58.527427+0000","last_active":"2024-09-21T11:59:09.722285+0000","last_peered":"2024-09-21T11:59:09.722285+0000","last_clean":"2024-09-21T11:59:09.722285+0000","last_became_active":"2024-09-21T11:57:58.527156+0000","last_became_peered":"2024-09-21T11:57:58.527156+0000","last_unstale":"2024-09-21T11:59:09.722285+0000","last_undegraded":"2024-09-21T11:59:09.722285+0000","last_fullsized":"2024-09-21T11:59:09.722285+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:38.938666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:38.938666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:49.629819+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":873,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720802+0000","last_change":"2024-09-21T11:56:29.528662+0000","last_active":"2024-09-21T11:59:09.720802+0000","last_peered":"2024-09-21T11:59:09.720802+0000","last_clean":"2024-09-21T11:59:09.720802+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:59:09.720802+0000","last_undegraded":"2024-09-21T11:59:09.720802+0000","last_fullsized":"2024-09-21T11:59:09.720802+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:56:29.528598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.528598+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":882,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.711418+0000","last_change":"2024-09-21T11:58:56.841707+0000","last_active":"2024-09-21T11:59:09.711418+0000","last_peered":"2024-09-21T11:59:09.711418+0000","last_clean":"2024-09-21T11:59:09.711418+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:59:09.711418+0000","last_undegraded":"2024-09-21T11:59:09.711418+0000","last_fullsized":"2024-09-21T11:59:09.711418+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:56.841648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:56.841648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:06.233795+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":870,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.715501+0000","last_change":"2024-09-21T11:58:20.347779+0000","last_active":"2024-09-21T11:59:09.715501+0000","last_peered":"2024-09-21T11:59:09.715501+0000","last_clean":"2024-09-21T11:59:09.715501+0000","last_became_active":"2024-09-21T11:58:17.529420+0000","last_became_peered":"2024-09-21T11:58:17.529420+0000","last_unstale":"2024-09-21T11:59:09.715501+0000","last_undegraded":"2024-09-21T11:59:09.715501+0000","last_fullsized":"2024-09-21T11:59:09.715501+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:20.347736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:20.347736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:47.925674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":866,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.725532+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:59:09.725532+0000","last_peered":"2024-09-21T11:59:09.725532+0000","last_clean":"2024-09-21T11:59:09.725532+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:59:09.725532+0000","last_undegraded":"2024-09-21T11:59:09.725532+0000","last_fullsized":"2024-09-21T11:59:09.725532+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":829,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.714884+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:59:09.714884+0000","last_peered":"2024-09-21T11:59:09.714884+0000","last_clean":"2024-09-21T11:59:09.714884+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:59:09.714884+0000","last_undegraded":"2024-09-21T11:59:09.714884+0000","last_fullsized":"2024-09-21T11:59:09.714884+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.174525+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":859,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720628+0000","last_change":"2024-09-21T11:56:50.696344+0000","last_active":"2024-09-21T11:59:09.720628+0000","last_peered":"2024-09-21T11:59:09.720628+0000","last_clean":"2024-09-21T11:59:09.720628+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:59:09.720628+0000","last_undegraded":"2024-09-21T11:59:09.720628+0000","last_fullsized":"2024-09-21T11:59:09.720628+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.696253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.696253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":834,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.714712+0000","last_change":"2024-09-21T11:57:12.518014+0000","last_active":"2024-09-21T11:59:09.714712+0000","last_peered":"2024-09-21T11:59:09.714712+0000","last_clean":"2024-09-21T11:59:09.714712+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:59:09.714712+0000","last_undegraded":"2024-09-21T11:59:09.714712+0000","last_fullsized":"2024-09-21T11:59:09.714712+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:12.517953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:12.517953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:52.060114+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":873,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.712621+0000","last_change":"2024-09-21T11:59:07.953847+0000","last_active":"2024-09-21T11:59:09.712621+0000","last_peered":"2024-09-21T11:59:09.712621+0000","last_clean":"2024-09-21T11:59:09.712621+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:59:09.712621+0000","last_undegraded":"2024-09-21T11:59:09.712621+0000","last_fullsized":"2024-09-21T11:59:09.712621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:59:07.953663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:07.953663+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:36.906230+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":861,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720956+0000","last_change":"2024-09-21T11:57:49.993421+0000","last_active":"2024-09-21T11:59:09.720956+0000","last_peered":"2024-09-21T11:59:09.720956+0000","last_clean":"2024-09-21T11:59:09.720956+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:59:09.720956+0000","last_undegraded":"2024-09-21T11:59:09.720956+0000","last_fullsized":"2024-09-21T11:59:09.720956+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:49.993339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:49.993339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1054,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:10.603592+0000","last_change":"2024-09-21T11:55:51.317564+0000","last_active":"2024-09-21T11:59:10.603592+0000","last_peered":"2024-09-21T11:59:10.603592+0000","last_clean":"2024-09-21T11:59:10.603592+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:59:10.603592+0000","last_undegraded":"2024-09-21T11:59:10.603592+0000","last_fullsized":"2024-09-21T11:59:10.603592+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:51.317473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.317473+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":332,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722691+0000","last_change":"2024-09-21T11:58:20.129924+0000","last_active":"2024-09-21T11:59:09.722691+0000","last_peered":"2024-09-21T11:59:09.722691+0000","last_clean":"2024-09-21T11:59:09.722691+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:59:09.722691+0000","last_undegraded":"2024-09-21T11:59:09.722691+0000","last_fullsized":"2024-09-21T11:59:09.722691+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:20.129854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:20.129854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:38.523985+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":894,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.724591+0000","last_change":"2024-09-21T11:57:16.786845+0000","last_active":"2024-09-21T11:59:09.724591+0000","last_peered":"2024-09-21T11:59:09.724591+0000","last_clean":"2024-09-21T11:59:09.724591+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:59:09.724591+0000","last_undegraded":"2024-09-21T11:59:09.724591+0000","last_fullsized":"2024-09-21T11:59:09.724591+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:57:16.786756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:16.786756+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":845,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.715896+0000","last_change":"2024-09-21T11:57:40.391494+0000","last_active":"2024-09-21T11:59:09.715896+0000","last_peered":"2024-09-21T11:59:09.715896+0000","last_clean":"2024-09-21T11:59:09.715896+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:59:09.715896+0000","last_undegraded":"2024-09-21T11:59:09.715896+0000","last_fullsized":"2024-09-21T11:59:09.715896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:40.391409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:40.391409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":1107,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:10.603001+0000","last_change":"2024-09-21T11:56:56.051487+0000","last_active":"2024-09-21T11:59:10.603001+0000","last_peered":"2024-09-21T11:59:10.603001+0000","last_clean":"2024-09-21T11:59:10.603001+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:59:10.603001+0000","last_undegraded":"2024-09-21T11:59:10.603001+0000","last_fullsized":"2024-09-21T11:59:10.603001+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:56.051300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:56.051300+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.828079+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":66,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.719743+0000","last_change":"2024-09-21T11:57:58.534490+0000","last_active":"2024-09-21T11:59:09.719743+0000","last_peered":"2024-09-21T11:59:09.719743+0000","last_clean":"2024-09-21T11:59:09.719743+0000","last_became_active":"2024-09-21T11:57:58.533810+0000","last_became_peered":"2024-09-21T11:57:58.533810+0000","last_unstale":"2024-09-21T11:59:09.719743+0000","last_undegraded":"2024-09-21T11:59:09.719743+0000","last_fullsized":"2024-09-21T11:59:09.719743+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.280698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.280698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":882,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722818+0000","last_change":"2024-09-21T11:58:28.658668+0000","last_active":"2024-09-21T11:59:09.722818+0000","last_peered":"2024-09-21T11:59:09.722818+0000","last_clean":"2024-09-21T11:59:09.722818+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:59:09.722818+0000","last_undegraded":"2024-09-21T11:59:09.722818+0000","last_fullsized":"2024-09-21T11:59:09.722818+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:28.658574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.658574+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:54.766743+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":348,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.719925+0000","last_change":"2024-09-21T11:58:56.695698+0000","last_active":"2024-09-21T11:59:09.719925+0000","last_peered":"2024-09-21T11:59:09.719925+0000","last_clean":"2024-09-21T11:59:09.719925+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:59:09.719925+0000","last_undegraded":"2024-09-21T11:59:09.719925+0000","last_fullsized":"2024-09-21T11:59:09.719925+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:56.695646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:58:56.695646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:21.948032+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":539,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723722+0000","last_change":"2024-09-21T11:56:00.441644+0000","last_active":"2024-09-21T11:59:09.723722+0000","last_peered":"2024-09-21T11:59:09.723722+0000","last_clean":"2024-09-21T11:59:09.723722+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:59:09.723722+0000","last_undegraded":"2024-09-21T11:59:09.723722+0000","last_fullsized":"2024-09-21T11:59:09.723722+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:00.441565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:00.441565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":832,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722274+0000","last_change":"2024-09-21T11:57:58.531539+0000","last_active":"2024-09-21T11:59:09.722274+0000","last_peered":"2024-09-21T11:59:09.722274+0000","last_clean":"2024-09-21T11:59:09.722274+0000","last_became_active":"2024-09-21T11:57:58.531213+0000","last_became_peered":"2024-09-21T11:57:58.531213+0000","last_unstale":"2024-09-21T11:59:09.722274+0000","last_undegraded":"2024-09-21T11:59:09.722274+0000","last_fullsized":"2024-09-21T11:59:09.722274+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:01.434123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.434123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":550,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723985+0000","last_change":"2024-09-21T11:55:59.419239+0000","last_active":"2024-09-21T11:59:09.723985+0000","last_peered":"2024-09-21T11:59:09.723985+0000","last_clean":"2024-09-21T11:59:09.723985+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:59:09.723985+0000","last_undegraded":"2024-09-21T11:59:09.723985+0000","last_fullsized":"2024-09-21T11:59:09.723985+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:59.419148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.419148+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":836,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.715775+0000","last_change":"2024-09-21T11:57:17.492529+0000","last_active":"2024-09-21T11:59:09.715775+0000","last_peered":"2024-09-21T11:59:09.715775+0000","last_clean":"2024-09-21T11:59:09.715775+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:59:09.715775+0000","last_undegraded":"2024-09-21T11:59:09.715775+0000","last_fullsized":"2024-09-21T11:59:09.715775+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:17.492466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:17.492466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:37.151802+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":844,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720292+0000","last_change":"2024-09-21T11:58:24.939301+0000","last_active":"2024-09-21T11:59:09.720292+0000","last_peered":"2024-09-21T11:59:09.720292+0000","last_clean":"2024-09-21T11:59:09.720292+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:59:09.720292+0000","last_undegraded":"2024-09-21T11:59:09.720292+0000","last_fullsized":"2024-09-21T11:59:09.720292+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:24.939230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:24.939230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.727906+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":872,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.714432+0000","last_change":"2024-09-21T11:57:37.071994+0000","last_active":"2024-09-21T11:59:09.714432+0000","last_peered":"2024-09-21T11:59:09.714432+0000","last_clean":"2024-09-21T11:59:09.714432+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:59:09.714432+0000","last_undegraded":"2024-09-21T11:59:09.714432+0000","last_fullsized":"2024-09-21T11:59:09.714432+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:57:37.071908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.071908+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":873,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722528+0000","last_change":"2024-09-21T11:57:18.759800+0000","last_active":"2024-09-21T11:59:09.722528+0000","last_peered":"2024-09-21T11:59:09.722528+0000","last_clean":"2024-09-21T11:59:09.722528+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:59:09.722528+0000","last_undegraded":"2024-09-21T11:59:09.722528+0000","last_fullsized":"2024-09-21T11:59:09.722528+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:57:18.759710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:18.759710+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":858,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.711494+0000","last_change":"2024-09-21T11:56:47.967479+0000","last_active":"2024-09-21T11:59:09.711494+0000","last_peered":"2024-09-21T11:59:09.711494+0000","last_clean":"2024-09-21T11:59:09.711494+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:59:09.711494+0000","last_undegraded":"2024-09-21T11:59:09.711494+0000","last_fullsized":"2024-09-21T11:59:09.711494+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:47.967392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:47.967392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:15.865477+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":821,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.716000+0000","last_change":"2024-09-21T11:58:23.430651+0000","last_active":"2024-09-21T11:59:09.716000+0000","last_peered":"2024-09-21T11:59:09.716000+0000","last_clean":"2024-09-21T11:59:09.716000+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:59:09.716000+0000","last_undegraded":"2024-09-21T11:59:09.716000+0000","last_fullsized":"2024-09-21T11:59:09.716000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:23.430517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:23.430517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:25.920377+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":848,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.720464+0000","last_change":"2024-09-21T11:57:58.537818+0000","last_active":"2024-09-21T11:59:09.720464+0000","last_peered":"2024-09-21T11:59:09.720464+0000","last_clean":"2024-09-21T11:59:09.720464+0000","last_became_active":"2024-09-21T11:57:58.537121+0000","last_became_peered":"2024-09-21T11:57:58.537121+0000","last_unstale":"2024-09-21T11:59:09.720464+0000","last_undegraded":"2024-09-21T11:59:09.720464+0000","last_fullsized":"2024-09-21T11:59:09.720464+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:43.803926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.803926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":877,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723197+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:59:09.723197+0000","last_peered":"2024-09-21T11:59:09.723197+0000","last_clean":"2024-09-21T11:59:09.723197+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:59:09.723197+0000","last_undegraded":"2024-09-21T11:59:09.723197+0000","last_fullsized":"2024-09-21T11:59:09.723197+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":828,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.721495+0000","last_change":"2024-09-21T11:56:53.232038+0000","last_active":"2024-09-21T11:59:09.721495+0000","last_peered":"2024-09-21T11:59:09.721495+0000","last_clean":"2024-09-21T11:59:09.721495+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:59:09.721495+0000","last_undegraded":"2024-09-21T11:59:09.721495+0000","last_fullsized":"2024-09-21T11:59:09.721495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.231963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.231963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":859,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.722396+0000","last_change":"2024-09-21T11:59:08.841142+0000","last_active":"2024-09-21T11:59:09.722396+0000","last_peered":"2024-09-21T11:59:09.722396+0000","last_clean":"2024-09-21T11:59:09.722396+0000","last_became_active":"2024-09-21T11:58:17.527743+0000","last_became_peered":"2024-09-21T11:58:17.527743+0000","last_unstale":"2024-09-21T11:59:09.722396+0000","last_undegraded":"2024-09-21T11:59:09.722396+0000","last_fullsized":"2024-09-21T11:59:09.722396+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:08.841066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.841066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:28.082425+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":849,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.714998+0000","last_change":"2024-09-21T11:58:53.713848+0000","last_active":"2024-09-21T11:59:09.714998+0000","last_peered":"2024-09-21T11:59:09.714998+0000","last_clean":"2024-09-21T11:59:09.714998+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:59:09.714998+0000","last_undegraded":"2024-09-21T11:59:09.714998+0000","last_fullsized":"2024-09-21T11:59:09.714998+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:53.713785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:53.713785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:04.557035+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":544,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723268+0000","last_change":"2024-09-21T11:58:17.532076+0000","last_active":"2024-09-21T11:59:09.723268+0000","last_peered":"2024-09-21T11:59:09.723268+0000","last_clean":"2024-09-21T11:59:09.723268+0000","last_became_active":"2024-09-21T11:58:17.531567+0000","last_became_peered":"2024-09-21T11:58:17.531567+0000","last_unstale":"2024-09-21T11:59:09.723268+0000","last_undegraded":"2024-09-21T11:59:09.723268+0000","last_fullsized":"2024-09-21T11:59:09.723268+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":873,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723564+0000","last_change":"2024-09-21T11:57:58.536812+0000","last_active":"2024-09-21T11:59:09.723564+0000","last_peered":"2024-09-21T11:59:09.723564+0000","last_clean":"2024-09-21T11:59:09.723564+0000","last_became_active":"2024-09-21T11:57:58.535015+0000","last_became_peered":"2024-09-21T11:57:58.535015+0000","last_unstale":"2024-09-21T11:59:09.723564+0000","last_undegraded":"2024-09-21T11:59:09.723564+0000","last_fullsized":"2024-09-21T11:59:09.723564+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:39.735027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:39.735027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":943,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.723556+0000","last_change":"2024-09-21T11:58:16.743891+0000","last_active":"2024-09-21T11:59:09.723556+0000","last_peered":"2024-09-21T11:59:09.723556+0000","last_clean":"2024-09-21T11:59:09.723556+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:59:09.723556+0000","last_undegraded":"2024-09-21T11:59:09.723556+0000","last_fullsized":"2024-09-21T11:59:09.723556+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:16.743830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:16.743830+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.512604+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":838,"reported_epoch":448,"state":"active+clean","last_fresh":"2024-09-21T11:59:09.725070+0000","last_change":"2024-09-21T11:57:48.045901+0000","last_active":"2024-09-21T11:59:09.725070+0000","last_peered":"2024-09-21T11:59:09.725070+0000","last_clean":"2024-09-21T11:59:09.725070+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:59:09.725070+0000","last_undegraded":"2024-09-21T11:59:09.725070+0000","last_fullsized":"2024-09-21T11:59:09.725070+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:48.045820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.045820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":3916693528,"num_objects":59765,"num_object_clones":0,"num_object_copies":179295,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4212,"num_objects_unfound":0,"num_objects_dirty":59765,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182701,"num_write_kb":7758849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15308,"num_bytes_recovered":1003159552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12235190272,"data_stored":12235178056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81048,"ondisk_log_size":81048,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":6904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1146,"num_read_kb":1141,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234220,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2477136,"kb_used_data":1701824,"kb_used_omap":1,"kb_used_meta":775294,"kb_avail":91268016,"statfs":{"total":95995035648,"available":93458448384,"internally_reserved":0,"allocated":1742667776,"data_stored":1742294519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":793901514},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463153,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2555208,"kb_used_data":1924216,"kb_used_omap":1,"kb_used_meta":630910,"kb_avail":91189944,"statfs":{"total":95995035648,"available":93378502656,"internally_reserved":0,"allocated":1970397184,"data_stored":1970032487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":646052294},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724790,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1629596,"kb_used_data":952780,"kb_used_omap":1,"kb_used_meta":676734,"kb_avail":92115556,"statfs":{"total":95995035648,"available":94326329344,"internally_reserved":0,"allocated":975646720,"data_stored":975261181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":692976070},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733921020,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1301536,"kb_used_data":702544,"kb_used_omap":1,"kb_used_meta":598974,"kb_avail":92443616,"statfs":{"total":95995035648,"available":94662262784,"internally_reserved":0,"allocated":719405056,"data_stored":719015952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":613349829},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117250,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2481168,"kb_used_data":1687808,"kb_used_omap":1,"kb_used_meta":793342,"kb_avail":91263984,"statfs":{"total":95995035648,"available":93454319616,"internally_reserved":0,"allocated":1728315392,"data_stored":1727942017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":812382663},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411592,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2212952,"kb_used_data":1437576,"kb_used_omap":1,"kb_used_meta":775294,"kb_avail":91532200,"statfs":{"total":95995035648,"available":93728972800,"internally_reserved":0,"allocated":1472077824,"data_stored":1471696992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":793901509},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575119,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2934892,"kb_used_data":2177628,"kb_used_omap":1,"kb_used_meta":757246,"kb_avail":90810260,"statfs":{"total":95995035648,"available":92989706240,"internally_reserved":0,"allocated":2229891072,"data_stored":2229489632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":775420358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738644,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2279012,"kb_used_data":1485584,"kb_used_omap":1,"kb_used_meta":793342,"kb_avail":91466140,"statfs":{"total":95995035648,"available":93661327360,"internally_reserved":0,"allocated":1521238016,"data_stored":1520849493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":812382661},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1504575488,"data_stored":1504575488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2197225472,"data_stored":2197225472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1454575616,"data_stored":1454571544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1700069376,"data_stored":1700069376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":722538496,"data_stored":722534424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":965935104,"data_stored":965935104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1967984640,"data_stored":1967980568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1722286080,"data_stored":1722286080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:59:21.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2023848829' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:21.302 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:21.334 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:21.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:21.335 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:21.436 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:21.463 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:21.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:21.464 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:21.565 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:21.599 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:21.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:21.600 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:21.701 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:21.730 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:21.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:21.732 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:21.832 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:21.859 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:21.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:21.860 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:21.960 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:21.985 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:21.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:21.986 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:22.057 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:21 smithi043 ceph-mon[36454]: pgmap v946: 158 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 789 op/s; 4212/179013 objects misplaced (2.353%); 12 MiB/s, 186 objects/s recovering 2024-09-21T11:59:22.059 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:21 smithi043 ceph-mon[29710]: pgmap v946: 158 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 789 op/s; 4212/179013 objects misplaced (2.353%); 12 MiB/s, 186 objects/s recovering 2024-09-21T11:59:22.087 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:22.117 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:22.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:22.118 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:22.219 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:22.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:21 smithi175 ceph-mon[32613]: pgmap v946: 158 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 154 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 789 op/s; 4212/179013 objects misplaced (2.353%); 12 MiB/s, 186 objects/s recovering 2024-09-21T11:59:22.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:59:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:59:22.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:22.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:22.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:22.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:22.399 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:22.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:22.511 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:59:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T11:59:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T11:59:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:59:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.511 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:166714 9.0 9:0db06241:::benchmark_data_smithi043_126002_object39648:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:18.113330+0000", 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 484.39284798199998, 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.182493339, 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 166714 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.512 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113330+0000", 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113330+0000", 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.513 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113331+0000", 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.068e-06 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113335+0000", 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6059999999999996e-06 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113349+0000", 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.323e-05 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.514 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113356+0000", 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.9920000000000001e-06 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113412+0000", 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6793999999999999e-05 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113528+0000", 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115573 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.515 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.113645+0000", 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000116731 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.116389+0000", 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027440640000000001 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.116875+0000", 2024-09-21T11:59:22.516 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00048631400000000001 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295714+0000", 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17883933099999999 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295772+0000", 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7215e-05 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:18.295823+0000", 2024-09-21T11:59:22.517 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1421000000000001e-05 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:211057 9.e e296/247 9:722bc66e:::benchmark_data_smithi043_126002_object83991:head v 296'13171, mlcod=296'13169)", 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409320+0000", 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 447.09685752399997, 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29737601699999999, 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.518 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409320+0000", 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409320+0000", 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409321+0000", 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0800000000000004e-07 2024-09-21T11:59:22.519 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409325+0000", 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4000000000000002e-06 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409333+0000", 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4669999999999998e-06 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.520 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409338+0000", 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.609e-06 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409445+0000", 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106464 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409463+0000", 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8719000000000002e-05 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.521 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706603+0000", 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.297139561 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706696+0000", 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3189000000000005e-05 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:211059 9.0 9:08679dd9:::benchmark_data_smithi043_126002_object83993:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:59:22.522 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409680+0000", 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 447.09649741300001, 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29635377000000002, 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 211059 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409680+0000", 2024-09-21T11:59:22.523 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409680+0000", 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409682+0000", 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3379999999999999e-06 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.524 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409687+0000", 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4929999999999998e-06 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409698+0000", 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0981e-05 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409706+0000", 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9680000000000006e-06 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.525 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.410134+0000", 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000427494 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.410245+0000", 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000111241 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.410365+0000", 2024-09-21T11:59:22.526 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000120077 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705854+0000", 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29548879099999997 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705865+0000", 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1168999999999999e-05 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.527 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705894+0000", 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9088999999999999e-05 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705988+0000", 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4019999999999998e-05 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706034+0000", 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6109000000000002e-05 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.528 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:211058 9.d e296/247 9:b1ac4faf:::benchmark_data_smithi043_126002_object83992:head v 296'13146, mlcod=296'13144)", 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409691+0000", 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 447.09648678299999, 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29170147000000002, 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409691+0000", 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409691+0000", 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409692+0000", 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0200000000000001e-07 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409698+0000", 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3860000000000004e-06 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409708+0000", 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3540000000000006e-06 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409714+0000", 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8120000000000002e-06 2024-09-21T11:59:22.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409791+0000", 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6365999999999994e-05 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409815+0000", 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4087000000000002e-05 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413064+0000", 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032496109999999999 2024-09-21T11:59:22.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701392+0000", 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.288328052 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:211060 9.0 9:0749ae3e:::benchmark_data_smithi043_126002_object83994:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.409751+0000", 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 447.09642648200003, 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.296470603, 2024-09-21T11:59:22.533 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 211060 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409751+0000", 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409751+0000", 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409752+0000", 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9299999999999997e-07 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409756+0000", 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8349999999999997e-06 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409764+0000", 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9580000000000007e-06 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.409771+0000", 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6819999999999997e-06 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701791+0000", 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29201998200000001 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701945+0000", 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000154518 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,6", 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.702140+0000", 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019451399999999999 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706071+0000", 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039315360000000002 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706081+0000", 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6900000000000004e-06 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706111+0000", 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9796999999999999e-05 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706188+0000", 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7074000000000001e-05 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706222+0000", 2024-09-21T11:59:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.4224000000000003e-05 2024-09-21T11:59:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25565.0:211067 9.4 9:2a2e5d4d:::benchmark_data_smithi043_126002_object84001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.412385+0000", 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 447.09379306, 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.30543494500000001, 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25565", 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 211067 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412385+0000", 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412385+0000", 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412386+0000", 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8029999999999999e-06 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412912+0000", 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00052593400000000004 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701342+0000", 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28842914800000002 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701361+0000", 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9381999999999998e-05 2024-09-21T11:59:22.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701424+0000", 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2634999999999999e-05 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701566+0000", 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014276 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701694+0000", 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012741900000000001 2024-09-21T11:59:22.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.707583+0000", 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0058892190000000002 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.707636+0000", 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3359999999999997e-05 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.717703+0000", 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010067066 2024-09-21T11:59:22.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.717763+0000", 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9849999999999998e-05 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.717820+0000", 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6369000000000002e-05 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25565.0:211066 9.a e296/247 9:51f636b8:::benchmark_data_smithi043_126002_object84000:head v 296'13283, mlcod=296'13281)", 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.412517+0000", 2024-09-21T11:59:22.545 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 447.09366047700001, 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29723240400000001, 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412517+0000", 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412517+0000", 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412518+0000", 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2139999999999999e-06 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412621+0000", 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000102394 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412632+0000", 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0849000000000001e-05 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.412639+0000", 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7769999999999997e-06 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701367+0000", 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28872841500000002 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701394+0000", 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7345999999999999e-05 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.709721+0000", 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083270189999999997 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.709750+0000", 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.8390000000000001e-05 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.549 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:211059 9.0 e296/247 ondisk, result = 0)", 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 447.09288507100001, 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.292746124, 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.550 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413293+0000", 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1399999999999996e-07 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.413297+0000", 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8070000000000001e-06 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.551 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701343+0000", 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28804594700000002 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701357+0000", 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3959e-05 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705885+0000", 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0045275940000000002 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.552 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705890+0000", 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2009999999999998e-06 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.706039+0000", 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014890199999999999 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop_reply(client.25565.0:211059 9.0 e296/247 ondisk, result = 0)", 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:51:55.414118+0000", 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 447.09205978, 2024-09-21T11:59:22.553 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29174787600000002, 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414118+0000", 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414118+0000", 2024-09-21T11:59:22.554 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414119+0000", 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5899999999999995e-07 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.414123+0000", 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5170000000000001e-06 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701349+0000", 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28722547300000001 2024-09-21T11:59:22.555 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.701359+0000", 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0664e-05 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705840+0000", 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0044810070000000004 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705848+0000", 2024-09-21T11:59:22.556 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.3900000000000004e-06 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:51:55.705866+0000", 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7966e-05 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:126300 10.b e348/313 10:d6278167:::benchmark_data_smithi043_150678_object3549:head v 348'7832, mlcod=348'7829)", 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 297.27310709, 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.23630744300000001, 2024-09-21T11:59:22.557 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.558 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233071+0000", 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1999999999999999e-07 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233078+0000", 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.6150000000000002e-06 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233088+0000", 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7499999999999998e-06 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.559 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233094+0000", 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.246e-06 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233148+0000", 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3943000000000002e-05 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.233168+0000", 2024-09-21T11:59:22.560 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9822000000000001e-05 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.469306+0000", 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.236138347 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.469378+0000", 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2000000000000002e-05 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.561 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:126310 10.2 10:429ff067:::benchmark_data_smithi043_150678_object3559:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.234619+0000", 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 297.27155863000002, 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.23258517300000001, 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 126310 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.562 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234619+0000", 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234619+0000", 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234621+0000", 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0660000000000002e-06 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.563 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.234627+0000", 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3090000000000001e-06 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456221+0000", 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.22159378199999999 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456236+0000", 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4304999999999999e-05 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.564 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456300+0000", 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4294999999999993e-05 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456588+0000", 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028769499999999998 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456701+0000", 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113275 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:22.566 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:22.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.464708+0000", 2024-09-21T11:59:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080072679999999997 2024-09-21T11:59:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.567 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.464728+0000", 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9700000000000001e-05 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467080+0000", 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00235214 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467137+0000", 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.7194999999999997e-05 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467204+0000", 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7143000000000004e-05 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:126311 10.0 e348/313 10:03ca3f46:::benchmark_data_smithi043_150678_object3560:head v 348'7980, mlcod=348'7978)", 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:25.238509+0000", 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 297.26766853100003, 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.22861142000000001, 2024-09-21T11:59:22.569 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238509+0000", 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238509+0000", 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238511+0000", 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4559999999999999e-06 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.238524+0000", 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2875e-05 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.455912+0000", 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.21738855400000001 2024-09-21T11:59:22.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.455934+0000", 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1517000000000001e-05 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456941+0000", 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0010073829999999999 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.456966+0000", 2024-09-21T11:59:22.572 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4556e-05 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467084+0000", 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010118240000000001 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:25.467121+0000", 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6838999999999997e-05 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:130557 10.2 10:4741c7dc:::benchmark_data_smithi043_150678_object7806:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.493746+0000", 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 293.01243159299997, 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.18472519600000001, 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 130557 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493746+0000", 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493746+0000", 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493748+0000", 2024-09-21T11:59:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0480000000000001e-06 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493754+0000", 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0769999999999996e-06 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493795+0000", 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0853999999999999e-05 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493804+0000", 2024-09-21T11:59:22.576 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4950000000000008e-06 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493862+0000", 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8749999999999998e-05 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.493968+0000", 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000105327 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.494070+0000", 2024-09-21T11:59:22.577 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010192 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497356+0000", 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.003286027 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678126+0000", 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.18076989199999999 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678302+0000", 2024-09-21T11:59:22.578 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017640099999999999 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678415+0000", 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000112965 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678471+0000", 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.6440000000000002e-05 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:130563 10.2 10:4b5202a2:::benchmark_data_smithi043_150678_object7812:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.497727+0000", 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 293.00845109800002, 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.183693038, 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 130563 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497727+0000", 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497727+0000", 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497728+0000", 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.807e-06 2024-09-21T11:59:22.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497735+0000", 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7490000000000001e-06 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497748+0000", 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2532e-05 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.497756+0000", 2024-09-21T11:59:22.582 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1100000000000003e-06 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.500514+0000", 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0027577190000000001 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675393+0000", 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17487897299999999 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675484+0000", 2024-09-21T11:59:22.583 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1860999999999993e-05 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678546+0000", 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0030618709999999999 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681236+0000", 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026896289999999998 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.584 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681299+0000", 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2710000000000001e-05 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681371+0000", 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2080000000000001e-05 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681420+0000", 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8996999999999999e-05 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.585 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:130569 10.2 10:46c1164d:::benchmark_data_smithi043_150678_object7818:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.502522+0000", 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 293.00365613399998, 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17911437499999999, 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 130569 2024-09-21T11:59:22.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502522+0000", 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502522+0000", 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.587 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502524+0000", 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1940000000000001e-06 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502531+0000", 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0960000000000003e-06 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675246+0000", 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.172715127 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.588 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675264+0000", 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8306999999999999e-05 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678690+0000", 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0034253310000000002 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678797+0000", 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107679 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.589 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.678873+0000", 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5402999999999996e-05 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681486+0000", 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0026133100000000002 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681534+0000", 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8352000000000001e-05 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.590 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681547+0000", 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2784e-05 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681595+0000", 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7403e-05 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.681636+0000", 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1389e-05 2024-09-21T11:59:22.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.25910.0:130568 10.0 e348/313 10:0d347089:::benchmark_data_smithi043_150678_object7817:head v 348'8247, mlcod=348'8245)", 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:54:29.502849+0000", 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 293.00332877800003, 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.180754058, 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.592 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502849+0000", 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502849+0000", 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502850+0000", 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-06 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.593 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.502860+0000", 2024-09-21T11:59:22.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0000000000000001e-05 2024-09-21T11:59:22.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.594 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.594 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675242+0000", 2024-09-21T11:59:22.594 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17238205500000001 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675254+0000", 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1874e-05 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675376+0000", 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012161 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.595 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.675398+0000", 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2949999999999999e-05 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.683567+0000", 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0081684180000000002 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:54:29.683603+0000", 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6127999999999997e-05 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:60092 11.c 11:39d222d2:::benchmark_data_smithi043_178089_object60091:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e402)", 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:57:13.474688+0000", 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 129.03149000299999, 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27958532800000002, 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 60092 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.597 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474688+0000", 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474688+0000", 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474689+0000", 2024-09-21T11:59:22.598 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.201e-06 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474709+0000", 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0089999999999999e-05 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474719+0000", 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0302000000000001e-05 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474725+0000", 2024-09-21T11:59:22.599 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3759999999999999e-06 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474776+0000", 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1663000000000003e-05 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.474920+0000", 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014358400000000001 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.475107+0000", 2024-09-21T11:59:22.600 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000186949 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.477348+0000", 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022414790000000002 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.477634+0000", 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028583999999999999 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.754107+0000", 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27647271099999998 2024-09-21T11:59:22.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.754213+0000", 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010594000000000001 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:57:13.754273+0000", 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0192999999999998e-05 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:155693 11.c 11:3c1f6998:::benchmark_data_smithi043_178089_object34458:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e442)", 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:58:47.225317+0000", 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.280860879999999, 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.18613823900000001, 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 155693 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.603 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.225317+0000", 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.225317+0000", 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.225318+0000", 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5799999999999999e-06 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.225325+0000", 2024-09-21T11:59:22.604 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4069999999999998e-06 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.225337+0000", 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1822e-05 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.225345+0000", 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0450000000000001e-06 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.226296+0000", 2024-09-21T11:59:22.605 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00095128200000000002 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.226418+0000", 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121766 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,4,7", 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.226548+0000", 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012986200000000001 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.232929+0000", 2024-09-21T11:59:22.606 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0063811750000000002 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.233599+0000", 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00067011600000000001 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.233716+0000", 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117098 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.411253+0000", 2024-09-21T11:59:22.607 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.177537259 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.411335+0000", 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2130000000000001e-05 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.411455+0000", 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000119697 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:155703 11.6 11:6086f008:::benchmark_data_smithi043_178089_object34468:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e442)", 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:58:47.232737+0000", 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.273440755999999, 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.182553101, 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 155703 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.232737+0000", 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.232737+0000", 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.232738+0000", 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.26e-06 2024-09-21T11:59:22.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.232743+0000", 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4279999999999998e-06 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.232753+0000", 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0220999999999999e-05 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.232760+0000", 2024-09-21T11:59:22.611 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.1559999999999998e-06 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.233058+0000", 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00029753899999999998 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.233171+0000", 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000113245 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,2,4", 2024-09-21T11:59:22.612 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.233731+0000", 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00056058599999999996 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.238448+0000", 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004716151 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.238535+0000", 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7573999999999999e-05 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.243490+0000", 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0049545470000000001 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.414963+0000", 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17147317400000001 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.415142+0000", 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017956100000000001 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.614 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.415290+0000", 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014765899999999999 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:155713 11.6 11:6470c697:::benchmark_data_smithi043_178089_object34478:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e442)", 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:58:47.239685+0000", 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 35.266493103999998, 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.176333025, 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:22.615 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:22.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:22.616 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:22.616 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 155713 2024-09-21T11:59:22.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.617 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:22.617 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.617 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:22.617 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.239685+0000", 2024-09-21T11:59:22.617 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.239685+0000", 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.239686+0000", 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.649e-06 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.239692+0000", 2024-09-21T11:59:22.618 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8529999999999997e-06 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.239710+0000", 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7473e-05 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.239717+0000", 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5379999999999997e-06 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.239777+0000", 2024-09-21T11:59:22.619 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9450999999999997e-05 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.239937+0000", 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000160576 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,2,4", 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.240168+0000", 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023050499999999999 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.620 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.243536+0000", 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033679869999999998 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.244499+0000", 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00096311300000000003 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.244543+0000", 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4656000000000001e-05 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:22.621 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.415858+0000", 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.17131471400000001 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.415911+0000", 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2862999999999999e-05 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:58:47.416018+0000", 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000106647 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.623 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:22.623 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:22.623 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:59:22.667 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:22.693 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:22.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:22.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:22.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:22.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:22.839 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:22.840 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:22.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:22.969 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:22.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:22.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:23.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:23.102 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:23.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:23.103 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:23.204 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:23.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:23.237 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:23.237 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:23.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:23.367 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:23.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:23.368 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:23.468 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:23.495 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:23.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:23.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:23.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:23.639 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:23.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:23.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:23.742 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:23.769 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:23.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:23.770 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:23.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:23.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:23.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:23.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:59:24.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:24.066 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:24.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:24.067 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:24.167 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:24.191 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:24.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:24.192 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:24.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:59:24.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:23 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:24.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:23 smithi043 ceph-mon[29710]: pgmap v947: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s; 3533/168858 objects misplaced (2.092%); 13 MiB/s, 201 objects/s recovering 2024-09-21T11:59:24.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:23 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:24.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:23 smithi043 ceph-mon[36454]: pgmap v947: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s; 3533/168858 objects misplaced (2.092%); 13 MiB/s, 201 objects/s recovering 2024-09-21T11:59:24.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:23 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:24.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:23 smithi175 ceph-mon[32613]: pgmap v947: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s; 3533/168858 objects misplaced (2.092%); 13 MiB/s, 201 objects/s recovering 2024-09-21T11:59:24.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:24.329 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:24.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:24.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:24.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:24.457 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:24.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:24.457 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:24.558 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:24.579 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:24.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:24.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:24.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:24.719 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:24.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:24.720 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:24.821 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:24.849 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:24.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:24.850 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:24.950 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:24.979 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:24.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:24.980 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:25.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:25.126 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:25.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:25.127 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:25.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:25.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:25.285 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:25.285 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:25.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:25.419 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:25.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:25.420 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:25.490 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:59:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:59:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:25.520 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:25.550 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:25.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:25.551 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:25.651 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:25.682 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:25.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:25.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:25.784 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:25.814 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:25.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:25.815 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:25.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T11:59:25.913 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:25.916 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:25.941 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:25.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:25.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:26.043 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:26.059 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:25 smithi043 ceph-mon[36454]: pgmap v948: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 592 op/s; 3533/168075 objects misplaced (2.102%); 4.2 MiB/s, 67 objects/s recovering 2024-09-21T11:59:26.059 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:25 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:59:26.059 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:25 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:59:26.059 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:25 smithi043 ceph-mon[36454]: osdmap e452: 8 total, 8 up, 8 in 2024-09-21T11:59:26.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:25 smithi043 ceph-mon[29710]: pgmap v948: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 592 op/s; 3533/168075 objects misplaced (2.102%); 4.2 MiB/s, 67 objects/s recovering 2024-09-21T11:59:26.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:25 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:59:26.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:25 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:59:26.061 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:25 smithi043 ceph-mon[29710]: osdmap e452: 8 total, 8 up, 8 in 2024-09-21T11:59:26.068 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:26.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:26.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:26.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:26.199 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:26.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:26.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:26.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:25 smithi175 ceph-mon[32613]: pgmap v948: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 592 op/s; 3533/168075 objects misplaced (2.102%); 4.2 MiB/s, 67 objects/s recovering 2024-09-21T11:59:26.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:25 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:59:26.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:25 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:59:26.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:25 smithi175 ceph-mon[32613]: osdmap e452: 8 total, 8 up, 8 in 2024-09-21T11:59:26.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:26.352 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:26.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:26.353 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:26.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:26.490 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:26.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:26.491 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:26.545 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:26.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:26.638 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:26.670 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:26.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:26.671 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:26.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:26.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:59:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:26.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:26.832 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:26.933 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:26.959 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:26.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:26.960 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:26.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:26 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:59:26.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:26 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:59:26.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:26 smithi043 ceph-mon[36454]: osdmap e453: 8 total, 8 up, 8 in 2024-09-21T11:59:26.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:26 smithi043 ceph-mon[36454]: pgmap v951: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.3 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 861 op/s; 3533/161811 objects misplaced (2.183%); 4.2 MiB/s, 67 objects/s recovering 2024-09-21T11:59:26.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:26 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:59:26.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:26 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:59:26.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:26 smithi043 ceph-mon[29710]: osdmap e453: 8 total, 8 up, 8 in 2024-09-21T11:59:26.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:26 smithi043 ceph-mon[29710]: pgmap v951: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.3 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 861 op/s; 3533/161811 objects misplaced (2.183%); 4.2 MiB/s, 67 objects/s recovering 2024-09-21T11:59:27.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:27.089 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:27.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:27.090 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:27.191 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:27.216 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:27.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:27.217 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:27.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:26 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:59:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:26 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:59:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:26 smithi175 ceph-mon[32613]: osdmap e453: 8 total, 8 up, 8 in 2024-09-21T11:59:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:26 smithi175 ceph-mon[32613]: pgmap v951: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.3 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 861 op/s; 3533/161811 objects misplaced (2.183%); 4.2 MiB/s, 67 objects/s recovering 2024-09-21T11:59:27.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:27.371 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:27.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:27.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:27.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:27.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:27.513 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:27.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:27.528 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.26339.0:193110 11.c e453/449 11:3668c400:::benchmark_data_smithi043_178089_object71875:head v 453'12112, mlcod=453'12110)", 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:27.496711+0000", 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.012851133000000001, 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.012877700000000001, 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:27.529 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.496711+0000", 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.496711+0000", 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.496711+0000", 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5300000000000004e-07 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.530 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.496718+0000", 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.737e-06 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.496726+0000", 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8630000000000008e-06 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.496731+0000", 2024-09-21T11:59:27.531 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2789999999999998e-06 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.496810+0000", 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.8790999999999998e-05 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.496876+0000", 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5990999999999999e-05 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:27.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.26339.0:193114 11.f e453/392 11:f9f0e5f3:::benchmark_data_smithi043_178089_object71879:head v 453'12403, mlcod=453'12401)", 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:27.497825+0000", 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.011737022, 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011818768, 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.533 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.497825+0000", 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.497825+0000", 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.497825+0000", 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0200000000000001e-07 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.497831+0000", 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8109999999999999e-06 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.497843+0000", 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2235999999999999e-05 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.497850+0000", 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7950000000000002e-06 2024-09-21T11:59:27.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.498150+0000", 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00030009500000000001 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.498169+0000", 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8329e-05 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.536 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.26339.0:193113 11.c e453/449 11:3a441798:::benchmark_data_smithi043_178089_object71878:head v 453'12113, mlcod=453'12110)", 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:27.504646+0000", 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0049156310000000002, 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050460749999999997, 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.504646+0000", 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:27.537 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.504646+0000", 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.504647+0000", 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.009e-06 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.504654+0000", 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.277e-06 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.538 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.504664+0000", 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0083e-05 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.504672+0000", 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2189999999999998e-06 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.504946+0000", 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00027391999999999998 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.504966+0000", 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0307999999999999e-05 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.26339.0:193118 11.c e453/449 11:38b5ae64:::benchmark_data_smithi043_178089_object71883:head v 453'12114, mlcod=453'12110)", 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:27.505401+0000", 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0041601420000000004, 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004338298, 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.505401+0000", 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.505401+0000", 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:27.541 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.505402+0000", 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7199999999999998e-07 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.505407+0000", 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1680000000000003e-06 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.505415+0000", 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1149999999999994e-06 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.505422+0000", 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3749999999999999e-06 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.507672+0000", 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0022503100000000002 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.507690+0000", 2024-09-21T11:59:27.543 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8156e-05 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_repop(client.26339.0:193117 11.6 e453/444 11:6fcfc3c3:::benchmark_data_smithi043_178089_object71882:head v 453'12111, mlcod=453'12109)", 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:27.506037+0000", 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0035247120000000002, 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037506499999999999, 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "started", 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.544 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.506037+0000", 2024-09-21T11:59:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.545 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.506037+0000", 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.506037+0000", 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7300000000000002e-07 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.506041+0000", 2024-09-21T11:59:27.546 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.704e-06 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.506045+0000", 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5759999999999999e-06 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.506051+0000", 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2050000000000001e-06 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.506584+0000", 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00053337000000000002 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:27.506603+0000", 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8609000000000001e-05 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 5 2024-09-21T11:59:27.548 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:59:27.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:27.644 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:27.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:27.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:27.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:27.770 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:59:27.773 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:59:27.786 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:27.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:27.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:59:27.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:27.910 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:27.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:27.911 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:28.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:28.050 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:28.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:28.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:28.112 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:27 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3618490606' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:28.112 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:27 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:28.112 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:27 smithi043 ceph-mon[36454]: from='client.16797 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:59:28.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:27 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3618490606' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:28.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:27 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:28.113 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:27 smithi043 ceph-mon[29710]: from='client.16797 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:59:28.152 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:28.183 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:28.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:28.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:28.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:27 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3618490606' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:27 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:27 smithi175 ceph-mon[32613]: from='client.16797 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:59:28.285 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:28.313 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:28.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:28.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:28.414 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:28.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:28.449 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:28.449 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:28.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T11:59:28.550 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:28.580 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:28.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:28.581 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:28.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:28.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:28.729 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:28.729 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:28.829 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:28.856 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:28.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:28.858 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:28.959 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:28.984 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:28.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:28.985 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:29.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:29.114 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:29.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:29.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:29.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:29.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:28 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:29.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:28 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:29.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:28 smithi043 ceph-mon[29710]: osdmap e454: 8 total, 8 up, 8 in 2024-09-21T11:59:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:28 smithi043 ceph-mon[29710]: pgmap v953: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 965 op/s; 3188/151482 objects misplaced (2.105%) 2024-09-21T11:59:29.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:28 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:29.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:28 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:29.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:28 smithi043 ceph-mon[36454]: osdmap e454: 8 total, 8 up, 8 in 2024-09-21T11:59:29.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:28 smithi043 ceph-mon[36454]: pgmap v953: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 965 op/s; 3188/151482 objects misplaced (2.105%) 2024-09-21T11:59:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:29.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:29.257 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:29.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:28 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:28 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:28 smithi175 ceph-mon[32613]: osdmap e454: 8 total, 8 up, 8 in 2024-09-21T11:59:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:28 smithi175 ceph-mon[32613]: pgmap v953: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 965 op/s; 3188/151482 objects misplaced (2.105%) 2024-09-21T11:59:29.357 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:29.386 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:29.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:29.387 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:29.488 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:29.515 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:29.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:29.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:29.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:29.645 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:29.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:29.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:29.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:29.787 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:29.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:29.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:29.872 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":951,"stamp":"2024-09-21T11:59:26.193204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3522304563,"num_objects":53937,"num_object_clones":0,"num_object_copies":161811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3533,"num_objects_unfound":0,"num_objects_dirty":53937,"num_whiteouts":0,"num_read":2274,"num_read_kb":2026,"num_write":189450,"num_write_kb":7758894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16117,"num_bytes_recovered":1032725379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82915,"ondisk_log_size":82915,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":476,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":16421436,"kb_used_data":10632164,"kb_used_omap":12,"kb_used_meta":5788659,"kb_avail":733539780,"statfs":{"total":767960285184,"available":751144734720,"internally_reserved":0,"allocated":10887335936,"data_stored":10884128982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":5927587378},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,1,6,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-375783424,"num_objects":-5734,"num_object_clones":0,"num_object_copies":-17202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-679,"num_objects_unfound":0,"num_objects_dirty":-5734,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5734,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":451,"num_bytes_recovered":29556736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.658695"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":72,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558552+0000","last_change":"2024-09-21T11:57:58.529638+0000","last_active":"2024-09-21T11:59:19.558552+0000","last_peered":"2024-09-21T11:59:19.558552+0000","last_clean":"2024-09-21T11:59:19.558552+0000","last_became_active":"2024-09-21T11:57:58.529245+0000","last_became_peered":"2024-09-21T11:57:58.529245+0000","last_unstale":"2024-09-21T11:59:19.558552+0000","last_undegraded":"2024-09-21T11:59:19.558552+0000","last_fullsized":"2024-09-21T11:59:19.558552+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:43.783030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.783030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":879,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834804+0000","last_change":"2024-09-21T11:57:37.936348+0000","last_active":"2024-09-21T11:59:25.834804+0000","last_peered":"2024-09-21T11:59:25.834804+0000","last_clean":"2024-09-21T11:59:25.834804+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:59:25.834804+0000","last_undegraded":"2024-09-21T11:59:25.834804+0000","last_fullsized":"2024-09-21T11:59:25.834804+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:37.936279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.936279+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:39.639770+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":553,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535223+0000","last_change":"2024-09-21T11:56:08.429983+0000","last_active":"2024-09-21T11:59:19.535223+0000","last_peered":"2024-09-21T11:59:19.535223+0000","last_clean":"2024-09-21T11:59:19.535223+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:59:19.535223+0000","last_undegraded":"2024-09-21T11:59:19.535223+0000","last_fullsized":"2024-09-21T11:59:19.535223+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:08.429912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.429912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":648,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589165+0000","last_change":"2024-09-21T11:58:21.827334+0000","last_active":"2024-09-21T11:59:19.589165+0000","last_peered":"2024-09-21T11:59:19.589165+0000","last_clean":"2024-09-21T11:59:19.589165+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:59:19.589165+0000","last_undegraded":"2024-09-21T11:59:19.589165+0000","last_fullsized":"2024-09-21T11:59:19.589165+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:21.827047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:21.827047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:43.889402+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":851,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.832702+0000","last_change":"2024-09-21T11:56:49.693769+0000","last_active":"2024-09-21T11:59:25.832702+0000","last_peered":"2024-09-21T11:59:25.832702+0000","last_clean":"2024-09-21T11:59:25.832702+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:59:25.832702+0000","last_undegraded":"2024-09-21T11:59:25.832702+0000","last_fullsized":"2024-09-21T11:59:25.832702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:49.693594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:49.693594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":841,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.833512+0000","last_change":"2024-09-21T11:57:37.711838+0000","last_active":"2024-09-21T11:59:25.833512+0000","last_peered":"2024-09-21T11:59:25.833512+0000","last_clean":"2024-09-21T11:59:25.833512+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:59:25.833512+0000","last_undegraded":"2024-09-21T11:59:25.833512+0000","last_fullsized":"2024-09-21T11:59:25.833512+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:37.711758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.711758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":886,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.833144+0000","last_change":"2024-09-21T11:56:31.572626+0000","last_active":"2024-09-21T11:59:25.833144+0000","last_peered":"2024-09-21T11:59:25.833144+0000","last_clean":"2024-09-21T11:59:25.833144+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:59:25.833144+0000","last_undegraded":"2024-09-21T11:59:25.833144+0000","last_fullsized":"2024-09-21T11:59:25.833144+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:31.572525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:31.572525+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":702,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834919+0000","last_change":"2024-09-21T11:58:30.584828+0000","last_active":"2024-09-21T11:59:25.834919+0000","last_peered":"2024-09-21T11:59:25.834919+0000","last_clean":"2024-09-21T11:59:25.834919+0000","last_became_active":"2024-09-21T11:58:17.527501+0000","last_became_peered":"2024-09-21T11:58:17.527501+0000","last_unstale":"2024-09-21T11:59:25.834919+0000","last_undegraded":"2024-09-21T11:59:25.834919+0000","last_fullsized":"2024-09-21T11:59:25.834919+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:30.584774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:30.584774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:50.421881+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":826,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.592740+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:59:19.592740+0000","last_peered":"2024-09-21T11:59:19.592740+0000","last_clean":"2024-09-21T11:59:19.592740+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:59:19.592740+0000","last_undegraded":"2024-09-21T11:59:19.592740+0000","last_fullsized":"2024-09-21T11:59:19.592740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.819967+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":879,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834212+0000","last_change":"2024-09-21T11:58:17.547443+0000","last_active":"2024-09-21T11:59:25.834212+0000","last_peered":"2024-09-21T11:59:25.834212+0000","last_clean":"2024-09-21T11:59:25.834212+0000","last_became_active":"2024-09-21T11:58:17.525792+0000","last_became_peered":"2024-09-21T11:58:17.525792+0000","last_unstale":"2024-09-21T11:59:25.834212+0000","last_undegraded":"2024-09-21T11:59:25.834212+0000","last_fullsized":"2024-09-21T11:59:25.834212+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:07.774892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:07.774892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":834,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589223+0000","last_change":"2024-09-21T11:58:58.842083+0000","last_active":"2024-09-21T11:59:19.589223+0000","last_peered":"2024-09-21T11:59:19.589223+0000","last_clean":"2024-09-21T11:59:19.589223+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:59:19.589223+0000","last_undegraded":"2024-09-21T11:59:19.589223+0000","last_fullsized":"2024-09-21T11:59:19.589223+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:58.842006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:58.842006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:22.061210+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":72,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535580+0000","last_change":"2024-09-21T11:57:58.527130+0000","last_active":"2024-09-21T11:59:19.535580+0000","last_peered":"2024-09-21T11:59:19.535580+0000","last_clean":"2024-09-21T11:59:19.535580+0000","last_became_active":"2024-09-21T11:57:58.526820+0000","last_became_peered":"2024-09-21T11:57:58.526820+0000","last_unstale":"2024-09-21T11:59:19.535580+0000","last_undegraded":"2024-09-21T11:59:19.535580+0000","last_fullsized":"2024-09-21T11:59:19.535580+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:36.422726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:36.422726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":872,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589715+0000","last_change":"2024-09-21T11:56:02.994041+0000","last_active":"2024-09-21T11:59:19.589715+0000","last_peered":"2024-09-21T11:59:19.589715+0000","last_clean":"2024-09-21T11:59:19.589715+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:59:19.589715+0000","last_undegraded":"2024-09-21T11:59:19.589715+0000","last_fullsized":"2024-09-21T11:59:19.589715+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.993966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.993966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:35.823113+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":877,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.833067+0000","last_change":"2024-09-21T11:58:17.549945+0000","last_active":"2024-09-21T11:59:25.833067+0000","last_peered":"2024-09-21T11:59:25.833067+0000","last_clean":"2024-09-21T11:59:25.833067+0000","last_became_active":"2024-09-21T11:58:17.525789+0000","last_became_peered":"2024-09-21T11:58:17.525789+0000","last_unstale":"2024-09-21T11:59:25.833067+0000","last_undegraded":"2024-09-21T11:59:25.833067+0000","last_fullsized":"2024-09-21T11:59:25.833067+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'7","last_scrub_stamp":"2024-09-21T11:56:59.659576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.659576+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":848,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587252+0000","last_change":"2024-09-21T11:58:22.381598+0000","last_active":"2024-09-21T11:59:19.587252+0000","last_peered":"2024-09-21T11:59:19.587252+0000","last_clean":"2024-09-21T11:59:19.587252+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:59:19.587252+0000","last_undegraded":"2024-09-21T11:59:19.587252+0000","last_fullsized":"2024-09-21T11:59:19.587252+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:22.381531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:22.381531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.811109+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":844,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589358+0000","last_change":"2024-09-21T11:58:24.861423+0000","last_active":"2024-09-21T11:59:19.589358+0000","last_peered":"2024-09-21T11:59:19.589358+0000","last_clean":"2024-09-21T11:59:19.589358+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:59:19.589358+0000","last_undegraded":"2024-09-21T11:59:19.589358+0000","last_fullsized":"2024-09-21T11:59:19.589358+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:24.861354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:24.861354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.549127+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":358,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835292+0000","last_change":"2024-09-21T11:59:15.886098+0000","last_active":"2024-09-21T11:59:25.835292+0000","last_peered":"2024-09-21T11:59:25.835292+0000","last_clean":"2024-09-21T11:59:25.835292+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:59:25.835292+0000","last_undegraded":"2024-09-21T11:59:25.835292+0000","last_fullsized":"2024-09-21T11:59:25.835292+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:15.886007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:15.886007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:23.801683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":72,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.536180+0000","last_change":"2024-09-21T11:57:58.534353+0000","last_active":"2024-09-21T11:59:19.536180+0000","last_peered":"2024-09-21T11:59:19.536180+0000","last_clean":"2024-09-21T11:59:19.536180+0000","last_became_active":"2024-09-21T11:57:58.530327+0000","last_became_peered":"2024-09-21T11:57:58.530327+0000","last_unstale":"2024-09-21T11:59:19.536180+0000","last_undegraded":"2024-09-21T11:59:19.536180+0000","last_fullsized":"2024-09-21T11:59:19.536180+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:27.176320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:27.176320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":998,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589327+0000","last_change":"2024-09-21T11:59:17.201834+0000","last_active":"2024-09-21T11:59:19.589327+0000","last_peered":"2024-09-21T11:59:19.589327+0000","last_clean":"2024-09-21T11:59:19.589327+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T11:59:19.589327+0000","last_undegraded":"2024-09-21T11:59:19.589327+0000","last_fullsized":"2024-09-21T11:59:19.589327+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:59:17.201563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:17.201563+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:38.988507+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":558,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.559152+0000","last_change":"2024-09-21T11:58:17.533215+0000","last_active":"2024-09-21T11:59:19.559152+0000","last_peered":"2024-09-21T11:59:19.559152+0000","last_clean":"2024-09-21T11:59:19.559152+0000","last_became_active":"2024-09-21T11:58:17.532890+0000","last_became_peered":"2024-09-21T11:58:17.532890+0000","last_unstale":"2024-09-21T11:59:19.559152+0000","last_undegraded":"2024-09-21T11:59:19.559152+0000","last_fullsized":"2024-09-21T11:59:19.559152+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:56.501356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:56.501356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":845,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.833483+0000","last_change":"2024-09-21T11:57:04.647951+0000","last_active":"2024-09-21T11:59:25.833483+0000","last_peered":"2024-09-21T11:59:25.833483+0000","last_clean":"2024-09-21T11:59:25.833483+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:59:25.833483+0000","last_undegraded":"2024-09-21T11:59:25.833483+0000","last_fullsized":"2024-09-21T11:59:25.833483+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:04.647749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:04.647749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":660,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535229+0000","last_change":"2024-09-21T11:55:57.442188+0000","last_active":"2024-09-21T11:59:19.535229+0000","last_peered":"2024-09-21T11:59:19.535229+0000","last_clean":"2024-09-21T11:59:19.535229+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:59:19.535229+0000","last_undegraded":"2024-09-21T11:59:19.535229+0000","last_fullsized":"2024-09-21T11:59:19.535229+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:57.441965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.441965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":905,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.832847+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:59:25.832847+0000","last_peered":"2024-09-21T11:59:25.832847+0000","last_clean":"2024-09-21T11:59:25.832847+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:59:25.832847+0000","last_undegraded":"2024-09-21T11:59:25.832847+0000","last_fullsized":"2024-09-21T11:59:25.832847+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":364,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834660+0000","last_change":"2024-09-21T11:58:17.529121+0000","last_active":"2024-09-21T11:59:25.834660+0000","last_peered":"2024-09-21T11:59:25.834660+0000","last_clean":"2024-09-21T11:59:25.834660+0000","last_became_active":"2024-09-21T11:58:17.528785+0000","last_became_peered":"2024-09-21T11:58:17.528785+0000","last_unstale":"2024-09-21T11:59:25.834660+0000","last_undegraded":"2024-09-21T11:59:25.834660+0000","last_fullsized":"2024-09-21T11:59:25.834660+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.487132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.487132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":341,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.536042+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:59:19.536042+0000","last_peered":"2024-09-21T11:59:19.536042+0000","last_clean":"2024-09-21T11:59:19.536042+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:59:19.536042+0000","last_undegraded":"2024-09-21T11:59:19.536042+0000","last_fullsized":"2024-09-21T11:59:19.536042+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":579,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535333+0000","last_change":"2024-09-21T11:58:17.554468+0000","last_active":"2024-09-21T11:59:19.535333+0000","last_peered":"2024-09-21T11:59:19.535333+0000","last_clean":"2024-09-21T11:59:19.535333+0000","last_became_active":"2024-09-21T11:58:17.529593+0000","last_became_peered":"2024-09-21T11:58:17.529593+0000","last_unstale":"2024-09-21T11:59:19.535333+0000","last_undegraded":"2024-09-21T11:59:19.535333+0000","last_fullsized":"2024-09-21T11:59:19.535333+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:10.530913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.530913+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":870,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558374+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:59:19.558374+0000","last_peered":"2024-09-21T11:59:19.558374+0000","last_clean":"2024-09-21T11:59:19.558374+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:59:19.558374+0000","last_undegraded":"2024-09-21T11:59:19.558374+0000","last_fullsized":"2024-09-21T11:59:19.558374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"432'10","reported_seq":1133,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835497+0000","last_change":"2024-09-21T11:58:18.736596+0000","last_active":"2024-09-21T11:59:25.835497+0000","last_peered":"2024-09-21T11:59:25.835497+0000","last_clean":"2024-09-21T11:59:25.835497+0000","last_became_active":"2024-09-21T11:58:17.529830+0000","last_became_peered":"2024-09-21T11:58:17.529830+0000","last_unstale":"2024-09-21T11:59:25.835497+0000","last_undegraded":"2024-09-21T11:59:25.835497+0000","last_fullsized":"2024-09-21T11:59:25.835497+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"383'9","last_scrub_stamp":"2024-09-21T11:58:18.736520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:18.736520+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:40.102209+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":88,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589284+0000","last_change":"2024-09-21T11:57:49.896884+0000","last_active":"2024-09-21T11:59:19.589284+0000","last_peered":"2024-09-21T11:59:19.589284+0000","last_clean":"2024-09-21T11:59:19.589284+0000","last_became_active":"2024-09-21T11:57:45.504680+0000","last_became_peered":"2024-09-21T11:57:45.504680+0000","last_unstale":"2024-09-21T11:59:19.589284+0000","last_undegraded":"2024-09-21T11:59:19.589284+0000","last_fullsized":"2024-09-21T11:59:19.589284+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:49.896819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:49.896819+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"451'11838","reported_seq":11968,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:21.858844+0000","last_change":"2024-09-21T11:56:36.507880+0000","last_active":"2024-09-21T11:59:21.858844+0000","last_peered":"2024-09-21T11:59:21.858844+0000","last_clean":"2024-09-21T11:59:21.858844+0000","last_became_active":"2024-09-21T11:56:36.507606+0000","last_became_peered":"2024-09-21T11:56:36.507606+0000","last_unstale":"2024-09-21T11:59:21.858844+0000","last_undegraded":"2024-09-21T11:59:21.858844+0000","last_fullsized":"2024-09-21T11:59:21.858844+0000","mapping_epoch":392,"log_start":"423'7022","ondisk_log_start":"423'7022","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4816,"log_dups_size":0,"ondisk_log_size":4816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":224133120,"num_objects":3420,"num_object_clones":0,"num_object_copies":10260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11838,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":579,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.559494+0000","last_change":"2024-09-21T11:55:52.562992+0000","last_active":"2024-09-21T11:59:19.559494+0000","last_peered":"2024-09-21T11:59:19.559494+0000","last_clean":"2024-09-21T11:59:19.559494+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:59:19.559494+0000","last_undegraded":"2024-09-21T11:59:19.559494+0000","last_fullsized":"2024-09-21T11:59:19.559494+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:55:52.562928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.562928+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":870,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835316+0000","last_change":"2024-09-21T11:57:13.738771+0000","last_active":"2024-09-21T11:59:25.835316+0000","last_peered":"2024-09-21T11:59:25.835316+0000","last_clean":"2024-09-21T11:59:25.835316+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:59:25.835316+0000","last_undegraded":"2024-09-21T11:59:25.835316+0000","last_fullsized":"2024-09-21T11:59:25.835316+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:13.738727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:13.738727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"432'11","reported_seq":488,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589680+0000","last_change":"2024-09-21T11:58:17.537763+0000","last_active":"2024-09-21T11:59:19.589680+0000","last_peered":"2024-09-21T11:59:19.589680+0000","last_clean":"2024-09-21T11:59:19.589680+0000","last_became_active":"2024-09-21T11:58:17.526752+0000","last_became_peered":"2024-09-21T11:58:17.526752+0000","last_unstale":"2024-09-21T11:59:19.589680+0000","last_undegraded":"2024-09-21T11:59:19.589680+0000","last_fullsized":"2024-09-21T11:59:19.589680+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"366'10","last_scrub_stamp":"2024-09-21T11:56:48.961017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.961017+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.739596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":891,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589263+0000","last_change":"2024-09-21T11:58:16.866853+0000","last_active":"2024-09-21T11:59:19.589263+0000","last_peered":"2024-09-21T11:59:19.589263+0000","last_clean":"2024-09-21T11:59:19.589263+0000","last_became_active":"2024-09-21T11:57:58.526927+0000","last_became_peered":"2024-09-21T11:57:58.526927+0000","last_unstale":"2024-09-21T11:59:19.589263+0000","last_undegraded":"2024-09-21T11:59:19.589263+0000","last_fullsized":"2024-09-21T11:59:19.589263+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:16.866782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:16.866782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.387681+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":96,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587960+0000","last_change":"2024-09-21T11:59:18.174455+0000","last_active":"2024-09-21T11:59:19.587960+0000","last_peered":"2024-09-21T11:59:19.587960+0000","last_clean":"2024-09-21T11:59:19.587960+0000","last_became_active":"2024-09-21T11:57:45.514811+0000","last_became_peered":"2024-09-21T11:57:45.514811+0000","last_unstale":"2024-09-21T11:59:19.587960+0000","last_undegraded":"2024-09-21T11:59:19.587960+0000","last_fullsized":"2024-09-21T11:59:19.587960+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:59:18.174384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:59:18.174384+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:22.819349+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"451'12013","reported_seq":12192,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:21.319592+0000","last_change":"2024-09-21T11:59:12.760513+0000","last_active":"2024-09-21T11:59:21.319592+0000","last_peered":"2024-09-21T11:59:21.319592+0000","last_clean":"2024-09-21T11:59:21.319592+0000","last_became_active":"2024-09-21T11:56:36.511191+0000","last_became_peered":"2024-09-21T11:56:36.511191+0000","last_unstale":"2024-09-21T11:59:21.319592+0000","last_undegraded":"2024-09-21T11:59:21.319592+0000","last_fullsized":"2024-09-21T11:59:21.319592+0000","mapping_epoch":392,"log_start":"423'7242","ondisk_log_start":"423'7242","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'11577","last_scrub_stamp":"2024-09-21T11:59:12.760365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:12.760365+0000","objects_scrubbed":4105,"log_size":4771,"log_dups_size":0,"ondisk_log_size":4771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:27.399210+0000","scrub_duration":1126,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238223360,"num_objects":3635,"num_object_clones":0,"num_object_copies":10905,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12013,"num_write_kb":500736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":961,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835023+0000","last_change":"2024-09-21T11:57:58.547816+0000","last_active":"2024-09-21T11:59:25.835023+0000","last_peered":"2024-09-21T11:59:25.835023+0000","last_clean":"2024-09-21T11:59:25.835023+0000","last_became_active":"2024-09-21T11:57:58.537184+0000","last_became_peered":"2024-09-21T11:57:58.537184+0000","last_unstale":"2024-09-21T11:59:25.835023+0000","last_undegraded":"2024-09-21T11:59:25.835023+0000","last_fullsized":"2024-09-21T11:59:25.835023+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:57:00.631769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:00.631769+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":864,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.833961+0000","last_change":"2024-09-21T11:56:53.701703+0000","last_active":"2024-09-21T11:59:25.833961+0000","last_peered":"2024-09-21T11:59:25.833961+0000","last_clean":"2024-09-21T11:59:25.833961+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:59:25.833961+0000","last_undegraded":"2024-09-21T11:59:25.833961+0000","last_fullsized":"2024-09-21T11:59:25.833961+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.701639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.701639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":350,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.832567+0000","last_change":"2024-09-21T11:58:19.959956+0000","last_active":"2024-09-21T11:59:25.832567+0000","last_peered":"2024-09-21T11:59:25.832567+0000","last_clean":"2024-09-21T11:59:25.832567+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:59:25.832567+0000","last_undegraded":"2024-09-21T11:59:25.832567+0000","last_fullsized":"2024-09-21T11:59:25.832567+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:19.959912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:19.959912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.737932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":356,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587880+0000","last_change":"2024-09-21T11:59:05.095763+0000","last_active":"2024-09-21T11:59:19.587880+0000","last_peered":"2024-09-21T11:59:19.587880+0000","last_clean":"2024-09-21T11:59:19.587880+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:59:19.587880+0000","last_undegraded":"2024-09-21T11:59:19.587880+0000","last_fullsized":"2024-09-21T11:59:19.587880+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:05.095722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:05.095722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:12.462739+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":97,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835236+0000","last_change":"2024-09-21T11:57:47.804542+0000","last_active":"2024-09-21T11:59:25.835236+0000","last_peered":"2024-09-21T11:59:25.835236+0000","last_clean":"2024-09-21T11:59:25.835236+0000","last_became_active":"2024-09-21T11:57:45.514189+0000","last_became_peered":"2024-09-21T11:57:45.514189+0000","last_unstale":"2024-09-21T11:59:25.835236+0000","last_undegraded":"2024-09-21T11:59:25.835236+0000","last_fullsized":"2024-09-21T11:59:25.835236+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:47.804466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:47.804466+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"451'11549","reported_seq":11699,"reported_epoch":451,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-21T11:59:21.835686+0000","last_change":"2024-09-21T11:59:18.695884+0000","last_active":"2024-09-21T11:59:21.835686+0000","last_peered":"2024-09-21T11:59:21.835686+0000","last_clean":"2024-09-21T11:58:17.235375+0000","last_became_active":"2024-09-21T11:58:18.605530+0000","last_became_peered":"2024-09-21T11:58:18.605530+0000","last_unstale":"2024-09-21T11:59:21.835686+0000","last_undegraded":"2024-09-21T11:59:21.835686+0000","last_fullsized":"2024-09-21T11:59:21.835686+0000","mapping_epoch":431,"log_start":"419'6767","ondisk_log_start":"419'6767","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4782,"log_dups_size":0,"ondisk_log_size":4782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231538688,"num_objects":3533,"num_object_clones":0,"num_object_copies":10599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3533,"num_objects_unfound":0,"num_objects_dirty":3533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11549,"num_write_kb":482624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,7,0],"avail_no_missing":["1","0","7"],"object_location_counts":[{"shards":"0,1,7","objects":3533}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":896,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589445+0000","last_change":"2024-09-21T11:58:17.828372+0000","last_active":"2024-09-21T11:59:19.589445+0000","last_peered":"2024-09-21T11:59:19.589445+0000","last_clean":"2024-09-21T11:59:19.589445+0000","last_became_active":"2024-09-21T11:58:17.525491+0000","last_became_peered":"2024-09-21T11:58:17.525491+0000","last_unstale":"2024-09-21T11:59:19.589445+0000","last_undegraded":"2024-09-21T11:59:19.589445+0000","last_fullsized":"2024-09-21T11:59:19.589445+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:58:17.828305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:17.828305+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.685496+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":76,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558471+0000","last_change":"2024-09-21T11:57:58.539709+0000","last_active":"2024-09-21T11:59:19.558471+0000","last_peered":"2024-09-21T11:59:19.558471+0000","last_clean":"2024-09-21T11:59:19.558471+0000","last_became_active":"2024-09-21T11:57:58.526938+0000","last_became_peered":"2024-09-21T11:57:58.526938+0000","last_unstale":"2024-09-21T11:59:19.558471+0000","last_undegraded":"2024-09-21T11:59:19.558471+0000","last_fullsized":"2024-09-21T11:59:19.558471+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:48.446635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.446635+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":1106,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834819+0000","last_change":"2024-09-21T11:56:05.567407+0000","last_active":"2024-09-21T11:59:25.834819+0000","last_peered":"2024-09-21T11:59:25.834819+0000","last_clean":"2024-09-21T11:59:25.834819+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:59:25.834819+0000","last_undegraded":"2024-09-21T11:59:25.834819+0000","last_fullsized":"2024-09-21T11:59:25.834819+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:05.567332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.567332+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":850,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587593+0000","last_change":"2024-09-21T11:56:34.581966+0000","last_active":"2024-09-21T11:59:19.587593+0000","last_peered":"2024-09-21T11:59:19.587593+0000","last_clean":"2024-09-21T11:59:19.587593+0000","last_became_active":"2024-09-21T11:56:34.581626+0000","last_became_peered":"2024-09-21T11:56:34.581626+0000","last_unstale":"2024-09-21T11:59:19.587593+0000","last_undegraded":"2024-09-21T11:59:19.587593+0000","last_fullsized":"2024-09-21T11:59:19.587593+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.643323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.643323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:35.106418+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":178,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835244+0000","last_change":"2024-09-21T11:57:45.998708+0000","last_active":"2024-09-21T11:59:25.835244+0000","last_peered":"2024-09-21T11:59:25.835244+0000","last_clean":"2024-09-21T11:59:25.835244+0000","last_became_active":"2024-09-21T11:57:45.514310+0000","last_became_peered":"2024-09-21T11:57:45.514310+0000","last_unstale":"2024-09-21T11:59:25.835244+0000","last_undegraded":"2024-09-21T11:59:25.835244+0000","last_fullsized":"2024-09-21T11:59:25.835244+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:45.998644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:45.998644+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:19.779779+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"451'11656","reported_seq":11841,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:21.990818+0000","last_change":"2024-09-21T11:59:01.985776+0000","last_active":"2024-09-21T11:59:21.990818+0000","last_peered":"2024-09-21T11:59:21.990818+0000","last_clean":"2024-09-21T11:59:21.990818+0000","last_became_active":"2024-09-21T11:56:36.623581+0000","last_became_peered":"2024-09-21T11:56:36.623581+0000","last_unstale":"2024-09-21T11:59:21.990818+0000","last_undegraded":"2024-09-21T11:59:21.990818+0000","last_fullsized":"2024-09-21T11:59:21.990818+0000","mapping_epoch":392,"log_start":"410'5516","ondisk_log_start":"410'5516","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"444'10565","last_scrub_stamp":"2024-09-21T11:59:01.985630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:01.985630+0000","objects_scrubbed":4571,"log_size":6140,"log_dups_size":0,"ondisk_log_size":6140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:14.582998+0000","scrub_duration":1821,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225640472,"num_objects":3444,"num_object_clones":0,"num_object_copies":10332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3444,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11656,"num_write_kb":483137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":610,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535531+0000","last_change":"2024-09-21T11:58:17.557543+0000","last_active":"2024-09-21T11:59:19.535531+0000","last_peered":"2024-09-21T11:59:19.535531+0000","last_clean":"2024-09-21T11:59:19.535531+0000","last_became_active":"2024-09-21T11:58:17.531001+0000","last_became_peered":"2024-09-21T11:58:17.531001+0000","last_unstale":"2024-09-21T11:59:19.535531+0000","last_undegraded":"2024-09-21T11:59:19.535531+0000","last_fullsized":"2024-09-21T11:59:19.535531+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:56:09.421193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.421193+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":876,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.593701+0000","last_change":"2024-09-21T11:56:51.036609+0000","last_active":"2024-09-21T11:59:19.593701+0000","last_peered":"2024-09-21T11:59:19.593701+0000","last_clean":"2024-09-21T11:59:19.593701+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:59:19.593701+0000","last_undegraded":"2024-09-21T11:59:19.593701+0000","last_fullsized":"2024-09-21T11:59:19.593701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:51.036535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:51.036535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:40.871456+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1088,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.559439+0000","last_change":"2024-09-21T11:57:40.102377+0000","last_active":"2024-09-21T11:59:19.559439+0000","last_peered":"2024-09-21T11:59:19.559439+0000","last_clean":"2024-09-21T11:59:19.559439+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:59:19.559439+0000","last_undegraded":"2024-09-21T11:59:19.559439+0000","last_fullsized":"2024-09-21T11:59:19.559439+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:40.102294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:40.102294+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":848,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535814+0000","last_change":"2024-09-21T11:57:36.109392+0000","last_active":"2024-09-21T11:59:19.535814+0000","last_peered":"2024-09-21T11:59:19.535814+0000","last_clean":"2024-09-21T11:59:19.535814+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:59:19.535814+0000","last_undegraded":"2024-09-21T11:59:19.535814+0000","last_fullsized":"2024-09-21T11:59:19.535814+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:36.109317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:36.109317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":186,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835244+0000","last_change":"2024-09-21T11:59:07.885015+0000","last_active":"2024-09-21T11:59:25.835244+0000","last_peered":"2024-09-21T11:59:25.835244+0000","last_clean":"2024-09-21T11:59:25.835244+0000","last_became_active":"2024-09-21T11:57:45.518764+0000","last_became_peered":"2024-09-21T11:57:45.518764+0000","last_unstale":"2024-09-21T11:59:25.835244+0000","last_undegraded":"2024-09-21T11:59:25.835244+0000","last_fullsized":"2024-09-21T11:59:25.835244+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:59:07.884939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:59:07.884939+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:13.711897+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.c","version":"451'11739","reported_seq":20742,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:21.851367+0000","last_change":"2024-09-21T11:59:18.569237+0000","last_active":"2024-09-21T11:59:21.851367+0000","last_peered":"2024-09-21T11:59:21.851367+0000","last_clean":"2024-09-21T11:59:21.851367+0000","last_became_active":"2024-09-21T11:59:18.539184+0000","last_became_peered":"2024-09-21T11:59:18.539184+0000","last_unstale":"2024-09-21T11:59:21.851367+0000","last_undegraded":"2024-09-21T11:59:21.851367+0000","last_fullsized":"2024-09-21T11:59:21.851367+0000","mapping_epoch":449,"log_start":"419'6879","ondisk_log_start":"419'6879","created":392,"last_epoch_clean":450,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4860,"log_dups_size":0,"ondisk_log_size":4860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225378304,"num_objects":3439,"num_object_clones":0,"num_object_copies":10317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11739,"num_write_kb":485696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4427,"num_bytes_recovered":290127872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":915,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558970+0000","last_change":"2024-09-21T11:57:50.976268+0000","last_active":"2024-09-21T11:59:19.558970+0000","last_peered":"2024-09-21T11:59:19.558970+0000","last_clean":"2024-09-21T11:59:19.558970+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:59:19.558970+0000","last_undegraded":"2024-09-21T11:59:19.558970+0000","last_fullsized":"2024-09-21T11:59:19.558970+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:57:50.976029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:50.976029+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":545,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558766+0000","last_change":"2024-09-21T11:56:58.434536+0000","last_active":"2024-09-21T11:59:19.558766+0000","last_peered":"2024-09-21T11:59:19.558766+0000","last_clean":"2024-09-21T11:59:19.558766+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:59:19.558766+0000","last_undegraded":"2024-09-21T11:59:19.558766+0000","last_fullsized":"2024-09-21T11:59:19.558766+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.434464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.434464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":832,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834900+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:59:25.834900+0000","last_peered":"2024-09-21T11:59:25.834900+0000","last_clean":"2024-09-21T11:59:25.834900+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:59:25.834900+0000","last_undegraded":"2024-09-21T11:59:25.834900+0000","last_fullsized":"2024-09-21T11:59:25.834900+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:34.633118+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":537,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535093+0000","last_change":"2024-09-21T11:55:51.602665+0000","last_active":"2024-09-21T11:59:19.535093+0000","last_peered":"2024-09-21T11:59:19.535093+0000","last_clean":"2024-09-21T11:59:19.535093+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:59:19.535093+0000","last_undegraded":"2024-09-21T11:59:19.535093+0000","last_fullsized":"2024-09-21T11:59:19.535093+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.602587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.602587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":88,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589145+0000","last_change":"2024-09-21T11:57:53.922876+0000","last_active":"2024-09-21T11:59:19.589145+0000","last_peered":"2024-09-21T11:59:19.589145+0000","last_clean":"2024-09-21T11:59:19.589145+0000","last_became_active":"2024-09-21T11:57:45.515242+0000","last_became_peered":"2024-09-21T11:57:45.515242+0000","last_unstale":"2024-09-21T11:59:19.589145+0000","last_undegraded":"2024-09-21T11:59:19.589145+0000","last_fullsized":"2024-09-21T11:59:19.589145+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:53.922795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:53.922795+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:20.497165+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.b","version":"453'11978","reported_seq":12112,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:26.145308+0000","last_change":"2024-09-21T11:56:36.510297+0000","last_active":"2024-09-21T11:59:26.145308+0000","last_peered":"2024-09-21T11:59:26.145308+0000","last_clean":"2024-09-21T11:59:26.145308+0000","last_became_active":"2024-09-21T11:56:36.510018+0000","last_became_peered":"2024-09-21T11:56:36.510018+0000","last_unstale":"2024-09-21T11:59:26.145308+0000","last_undegraded":"2024-09-21T11:59:26.145308+0000","last_fullsized":"2024-09-21T11:59:26.145308+0000","mapping_epoch":392,"log_start":"426'7321","ondisk_log_start":"426'7321","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4657,"log_dups_size":0,"ondisk_log_size":4657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205520896,"num_objects":3136,"num_object_clones":0,"num_object_copies":9408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11978,"num_write_kb":483648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":709,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535604+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:59:19.535604+0000","last_peered":"2024-09-21T11:59:19.535604+0000","last_clean":"2024-09-21T11:59:19.535604+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:59:19.535604+0000","last_undegraded":"2024-09-21T11:59:19.535604+0000","last_fullsized":"2024-09-21T11:59:19.535604+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":371,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.559129+0000","last_change":"2024-09-21T11:57:51.934799+0000","last_active":"2024-09-21T11:59:19.559129+0000","last_peered":"2024-09-21T11:59:19.559129+0000","last_clean":"2024-09-21T11:59:19.559129+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:59:19.559129+0000","last_undegraded":"2024-09-21T11:59:19.559129+0000","last_fullsized":"2024-09-21T11:59:19.559129+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:51.934563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.934563+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":876,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.590083+0000","last_change":"2024-09-21T11:58:59.855226+0000","last_active":"2024-09-21T11:59:19.590083+0000","last_peered":"2024-09-21T11:59:19.590083+0000","last_clean":"2024-09-21T11:59:19.590083+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:59:19.590083+0000","last_undegraded":"2024-09-21T11:59:19.590083+0000","last_fullsized":"2024-09-21T11:59:19.590083+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:58:59.855148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:58:59.855148+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:09.486850+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":79,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.536123+0000","last_change":"2024-09-21T11:58:17.527987+0000","last_active":"2024-09-21T11:59:19.536123+0000","last_peered":"2024-09-21T11:59:19.536123+0000","last_clean":"2024-09-21T11:59:19.536123+0000","last_became_active":"2024-09-21T11:58:17.527653+0000","last_became_peered":"2024-09-21T11:58:17.527653+0000","last_unstale":"2024-09-21T11:59:19.536123+0000","last_undegraded":"2024-09-21T11:59:19.536123+0000","last_fullsized":"2024-09-21T11:59:19.536123+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":828,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.833235+0000","last_change":"2024-09-21T11:57:54.859726+0000","last_active":"2024-09-21T11:59:25.833235+0000","last_peered":"2024-09-21T11:59:25.833235+0000","last_clean":"2024-09-21T11:59:25.833235+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:59:25.833235+0000","last_undegraded":"2024-09-21T11:59:25.833235+0000","last_fullsized":"2024-09-21T11:59:25.833235+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:54.859668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.859668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":968,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587145+0000","last_change":"2024-09-21T11:58:17.529673+0000","last_active":"2024-09-21T11:59:19.587145+0000","last_peered":"2024-09-21T11:59:19.587145+0000","last_clean":"2024-09-21T11:59:19.587145+0000","last_became_active":"2024-09-21T11:58:17.529304+0000","last_became_peered":"2024-09-21T11:58:17.529304+0000","last_unstale":"2024-09-21T11:59:19.587145+0000","last_undegraded":"2024-09-21T11:59:19.587145+0000","last_fullsized":"2024-09-21T11:59:19.587145+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:50.548003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:50.548003+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"453'12045","reported_seq":12179,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:26.120693+0000","last_change":"2024-09-21T11:56:36.509094+0000","last_active":"2024-09-21T11:59:26.120693+0000","last_peered":"2024-09-21T11:59:26.120693+0000","last_clean":"2024-09-21T11:59:26.120693+0000","last_became_active":"2024-09-21T11:56:36.508840+0000","last_became_peered":"2024-09-21T11:56:36.508840+0000","last_unstale":"2024-09-21T11:59:26.120693+0000","last_undegraded":"2024-09-21T11:59:26.120693+0000","last_fullsized":"2024-09-21T11:59:26.120693+0000","mapping_epoch":392,"log_start":"426'7395","ondisk_log_start":"426'7395","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4650,"log_dups_size":0,"ondisk_log_size":4650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":208470016,"num_objects":3181,"num_object_clones":0,"num_object_copies":9543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12045,"num_write_kb":487232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":534,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.588102+0000","last_change":"2024-09-21T11:59:04.130363+0000","last_active":"2024-09-21T11:59:19.588102+0000","last_peered":"2024-09-21T11:59:19.588102+0000","last_clean":"2024-09-21T11:59:19.588102+0000","last_became_active":"2024-09-21T11:57:58.530507+0000","last_became_peered":"2024-09-21T11:57:58.530507+0000","last_unstale":"2024-09-21T11:59:19.588102+0000","last_undegraded":"2024-09-21T11:59:19.588102+0000","last_fullsized":"2024-09-21T11:59:19.588102+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:59:04.130113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:04.130113+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:17.734192+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":550,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.534994+0000","last_change":"2024-09-21T11:57:43.583962+0000","last_active":"2024-09-21T11:59:19.534994+0000","last_peered":"2024-09-21T11:59:19.534994+0000","last_clean":"2024-09-21T11:59:19.534994+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:59:19.534994+0000","last_undegraded":"2024-09-21T11:59:19.534994+0000","last_fullsized":"2024-09-21T11:59:19.534994+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:43.583736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.583736+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":870,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587819+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:59:19.587819+0000","last_peered":"2024-09-21T11:59:19.587819+0000","last_clean":"2024-09-21T11:59:19.587819+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:59:19.587819+0000","last_undegraded":"2024-09-21T11:59:19.587819+0000","last_fullsized":"2024-09-21T11:59:19.587819+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.044139+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":852,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.588330+0000","last_change":"2024-09-21T11:59:09.091725+0000","last_active":"2024-09-21T11:59:19.588330+0000","last_peered":"2024-09-21T11:59:19.588330+0000","last_clean":"2024-09-21T11:59:19.588330+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:59:19.588330+0000","last_undegraded":"2024-09-21T11:59:19.588330+0000","last_fullsized":"2024-09-21T11:59:19.588330+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:09.091520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:59:09.091520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:10.832593+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":856,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587366+0000","last_change":"2024-09-21T11:58:30.226389+0000","last_active":"2024-09-21T11:59:19.587366+0000","last_peered":"2024-09-21T11:59:19.587366+0000","last_clean":"2024-09-21T11:59:19.587366+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:59:19.587366+0000","last_undegraded":"2024-09-21T11:59:19.587366+0000","last_fullsized":"2024-09-21T11:59:19.587366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:30.226345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:30.226345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:50.129480+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":840,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558657+0000","last_change":"2024-09-21T11:58:30.794254+0000","last_active":"2024-09-21T11:59:19.558657+0000","last_peered":"2024-09-21T11:59:19.558657+0000","last_clean":"2024-09-21T11:59:19.558657+0000","last_became_active":"2024-09-21T11:58:30.793717+0000","last_became_peered":"2024-09-21T11:58:30.793717+0000","last_unstale":"2024-09-21T11:59:19.558657+0000","last_undegraded":"2024-09-21T11:59:19.558657+0000","last_fullsized":"2024-09-21T11:59:19.558657+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":436,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:48.627711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.627711+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"453'12116","reported_seq":22177,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:26.169559+0000","last_change":"2024-09-21T11:59:19.581217+0000","last_active":"2024-09-21T11:59:26.169559+0000","last_peered":"2024-09-21T11:59:26.169559+0000","last_clean":"2024-09-21T11:59:26.169559+0000","last_became_active":"2024-09-21T11:59:19.559837+0000","last_became_peered":"2024-09-21T11:59:19.559837+0000","last_unstale":"2024-09-21T11:59:26.169559+0000","last_undegraded":"2024-09-21T11:59:26.169559+0000","last_fullsized":"2024-09-21T11:59:26.169559+0000","mapping_epoch":450,"log_start":"425'7445","ondisk_log_start":"425'7445","created":392,"last_epoch_clean":451,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4671,"log_dups_size":0,"ondisk_log_size":4671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":211681280,"num_objects":3230,"num_object_clones":0,"num_object_copies":9690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12116,"num_write_kb":491072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4953,"num_bytes_recovered":324599808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":86,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.536439+0000","last_change":"2024-09-21T11:57:58.545640+0000","last_active":"2024-09-21T11:59:19.536439+0000","last_peered":"2024-09-21T11:59:19.536439+0000","last_clean":"2024-09-21T11:59:19.536439+0000","last_became_active":"2024-09-21T11:57:58.527381+0000","last_became_peered":"2024-09-21T11:57:58.527381+0000","last_unstale":"2024-09-21T11:59:19.536439+0000","last_undegraded":"2024-09-21T11:59:19.536439+0000","last_fullsized":"2024-09-21T11:59:19.536439+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:57:54.679272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.679272+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":882,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.536240+0000","last_change":"2024-09-21T11:58:17.528448+0000","last_active":"2024-09-21T11:59:19.536240+0000","last_peered":"2024-09-21T11:59:19.536240+0000","last_clean":"2024-09-21T11:59:19.536240+0000","last_became_active":"2024-09-21T11:58:17.528154+0000","last_became_peered":"2024-09-21T11:58:17.528154+0000","last_unstale":"2024-09-21T11:59:19.536240+0000","last_undegraded":"2024-09-21T11:59:19.536240+0000","last_fullsized":"2024-09-21T11:59:19.536240+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:01.083437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.083437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":2046,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.725305+0000","last_change":"2024-09-21T11:56:09.604887+0000","last_active":"2024-09-21T11:59:19.725305+0000","last_peered":"2024-09-21T11:59:19.725305+0000","last_clean":"2024-09-21T11:59:19.725305+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:59:19.725305+0000","last_undegraded":"2024-09-21T11:59:19.725305+0000","last_fullsized":"2024-09-21T11:59:19.725305+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:56:09.604808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.604808+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.102017+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1151,"num_read_kb":1146,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":865,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.590448+0000","last_change":"2024-09-21T11:59:11.972760+0000","last_active":"2024-09-21T11:59:19.590448+0000","last_peered":"2024-09-21T11:59:19.590448+0000","last_clean":"2024-09-21T11:59:19.590448+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:59:19.590448+0000","last_undegraded":"2024-09-21T11:59:19.590448+0000","last_fullsized":"2024-09-21T11:59:19.590448+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:11.972680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:59:11.972680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:33.037868+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":537,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535545+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:59:19.535545+0000","last_peered":"2024-09-21T11:59:19.535545+0000","last_clean":"2024-09-21T11:59:19.535545+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:59:19.535545+0000","last_undegraded":"2024-09-21T11:59:19.535545+0000","last_fullsized":"2024-09-21T11:59:19.535545+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"453'11976","reported_seq":12110,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:26.156545+0000","last_change":"2024-09-21T11:56:36.509270+0000","last_active":"2024-09-21T11:59:26.156545+0000","last_peered":"2024-09-21T11:59:26.156545+0000","last_clean":"2024-09-21T11:59:26.156545+0000","last_became_active":"2024-09-21T11:56:36.508533+0000","last_became_peered":"2024-09-21T11:56:36.508533+0000","last_unstale":"2024-09-21T11:59:26.156545+0000","last_undegraded":"2024-09-21T11:59:26.156545+0000","last_fullsized":"2024-09-21T11:59:26.156545+0000","mapping_epoch":392,"log_start":"426'7322","ondisk_log_start":"426'7322","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4654,"log_dups_size":0,"ondisk_log_size":4654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":203948032,"num_objects":3112,"num_object_clones":0,"num_object_copies":9336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11976,"num_write_kb":482816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":1134,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835483+0000","last_change":"2024-09-21T11:59:05.905840+0000","last_active":"2024-09-21T11:59:25.835483+0000","last_peered":"2024-09-21T11:59:25.835483+0000","last_clean":"2024-09-21T11:59:25.835483+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:59:25.835483+0000","last_undegraded":"2024-09-21T11:59:25.835483+0000","last_fullsized":"2024-09-21T11:59:25.835483+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:59:05.905765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:05.905765+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:34.307948+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":857,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834442+0000","last_change":"2024-09-21T11:57:14.756298+0000","last_active":"2024-09-21T11:59:25.834442+0000","last_peered":"2024-09-21T11:59:25.834442+0000","last_clean":"2024-09-21T11:59:25.834442+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:59:25.834442+0000","last_undegraded":"2024-09-21T11:59:25.834442+0000","last_fullsized":"2024-09-21T11:59:25.834442+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:14.756227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:14.756227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":971,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587050+0000","last_change":"2024-09-21T11:58:54.745261+0000","last_active":"2024-09-21T11:59:19.587050+0000","last_peered":"2024-09-21T11:59:19.587050+0000","last_clean":"2024-09-21T11:59:19.587050+0000","last_became_active":"2024-09-21T11:57:58.528825+0000","last_became_peered":"2024-09-21T11:57:58.528825+0000","last_unstale":"2024-09-21T11:59:19.587050+0000","last_undegraded":"2024-09-21T11:59:19.587050+0000","last_fullsized":"2024-09-21T11:59:19.587050+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:58:54.745202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:58:54.745202+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:08.610072+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":844,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587414+0000","last_change":"2024-09-21T11:58:28.274182+0000","last_active":"2024-09-21T11:59:19.587414+0000","last_peered":"2024-09-21T11:59:19.587414+0000","last_clean":"2024-09-21T11:59:19.587414+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:59:19.587414+0000","last_undegraded":"2024-09-21T11:59:19.587414+0000","last_fullsized":"2024-09-21T11:59:19.587414+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:28.274120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.274120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.924347+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":855,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.590358+0000","last_change":"2024-09-21T11:57:05.904684+0000","last_active":"2024-09-21T11:59:19.590358+0000","last_peered":"2024-09-21T11:59:19.590358+0000","last_clean":"2024-09-21T11:59:19.590358+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:59:19.590358+0000","last_undegraded":"2024-09-21T11:59:19.590358+0000","last_fullsized":"2024-09-21T11:59:19.590358+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:05.904621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:05.904621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:46.758787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"453'12024","reported_seq":12206,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.914063+0000","last_change":"2024-09-21T11:59:16.343176+0000","last_active":"2024-09-21T11:59:25.914063+0000","last_peered":"2024-09-21T11:59:25.914063+0000","last_clean":"2024-09-21T11:59:25.914063+0000","last_became_active":"2024-09-21T11:56:36.507849+0000","last_became_peered":"2024-09-21T11:56:36.507849+0000","last_unstale":"2024-09-21T11:59:25.914063+0000","last_undegraded":"2024-09-21T11:59:25.914063+0000","last_fullsized":"2024-09-21T11:59:25.914063+0000","mapping_epoch":392,"log_start":"418'6604","ondisk_log_start":"418'6604","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'11497","last_scrub_stamp":"2024-09-21T11:59:16.342973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:16.342973+0000","objects_scrubbed":3687,"log_size":5420,"log_dups_size":0,"ondisk_log_size":5420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:18.053609+0000","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":203292672,"num_objects":3102,"num_object_clones":0,"num_object_copies":9306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12024,"num_write_kb":484032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":881,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835352+0000","last_change":"2024-09-21T11:58:56.825792+0000","last_active":"2024-09-21T11:59:25.835352+0000","last_peered":"2024-09-21T11:59:25.835352+0000","last_clean":"2024-09-21T11:59:25.835352+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:59:25.835352+0000","last_undegraded":"2024-09-21T11:59:25.835352+0000","last_fullsized":"2024-09-21T11:59:25.835352+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:56.825707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:56.825707+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:07.271040+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":360,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587721+0000","last_change":"2024-09-21T11:58:17.529749+0000","last_active":"2024-09-21T11:59:19.587721+0000","last_peered":"2024-09-21T11:59:19.587721+0000","last_clean":"2024-09-21T11:59:19.587721+0000","last_became_active":"2024-09-21T11:58:17.529395+0000","last_became_peered":"2024-09-21T11:58:17.529395+0000","last_unstale":"2024-09-21T11:59:19.587721+0000","last_undegraded":"2024-09-21T11:59:19.587721+0000","last_fullsized":"2024-09-21T11:59:19.587721+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:05.663102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.663102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:47.080734+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":857,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.590226+0000","last_change":"2024-09-21T11:58:29.807560+0000","last_active":"2024-09-21T11:59:19.590226+0000","last_peered":"2024-09-21T11:59:19.590226+0000","last_clean":"2024-09-21T11:59:19.590226+0000","last_became_active":"2024-09-21T11:58:17.527945+0000","last_became_peered":"2024-09-21T11:58:17.527945+0000","last_unstale":"2024-09-21T11:59:19.590226+0000","last_undegraded":"2024-09-21T11:59:19.590226+0000","last_fullsized":"2024-09-21T11:59:19.590226+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:29.807504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:29.807504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:58.143758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":850,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834544+0000","last_change":"2024-09-21T11:56:28.556475+0000","last_active":"2024-09-21T11:59:25.834544+0000","last_peered":"2024-09-21T11:59:25.834544+0000","last_clean":"2024-09-21T11:59:25.834544+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:59:25.834544+0000","last_undegraded":"2024-09-21T11:59:25.834544+0000","last_fullsized":"2024-09-21T11:59:25.834544+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.556423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.556423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":95,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.590230+0000","last_change":"2024-09-21T11:57:54.931806+0000","last_active":"2024-09-21T11:59:19.590230+0000","last_peered":"2024-09-21T11:59:19.590230+0000","last_clean":"2024-09-21T11:59:19.590230+0000","last_became_active":"2024-09-21T11:57:45.534494+0000","last_became_peered":"2024-09-21T11:57:45.534494+0000","last_unstale":"2024-09-21T11:59:19.590230+0000","last_undegraded":"2024-09-21T11:59:19.590230+0000","last_fullsized":"2024-09-21T11:59:19.590230+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:54.931704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.931704+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:23.002342+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2","version":"451'11649","reported_seq":11833,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:21.909715+0000","last_change":"2024-09-21T11:59:01.566946+0000","last_active":"2024-09-21T11:59:21.909715+0000","last_peered":"2024-09-21T11:59:21.909715+0000","last_clean":"2024-09-21T11:59:21.909715+0000","last_became_active":"2024-09-21T11:56:36.511642+0000","last_became_peered":"2024-09-21T11:56:36.511642+0000","last_unstale":"2024-09-21T11:59:21.909715+0000","last_undegraded":"2024-09-21T11:59:21.909715+0000","last_fullsized":"2024-09-21T11:59:21.909715+0000","mapping_epoch":392,"log_start":"410'5497","ondisk_log_start":"410'5497","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'10535","last_scrub_stamp":"2024-09-21T11:59:01.566731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:01.566731+0000","objects_scrubbed":4558,"log_size":6152,"log_dups_size":0,"ondisk_log_size":6152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:17.985934+0000","scrub_duration":2440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221577216,"num_objects":3381,"num_object_clones":0,"num_object_copies":10143,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11649,"num_write_kb":480960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":974,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.588070+0000","last_change":"2024-09-21T11:58:17.555274+0000","last_active":"2024-09-21T11:59:19.588070+0000","last_peered":"2024-09-21T11:59:19.588070+0000","last_clean":"2024-09-21T11:59:19.588070+0000","last_became_active":"2024-09-21T11:58:17.527029+0000","last_became_peered":"2024-09-21T11:58:17.527029+0000","last_unstale":"2024-09-21T11:59:19.588070+0000","last_undegraded":"2024-09-21T11:59:19.588070+0000","last_fullsized":"2024-09-21T11:59:19.588070+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:56:03.667750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:03.667750+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:37.084768+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":885,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835289+0000","last_change":"2024-09-21T11:59:02.898115+0000","last_active":"2024-09-21T11:59:25.835289+0000","last_peered":"2024-09-21T11:59:25.835289+0000","last_clean":"2024-09-21T11:59:25.835289+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:59:25.835289+0000","last_undegraded":"2024-09-21T11:59:25.835289+0000","last_fullsized":"2024-09-21T11:59:25.835289+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:02.898000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:59:02.898000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:26.416721+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":545,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535467+0000","last_change":"2024-09-21T11:57:42.599025+0000","last_active":"2024-09-21T11:59:19.535467+0000","last_peered":"2024-09-21T11:59:19.535467+0000","last_clean":"2024-09-21T11:59:19.535467+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:59:19.535467+0000","last_undegraded":"2024-09-21T11:59:19.535467+0000","last_fullsized":"2024-09-21T11:59:19.535467+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:42.598710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:42.598710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":856,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.588952+0000","last_change":"2024-09-21T11:59:16.184608+0000","last_active":"2024-09-21T11:59:19.588952+0000","last_peered":"2024-09-21T11:59:19.588952+0000","last_clean":"2024-09-21T11:59:19.588952+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:59:19.588952+0000","last_undegraded":"2024-09-21T11:59:19.588952+0000","last_fullsized":"2024-09-21T11:59:19.588952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:16.184540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:16.184540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:21.327851+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":969,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587073+0000","last_change":"2024-09-21T11:59:10.608778+0000","last_active":"2024-09-21T11:59:19.587073+0000","last_peered":"2024-09-21T11:59:19.587073+0000","last_clean":"2024-09-21T11:59:19.587073+0000","last_became_active":"2024-09-21T11:57:45.516610+0000","last_became_peered":"2024-09-21T11:57:45.516610+0000","last_unstale":"2024-09-21T11:59:19.587073+0000","last_undegraded":"2024-09-21T11:59:19.587073+0000","last_fullsized":"2024-09-21T11:59:19.587073+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:59:10.608731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:59:10.608731+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:38.999852+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3","version":"453'11937","reported_seq":12151,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.912599+0000","last_change":"2024-09-21T11:58:22.114392+0000","last_active":"2024-09-21T11:59:25.912599+0000","last_peered":"2024-09-21T11:59:25.912599+0000","last_clean":"2024-09-21T11:59:25.912599+0000","last_became_active":"2024-09-21T11:56:36.504516+0000","last_became_peered":"2024-09-21T11:56:36.504516+0000","last_unstale":"2024-09-21T11:59:25.912599+0000","last_undegraded":"2024-09-21T11:59:25.912599+0000","last_fullsized":"2024-09-21T11:59:25.912599+0000","mapping_epoch":392,"log_start":"418'6495","ondisk_log_start":"418'6495","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"432'8216","last_scrub_stamp":"2024-09-21T11:58:22.114283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:58:22.114283+0000","objects_scrubbed":7071,"log_size":5442,"log_dups_size":0,"ondisk_log_size":5442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:43.111443+0000","scrub_duration":2344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":215547904,"num_objects":3289,"num_object_clones":0,"num_object_copies":9867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11937,"num_write_kb":487232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":834,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834937+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:59:25.834937+0000","last_peered":"2024-09-21T11:59:25.834937+0000","last_clean":"2024-09-21T11:59:25.834937+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:59:25.834937+0000","last_undegraded":"2024-09-21T11:59:25.834937+0000","last_fullsized":"2024-09-21T11:59:25.834937+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:20.087582+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":854,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.833233+0000","last_change":"2024-09-21T11:57:58.535943+0000","last_active":"2024-09-21T11:59:25.833233+0000","last_peered":"2024-09-21T11:59:25.833233+0000","last_clean":"2024-09-21T11:59:25.833233+0000","last_became_active":"2024-09-21T11:57:58.535642+0000","last_became_peered":"2024-09-21T11:57:58.535642+0000","last_unstale":"2024-09-21T11:59:25.833233+0000","last_undegraded":"2024-09-21T11:59:25.833233+0000","last_fullsized":"2024-09-21T11:59:25.833233+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:09.486386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.486386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1093,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.588081+0000","last_change":"2024-09-21T11:56:08.680971+0000","last_active":"2024-09-21T11:59:19.588081+0000","last_peered":"2024-09-21T11:59:19.588081+0000","last_clean":"2024-09-21T11:59:19.588081+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:59:19.588081+0000","last_undegraded":"2024-09-21T11:59:19.588081+0000","last_fullsized":"2024-09-21T11:59:19.588081+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:56:08.680766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.680766+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:46.070618+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":679,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.559566+0000","last_change":"2024-09-21T11:58:17.532859+0000","last_active":"2024-09-21T11:59:19.559566+0000","last_peered":"2024-09-21T11:59:19.559566+0000","last_clean":"2024-09-21T11:59:19.559566+0000","last_became_active":"2024-09-21T11:58:17.531268+0000","last_became_peered":"2024-09-21T11:58:17.531268+0000","last_unstale":"2024-09-21T11:59:19.559566+0000","last_undegraded":"2024-09-21T11:59:19.559566+0000","last_fullsized":"2024-09-21T11:59:19.559566+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:51.665137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.665137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"451'11573","reported_seq":11750,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:21.322339+0000","last_change":"2024-09-21T11:59:13.679316+0000","last_active":"2024-09-21T11:59:21.322339+0000","last_peered":"2024-09-21T11:59:21.322339+0000","last_clean":"2024-09-21T11:59:21.322339+0000","last_became_active":"2024-09-21T11:56:36.508299+0000","last_became_peered":"2024-09-21T11:56:36.508299+0000","last_unstale":"2024-09-21T11:59:21.322339+0000","last_undegraded":"2024-09-21T11:59:21.322339+0000","last_fullsized":"2024-09-21T11:59:21.322339+0000","mapping_epoch":392,"log_start":"419'6715","ondisk_log_start":"419'6715","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'11214","last_scrub_stamp":"2024-09-21T11:59:13.679136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:13.679136+0000","objects_scrubbed":3897,"log_size":4858,"log_dups_size":0,"ondisk_log_size":4858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:20.950833+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":229834752,"num_objects":3507,"num_object_clones":0,"num_object_copies":10521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11573,"num_write_kb":482560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":890,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558840+0000","last_change":"2024-09-21T11:56:48.169371+0000","last_active":"2024-09-21T11:59:19.558840+0000","last_peered":"2024-09-21T11:59:19.558840+0000","last_clean":"2024-09-21T11:59:19.558840+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:59:19.558840+0000","last_undegraded":"2024-09-21T11:59:19.558840+0000","last_fullsized":"2024-09-21T11:59:19.558840+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:48.168795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.168795+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":883,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589891+0000","last_change":"2024-09-21T11:59:12.932516+0000","last_active":"2024-09-21T11:59:19.589891+0000","last_peered":"2024-09-21T11:59:19.589891+0000","last_clean":"2024-09-21T11:59:19.589891+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:59:19.589891+0000","last_undegraded":"2024-09-21T11:59:19.589891+0000","last_fullsized":"2024-09-21T11:59:19.589891+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:12.932440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:59:12.932440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:25.638974+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":850,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587832+0000","last_change":"2024-09-21T11:58:21.147985+0000","last_active":"2024-09-21T11:59:19.587832+0000","last_peered":"2024-09-21T11:59:19.587832+0000","last_clean":"2024-09-21T11:59:19.587832+0000","last_became_active":"2024-09-21T11:58:17.525791+0000","last_became_peered":"2024-09-21T11:58:17.525791+0000","last_unstale":"2024-09-21T11:59:19.587832+0000","last_undegraded":"2024-09-21T11:59:19.587832+0000","last_fullsized":"2024-09-21T11:59:19.587832+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:21.147942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:21.147942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:41.780615+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":72,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.536148+0000","last_change":"2024-09-21T11:57:58.529616+0000","last_active":"2024-09-21T11:59:19.536148+0000","last_peered":"2024-09-21T11:59:19.536148+0000","last_clean":"2024-09-21T11:59:19.536148+0000","last_became_active":"2024-09-21T11:57:58.529278+0000","last_became_peered":"2024-09-21T11:57:58.529278+0000","last_unstale":"2024-09-21T11:59:19.536148+0000","last_undegraded":"2024-09-21T11:59:19.536148+0000","last_fullsized":"2024-09-21T11:59:19.536148+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:35.532577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:35.532577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"451'11439","reported_seq":11618,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:21.329558+0000","last_change":"2024-09-21T11:59:08.338604+0000","last_active":"2024-09-21T11:59:21.329558+0000","last_peered":"2024-09-21T11:59:21.329558+0000","last_clean":"2024-09-21T11:59:21.329558+0000","last_became_active":"2024-09-21T11:56:36.506694+0000","last_became_peered":"2024-09-21T11:56:36.506694+0000","last_unstale":"2024-09-21T11:59:21.329558+0000","last_undegraded":"2024-09-21T11:59:21.329558+0000","last_fullsized":"2024-09-21T11:59:21.329558+0000","mapping_epoch":392,"log_start":"418'6619","ondisk_log_start":"418'6619","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"447'10791","last_scrub_stamp":"2024-09-21T11:59:08.338518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.338518+0000","objects_scrubbed":4166,"log_size":4820,"log_dups_size":0,"ondisk_log_size":4820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:31.958551+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":229572608,"num_objects":3503,"num_object_clones":0,"num_object_copies":10509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11439,"num_write_kb":478144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":960,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835026+0000","last_change":"2024-09-21T11:58:29.624654+0000","last_active":"2024-09-21T11:59:25.835026+0000","last_peered":"2024-09-21T11:59:25.835026+0000","last_clean":"2024-09-21T11:59:25.835026+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:59:25.835026+0000","last_undegraded":"2024-09-21T11:59:25.835026+0000","last_fullsized":"2024-09-21T11:59:25.835026+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:58:29.624549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:29.624549+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:48.198468+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":851,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587635+0000","last_change":"2024-09-21T11:59:17.687953+0000","last_active":"2024-09-21T11:59:19.587635+0000","last_peered":"2024-09-21T11:59:19.587635+0000","last_clean":"2024-09-21T11:59:19.587635+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:59:19.587635+0000","last_undegraded":"2024-09-21T11:59:19.587635+0000","last_fullsized":"2024-09-21T11:59:19.587635+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:17.687904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:59:17.687904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:21.256703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":852,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587800+0000","last_change":"2024-09-21T11:58:28.041091+0000","last_active":"2024-09-21T11:59:19.587800+0000","last_peered":"2024-09-21T11:59:19.587800+0000","last_clean":"2024-09-21T11:59:19.587800+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:59:19.587800+0000","last_undegraded":"2024-09-21T11:59:19.587800+0000","last_fullsized":"2024-09-21T11:59:19.587800+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:28.041026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.041026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:52.616083+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":855,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587038+0000","last_change":"2024-09-21T11:59:08.676283+0000","last_active":"2024-09-21T11:59:19.587038+0000","last_peered":"2024-09-21T11:59:19.587038+0000","last_clean":"2024-09-21T11:59:19.587038+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:59:19.587038+0000","last_undegraded":"2024-09-21T11:59:19.587038+0000","last_fullsized":"2024-09-21T11:59:19.587038+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:08.676235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.676235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:13.547786+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"451'11774","reported_seq":24677,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:22.622643+0000","last_change":"2024-09-21T11:59:02.700721+0000","last_active":"2024-09-21T11:59:22.622643+0000","last_peered":"2024-09-21T11:59:22.622643+0000","last_clean":"2024-09-21T11:59:22.622643+0000","last_became_active":"2024-09-21T11:59:02.673387+0000","last_became_peered":"2024-09-21T11:59:02.673387+0000","last_unstale":"2024-09-21T11:59:22.622643+0000","last_undegraded":"2024-09-21T11:59:22.622643+0000","last_fullsized":"2024-09-21T11:59:22.622643+0000","mapping_epoch":444,"log_start":"418'6415","ondisk_log_start":"418'6415","created":392,"last_epoch_clean":445,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223608832,"num_objects":3412,"num_object_clones":0,"num_object_copies":10236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11774,"num_write_kb":485952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6378,"num_bytes_recovered":417988608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":870,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535299+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:59:19.535299+0000","last_peered":"2024-09-21T11:59:19.535299+0000","last_clean":"2024-09-21T11:59:19.535299+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:59:19.535299+0000","last_undegraded":"2024-09-21T11:59:19.535299+0000","last_fullsized":"2024-09-21T11:59:19.535299+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":910,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587893+0000","last_change":"2024-09-21T11:58:18.420036+0000","last_active":"2024-09-21T11:59:19.587893+0000","last_peered":"2024-09-21T11:59:19.587893+0000","last_clean":"2024-09-21T11:59:19.587893+0000","last_became_active":"2024-09-21T11:58:17.530676+0000","last_became_peered":"2024-09-21T11:58:17.530676+0000","last_unstale":"2024-09-21T11:59:19.587893+0000","last_undegraded":"2024-09-21T11:59:19.587893+0000","last_fullsized":"2024-09-21T11:59:19.587893+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:18.419990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:58:18.419990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:25.445566+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":858,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587539+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:59:19.587539+0000","last_peered":"2024-09-21T11:59:19.587539+0000","last_clean":"2024-09-21T11:59:19.587539+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:59:19.587539+0000","last_undegraded":"2024-09-21T11:59:19.587539+0000","last_fullsized":"2024-09-21T11:59:19.587539+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.052833+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":850,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.832910+0000","last_change":"2024-09-21T11:56:02.541452+0000","last_active":"2024-09-21T11:59:25.832910+0000","last_peered":"2024-09-21T11:59:25.832910+0000","last_clean":"2024-09-21T11:59:25.832910+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:59:25.832910+0000","last_undegraded":"2024-09-21T11:59:25.832910+0000","last_fullsized":"2024-09-21T11:59:25.832910+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.541232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.541232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"451'11422","reported_seq":11600,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:21.493410+0000","last_change":"2024-09-21T11:59:11.044768+0000","last_active":"2024-09-21T11:59:21.493410+0000","last_peered":"2024-09-21T11:59:21.493410+0000","last_clean":"2024-09-21T11:59:21.493410+0000","last_became_active":"2024-09-21T11:56:36.623765+0000","last_became_peered":"2024-09-21T11:56:36.623765+0000","last_unstale":"2024-09-21T11:59:21.493410+0000","last_undegraded":"2024-09-21T11:59:21.493410+0000","last_fullsized":"2024-09-21T11:59:21.493410+0000","mapping_epoch":392,"log_start":"418'6199","ondisk_log_start":"418'6199","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'10886","last_scrub_stamp":"2024-09-21T11:59:11.044452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:11.044452+0000","objects_scrubbed":3980,"log_size":5223,"log_dups_size":0,"ondisk_log_size":5223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:23.581441+0000","scrub_duration":1112,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":223739904,"num_objects":3414,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11422,"num_write_kb":474752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":898,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.590367+0000","last_change":"2024-09-21T11:59:08.919308+0000","last_active":"2024-09-21T11:59:19.590367+0000","last_peered":"2024-09-21T11:59:19.590367+0000","last_clean":"2024-09-21T11:59:19.590367+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:59:19.590367+0000","last_undegraded":"2024-09-21T11:59:19.590367+0000","last_fullsized":"2024-09-21T11:59:19.590367+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:59:08.919232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.919232+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:19.454285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":905,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587836+0000","last_change":"2024-09-21T11:57:58.539544+0000","last_active":"2024-09-21T11:59:19.587836+0000","last_peered":"2024-09-21T11:59:19.587836+0000","last_clean":"2024-09-21T11:59:19.587836+0000","last_became_active":"2024-09-21T11:57:58.528814+0000","last_became_peered":"2024-09-21T11:57:58.528814+0000","last_unstale":"2024-09-21T11:59:19.587836+0000","last_undegraded":"2024-09-21T11:59:19.587836+0000","last_fullsized":"2024-09-21T11:59:19.587836+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:57:01.221960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.221960+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.207576+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1092,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587048+0000","last_change":"2024-09-21T11:57:10.282180+0000","last_active":"2024-09-21T11:59:19.587048+0000","last_peered":"2024-09-21T11:59:19.587048+0000","last_clean":"2024-09-21T11:59:19.587048+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:59:19.587048+0000","last_undegraded":"2024-09-21T11:59:19.587048+0000","last_fullsized":"2024-09-21T11:59:19.587048+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:10.282134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.282134+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:18.729887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":839,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589456+0000","last_change":"2024-09-21T11:59:13.897600+0000","last_active":"2024-09-21T11:59:19.589456+0000","last_peered":"2024-09-21T11:59:19.589456+0000","last_clean":"2024-09-21T11:59:19.589456+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:59:19.589456+0000","last_undegraded":"2024-09-21T11:59:19.589456+0000","last_fullsized":"2024-09-21T11:59:19.589456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:13.897540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:13.897540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:34.214481+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":885,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834980+0000","last_change":"2024-09-21T11:57:58.527427+0000","last_active":"2024-09-21T11:59:25.834980+0000","last_peered":"2024-09-21T11:59:25.834980+0000","last_clean":"2024-09-21T11:59:25.834980+0000","last_became_active":"2024-09-21T11:57:58.527156+0000","last_became_peered":"2024-09-21T11:57:58.527156+0000","last_unstale":"2024-09-21T11:59:25.834980+0000","last_undegraded":"2024-09-21T11:59:25.834980+0000","last_fullsized":"2024-09-21T11:59:25.834980+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:38.938666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:38.938666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:49.629819+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":884,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.833853+0000","last_change":"2024-09-21T11:56:29.528662+0000","last_active":"2024-09-21T11:59:25.833853+0000","last_peered":"2024-09-21T11:59:25.833853+0000","last_clean":"2024-09-21T11:59:25.833853+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:59:25.833853+0000","last_undegraded":"2024-09-21T11:59:25.833853+0000","last_fullsized":"2024-09-21T11:59:25.833853+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:56:29.528598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.528598+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":888,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589720+0000","last_change":"2024-09-21T11:58:56.841707+0000","last_active":"2024-09-21T11:59:19.589720+0000","last_peered":"2024-09-21T11:59:19.589720+0000","last_clean":"2024-09-21T11:59:19.589720+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:59:19.589720+0000","last_undegraded":"2024-09-21T11:59:19.589720+0000","last_fullsized":"2024-09-21T11:59:19.589720+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:56.841648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:56.841648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:06.233795+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":876,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587326+0000","last_change":"2024-09-21T11:58:20.347779+0000","last_active":"2024-09-21T11:59:19.587326+0000","last_peered":"2024-09-21T11:59:19.587326+0000","last_clean":"2024-09-21T11:59:19.587326+0000","last_became_active":"2024-09-21T11:58:17.529420+0000","last_became_peered":"2024-09-21T11:58:17.529420+0000","last_unstale":"2024-09-21T11:59:19.587326+0000","last_undegraded":"2024-09-21T11:59:19.587326+0000","last_fullsized":"2024-09-21T11:59:19.587326+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:20.347736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:20.347736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:47.925674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":872,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558635+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:59:19.558635+0000","last_peered":"2024-09-21T11:59:19.558635+0000","last_clean":"2024-09-21T11:59:19.558635+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:59:19.558635+0000","last_undegraded":"2024-09-21T11:59:19.558635+0000","last_fullsized":"2024-09-21T11:59:19.558635+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":835,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587204+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:59:19.587204+0000","last_peered":"2024-09-21T11:59:19.587204+0000","last_clean":"2024-09-21T11:59:19.587204+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:59:19.587204+0000","last_undegraded":"2024-09-21T11:59:19.587204+0000","last_fullsized":"2024-09-21T11:59:19.587204+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.174525+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":870,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.833623+0000","last_change":"2024-09-21T11:56:50.696344+0000","last_active":"2024-09-21T11:59:25.833623+0000","last_peered":"2024-09-21T11:59:25.833623+0000","last_clean":"2024-09-21T11:59:25.833623+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:59:25.833623+0000","last_undegraded":"2024-09-21T11:59:25.833623+0000","last_fullsized":"2024-09-21T11:59:25.833623+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.696253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.696253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":840,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.588090+0000","last_change":"2024-09-21T11:57:12.518014+0000","last_active":"2024-09-21T11:59:19.588090+0000","last_peered":"2024-09-21T11:59:19.588090+0000","last_clean":"2024-09-21T11:59:19.588090+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:59:19.588090+0000","last_undegraded":"2024-09-21T11:59:19.588090+0000","last_fullsized":"2024-09-21T11:59:19.588090+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:12.517953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:12.517953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:52.060114+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":879,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.592646+0000","last_change":"2024-09-21T11:59:07.953847+0000","last_active":"2024-09-21T11:59:19.592646+0000","last_peered":"2024-09-21T11:59:19.592646+0000","last_clean":"2024-09-21T11:59:19.592646+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:59:19.592646+0000","last_undegraded":"2024-09-21T11:59:19.592646+0000","last_fullsized":"2024-09-21T11:59:19.592646+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:59:07.953663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:07.953663+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:36.906230+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":867,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.557964+0000","last_change":"2024-09-21T11:57:49.993421+0000","last_active":"2024-09-21T11:59:19.557964+0000","last_peered":"2024-09-21T11:59:19.557964+0000","last_clean":"2024-09-21T11:59:19.557964+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:59:19.557964+0000","last_undegraded":"2024-09-21T11:59:19.557964+0000","last_fullsized":"2024-09-21T11:59:19.557964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:49.993339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:49.993339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1066,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834758+0000","last_change":"2024-09-21T11:55:51.317564+0000","last_active":"2024-09-21T11:59:25.834758+0000","last_peered":"2024-09-21T11:59:25.834758+0000","last_clean":"2024-09-21T11:59:25.834758+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:59:25.834758+0000","last_undegraded":"2024-09-21T11:59:25.834758+0000","last_fullsized":"2024-09-21T11:59:25.834758+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:51.317473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.317473+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":338,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.588604+0000","last_change":"2024-09-21T11:58:20.129924+0000","last_active":"2024-09-21T11:59:19.588604+0000","last_peered":"2024-09-21T11:59:19.588604+0000","last_clean":"2024-09-21T11:59:19.588604+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:59:19.588604+0000","last_undegraded":"2024-09-21T11:59:19.588604+0000","last_fullsized":"2024-09-21T11:59:19.588604+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:20.129854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:20.129854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:38.523985+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":904,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835037+0000","last_change":"2024-09-21T11:57:16.786845+0000","last_active":"2024-09-21T11:59:25.835037+0000","last_peered":"2024-09-21T11:59:25.835037+0000","last_clean":"2024-09-21T11:59:25.835037+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:59:25.835037+0000","last_undegraded":"2024-09-21T11:59:25.835037+0000","last_fullsized":"2024-09-21T11:59:25.835037+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:57:16.786756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:16.786756+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":851,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587961+0000","last_change":"2024-09-21T11:57:40.391494+0000","last_active":"2024-09-21T11:59:19.587961+0000","last_peered":"2024-09-21T11:59:19.587961+0000","last_clean":"2024-09-21T11:59:19.587961+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:59:19.587961+0000","last_undegraded":"2024-09-21T11:59:19.587961+0000","last_fullsized":"2024-09-21T11:59:19.587961+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:40.391409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:40.391409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":1114,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589458+0000","last_change":"2024-09-21T11:56:56.051487+0000","last_active":"2024-09-21T11:59:19.589458+0000","last_peered":"2024-09-21T11:59:19.589458+0000","last_clean":"2024-09-21T11:59:19.589458+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:59:19.589458+0000","last_undegraded":"2024-09-21T11:59:19.589458+0000","last_fullsized":"2024-09-21T11:59:19.589458+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:56.051300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:56.051300+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.828079+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":72,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.536002+0000","last_change":"2024-09-21T11:57:58.534490+0000","last_active":"2024-09-21T11:59:19.536002+0000","last_peered":"2024-09-21T11:59:19.536002+0000","last_clean":"2024-09-21T11:59:19.536002+0000","last_became_active":"2024-09-21T11:57:58.533810+0000","last_became_peered":"2024-09-21T11:57:58.533810+0000","last_unstale":"2024-09-21T11:59:19.536002+0000","last_undegraded":"2024-09-21T11:59:19.536002+0000","last_fullsized":"2024-09-21T11:59:19.536002+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.280698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.280698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":892,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835191+0000","last_change":"2024-09-21T11:58:28.658668+0000","last_active":"2024-09-21T11:59:25.835191+0000","last_peered":"2024-09-21T11:59:25.835191+0000","last_clean":"2024-09-21T11:59:25.835191+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:59:25.835191+0000","last_undegraded":"2024-09-21T11:59:25.835191+0000","last_fullsized":"2024-09-21T11:59:25.835191+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:28.658574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.658574+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:54.766743+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":354,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.586631+0000","last_change":"2024-09-21T11:58:56.695698+0000","last_active":"2024-09-21T11:59:19.586631+0000","last_peered":"2024-09-21T11:59:19.586631+0000","last_clean":"2024-09-21T11:59:19.586631+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:59:19.586631+0000","last_undegraded":"2024-09-21T11:59:19.586631+0000","last_fullsized":"2024-09-21T11:59:19.586631+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:56.695646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:58:56.695646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:21.948032+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":545,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535428+0000","last_change":"2024-09-21T11:56:00.441644+0000","last_active":"2024-09-21T11:59:19.535428+0000","last_peered":"2024-09-21T11:59:19.535428+0000","last_clean":"2024-09-21T11:59:19.535428+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:59:19.535428+0000","last_undegraded":"2024-09-21T11:59:19.535428+0000","last_fullsized":"2024-09-21T11:59:19.535428+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:00.441565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:00.441565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":842,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.832477+0000","last_change":"2024-09-21T11:57:58.531539+0000","last_active":"2024-09-21T11:59:25.832477+0000","last_peered":"2024-09-21T11:59:25.832477+0000","last_clean":"2024-09-21T11:59:25.832477+0000","last_became_active":"2024-09-21T11:57:58.531213+0000","last_became_peered":"2024-09-21T11:57:58.531213+0000","last_unstale":"2024-09-21T11:59:25.832477+0000","last_undegraded":"2024-09-21T11:59:25.832477+0000","last_fullsized":"2024-09-21T11:59:25.832477+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:01.434123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.434123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":556,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.559731+0000","last_change":"2024-09-21T11:55:59.419239+0000","last_active":"2024-09-21T11:59:19.559731+0000","last_peered":"2024-09-21T11:59:19.559731+0000","last_clean":"2024-09-21T11:59:19.559731+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:59:19.559731+0000","last_undegraded":"2024-09-21T11:59:19.559731+0000","last_fullsized":"2024-09-21T11:59:19.559731+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:59.419148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.419148+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":843,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587536+0000","last_change":"2024-09-21T11:57:17.492529+0000","last_active":"2024-09-21T11:59:19.587536+0000","last_peered":"2024-09-21T11:59:19.587536+0000","last_clean":"2024-09-21T11:59:19.587536+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:59:19.587536+0000","last_undegraded":"2024-09-21T11:59:19.587536+0000","last_fullsized":"2024-09-21T11:59:19.587536+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:17.492466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:17.492466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:37.151802+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":854,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.832990+0000","last_change":"2024-09-21T11:58:24.939301+0000","last_active":"2024-09-21T11:59:25.832990+0000","last_peered":"2024-09-21T11:59:25.832990+0000","last_clean":"2024-09-21T11:59:25.832990+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:59:25.832990+0000","last_undegraded":"2024-09-21T11:59:25.832990+0000","last_fullsized":"2024-09-21T11:59:25.832990+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:24.939230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:24.939230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.727906+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":878,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.535711+0000","last_change":"2024-09-21T11:57:37.071994+0000","last_active":"2024-09-21T11:59:19.535711+0000","last_peered":"2024-09-21T11:59:19.535711+0000","last_clean":"2024-09-21T11:59:19.535711+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:59:19.535711+0000","last_undegraded":"2024-09-21T11:59:19.535711+0000","last_fullsized":"2024-09-21T11:59:19.535711+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:57:37.071908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.071908+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":883,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.832702+0000","last_change":"2024-09-21T11:57:18.759800+0000","last_active":"2024-09-21T11:59:25.832702+0000","last_peered":"2024-09-21T11:59:25.832702+0000","last_clean":"2024-09-21T11:59:25.832702+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:59:25.832702+0000","last_undegraded":"2024-09-21T11:59:25.832702+0000","last_fullsized":"2024-09-21T11:59:25.832702+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:57:18.759710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:18.759710+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":873,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.589873+0000","last_change":"2024-09-21T11:59:17.879144+0000","last_active":"2024-09-21T11:59:19.589873+0000","last_peered":"2024-09-21T11:59:19.589873+0000","last_clean":"2024-09-21T11:59:19.589873+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:59:19.589873+0000","last_undegraded":"2024-09-21T11:59:19.589873+0000","last_fullsized":"2024-09-21T11:59:19.589873+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:17.879101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:59:17.879101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:43.295771+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":827,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.586726+0000","last_change":"2024-09-21T11:58:23.430651+0000","last_active":"2024-09-21T11:59:19.586726+0000","last_peered":"2024-09-21T11:59:19.586726+0000","last_clean":"2024-09-21T11:59:19.586726+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:59:19.586726+0000","last_undegraded":"2024-09-21T11:59:19.586726+0000","last_fullsized":"2024-09-21T11:59:19.586726+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:23.430517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:23.430517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:25.920377+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":858,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.833152+0000","last_change":"2024-09-21T11:57:58.537818+0000","last_active":"2024-09-21T11:59:25.833152+0000","last_peered":"2024-09-21T11:59:25.833152+0000","last_clean":"2024-09-21T11:59:25.833152+0000","last_became_active":"2024-09-21T11:57:58.537121+0000","last_became_peered":"2024-09-21T11:57:58.537121+0000","last_unstale":"2024-09-21T11:59:25.833152+0000","last_undegraded":"2024-09-21T11:59:25.833152+0000","last_fullsized":"2024-09-21T11:59:25.833152+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:43.803926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.803926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":887,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.834928+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:59:25.834928+0000","last_peered":"2024-09-21T11:59:25.834928+0000","last_clean":"2024-09-21T11:59:25.834928+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:59:25.834928+0000","last_undegraded":"2024-09-21T11:59:25.834928+0000","last_fullsized":"2024-09-21T11:59:25.834928+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":835,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558723+0000","last_change":"2024-09-21T11:56:53.232038+0000","last_active":"2024-09-21T11:59:19.558723+0000","last_peered":"2024-09-21T11:59:19.558723+0000","last_clean":"2024-09-21T11:59:19.558723+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:59:19.558723+0000","last_undegraded":"2024-09-21T11:59:19.558723+0000","last_fullsized":"2024-09-21T11:59:19.558723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.231963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.231963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":869,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.836266+0000","last_change":"2024-09-21T11:59:08.841142+0000","last_active":"2024-09-21T11:59:25.836266+0000","last_peered":"2024-09-21T11:59:25.836266+0000","last_clean":"2024-09-21T11:59:25.836266+0000","last_became_active":"2024-09-21T11:58:17.527743+0000","last_became_peered":"2024-09-21T11:58:17.527743+0000","last_unstale":"2024-09-21T11:59:25.836266+0000","last_undegraded":"2024-09-21T11:59:25.836266+0000","last_fullsized":"2024-09-21T11:59:25.836266+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:08.841066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.841066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:28.082425+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":855,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.587245+0000","last_change":"2024-09-21T11:58:53.713848+0000","last_active":"2024-09-21T11:59:19.587245+0000","last_peered":"2024-09-21T11:59:19.587245+0000","last_clean":"2024-09-21T11:59:19.587245+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:59:19.587245+0000","last_undegraded":"2024-09-21T11:59:19.587245+0000","last_fullsized":"2024-09-21T11:59:19.587245+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:53.713785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:53.713785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:04.557035+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":550,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558956+0000","last_change":"2024-09-21T11:58:17.532076+0000","last_active":"2024-09-21T11:59:19.558956+0000","last_peered":"2024-09-21T11:59:19.558956+0000","last_clean":"2024-09-21T11:59:19.558956+0000","last_became_active":"2024-09-21T11:58:17.531567+0000","last_became_peered":"2024-09-21T11:58:17.531567+0000","last_unstale":"2024-09-21T11:59:19.558956+0000","last_undegraded":"2024-09-21T11:59:19.558956+0000","last_fullsized":"2024-09-21T11:59:19.558956+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":884,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835209+0000","last_change":"2024-09-21T11:57:58.536812+0000","last_active":"2024-09-21T11:59:25.835209+0000","last_peered":"2024-09-21T11:59:25.835209+0000","last_clean":"2024-09-21T11:59:25.835209+0000","last_became_active":"2024-09-21T11:57:58.535015+0000","last_became_peered":"2024-09-21T11:57:58.535015+0000","last_unstale":"2024-09-21T11:59:25.835209+0000","last_undegraded":"2024-09-21T11:59:25.835209+0000","last_fullsized":"2024-09-21T11:59:25.835209+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:39.735027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:39.735027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":953,"reported_epoch":453,"state":"active+clean","last_fresh":"2024-09-21T11:59:25.835527+0000","last_change":"2024-09-21T11:58:16.743891+0000","last_active":"2024-09-21T11:59:25.835527+0000","last_peered":"2024-09-21T11:59:25.835527+0000","last_clean":"2024-09-21T11:59:25.835527+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:59:25.835527+0000","last_undegraded":"2024-09-21T11:59:25.835527+0000","last_fullsized":"2024-09-21T11:59:25.835527+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:16.743830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:16.743830+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.512604+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":844,"reported_epoch":451,"state":"active+clean","last_fresh":"2024-09-21T11:59:19.558175+0000","last_change":"2024-09-21T11:57:48.045901+0000","last_active":"2024-09-21T11:59:19.558175+0000","last_peered":"2024-09-21T11:59:19.558175+0000","last_clean":"2024-09-21T11:59:19.558175+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:59:19.558175+0000","last_undegraded":"2024-09-21T11:59:19.558175+0000","last_fullsized":"2024-09-21T11:59:19.558175+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:48.045820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.045820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":3521708056,"num_objects":53738,"num_object_clones":0,"num_object_copies":161214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3533,"num_objects_unfound":0,"num_objects_dirty":53738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188728,"num_write_kb":7758849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15759,"num_bytes_recovered":1032716288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10790842368,"data_stored":10790830152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81275,"ondisk_log_size":81275,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":6904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1151,"num_read_kb":1146,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234221,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2145272,"kb_used_data":1378276,"kb_used_omap":1,"kb_used_meta":766974,"kb_avail":91599880,"statfs":{"total":95995035648,"available":93798277120,"internally_reserved":0,"allocated":1411354624,"data_stored":1410965823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":785381834},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463155,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2262728,"kb_used_data":1631796,"kb_used_omap":1,"kb_used_meta":630910,"kb_avail":91482424,"statfs":{"total":95995035648,"available":93678002176,"internally_reserved":0,"allocated":1670959104,"data_stored":1670568529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":646052294},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724791,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1577792,"kb_used_data":896816,"kb_used_omap":1,"kb_used_meta":680894,"kb_avail":92167360,"statfs":{"total":95995035648,"available":94379376640,"internally_reserved":0,"allocated":918339584,"data_stored":917938501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":697235910},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733921022,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1214548,"kb_used_data":619712,"kb_used_omap":1,"kb_used_meta":594814,"kb_avail":92530604,"statfs":{"total":95995035648,"available":94751338496,"internally_reserved":0,"allocated":634585088,"data_stored":634175273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":609089989},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117252,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2195284,"kb_used_data":1405948,"kb_used_omap":1,"kb_used_meta":789182,"kb_avail":91549868,"statfs":{"total":95995035648,"available":93747064832,"internally_reserved":0,"allocated":1439690752,"data_stored":1439291499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":808122823},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411593,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2136572,"kb_used_data":1352748,"kb_used_omap":1,"kb_used_meta":783614,"kb_avail":91608580,"statfs":{"total":95995035648,"available":93807185920,"internally_reserved":0,"allocated":1385213952,"data_stored":1384817576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":802421189},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575120,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2768596,"kb_used_data":2019648,"kb_used_omap":1,"kb_used_meta":748926,"kb_avail":90976556,"statfs":{"total":95995035648,"available":93159993344,"internally_reserved":0,"allocated":2068119552,"data_stored":2067702568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":766900678},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738646,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2120644,"kb_used_data":1327220,"kb_used_omap":1,"kb_used_meta":793342,"kb_avail":91624508,"statfs":{"total":95995035648,"available":93823496192,"internally_reserved":0,"allocated":1359073280,"data_stored":1358669213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":812382661},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1338572800,"data_stored":1338572800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2063728640,"data_stored":2063728640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1359876096,"data_stored":1359872024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1436090368,"data_stored":1436090368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":622333952,"data_stored":622329880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":906493952,"data_stored":906493952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1669074944,"data_stored":1669070872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1394671616,"data_stored":1394671616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:59:29.878 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-21T11:59:29.879 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:59:29.879 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_4 2024-09-21T11:59:29.879 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_4 by 10 min 8 2024-09-21T11:59:29.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T11:59:29.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:29.926 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:29.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:29.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:30.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:30.084 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:30.085 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:29 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:59:30.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:29 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:59:30.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:30.087 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:30.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:30.241 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:30.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:30.243 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:30.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:29 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T11:59:30.344 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:30.369 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:30.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:30.370 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:30.471 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:30.499 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:30.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:30.500 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:30.601 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:30.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:30.631 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:30.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:30.732 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:30.759 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:30.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:30.760 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:30.815 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:59:30.815 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:59:30.852 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:59:30.853 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:59:30.861 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:30.894 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:30.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:30.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:30.898 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:59:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:59:30.943 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:59:30.944 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:59:30.984 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:59:30.984 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:59:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:30 smithi043 ceph-mon[36454]: pgmap v954: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 921 op/s; 3188/151482 objects misplaced (2.105%) 2024-09-21T11:59:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:59:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:30 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:59:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:30 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:30 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2598494596' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:30 smithi043 ceph-mon[29710]: pgmap v954: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 921 op/s; 3188/151482 objects misplaced (2.105%) 2024-09-21T11:59:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:59:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:30 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:59:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:30 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:30.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:30 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2598494596' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:30.996 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:31.034 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:59:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:59:31.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:31.035 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:31.035 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:31.053 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:59:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:59:31.073 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:59:31.073 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:59:31.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:31.159 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:31.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:31.160 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:31.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:30 smithi175 ceph-mon[32613]: pgmap v954: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 3.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 921 op/s; 3188/151482 objects misplaced (2.105%) 2024-09-21T11:59:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T11:59:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:30 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T11:59:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:30 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:30 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2598494596' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:31.261 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:31.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:31.299 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:31.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:31.439 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:31.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:31.440 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:31.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T11:59:31.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:31.589 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:31.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:31.590 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:31.605 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:59:31.605 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:59:31.690 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:31.719 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:31.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:31.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:31.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:31.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:31.867 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:31.867 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:31.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:31.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:31.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:31.994 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:32.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:32.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:32.134 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:32.234 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:32.260 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:59:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:59:32.262 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:32.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:32.263 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:32.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:32.402 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:32.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:32.403 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:32.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T11:59:32.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:32.583 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:32.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:32.584 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:32.685 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:32.710 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:32.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:32.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:32.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:32.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:32.863 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:32.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:32.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:32.993 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:32.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:32.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:33.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:33.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:33.170 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:33.176 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:33.176 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:33.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:33.316 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:33.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:33.317 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:33.379 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:33 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:33.380 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:33 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:33.380 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:33 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:33.380 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:33 smithi043 ceph-mon[29710]: pgmap v955: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.39k op/s; 2883/137235 objects misplaced (2.101%) 2024-09-21T11:59:33.380 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:33 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:33.380 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:33 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:33.380 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:33 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:33.380 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:33 smithi043 ceph-mon[36454]: pgmap v955: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.39k op/s; 2883/137235 objects misplaced (2.101%) 2024-09-21T11:59:33.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:33.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:33.463 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:33.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:33.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:33 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:33.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:33 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:33.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:33 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:33.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:33 smithi175 ceph-mon[32613]: pgmap v955: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.39k op/s; 2883/137235 objects misplaced (2.101%) 2024-09-21T11:59:33.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:33.592 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:33.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:33.593 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:33.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:33.726 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:33.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:33.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:33.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:33.867 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:33.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:33.868 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:33.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:34.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:34.015 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:34.015 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:34.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:34.160 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:34.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:34.161 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:34.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:34.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:34.303 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:34.347 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:34.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:34.455 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:34.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:34.457 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T11:59:34.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T11:59:34.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:34.598 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:34.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:34.599 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:34.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:34.722 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:34.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:34.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:34.823 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:34.844 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:34.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:34.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:34.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:34.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:34.970 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:34.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:35.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:35.099 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:35.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:35.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:35.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:59:35.200 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:35.227 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:35.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:35.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:35.267 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T11:59:35.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:35.375 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:35.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:35.376 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:35.477 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:59:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:59:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:35 smithi043 ceph-mon[29710]: pgmap v956: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 2883/136275 objects misplaced (2.116%) 2024-09-21T11:59:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:35 smithi043 ceph-mon[29710]: from='client.16809 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:59:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:35 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2520700203' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:59:35.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:35 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:59:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:35 smithi043 ceph-mon[36454]: pgmap v956: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 2883/136275 objects misplaced (2.116%) 2024-09-21T11:59:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:35 smithi043 ceph-mon[36454]: from='client.16809 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:59:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:35 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2520700203' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:59:35.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:35 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:59:35.529 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:35.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:35.530 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:35.630 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:35.660 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:35.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:35.661 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:35 smithi175 ceph-mon[32613]: pgmap v956: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.8 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.27k op/s; 2883/136275 objects misplaced (2.116%) 2024-09-21T11:59:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:35 smithi175 ceph-mon[32613]: from='client.16809 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T11:59:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:35 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2520700203' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:59:35.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:35 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T11:59:35.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:35.784 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:35.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:35.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:35.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:35.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:35.922 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:35.922 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:36.023 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:36.049 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:36.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:36.050 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:36.150 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:36.181 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:36.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:36.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:36.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:36.319 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:36.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:36.320 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:36.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:36.478 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:36.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:36.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:36.538 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:36 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:36.538 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:36 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:59:36.538 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:36 smithi043 ceph-mon[36454]: osdmap e455: 8 total, 8 up, 8 in 2024-09-21T11:59:36.538 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:36 smithi043 ceph-mon[36454]: 3.1c scrub starts 2024-09-21T11:59:36.538 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:36 smithi043 ceph-mon[36454]: 3.1c scrub ok 2024-09-21T11:59:36.538 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:36 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:36.539 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:36 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:59:36.539 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:36 smithi043 ceph-mon[29710]: osdmap e455: 8 total, 8 up, 8 in 2024-09-21T11:59:36.539 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:36 smithi043 ceph-mon[29710]: 3.1c scrub starts 2024-09-21T11:59:36.539 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:36 smithi043 ceph-mon[29710]: 3.1c scrub ok 2024-09-21T11:59:36.580 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:36.613 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:36.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:36.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:36.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:36.752 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:36.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:36.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:36.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:36 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:36.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:36 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T11:59:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:36 smithi175 ceph-mon[32613]: osdmap e455: 8 total, 8 up, 8 in 2024-09-21T11:59:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:36 smithi175 ceph-mon[32613]: 3.1c scrub starts 2024-09-21T11:59:36.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:36 smithi175 ceph-mon[32613]: 3.1c scrub ok 2024-09-21T11:59:36.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:36.884 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:36.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:36.885 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:36.986 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:37.012 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:37.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:37.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:37.114 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:37.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:37.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:37.161 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:59:37.162 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:59:37.256 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:37.285 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:37.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:37.286 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:37.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:37.415 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:37.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:37.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:37.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:37 smithi043 ceph-mon[36454]: pgmap v958: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 613 B/s rd, 0 B/s wr, 828 op/s; 2883/130755 objects misplaced (2.205%) 2024-09-21T11:59:37.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:37 smithi043 ceph-mon[36454]: 5.8 scrub starts 2024-09-21T11:59:37.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:37 smithi043 ceph-mon[36454]: 5.8 scrub ok 2024-09-21T11:59:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:37 smithi043 ceph-mon[29710]: pgmap v958: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 613 B/s rd, 0 B/s wr, 828 op/s; 2883/130755 objects misplaced (2.205%) 2024-09-21T11:59:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:37 smithi043 ceph-mon[29710]: 5.8 scrub starts 2024-09-21T11:59:37.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:37 smithi043 ceph-mon[29710]: 5.8 scrub ok 2024-09-21T11:59:37.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:37.544 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:37.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:37.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:37.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:37.683 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:37.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:37.683 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:37.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:37 smithi175 ceph-mon[32613]: pgmap v958: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 613 B/s rd, 0 B/s wr, 828 op/s; 2883/130755 objects misplaced (2.205%) 2024-09-21T11:59:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:37 smithi175 ceph-mon[32613]: 5.8 scrub starts 2024-09-21T11:59:37.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:37 smithi175 ceph-mon[32613]: 5.8 scrub ok 2024-09-21T11:59:37.784 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:37.811 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:37.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:37.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:37.887 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":956,"stamp":"2024-09-21T11:59:34.201309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2964462131,"num_objects":45425,"num_object_clones":0,"num_object_copies":136275,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2883,"num_objects_unfound":0,"num_objects_dirty":45425,"num_whiteouts":0,"num_read":2284,"num_read_kb":2036,"num_write":197962,"num_write_kb":7758894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16117,"num_bytes_recovered":1032725379,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83027,"ondisk_log_size":83027,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":476,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":14806924,"kb_used_data":8965064,"kb_used_omap":12,"kb_used_meta":5841395,"kb_avail":735154292,"statfs":{"total":767960285184,"available":752797995008,"internally_reserved":0,"allocated":9180225536,"data_stored":9176920009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":5981589042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,6,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-694681600,"num_objects":-10600,"num_object_clones":0,"num_object_copies":-31800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-650,"num_objects_unfound":0,"num_objects_dirty":-10600,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10600,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.380474"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":78,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.861350+0000","last_change":"2024-09-21T11:57:58.529638+0000","last_active":"2024-09-21T11:59:27.861350+0000","last_peered":"2024-09-21T11:59:27.861350+0000","last_clean":"2024-09-21T11:59:27.861350+0000","last_became_active":"2024-09-21T11:57:58.529245+0000","last_became_peered":"2024-09-21T11:57:58.529245+0000","last_unstale":"2024-09-21T11:59:27.861350+0000","last_undegraded":"2024-09-21T11:59:27.861350+0000","last_fullsized":"2024-09-21T11:59:27.861350+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:43.783030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.783030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"269'15","reported_seq":881,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860307+0000","last_change":"2024-09-21T11:57:37.936348+0000","last_active":"2024-09-21T11:59:27.860307+0000","last_peered":"2024-09-21T11:59:27.860307+0000","last_clean":"2024-09-21T11:59:27.860307+0000","last_became_active":"2024-09-21T11:51:43.527076+0000","last_became_peered":"2024-09-21T11:51:43.527076+0000","last_unstale":"2024-09-21T11:59:27.860307+0000","last_undegraded":"2024-09-21T11:59:27.860307+0000","last_fullsized":"2024-09-21T11:59:27.860307+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:37.936279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.936279+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:39.639770+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":559,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.855133+0000","last_change":"2024-09-21T11:56:08.429983+0000","last_active":"2024-09-21T11:59:27.855133+0000","last_peered":"2024-09-21T11:59:27.855133+0000","last_clean":"2024-09-21T11:59:27.855133+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T11:59:27.855133+0000","last_undegraded":"2024-09-21T11:59:27.855133+0000","last_fullsized":"2024-09-21T11:59:27.855133+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:08.429912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.429912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":654,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.882243+0000","last_change":"2024-09-21T11:58:21.827334+0000","last_active":"2024-09-21T11:59:27.882243+0000","last_peered":"2024-09-21T11:59:27.882243+0000","last_clean":"2024-09-21T11:59:27.882243+0000","last_became_active":"2024-09-21T11:51:43.528263+0000","last_became_peered":"2024-09-21T11:51:43.528263+0000","last_unstale":"2024-09-21T11:59:27.882243+0000","last_undegraded":"2024-09-21T11:59:27.882243+0000","last_fullsized":"2024-09-21T11:59:27.882243+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:21.827047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:21.827047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:43.889402+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":853,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863344+0000","last_change":"2024-09-21T11:56:49.693769+0000","last_active":"2024-09-21T11:59:27.863344+0000","last_peered":"2024-09-21T11:59:27.863344+0000","last_clean":"2024-09-21T11:59:27.863344+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T11:59:27.863344+0000","last_undegraded":"2024-09-21T11:59:27.863344+0000","last_fullsized":"2024-09-21T11:59:27.863344+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:49.693594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:49.693594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":843,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863706+0000","last_change":"2024-09-21T11:57:37.711838+0000","last_active":"2024-09-21T11:59:27.863706+0000","last_peered":"2024-09-21T11:59:27.863706+0000","last_clean":"2024-09-21T11:59:27.863706+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T11:59:27.863706+0000","last_undegraded":"2024-09-21T11:59:27.863706+0000","last_fullsized":"2024-09-21T11:59:27.863706+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:37.711758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.711758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"269'27","reported_seq":888,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.865775+0000","last_change":"2024-09-21T11:56:31.572626+0000","last_active":"2024-09-21T11:59:27.865775+0000","last_peered":"2024-09-21T11:59:27.865775+0000","last_clean":"2024-09-21T11:59:27.865775+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T11:59:27.865775+0000","last_undegraded":"2024-09-21T11:59:27.865775+0000","last_fullsized":"2024-09-21T11:59:27.865775+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:56:31.572525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:31.572525+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":704,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860483+0000","last_change":"2024-09-21T11:58:30.584828+0000","last_active":"2024-09-21T11:59:27.860483+0000","last_peered":"2024-09-21T11:59:27.860483+0000","last_clean":"2024-09-21T11:59:27.860483+0000","last_became_active":"2024-09-21T11:58:17.527501+0000","last_became_peered":"2024-09-21T11:58:17.527501+0000","last_unstale":"2024-09-21T11:59:27.860483+0000","last_undegraded":"2024-09-21T11:59:27.860483+0000","last_fullsized":"2024-09-21T11:59:27.860483+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:30.584774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:30.584774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:50.421881+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":832,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.884264+0000","last_change":"2024-09-21T11:51:15.585553+0000","last_active":"2024-09-21T11:59:27.884264+0000","last_peered":"2024-09-21T11:59:27.884264+0000","last_clean":"2024-09-21T11:59:27.884264+0000","last_became_active":"2024-09-21T11:40:00.313842+0000","last_became_peered":"2024-09-21T11:40:00.313842+0000","last_unstale":"2024-09-21T11:59:27.884264+0000","last_undegraded":"2024-09-21T11:59:27.884264+0000","last_fullsized":"2024-09-21T11:59:27.884264+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:15.585504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:15.585504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.819967+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"269'15","reported_seq":881,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.866774+0000","last_change":"2024-09-21T11:58:17.547443+0000","last_active":"2024-09-21T11:59:27.866774+0000","last_peered":"2024-09-21T11:59:27.866774+0000","last_clean":"2024-09-21T11:59:27.866774+0000","last_became_active":"2024-09-21T11:58:17.525792+0000","last_became_peered":"2024-09-21T11:58:17.525792+0000","last_unstale":"2024-09-21T11:59:27.866774+0000","last_undegraded":"2024-09-21T11:59:27.866774+0000","last_fullsized":"2024-09-21T11:59:27.866774+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:57:07.774892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:07.774892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":840,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.882373+0000","last_change":"2024-09-21T11:58:58.842083+0000","last_active":"2024-09-21T11:59:27.882373+0000","last_peered":"2024-09-21T11:59:27.882373+0000","last_clean":"2024-09-21T11:59:27.882373+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T11:59:27.882373+0000","last_undegraded":"2024-09-21T11:59:27.882373+0000","last_fullsized":"2024-09-21T11:59:27.882373+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:58.842006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:58.842006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:22.061210+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":78,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863576+0000","last_change":"2024-09-21T11:57:58.527130+0000","last_active":"2024-09-21T11:59:27.863576+0000","last_peered":"2024-09-21T11:59:27.863576+0000","last_clean":"2024-09-21T11:59:27.863576+0000","last_became_active":"2024-09-21T11:57:58.526820+0000","last_became_peered":"2024-09-21T11:57:58.526820+0000","last_unstale":"2024-09-21T11:59:27.863576+0000","last_undegraded":"2024-09-21T11:59:27.863576+0000","last_fullsized":"2024-09-21T11:59:27.863576+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:36.422726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:36.422726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":878,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.882811+0000","last_change":"2024-09-21T11:56:02.994041+0000","last_active":"2024-09-21T11:59:27.882811+0000","last_peered":"2024-09-21T11:59:27.882811+0000","last_clean":"2024-09-21T11:59:27.882811+0000","last_became_active":"2024-09-21T11:47:03.384741+0000","last_became_peered":"2024-09-21T11:47:03.384741+0000","last_unstale":"2024-09-21T11:59:27.882811+0000","last_undegraded":"2024-09-21T11:59:27.882811+0000","last_fullsized":"2024-09-21T11:59:27.882811+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.993966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.993966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:35.823113+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"269'7","reported_seq":879,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863005+0000","last_change":"2024-09-21T11:58:17.549945+0000","last_active":"2024-09-21T11:59:27.863005+0000","last_peered":"2024-09-21T11:59:27.863005+0000","last_clean":"2024-09-21T11:59:27.863005+0000","last_became_active":"2024-09-21T11:58:17.525789+0000","last_became_peered":"2024-09-21T11:58:17.525789+0000","last_unstale":"2024-09-21T11:59:27.863005+0000","last_undegraded":"2024-09-21T11:59:27.863005+0000","last_fullsized":"2024-09-21T11:59:27.863005+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'7","last_scrub_stamp":"2024-09-21T11:56:59.659576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:59.659576+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":854,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862619+0000","last_change":"2024-09-21T11:58:22.381598+0000","last_active":"2024-09-21T11:59:27.862619+0000","last_peered":"2024-09-21T11:59:27.862619+0000","last_clean":"2024-09-21T11:59:27.862619+0000","last_became_active":"2024-09-21T11:47:03.368302+0000","last_became_peered":"2024-09-21T11:47:03.368302+0000","last_unstale":"2024-09-21T11:59:27.862619+0000","last_undegraded":"2024-09-21T11:59:27.862619+0000","last_fullsized":"2024-09-21T11:59:27.862619+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:22.381531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:22.381531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.811109+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":851,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.883531+0000","last_change":"2024-09-21T11:58:24.861423+0000","last_active":"2024-09-21T11:59:27.883531+0000","last_peered":"2024-09-21T11:59:27.883531+0000","last_clean":"2024-09-21T11:59:27.883531+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T11:59:27.883531+0000","last_undegraded":"2024-09-21T11:59:27.883531+0000","last_fullsized":"2024-09-21T11:59:27.883531+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:24.861354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:24.861354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.549127+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":360,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860597+0000","last_change":"2024-09-21T11:59:15.886098+0000","last_active":"2024-09-21T11:59:27.860597+0000","last_peered":"2024-09-21T11:59:27.860597+0000","last_clean":"2024-09-21T11:59:27.860597+0000","last_became_active":"2024-09-21T11:51:43.524237+0000","last_became_peered":"2024-09-21T11:51:43.524237+0000","last_unstale":"2024-09-21T11:59:27.860597+0000","last_undegraded":"2024-09-21T11:59:27.860597+0000","last_fullsized":"2024-09-21T11:59:27.860597+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:15.886007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:15.886007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:23.801683+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,5],"acting":[6,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":78,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863189+0000","last_change":"2024-09-21T11:57:58.534353+0000","last_active":"2024-09-21T11:59:27.863189+0000","last_peered":"2024-09-21T11:59:27.863189+0000","last_clean":"2024-09-21T11:59:27.863189+0000","last_became_active":"2024-09-21T11:57:58.530327+0000","last_became_peered":"2024-09-21T11:57:58.530327+0000","last_unstale":"2024-09-21T11:59:27.863189+0000","last_undegraded":"2024-09-21T11:59:27.863189+0000","last_fullsized":"2024-09-21T11:59:27.863189+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:27.176320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:27.176320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"269'27","reported_seq":1004,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.857626+0000","last_change":"2024-09-21T11:59:17.201834+0000","last_active":"2024-09-21T11:59:27.857626+0000","last_peered":"2024-09-21T11:59:27.857626+0000","last_clean":"2024-09-21T11:59:27.857626+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T11:59:27.857626+0000","last_undegraded":"2024-09-21T11:59:27.857626+0000","last_fullsized":"2024-09-21T11:59:27.857626+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:59:17.201563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:17.201563+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:38.988507+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":564,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.858664+0000","last_change":"2024-09-21T11:58:17.533215+0000","last_active":"2024-09-21T11:59:27.858664+0000","last_peered":"2024-09-21T11:59:27.858664+0000","last_clean":"2024-09-21T11:59:27.858664+0000","last_became_active":"2024-09-21T11:58:17.532890+0000","last_became_peered":"2024-09-21T11:58:17.532890+0000","last_unstale":"2024-09-21T11:59:27.858664+0000","last_undegraded":"2024-09-21T11:59:27.858664+0000","last_fullsized":"2024-09-21T11:59:27.858664+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:56.501356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:56.501356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:27.816777+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":847,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863915+0000","last_change":"2024-09-21T11:57:04.647951+0000","last_active":"2024-09-21T11:59:27.863915+0000","last_peered":"2024-09-21T11:59:27.863915+0000","last_clean":"2024-09-21T11:59:27.863915+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T11:59:27.863915+0000","last_undegraded":"2024-09-21T11:59:27.863915+0000","last_fullsized":"2024-09-21T11:59:27.863915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:04.647749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:04.647749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":666,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.852440+0000","last_change":"2024-09-21T11:55:57.442188+0000","last_active":"2024-09-21T11:59:27.852440+0000","last_peered":"2024-09-21T11:59:27.852440+0000","last_clean":"2024-09-21T11:59:27.852440+0000","last_became_active":"2024-09-21T11:51:43.543074+0000","last_became_peered":"2024-09-21T11:51:43.543074+0000","last_unstale":"2024-09-21T11:59:27.852440+0000","last_undegraded":"2024-09-21T11:59:27.852440+0000","last_fullsized":"2024-09-21T11:59:27.852440+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:57.441965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:55:57.441965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"269'18","reported_seq":907,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863140+0000","last_change":"2024-09-21T11:54:37.403365+0000","last_active":"2024-09-21T11:59:27.863140+0000","last_peered":"2024-09-21T11:59:27.863140+0000","last_clean":"2024-09-21T11:59:27.863140+0000","last_became_active":"2024-09-21T11:47:03.382550+0000","last_became_peered":"2024-09-21T11:47:03.382550+0000","last_unstale":"2024-09-21T11:59:27.863140+0000","last_undegraded":"2024-09-21T11:59:27.863140+0000","last_fullsized":"2024-09-21T11:59:27.863140+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:54:37.403058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:37.403058+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":366,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863532+0000","last_change":"2024-09-21T11:58:17.529121+0000","last_active":"2024-09-21T11:59:27.863532+0000","last_peered":"2024-09-21T11:59:27.863532+0000","last_clean":"2024-09-21T11:59:27.863532+0000","last_became_active":"2024-09-21T11:58:17.528785+0000","last_became_peered":"2024-09-21T11:58:17.528785+0000","last_unstale":"2024-09-21T11:59:27.863532+0000","last_undegraded":"2024-09-21T11:59:27.863532+0000","last_fullsized":"2024-09-21T11:59:27.863532+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:33.487132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:33.487132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":347,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.861077+0000","last_change":"2024-09-21T11:51:31.479794+0000","last_active":"2024-09-21T11:59:27.861077+0000","last_peered":"2024-09-21T11:59:27.861077+0000","last_clean":"2024-09-21T11:59:27.861077+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T11:59:27.861077+0000","last_undegraded":"2024-09-21T11:59:27.861077+0000","last_fullsized":"2024-09-21T11:59:27.861077+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:50:27.321736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:50:27.321736+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":585,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.855305+0000","last_change":"2024-09-21T11:58:17.554468+0000","last_active":"2024-09-21T11:59:27.855305+0000","last_peered":"2024-09-21T11:59:27.855305+0000","last_clean":"2024-09-21T11:59:27.855305+0000","last_became_active":"2024-09-21T11:58:17.529593+0000","last_became_peered":"2024-09-21T11:58:17.529593+0000","last_unstale":"2024-09-21T11:59:27.855305+0000","last_undegraded":"2024-09-21T11:59:27.855305+0000","last_fullsized":"2024-09-21T11:59:27.855305+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:10.530913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.530913+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"269'20","reported_seq":876,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.870381+0000","last_change":"2024-09-21T11:53:27.541799+0000","last_active":"2024-09-21T11:59:27.870381+0000","last_peered":"2024-09-21T11:59:27.870381+0000","last_clean":"2024-09-21T11:59:27.870381+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T11:59:27.870381+0000","last_undegraded":"2024-09-21T11:59:27.870381+0000","last_fullsized":"2024-09-21T11:59:27.870381+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:53:27.541726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:27.541726+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"432'10","reported_seq":1136,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860977+0000","last_change":"2024-09-21T11:58:18.736596+0000","last_active":"2024-09-21T11:59:27.860977+0000","last_peered":"2024-09-21T11:59:27.860977+0000","last_clean":"2024-09-21T11:59:27.860977+0000","last_became_active":"2024-09-21T11:58:17.529830+0000","last_became_peered":"2024-09-21T11:58:17.529830+0000","last_unstale":"2024-09-21T11:59:27.860977+0000","last_undegraded":"2024-09-21T11:59:27.860977+0000","last_fullsized":"2024-09-21T11:59:27.860977+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"383'9","last_scrub_stamp":"2024-09-21T11:58:18.736520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:18.736520+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:40.102209+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":94,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.882089+0000","last_change":"2024-09-21T11:57:49.896884+0000","last_active":"2024-09-21T11:59:27.882089+0000","last_peered":"2024-09-21T11:59:27.882089+0000","last_clean":"2024-09-21T11:59:27.882089+0000","last_became_active":"2024-09-21T11:57:45.504680+0000","last_became_peered":"2024-09-21T11:57:45.504680+0000","last_unstale":"2024-09-21T11:59:27.882089+0000","last_undegraded":"2024-09-21T11:59:27.882089+0000","last_fullsized":"2024-09-21T11:59:27.882089+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:49.896819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:49.896819+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:20.946295+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"454'12497","reported_seq":12633,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.832715+0000","last_change":"2024-09-21T11:56:36.507880+0000","last_active":"2024-09-21T11:59:31.832715+0000","last_peered":"2024-09-21T11:59:31.832715+0000","last_clean":"2024-09-21T11:59:31.832715+0000","last_became_active":"2024-09-21T11:56:36.507606+0000","last_became_peered":"2024-09-21T11:56:36.507606+0000","last_unstale":"2024-09-21T11:59:31.832715+0000","last_undegraded":"2024-09-21T11:59:31.832715+0000","last_fullsized":"2024-09-21T11:59:31.832715+0000","mapping_epoch":392,"log_start":"427'7722","ondisk_log_start":"427'7722","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4775,"log_dups_size":0,"ondisk_log_size":4775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180944896,"num_objects":2761,"num_object_clones":0,"num_object_copies":8283,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12497,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"269'27","reported_seq":585,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.858736+0000","last_change":"2024-09-21T11:55:52.562992+0000","last_active":"2024-09-21T11:59:27.858736+0000","last_peered":"2024-09-21T11:59:27.858736+0000","last_clean":"2024-09-21T11:59:27.858736+0000","last_became_active":"2024-09-21T11:47:03.376597+0000","last_became_peered":"2024-09-21T11:47:03.376597+0000","last_unstale":"2024-09-21T11:59:27.858736+0000","last_undegraded":"2024-09-21T11:59:27.858736+0000","last_fullsized":"2024-09-21T11:59:27.858736+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:55:52.562928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:52.562928+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":873,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.865337+0000","last_change":"2024-09-21T11:57:13.738771+0000","last_active":"2024-09-21T11:59:27.865337+0000","last_peered":"2024-09-21T11:59:27.865337+0000","last_clean":"2024-09-21T11:59:27.865337+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T11:59:27.865337+0000","last_undegraded":"2024-09-21T11:59:27.865337+0000","last_fullsized":"2024-09-21T11:59:27.865337+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:13.738727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:13.738727+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"432'11","reported_seq":496,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.883191+0000","last_change":"2024-09-21T11:58:17.537763+0000","last_active":"2024-09-21T11:59:27.883191+0000","last_peered":"2024-09-21T11:59:27.883191+0000","last_clean":"2024-09-21T11:59:27.883191+0000","last_became_active":"2024-09-21T11:58:17.526752+0000","last_became_peered":"2024-09-21T11:58:17.526752+0000","last_unstale":"2024-09-21T11:59:27.883191+0000","last_undegraded":"2024-09-21T11:59:27.883191+0000","last_fullsized":"2024-09-21T11:59:27.883191+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"366'10","last_scrub_stamp":"2024-09-21T11:56:48.961017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.961017+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.739596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":897,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.883824+0000","last_change":"2024-09-21T11:58:16.866853+0000","last_active":"2024-09-21T11:59:27.883824+0000","last_peered":"2024-09-21T11:59:27.883824+0000","last_clean":"2024-09-21T11:59:27.883824+0000","last_became_active":"2024-09-21T11:57:58.526927+0000","last_became_peered":"2024-09-21T11:57:58.526927+0000","last_unstale":"2024-09-21T11:59:27.883824+0000","last_undegraded":"2024-09-21T11:59:27.883824+0000","last_fullsized":"2024-09-21T11:59:27.883824+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:16.866782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:16.866782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.387681+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":102,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.856756+0000","last_change":"2024-09-21T11:59:18.174455+0000","last_active":"2024-09-21T11:59:27.856756+0000","last_peered":"2024-09-21T11:59:27.856756+0000","last_clean":"2024-09-21T11:59:27.856756+0000","last_became_active":"2024-09-21T11:57:45.514811+0000","last_became_peered":"2024-09-21T11:57:45.514811+0000","last_unstale":"2024-09-21T11:59:27.856756+0000","last_undegraded":"2024-09-21T11:59:27.856756+0000","last_fullsized":"2024-09-21T11:59:27.856756+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:59:18.174384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:59:18.174384+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:22.819349+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.f","version":"454'12684","reported_seq":12869,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.327494+0000","last_change":"2024-09-21T11:59:12.760513+0000","last_active":"2024-09-21T11:59:31.327494+0000","last_peered":"2024-09-21T11:59:31.327494+0000","last_clean":"2024-09-21T11:59:31.327494+0000","last_became_active":"2024-09-21T11:56:36.511191+0000","last_became_peered":"2024-09-21T11:56:36.511191+0000","last_unstale":"2024-09-21T11:59:31.327494+0000","last_undegraded":"2024-09-21T11:59:31.327494+0000","last_fullsized":"2024-09-21T11:59:31.327494+0000","mapping_epoch":392,"log_start":"426'7842","ondisk_log_start":"426'7842","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'11577","last_scrub_stamp":"2024-09-21T11:59:12.760365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:12.760365+0000","objects_scrubbed":4105,"log_size":4842,"log_dups_size":0,"ondisk_log_size":4842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:27.399210+0000","scrub_duration":1126,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":194248704,"num_objects":2964,"num_object_clones":0,"num_object_copies":8892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12684,"num_write_kb":500736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"269'30","reported_seq":963,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.865038+0000","last_change":"2024-09-21T11:57:58.547816+0000","last_active":"2024-09-21T11:59:27.865038+0000","last_peered":"2024-09-21T11:59:27.865038+0000","last_clean":"2024-09-21T11:59:27.865038+0000","last_became_active":"2024-09-21T11:57:58.537184+0000","last_became_peered":"2024-09-21T11:57:58.537184+0000","last_unstale":"2024-09-21T11:59:27.865038+0000","last_undegraded":"2024-09-21T11:59:27.865038+0000","last_fullsized":"2024-09-21T11:59:27.865038+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'30","last_scrub_stamp":"2024-09-21T11:57:00.631769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:00.631769+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":867,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.864431+0000","last_change":"2024-09-21T11:56:53.701703+0000","last_active":"2024-09-21T11:59:27.864431+0000","last_peered":"2024-09-21T11:59:27.864431+0000","last_clean":"2024-09-21T11:59:27.864431+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T11:59:27.864431+0000","last_undegraded":"2024-09-21T11:59:27.864431+0000","last_fullsized":"2024-09-21T11:59:27.864431+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.701639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.701639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":352,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.864614+0000","last_change":"2024-09-21T11:58:19.959956+0000","last_active":"2024-09-21T11:59:27.864614+0000","last_peered":"2024-09-21T11:59:27.864614+0000","last_clean":"2024-09-21T11:59:27.864614+0000","last_became_active":"2024-09-21T11:51:43.541598+0000","last_became_peered":"2024-09-21T11:51:43.541598+0000","last_unstale":"2024-09-21T11:59:27.864614+0000","last_undegraded":"2024-09-21T11:59:27.864614+0000","last_fullsized":"2024-09-21T11:59:27.864614+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:19.959912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:19.959912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.737932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":362,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.856482+0000","last_change":"2024-09-21T11:59:05.095763+0000","last_active":"2024-09-21T11:59:27.856482+0000","last_peered":"2024-09-21T11:59:27.856482+0000","last_clean":"2024-09-21T11:59:27.856482+0000","last_became_active":"2024-09-21T11:52:01.954394+0000","last_became_peered":"2024-09-21T11:52:01.954394+0000","last_unstale":"2024-09-21T11:59:27.856482+0000","last_undegraded":"2024-09-21T11:59:27.856482+0000","last_fullsized":"2024-09-21T11:59:27.856482+0000","mapping_epoch":297,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":298,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:05.095722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:05.095722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:12.462739+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":99,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862719+0000","last_change":"2024-09-21T11:57:47.804542+0000","last_active":"2024-09-21T11:59:27.862719+0000","last_peered":"2024-09-21T11:59:27.862719+0000","last_clean":"2024-09-21T11:59:27.862719+0000","last_became_active":"2024-09-21T11:57:45.514189+0000","last_became_peered":"2024-09-21T11:57:45.514189+0000","last_unstale":"2024-09-21T11:59:27.862719+0000","last_undegraded":"2024-09-21T11:59:27.862719+0000","last_fullsized":"2024-09-21T11:59:27.862719+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:47.804466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:47.804466+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"454'12199","reported_seq":12355,"reported_epoch":454,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-21T11:59:31.839741+0000","last_change":"2024-09-21T11:59:18.695884+0000","last_active":"2024-09-21T11:59:31.839741+0000","last_peered":"2024-09-21T11:59:31.839741+0000","last_clean":"2024-09-21T11:58:17.235375+0000","last_became_active":"2024-09-21T11:58:18.605530+0000","last_became_peered":"2024-09-21T11:58:18.605530+0000","last_unstale":"2024-09-21T11:59:31.839741+0000","last_undegraded":"2024-09-21T11:59:31.839741+0000","last_fullsized":"2024-09-21T11:59:31.839741+0000","mapping_epoch":431,"log_start":"426'7367","ondisk_log_start":"426'7367","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4832,"log_dups_size":0,"ondisk_log_size":4832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188940288,"num_objects":2883,"num_object_clones":0,"num_object_copies":8649,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2883,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12199,"num_write_kb":482624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,7],"acting":[1,7,0],"avail_no_missing":["1","0","7"],"object_location_counts":[{"shards":"0,1,7","objects":2883}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"269'28","reported_seq":902,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.883574+0000","last_change":"2024-09-21T11:58:17.828372+0000","last_active":"2024-09-21T11:59:27.883574+0000","last_peered":"2024-09-21T11:59:27.883574+0000","last_clean":"2024-09-21T11:59:27.883574+0000","last_became_active":"2024-09-21T11:58:17.525491+0000","last_became_peered":"2024-09-21T11:58:17.525491+0000","last_unstale":"2024-09-21T11:59:27.883574+0000","last_undegraded":"2024-09-21T11:59:27.883574+0000","last_fullsized":"2024-09-21T11:59:27.883574+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:58:17.828305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:17.828305+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.685496+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":82,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.867281+0000","last_change":"2024-09-21T11:57:58.539709+0000","last_active":"2024-09-21T11:59:27.867281+0000","last_peered":"2024-09-21T11:59:27.867281+0000","last_clean":"2024-09-21T11:59:27.867281+0000","last_became_active":"2024-09-21T11:57:58.526938+0000","last_became_peered":"2024-09-21T11:57:58.526938+0000","last_unstale":"2024-09-21T11:59:27.867281+0000","last_undegraded":"2024-09-21T11:59:27.867281+0000","last_fullsized":"2024-09-21T11:59:27.867281+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:56:48.446635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.446635+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"205'10","reported_seq":1109,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.864871+0000","last_change":"2024-09-21T11:56:05.567407+0000","last_active":"2024-09-21T11:59:27.864871+0000","last_peered":"2024-09-21T11:59:27.864871+0000","last_clean":"2024-09-21T11:59:27.864871+0000","last_became_active":"2024-09-21T11:47:03.375873+0000","last_became_peered":"2024-09-21T11:47:03.375873+0000","last_unstale":"2024-09-21T11:59:27.864871+0000","last_undegraded":"2024-09-21T11:59:27.864871+0000","last_fullsized":"2024-09-21T11:59:27.864871+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:05.567332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.567332+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":856,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862854+0000","last_change":"2024-09-21T11:56:34.581966+0000","last_active":"2024-09-21T11:59:27.862854+0000","last_peered":"2024-09-21T11:59:27.862854+0000","last_clean":"2024-09-21T11:59:27.862854+0000","last_became_active":"2024-09-21T11:56:34.581626+0000","last_became_peered":"2024-09-21T11:56:34.581626+0000","last_unstale":"2024-09-21T11:59:27.862854+0000","last_undegraded":"2024-09-21T11:59:27.862854+0000","last_fullsized":"2024-09-21T11:59:27.862854+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.643323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.643323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:35.106418+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":180,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860613+0000","last_change":"2024-09-21T11:57:45.998708+0000","last_active":"2024-09-21T11:59:27.860613+0000","last_peered":"2024-09-21T11:59:27.860613+0000","last_clean":"2024-09-21T11:59:27.860613+0000","last_became_active":"2024-09-21T11:57:45.514310+0000","last_became_peered":"2024-09-21T11:57:45.514310+0000","last_unstale":"2024-09-21T11:59:27.860613+0000","last_undegraded":"2024-09-21T11:59:27.860613+0000","last_fullsized":"2024-09-21T11:59:27.860613+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:45.998644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:45.998644+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:19.779779+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"454'12315","reported_seq":12506,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.972427+0000","last_change":"2024-09-21T11:59:01.985776+0000","last_active":"2024-09-21T11:59:31.972427+0000","last_peered":"2024-09-21T11:59:31.972427+0000","last_clean":"2024-09-21T11:59:31.972427+0000","last_became_active":"2024-09-21T11:56:36.623581+0000","last_became_peered":"2024-09-21T11:56:36.623581+0000","last_unstale":"2024-09-21T11:59:31.972427+0000","last_undegraded":"2024-09-21T11:59:31.972427+0000","last_fullsized":"2024-09-21T11:59:31.972427+0000","mapping_epoch":392,"log_start":"417'6116","ondisk_log_start":"417'6116","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"444'10565","last_scrub_stamp":"2024-09-21T11:59:01.985630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:01.985630+0000","objects_scrubbed":4571,"log_size":6199,"log_dups_size":0,"ondisk_log_size":6199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:14.582998+0000","scrub_duration":1821,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182452248,"num_objects":2785,"num_object_clones":0,"num_object_copies":8355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12315,"num_write_kb":483137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"269'20","reported_seq":616,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.852713+0000","last_change":"2024-09-21T11:58:17.557543+0000","last_active":"2024-09-21T11:59:27.852713+0000","last_peered":"2024-09-21T11:59:27.852713+0000","last_clean":"2024-09-21T11:59:27.852713+0000","last_became_active":"2024-09-21T11:58:17.531001+0000","last_became_peered":"2024-09-21T11:58:17.531001+0000","last_unstale":"2024-09-21T11:59:27.852713+0000","last_undegraded":"2024-09-21T11:59:27.852713+0000","last_fullsized":"2024-09-21T11:59:27.852713+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:56:09.421193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.421193+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:25.744190+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":882,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.884331+0000","last_change":"2024-09-21T11:56:51.036609+0000","last_active":"2024-09-21T11:59:27.884331+0000","last_peered":"2024-09-21T11:59:27.884331+0000","last_clean":"2024-09-21T11:59:27.884331+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T11:59:27.884331+0000","last_undegraded":"2024-09-21T11:59:27.884331+0000","last_fullsized":"2024-09-21T11:59:27.884331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:51.036535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:51.036535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:40.871456+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1096,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863529+0000","last_change":"2024-09-21T11:57:40.102377+0000","last_active":"2024-09-21T11:59:27.863529+0000","last_peered":"2024-09-21T11:59:27.863529+0000","last_clean":"2024-09-21T11:59:27.863529+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T11:59:27.863529+0000","last_undegraded":"2024-09-21T11:59:27.863529+0000","last_fullsized":"2024-09-21T11:59:27.863529+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:40.102294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:40.102294+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":854,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863883+0000","last_change":"2024-09-21T11:57:36.109392+0000","last_active":"2024-09-21T11:59:27.863883+0000","last_peered":"2024-09-21T11:59:27.863883+0000","last_clean":"2024-09-21T11:59:27.863883+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T11:59:27.863883+0000","last_undegraded":"2024-09-21T11:59:27.863883+0000","last_fullsized":"2024-09-21T11:59:27.863883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:36.109317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:36.109317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":188,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860551+0000","last_change":"2024-09-21T11:59:07.885015+0000","last_active":"2024-09-21T11:59:27.860551+0000","last_peered":"2024-09-21T11:59:27.860551+0000","last_clean":"2024-09-21T11:59:27.860551+0000","last_became_active":"2024-09-21T11:57:45.518764+0000","last_became_peered":"2024-09-21T11:57:45.518764+0000","last_unstale":"2024-09-21T11:59:27.860551+0000","last_undegraded":"2024-09-21T11:59:27.860551+0000","last_fullsized":"2024-09-21T11:59:27.860551+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:59:07.884939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:59:07.884939+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:13.711897+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.c","version":"454'12414","reported_seq":21422,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.854405+0000","last_change":"2024-09-21T11:59:18.569237+0000","last_active":"2024-09-21T11:59:31.854405+0000","last_peered":"2024-09-21T11:59:31.854405+0000","last_clean":"2024-09-21T11:59:31.854405+0000","last_became_active":"2024-09-21T11:59:18.539184+0000","last_became_peered":"2024-09-21T11:59:18.539184+0000","last_unstale":"2024-09-21T11:59:31.854405+0000","last_undegraded":"2024-09-21T11:59:31.854405+0000","last_fullsized":"2024-09-21T11:59:31.854405+0000","mapping_epoch":449,"log_start":"426'7579","ondisk_log_start":"426'7579","created":392,"last_epoch_clean":450,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4835,"log_dups_size":0,"ondisk_log_size":4835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":181141504,"num_objects":2764,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12414,"num_write_kb":485696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4427,"num_bytes_recovered":290127872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"269'29","reported_seq":921,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863680+0000","last_change":"2024-09-21T11:57:50.976268+0000","last_active":"2024-09-21T11:59:27.863680+0000","last_peered":"2024-09-21T11:59:27.863680+0000","last_clean":"2024-09-21T11:59:27.863680+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T11:59:27.863680+0000","last_undegraded":"2024-09-21T11:59:27.863680+0000","last_fullsized":"2024-09-21T11:59:27.863680+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'29","last_scrub_stamp":"2024-09-21T11:57:50.976029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:50.976029+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":551,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.859010+0000","last_change":"2024-09-21T11:56:58.434536+0000","last_active":"2024-09-21T11:59:27.859010+0000","last_peered":"2024-09-21T11:59:27.859010+0000","last_clean":"2024-09-21T11:59:27.859010+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T11:59:27.859010+0000","last_undegraded":"2024-09-21T11:59:27.859010+0000","last_fullsized":"2024-09-21T11:59:27.859010+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.434464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.434464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":834,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860434+0000","last_change":"2024-09-21T11:54:25.302380+0000","last_active":"2024-09-21T11:59:27.860434+0000","last_peered":"2024-09-21T11:59:27.860434+0000","last_clean":"2024-09-21T11:59:27.860434+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T11:59:27.860434+0000","last_undegraded":"2024-09-21T11:59:27.860434+0000","last_fullsized":"2024-09-21T11:59:27.860434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:25.302147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:25.302147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:34.633118+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":543,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.854969+0000","last_change":"2024-09-21T11:55:51.602665+0000","last_active":"2024-09-21T11:59:27.854969+0000","last_peered":"2024-09-21T11:59:27.854969+0000","last_clean":"2024-09-21T11:59:27.854969+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T11:59:27.854969+0000","last_undegraded":"2024-09-21T11:59:27.854969+0000","last_fullsized":"2024-09-21T11:59:27.854969+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:51.602587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.602587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":94,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.881873+0000","last_change":"2024-09-21T11:57:53.922876+0000","last_active":"2024-09-21T11:59:27.881873+0000","last_peered":"2024-09-21T11:59:27.881873+0000","last_clean":"2024-09-21T11:59:27.881873+0000","last_became_active":"2024-09-21T11:57:45.515242+0000","last_became_peered":"2024-09-21T11:57:45.515242+0000","last_unstale":"2024-09-21T11:59:27.881873+0000","last_undegraded":"2024-09-21T11:59:27.881873+0000","last_fullsized":"2024-09-21T11:59:27.881873+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:53.922795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:53.922795+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:21.979702+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.b","version":"454'12303","reported_seq":12438,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.128055+0000","last_change":"2024-09-21T11:56:36.510297+0000","last_active":"2024-09-21T11:59:31.128055+0000","last_peered":"2024-09-21T11:59:31.128055+0000","last_clean":"2024-09-21T11:59:31.128055+0000","last_became_active":"2024-09-21T11:56:36.510018+0000","last_became_peered":"2024-09-21T11:56:36.510018+0000","last_unstale":"2024-09-21T11:59:31.128055+0000","last_undegraded":"2024-09-21T11:59:31.128055+0000","last_fullsized":"2024-09-21T11:59:31.128055+0000","mapping_epoch":392,"log_start":"426'7621","ondisk_log_start":"426'7621","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4682,"log_dups_size":0,"ondisk_log_size":4682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184221696,"num_objects":2811,"num_object_clones":0,"num_object_copies":8433,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12303,"num_write_kb":483648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"269'43","reported_seq":717,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.858956+0000","last_change":"2024-09-21T11:51:47.299961+0000","last_active":"2024-09-21T11:59:27.858956+0000","last_peered":"2024-09-21T11:59:27.858956+0000","last_clean":"2024-09-21T11:59:27.858956+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T11:59:27.858956+0000","last_undegraded":"2024-09-21T11:59:27.858956+0000","last_fullsized":"2024-09-21T11:59:27.858956+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'43","last_scrub_stamp":"2024-09-21T11:51:47.299856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:51:47.299856+0000","objects_scrubbed":10,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:21.777100+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":377,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.866723+0000","last_change":"2024-09-21T11:57:51.934799+0000","last_active":"2024-09-21T11:59:27.866723+0000","last_peered":"2024-09-21T11:59:27.866723+0000","last_clean":"2024-09-21T11:59:27.866723+0000","last_became_active":"2024-09-21T11:51:43.532442+0000","last_became_peered":"2024-09-21T11:51:43.532442+0000","last_unstale":"2024-09-21T11:59:27.866723+0000","last_undegraded":"2024-09-21T11:59:27.866723+0000","last_fullsized":"2024-09-21T11:59:27.866723+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:51.934563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.934563+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6],"acting":[0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":882,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.882638+0000","last_change":"2024-09-21T11:58:59.855226+0000","last_active":"2024-09-21T11:59:27.882638+0000","last_peered":"2024-09-21T11:59:27.882638+0000","last_clean":"2024-09-21T11:59:27.882638+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T11:59:27.882638+0000","last_undegraded":"2024-09-21T11:59:27.882638+0000","last_fullsized":"2024-09-21T11:59:27.882638+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T11:58:59.855148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:58:59.855148+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:09.486850+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":85,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.861620+0000","last_change":"2024-09-21T11:58:17.527987+0000","last_active":"2024-09-21T11:59:27.861620+0000","last_peered":"2024-09-21T11:59:27.861620+0000","last_clean":"2024-09-21T11:59:27.861620+0000","last_became_active":"2024-09-21T11:58:17.527653+0000","last_became_peered":"2024-09-21T11:58:17.527653+0000","last_unstale":"2024-09-21T11:59:27.861620+0000","last_undegraded":"2024-09-21T11:59:27.861620+0000","last_fullsized":"2024-09-21T11:59:27.861620+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:50:37.858329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:50:37.858329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":830,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863507+0000","last_change":"2024-09-21T11:57:54.859726+0000","last_active":"2024-09-21T11:59:27.863507+0000","last_peered":"2024-09-21T11:59:27.863507+0000","last_clean":"2024-09-21T11:59:27.863507+0000","last_became_active":"2024-09-21T11:40:06.350454+0000","last_became_peered":"2024-09-21T11:40:06.350454+0000","last_unstale":"2024-09-21T11:59:27.863507+0000","last_undegraded":"2024-09-21T11:59:27.863507+0000","last_fullsized":"2024-09-21T11:59:27.863507+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:54.859668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.859668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":974,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862375+0000","last_change":"2024-09-21T11:58:17.529673+0000","last_active":"2024-09-21T11:59:27.862375+0000","last_peered":"2024-09-21T11:59:27.862375+0000","last_clean":"2024-09-21T11:59:27.862375+0000","last_became_active":"2024-09-21T11:58:17.529304+0000","last_became_peered":"2024-09-21T11:58:17.529304+0000","last_unstale":"2024-09-21T11:59:27.862375+0000","last_undegraded":"2024-09-21T11:59:27.862375+0000","last_fullsized":"2024-09-21T11:59:27.862375+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:50.548003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:50.548003+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:24.779078+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"454'12377","reported_seq":12513,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.173542+0000","last_change":"2024-09-21T11:56:36.509094+0000","last_active":"2024-09-21T11:59:31.173542+0000","last_peered":"2024-09-21T11:59:31.173542+0000","last_clean":"2024-09-21T11:59:31.173542+0000","last_became_active":"2024-09-21T11:56:36.508840+0000","last_became_peered":"2024-09-21T11:56:36.508840+0000","last_unstale":"2024-09-21T11:59:31.173542+0000","last_undegraded":"2024-09-21T11:59:31.173542+0000","last_fullsized":"2024-09-21T11:59:31.173542+0000","mapping_epoch":392,"log_start":"426'7695","ondisk_log_start":"426'7695","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4682,"log_dups_size":0,"ondisk_log_size":4682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":186712064,"num_objects":2849,"num_object_clones":0,"num_object_copies":8547,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12377,"num_write_kb":487232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"269'17","reported_seq":541,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.851396+0000","last_change":"2024-09-21T11:59:04.130363+0000","last_active":"2024-09-21T11:59:27.851396+0000","last_peered":"2024-09-21T11:59:27.851396+0000","last_clean":"2024-09-21T11:59:27.851396+0000","last_became_active":"2024-09-21T11:57:58.530507+0000","last_became_peered":"2024-09-21T11:57:58.530507+0000","last_unstale":"2024-09-21T11:59:27.851396+0000","last_undegraded":"2024-09-21T11:59:27.851396+0000","last_fullsized":"2024-09-21T11:59:27.851396+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'17","last_scrub_stamp":"2024-09-21T11:59:04.130113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:04.130113+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:17.734192+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":556,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.853670+0000","last_change":"2024-09-21T11:57:43.583962+0000","last_active":"2024-09-21T11:59:27.853670+0000","last_peered":"2024-09-21T11:59:27.853670+0000","last_clean":"2024-09-21T11:59:27.853670+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T11:59:27.853670+0000","last_undegraded":"2024-09-21T11:59:27.853670+0000","last_fullsized":"2024-09-21T11:59:27.853670+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:57:43.583736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.583736+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":876,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.850721+0000","last_change":"2024-09-21T11:53:20.956563+0000","last_active":"2024-09-21T11:59:27.850721+0000","last_peered":"2024-09-21T11:59:27.850721+0000","last_clean":"2024-09-21T11:59:27.850721+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T11:59:27.850721+0000","last_undegraded":"2024-09-21T11:59:27.850721+0000","last_fullsized":"2024-09-21T11:59:27.850721+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T11:53:20.956479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:53:20.956479+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.044139+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":858,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.857830+0000","last_change":"2024-09-21T11:59:09.091725+0000","last_active":"2024-09-21T11:59:27.857830+0000","last_peered":"2024-09-21T11:59:27.857830+0000","last_clean":"2024-09-21T11:59:27.857830+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T11:59:27.857830+0000","last_undegraded":"2024-09-21T11:59:27.857830+0000","last_fullsized":"2024-09-21T11:59:27.857830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:09.091520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:59:09.091520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:10.832593+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":862,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863143+0000","last_change":"2024-09-21T11:58:30.226389+0000","last_active":"2024-09-21T11:59:27.863143+0000","last_peered":"2024-09-21T11:59:27.863143+0000","last_clean":"2024-09-21T11:59:27.863143+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T11:59:27.863143+0000","last_undegraded":"2024-09-21T11:59:27.863143+0000","last_fullsized":"2024-09-21T11:59:27.863143+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:30.226345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:30.226345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:50.129480+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":846,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.852477+0000","last_change":"2024-09-21T11:58:30.794254+0000","last_active":"2024-09-21T11:59:27.852477+0000","last_peered":"2024-09-21T11:59:27.852477+0000","last_clean":"2024-09-21T11:59:27.852477+0000","last_became_active":"2024-09-21T11:58:30.793717+0000","last_became_peered":"2024-09-21T11:58:30.793717+0000","last_unstale":"2024-09-21T11:59:27.852477+0000","last_undegraded":"2024-09-21T11:59:27.852477+0000","last_fullsized":"2024-09-21T11:59:27.852477+0000","mapping_epoch":435,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":436,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:48.627711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.627711+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.9","version":"454'12437","reported_seq":22499,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.170105+0000","last_change":"2024-09-21T11:59:19.581217+0000","last_active":"2024-09-21T11:59:31.170105+0000","last_peered":"2024-09-21T11:59:31.170105+0000","last_clean":"2024-09-21T11:59:31.170105+0000","last_became_active":"2024-09-21T11:59:19.559837+0000","last_became_peered":"2024-09-21T11:59:19.559837+0000","last_unstale":"2024-09-21T11:59:31.170105+0000","last_undegraded":"2024-09-21T11:59:31.170105+0000","last_fullsized":"2024-09-21T11:59:31.170105+0000","mapping_epoch":450,"log_start":"426'7745","ondisk_log_start":"426'7745","created":392,"last_epoch_clean":451,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4692,"log_dups_size":0,"ondisk_log_size":4692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190644224,"num_objects":2909,"num_object_clones":0,"num_object_copies":8727,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12437,"num_write_kb":491072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4953,"num_bytes_recovered":324599808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"269'24","reported_seq":92,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862683+0000","last_change":"2024-09-21T11:57:58.545640+0000","last_active":"2024-09-21T11:59:27.862683+0000","last_peered":"2024-09-21T11:59:27.862683+0000","last_clean":"2024-09-21T11:59:27.862683+0000","last_became_active":"2024-09-21T11:57:58.527381+0000","last_became_peered":"2024-09-21T11:57:58.527381+0000","last_unstale":"2024-09-21T11:59:27.862683+0000","last_undegraded":"2024-09-21T11:59:27.862683+0000","last_fullsized":"2024-09-21T11:59:27.862683+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'24","last_scrub_stamp":"2024-09-21T11:57:54.679272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.679272+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":888,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.869618+0000","last_change":"2024-09-21T11:58:17.528448+0000","last_active":"2024-09-21T11:59:27.869618+0000","last_peered":"2024-09-21T11:59:27.869618+0000","last_clean":"2024-09-21T11:59:27.869618+0000","last_became_active":"2024-09-21T11:58:17.528154+0000","last_became_peered":"2024-09-21T11:58:17.528154+0000","last_unstale":"2024-09-21T11:59:27.869618+0000","last_undegraded":"2024-09-21T11:59:27.869618+0000","last_fullsized":"2024-09-21T11:59:27.869618+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:01.083437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.083437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":2062,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:29.737251+0000","last_change":"2024-09-21T11:56:09.604887+0000","last_active":"2024-09-21T11:59:29.737251+0000","last_peered":"2024-09-21T11:59:29.737251+0000","last_clean":"2024-09-21T11:59:29.737251+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T11:59:29.737251+0000","last_undegraded":"2024-09-21T11:59:29.737251+0000","last_fullsized":"2024-09-21T11:59:29.737251+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T11:56:09.604808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.604808+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.102017+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1161,"num_read_kb":1156,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":871,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.883192+0000","last_change":"2024-09-21T11:59:11.972760+0000","last_active":"2024-09-21T11:59:27.883192+0000","last_peered":"2024-09-21T11:59:27.883192+0000","last_clean":"2024-09-21T11:59:27.883192+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T11:59:27.883192+0000","last_undegraded":"2024-09-21T11:59:27.883192+0000","last_fullsized":"2024-09-21T11:59:27.883192+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:11.972680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:59:11.972680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:33.037868+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":543,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.858803+0000","last_change":"2024-09-21T11:52:38.154347+0000","last_active":"2024-09-21T11:59:27.858803+0000","last_peered":"2024-09-21T11:59:27.858803+0000","last_clean":"2024-09-21T11:59:27.858803+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T11:59:27.858803+0000","last_undegraded":"2024-09-21T11:59:27.858803+0000","last_fullsized":"2024-09-21T11:59:27.858803+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:38.154301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:38.154301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:23.782082+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"454'12255","reported_seq":12391,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.181696+0000","last_change":"2024-09-21T11:56:36.509270+0000","last_active":"2024-09-21T11:59:31.181696+0000","last_peered":"2024-09-21T11:59:31.181696+0000","last_clean":"2024-09-21T11:59:31.181696+0000","last_became_active":"2024-09-21T11:56:36.508533+0000","last_became_peered":"2024-09-21T11:56:36.508533+0000","last_unstale":"2024-09-21T11:59:31.181696+0000","last_undegraded":"2024-09-21T11:59:31.181696+0000","last_fullsized":"2024-09-21T11:59:31.181696+0000","mapping_epoch":392,"log_start":"426'7622","ondisk_log_start":"426'7622","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":4633,"log_dups_size":0,"ondisk_log_size":4633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185663488,"num_objects":2833,"num_object_clones":0,"num_object_copies":8499,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12255,"num_write_kb":482816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"269'28","reported_seq":1137,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860946+0000","last_change":"2024-09-21T11:59:05.905840+0000","last_active":"2024-09-21T11:59:27.860946+0000","last_peered":"2024-09-21T11:59:27.860946+0000","last_clean":"2024-09-21T11:59:27.860946+0000","last_became_active":"2024-09-21T11:40:02.398855+0000","last_became_peered":"2024-09-21T11:40:02.398855+0000","last_unstale":"2024-09-21T11:59:27.860946+0000","last_undegraded":"2024-09-21T11:59:27.860946+0000","last_fullsized":"2024-09-21T11:59:27.860946+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'28","last_scrub_stamp":"2024-09-21T11:59:05.905765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:05.905765+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:34.307948+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":859,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.867009+0000","last_change":"2024-09-21T11:57:14.756298+0000","last_active":"2024-09-21T11:59:27.867009+0000","last_peered":"2024-09-21T11:59:27.867009+0000","last_clean":"2024-09-21T11:59:27.867009+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T11:59:27.867009+0000","last_undegraded":"2024-09-21T11:59:27.867009+0000","last_fullsized":"2024-09-21T11:59:27.867009+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:14.756227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:14.756227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":977,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862652+0000","last_change":"2024-09-21T11:58:54.745261+0000","last_active":"2024-09-21T11:59:27.862652+0000","last_peered":"2024-09-21T11:59:27.862652+0000","last_clean":"2024-09-21T11:59:27.862652+0000","last_became_active":"2024-09-21T11:57:58.528825+0000","last_became_peered":"2024-09-21T11:57:58.528825+0000","last_unstale":"2024-09-21T11:59:27.862652+0000","last_undegraded":"2024-09-21T11:59:27.862652+0000","last_fullsized":"2024-09-21T11:59:27.862652+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:58:54.745202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:58:54.745202+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:08.610072+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":850,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863227+0000","last_change":"2024-09-21T11:58:28.274182+0000","last_active":"2024-09-21T11:59:27.863227+0000","last_peered":"2024-09-21T11:59:27.863227+0000","last_clean":"2024-09-21T11:59:27.863227+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T11:59:27.863227+0000","last_undegraded":"2024-09-21T11:59:27.863227+0000","last_fullsized":"2024-09-21T11:59:27.863227+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:28.274120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.274120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:44.924347+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":861,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.883058+0000","last_change":"2024-09-21T11:57:05.904684+0000","last_active":"2024-09-21T11:59:27.883058+0000","last_peered":"2024-09-21T11:59:27.883058+0000","last_clean":"2024-09-21T11:59:27.883058+0000","last_became_active":"2024-09-21T11:40:06.349587+0000","last_became_peered":"2024-09-21T11:40:06.349587+0000","last_unstale":"2024-09-21T11:59:27.883058+0000","last_undegraded":"2024-09-21T11:59:27.883058+0000","last_fullsized":"2024-09-21T11:59:27.883058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:05.904621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:05.904621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:46.758787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"454'12330","reported_seq":12514,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:30.906017+0000","last_change":"2024-09-21T11:59:16.343176+0000","last_active":"2024-09-21T11:59:30.906017+0000","last_peered":"2024-09-21T11:59:30.906017+0000","last_clean":"2024-09-21T11:59:30.906017+0000","last_became_active":"2024-09-21T11:56:36.507849+0000","last_became_peered":"2024-09-21T11:56:36.507849+0000","last_unstale":"2024-09-21T11:59:30.906017+0000","last_undegraded":"2024-09-21T11:59:30.906017+0000","last_fullsized":"2024-09-21T11:59:30.906017+0000","mapping_epoch":392,"log_start":"423'6904","ondisk_log_start":"423'6904","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'11497","last_scrub_stamp":"2024-09-21T11:59:16.342973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:16.342973+0000","objects_scrubbed":3687,"log_size":5426,"log_dups_size":0,"ondisk_log_size":5426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:18.053609+0000","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183238656,"num_objects":2796,"num_object_clones":0,"num_object_copies":8388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12330,"num_write_kb":484032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"269'19","reported_seq":883,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860685+0000","last_change":"2024-09-21T11:58:56.825792+0000","last_active":"2024-09-21T11:59:27.860685+0000","last_peered":"2024-09-21T11:59:27.860685+0000","last_clean":"2024-09-21T11:59:27.860685+0000","last_became_active":"2024-09-21T11:51:43.526433+0000","last_became_peered":"2024-09-21T11:51:43.526433+0000","last_unstale":"2024-09-21T11:59:27.860685+0000","last_undegraded":"2024-09-21T11:59:27.860685+0000","last_fullsized":"2024-09-21T11:59:27.860685+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:56.825707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:56.825707+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:07.271040+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":366,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.850573+0000","last_change":"2024-09-21T11:58:17.529749+0000","last_active":"2024-09-21T11:59:27.850573+0000","last_peered":"2024-09-21T11:59:27.850573+0000","last_clean":"2024-09-21T11:59:27.850573+0000","last_became_active":"2024-09-21T11:58:17.529395+0000","last_became_peered":"2024-09-21T11:58:17.529395+0000","last_unstale":"2024-09-21T11:59:27.850573+0000","last_undegraded":"2024-09-21T11:59:27.850573+0000","last_fullsized":"2024-09-21T11:59:27.850573+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:05.663102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:05.663102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:47.080734+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":863,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.882914+0000","last_change":"2024-09-21T11:58:29.807560+0000","last_active":"2024-09-21T11:59:27.882914+0000","last_peered":"2024-09-21T11:59:27.882914+0000","last_clean":"2024-09-21T11:59:27.882914+0000","last_became_active":"2024-09-21T11:58:17.527945+0000","last_became_peered":"2024-09-21T11:58:17.527945+0000","last_unstale":"2024-09-21T11:59:27.882914+0000","last_undegraded":"2024-09-21T11:59:27.882914+0000","last_fullsized":"2024-09-21T11:59:27.882914+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:29.807504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:29.807504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:58.143758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":852,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863447+0000","last_change":"2024-09-21T11:56:28.556475+0000","last_active":"2024-09-21T11:59:27.863447+0000","last_peered":"2024-09-21T11:59:27.863447+0000","last_clean":"2024-09-21T11:59:27.863447+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T11:59:27.863447+0000","last_undegraded":"2024-09-21T11:59:27.863447+0000","last_fullsized":"2024-09-21T11:59:27.863447+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:28.556423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:28.556423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":101,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.884661+0000","last_change":"2024-09-21T11:57:54.931806+0000","last_active":"2024-09-21T11:59:27.884661+0000","last_peered":"2024-09-21T11:59:27.884661+0000","last_clean":"2024-09-21T11:59:27.884661+0000","last_became_active":"2024-09-21T11:57:45.534494+0000","last_became_peered":"2024-09-21T11:57:45.534494+0000","last_unstale":"2024-09-21T11:59:27.884661+0000","last_undegraded":"2024-09-21T11:59:27.884661+0000","last_fullsized":"2024-09-21T11:59:27.884661+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:57:54.931704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:57:54.931704+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:23.897340+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2","version":"454'12337","reported_seq":12527,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.990807+0000","last_change":"2024-09-21T11:59:01.566946+0000","last_active":"2024-09-21T11:59:31.990807+0000","last_peered":"2024-09-21T11:59:31.990807+0000","last_clean":"2024-09-21T11:59:31.990807+0000","last_became_active":"2024-09-21T11:56:36.511642+0000","last_became_peered":"2024-09-21T11:56:36.511642+0000","last_unstale":"2024-09-21T11:59:31.990807+0000","last_undegraded":"2024-09-21T11:59:31.990807+0000","last_fullsized":"2024-09-21T11:59:31.990807+0000","mapping_epoch":392,"log_start":"418'6197","ondisk_log_start":"418'6197","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"443'10535","last_scrub_stamp":"2024-09-21T11:59:01.566731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:01.566731+0000","objects_scrubbed":4558,"log_size":6140,"log_dups_size":0,"ondisk_log_size":6140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:17.985934+0000","scrub_duration":2440,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176488448,"num_objects":2693,"num_object_clones":0,"num_object_copies":8079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12337,"num_write_kb":480960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"269'31","reported_seq":980,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.856889+0000","last_change":"2024-09-21T11:58:17.555274+0000","last_active":"2024-09-21T11:59:27.856889+0000","last_peered":"2024-09-21T11:59:27.856889+0000","last_clean":"2024-09-21T11:59:27.856889+0000","last_became_active":"2024-09-21T11:58:17.527029+0000","last_became_peered":"2024-09-21T11:58:17.527029+0000","last_unstale":"2024-09-21T11:59:27.856889+0000","last_undegraded":"2024-09-21T11:59:27.856889+0000","last_fullsized":"2024-09-21T11:59:27.856889+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'31","last_scrub_stamp":"2024-09-21T11:56:03.667750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:03.667750+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:37.084768+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":887,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862550+0000","last_change":"2024-09-21T11:59:02.898115+0000","last_active":"2024-09-21T11:59:27.862550+0000","last_peered":"2024-09-21T11:59:27.862550+0000","last_clean":"2024-09-21T11:59:27.862550+0000","last_became_active":"2024-09-21T11:47:03.364435+0000","last_became_peered":"2024-09-21T11:47:03.364435+0000","last_unstale":"2024-09-21T11:59:27.862550+0000","last_undegraded":"2024-09-21T11:59:27.862550+0000","last_fullsized":"2024-09-21T11:59:27.862550+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:02.898000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:59:02.898000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:26.416721+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":551,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.858890+0000","last_change":"2024-09-21T11:57:42.599025+0000","last_active":"2024-09-21T11:59:27.858890+0000","last_peered":"2024-09-21T11:59:27.858890+0000","last_clean":"2024-09-21T11:59:27.858890+0000","last_became_active":"2024-09-21T11:47:03.374781+0000","last_became_peered":"2024-09-21T11:47:03.374781+0000","last_unstale":"2024-09-21T11:59:27.858890+0000","last_undegraded":"2024-09-21T11:59:27.858890+0000","last_fullsized":"2024-09-21T11:59:27.858890+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:42.598710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:42.598710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":862,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.851641+0000","last_change":"2024-09-21T11:59:16.184608+0000","last_active":"2024-09-21T11:59:27.851641+0000","last_peered":"2024-09-21T11:59:27.851641+0000","last_clean":"2024-09-21T11:59:27.851641+0000","last_became_active":"2024-09-21T11:40:06.348750+0000","last_became_peered":"2024-09-21T11:40:06.348750+0000","last_unstale":"2024-09-21T11:59:27.851641+0000","last_undegraded":"2024-09-21T11:59:27.851641+0000","last_fullsized":"2024-09-21T11:59:27.851641+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:16.184540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:16.184540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:21.327851+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":975,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862415+0000","last_change":"2024-09-21T11:59:10.608778+0000","last_active":"2024-09-21T11:59:27.862415+0000","last_peered":"2024-09-21T11:59:27.862415+0000","last_clean":"2024-09-21T11:59:27.862415+0000","last_became_active":"2024-09-21T11:57:45.516610+0000","last_became_peered":"2024-09-21T11:57:45.516610+0000","last_unstale":"2024-09-21T11:59:27.862415+0000","last_undegraded":"2024-09-21T11:59:27.862415+0000","last_fullsized":"2024-09-21T11:59:27.862415+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T11:59:10.608731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T11:59:10.608731+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:38.999852+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3","version":"454'12241","reported_seq":12457,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:30.879831+0000","last_change":"2024-09-21T11:58:22.114392+0000","last_active":"2024-09-21T11:59:30.879831+0000","last_peered":"2024-09-21T11:59:30.879831+0000","last_clean":"2024-09-21T11:59:30.879831+0000","last_became_active":"2024-09-21T11:56:36.504516+0000","last_became_peered":"2024-09-21T11:56:36.504516+0000","last_unstale":"2024-09-21T11:59:30.879831+0000","last_undegraded":"2024-09-21T11:59:30.879831+0000","last_fullsized":"2024-09-21T11:59:30.879831+0000","mapping_epoch":392,"log_start":"419'6795","ondisk_log_start":"419'6795","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"432'8216","last_scrub_stamp":"2024-09-21T11:58:22.114283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:58:22.114283+0000","objects_scrubbed":7071,"log_size":5446,"log_dups_size":0,"ondisk_log_size":5446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:43.111443+0000","scrub_duration":2344,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195624960,"num_objects":2985,"num_object_clones":0,"num_object_copies":8955,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12241,"num_write_kb":487232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"269'20","reported_seq":836,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863765+0000","last_change":"2024-09-21T11:51:43.549155+0000","last_active":"2024-09-21T11:59:27.863765+0000","last_peered":"2024-09-21T11:59:27.863765+0000","last_clean":"2024-09-21T11:59:27.863765+0000","last_became_active":"2024-09-21T11:51:43.548096+0000","last_became_peered":"2024-09-21T11:51:43.548096+0000","last_unstale":"2024-09-21T11:59:27.863765+0000","last_undegraded":"2024-09-21T11:59:27.863765+0000","last_fullsized":"2024-09-21T11:59:27.863765+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-21T11:50:06.224944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:06.224944+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:20.087582+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,6],"acting":[3,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":856,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863171+0000","last_change":"2024-09-21T11:57:58.535943+0000","last_active":"2024-09-21T11:59:27.863171+0000","last_peered":"2024-09-21T11:59:27.863171+0000","last_clean":"2024-09-21T11:59:27.863171+0000","last_became_active":"2024-09-21T11:57:58.535642+0000","last_became_peered":"2024-09-21T11:57:58.535642+0000","last_unstale":"2024-09-21T11:59:27.863171+0000","last_undegraded":"2024-09-21T11:59:27.863171+0000","last_fullsized":"2024-09-21T11:59:27.863171+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:09.486386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:09.486386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1101,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.857194+0000","last_change":"2024-09-21T11:56:08.680971+0000","last_active":"2024-09-21T11:59:27.857194+0000","last_peered":"2024-09-21T11:59:27.857194+0000","last_clean":"2024-09-21T11:59:27.857194+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T11:59:27.857194+0000","last_undegraded":"2024-09-21T11:59:27.857194+0000","last_fullsized":"2024-09-21T11:59:27.857194+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:56:08.680766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:08.680766+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:46.070618+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":685,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.859767+0000","last_change":"2024-09-21T11:58:17.532859+0000","last_active":"2024-09-21T11:59:27.859767+0000","last_peered":"2024-09-21T11:59:27.859767+0000","last_clean":"2024-09-21T11:59:27.859767+0000","last_became_active":"2024-09-21T11:58:17.531268+0000","last_became_peered":"2024-09-21T11:58:17.531268+0000","last_unstale":"2024-09-21T11:59:27.859767+0000","last_undegraded":"2024-09-21T11:59:27.859767+0000","last_fullsized":"2024-09-21T11:59:27.859767+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:51.665137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:51.665137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"454'12243","reported_seq":12424,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.341814+0000","last_change":"2024-09-21T11:59:13.679316+0000","last_active":"2024-09-21T11:59:31.341814+0000","last_peered":"2024-09-21T11:59:31.341814+0000","last_clean":"2024-09-21T11:59:31.341814+0000","last_became_active":"2024-09-21T11:56:36.508299+0000","last_became_peered":"2024-09-21T11:56:36.508299+0000","last_unstale":"2024-09-21T11:59:31.341814+0000","last_undegraded":"2024-09-21T11:59:31.341814+0000","last_fullsized":"2024-09-21T11:59:31.341814+0000","mapping_epoch":392,"log_start":"426'7415","ondisk_log_start":"426'7415","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'11214","last_scrub_stamp":"2024-09-21T11:59:13.679136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:13.679136+0000","objects_scrubbed":3897,"log_size":4828,"log_dups_size":0,"ondisk_log_size":4828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:20.950833+0000","scrub_duration":1075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185925632,"num_objects":2837,"num_object_clones":0,"num_object_copies":8511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12243,"num_write_kb":482560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"269'15","reported_seq":896,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863585+0000","last_change":"2024-09-21T11:56:48.169371+0000","last_active":"2024-09-21T11:59:27.863585+0000","last_peered":"2024-09-21T11:59:27.863585+0000","last_clean":"2024-09-21T11:59:27.863585+0000","last_became_active":"2024-09-21T11:47:03.385341+0000","last_became_peered":"2024-09-21T11:47:03.385341+0000","last_unstale":"2024-09-21T11:59:27.863585+0000","last_undegraded":"2024-09-21T11:59:27.863585+0000","last_fullsized":"2024-09-21T11:59:27.863585+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:56:48.168795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:48.168795+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":889,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.882950+0000","last_change":"2024-09-21T11:59:12.932516+0000","last_active":"2024-09-21T11:59:27.882950+0000","last_peered":"2024-09-21T11:59:27.882950+0000","last_clean":"2024-09-21T11:59:27.882950+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T11:59:27.882950+0000","last_undegraded":"2024-09-21T11:59:27.882950+0000","last_fullsized":"2024-09-21T11:59:27.882950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:12.932440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:59:12.932440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:25.638974+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":856,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.856626+0000","last_change":"2024-09-21T11:58:21.147985+0000","last_active":"2024-09-21T11:59:27.856626+0000","last_peered":"2024-09-21T11:59:27.856626+0000","last_clean":"2024-09-21T11:59:27.856626+0000","last_became_active":"2024-09-21T11:58:17.525791+0000","last_became_peered":"2024-09-21T11:58:17.525791+0000","last_unstale":"2024-09-21T11:59:27.856626+0000","last_undegraded":"2024-09-21T11:59:27.856626+0000","last_fullsized":"2024-09-21T11:59:27.856626+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:21.147942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:21.147942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:41.780615+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":78,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.869532+0000","last_change":"2024-09-21T11:57:58.529616+0000","last_active":"2024-09-21T11:59:27.869532+0000","last_peered":"2024-09-21T11:59:27.869532+0000","last_clean":"2024-09-21T11:59:27.869532+0000","last_became_active":"2024-09-21T11:57:58.529278+0000","last_became_peered":"2024-09-21T11:57:58.529278+0000","last_unstale":"2024-09-21T11:59:27.869532+0000","last_undegraded":"2024-09-21T11:59:27.869532+0000","last_fullsized":"2024-09-21T11:59:27.869532+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:35.532577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:35.532577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"454'12116","reported_seq":12301,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.334304+0000","last_change":"2024-09-21T11:59:08.338604+0000","last_active":"2024-09-21T11:59:31.334304+0000","last_peered":"2024-09-21T11:59:31.334304+0000","last_clean":"2024-09-21T11:59:31.334304+0000","last_became_active":"2024-09-21T11:56:36.506694+0000","last_became_peered":"2024-09-21T11:56:36.506694+0000","last_unstale":"2024-09-21T11:59:31.334304+0000","last_undegraded":"2024-09-21T11:59:31.334304+0000","last_fullsized":"2024-09-21T11:59:31.334304+0000","mapping_epoch":392,"log_start":"426'7319","ondisk_log_start":"426'7319","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"447'10791","last_scrub_stamp":"2024-09-21T11:59:08.338518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.338518+0000","objects_scrubbed":4166,"log_size":4797,"log_dups_size":0,"ondisk_log_size":4797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:31.958551+0000","scrub_duration":707,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185204736,"num_objects":2826,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12116,"num_write_kb":478144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"269'35","reported_seq":962,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862247+0000","last_change":"2024-09-21T11:58:29.624654+0000","last_active":"2024-09-21T11:59:27.862247+0000","last_peered":"2024-09-21T11:59:27.862247+0000","last_clean":"2024-09-21T11:59:27.862247+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T11:59:27.862247+0000","last_undegraded":"2024-09-21T11:59:27.862247+0000","last_fullsized":"2024-09-21T11:59:27.862247+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'35","last_scrub_stamp":"2024-09-21T11:58:29.624549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:29.624549+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:48.198468+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":857,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863428+0000","last_change":"2024-09-21T11:59:17.687953+0000","last_active":"2024-09-21T11:59:27.863428+0000","last_peered":"2024-09-21T11:59:27.863428+0000","last_clean":"2024-09-21T11:59:27.863428+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T11:59:27.863428+0000","last_undegraded":"2024-09-21T11:59:27.863428+0000","last_fullsized":"2024-09-21T11:59:27.863428+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:17.687904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:59:17.687904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:21.256703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":858,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.850877+0000","last_change":"2024-09-21T11:58:28.041091+0000","last_active":"2024-09-21T11:59:27.850877+0000","last_peered":"2024-09-21T11:59:27.850877+0000","last_clean":"2024-09-21T11:59:27.850877+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T11:59:27.850877+0000","last_undegraded":"2024-09-21T11:59:27.850877+0000","last_fullsized":"2024-09-21T11:59:27.850877+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:28.041026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.041026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:52.616083+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":861,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862432+0000","last_change":"2024-09-21T11:59:08.676283+0000","last_active":"2024-09-21T11:59:27.862432+0000","last_peered":"2024-09-21T11:59:27.862432+0000","last_clean":"2024-09-21T11:59:27.862432+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T11:59:27.862432+0000","last_undegraded":"2024-09-21T11:59:27.862432+0000","last_fullsized":"2024-09-21T11:59:27.862432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:08.676235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.676235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:13.547786+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"454'12436","reported_seq":25345,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:32.624935+0000","last_change":"2024-09-21T11:59:02.700721+0000","last_active":"2024-09-21T11:59:32.624935+0000","last_peered":"2024-09-21T11:59:32.624935+0000","last_clean":"2024-09-21T11:59:32.624935+0000","last_became_active":"2024-09-21T11:59:02.673387+0000","last_became_peered":"2024-09-21T11:59:02.673387+0000","last_unstale":"2024-09-21T11:59:32.624935+0000","last_undegraded":"2024-09-21T11:59:32.624935+0000","last_fullsized":"2024-09-21T11:59:32.624935+0000","mapping_epoch":444,"log_start":"424'7115","ondisk_log_start":"424'7115","created":392,"last_epoch_clean":445,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:56:35.484863+0000","objects_scrubbed":0,"log_size":5321,"log_dups_size":0,"ondisk_log_size":5321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180224000,"num_objects":2750,"num_object_clones":0,"num_object_copies":8250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12436,"num_write_kb":485952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6378,"num_bytes_recovered":417988608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"269'18","reported_seq":876,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.855216+0000","last_change":"2024-09-21T11:51:43.538886+0000","last_active":"2024-09-21T11:59:27.855216+0000","last_peered":"2024-09-21T11:59:27.855216+0000","last_clean":"2024-09-21T11:59:27.855216+0000","last_became_active":"2024-09-21T11:51:43.538471+0000","last_became_peered":"2024-09-21T11:51:43.538471+0000","last_unstale":"2024-09-21T11:59:27.855216+0000","last_undegraded":"2024-09-21T11:59:27.855216+0000","last_fullsized":"2024-09-21T11:59:27.855216+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'18","last_scrub_stamp":"2024-09-21T11:50:34.362744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:50:34.362744+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:20.751918+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863395+0000","last_change":"2024-09-21T11:58:18.420036+0000","last_active":"2024-09-21T11:59:27.863395+0000","last_peered":"2024-09-21T11:59:27.863395+0000","last_clean":"2024-09-21T11:59:27.863395+0000","last_became_active":"2024-09-21T11:58:17.530676+0000","last_became_peered":"2024-09-21T11:58:17.530676+0000","last_unstale":"2024-09-21T11:59:27.863395+0000","last_undegraded":"2024-09-21T11:59:27.863395+0000","last_fullsized":"2024-09-21T11:59:27.863395+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:18.419990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:58:18.419990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":864,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.853180+0000","last_change":"2024-09-21T11:54:01.146126+0000","last_active":"2024-09-21T11:59:27.853180+0000","last_peered":"2024-09-21T11:59:27.853180+0000","last_clean":"2024-09-21T11:59:27.853180+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T11:59:27.853180+0000","last_undegraded":"2024-09-21T11:59:27.853180+0000","last_fullsized":"2024-09-21T11:59:27.853180+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:54:01.145859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:54:01.145859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:36.052833+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":852,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862827+0000","last_change":"2024-09-21T11:56:02.541452+0000","last_active":"2024-09-21T11:59:27.862827+0000","last_peered":"2024-09-21T11:59:27.862827+0000","last_clean":"2024-09-21T11:59:27.862827+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T11:59:27.862827+0000","last_undegraded":"2024-09-21T11:59:27.862827+0000","last_fullsized":"2024-09-21T11:59:27.862827+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:02.541232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:02.541232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"454'12056","reported_seq":12241,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:31.504505+0000","last_change":"2024-09-21T11:59:11.044768+0000","last_active":"2024-09-21T11:59:31.504505+0000","last_peered":"2024-09-21T11:59:31.504505+0000","last_clean":"2024-09-21T11:59:31.504505+0000","last_became_active":"2024-09-21T11:56:36.623765+0000","last_became_peered":"2024-09-21T11:56:36.623765+0000","last_unstale":"2024-09-21T11:59:31.504505+0000","last_undegraded":"2024-09-21T11:59:31.504505+0000","last_fullsized":"2024-09-21T11:59:31.504505+0000","mapping_epoch":392,"log_start":"423'6799","ondisk_log_start":"423'6799","created":392,"last_epoch_clean":393,"parent":"0.0","parent_split_bits":0,"last_scrub":"448'10886","last_scrub_stamp":"2024-09-21T11:59:11.044452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:56:35.484863+0000","last_clean_scrub_stamp":"2024-09-21T11:59:11.044452+0000","objects_scrubbed":3980,"log_size":5257,"log_dups_size":0,"ondisk_log_size":5257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:23.581441+0000","scrub_duration":1112,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182190080,"num_objects":2780,"num_object_clones":0,"num_object_copies":8340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12056,"num_write_kb":474752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"269'27","reported_seq":904,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.885040+0000","last_change":"2024-09-21T11:59:08.919308+0000","last_active":"2024-09-21T11:59:27.885040+0000","last_peered":"2024-09-21T11:59:27.885040+0000","last_clean":"2024-09-21T11:59:27.885040+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T11:59:27.885040+0000","last_undegraded":"2024-09-21T11:59:27.885040+0000","last_fullsized":"2024-09-21T11:59:27.885040+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'27","last_scrub_stamp":"2024-09-21T11:59:08.919232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.919232+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:19.454285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":911,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863313+0000","last_change":"2024-09-21T11:57:58.539544+0000","last_active":"2024-09-21T11:59:27.863313+0000","last_peered":"2024-09-21T11:59:27.863313+0000","last_clean":"2024-09-21T11:59:27.863313+0000","last_became_active":"2024-09-21T11:57:58.528814+0000","last_became_peered":"2024-09-21T11:57:58.528814+0000","last_unstale":"2024-09-21T11:59:27.863313+0000","last_undegraded":"2024-09-21T11:59:27.863313+0000","last_fullsized":"2024-09-21T11:59:27.863313+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T11:57:01.221960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:01.221960+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.207576+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1100,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862462+0000","last_change":"2024-09-21T11:57:10.282180+0000","last_active":"2024-09-21T11:59:27.862462+0000","last_peered":"2024-09-21T11:59:27.862462+0000","last_clean":"2024-09-21T11:59:27.862462+0000","last_became_active":"2024-09-21T11:40:04.331716+0000","last_became_peered":"2024-09-21T11:40:04.331716+0000","last_unstale":"2024-09-21T11:59:27.862462+0000","last_undegraded":"2024-09-21T11:59:27.862462+0000","last_fullsized":"2024-09-21T11:59:27.862462+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:57:10.282134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:10.282134+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:18.729887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":845,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.882323+0000","last_change":"2024-09-21T11:59:13.897600+0000","last_active":"2024-09-21T11:59:27.882323+0000","last_peered":"2024-09-21T11:59:27.882323+0000","last_clean":"2024-09-21T11:59:27.882323+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T11:59:27.882323+0000","last_undegraded":"2024-09-21T11:59:27.882323+0000","last_fullsized":"2024-09-21T11:59:27.882323+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:13.897540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:59:13.897540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:34.214481+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":887,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860479+0000","last_change":"2024-09-21T11:57:58.527427+0000","last_active":"2024-09-21T11:59:27.860479+0000","last_peered":"2024-09-21T11:59:27.860479+0000","last_clean":"2024-09-21T11:59:27.860479+0000","last_became_active":"2024-09-21T11:57:58.527156+0000","last_became_peered":"2024-09-21T11:57:58.527156+0000","last_unstale":"2024-09-21T11:59:27.860479+0000","last_undegraded":"2024-09-21T11:59:27.860479+0000","last_fullsized":"2024-09-21T11:59:27.860479+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:38.938666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:38.938666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:49.629819+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"269'12","reported_seq":886,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.864212+0000","last_change":"2024-09-21T11:56:29.528662+0000","last_active":"2024-09-21T11:59:27.864212+0000","last_peered":"2024-09-21T11:59:27.864212+0000","last_clean":"2024-09-21T11:59:27.864212+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T11:59:27.864212+0000","last_undegraded":"2024-09-21T11:59:27.864212+0000","last_fullsized":"2024-09-21T11:59:27.864212+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'12","last_scrub_stamp":"2024-09-21T11:56:29.528598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:56:29.528598+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":895,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.883234+0000","last_change":"2024-09-21T11:58:56.841707+0000","last_active":"2024-09-21T11:59:27.883234+0000","last_peered":"2024-09-21T11:59:27.883234+0000","last_clean":"2024-09-21T11:59:27.883234+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T11:59:27.883234+0000","last_undegraded":"2024-09-21T11:59:27.883234+0000","last_fullsized":"2024-09-21T11:59:27.883234+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:56.841648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:56.841648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:06.233795+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":883,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863123+0000","last_change":"2024-09-21T11:58:20.347779+0000","last_active":"2024-09-21T11:59:27.863123+0000","last_peered":"2024-09-21T11:59:27.863123+0000","last_clean":"2024-09-21T11:59:27.863123+0000","last_became_active":"2024-09-21T11:58:17.529420+0000","last_became_peered":"2024-09-21T11:58:17.529420+0000","last_unstale":"2024-09-21T11:59:27.863123+0000","last_undegraded":"2024-09-21T11:59:27.863123+0000","last_fullsized":"2024-09-21T11:59:27.863123+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:20.347736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:20.347736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:47.925674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"269'15","reported_seq":879,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.867440+0000","last_change":"2024-09-21T11:53:29.499330+0000","last_active":"2024-09-21T11:59:27.867440+0000","last_peered":"2024-09-21T11:59:27.867440+0000","last_clean":"2024-09-21T11:59:27.867440+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T11:59:27.867440+0000","last_undegraded":"2024-09-21T11:59:27.867440+0000","last_fullsized":"2024-09-21T11:59:27.867440+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:53:29.499250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:53:29.499250+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":841,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862562+0000","last_change":"2024-09-21T11:51:42.236644+0000","last_active":"2024-09-21T11:59:27.862562+0000","last_peered":"2024-09-21T11:59:27.862562+0000","last_clean":"2024-09-21T11:59:27.862562+0000","last_became_active":"2024-09-21T11:40:00.308660+0000","last_became_peered":"2024-09-21T11:40:00.308660+0000","last_unstale":"2024-09-21T11:59:27.862562+0000","last_undegraded":"2024-09-21T11:59:27.862562+0000","last_fullsized":"2024-09-21T11:59:27.862562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:51:42.236496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:51:42.236496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:32.174525+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":872,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.864013+0000","last_change":"2024-09-21T11:56:50.696344+0000","last_active":"2024-09-21T11:59:27.864013+0000","last_peered":"2024-09-21T11:59:27.864013+0000","last_clean":"2024-09-21T11:59:27.864013+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T11:59:27.864013+0000","last_undegraded":"2024-09-21T11:59:27.864013+0000","last_fullsized":"2024-09-21T11:59:27.864013+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:50.696253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:50.696253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":846,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.856753+0000","last_change":"2024-09-21T11:57:12.518014+0000","last_active":"2024-09-21T11:59:27.856753+0000","last_peered":"2024-09-21T11:59:27.856753+0000","last_clean":"2024-09-21T11:59:27.856753+0000","last_became_active":"2024-09-21T11:40:06.343450+0000","last_became_peered":"2024-09-21T11:40:06.343450+0000","last_unstale":"2024-09-21T11:59:27.856753+0000","last_undegraded":"2024-09-21T11:59:27.856753+0000","last_fullsized":"2024-09-21T11:59:27.856753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:12.517953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:12.517953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:52.060114+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"269'15","reported_seq":886,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.884187+0000","last_change":"2024-09-21T11:59:07.953847+0000","last_active":"2024-09-21T11:59:27.884187+0000","last_peered":"2024-09-21T11:59:27.884187+0000","last_clean":"2024-09-21T11:59:27.884187+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T11:59:27.884187+0000","last_undegraded":"2024-09-21T11:59:27.884187+0000","last_fullsized":"2024-09-21T11:59:27.884187+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:59:07.953663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:59:07.953663+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:36.906230+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":873,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862427+0000","last_change":"2024-09-21T11:57:49.993421+0000","last_active":"2024-09-21T11:59:27.862427+0000","last_peered":"2024-09-21T11:59:27.862427+0000","last_clean":"2024-09-21T11:59:27.862427+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T11:59:27.862427+0000","last_undegraded":"2024-09-21T11:59:27.862427+0000","last_fullsized":"2024-09-21T11:59:27.862427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:49.993339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:49.993339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1069,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863651+0000","last_change":"2024-09-21T11:55:51.317564+0000","last_active":"2024-09-21T11:59:27.863651+0000","last_peered":"2024-09-21T11:59:27.863651+0000","last_clean":"2024-09-21T11:59:27.863651+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T11:59:27.863651+0000","last_undegraded":"2024-09-21T11:59:27.863651+0000","last_fullsized":"2024-09-21T11:59:27.863651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T11:55:51.317473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:55:51.317473+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":344,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.851556+0000","last_change":"2024-09-21T11:58:20.129924+0000","last_active":"2024-09-21T11:59:27.851556+0000","last_peered":"2024-09-21T11:59:27.851556+0000","last_clean":"2024-09-21T11:59:27.851556+0000","last_became_active":"2024-09-21T11:51:43.530898+0000","last_became_peered":"2024-09-21T11:51:43.530898+0000","last_unstale":"2024-09-21T11:59:27.851556+0000","last_undegraded":"2024-09-21T11:59:27.851556+0000","last_fullsized":"2024-09-21T11:59:27.851556+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:20.129854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:20.129854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:38.523985+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.14","version":"269'20","reported_seq":906,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863861+0000","last_change":"2024-09-21T11:57:16.786845+0000","last_active":"2024-09-21T11:59:27.863861+0000","last_peered":"2024-09-21T11:59:27.863861+0000","last_clean":"2024-09-21T11:59:27.863861+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T11:59:27.863861+0000","last_undegraded":"2024-09-21T11:59:27.863861+0000","last_fullsized":"2024-09-21T11:59:27.863861+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'20","last_scrub_stamp":"2024-09-21T11:57:16.786756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:16.786756+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":857,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863032+0000","last_change":"2024-09-21T11:57:40.391494+0000","last_active":"2024-09-21T11:59:27.863032+0000","last_peered":"2024-09-21T11:59:27.863032+0000","last_clean":"2024-09-21T11:59:27.863032+0000","last_became_active":"2024-09-21T11:40:00.314349+0000","last_became_peered":"2024-09-21T11:40:00.314349+0000","last_unstale":"2024-09-21T11:59:27.863032+0000","last_undegraded":"2024-09-21T11:59:27.863032+0000","last_fullsized":"2024-09-21T11:59:27.863032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:40.391409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:40.391409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:25.790004+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":1123,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.884010+0000","last_change":"2024-09-21T11:56:56.051487+0000","last_active":"2024-09-21T11:59:27.884010+0000","last_peered":"2024-09-21T11:59:27.884010+0000","last_clean":"2024-09-21T11:59:27.884010+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T11:59:27.884010+0000","last_undegraded":"2024-09-21T11:59:27.884010+0000","last_fullsized":"2024-09-21T11:59:27.884010+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T11:56:56.051300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:56.051300+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.828079+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":78,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862577+0000","last_change":"2024-09-21T11:57:58.534490+0000","last_active":"2024-09-21T11:59:27.862577+0000","last_peered":"2024-09-21T11:59:27.862577+0000","last_clean":"2024-09-21T11:59:27.862577+0000","last_became_active":"2024-09-21T11:57:58.533810+0000","last_became_peered":"2024-09-21T11:57:58.533810+0000","last_unstale":"2024-09-21T11:59:27.862577+0000","last_undegraded":"2024-09-21T11:59:27.862577+0000","last_fullsized":"2024-09-21T11:59:27.862577+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:58.280698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:56:58.280698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"269'19","reported_seq":894,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860766+0000","last_change":"2024-09-21T11:58:28.658668+0000","last_active":"2024-09-21T11:59:27.860766+0000","last_peered":"2024-09-21T11:59:27.860766+0000","last_clean":"2024-09-21T11:59:27.860766+0000","last_became_active":"2024-09-21T11:51:43.533280+0000","last_became_peered":"2024-09-21T11:51:43.533280+0000","last_unstale":"2024-09-21T11:59:27.860766+0000","last_undegraded":"2024-09-21T11:59:27.860766+0000","last_fullsized":"2024-09-21T11:59:27.860766+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:28.658574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:28.658574+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:54.766743+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":360,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863193+0000","last_change":"2024-09-21T11:58:56.695698+0000","last_active":"2024-09-21T11:59:27.863193+0000","last_peered":"2024-09-21T11:59:27.863193+0000","last_clean":"2024-09-21T11:59:27.863193+0000","last_became_active":"2024-09-21T11:51:43.523229+0000","last_became_peered":"2024-09-21T11:51:43.523229+0000","last_unstale":"2024-09-21T11:59:27.863193+0000","last_undegraded":"2024-09-21T11:59:27.863193+0000","last_fullsized":"2024-09-21T11:59:27.863193+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:56.695646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:58:56.695646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:21.948032+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":551,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.852630+0000","last_change":"2024-09-21T11:56:00.441644+0000","last_active":"2024-09-21T11:59:27.852630+0000","last_peered":"2024-09-21T11:59:27.852630+0000","last_clean":"2024-09-21T11:59:27.852630+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T11:59:27.852630+0000","last_undegraded":"2024-09-21T11:59:27.852630+0000","last_fullsized":"2024-09-21T11:59:27.852630+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:00.441565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:56:00.441565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":844,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863947+0000","last_change":"2024-09-21T11:57:58.531539+0000","last_active":"2024-09-21T11:59:27.863947+0000","last_peered":"2024-09-21T11:59:27.863947+0000","last_clean":"2024-09-21T11:59:27.863947+0000","last_became_active":"2024-09-21T11:57:58.531213+0000","last_became_peered":"2024-09-21T11:57:58.531213+0000","last_unstale":"2024-09-21T11:59:27.863947+0000","last_undegraded":"2024-09-21T11:59:27.863947+0000","last_fullsized":"2024-09-21T11:59:27.863947+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:55:01.434123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:55:01.434123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"269'15","reported_seq":562,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.852850+0000","last_change":"2024-09-21T11:55:59.419239+0000","last_active":"2024-09-21T11:59:27.852850+0000","last_peered":"2024-09-21T11:59:27.852850+0000","last_clean":"2024-09-21T11:59:27.852850+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T11:59:27.852850+0000","last_undegraded":"2024-09-21T11:59:27.852850+0000","last_fullsized":"2024-09-21T11:59:27.852850+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'15","last_scrub_stamp":"2024-09-21T11:55:59.419148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:55:59.419148+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":849,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863303+0000","last_change":"2024-09-21T11:57:17.492529+0000","last_active":"2024-09-21T11:59:27.863303+0000","last_peered":"2024-09-21T11:59:27.863303+0000","last_clean":"2024-09-21T11:59:27.863303+0000","last_became_active":"2024-09-21T11:40:00.309405+0000","last_became_peered":"2024-09-21T11:40:00.309405+0000","last_unstale":"2024-09-21T11:59:27.863303+0000","last_undegraded":"2024-09-21T11:59:27.863303+0000","last_fullsized":"2024-09-21T11:59:27.863303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:17.492466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:17.492466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:37.151802+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":856,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862917+0000","last_change":"2024-09-21T11:58:24.939301+0000","last_active":"2024-09-21T11:59:27.862917+0000","last_peered":"2024-09-21T11:59:27.862917+0000","last_clean":"2024-09-21T11:59:27.862917+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T11:59:27.862917+0000","last_undegraded":"2024-09-21T11:59:27.862917+0000","last_fullsized":"2024-09-21T11:59:27.862917+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:24.939230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:24.939230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:31.727906+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":884,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863674+0000","last_change":"2024-09-21T11:57:37.071994+0000","last_active":"2024-09-21T11:59:27.863674+0000","last_peered":"2024-09-21T11:59:27.863674+0000","last_clean":"2024-09-21T11:59:27.863674+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T11:59:27.863674+0000","last_undegraded":"2024-09-21T11:59:27.863674+0000","last_fullsized":"2024-09-21T11:59:27.863674+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T11:57:37.071908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:37.071908+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"269'19","reported_seq":885,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.864803+0000","last_change":"2024-09-21T11:57:18.759800+0000","last_active":"2024-09-21T11:59:27.864803+0000","last_peered":"2024-09-21T11:59:27.864803+0000","last_clean":"2024-09-21T11:59:27.864803+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T11:59:27.864803+0000","last_undegraded":"2024-09-21T11:59:27.864803+0000","last_fullsized":"2024-09-21T11:59:27.864803+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:57:18.759710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:57:18.759710+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":879,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.883307+0000","last_change":"2024-09-21T11:59:17.879144+0000","last_active":"2024-09-21T11:59:27.883307+0000","last_peered":"2024-09-21T11:59:27.883307+0000","last_clean":"2024-09-21T11:59:27.883307+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T11:59:27.883307+0000","last_undegraded":"2024-09-21T11:59:27.883307+0000","last_fullsized":"2024-09-21T11:59:27.883307+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:17.879101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:59:17.879101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:43.295771+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":833,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862160+0000","last_change":"2024-09-21T11:58:23.430651+0000","last_active":"2024-09-21T11:59:27.862160+0000","last_peered":"2024-09-21T11:59:27.862160+0000","last_clean":"2024-09-21T11:59:27.862160+0000","last_became_active":"2024-09-21T11:40:04.331689+0000","last_became_peered":"2024-09-21T11:40:04.331689+0000","last_unstale":"2024-09-21T11:59:27.862160+0000","last_undegraded":"2024-09-21T11:59:27.862160+0000","last_fullsized":"2024-09-21T11:59:27.862160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:23.430517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:58:23.430517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":860,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.863093+0000","last_change":"2024-09-21T11:57:58.537818+0000","last_active":"2024-09-21T11:59:27.863093+0000","last_peered":"2024-09-21T11:59:27.863093+0000","last_clean":"2024-09-21T11:59:27.863093+0000","last_became_active":"2024-09-21T11:57:58.537121+0000","last_became_peered":"2024-09-21T11:57:58.537121+0000","last_unstale":"2024-09-21T11:59:27.863093+0000","last_undegraded":"2024-09-21T11:59:27.863093+0000","last_fullsized":"2024-09-21T11:59:27.863093+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:43.803926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:57:43.803926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"269'8","reported_seq":889,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.864923+0000","last_change":"2024-09-21T11:54:00.425949+0000","last_active":"2024-09-21T11:59:27.864923+0000","last_peered":"2024-09-21T11:59:27.864923+0000","last_clean":"2024-09-21T11:59:27.864923+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T11:59:27.864923+0000","last_undegraded":"2024-09-21T11:59:27.864923+0000","last_fullsized":"2024-09-21T11:59:27.864923+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'8","last_scrub_stamp":"2024-09-21T11:54:00.425862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:54:00.425862+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:27.148690+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":841,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.861471+0000","last_change":"2024-09-21T11:56:53.232038+0000","last_active":"2024-09-21T11:59:27.861471+0000","last_peered":"2024-09-21T11:59:27.861471+0000","last_clean":"2024-09-21T11:59:27.861471+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T11:59:27.861471+0000","last_undegraded":"2024-09-21T11:59:27.861471+0000","last_fullsized":"2024-09-21T11:59:27.861471+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:56:53.231963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:56:53.231963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":871,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862670+0000","last_change":"2024-09-21T11:59:08.841142+0000","last_active":"2024-09-21T11:59:27.862670+0000","last_peered":"2024-09-21T11:59:27.862670+0000","last_clean":"2024-09-21T11:59:27.862670+0000","last_became_active":"2024-09-21T11:58:17.527743+0000","last_became_peered":"2024-09-21T11:58:17.527743+0000","last_unstale":"2024-09-21T11:59:27.862670+0000","last_undegraded":"2024-09-21T11:59:27.862670+0000","last_fullsized":"2024-09-21T11:59:27.862670+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:59:08.841066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:59:08.841066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:28.082425+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":861,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862475+0000","last_change":"2024-09-21T11:58:53.713848+0000","last_active":"2024-09-21T11:59:27.862475+0000","last_peered":"2024-09-21T11:59:27.862475+0000","last_clean":"2024-09-21T11:59:27.862475+0000","last_became_active":"2024-09-21T11:40:06.342713+0000","last_became_peered":"2024-09-21T11:40:06.342713+0000","last_unstale":"2024-09-21T11:59:27.862475+0000","last_undegraded":"2024-09-21T11:59:27.862475+0000","last_fullsized":"2024-09-21T11:59:27.862475+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:58:53.713785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:58:53.713785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:04.557035+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":556,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.858507+0000","last_change":"2024-09-21T11:58:17.532076+0000","last_active":"2024-09-21T11:59:27.858507+0000","last_peered":"2024-09-21T11:59:27.858507+0000","last_clean":"2024-09-21T11:59:27.858507+0000","last_became_active":"2024-09-21T11:58:17.531567+0000","last_became_peered":"2024-09-21T11:58:17.531567+0000","last_unstale":"2024-09-21T11:59:27.858507+0000","last_undegraded":"2024-09-21T11:59:27.858507+0000","last_fullsized":"2024-09-21T11:59:27.858507+0000","mapping_epoch":430,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":431,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:52:12.348629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T11:52:12.348629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:00:22.768976+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":886,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.865170+0000","last_change":"2024-09-21T11:57:58.536812+0000","last_active":"2024-09-21T11:59:27.865170+0000","last_peered":"2024-09-21T11:59:27.865170+0000","last_clean":"2024-09-21T11:59:27.865170+0000","last_became_active":"2024-09-21T11:57:58.535015+0000","last_became_peered":"2024-09-21T11:57:58.535015+0000","last_unstale":"2024-09-21T11:59:27.865170+0000","last_undegraded":"2024-09-21T11:59:27.865170+0000","last_fullsized":"2024-09-21T11:59:27.865170+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:39.735027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T11:57:39.735027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"269'19","reported_seq":955,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.860965+0000","last_change":"2024-09-21T11:58:16.743891+0000","last_active":"2024-09-21T11:59:27.860965+0000","last_peered":"2024-09-21T11:59:27.860965+0000","last_clean":"2024-09-21T11:59:27.860965+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T11:59:27.860965+0000","last_undegraded":"2024-09-21T11:59:27.860965+0000","last_fullsized":"2024-09-21T11:59:27.860965+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'19","last_scrub_stamp":"2024-09-21T11:58:16.743830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T11:58:16.743830+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T11:59:42.512604+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":850,"reported_epoch":454,"state":"active+clean","last_fresh":"2024-09-21T11:59:27.862924+0000","last_change":"2024-09-21T11:57:48.045901+0000","last_active":"2024-09-21T11:59:27.862924+0000","last_peered":"2024-09-21T11:59:27.862924+0000","last_clean":"2024-09-21T11:59:27.862924+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T11:59:27.862924+0000","last_undegraded":"2024-09-21T11:59:27.862924+0000","last_fullsized":"2024-09-21T11:59:27.862924+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T11:57:48.045820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T11:57:48.045820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":2963865624,"num_objects":45226,"num_object_clones":0,"num_object_copies":135678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2883,"num_objects_unfound":0,"num_objects_dirty":45226,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197240,"num_write_kb":7758849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15759,"num_bytes_recovered":1032716288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9098178560,"data_stored":9098166344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81387,"ondisk_log_size":81387,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1112,"num_read_kb":869,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":6904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1161,"num_read_kb":1156,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234223,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1888344,"kb_used_data":1121352,"kb_used_omap":1,"kb_used_meta":766974,"kb_avail":91856808,"statfs":{"total":95995035648,"available":94061371392,"internally_reserved":0,"allocated":1148264448,"data_stored":1147860096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":785381834},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463156,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2117648,"kb_used_data":1472832,"kb_used_omap":1,"kb_used_meta":644798,"kb_avail":91627504,"statfs":{"total":95995035648,"available":93826564096,"internally_reserved":0,"allocated":1508179968,"data_stored":1507784176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":660273606},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724793,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1418724,"kb_used_data":729492,"kb_used_omap":1,"kb_used_meta":689214,"kb_avail":92326428,"statfs":{"total":95995035648,"available":94542262272,"internally_reserved":0,"allocated":746999808,"data_stored":746583174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":705755590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733921023,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1155756,"kb_used_data":556760,"kb_used_omap":1,"kb_used_meta":598974,"kb_avail":92589396,"statfs":{"total":95995035648,"available":94811541504,"internally_reserved":0,"allocated":570122240,"data_stored":569702041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":613349829},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117253,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2081368,"kb_used_data":1265800,"kb_used_omap":1,"kb_used_meta":815550,"kb_avail":91663784,"statfs":{"total":95995035648,"available":93863714816,"internally_reserved":0,"allocated":1296179200,"data_stored":1295774730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":835123655},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411595,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1869280,"kb_used_data":1089744,"kb_used_omap":1,"kb_used_meta":779454,"kb_avail":91875872,"statfs":{"total":95995035648,"available":94080892928,"internally_reserved":0,"allocated":1115897856,"data_stored":1115485929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":798161349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575122,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2414644,"kb_used_data":1661348,"kb_used_omap":1,"kb_used_meta":753086,"kb_avail":91330508,"statfs":{"total":95995035648,"available":93522440192,"internally_reserved":0,"allocated":1701220352,"data_stored":1700787817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":771160518},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738648,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1861160,"kb_used_data":1067736,"kb_used_omap":1,"kb_used_meta":793342,"kb_avail":91883992,"statfs":{"total":95995035648,"available":94089207808,"internally_reserved":0,"allocated":1093361664,"data_stored":1092942046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":812382661},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1074921472,"data_stored":1074921472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1697447936,"data_stored":1697447936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1096880128,"data_stored":1096876056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1293156352,"data_stored":1293156352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":559353856,"data_stored":559349784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":736100352,"data_stored":736100352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1505234944,"data_stored":1505230872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1135083520,"data_stored":1135083520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T11:59:37.893 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-21T11:59:37.913 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:37.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:37.940 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:37.940 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:38.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:38.069 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:38.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:38.070 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:38.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:38.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:38.208 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:38.309 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:38.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:38.340 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:38.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:38.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:38.480 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T11:59:38.482 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:38.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:38.483 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:38.572 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:38.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:38.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:38.644 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:38.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:38.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:38 smithi043 ceph-mon[36454]: 4.16 scrub starts 2024-09-21T11:59:38.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:38 smithi043 ceph-mon[36454]: 4.16 scrub ok 2024-09-21T11:59:38.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:38 smithi043 ceph-mon[36454]: 3.11 scrub starts 2024-09-21T11:59:38.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:38 smithi043 ceph-mon[36454]: 3.11 scrub ok 2024-09-21T11:59:38.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:38 smithi043 ceph-mon[29710]: 4.16 scrub starts 2024-09-21T11:59:38.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:38 smithi043 ceph-mon[29710]: 4.16 scrub ok 2024-09-21T11:59:38.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:38 smithi043 ceph-mon[29710]: 3.11 scrub starts 2024-09-21T11:59:38.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:38 smithi043 ceph-mon[29710]: 3.11 scrub ok 2024-09-21T11:59:38.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:38.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:38 smithi175 ceph-mon[32613]: 4.16 scrub starts 2024-09-21T11:59:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:38 smithi175 ceph-mon[32613]: 4.16 scrub ok 2024-09-21T11:59:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:38 smithi175 ceph-mon[32613]: 3.11 scrub starts 2024-09-21T11:59:38.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:38 smithi175 ceph-mon[32613]: 3.11 scrub ok 2024-09-21T11:59:38.809 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:38.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:38.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:38.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:38.981 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:38.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:38.983 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:39.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:39.111 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:39.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:39.113 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:39.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:39.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T11:59:39.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:39.263 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:39.264 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:39.276 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:59:39.276 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:59:39.303 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:39.323 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:59:39.324 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:59:39.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:39.365 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:59:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:59:39.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[36454]: 5.2 scrub starts 2024-09-21T11:59:39.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[36454]: 5.2 scrub ok 2024-09-21T11:59:39.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[36454]: 6.9 scrub starts 2024-09-21T11:59:39.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[36454]: 6.9 scrub ok 2024-09-21T11:59:39.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[36454]: pgmap v959: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.00k op/s; 2553/121410 objects misplaced (2.103%) 2024-09-21T11:59:39.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[36454]: 3.1a scrub starts 2024-09-21T11:59:39.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[36454]: 3.1a scrub ok 2024-09-21T11:59:39.367 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3668247139' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:39.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[29710]: 5.2 scrub starts 2024-09-21T11:59:39.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[29710]: 5.2 scrub ok 2024-09-21T11:59:39.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[29710]: 6.9 scrub starts 2024-09-21T11:59:39.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[29710]: 6.9 scrub ok 2024-09-21T11:59:39.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[29710]: pgmap v959: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.00k op/s; 2553/121410 objects misplaced (2.103%) 2024-09-21T11:59:39.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[29710]: 3.1a scrub starts 2024-09-21T11:59:39.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[29710]: 3.1a scrub ok 2024-09-21T11:59:39.367 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3668247139' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:39.387 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:39.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:39.388 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:39.403 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:59:39.404 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:59:39.441 INFO:teuthology.orchestra.run.smithi043.stdout:osd.4: {} 2024-09-21T11:59:39.442 INFO:teuthology.orchestra.run.smithi043.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-21T11:59:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:59:39.475 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:59:39.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:39.513 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:59:39.513 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:59:39.549 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:59:39.549 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:59:39.551 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:39.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:39.552 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:39.653 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:39.681 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:39.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:39.682 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:39.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:39 smithi175 ceph-mon[32613]: 5.2 scrub starts 2024-09-21T11:59:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:39 smithi175 ceph-mon[32613]: 5.2 scrub ok 2024-09-21T11:59:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:39 smithi175 ceph-mon[32613]: 6.9 scrub starts 2024-09-21T11:59:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:39 smithi175 ceph-mon[32613]: 6.9 scrub ok 2024-09-21T11:59:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:39 smithi175 ceph-mon[32613]: pgmap v959: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.00k op/s; 2553/121410 objects misplaced (2.103%) 2024-09-21T11:59:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:39 smithi175 ceph-mon[32613]: 3.1a scrub starts 2024-09-21T11:59:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:39 smithi175 ceph-mon[32613]: 3.1a scrub ok 2024-09-21T11:59:39.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3668247139' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:39.782 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:39.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:39.820 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:39.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:39.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:39.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:40.057 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:40.098 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:40.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:40.099 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:40.199 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:40.227 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:40.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:40.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:40.328 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:40.358 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:40.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:40.359 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:40.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:40.485 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:40.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:40.486 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:40.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:40.622 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:40.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:40.623 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:40.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[29710]: 4.d scrub starts 2024-09-21T11:59:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[29710]: 4.d scrub ok 2024-09-21T11:59:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[29710]: 4.12 scrub starts 2024-09-21T11:59:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[29710]: 6.3 scrub starts 2024-09-21T11:59:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[29710]: 4.12 scrub ok 2024-09-21T11:59:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[29710]: 6.3 scrub ok 2024-09-21T11:59:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[29710]: 4.1 scrub starts 2024-09-21T11:59:40.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[29710]: 4.1 scrub ok 2024-09-21T11:59:40.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[36454]: 4.d scrub starts 2024-09-21T11:59:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[36454]: 4.d scrub ok 2024-09-21T11:59:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[36454]: 4.12 scrub starts 2024-09-21T11:59:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[36454]: 6.3 scrub starts 2024-09-21T11:59:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[36454]: 4.12 scrub ok 2024-09-21T11:59:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[36454]: 6.3 scrub ok 2024-09-21T11:59:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[36454]: 4.1 scrub starts 2024-09-21T11:59:40.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:40 smithi043 ceph-mon[36454]: 4.1 scrub ok 2024-09-21T11:59:40.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:40 smithi175 ceph-mon[32613]: 4.d scrub starts 2024-09-21T11:59:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:40 smithi175 ceph-mon[32613]: 4.d scrub ok 2024-09-21T11:59:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:40 smithi175 ceph-mon[32613]: 4.12 scrub starts 2024-09-21T11:59:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:40 smithi175 ceph-mon[32613]: 6.3 scrub starts 2024-09-21T11:59:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:40 smithi175 ceph-mon[32613]: 4.12 scrub ok 2024-09-21T11:59:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:40 smithi175 ceph-mon[32613]: 6.3 scrub ok 2024-09-21T11:59:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:40 smithi175 ceph-mon[32613]: 4.1 scrub starts 2024-09-21T11:59:40.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:40 smithi175 ceph-mon[32613]: 4.1 scrub ok 2024-09-21T11:59:40.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:40.767 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:40.767 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:40.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:40.902 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:40.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:40.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:41.004 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:41.040 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:41.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:41.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:41.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:41.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:41.181 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:41.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:41.324 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:41.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:41.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:41.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:41.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:41.463 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:41.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:41.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:41.588 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:41.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:41.590 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:41.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:41.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:41.739 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:41 smithi043 ceph-mon[29710]: 4.18 scrub starts 2024-09-21T11:59:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:41 smithi043 ceph-mon[29710]: 4.18 scrub ok 2024-09-21T11:59:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:41 smithi043 ceph-mon[29710]: pgmap v960: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.00k op/s; 2553/121410 objects misplaced (2.103%) 2024-09-21T11:59:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:41 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:59:41.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:41 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:59:41.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:41 smithi043 ceph-mon[36454]: 4.18 scrub starts 2024-09-21T11:59:41.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:41 smithi043 ceph-mon[36454]: 4.18 scrub ok 2024-09-21T11:59:41.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:41 smithi043 ceph-mon[36454]: pgmap v960: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.00k op/s; 2553/121410 objects misplaced (2.103%) 2024-09-21T11:59:41.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:41 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:59:41.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:41 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:59:41.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:41 smithi175 ceph-mon[32613]: 4.18 scrub starts 2024-09-21T11:59:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:41 smithi175 ceph-mon[32613]: 4.18 scrub ok 2024-09-21T11:59:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:41 smithi175 ceph-mon[32613]: pgmap v960: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.5 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.00k op/s; 2553/121410 objects misplaced (2.103%) 2024-09-21T11:59:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:41 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:59:41.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:41 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:59:41.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:41.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:41.866 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:41.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:41.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:41.992 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:41.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:41.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:42.094 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:42.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:42.179 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:42.179 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:42.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:59:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:59:42.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:42.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:42.309 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:42.309 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:42.410 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4 2024-09-21T11:59:42.440 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:42.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:42.441 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.4' 2024-09-21T11:59:42.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:42.576 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:42.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:42.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:42.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[29710]: 3.1b scrub starts 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[29710]: 3.1b scrub ok 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[29710]: osdmap e456: 8 total, 8 up, 8 in 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[29710]: 6.17 scrub starts 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[29710]: 6.17 scrub ok 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[29710]: 2.0 scrub starts 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[29710]: 2.0 scrub ok 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[36454]: 3.1b scrub starts 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[36454]: 3.1b scrub ok 2024-09-21T11:59:42.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[36454]: osdmap e456: 8 total, 8 up, 8 in 2024-09-21T11:59:42.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[36454]: 6.17 scrub starts 2024-09-21T11:59:42.688 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[36454]: 6.17 scrub ok 2024-09-21T11:59:42.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[36454]: 2.0 scrub starts 2024-09-21T11:59:42.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[36454]: 2.0 scrub ok 2024-09-21T11:59:42.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T11:59:42.689 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:42 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T11:59:42.724 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:42.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:42.726 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:42.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:42 smithi175 ceph-mon[32613]: 3.1b scrub starts 2024-09-21T11:59:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:42 smithi175 ceph-mon[32613]: 3.1b scrub ok 2024-09-21T11:59:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:42 smithi175 ceph-mon[32613]: osdmap e456: 8 total, 8 up, 8 in 2024-09-21T11:59:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:42 smithi175 ceph-mon[32613]: 6.17 scrub starts 2024-09-21T11:59:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:42 smithi175 ceph-mon[32613]: 6.17 scrub ok 2024-09-21T11:59:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:42 smithi175 ceph-mon[32613]: 2.0 scrub starts 2024-09-21T11:59:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:42 smithi175 ceph-mon[32613]: 2.0 scrub ok 2024-09-21T11:59:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:42 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T11:59:42.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:42 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T11:59:42.826 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:42.852 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:42.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:42.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:42.894 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:59:42.894 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-21T11:59:42.894 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-21T11:59:42.894 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T11:59:42.895 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.4 2024-09-21T11:59:42.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:43.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:43.000 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:43.000 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:43.062 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:59:43.062 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T11:59:43.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.062 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:209288 11.b 11:d2cf3d31:::benchmark_data_smithi043_178089_object88053:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e458)", 2024-09-21T11:59:43.062 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:43.050180+0000", 2024-09-21T11:59:43.062 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.007442911, 2024-09-21T11:59:43.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007475219, 2024-09-21T11:59:43.062 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:43.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:43.062 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 209288 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.050180+0000", 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.050180+0000", 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.050182+0000", 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.0339999999999999e-06 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.050190+0000", 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.9300000000000003e-06 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.050212+0000", 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2232999999999999e-05 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.050222+0000", 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.4870000000000008e-06 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.050343+0000", 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000121068 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:43.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.050467+0000", 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123931 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,1", 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.050590+0000", 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000123101 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:209290 11.a 11:5c0386a5:::benchmark_data_smithi043_178089_object88055:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e458)", 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:43.053295+0000", 2024-09-21T11:59:43.066 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.0043280259999999996, 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.004439363, 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 209290 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.053295+0000", 2024-09-21T11:59:43.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.053295+0000", 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.053297+0000", 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.7230000000000001e-06 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:43.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.053303+0000", 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8649999999999998e-06 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.053314+0000", 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1555e-05 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.053323+0000", 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5290000000000008e-06 2024-09-21T11:59:43.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.054311+0000", 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00098803799999999989 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.054426+0000", 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114827 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 5,6", 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.054541+0000", 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115321 2024-09-21T11:59:43.070 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:209295 11.9 11:998cf3c8:::benchmark_data_smithi043_178089_object88060:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e458)", 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:43.056288+0000", 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 0.00133471, 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.001520155, 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "waiting for sub ops", 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T11:59:43.071 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 209295 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.056288+0000", 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.056288+0000", 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T11:59:43.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.056290+0000", 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.685e-06 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.056295+0000", 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4110000000000004e-06 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.056317+0000", 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1835999999999999e-05 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.056327+0000", 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0179e-05 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.056442+0000", 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114437 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.056572+0000", 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013007900000000001 2024-09-21T11:59:43.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 0,6", 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:43.056696+0000", 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012454499999999999 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout: "num_ops": 3 2024-09-21T11:59:43.075 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:59:43.076 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:43.101 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:43.130 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:43.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:43.131 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:43.232 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:43.260 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:42 smithi175 systemd[1]: Stopping Ceph osd.4 for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T11:59:43.278 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:43.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:43.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:43.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:43.411 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:43.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:43.412 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:43.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:43.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:43.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:43.654 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:43.675 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:43.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:43.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: 5.12 scrub starts 2024-09-21T11:59:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: 5.12 scrub ok 2024-09-21T11:59:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:59:43.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: pgmap v962: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s; 2202/105945 objects misplaced (2.078%) 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: osdmap e457: 8 total, 8 up, 8 in 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: osdmap e458: 8 total, 8 up, 8 in 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: 3.f scrub starts 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: 3.f scrub ok 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: 5.15 scrub starts 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: 5.15 scrub ok 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[29710]: osd.4 marked itself down and dead 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: 5.12 scrub starts 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: 5.12 scrub ok 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:43.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:59:43.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:59:43.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: pgmap v962: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s; 2202/105945 objects misplaced (2.078%) 2024-09-21T11:59:43.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: osdmap e457: 8 total, 8 up, 8 in 2024-09-21T11:59:43.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: osdmap e458: 8 total, 8 up, 8 in 2024-09-21T11:59:43.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: 3.f scrub starts 2024-09-21T11:59:43.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: 3.f scrub ok 2024-09-21T11:59:43.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: 5.15 scrub starts 2024-09-21T11:59:43.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: 5.15 scrub ok 2024-09-21T11:59:43.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:43 smithi043 ceph-mon[36454]: osd.4 marked itself down and dead 2024-09-21T11:59:43.761 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[36714]: 2024-09-21T11:59:43.283+0000 7f7133841640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T11:59:43.761 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[36714]: 2024-09-21T11:59:43.283+0000 7f7133841640 -1 osd.4 458 *** Got signal Terminated *** 2024-09-21T11:59:43.761 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[36714]: 2024-09-21T11:59:43.283+0000 7f7133841640 -1 osd.4 458 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T11:59:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: 5.12 scrub starts 2024-09-21T11:59:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: 5.12 scrub ok 2024-09-21T11:59:43.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:43.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:59:43.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:59:43.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: pgmap v962: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s; 2202/105945 objects misplaced (2.078%) 2024-09-21T11:59:43.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: osdmap e457: 8 total, 8 up, 8 in 2024-09-21T11:59:43.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: osdmap e458: 8 total, 8 up, 8 in 2024-09-21T11:59:43.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: 3.f scrub starts 2024-09-21T11:59:43.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: 3.f scrub ok 2024-09-21T11:59:43.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: 5.15 scrub starts 2024-09-21T11:59:43.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: 5.15 scrub ok 2024-09-21T11:59:43.762 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:43 smithi175 ceph-mon[32613]: osd.4 marked itself down and dead 2024-09-21T11:59:43.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:43.799 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:43.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:43.800 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:43.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T11:59:43.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:43.960 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:43.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:43.961 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:44.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:44.083 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:44.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:44.084 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:44.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:44.206 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:44.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:44.207 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:44.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:44.309 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:44 smithi175 podman[147213]: 2024-09-21 11:59:44.038100284 +0000 UTC m=+0.899124927 container died e4cfa0b3eae81b1262587d79747604f77dc8c34c5519525bc53867c7bd6e2eea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T11:59:44.332 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:44.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:44.333 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:44.434 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:44.455 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:44.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:44.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: 11.6 scrub starts 2024-09-21T11:59:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: 5.1f scrub starts 2024-09-21T11:59:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: 5.1f scrub ok 2024-09-21T11:59:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: 11.0 scrub starts 2024-09-21T11:59:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: 5.a scrub starts 2024-09-21T11:59:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: 5.a scrub ok 2024-09-21T11:59:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T11:59:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: osdmap e459: 8 total, 7 up, 8 in 2024-09-21T11:59:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: 11.0 scrub ok 2024-09-21T11:59:44.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T11:59:44.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T11:59:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: 11.6 scrub starts 2024-09-21T11:59:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: 5.1f scrub starts 2024-09-21T11:59:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: 5.1f scrub ok 2024-09-21T11:59:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: 11.0 scrub starts 2024-09-21T11:59:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: 5.a scrub starts 2024-09-21T11:59:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: 5.a scrub ok 2024-09-21T11:59:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T11:59:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: osdmap e459: 8 total, 7 up, 8 in 2024-09-21T11:59:44.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: 11.0 scrub ok 2024-09-21T11:59:44.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T11:59:44.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:44 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T11:59:44.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:44.584 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:44.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:44.585 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:44.685 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:44.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:44.712 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:44.712 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:44.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:59:44.760 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:44 smithi175 podman[147213]: 2024-09-21 11:59:44.489658155 +0000 UTC m=+1.350682839 container cleanup e4cfa0b3eae81b1262587d79747604f77dc8c34c5519525bc53867c7bd6e2eea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T11:59:44.761 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:44 smithi175 bash[147213]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: 11.6 scrub starts 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: 5.1f scrub starts 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: 5.1f scrub ok 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: 11.0 scrub starts 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: 5.a scrub starts 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: 5.a scrub ok 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: osdmap e459: 8 total, 7 up, 8 in 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: 11.0 scrub ok 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T11:59:44.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:44 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T11:59:44.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:44.869 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:44.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:44.870 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:44.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:45.012 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T11:59:45.012 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T11:59:45.012 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T11:59:45.012 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T11:59:45.012 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T11:59:45.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:45.022 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:45.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:45.123 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:45.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:45.154 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:45.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:45.221 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:44 smithi175 podman[147263]: 2024-09-21 11:59:44.850053097 +0000 UTC m=+0.801878777 container remove e4cfa0b3eae81b1262587d79747604f77dc8c34c5519525bc53867c7bd6e2eea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T11:59:45.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:45.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:45.301 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:45.302 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:45.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:45.455 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:45.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:45.457 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[29710]: 11.6 scrub ok 2024-09-21T11:59:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T11:59:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T11:59:45.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[29710]: 3.4 scrub starts 2024-09-21T11:59:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[29710]: 3.4 scrub ok 2024-09-21T11:59:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[29710]: pgmap v966: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.37k op/s; 2202/104967 objects misplaced (2.098%) 2024-09-21T11:59:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[29710]: osdmap e460: 8 total, 7 up, 8 in 2024-09-21T11:59:45.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/702738052' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:45.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:59:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[36454]: 11.6 scrub ok 2024-09-21T11:59:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T11:59:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T11:59:45.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[36454]: 3.4 scrub starts 2024-09-21T11:59:45.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[36454]: 3.4 scrub ok 2024-09-21T11:59:45.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[36454]: pgmap v966: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.37k op/s; 2202/104967 objects misplaced (2.098%) 2024-09-21T11:59:45.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[36454]: osdmap e460: 8 total, 7 up, 8 in 2024-09-21T11:59:45.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/702738052' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:45.510 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:45 smithi175 podman[147434]: 2024-09-21 11:59:45.126516825 +0000 UTC m=+0.029953840 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T11:59:45.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:45 smithi175 ceph-mon[32613]: 11.6 scrub ok 2024-09-21T11:59:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:45 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T11:59:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:45 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T11:59:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:45 smithi175 ceph-mon[32613]: 3.4 scrub starts 2024-09-21T11:59:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:45 smithi175 ceph-mon[32613]: 3.4 scrub ok 2024-09-21T11:59:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:45 smithi175 ceph-mon[32613]: pgmap v966: 158 pgs: 1 active+remapped+backfill_toofull, 157 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.37k op/s; 2202/104967 objects misplaced (2.098%) 2024-09-21T11:59:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:45 smithi175 ceph-mon[32613]: osdmap e460: 8 total, 7 up, 8 in 2024-09-21T11:59:45.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/702738052' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T11:59:45.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:45.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:45.607 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:45.607 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:45.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:45.732 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:45.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:45.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:45.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:45.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:45.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:45.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:46.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:46.014 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:46.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:46.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:46.140 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:46.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:46.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:46.242 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:46.270 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:46.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:46.271 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:46.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T11:59:46.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:46.420 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:46.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:46.422 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:46.523 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:46.551 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:46.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:46.552 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:46.587 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:46 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:59:46.588 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:46 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/702738052' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:46.588 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:46 smithi043 ceph-mon[29710]: osdmap e461: 8 total, 7 up, 8 in 2024-09-21T11:59:46.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:46 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:59:46.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:46 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/702738052' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:46.588 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:46 smithi043 ceph-mon[36454]: osdmap e461: 8 total, 7 up, 8 in 2024-09-21T11:59:46.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:46.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:46.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:46 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T11:59:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:46 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/702738052' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:46.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:46 smithi175 ceph-mon[32613]: osdmap e461: 8 total, 7 up, 8 in 2024-09-21T11:59:46.761 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:46 smithi175 podman[147434]: 2024-09-21 11:59:46.391465238 +0000 UTC m=+1.294902244 container create 170899026b575912fe36383397d35c8813ea8ace5536e02e03fb679aa45cfc4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T11:59:46.806 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:46.833 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:46.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:46.834 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:46.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:46.983 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:46.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:46.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:47.085 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:47.112 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:47.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:47.113 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:47.214 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:47.237 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:47.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:47.238 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:47.339 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:47.365 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:47.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:47.367 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:47.467 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:47.489 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:47.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:47.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:47.524 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[36454]: pgmap v969: 158 pgs: 4 active+undersized, 5 active+undersized+degraded, 1 active+remapped+backfill_toofull, 148 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 214 op/s; 23/103449 objects degraded (0.022%); 2202/103449 objects misplaced (2.129%) 2024-09-21T11:59:47.524 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[36454]: 5.d scrub starts 2024-09-21T11:59:47.524 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[36454]: 5.d scrub ok 2024-09-21T11:59:47.524 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T11:59:47.524 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T11:59:47.524 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[36454]: Health check failed: Degraded data redundancy: 23/103449 objects degraded (0.022%), 5 pgs degraded (PG_DEGRADED) 2024-09-21T11:59:47.524 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:47.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:47.525 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3412601007' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:47.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[29710]: pgmap v969: 158 pgs: 4 active+undersized, 5 active+undersized+degraded, 1 active+remapped+backfill_toofull, 148 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 214 op/s; 23/103449 objects degraded (0.022%); 2202/103449 objects misplaced (2.129%) 2024-09-21T11:59:47.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[29710]: 5.d scrub starts 2024-09-21T11:59:47.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[29710]: 5.d scrub ok 2024-09-21T11:59:47.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T11:59:47.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T11:59:47.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[29710]: Health check failed: Degraded data redundancy: 23/103449 objects degraded (0.022%), 5 pgs degraded (PG_DEGRADED) 2024-09-21T11:59:47.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:47.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:47.526 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3412601007' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:47.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:47.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:47.639 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:47.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:47.687 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:47 smithi175 ceph-mon[32613]: pgmap v969: 158 pgs: 4 active+undersized, 5 active+undersized+degraded, 1 active+remapped+backfill_toofull, 148 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 214 op/s; 23/103449 objects degraded (0.022%); 2202/103449 objects misplaced (2.129%) 2024-09-21T11:59:47.687 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:47 smithi175 ceph-mon[32613]: 5.d scrub starts 2024-09-21T11:59:47.687 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:47 smithi175 ceph-mon[32613]: 5.d scrub ok 2024-09-21T11:59:47.687 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:47 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T11:59:47.687 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:47 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T11:59:47.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:47 smithi175 ceph-mon[32613]: Health check failed: Degraded data redundancy: 23/103449 objects degraded (0.022%), 5 pgs degraded (PG_DEGRADED) 2024-09-21T11:59:47.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:47 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T11:59:47.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:47 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T11:59:47.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3412601007' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:47.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:47.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:47.782 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:47.782 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:47.883 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:47.925 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:47.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:47.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:48.027 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:48.054 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:48.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:48.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:48.155 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:48.177 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:48.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:48.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:48.245 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:59:48.245 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:59:48.246 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:59:48.246 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:59:48.246 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:59:48.246 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:59:48.246 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:59:48.246 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:59:48.246 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:59:48.246 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:59:48.246 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:59:48.246 INFO:teuthology.orchestra.run.smithi043.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T11:59:48.247 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:59:48.247 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:59:48.247 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:59:48.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:48.317 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:48.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:48.318 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:48.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:48.453 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:48.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:48.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:48 smithi043 ceph-mon[29710]: osdmap e462: 8 total, 7 up, 8 in 2024-09-21T11:59:48.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:48 smithi043 ceph-mon[29710]: osdmap e463: 8 total, 7 up, 8 in 2024-09-21T11:59:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:48 smithi043 ceph-mon[36454]: osdmap e462: 8 total, 7 up, 8 in 2024-09-21T11:59:48.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:48 smithi043 ceph-mon[36454]: osdmap e463: 8 total, 7 up, 8 in 2024-09-21T11:59:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:48.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:48.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:48.595 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:48.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:48.604 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:48 smithi175 ceph-mon[32613]: osdmap e462: 8 total, 7 up, 8 in 2024-09-21T11:59:48.604 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:48 smithi175 ceph-mon[32613]: osdmap e463: 8 total, 7 up, 8 in 2024-09-21T11:59:48.604 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:48 smithi175 podman[147434]: 2024-09-21 11:59:48.441214162 +0000 UTC m=+3.344651183 container init 170899026b575912fe36383397d35c8813ea8ace5536e02e03fb679aa45cfc4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T11:59:48.604 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:48 smithi175 podman[147434]: 2024-09-21 11:59:48.450240714 +0000 UTC m=+3.353677723 container start 170899026b575912fe36383397d35c8813ea8ace5536e02e03fb679aa45cfc4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=) 2024-09-21T11:59:48.696 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:48.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:48.733 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:48.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:48.834 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:48.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:48.865 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:48.865 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:48.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:49.004 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T11:59:49.004 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T11:59:49.010 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:48 smithi175 podman[147434]: 2024-09-21 11:59:48.602954035 +0000 UTC m=+3.506391041 container attach 170899026b575912fe36383397d35c8813ea8ace5536e02e03fb679aa45cfc4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T11:59:49.011 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 11:59:48 smithi175 podman[147434]: 2024-09-21 11:59:48.759116983 +0000 UTC m=+3.662553990 container died 170899026b575912fe36383397d35c8813ea8ace5536e02e03fb679aa45cfc4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T11:59:49.015 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:49.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:49.016 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:49.117 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:49.143 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:49.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:49.144 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:49.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:49.291 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:49.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:49.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:49.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:49.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:49.427 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:49.427 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:49.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:49.561 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:49.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:49.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:49.574 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.4.service' 2024-09-21T11:59:49.580 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:49 smithi043 ceph-mon[29710]: pgmap v972: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.01k op/s; 5695/92874 objects degraded (6.132%); 1933/92874 objects misplaced (2.081%); 0 B/s, 0 objects/s recovering 2024-09-21T11:59:49.580 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:49 smithi043 ceph-mon[29710]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-21T11:59:49.581 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:49 smithi043 ceph-mon[36454]: pgmap v972: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.01k op/s; 5695/92874 objects degraded (6.132%); 1933/92874 objects misplaced (2.081%); 0 B/s, 0 objects/s recovering 2024-09-21T11:59:49.581 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:49 smithi043 ceph-mon[36454]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-21T11:59:49.651 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T11:59:49.651 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:59:49.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:49 smithi175 ceph-mon[32613]: pgmap v972: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.01k op/s; 5695/92874 objects degraded (6.132%); 1933/92874 objects misplaced (2.081%); 0 B/s, 0 objects/s recovering 2024-09-21T11:59:49.661 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:49 smithi175 ceph-mon[32613]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-21T11:59:49.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T11:59:49.663 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T11:59:49.663 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-21T11:59:49.663 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T11:59:49.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -n osd.4 --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-21T11:59:49.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:49.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:49.703 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:49.804 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:49.838 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:49.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:49.839 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:49.938 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/osd.4/config 2024-09-21T11:59:49.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:49.969 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:49.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:49.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:50.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:50.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:50.103 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:50.103 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:50.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T11:59:50.204 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:50.230 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:50.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:50.231 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:50.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:50.384 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:50.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:50.385 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:50.486 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:50.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T11:59:50.513 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:50.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:50.514 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:50.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:50.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:50.672 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:50.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:50.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:50.799 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:50.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:50.800 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:50.901 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:50.927 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:50.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:50.928 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:51.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T11:59:51.030 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:51.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:51.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:51.192 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:51.215 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:51.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:51.216 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:51.317 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:51.341 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:51.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:51.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:51.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:51.492 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:51.509 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:51 smithi175 ceph-mon[32613]: 3.10 scrub starts 2024-09-21T11:59:51.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:51 smithi175 ceph-mon[32613]: 3.10 scrub ok 2024-09-21T11:59:51.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:51 smithi175 ceph-mon[32613]: pgmap v973: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 712 op/s; 5695/92874 objects degraded (6.132%); 1933/92874 objects misplaced (2.081%); 0 B/s, 0 objects/s recovering 2024-09-21T11:59:51.512 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:59:51.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:51.515 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:51.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:51.566 INFO:teuthology.orchestra.run.smithi175.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-21T11:59:51.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:51.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:51 smithi043 ceph-mon[29710]: 3.10 scrub starts 2024-09-21T11:59:51.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:51 smithi043 ceph-mon[29710]: 3.10 scrub ok 2024-09-21T11:59:51.643 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:51 smithi043 ceph-mon[29710]: pgmap v973: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 712 op/s; 5695/92874 objects degraded (6.132%); 1933/92874 objects misplaced (2.081%); 0 B/s, 0 objects/s recovering 2024-09-21T11:59:51.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:51 smithi043 ceph-mon[36454]: 3.10 scrub starts 2024-09-21T11:59:51.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:51 smithi043 ceph-mon[36454]: 3.10 scrub ok 2024-09-21T11:59:51.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:51 smithi043 ceph-mon[36454]: pgmap v973: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.9 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 712 op/s; 5695/92874 objects degraded (6.132%); 1933/92874 objects misplaced (2.081%); 0 B/s, 0 objects/s recovering 2024-09-21T11:59:51.671 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:51.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:51.672 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:51.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:51.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:51.812 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:51.812 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:51.913 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:51.941 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:51.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:51.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:52.043 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:52.070 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:52.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:52.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:52.074 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 11:59:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T11:59:52.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:52.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:52.210 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:52.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:52.333 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:59:52.333 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:59:52.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:52.346 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:52.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:52.407 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T11:59:52.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:52.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:52.473 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:52.474 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:52.574 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:52.610 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:52.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:52.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:52.622 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:59:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:59:52.623 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:52 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2946316668' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:52.623 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:52 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:52.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:52 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2946316668' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:52.623 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:52 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:52.663 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:59:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:59:52.701 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:59:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:59:52.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:52.739 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:59:52.740 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:59:52.740 INFO:teuthology.orchestra.run.smithi043.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T11:59:52.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:52.742 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:52.742 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:52 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2946316668' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:52.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:52 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T11:59:52.772 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:59:52.772 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:59:52.810 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:59:52.811 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:59:52.843 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:52.851 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:59:52.852 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:59:52.873 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:52.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:52.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:52.974 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:53.005 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:53.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:53.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:53.107 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:53.135 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:53.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:53.136 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:53.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:53.279 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:53.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:53.280 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:53.283 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T11:59:53.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:53.415 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:53.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:53.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:53.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:53 smithi175 ceph-mon[32613]: 6.18 scrub starts 2024-09-21T11:59:53.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:53 smithi175 ceph-mon[32613]: 6.18 scrub ok 2024-09-21T11:59:53.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:53 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:53.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:53 smithi175 ceph-mon[32613]: pgmap v974: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.34k op/s; 4718/78237 objects degraded (6.030%); 1610/78237 objects misplaced (2.058%); 0 B/s, 0 objects/s recovering 2024-09-21T11:59:53.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:53 smithi175 ceph-mon[32613]: Health check update: Degraded data redundancy: 4718/78237 objects degraded (6.030%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T11:59:53.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:53 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:53.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:53 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:59:53.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:53 smithi175 ceph-mon[32613]: osdmap e464: 8 total, 7 up, 8 in 2024-09-21T11:59:53.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:53 smithi175 ceph-mon[32613]: osdmap e465: 8 total, 7 up, 8 in 2024-09-21T11:59:53.513 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:53 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3101511446' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:53.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:53.557 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:53.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:53.559 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:53.659 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:53.665 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:59:53.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:53.698 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:53.698 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[29710]: 6.18 scrub starts 2024-09-21T11:59:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[29710]: 6.18 scrub ok 2024-09-21T11:59:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:53.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[29710]: pgmap v974: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.34k op/s; 4718/78237 objects degraded (6.030%); 1610/78237 objects misplaced (2.058%); 0 B/s, 0 objects/s recovering 2024-09-21T11:59:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[29710]: Health check update: Degraded data redundancy: 4718/78237 objects degraded (6.030%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T11:59:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:59:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[29710]: osdmap e464: 8 total, 7 up, 8 in 2024-09-21T11:59:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[29710]: osdmap e465: 8 total, 7 up, 8 in 2024-09-21T11:59:53.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3101511446' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[36454]: 6.18 scrub starts 2024-09-21T11:59:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[36454]: 6.18 scrub ok 2024-09-21T11:59:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T11:59:53.742 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[36454]: pgmap v974: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.34k op/s; 4718/78237 objects degraded (6.030%); 1610/78237 objects misplaced (2.058%); 0 B/s, 0 objects/s recovering 2024-09-21T11:59:53.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[36454]: Health check update: Degraded data redundancy: 4718/78237 objects degraded (6.030%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T11:59:53.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:53.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T11:59:53.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[36454]: osdmap e464: 8 total, 7 up, 8 in 2024-09-21T11:59:53.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[36454]: osdmap e465: 8 total, 7 up, 8 in 2024-09-21T11:59:53.743 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:53 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3101511446' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:53.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:53.834 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:53.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:53.835 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:53.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:53.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:53.966 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:53.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:54.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:54.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:54.106 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:54.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:54.145 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T11:59:54.146 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T11:59:54.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:54.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:54.249 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:54.249 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:54.302 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:59:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T11:59:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T11:59:54.302 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T11:59:54.302 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:59:54.350 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:54.387 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:54.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:54.388 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:54.488 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:54.522 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:54.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:54.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:54.624 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:54.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:54.664 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:54.665 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:54.765 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:54.798 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:54.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:54.799 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:54.900 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:54.933 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:54.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:54.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:55.035 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:55.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:55.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:55.154 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T11:59:55.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:55.216 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:55.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:55.217 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:55.318 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:55.355 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:55.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:55.356 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:55.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:55.491 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:55 smithi043 ceph-mon[29710]: pgmap v977: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 818 op/s; 4718/77223 objects degraded (6.110%); 1610/77223 objects misplaced (2.085%) 2024-09-21T11:59:55.492 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 11:59:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:11:59:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T11:59:55.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:55 smithi043 ceph-mon[36454]: pgmap v977: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 818 op/s; 4718/77223 objects degraded (6.110%); 1610/77223 objects misplaced (2.085%) 2024-09-21T11:59:55.493 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:55.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:55.494 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:55.517 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:59:55.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:55.619 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-21T11:59:55.624 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:55.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:55.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:55.645 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:55 smithi175 ceph-mon[32613]: pgmap v977: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 818 op/s; 4718/77223 objects degraded (6.110%); 1610/77223 objects misplaced (2.085%) 2024-09-21T11:59:55.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:55.771 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:55.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:55.772 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:55.873 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:55.912 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:55.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:55.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:56.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:56.047 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:56.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:56.048 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:56.116 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:56.127 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T11:59:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T11:59:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T11:59:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T11:59:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25565.0:211066 9.a 9:51f636b8:::benchmark_data_smithi043_126002_object84000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e296)", 2024-09-21T11:59:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:59:56.127 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 480.71168852800002, 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.30144848499999999, 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25565", 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/936049955", 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 211066 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.128 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411972+0000", 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3099999999999999e-07 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.129 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411975+0000", 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.401e-06 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411979+0000", 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4889999999999997e-06 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.411984+0000", 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2530000000000004e-06 2024-09-21T11:59:56.130 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412132+0000", 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014723399999999999 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412228+0000", 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6713999999999996e-05 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,7", 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.412367+0000", 2024-09-21T11:59:56.131 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013867400000000001 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.415401+0000", 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0030337630000000001 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.421153+0000", 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0057522909999999997 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.132 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.713334+0000", 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29218145600000001 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.713381+0000", 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.6492000000000001e-05 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:51:55.713420+0000", 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.9387000000000001e-05 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.133 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67834 10.6 10:6b4bd2cf:::benchmark_data_smithi043_150678_object67833:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.042578+0000", 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 387.081082186, 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28913464900000002, 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67834 2024-09-21T11:59:56.134 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042578+0000", 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042578+0000", 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.135 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042580+0000", 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.443e-06 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042613+0000", 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.3355000000000002e-05 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042623+0000", 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8290000000000003e-06 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.136 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042630+0000", 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7519999999999999e-06 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042846+0000", 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00021617899999999999 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.042970+0000", 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123807 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.137 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043305+0000", 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033573499999999998 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331521+0000", 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28821593200000001 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331551+0000", 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9621999999999999e-05 2024-09-21T11:59:56.138 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331587+0000", 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6513000000000001e-05 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331664+0000", 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6893999999999999e-05 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331713+0000", 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8588000000000003e-05 2024-09-21T11:59:56.139 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67836 10.6 10:654cbc8c:::benchmark_data_smithi043_150678_object67835:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.043478+0000", 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 387.080182697, 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28847367400000001, 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.140 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:59:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67836 2024-09-21T11:59:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043478+0000", 2024-09-21T11:59:56.141 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043478+0000", 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043479+0000", 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0780000000000001e-06 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.142 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043547+0000", 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7952000000000003e-05 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043557+0000", 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0547000000000001e-05 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.043564+0000", 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0550000000000002e-06 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046156+0000", 2024-09-21T11:59:56.143 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002592123 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046282+0000", 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000125747 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046402+0000", 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012027999999999999 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331787+0000", 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28538471399999998 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.144 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331797+0000", 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6050000000000004e-06 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331831+0000", 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4458000000000001e-05 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331911+0000", 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9338999999999994e-05 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331951+0000", 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0775999999999997e-05 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:67834 10.6 e331/313 ondisk, result = 0)", 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046066+0000", 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 387.07759413100001, 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28548575300000001, 2024-09-21T11:59:56.146 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046066+0000", 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046066+0000", 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.147 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046067+0000", 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0210000000000001e-06 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046072+0000", 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.152e-06 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046082+0000", 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3449999999999995e-06 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.148 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046089+0000", 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0740000000000004e-06 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331506+0000", 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28541750199999999 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331515+0000", 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8950000000000004e-06 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.149 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331552+0000", 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6764000000000002e-05 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:67834 10.6 e331/313 ondisk, result = 0)", 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046496+0000", 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 387.077163918, 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.285227234, 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:59:56.150 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046496+0000", 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046496+0000", 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.151 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046497+0000", 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0299999999999999e-07 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046501+0000", 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2359999999999998e-06 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046509+0000", 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9719999999999992e-06 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.152 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046516+0000", 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3679999999999998e-06 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331575+0000", 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28505966199999999 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331582+0000", 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2790000000000004e-06 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.153 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331724+0000", 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014211399999999999 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67837 10.a e331/313 10:5163063c:::benchmark_data_smithi043_150678_object67836:head v 331'4187, mlcod=331'4185)", 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.046734+0000", 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 387.07692584, 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28267978599999999, 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.154 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046734+0000", 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046734+0000", 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046735+0000", 2024-09-21T11:59:56.155 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7300000000000005e-07 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046777+0000", 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1236999999999998e-05 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046786+0000", 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6069999999999997e-06 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046793+0000", 2024-09-21T11:59:56.156 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3990000000000001e-06 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046836+0000", 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2938999999999997e-05 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.046853+0000", 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7424000000000001e-05 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329365+0000", 2024-09-21T11:59:56.157 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28251246899999999 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329414+0000", 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8838000000000002e-05 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67842 10.7 e331/313 10:e64204f5:::benchmark_data_smithi043_150678_object67841:head v 331'4261, mlcod=331'4259)", 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 387.07621160000002, 2024-09-21T11:59:56.158 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28956349100000001, 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.159 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047449+0000", 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.7999999999999995e-07 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047488+0000", 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8732000000000001e-05 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047497+0000", 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8599999999999999e-06 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.160 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047503+0000", 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7289999999999997e-06 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047550+0000", 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7571000000000002e-05 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329323+0000", 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28177280300000002 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.161 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.336914+0000", 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0075908069999999998 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337012+0000", 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8308999999999997e-05 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop(client.25910.0:67844 10.3 e331/313 10:c750a974:::benchmark_data_smithi043_150678_object67843:head v 331'4259, mlcod=331'4257)", 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.047834+0000", 2024-09-21T11:59:56.162 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 387.07582628599999, 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.29078393699999999, 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047834+0000", 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047834+0000", 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.163 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047835+0000", 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9999999999999996e-07 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.047887+0000", 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1690999999999997e-05 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329326+0000", 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28143986999999998 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.164 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329343+0000", 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6688999999999999e-05 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329400+0000", 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6820999999999997e-05 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329424+0000", 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3693000000000002e-05 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.338574+0000", 2024-09-21T11:59:56.165 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0091507940000000003 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.338618+0000", 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.3578999999999997e-05 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:67847 10.6 10:65456b12:::benchmark_data_smithi043_150678_object67846:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e331)", 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.052219+0000", 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 387.07144135599998, 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.28538154100000002, 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.166 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.167 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.167 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T11:59:56.167 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:56.168 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:59:56.168 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 67847 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052219+0000", 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052219+0000", 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.169 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052221+0000", 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.108e-06 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.052286+0000", 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4874000000000003e-05 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329342+0000", 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27705603200000001 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.170 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329356+0000", 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4419e-05 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331988+0000", 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0026312319999999998 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.332118+0000", 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012998 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.171 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.332241+0000", 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012306900000000001 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337426+0000", 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.005185449 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337464+0000", 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7957999999999998e-05 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.172 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337515+0000", 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1017000000000003e-05 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337562+0000", 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7200999999999998e-05 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.337601+0000", 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8201999999999998e-05 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.173 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_repop_reply(client.25910.0:67836 10.6 e331/313 ondisk, result = 0)", 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:29.053932+0000", 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 387.06972842900001, 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27786533400000002, 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053932+0000", 2024-09-21T11:59:56.174 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053932+0000", 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053933+0000", 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.381e-06 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.175 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.053940+0000", 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1810000000000004e-06 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329343+0000", 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.27540220300000001 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.329355+0000", 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2237e-05 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331776+0000", 2024-09-21T11:59:56.176 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024212000000000001 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331783+0000", 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5429999999999999e-06 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:29.331797+0000", 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4589e-05 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.177 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:72213 10.6 10:6e0a855e:::benchmark_data_smithi043_150678_object72212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e332)", 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:32.969718+0000", 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 383.15394209900001, 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23677020700000001, 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 72213 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.178 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969718+0000", 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969718+0000", 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969720+0000", 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.522e-06 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.179 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969793+0000", 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.3422000000000003e-05 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969810+0000", 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7014e-05 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969819+0000", 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.653e-06 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.180 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.969874+0000", 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5568000000000001e-05 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970035+0000", 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00016056600000000001 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 0,4", 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.970175+0000", 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00014015300000000001 2024-09-21T11:59:56.181 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.972425+0000", 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022500749999999998 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:32.973522+0000", 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0010972670000000001 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206358+0000", 2024-09-21T11:59:56.182 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23283527300000001 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206438+0000", 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9804999999999999e-05 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:33.206488+0000", 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.0889e-05 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.183 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:85555 10.9 10:9e997645:::benchmark_data_smithi043_150678_object85554:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.837759+0000", 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 371.28590105199999, 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24446227800000001, 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 85555 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.184 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837759+0000", 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837759+0000", 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837761+0000", 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1500000000000002e-06 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.185 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837839+0000", 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7088999999999993e-05 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837852+0000", 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3409e-05 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837859+0000", 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.635e-06 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.186 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.837907+0000", 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8232000000000002e-05 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.838017+0000", 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011044000000000001 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.838100+0000", 2024-09-21T11:59:56.187 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3128999999999996e-05 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.840553+0000", 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024521650000000001 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.840749+0000", 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00019673600000000001 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082065+0000", 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.241316108 2024-09-21T11:59:56.188 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082158+0000", 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2916000000000003e-05 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082222+0000", 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.3268999999999993e-05 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.189 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:85563 10.9 10:97d65cde:::benchmark_data_smithi043_150678_object85562:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.847840+0000", 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 371.275819974, 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.234536209, 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 85563 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.190 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847840+0000", 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847840+0000", 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847842+0000", 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.482e-06 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.191 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847894+0000", 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2577999999999997e-05 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847905+0000", 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0764000000000001e-05 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847911+0000", 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7960000000000001e-06 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.192 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.847964+0000", 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3365000000000002e-05 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848068+0000", 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000103872 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848154+0000", 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6026000000000006e-05 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.193 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.851858+0000", 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0037039949999999999 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.851882+0000", 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3261999999999999e-05 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082261+0000", 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.23037903200000001 2024-09-21T11:59:56.194 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082340+0000", 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.9379e-05 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.082377+0000", 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.6658e-05 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.195 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.25910.0:85564 10.9 10:98f1ca55:::benchmark_data_smithi043_150678_object85563:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:53:44.848489+0000", 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 371.275171457, 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24391052699999999, 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.25910", 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 85564 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.196 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848489+0000", 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848489+0000", 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848490+0000", 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.8100000000000002e-07 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.197 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848561+0000", 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1595999999999997e-05 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848569+0000", 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6760000000000007e-06 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.848573+0000", 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4850000000000003e-06 2024-09-21T11:59:56.198 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.850273+0000", 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001699952 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.850369+0000", 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5712999999999997e-05 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 2,7", 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.850456+0000", 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7298000000000003e-05 2024-09-21T11:59:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.851904+0000", 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001447439 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:44.852025+0000", 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000121432 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.092269+0000", 2024-09-21T11:59:56.200 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.24024353500000001 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.092357+0000", 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8367000000000003e-05 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:53:45.092399+0000", 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.2252999999999997e-05 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.201 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.26339.0:209769 11.7 11:e6ef2b8b:::benchmark_data_smithi043_178089_object88534:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e458)", 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:59:43.545261+0000", 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.578399332, 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.067876708, 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.26339", 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 209769 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.202 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.545261+0000", 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.545261+0000", 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.545263+0000", 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5430000000000001e-06 2024-09-21T11:59:56.203 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.545276+0000", 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2954e-05 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.545288+0000", 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2612000000000001e-05 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.545297+0000", 2024-09-21T11:59:56.204 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.8640000000000002e-06 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.546653+0000", 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001355583 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.546663+0000", 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0180000000000001e-05 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.605190+0000", 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.058526791 2024-09-21T11:59:56.205 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.605285+0000", 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5191000000000004e-05 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1", 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.605349+0000", 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.4733999999999994e-05 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.612971+0000", 2024-09-21T11:59:56.206 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0076211869999999998 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.613024+0000", 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3038999999999998e-05 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.613075+0000", 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1252000000000003e-05 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.613138+0000", 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.2778000000000001e-05 2024-09-21T11:59:56.207 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.209 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.209 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.209 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.209 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.209 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.26339.0:209780 11.7 11:e032b12f:::benchmark_data_smithi043_178089_object88545:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e459)", 2024-09-21T11:59:56.209 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:59:43.577023+0000", 2024-09-21T11:59:56.209 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.546637744, 2024-09-21T11:59:56.209 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0454195799999999, 2024-09-21T11:59:56.209 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.26339", 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 209780 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577023+0000", 2024-09-21T11:59:56.210 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577023+0000", 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577024+0000", 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.339e-06 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.211 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577028+0000", 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4329999999999997e-06 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577038+0000", 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0020999999999999e-05 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577044+0000", 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.711e-06 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577085+0000", 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1282999999999998e-05 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577099+0000", 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3653e-05 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.608369+0000", 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.031270337 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.213 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.608456+0000", 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.6978999999999995e-05 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1", 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.608576+0000", 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000119463 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.615710+0000", 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0071342580000000001 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.214 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.622286+0000", 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0065755099999999997 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.622347+0000", 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1340000000000006e-05 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.622442+0000", 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.5253000000000004e-05 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.215 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.26339.0:209781 11.7 11:e0bef711:::benchmark_data_smithi043_178089_object88546:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e459)", 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:59:43.577671+0000", 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.545989628999999, 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.051724399, 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.26339", 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 209781 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.216 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577671+0000", 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577671+0000", 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577671+0000", 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5799999999999998e-07 2024-09-21T11:59:56.217 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577676+0000", 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.0380000000000001e-06 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577684+0000", 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.7399999999999993e-06 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577689+0000", 2024-09-21T11:59:56.218 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9509999999999997e-06 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577726+0000", 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.7047e-05 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.577732+0000", 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1550000000000004e-06 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.610840+0000", 2024-09-21T11:59:56.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0331077769999999 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.610956+0000", 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00011556 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1", 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.611068+0000", 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000112478 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.619657+0000", 2024-09-21T11:59:56.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0085884350000000002 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.629285+0000", 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0096279090000000005 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.629320+0000", 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.4962000000000002e-05 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.629395+0000", 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5588999999999997e-05 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.26339.0:209785 11.7 11:eb559e09:::benchmark_data_smithi043_178089_object88550:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e459)", 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:59:43.582253+0000", 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.541407076, 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.047251755, 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.222 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.26339", 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 209785 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.582253+0000", 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.582253+0000", 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.582255+0000", 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.26e-06 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.582259+0000", 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.1699999999999999e-06 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.582269+0000", 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9739999999999997e-06 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.582274+0000", 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7370000000000003e-06 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.582313+0000", 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.8902000000000001e-05 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.582319+0000", 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.823e-06 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.613186+0000", 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0308667789999999 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.613301+0000", 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000114678 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1", 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.613425+0000", 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000123998 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.619667+0000", 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0062422770000000001 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.629428+0000", 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0097615040000000007 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.629458+0000", 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.9241e-05 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.629505+0000", 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.7411999999999999e-05 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.26339.0:209799 11.7 11:eae7f4b8:::benchmark_data_smithi043_178089_object88564:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e459)", 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:59:43.613098+0000", 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.510562609000001, 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0165171719999999, 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.228 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.26339", 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 209799 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.613098+0000", 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.613098+0000", 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.613099+0000", 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.513e-06 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.613104+0000", 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.4270000000000001e-06 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.613115+0000", 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1420000000000001e-05 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.230 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.613121+0000", 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8830000000000003e-06 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.613179+0000", 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.7855000000000001e-05 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:43.613189+0000", 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0175e-05 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.615750+0000", 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.00256123 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.615857+0000", 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000106873 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1", 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.615993+0000", 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000136131 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.619672+0000", 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0036785989999999998 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.629536+0000", 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0098644930000000002 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "commit_sent", 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.629564+0000", 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.7497000000000001e-05 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.629615+0000", 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.1075999999999997e-05 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.14445.0:3791 5.e 5:7759931f:::notify.3:head [watch reconnect cookie 93908746614400 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e459)", 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T11:59:44.029400+0000", 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 12.094259867, 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.53580533500000005, 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T11:59:56.234 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.14445", 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/2311158182", 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 3791 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.029400+0000", 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.029400+0000", 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T11:59:56.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.029403+0000", 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6060000000000001e-06 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.029411+0000", 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.3080000000000008e-06 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.029428+0000", 2024-09-21T11:59:56.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6186999999999999e-05 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.029436+0000", 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.1999999999999994e-06 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.029511+0000", 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5643999999999996e-05 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for peered", 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.029521+0000", 2024-09-21T11:59:56.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.764e-06 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.562088+0000", 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.53256632400000004 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.562212+0000", 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000124184 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T11:59:56.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T11:59:44.565206+0000", 2024-09-21T11:59:56.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002994118 2024-09-21T11:59:56.239 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.239 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.239 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.239 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T11:59:56.239 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T11:59:56.239 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T11:59:56.244 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:56.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:56.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:56.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:56.403 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:56.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:56.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:56.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:56.533 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:56.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:56.534 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:56.635 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:56.666 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:56.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:56.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:6.f 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:6.e 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:6.6 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:6.2 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:6.1e 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:6.7 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:4.1a 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:4.1 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:3.15 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:4.18 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:1.2 2024-09-21T11:59:56.681 INFO:teuthology.orchestra.run.smithi175.stdout:3.f 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:3.e 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:3.9 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:3.b 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:6.14 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:1.3 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:3.1d 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:3.19 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:5.0 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:1.5 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:11.7 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:4.13 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:11.e 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:4.f 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:3.13 2024-09-21T11:59:56.682 INFO:teuthology.orchestra.run.smithi175.stdout:5.18 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:3.6 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:11.d 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:4.1b 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:3.a 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:3.1b 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:3.14 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:3.11 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:3.3 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:3.1a 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:4.2 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:4.5 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:3.1 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:5.a 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:4.9 2024-09-21T11:59:56.683 INFO:teuthology.orchestra.run.smithi175.stdout:4.c 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:4.b 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:11.c 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:5.1 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:4.1d 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:4.d 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:4.e 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:5.10 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:5.11 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:5.1a 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:6.1d 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:5.1c 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:6.12 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:5.1d 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:5.1f 2024-09-21T11:59:56.684 INFO:teuthology.orchestra.run.smithi175.stdout:5.5 2024-09-21T11:59:56.685 INFO:teuthology.orchestra.run.smithi175.stdout:3.1c 2024-09-21T11:59:56.685 INFO:teuthology.orchestra.run.smithi175.stdout:5.9 2024-09-21T11:59:56.685 INFO:teuthology.orchestra.run.smithi175.stdout:5.c 2024-09-21T11:59:56.685 INFO:teuthology.orchestra.run.smithi175.stdout:6.17 2024-09-21T11:59:56.685 INFO:teuthology.orchestra.run.smithi175.stdout:5.e 2024-09-21T11:59:56.685 INFO:teuthology.orchestra.run.smithi175.stdout:5.f 2024-09-21T11:59:56.685 INFO:teuthology.orchestra.run.smithi175.stdout:6.15 2024-09-21T11:59:56.685 INFO:teuthology.orchestra.run.smithi175.stdout:6.1a 2024-09-21T11:59:56.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:56.821 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:56.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:56.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T11:59:56.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:56.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:56.983 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:57.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:57.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:57.136 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:57.237 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:57.269 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:57.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:57.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:57.370 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:57.393 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is set 2024-09-21T11:59:57.396 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:57.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:57.397 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:57.498 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:57.524 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:57.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:57.525 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:57.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:57.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:57.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:57.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:57 smithi043 ceph-mon[36454]: pgmap v978: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 921 op/s; 4718/70755 objects degraded (6.668%); 1610/70755 objects misplaced (2.275%) 2024-09-21T11:59:57.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:57 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2543119945' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:57.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:57 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:57 smithi043 ceph-mon[29710]: pgmap v978: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 921 op/s; 4718/70755 objects degraded (6.668%); 1610/70755 objects misplaced (2.275%) 2024-09-21T11:59:57.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2543119945' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:57.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:57 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:57.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:57 smithi175 ceph-mon[32613]: pgmap v978: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.4 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 921 op/s; 4718/70755 objects degraded (6.668%); 1610/70755 objects misplaced (2.275%) 2024-09-21T11:59:57.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:57 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2543119945' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:57.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:57 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-21T11:59:57.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:57.797 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:57.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:57.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:57.874 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T11:59:57.899 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:57.938 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:57.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:57.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:58.040 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:58.066 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:58.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:58.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:58.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:58.212 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:58.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:58.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:58.314 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:58.338 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:58.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:58.339 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:58.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:58.481 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:58.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:58.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:58.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:58 smithi175 ceph-mon[32613]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:58.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:58 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:58.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:58 smithi175 ceph-mon[32613]: osdmap e466: 8 total, 7 up, 8 in 2024-09-21T11:59:58.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:58 smithi175 ceph-mon[32613]: Health check update: Degraded data redundancy: 4718/70755 objects degraded (6.668%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T11:59:58.576 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:58 smithi175 ceph-mon[32613]: osdmap e467: 8 total, 7 up, 8 in 2024-09-21T11:59:58.583 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T11:59:58.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:58.612 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:58.612 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T11:59:58.624 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -n osd.4 --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op export --pgid 5.1 --file /var/log/ceph/exp.5.1.4 2024-09-21T11:59:58.649 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T11:59:58.713 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T11:59:58.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:58 smithi043 ceph-mon[36454]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:58.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:58 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:58.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:58 smithi043 ceph-mon[36454]: osdmap e466: 8 total, 7 up, 8 in 2024-09-21T11:59:58.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:58 smithi043 ceph-mon[36454]: Health check update: Degraded data redundancy: 4718/70755 objects degraded (6.668%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T11:59:58.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:58 smithi043 ceph-mon[36454]: osdmap e467: 8 total, 7 up, 8 in 2024-09-21T11:59:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:58 smithi043 ceph-mon[29710]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T11:59:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:58 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-21T11:59:58.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:58 smithi043 ceph-mon[29710]: osdmap e466: 8 total, 7 up, 8 in 2024-09-21T11:59:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:58 smithi043 ceph-mon[29710]: Health check update: Degraded data redundancy: 4718/70755 objects degraded (6.668%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T11:59:58.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:58 smithi043 ceph-mon[29710]: osdmap e467: 8 total, 7 up, 8 in 2024-09-21T11:59:58.751 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:58.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:58.752 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T11:59:58.853 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:58.890 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:58.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:58.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T11:59:58.919 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/osd.4/config 2024-09-21T11:59:58.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:59.022 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:59.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:59.023 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:59.123 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:59.147 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:59.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:59.148 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:59.248 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T11:59:59.272 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T11:59:59.273 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T11:59:59.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:59.295 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:59.295 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T11:59:59.304 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T11:59:59.305 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T11:59:59.338 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T11:59:59.339 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T11:59:59.376 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T11:59:59.376 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T11:59:59.382 INFO:teuthology.orchestra.run.smithi043.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T11:59:59.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T11:59:59.420 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T11:59:59.421 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T11:59:59.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:59.445 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:59.445 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T11:59:59.453 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T11:59:59.453 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T11:59:59.489 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T11:59:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T11:59:59.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T11:59:59.590 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:59.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:59.591 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T11:59:59.692 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:59.720 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:59.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:59.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:59 smithi043 ceph-mon[29710]: pgmap v981: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.2 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 880 B/s rd, 0 B/s wr, 1.04k op/s; 3674/60057 objects degraded (6.118%); 1222/60057 objects misplaced (2.035%) 2024-09-21T11:59:59.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 11:59:59 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/326369383' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:59 smithi043 ceph-mon[36454]: pgmap v981: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.2 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 880 B/s rd, 0 B/s wr, 1.04k op/s; 3674/60057 objects degraded (6.118%); 1222/60057 objects misplaced (2.035%) 2024-09-21T11:59:59.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 11:59:59 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/326369383' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:59.822 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T11:59:59.849 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T11:59:59.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:59.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T11:59:59.852 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T11:59:59.852 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T11:59:59.852 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:59 smithi175 ceph-mon[32613]: pgmap v981: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.2 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 880 B/s rd, 0 B/s wr, 1.04k op/s; 3674/60057 objects degraded (6.118%); 1222/60057 objects misplaced (2.035%) 2024-09-21T11:59:59.852 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 11:59:59 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/326369383' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T11:59:59.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T11:59:59.986 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T11:59:59.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T11:59:59.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:00.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:00.129 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:00.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:00.130 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:00.231 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:00.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:00.263 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:00.263 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:00.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:00.403 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:00.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:00.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:00.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:00.531 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:00.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:00.532 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:00.607 INFO:teuthology.orchestra.run.smithi175.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-21T12:00:00.633 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:00.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:00.676 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:00.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:00.777 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:00.820 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:00.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:00.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:00.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 1 osds down; Degraded data redundancy: 3674/60057 objects degraded (6.118%), 25 pgs degraded 2024-09-21T12:00:00.898 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: [WRN] OSD_DOWN: 1 osds down 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: osd.4 (root=default,host=smithi175) is down 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: [WRN] PG_DEGRADED: Degraded data redundancy: 3674/60057 objects degraded (6.118%), 25 pgs degraded 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 3.3 is active+undersized+degraded, acting [0,6] 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 3.6 is active+undersized+degraded, acting [0,1] 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 3.f is active+undersized+degraded, acting [7,0] 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 3.19 is active+undersized+degraded, acting [1,3] 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.1 is active+undersized+degraded, acting [5,6] 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.2 is active+undersized+degraded, acting [1,5] 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.5 is active+undersized+degraded, acting [6,7] 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.9 is active+undersized+degraded, acting [1,3] 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.b is active+undersized+degraded, acting [0,1] 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.c is active+undersized+degraded, acting [3,6] 2024-09-21T12:00:00.899 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.d is active+undersized+degraded, acting [2,5] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.e is active+undersized+degraded, acting [6,1] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.f is active+undersized+degraded, acting [1,3] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.13 is active+undersized+degraded, acting [6,2] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.18 is active+undersized+degraded, acting [6,1] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.1a is active+undersized+degraded, acting [3,0] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.1b is active+undersized+degraded, acting [3,1] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 4.1d is active+undersized+degraded, acting [3,1] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 5.0 is active+undersized+degraded, acting [3,1] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 5.9 is active+undersized+degraded, acting [7,6] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 5.e is active+undersized+degraded, acting [5,0] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 5.1f is active+undersized+degraded, acting [6,7] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 11.7 is active+undersized+degraded, acting [5,1] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 11.c is active+undersized+degraded, acting [1,2] 2024-09-21T12:00:00.900 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[36454]: pg 11.d is active+undersized+degraded, acting [2,6] 2024-09-21T12:00:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 1 osds down; Degraded data redundancy: 3674/60057 objects degraded (6.118%), 25 pgs degraded 2024-09-21T12:00:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-21T12:00:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: [WRN] OSD_DOWN: 1 osds down 2024-09-21T12:00:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: osd.4 (root=default,host=smithi175) is down 2024-09-21T12:00:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: [WRN] PG_DEGRADED: Degraded data redundancy: 3674/60057 objects degraded (6.118%), 25 pgs degraded 2024-09-21T12:00:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 3.3 is active+undersized+degraded, acting [0,6] 2024-09-21T12:00:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 3.6 is active+undersized+degraded, acting [0,1] 2024-09-21T12:00:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 3.f is active+undersized+degraded, acting [7,0] 2024-09-21T12:00:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 3.19 is active+undersized+degraded, acting [1,3] 2024-09-21T12:00:00.901 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.1 is active+undersized+degraded, acting [5,6] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.2 is active+undersized+degraded, acting [1,5] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.5 is active+undersized+degraded, acting [6,7] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.9 is active+undersized+degraded, acting [1,3] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.b is active+undersized+degraded, acting [0,1] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.c is active+undersized+degraded, acting [3,6] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.d is active+undersized+degraded, acting [2,5] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.e is active+undersized+degraded, acting [6,1] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.f is active+undersized+degraded, acting [1,3] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.13 is active+undersized+degraded, acting [6,2] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.18 is active+undersized+degraded, acting [6,1] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.1a is active+undersized+degraded, acting [3,0] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.1b is active+undersized+degraded, acting [3,1] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 4.1d is active+undersized+degraded, acting [3,1] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 5.0 is active+undersized+degraded, acting [3,1] 2024-09-21T12:00:00.902 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 5.9 is active+undersized+degraded, acting [7,6] 2024-09-21T12:00:00.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 5.e is active+undersized+degraded, acting [5,0] 2024-09-21T12:00:00.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 5.1f is active+undersized+degraded, acting [6,7] 2024-09-21T12:00:00.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 11.7 is active+undersized+degraded, acting [5,1] 2024-09-21T12:00:00.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 11.c is active+undersized+degraded, acting [1,2] 2024-09-21T12:00:00.903 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:00 smithi043 ceph-mon[29710]: pg 11.d is active+undersized+degraded, acting [2,6] 2024-09-21T12:00:00.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:00.950 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T12:00:00.950 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T12:00:00.969 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:00.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:00.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set; 1 osds down; Degraded data redundancy: 3674/60057 objects degraded (6.118%), 25 pgs degraded 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: [WRN] OSD_DOWN: 1 osds down 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: osd.4 (root=default,host=smithi175) is down 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: [WRN] PG_DEGRADED: Degraded data redundancy: 3674/60057 objects degraded (6.118%), 25 pgs degraded 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 3.3 is active+undersized+degraded, acting [0,6] 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 3.6 is active+undersized+degraded, acting [0,1] 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 3.f is active+undersized+degraded, acting [7,0] 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 3.19 is active+undersized+degraded, acting [1,3] 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.1 is active+undersized+degraded, acting [5,6] 2024-09-21T12:00:01.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.2 is active+undersized+degraded, acting [1,5] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.5 is active+undersized+degraded, acting [6,7] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.9 is active+undersized+degraded, acting [1,3] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.b is active+undersized+degraded, acting [0,1] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.c is active+undersized+degraded, acting [3,6] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.d is active+undersized+degraded, acting [2,5] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.e is active+undersized+degraded, acting [6,1] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.f is active+undersized+degraded, acting [1,3] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.13 is active+undersized+degraded, acting [6,2] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.18 is active+undersized+degraded, acting [6,1] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.1a is active+undersized+degraded, acting [3,0] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.1b is active+undersized+degraded, acting [3,1] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 4.1d is active+undersized+degraded, acting [3,1] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 5.0 is active+undersized+degraded, acting [3,1] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 5.9 is active+undersized+degraded, acting [7,6] 2024-09-21T12:00:01.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 5.e is active+undersized+degraded, acting [5,0] 2024-09-21T12:00:01.013 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 5.1f is active+undersized+degraded, acting [6,7] 2024-09-21T12:00:01.013 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 11.7 is active+undersized+degraded, acting [5,1] 2024-09-21T12:00:01.013 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 11.c is active+undersized+degraded, acting [1,2] 2024-09-21T12:00:01.013 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:00 smithi175 ceph-mon[32613]: pg 11.d is active+undersized+degraded, acting [2,6] 2024-09-21T12:00:01.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:01.101 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:01.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:01.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:01.128 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T12:00:01.203 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:01.243 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:01.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:01.244 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:01.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:01.384 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:01.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:01.385 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:01.486 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:01.513 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:01.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:01.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:01.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:01.624 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:01 smithi175 ceph-mon[32613]: pgmap v982: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.2 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 667 B/s rd, 0 B/s wr, 790 op/s; 3674/60057 objects degraded (6.118%); 1222/60057 objects misplaced (2.035%) 2024-09-21T12:00:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:01.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:01.658 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:01.700 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:01.759 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:01.787 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:01.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:01.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:01.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:01.925 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:01.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:01.926 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:01.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:01 smithi043 ceph-mon[36454]: pgmap v982: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.2 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 667 B/s rd, 0 B/s wr, 790 op/s; 3674/60057 objects degraded (6.118%); 1222/60057 objects misplaced (2.035%) 2024-09-21T12:00:01.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:01 smithi043 ceph-mon[29710]: pgmap v982: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 1.2 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 667 B/s rd, 0 B/s wr, 790 op/s; 3674/60057 objects degraded (6.118%); 1222/60057 objects misplaced (2.035%) 2024-09-21T12:00:02.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:02.064 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:02.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:02.065 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:02.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:02.202 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:02.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:02.203 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:02.262 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:00:01 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:00:02.304 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:02.340 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:02.340 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:02.345 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:02.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:02.346 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:02.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:02.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:02.486 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:02.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:02.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:02.620 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:02.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:02.621 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:02.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:02.762 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:02.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:02.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:02.864 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:02.891 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:02.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:02.892 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:02.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T12:00:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:00:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:00:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[36454]: osdmap e468: 8 total, 7 up, 8 in 2024-09-21T12:00:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[36454]: Health check update: Degraded data redundancy: 3674/60057 objects degraded (6.118%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:02.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[36454]: osdmap e469: 8 total, 7 up, 8 in 2024-09-21T12:00:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:00:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:00:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[29710]: osdmap e468: 8 total, 7 up, 8 in 2024-09-21T12:00:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[29710]: Health check update: Degraded data redundancy: 3674/60057 objects degraded (6.118%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:02.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:02 smithi043 ceph-mon[29710]: osdmap e469: 8 total, 7 up, 8 in 2024-09-21T12:00:02.993 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:03.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T12:00:03.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:02 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:00:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:02 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:00:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:02 smithi175 ceph-mon[32613]: osdmap e468: 8 total, 7 up, 8 in 2024-09-21T12:00:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:02 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:02 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:02 smithi175 ceph-mon[32613]: Health check update: Degraded data redundancy: 3674/60057 objects degraded (6.118%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:03.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:02 smithi175 ceph-mon[32613]: osdmap e469: 8 total, 7 up, 8 in 2024-09-21T12:00:03.018 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:03.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:03.019 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:03.120 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:03.147 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:03.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:03.148 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:03.248 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:03.294 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:03.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:03.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:03.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:03.440 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:03.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:03.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:03.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:03.562 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:03.574 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:03.600 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:03.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:03.601 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:03.702 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:03.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:03.742 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:03.742 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:03.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:03.881 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:03.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:03.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:03.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:03 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:03.931 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:03 smithi175 ceph-mon[32613]: pgmap v984: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 917 MiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.46k op/s; 2636/44580 objects degraded (5.913%); 847/44580 objects misplaced (1.900%) 2024-09-21T12:00:03.983 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:03 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:03.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:03 smithi043 ceph-mon[36454]: pgmap v984: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 917 MiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.46k op/s; 2636/44580 objects degraded (5.913%); 847/44580 objects misplaced (1.900%) 2024-09-21T12:00:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:03 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:03.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:03 smithi043 ceph-mon[29710]: pgmap v984: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 917 MiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.46k op/s; 2636/44580 objects degraded (5.913%); 847/44580 objects misplaced (1.900%) 2024-09-21T12:00:04.010 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:04.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:04.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:04.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:04.133 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:04.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:04.134 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:04.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:04.257 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:04.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:04.258 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:04.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:04.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:04.399 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:04.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:04.520 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:04.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:04.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:04.622 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:04.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:04.657 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:04.657 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:04.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:04.792 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:04.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:04.793 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:04.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:04.926 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:04.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:04.927 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:05.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:05.063 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T12:00:05.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:05.074 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:05.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:05.175 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:05.203 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:05.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:05.204 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:05.305 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:05.319 INFO:teuthology.orchestra.run.smithi175.stderr:Exporting 5.1 info 5.1( empty local-lis/les=293/294 n=0 ec=62/62 lis/c=293/293 les/c/f=294/294/0 sis=293) 2024-09-21T12:00:05.319 INFO:teuthology.orchestra.run.smithi175.stderr:Export successful 2024-09-21T12:00:05.340 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:05.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:05.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:05.415 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:00:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:00:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:05.416 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:05.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:05.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:05.483 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:05.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:05.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:05.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:05.622 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:05.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:05.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:05.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:05 smithi043 ceph-mon[29710]: pgmap v986: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 893 MiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 884 op/s; 2636/43458 objects degraded (6.066%); 847/43458 objects misplaced (1.949%) 2024-09-21T12:00:05.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:05 smithi043 ceph-mon[36454]: pgmap v986: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 893 MiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 884 op/s; 2636/43458 objects degraded (6.066%); 847/43458 objects misplaced (1.949%) 2024-09-21T12:00:05.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:05.777 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:05.823 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:05 smithi175 ceph-mon[32613]: pgmap v986: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 893 MiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 884 op/s; 2636/43458 objects degraded (6.066%); 847/43458 objects misplaced (1.949%) 2024-09-21T12:00:05.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:05.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:05.915 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:05.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:05.967 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:05.967 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:06.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:06.054 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:06.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:06.055 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:06.156 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:06.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:06.200 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:06.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:06.301 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:06.329 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:06.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:06.330 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:06.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:06.449 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T12:00:06.450 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T12:00:06.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:06.470 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:06.470 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:06.489 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T12:00:06.489 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T12:00:06.524 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T12:00:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T12:00:06.559 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T12:00:06.559 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T12:00:06.562 INFO:teuthology.orchestra.run.smithi043.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T12:00:06.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:06.596 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T12:00:06.596 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T12:00:06.612 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:06.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:06.613 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:06.619 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T12:00:06.634 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T12:00:06.635 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T12:00:06.671 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T12:00:06.671 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T12:00:06.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:06.755 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:06.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:06.756 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:06.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:06.857 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:06 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1558156976' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T12:00:06.857 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:06 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T12:00:06.857 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:06 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3052242998' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:06.858 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -n osd.4 --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --force --op remove --pgid 5.1 2024-09-21T12:00:06.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:06 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1558156976' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T12:00:06.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:06 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T12:00:06.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:06 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3052242998' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:06.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:06 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1558156976' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T12:00:06.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:06 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T12:00:06.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:06 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3052242998' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:06.894 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:06.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:06.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:06.996 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:07.023 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:07.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:07.024 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:07.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:07.164 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:07.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:07.165 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:07.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:07.295 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:07.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:07.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:07.299 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/osd.4/config 2024-09-21T12:00:07.322 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T12:00:07.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:07.440 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:07.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:07.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:07.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:07.586 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:07.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:07.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:07.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:07.704 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T12:00:07.704 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T12:00:07.733 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:07.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:07.735 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:07.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:07.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:07.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:07.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[36454]: pgmap v987: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 849 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 781 op/s; 2636/41307 objects degraded (6.381%); 847/41307 objects misplaced (2.050%) 2024-09-21T12:00:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[36454]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T12:00:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T12:00:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[36454]: osdmap e470: 8 total, 7 up, 8 in 2024-09-21T12:00:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T12:00:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T12:00:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[36454]: Health check update: Degraded data redundancy: 2636/41307 objects degraded (6.381%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:07.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[36454]: osdmap e471: 8 total, 7 up, 8 in 2024-09-21T12:00:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[29710]: pgmap v987: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 849 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 781 op/s; 2636/41307 objects degraded (6.381%); 847/41307 objects misplaced (2.050%) 2024-09-21T12:00:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[29710]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T12:00:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T12:00:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[29710]: osdmap e470: 8 total, 7 up, 8 in 2024-09-21T12:00:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T12:00:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T12:00:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[29710]: Health check update: Degraded data redundancy: 2636/41307 objects degraded (6.381%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:07.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:07 smithi043 ceph-mon[29710]: osdmap e471: 8 total, 7 up, 8 in 2024-09-21T12:00:08.004 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:08.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:08.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:07 smithi175 ceph-mon[32613]: pgmap v987: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 849 MiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 781 op/s; 2636/41307 objects degraded (6.381%); 847/41307 objects misplaced (2.050%) 2024-09-21T12:00:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:07 smithi175 ceph-mon[32613]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T12:00:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:07 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T12:00:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:07 smithi175 ceph-mon[32613]: osdmap e470: 8 total, 7 up, 8 in 2024-09-21T12:00:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:07 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T12:00:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:07 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T12:00:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:07 smithi175 ceph-mon[32613]: Health check update: Degraded data redundancy: 2636/41307 objects degraded (6.381%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:08.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:07 smithi175 ceph-mon[32613]: osdmap e471: 8 total, 7 up, 8 in 2024-09-21T12:00:08.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:08.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:08.146 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:08.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:08.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:08.283 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:08.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:08.284 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:08.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:08.425 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:08.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:08.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:08.527 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:08.552 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:08.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:08.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:08.655 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:08.667 INFO:teuthology.orchestra.run.smithi175.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-21T12:00:08.684 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:08.692 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:08.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:08.693 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:08.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:08.818 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:08.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:08.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:08.920 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:08.949 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:08.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:08.950 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:09.050 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:09.083 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:09.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:09.084 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:09.185 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:09.212 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:09.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:09.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:09.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:09.337 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:09.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:09.338 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:09.439 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:09.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:09.467 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:09.468 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:09.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:09.612 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:09.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:09.613 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:09.703 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:09 smithi175 ceph-mon[32613]: pgmap v990: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 564 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 942 op/s; 1687/27660 objects degraded (6.099%); 534/27660 objects misplaced (1.931%) 2024-09-21T12:00:09.703 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:09 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T12:00:09.703 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:09 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T12:00:09.703 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T12:00:09.703 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T12:00:09.703 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.703 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.26339.0:237125 11.f 11:fd11117e:::benchmark_data_smithi043_178089_object115890:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e471)", 2024-09-21T12:00:09.703 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:09.694057+0000", 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0070634249999999999, 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0070821410000000001, 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.26339", 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 237125 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.694057+0000", 2024-09-21T12:00:09.704 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.694057+0000", 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.694059+0000", 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4449999999999999e-06 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.694064+0000", 2024-09-21T12:00:09.705 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2870000000000004e-06 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.694078+0000", 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3788000000000001e-05 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.694086+0000", 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7340000000000008e-06 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:09.706 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.694146+0000", 2024-09-21T12:00:09.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.0231e-05 2024-09-21T12:00:09.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:09.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.694266+0000", 2024-09-21T12:00:09.707 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012005700000000001 2024-09-21T12:00:09.707 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.707 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.707 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:09.707 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.694379+0000", 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113037 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.698540+0000", 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0041612849999999998 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.701036+0000", 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002495986 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:09.708 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "osd_op(client.26339.0:237127 11.f 11:f3832803:::benchmark_data_smithi043_178089_object115892:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e471)", 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:09.695810+0000", 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 0.0053106430000000003, 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0053743389999999997, 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "waiting for sub ops", 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_info": { 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client": "client.26339", 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "tid": 237127 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.709 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.695810+0000", 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.695810+0000", 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.695811+0000", 2024-09-21T12:00:09.710 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.5099999999999999e-07 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.695818+0000", 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8789999999999997e-06 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.695828+0000", 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8959999999999998e-06 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.695833+0000", 2024-09-21T12:00:09.711 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6699999999999999e-06 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.696404+0000", 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00057012700000000003 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.696481+0000", 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.7108999999999996e-05 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:09.712 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.696549+0000", 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8477000000000002e-05 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "op_commit", 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.698672+0000", 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.002123163 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:09.701074+0000", 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0024013939999999998 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:09.713 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:09.714 INFO:teuthology.orchestra.run.smithi175.stdout: ], 2024-09-21T12:00:09.714 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 2 2024-09-21T12:00:09.714 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T12:00:09.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T12:00:09.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:09.761 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:09.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:09.762 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:09.863 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:09.891 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:09.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:09.892 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:09 smithi043 ceph-mon[36454]: pgmap v990: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 564 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 942 op/s; 1687/27660 objects degraded (6.099%); 534/27660 objects misplaced (1.931%) 2024-09-21T12:00:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:09 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T12:00:09.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:09 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T12:00:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:09 smithi043 ceph-mon[29710]: pgmap v990: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 564 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 942 op/s; 1687/27660 objects degraded (6.099%); 534/27660 objects misplaced (1.931%) 2024-09-21T12:00:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:09 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T12:00:09.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:09 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T12:00:09.993 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:10.019 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:10.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:10.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:10.121 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:10.146 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:10.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:10.147 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:10.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T12:00:10.248 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:10.278 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:10.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:10.279 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:10.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:10.408 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:10.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:10.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:10.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:10.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T12:00:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:10.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:10.572 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:10.672 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:10.707 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:10.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:10.709 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:10.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:10 smithi175 ceph-mon[32613]: 2.2 scrub starts 2024-09-21T12:00:10.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:10 smithi175 ceph-mon[32613]: 2.2 scrub ok 2024-09-21T12:00:10.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:10.843 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:10 smithi043 ceph-mon[36454]: 2.2 scrub starts 2024-09-21T12:00:10.843 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:10 smithi043 ceph-mon[36454]: 2.2 scrub ok 2024-09-21T12:00:10.844 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:10 smithi043 ceph-mon[29710]: 2.2 scrub starts 2024-09-21T12:00:10.844 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:10 smithi043 ceph-mon[29710]: 2.2 scrub ok 2024-09-21T12:00:10.851 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:10.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:10.852 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:10.894 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:10.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:10.959 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T12:00:10.959 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T12:00:10.995 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:10.999 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T12:00:11.000 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T12:00:11.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:11.006 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:11.043 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T12:00:11.043 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T12:00:11.077 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T12:00:11.078 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T12:00:11.078 INFO:teuthology.orchestra.run.smithi043.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T12:00:11.107 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:11.111 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T12:00:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T12:00:11.134 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:11.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:11.135 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:11.147 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T12:00:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T12:00:11.183 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T12:00:11.183 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T12:00:11.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:11.264 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:11.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:11.265 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:11.366 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:11.399 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:11.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:11.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:11.496 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:11.496 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:11.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:11.535 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:11.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:11.536 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:11.637 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:11.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:11.666 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:11.667 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:11.767 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:11.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:11.791 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:11.791 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:11.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:11.915 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:11.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:11.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:11.989 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:11 smithi175 ceph-mon[32613]: pgmap v991: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 564 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 737 op/s; 1687/27660 objects degraded (6.099%); 534/27660 objects misplaced (1.931%) 2024-09-21T12:00:11.989 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:11 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1103306589' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:11.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:11 smithi043 ceph-mon[36454]: pgmap v991: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 564 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 737 op/s; 1687/27660 objects degraded (6.099%); 534/27660 objects misplaced (1.931%) 2024-09-21T12:00:11.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:11 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1103306589' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:11 smithi043 ceph-mon[29710]: pgmap v991: 158 pgs: 1 active+clean+remapped, 39 active+undersized, 25 active+undersized+degraded, 93 active+clean; 564 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 737 op/s; 1687/27660 objects degraded (6.099%); 534/27660 objects misplaced (1.931%) 2024-09-21T12:00:11.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:11 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1103306589' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:12.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:12.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:12.057 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:12.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:12.158 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:12.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:12.192 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:12.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:12.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:00:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:00:12.277 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T12:00:12.294 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:12.355 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:12.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:12.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:12.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:12.479 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:12.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:12.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:12.569 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:12.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:12.604 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:12.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:12.605 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:12.705 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:12.732 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:12.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:12.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:12.834 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:12.864 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:12.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:12.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:12.914 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:12 smithi175 ceph-mon[32613]: 5.17 scrub starts 2024-09-21T12:00:12.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:12 smithi175 ceph-mon[32613]: 5.17 scrub ok 2024-09-21T12:00:12.915 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:12 smithi175 ceph-mon[32613]: Health check update: Degraded data redundancy: 1687/27660 objects degraded (6.099%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:12.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:12 smithi043 ceph-mon[29710]: 5.17 scrub starts 2024-09-21T12:00:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:12 smithi043 ceph-mon[29710]: 5.17 scrub ok 2024-09-21T12:00:12.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:12 smithi043 ceph-mon[29710]: Health check update: Degraded data redundancy: 1687/27660 objects degraded (6.099%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:12 smithi043 ceph-mon[36454]: 5.17 scrub starts 2024-09-21T12:00:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:12 smithi043 ceph-mon[36454]: 5.17 scrub ok 2024-09-21T12:00:12.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:12 smithi043 ceph-mon[36454]: Health check update: Degraded data redundancy: 1687/27660 objects degraded (6.099%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:13.005 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:13.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:13.007 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:13.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:13.138 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:13.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:13.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:13.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:13.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:13.277 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:13.278 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:13.343 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:13.344 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:13.378 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:13.415 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:13.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:13.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:13.517 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:13.544 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:13.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:13.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:13.646 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:13.673 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:13.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:13.674 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:13.775 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:13.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:13.802 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:13.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:13.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:13.908 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:13.908 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[36454]: pgmap v992: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 39 active+undersized, 25 active+undersized+degraded, 92 active+clean; 234 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 47 KiB/s rd, 0 B/s wr, 1.40k op/s; 668/11790 objects degraded (5.666%); 172/11790 objects misplaced (1.459%) 2024-09-21T12:00:13.908 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[36454]: 11.8 scrub starts 2024-09-21T12:00:13.908 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[36454]: 11.8 scrub ok 2024-09-21T12:00:13.908 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[36454]: 5.6 scrub starts 2024-09-21T12:00:13.908 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[36454]: 5.6 scrub ok 2024-09-21T12:00:13.908 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:13.908 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[29710]: pgmap v992: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 39 active+undersized, 25 active+undersized+degraded, 92 active+clean; 234 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 47 KiB/s rd, 0 B/s wr, 1.40k op/s; 668/11790 objects degraded (5.666%); 172/11790 objects misplaced (1.459%) 2024-09-21T12:00:13.908 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[29710]: 11.8 scrub starts 2024-09-21T12:00:13.909 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[29710]: 11.8 scrub ok 2024-09-21T12:00:13.909 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[29710]: 5.6 scrub starts 2024-09-21T12:00:13.909 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:13 smithi043 ceph-mon[29710]: 5.6 scrub ok 2024-09-21T12:00:13.943 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:13.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:13.944 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:14.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:13 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:13 smithi175 ceph-mon[32613]: pgmap v992: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 39 active+undersized, 25 active+undersized+degraded, 92 active+clean; 234 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 47 KiB/s rd, 0 B/s wr, 1.40k op/s; 668/11790 objects degraded (5.666%); 172/11790 objects misplaced (1.459%) 2024-09-21T12:00:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:13 smithi175 ceph-mon[32613]: 11.8 scrub starts 2024-09-21T12:00:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:13 smithi175 ceph-mon[32613]: 11.8 scrub ok 2024-09-21T12:00:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:13 smithi175 ceph-mon[32613]: 5.6 scrub starts 2024-09-21T12:00:14.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:13 smithi175 ceph-mon[32613]: 5.6 scrub ok 2024-09-21T12:00:14.045 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:14.070 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:14.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:14.071 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:14.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:14.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:14.212 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:14.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:14.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:14.340 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:14.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:14.341 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:14.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:14.482 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:14.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:14.483 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:14.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:14.621 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:14.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:14.622 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:14.666 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T12:00:14.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-21T12:00:14.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:14.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:14.796 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:14.796 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:14.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:14.939 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:14.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:14.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:14.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2945915259' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T12:00:14.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2945915259' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T12:00:15.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2945915259' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T12:00:15.042 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:15.069 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:15.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:15.070 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:15.147 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T12:00:15.147 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T12:00:15.171 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:15.174 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Removed 121232 objects 2024-09-21T12:00:15.174 INFO:tasks.radosbench.radosbench.0.smithi043.stdout:Clean up completed and total clean up time :124.453 2024-09-21T12:00:15.193 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:15.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:15.194 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:15.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:15.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:15.317 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:15.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:15.382 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:00:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:15.419 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:15.423 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:15.441 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:15.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:15.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:15.543 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:15.564 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:15.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:15.565 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:15.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:15.688 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:15.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:15.689 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:15.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:15.811 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:15.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:15.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:15.913 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:15.934 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:15.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:15.935 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[29710]: pgmap v993: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 39 active+undersized, 25 active+undersized+degraded, 92 active+clean; 213 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 1.36k op/s; 668/10785 objects degraded (6.194%); 172/10785 objects misplaced (1.595%) 2024-09-21T12:00:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:00:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:00:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T12:00:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2945915259' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T12:00:15.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[29710]: osdmap e472: 8 total, 7 up, 8 in 2024-09-21T12:00:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[29710]: 1.6 scrub starts 2024-09-21T12:00:15.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[29710]: 1.6 scrub ok 2024-09-21T12:00:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[36454]: pgmap v993: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 39 active+undersized, 25 active+undersized+degraded, 92 active+clean; 213 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 1.36k op/s; 668/10785 objects degraded (6.194%); 172/10785 objects misplaced (1.595%) 2024-09-21T12:00:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:00:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:00:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T12:00:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2945915259' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T12:00:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[36454]: osdmap e472: 8 total, 7 up, 8 in 2024-09-21T12:00:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[36454]: 1.6 scrub starts 2024-09-21T12:00:15.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:15 smithi043 ceph-mon[36454]: 1.6 scrub ok 2024-09-21T12:00:16.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:15 smithi175 ceph-mon[32613]: pgmap v993: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 39 active+undersized, 25 active+undersized+degraded, 92 active+clean; 213 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 1.36k op/s; 668/10785 objects degraded (6.194%); 172/10785 objects misplaced (1.595%) 2024-09-21T12:00:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:15 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:00:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:15 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:00:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:15 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-21T12:00:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2945915259' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T12:00:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:15 smithi175 ceph-mon[32613]: osdmap e472: 8 total, 7 up, 8 in 2024-09-21T12:00:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:15 smithi175 ceph-mon[32613]: 1.6 scrub starts 2024-09-21T12:00:16.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:15 smithi175 ceph-mon[32613]: 1.6 scrub ok 2024-09-21T12:00:16.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:16.060 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:16.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:16.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:16.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:16.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:16.185 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:16.185 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:16.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:16.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:16.312 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:16.383 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:16.383 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:16.402 INFO:teuthology.orchestra.run.smithi175.stdout: marking collection for removal 2024-09-21T12:00:16.402 INFO:teuthology.orchestra.run.smithi175.stdout:setting '_remove' omap key 2024-09-21T12:00:16.402 INFO:teuthology.orchestra.run.smithi175.stdout:finish_remove_pgs 5.1_head removing 5.1 2024-09-21T12:00:16.403 INFO:teuthology.orchestra.run.smithi175.stdout:Remove successful 2024-09-21T12:00:16.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:16.436 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:16.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:16.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:16.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:16.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:16.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:16.663 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:16.685 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:16.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:16.686 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:16.787 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:16.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:16.810 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:16.810 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:16.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:16.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:16 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:00:16.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:16 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:00:16.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:16 smithi175 ceph-mon[32613]: osdmap e473: 8 total, 7 up, 8 in 2024-09-21T12:00:16.933 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:16.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:16.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:16 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:00:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:16 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:00:16.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:16 smithi043 ceph-mon[36454]: osdmap e473: 8 total, 7 up, 8 in 2024-09-21T12:00:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:16 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:00:16.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:16 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:00:16.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:16 smithi043 ceph-mon[29710]: osdmap e473: 8 total, 7 up, 8 in 2024-09-21T12:00:17.035 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:17.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:17.057 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:17.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:17.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-21T12:00:17.158 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:17.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-21T12:00:17.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:17.180 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:17.180 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:17.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:17.305 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:17.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:17.305 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:17.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:17.418 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:17.431 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:17.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:17.432 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:17.533 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:17.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:17.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:17.650 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell -n osd.4 --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.5.1.4 2024-09-21T12:00:17.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:17.681 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:17.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:17.682 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:17.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:17.805 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:17.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:17.805 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:17.841 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/osd.4/config 2024-09-21T12:00:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[29710]: pgmap v996: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 39 active+undersized, 25 active+undersized+degraded, 92 active+clean; 179 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 53 KiB/s rd, 0 B/s wr, 859 op/s; 668/9156 objects degraded (7.296%); 172/9156 objects misplaced (1.879%) 2024-09-21T12:00:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[29710]: 6.1b scrub starts 2024-09-21T12:00:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[29710]: 6.1b scrub ok 2024-09-21T12:00:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:17.860 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[29710]: Health check update: Degraded data redundancy: 668/9156 objects degraded (7.296%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:17.860 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[36454]: pgmap v996: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 39 active+undersized, 25 active+undersized+degraded, 92 active+clean; 179 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 53 KiB/s rd, 0 B/s wr, 859 op/s; 668/9156 objects degraded (7.296%); 172/9156 objects misplaced (1.879%) 2024-09-21T12:00:17.861 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[36454]: 6.1b scrub starts 2024-09-21T12:00:17.861 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[36454]: 6.1b scrub ok 2024-09-21T12:00:17.861 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:17.861 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:17.861 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:17 smithi043 ceph-mon[36454]: Health check update: Degraded data redundancy: 668/9156 objects degraded (7.296%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:17.906 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:17.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:17.930 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:17.930 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:17.956 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:17 smithi175 ceph-mon[32613]: pgmap v996: 158 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 39 active+undersized, 25 active+undersized+degraded, 92 active+clean; 179 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 53 KiB/s rd, 0 B/s wr, 859 op/s; 668/9156 objects degraded (7.296%); 172/9156 objects misplaced (1.879%) 2024-09-21T12:00:17.956 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:17 smithi175 ceph-mon[32613]: 6.1b scrub starts 2024-09-21T12:00:17.956 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:17 smithi175 ceph-mon[32613]: 6.1b scrub ok 2024-09-21T12:00:17.956 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:17 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:17.957 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:17 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:17.957 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:17 smithi175 ceph-mon[32613]: Health check update: Degraded data redundancy: 668/9156 objects degraded (7.296%), 25 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:17.962 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T12:00:17.962 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T12:00:17.979 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T12:00:17.979 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T12:00:17.995 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T12:00:17.996 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T12:00:18.013 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T12:00:18.014 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T12:00:18.014 INFO:teuthology.orchestra.run.smithi043.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T12:00:18.014 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:18.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:18.032 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T12:00:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T12:00:18.050 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T12:00:18.050 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T12:00:18.055 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:18.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:18.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:18.069 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T12:00:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T12:00:18.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd set noscrub 2024-09-21T12:00:18.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:18.200 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:18.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:18.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:18.302 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:18.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:18.325 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:18.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:18.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:18.449 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:18.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:18.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:18.551 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:18.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:18.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:18.676 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:18.698 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:18.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:18.699 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:18.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:18.823 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:18.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:18.824 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:18.924 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:18.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:18.948 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:18.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:18 smithi043 ceph-mon[29710]: 6.8 scrub starts 2024-09-21T12:00:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:18 smithi043 ceph-mon[29710]: 6.8 scrub ok 2024-09-21T12:00:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:18 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/294018873' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:18 smithi043 ceph-mon[29710]: 4.0 scrub starts 2024-09-21T12:00:18.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:18 smithi043 ceph-mon[29710]: 4.0 scrub ok 2024-09-21T12:00:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:18 smithi043 ceph-mon[36454]: 6.8 scrub starts 2024-09-21T12:00:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:18 smithi043 ceph-mon[36454]: 6.8 scrub ok 2024-09-21T12:00:18.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:18 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/294018873' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:18 smithi043 ceph-mon[36454]: 4.0 scrub starts 2024-09-21T12:00:18.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:18 smithi043 ceph-mon[36454]: 4.0 scrub ok 2024-09-21T12:00:19.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:18 smithi175 ceph-mon[32613]: 6.8 scrub starts 2024-09-21T12:00:19.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:18 smithi175 ceph-mon[32613]: 6.8 scrub ok 2024-09-21T12:00:19.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:18 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/294018873' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:19.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:18 smithi175 ceph-mon[32613]: 4.0 scrub starts 2024-09-21T12:00:19.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:18 smithi175 ceph-mon[32613]: 4.0 scrub ok 2024-09-21T12:00:19.049 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:19.074 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:19.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:19.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:19.098 INFO:teuthology.orchestra.run.smithi175.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-21T12:00:19.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:19.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:19.205 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:19.306 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:19.329 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:19.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:19.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:19.408 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T12:00:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [], 2024-09-21T12:00:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: "complaint_time": 30, 2024-09-21T12:00:19.408 INFO:teuthology.orchestra.run.smithi043.stdout: "num_blocked_ops": 0 2024-09-21T12:00:19.408 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T12:00:19.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:19.455 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:19.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:19.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:19.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0 2024-09-21T12:00:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:19.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:19.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.0' 2024-09-21T12:00:19.680 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:19.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:19.704 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:19.704 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:19.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:19 smithi043 ceph-mon[36454]: pgmap v997: 158 pgs: 1 active+clean+remapped, 42 active+undersized, 22 active+undersized+degraded, 93 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 1.22k op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:19.741 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:19 smithi043 ceph-mon[29710]: pgmap v997: 158 pgs: 1 active+clean+remapped, 42 active+undersized, 22 active+undersized+degraded, 93 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 1.22k op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:19.805 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:19.828 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:19.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:19.829 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:19.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:19.952 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:19.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:19.953 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:20.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:19 smithi175 ceph-mon[32613]: pgmap v997: 158 pgs: 1 active+clean+remapped, 42 active+undersized, 22 active+undersized+degraded, 93 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 1.22k op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:20.054 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:20.067 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T12:00:20.067 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T12:00:20.079 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:20.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:20.080 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:20.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:20.193 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-21T12:00:20.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-21T12:00:20.206 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:20.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:20.207 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:20.308 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:20.333 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:20.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:20.334 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:20.435 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:20.460 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:20.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:20.461 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:20.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:20.585 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:20.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:20.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:20.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-21T12:00:20.638 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:20.687 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:20.712 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:20.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:20.713 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:20.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:20.837 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:20.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:20.838 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:20.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:20.993 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:20.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:20.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:21.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:21.121 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:21.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:21.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:21.145 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:21.223 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:21.247 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:21.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:21.248 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:21.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:21.370 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:21.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:21.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:21.471 INFO:teuthology.orchestra.run.smithi175.stdout:get_pg_num_history pg_num_history pg_num_history(e458 pg_nums {1={20=1,414=11},2={57=3},3={58=32},4={60=32},5={62=32},6={64=32},11={392=16}} deleted_pools ) 2024-09-21T12:00:21.471 INFO:teuthology.orchestra.run.smithi175.stdout:Importing pgid 5.1 2024-09-21T12:00:21.471 INFO:teuthology.orchestra.run.smithi175.stdout:write_pg epoch 432 info 5.1( empty local-lis/les=293/294 n=0 ec=62/62 lis/c=293/293 les/c/f=294/294/0 sis=293) 2024-09-21T12:00:21.471 INFO:teuthology.orchestra.run.smithi175.stdout:Import successful 2024-09-21T12:00:21.472 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:21.496 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:21.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:21.497 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:21.598 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:21.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:21.622 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:21.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:21.684 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is set 2024-09-21T12:00:21.722 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:21.746 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:21.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:21.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:21.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:21.870 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:21.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:21.871 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:21.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:21.989 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:21 smithi175 ceph-mon[32613]: pgmap v998: 158 pgs: 1 active+clean+remapped, 42 active+undersized, 22 active+undersized+degraded, 93 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 8.1 KiB/s rd, 0 B/s wr, 479 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:21.989 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:21 smithi175 ceph-mon[32613]: 1.7 scrub starts 2024-09-21T12:00:21.989 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:21 smithi175 ceph-mon[32613]: 1.7 scrub ok 2024-09-21T12:00:21.989 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:21 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3722395833' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T12:00:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:21 smithi043 ceph-mon[29710]: pgmap v998: 158 pgs: 1 active+clean+remapped, 42 active+undersized, 22 active+undersized+degraded, 93 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 8.1 KiB/s rd, 0 B/s wr, 479 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:21 smithi043 ceph-mon[29710]: 1.7 scrub starts 2024-09-21T12:00:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:21 smithi043 ceph-mon[29710]: 1.7 scrub ok 2024-09-21T12:00:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3722395833' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T12:00:21.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:21 smithi043 ceph-mon[36454]: pgmap v998: 158 pgs: 1 active+clean+remapped, 42 active+undersized, 22 active+undersized+degraded, 93 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 8.1 KiB/s rd, 0 B/s wr, 479 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:21.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:21 smithi043 ceph-mon[36454]: 1.7 scrub starts 2024-09-21T12:00:21.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:21 smithi043 ceph-mon[36454]: 1.7 scrub ok 2024-09-21T12:00:21.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:21 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3722395833' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-21T12:00:21.994 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:21.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:21.995 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:22.068 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-21T12:00:22.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:22.122 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:22.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:22.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:22.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:22.246 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:22.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:22.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:22.260 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:00:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:00:22.348 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:22.374 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:22.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:22.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:22.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:22.498 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:22.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:22.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'unique_pool_4' removed 2024-09-21T12:00:22.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:22.651 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:22.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:22.652 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:22.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:22.775 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:22.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:22.776 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:22.818 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/exp.5.1.4 2024-09-21T12:00:22.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: 6.16 scrub starts 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: 6.16 scrub ok 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: 11.4 scrub starts 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: 11.4 scrub ok 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: 4.7 scrub starts 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: 4.7 scrub ok 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3722395833' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: osdmap e474: 8 total, 7 up, 8 in 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2851762579' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: Health check update: Degraded data redundancy: 83/597 objects degraded (13.903%), 22 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:22.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2851762579' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[29710]: osdmap e475: 8 total, 7 up, 8 in 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: 6.16 scrub starts 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: 6.16 scrub ok 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: 11.4 scrub starts 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: 11.4 scrub ok 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: 4.7 scrub starts 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: 4.7 scrub ok 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3722395833' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: osdmap e474: 8 total, 7 up, 8 in 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2851762579' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: Health check update: Degraded data redundancy: 83/597 objects degraded (13.903%), 22 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:22.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2851762579' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T12:00:22.897 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:22 smithi043 ceph-mon[36454]: osdmap e475: 8 total, 7 up, 8 in 2024-09-21T12:00:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:22.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:22.901 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.2' 2024-09-21T12:00:22.999 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-21T12:00:22.999 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 20, 2024-09-21T12:00:22.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 600, 2024-09-21T12:00:22.999 INFO:teuthology.orchestra.run.smithi043.stdout: "ops": [ 2024-09-21T12:00:22.999 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:22.999 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47521 10.4 10:207ae421:::benchmark_data_smithi043_150678_object47520:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T12:00:22.999 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.835329+0000", 2024-09-21T12:00:22.999 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 429.16077599099998, 2024-09-21T12:00:22.999 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.30159633899999999, 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47521 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835329+0000", 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.000 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835329+0000", 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835330+0000", 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.023e-06 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835511+0000", 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000180896 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.001 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835570+0000", 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.914e-05 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.835577+0000", 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4549999999999998e-06 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.839940+0000", 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0043626489999999997 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.002 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.840064+0000", 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012452599999999999 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.840193+0000", 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012866900000000001 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.843992+0000", 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0037985509999999998 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.845585+0000", 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0015931949999999999 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136797+0000", 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29121228599999999 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136857+0000", 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0177999999999999e-05 2024-09-21T12:00:23.004 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136925+0000", 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7770999999999999e-05 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47525 10.4 10:29bd7892:::benchmark_data_smithi043_150678_object47524:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.841626+0000", 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 429.15447827399998, 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.29545084599999999, 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.005 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47525 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841626+0000", 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.006 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841626+0000", 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841628+0000", 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4610000000000001e-06 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841655+0000", 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.6658000000000001e-05 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.007 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841682+0000", 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7674999999999998e-05 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841691+0000", 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5520000000000005e-06 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.841931+0000", 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023983500000000001 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.008 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842058+0000", 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012696499999999999 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.842192+0000", 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00013453200000000001 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.845735+0000", 2024-09-21T12:00:23.009 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0035430750000000001 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.847002+0000", 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0012665899999999999 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.136972+0000", 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28997065100000002 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137022+0000", 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9577999999999999e-05 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137077+0000", 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5274e-05 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:47558 10.4 10:2e79000f:::benchmark_data_smithi043_150678_object47557:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e325)", 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:13.855768+0000", 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 429.14033697999997, 2024-09-21T12:00:23.011 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28151058899999998, 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 47558 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855768+0000", 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.012 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855768+0000", 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855769+0000", 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.33e-06 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855788+0000", 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9289000000000001e-05 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.013 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855811+0000", 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3139e-05 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855819+0000", 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2819999999999999e-06 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.855885+0000", 2024-09-21T12:00:23.014 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5813000000000002e-05 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.856043+0000", 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015814 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 3,4", 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.856194+0000", 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015090899999999999 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.015 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.859384+0000", 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0031903259999999998 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:13.860384+0000", 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00099984199999999992 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137145+0000", 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27676123800000002 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.016 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137217+0000", 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2330999999999996e-05 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:14.137278+0000", 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.0949999999999998e-05 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.25910.0:85498 10.8 10:1841726b:::benchmark_data_smithi043_150678_object85497:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e336)", 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:53:44.793462+0000", 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 398.20264238599998, 2024-09-21T12:00:23.017 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.28091800300000003, 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.25910", 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/3955125255", 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 85498 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793462+0000", 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793462+0000", 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793464+0000", 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.282e-06 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793488+0000", 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.4277000000000002e-05 2024-09-21T12:00:23.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793498+0000", 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0564000000000001e-05 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793512+0000", 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3858e-05 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793594+0000", 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1499e-05 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.020 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793742+0000", 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00014823599999999999 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 2,7", 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.793877+0000", 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000135227 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.797141+0000", 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0032633900000000001 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.021 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:44.798058+0000", 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00091767599999999997 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074222+0000", 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.27616407700000001 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074310+0000", 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.7566999999999997e-05 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.022 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:53:45.074380+0000", 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.0350000000000002e-05 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169230 11.6 11:66274308:::benchmark_data_smithi043_178089_object47995:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.642927+0000", 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.353177729999999, 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0574667170000001, 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-21T12:00:23.023 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169230 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642927+0000", 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642927+0000", 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.024 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:23.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: 6.16 scrub starts 2024-09-21T12:00:23.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: 6.16 scrub ok 2024-09-21T12:00:23.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: 11.4 scrub starts 2024-09-21T12:00:23.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: 11.4 scrub ok 2024-09-21T12:00:23.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: 4.7 scrub starts 2024-09-21T12:00:23.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: 4.7 scrub ok 2024-09-21T12:00:23.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-21T12:00:23.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3722395833' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-21T12:00:23.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: osdmap e474: 8 total, 7 up, 8 in 2024-09-21T12:00:23.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2851762579' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T12:00:23.027 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: Health check update: Degraded data redundancy: 83/597 objects degraded (13.903%), 22 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:23.027 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2851762579' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T12:00:23.027 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:22 smithi175 ceph-mon[32613]: osdmap e475: 8 total, 7 up, 8 in 2024-09-21T12:00:23.027 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.027 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642928+0000", 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4699999999999999e-06 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642933+0000", 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.8069999999999999e-06 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642948+0000", 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4883e-05 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642956+0000", 2024-09-21T12:00:23.028 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.0320000000000003e-06 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.643117+0000", 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016093300000000001 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.643126+0000", 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0659999999999993e-06 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.700321+0000", 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.057195058 2024-09-21T12:00:23.029 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.700394+0000", 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2467999999999998e-05 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169232 11.6 11:6129255c:::benchmark_data_smithi043_178089_object47997:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.642971+0000", 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.353134046999998, 2024-09-21T12:00:23.030 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0580335839999999, 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "reached pg", 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169232 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642971+0000", 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.031 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642971+0000", 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642971+0000", 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0699999999999997e-07 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642973+0000", 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1169999999999998e-06 2024-09-21T12:00:23.032 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642979+0000", 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.327e-06 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.642983+0000", 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7960000000000003e-06 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.643151+0000", 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016791000000000001 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.033 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.643157+0000", 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.942e-06 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.700929+0000", 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.057771743 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.701004+0000", 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.5241999999999996e-05 2024-09-21T12:00:23.034 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169240 11.6 11:6130f782:::benchmark_data_smithi043_178089_object48005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.644440+0000", 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.351665025000003, 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0627846919999999, 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.035 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169240 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.644440+0000", 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.644440+0000", 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.644441+0000", 2024-09-21T12:00:23.036 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.395e-06 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.644446+0000", 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1499999999999998e-06 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.645101+0000", 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00065460300000000002 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.645112+0000", 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0869e-05 2024-09-21T12:00:23.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.645305+0000", 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000193055 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.645328+0000", 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3748999999999999e-05 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.701033+0000", 2024-09-21T12:00:23.038 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.055704701 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.701144+0000", 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011051400000000001 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.701251+0000", 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107648 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.706993+0000", 2024-09-21T12:00:23.039 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005742034 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.707084+0000", 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.1019000000000004e-05 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.707128+0000", 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3906999999999998e-05 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.707180+0000", 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.2061000000000002e-05 2024-09-21T12:00:23.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.707224+0000", 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3986999999999997e-05 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169263 11.6 11:64d2afa2:::benchmark_data_smithi043_178089_object48028:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.663695+0000", 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.332409398999999, 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0494875340000001, 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.041 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169263 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.663695+0000", 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.042 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.663695+0000", 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.663697+0000", 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.6479999999999999e-06 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.663702+0000", 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.412e-06 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.663717+0000", 2024-09-21T12:00:23.043 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.502e-05 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.663726+0000", 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.3690000000000007e-06 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.663927+0000", 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00020144999999999999 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.663951+0000", 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3768e-05 2024-09-21T12:00:23.044 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.704338+0000", 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.040386638 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.704445+0000", 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010785099999999999 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.704605+0000", 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015965900000000001 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.045 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.712982+0000", 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083773709999999998 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.713033+0000", 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0235999999999998e-05 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.713083+0000", 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.0457999999999997e-05 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.046 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.713139+0000", 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5455999999999999e-05 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.713183+0000", 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.4197999999999999e-05 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169280 11.6 11:64002507:::benchmark_data_smithi043_178089_object48045:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.681238+0000", 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.314866168999998, 2024-09-21T12:00:23.047 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.039875726, 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169280 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.681238+0000", 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.048 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.681238+0000", 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.681241+0000", 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1780000000000002e-06 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.681247+0000", 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8490000000000002e-06 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.049 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.681278+0000", 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1501000000000002e-05 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.681287+0000", 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.541e-06 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.050 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.681355+0000", 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.8384000000000001e-05 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.681365+0000", 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0121e-05 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.707277+0000", 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0259118039999999 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.051 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.707410+0000", 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000132741 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.707514+0000", 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010482800000000001 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.713226+0000", 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.005711603 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.718395+0000", 2024-09-21T12:00:23.052 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0051692409999999998 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.720943+0000", 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0025478250000000001 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.720997+0000", 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3449000000000003e-05 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.721114+0000", 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000117661 2024-09-21T12:00:23.053 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169287 11.6 11:6bd454e6:::benchmark_data_smithi043_178089_object48052:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.686456+0000", 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.309648198000005, 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0431420840000001, 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.054 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169287 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.686456+0000", 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.686456+0000", 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.055 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.686458+0000", 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8789999999999999e-06 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.686464+0000", 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3120000000000001e-06 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.686478+0000", 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4698000000000001e-05 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.686486+0000", 2024-09-21T12:00:23.056 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7600000000000002e-06 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.686549+0000", 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2799000000000006e-05 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.686587+0000", 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8167000000000002e-05 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.710382+0000", 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0237945530000001 2024-09-21T12:00:23.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.710492+0000", 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000110229 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.710607+0000", 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000115369 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.058 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.720893+0000", 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010285503999999999 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729263+0000", 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083700739999999999 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729505+0000", 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00024184 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729558+0000", 2024-09-21T12:00:23.059 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.3325000000000002e-05 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729599+0000", 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0574999999999997e-05 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169315 11.6 11:6328544f:::benchmark_data_smithi043_178089_object48080:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.715826+0000", 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.280278526000004, 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.014004232, 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.060 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169315 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.715826+0000", 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.715826+0000", 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.715827+0000", 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.336e-06 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.715831+0000", 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.799e-06 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.715844+0000", 2024-09-21T12:00:23.062 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.2354000000000001e-05 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.715850+0000", 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.5660000000000003e-06 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.715925+0000", 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.4595e-05 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.715933+0000", 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.2560000000000002e-06 2024-09-21T12:00:23.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.713277+0000", 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99734421900000003 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.713391+0000", 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000114199 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.713514+0000", 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00012280799999999999 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.064 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.720898+0000", 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.007384084 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729330+0000", 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0084318280000000006 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729752+0000", 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00042206200000000001 2024-09-21T12:00:23.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729795+0000", 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2605000000000003e-05 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729830+0000", 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.5521000000000001e-05 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.066 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169320 11.6 11:604cd133:::benchmark_data_smithi043_178089_object48085:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.721018+0000", 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.27508675, 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.008927216, 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169320 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.067 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.721018+0000", 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.721018+0000", 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.721020+0000", 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.795e-06 2024-09-21T12:00:23.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.721024+0000", 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5299999999999998e-06 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.721039+0000", 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4782e-05 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.721047+0000", 2024-09-21T12:00:23.069 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.7689999999999996e-06 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.721229+0000", 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00018234100000000001 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.721274+0000", 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5111999999999998e-05 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.715805+0000", 2024-09-21T12:00:23.070 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99453095199999997 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.715957+0000", 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00015169900000000001 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.716077+0000", 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00011977999999999999 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.721126+0000", 2024-09-21T12:00:23.071 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0050493229999999997 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729458+0000", 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0083317370000000005 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729870+0000", 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00041275399999999999 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729914+0000", 2024-09-21T12:00:23.072 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3510000000000002e-05 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729945+0000", 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1131999999999999e-05 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169334 11.6 11:67057a83:::benchmark_data_smithi043_178089_object48099:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.734526+0000", 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.261578631999996, 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.99552708899999998, 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.073 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169334 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.734526+0000", 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.734526+0000", 2024-09-21T12:00:23.074 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.734528+0000", 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5880000000000001e-06 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.734532+0000", 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6929999999999998e-06 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.734543+0000", 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0332e-05 2024-09-21T12:00:23.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.734549+0000", 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.4280000000000001e-06 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.734595+0000", 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.6319999999999997e-05 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.734601+0000", 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.4670000000000003e-06 2024-09-21T12:00:23.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.718423+0000", 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98382248500000002 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.718521+0000", 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.8021000000000002e-05 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.718627+0000", 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010569300000000001 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.077 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.078 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729286+0000", 2024-09-21T12:00:23.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.010658986000000001 2024-09-21T12:00:23.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.079 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.079 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729661+0000", 2024-09-21T12:00:23.079 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00037515900000000002 2024-09-21T12:00:23.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.079 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729986+0000", 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00032480600000000002 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.730024+0000", 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.7936999999999999e-05 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.730053+0000", 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.9173999999999999e-05 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169335 11.6 11:6adba0cf:::benchmark_data_smithi043_178089_object48100:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.735046+0000", 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.261058254999995, 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0006641940000001, 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169335 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.081 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.735046+0000", 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.735046+0000", 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.735047+0000", 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.9900000000000009e-07 2024-09-21T12:00:23.082 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.735052+0000", 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.831e-06 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.735219+0000", 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000166737 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.735229+0000", 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.046e-05 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.083 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.735292+0000", 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.2872000000000004e-05 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.735301+0000", 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.6649999999999992e-06 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.721153+0000", 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.98585232599999995 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.084 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.721251+0000", 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.7868000000000006e-05 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.721358+0000", 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000107228 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729341+0000", 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079827100000000005 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.085 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729701+0000", 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00036029299999999997 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.735616+0000", 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0059144710000000001 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.735667+0000", 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.1480999999999997e-05 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.086 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.735711+0000", 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3253000000000001e-05 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169358 11.6 11:631c8045:::benchmark_data_smithi043_178089_object48123:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.765074+0000", 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.231030719000003, 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.97074326300000002, 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.087 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169358 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.765074+0000", 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.765074+0000", 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.088 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.765075+0000", 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4759999999999999e-06 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.765086+0000", 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0604999999999999e-05 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.765102+0000", 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5951999999999999e-05 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.089 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.765109+0000", 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2830000000000004e-06 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.765279+0000", 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00017006399999999999 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.765302+0000", 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.3116999999999999e-05 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.090 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.726374+0000", 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.96107192600000002 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.726483+0000", 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010877799999999999 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.726589+0000", 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00010537800000000001 2024-09-21T12:00:23.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.729952+0000", 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0033639379999999999 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.735579+0000", 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0056264899999999996 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.735748+0000", 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016918599999999999 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.092 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.735787+0000", 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.8481e-05 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.735817+0000", 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0589000000000001e-05 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169377 11.6 11:626caae1:::benchmark_data_smithi043_178089_object48142:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.093 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.788380+0000", 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.207724749999997, 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.96130386800000001, 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169377 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.788380+0000", 2024-09-21T12:00:23.094 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.788380+0000", 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.788382+0000", 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.9309999999999998e-06 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.788389+0000", 2024-09-21T12:00:23.095 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2250000000000003e-06 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.788410+0000", 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.1297999999999998e-05 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.788495+0000", 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.4645e-05 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.788687+0000", 2024-09-21T12:00:23.096 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00019210200000000001 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.788723+0000", 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.6072000000000003e-05 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.730079+0000", 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.94135585099999997 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.730172+0000", 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.3282000000000006e-05 2024-09-21T12:00:23.097 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.730273+0000", 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100989 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.741510+0000", 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.011236276999999999 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.744317+0000", 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0028077140000000002 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749598+0000", 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0052807280000000002 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749642+0000", 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.3927999999999997e-05 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.099 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749684+0000", 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1826000000000002e-05 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169385 11.6 11:6c4ed501:::benchmark_data_smithi043_178089_object48150:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.798275+0000", 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.197829548000001, 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.95151172699999997, 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.100 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169385 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.798275+0000", 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.798275+0000", 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.101 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.798277+0000", 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.5209999999999999e-06 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.798281+0000", 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7690000000000004e-06 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.798296+0000", 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.4303e-05 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.102 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.798302+0000", 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.7399999999999998e-06 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.798472+0000", 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00016985500000000001 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.798495+0000", 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.2447999999999999e-05 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.732779+0000", 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.93428406900000005 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.732875+0000", 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6558999999999995e-05 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.732972+0000", 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.6367999999999996e-05 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.741515+0000", 2024-09-21T12:00:23.104 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0085431540000000007 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749486+0000", 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0079707249999999997 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749716+0000", 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00023007799999999999 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749756+0000", 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0110000000000001e-05 2024-09-21T12:00:23.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749787+0000", 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1028000000000002e-05 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169424 11.6 11:67e2960a:::benchmark_data_smithi043_178089_object48189:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.901444+0000", 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.094660774000005, 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.84845068999999995, 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.106 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169424 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.901444+0000", 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.107 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.901444+0000", 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.901446+0000", 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.8020000000000001e-06 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.901451+0000", 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.023e-06 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.108 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.901462+0000", 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.0808999999999999e-05 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.901469+0000", 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.2980000000000003e-06 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.901528+0000", 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.9168e-05 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.901536+0000", 2024-09-21T12:00:23.109 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1389999999999995e-06 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.735844+0000", 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.83430745699999997 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.735952+0000", 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108578 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.110 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.736053+0000", 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000100793 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749498+0000", 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.013445332000000001 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749529+0000", 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.0766999999999999e-05 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.111 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749818+0000", 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.00028851499999999998 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749863+0000", 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.5164000000000003e-05 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749895+0000", 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 3.1844999999999997e-05 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.112 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169434 11.6 11:6650bf37:::benchmark_data_smithi043_178089_object48199:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:01.924762+0000", 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 81.071342676, 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.82787440000000001, 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169434 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.113 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.924762+0000", 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.924762+0000", 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.924763+0000", 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.203e-06 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.924767+0000", 2024-09-21T12:00:23.114 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.1479999999999999e-06 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.924779+0000", 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.1222e-05 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.924786+0000", 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.763e-06 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.924863+0000", 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 7.6624000000000004e-05 2024-09-21T12:00:23.115 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:01.924872+0000", 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.8200000000000003e-06 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.738914+0000", 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.81404189500000002 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.738999+0000", 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.5792999999999997e-05 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.116 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.739090+0000", 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 9.0488000000000005e-05 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749539+0000", 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.01044866 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.749566+0000", 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 2.7521999999999998e-05 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.117 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.752544+0000", 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0029775169999999998 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.752594+0000", 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.9913999999999997e-05 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.752636+0000", 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.2830999999999997e-05 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.118 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "description": "osd_op(client.26339.0:169482 11.6 11:6938e113:::benchmark_data_smithi043_178089_object48247:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e444)", 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "initiated_at": "2024-09-21T11:59:02.187739+0000", 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "age": 80.808365401000003, 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.57436611199999998, 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "continuous": false, 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "type_data": { 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "client_info": { 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "client": "client.26339", 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "client_addr": "172.21.15.43:0/4015095582", 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: "tid": 169482 2024-09-21T12:00:23.119 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: "events": [ 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "initiated", 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.187739+0000", 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "header_read", 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.187739+0000", 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "throttled", 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.187741+0000", 2024-09-21T12:00:23.120 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.463e-06 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "all_read", 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.187746+0000", 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.5319999999999997e-06 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "dispatched", 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.187759+0000", 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 1.3118999999999999e-05 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "queued_for_pg", 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.187768+0000", 2024-09-21T12:00:23.121 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 8.1610000000000003e-06 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.187831+0000", 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 6.3484000000000004e-05 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for peered", 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.187837+0000", 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 5.8669999999999999e-06 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "reached_pg", 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.744357+0000", 2024-09-21T12:00:23.122 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.55652043200000001 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "started", 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.744466+0000", 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108968 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "waiting for subops from 1,2", 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.744575+0000", 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.000108832 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "op_commit", 2024-09-21T12:00:23.123 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.752647+0000", 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0080716610000000008 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.758090+0000", 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0054436099999999998 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "sub_op_commit_rec", 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.762017+0000", 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 0.0039268300000000001 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.124 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "commit_sent", 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.762065+0000", 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.7922e-05 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: "event": "done", 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time": "2024-09-21T11:59:02.762105+0000", 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: "duration": 4.0231000000000002e-05 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-21T12:00:23.125 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:23.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:23.126 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:23.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:23.227 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:23.249 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:23.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:23.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:23.350 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:23.374 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:23.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:23.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:23.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7 2024-09-21T12:00:23.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:23.498 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:23.498 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.7' 2024-09-21T12:00:23.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6 2024-09-21T12:00:23.620 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:23.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:23.621 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.6' 2024-09-21T12:00:23.722 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:23.743 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:23.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:23.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:23.845 DEBUG:teuthology.orchestra.run.smithi175:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5 2024-09-21T12:00:23.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:23.867 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: docker: command not found 2024-09-21T12:00:23.867 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi175 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.5' 2024-09-21T12:00:23.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1 2024-09-21T12:00:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:23 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:23.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:23 smithi043 ceph-mon[29710]: pgmap v1000: 158 pgs: 1 active+clean+remapped, 42 active+undersized, 22 active+undersized+degraded, 93 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 450 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:23 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:23.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:23 smithi043 ceph-mon[36454]: pgmap v1000: 158 pgs: 1 active+clean+remapped, 42 active+undersized, 22 active+undersized+degraded, 93 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 450 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:23.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:23.992 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:23.992 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.1' 2024-09-21T12:00:24.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:23 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:24.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:23 smithi175 ceph-mon[32613]: pgmap v1000: 158 pgs: 1 active+clean+remapped, 42 active+undersized, 22 active+undersized+degraded, 93 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 450 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:24.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3 2024-09-21T12:00:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: docker: command not found 2024-09-21T12:00:24.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:00:24.116 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi043 with status 1: 'sudo docker kill -s 1 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd.3' 2024-09-21T12:00:24.192 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-21T12:00:24.208 INFO:tasks.thrashosds:joining thrashosds 2024-09-21T12:00:24.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset noscrub 2024-09-21T12:00:24.717 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-21T12:00:25.169 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:25.200 INFO:teuthology.orchestra.run.smithi043.stdout:osd.0: {} 2024-09-21T12:00:25.200 INFO:teuthology.orchestra.run.smithi043.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-21T12:00:25.219 INFO:teuthology.orchestra.run.smithi043.stdout:osd.1: {} 2024-09-21T12:00:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-21T12:00:25.236 INFO:teuthology.orchestra.run.smithi043.stdout:osd.2: {} 2024-09-21T12:00:25.236 INFO:teuthology.orchestra.run.smithi043.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-21T12:00:25.252 INFO:teuthology.orchestra.run.smithi043.stdout:osd.3: {} 2024-09-21T12:00:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-21T12:00:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-21T12:00:25.270 INFO:teuthology.orchestra.run.smithi043.stdout:osd.5: {} 2024-09-21T12:00:25.270 INFO:teuthology.orchestra.run.smithi043.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-21T12:00:25.288 INFO:teuthology.orchestra.run.smithi043.stdout:osd.6: {} 2024-09-21T12:00:25.288 INFO:teuthology.orchestra.run.smithi043.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-21T12:00:25.305 INFO:teuthology.orchestra.run.smithi043.stdout:osd.7: {} 2024-09-21T12:00:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-21T12:00:25.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:00:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:00:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:25 smithi043 ceph-mon[29710]: pgmap v1002: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:25.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:25 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1192361395' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:25.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:25 smithi043 ceph-mon[36454]: pgmap v1002: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:25.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:25 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1192361395' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:26.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:25 smithi175 ceph-mon[32613]: pgmap v1002: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:26.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:25 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1192361395' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-21T12:00:26.140 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-21T12:00:26.140 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-21T12:00:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:noscrub is unset 2024-09-21T12:00:26.891 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:26.892 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:26.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:26 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1485158987' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T12:00:26.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:26 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1485158987' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T12:00:27.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:26 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1485158987' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-21T12:00:27.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-21T12:00:27.850 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-21T12:00:27.850 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-21T12:00:27.851 INFO:tasks.cephadm.osd.4:Restarting osd.4 (starting--it wasn't running)... 2024-09-21T12:00:27.851 DEBUG:teuthology.orchestra.run.smithi175:osd.4> sudo journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.4.service 2024-09-21T12:00:27.854 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl start ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.4 2024-09-21T12:00:27.943 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:27 smithi175 ceph-mon[32613]: pgmap v1003: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:27.943 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:27 smithi175 ceph-mon[32613]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-21T12:00:27.943 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:27 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1485158987' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T12:00:27.943 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:27 smithi175 ceph-mon[32613]: osdmap e476: 8 total, 7 up, 8 in 2024-09-21T12:00:27.943 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:27 smithi175 ceph-mon[32613]: 4.8 scrub starts 2024-09-21T12:00:27.943 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:27 smithi175 ceph-mon[32613]: 4.8 scrub ok 2024-09-21T12:00:27.943 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:27 smithi175 ceph-mon[32613]: osdmap e477: 8 total, 7 up, 8 in 2024-09-21T12:00:27.943 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:27 smithi175 systemd[1]: Starting Ceph osd.4 for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:00:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[36454]: pgmap v1003: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[36454]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-21T12:00:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1485158987' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T12:00:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[36454]: osdmap e476: 8 total, 7 up, 8 in 2024-09-21T12:00:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[36454]: 4.8 scrub starts 2024-09-21T12:00:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[36454]: 4.8 scrub ok 2024-09-21T12:00:27.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[36454]: osdmap e477: 8 total, 7 up, 8 in 2024-09-21T12:00:27.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[29710]: pgmap v1003: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[29710]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-21T12:00:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1485158987' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-21T12:00:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[29710]: osdmap e476: 8 total, 7 up, 8 in 2024-09-21T12:00:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[29710]: 4.8 scrub starts 2024-09-21T12:00:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[29710]: 4.8 scrub ok 2024-09-21T12:00:27.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:27 smithi043 ceph-mon[29710]: osdmap e477: 8 total, 7 up, 8 in 2024-09-21T12:00:28.443 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:28 smithi175 podman[152328]: 2024-09-21 12:00:28.078162449 +0000 UTC m=+0.016532369 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:00:28.444 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:28 smithi175 podman[152328]: 2024-09-21 12:00:28.206785577 +0000 UTC m=+0.145155468 container create 391df97c5c90802b75c24de983e88cb0e2e42835a75485984467a07abeff3d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:00:28.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-21T12:00:28.684 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:28.698 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:28 smithi175 podman[152328]: 2024-09-21 12:00:28.442638632 +0000 UTC m=+0.381008524 container init 391df97c5c90802b75c24de983e88cb0e2e42835a75485984467a07abeff3d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T12:00:28.698 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:28 smithi175 podman[152328]: 2024-09-21 12:00:28.446109601 +0000 UTC m=+0.384479502 container start 391df97c5c90802b75c24de983e88cb0e2e42835a75485984467a07abeff3d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:00:28.698 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:28 smithi175 podman[152328]: 2024-09-21 12:00:28.559744464 +0000 UTC m=+0.498114359 container attach 391df97c5c90802b75c24de983e88cb0e2e42835a75485984467a07abeff3d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T12:00:28.698 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:28 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T12:00:28.698 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:28 smithi175 bash[152328]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T12:00:28.698 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:28 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T12:00:28.698 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:28 smithi175 bash[152328]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T12:00:28.809 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:28 smithi043 ceph-mon[29710]: 5.1b scrub starts 2024-09-21T12:00:28.810 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:28 smithi043 ceph-mon[29710]: 5.1b scrub ok 2024-09-21T12:00:28.810 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:28 smithi043 ceph-mon[36454]: 5.1b scrub starts 2024-09-21T12:00:28.810 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:28 smithi043 ceph-mon[36454]: 5.1b scrub ok 2024-09-21T12:00:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:28 smithi175 ceph-mon[32613]: 5.1b scrub starts 2024-09-21T12:00:29.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:28 smithi175 ceph-mon[32613]: 5.1b scrub ok 2024-09-21T12:00:29.324 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:29.709 INFO:teuthology.orchestra.run.smithi043.stderr:nodeep-scrub is unset 2024-09-21T12:00:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[29710]: 6.10 scrub starts 2024-09-21T12:00:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[29710]: 6.10 scrub ok 2024-09-21T12:00:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[29710]: pgmap v1006: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[29710]: 5.1e scrub starts 2024-09-21T12:00:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[29710]: 5.1e scrub ok 2024-09-21T12:00:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4284454974' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T12:00:29.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T12:00:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[36454]: 6.10 scrub starts 2024-09-21T12:00:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[36454]: 6.10 scrub ok 2024-09-21T12:00:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[36454]: pgmap v1006: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[36454]: 5.1e scrub starts 2024-09-21T12:00:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[36454]: 5.1e scrub ok 2024-09-21T12:00:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4284454974' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T12:00:29.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:29 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T12:00:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:29 smithi175 ceph-mon[32613]: 6.10 scrub starts 2024-09-21T12:00:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:29 smithi175 ceph-mon[32613]: 6.10 scrub ok 2024-09-21T12:00:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:29 smithi175 ceph-mon[32613]: pgmap v1006: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 682 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:29 smithi175 ceph-mon[32613]: 5.1e scrub starts 2024-09-21T12:00:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:29 smithi175 ceph-mon[32613]: 5.1e scrub ok 2024-09-21T12:00:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:29 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4284454974' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T12:00:30.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:29 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-21T12:00:30.187 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:30.187 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:30.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-21T12:00:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:30 smithi043 ceph-mon[29710]: 3.0 scrub starts 2024-09-21T12:00:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:30 smithi043 ceph-mon[29710]: 3.0 scrub ok 2024-09-21T12:00:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:30 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T12:00:30.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:30 smithi043 ceph-mon[29710]: osdmap e478: 8 total, 7 up, 8 in 2024-09-21T12:00:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:30 smithi043 ceph-mon[36454]: 3.0 scrub starts 2024-09-21T12:00:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:30 smithi043 ceph-mon[36454]: 3.0 scrub ok 2024-09-21T12:00:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:30 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T12:00:30.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:30 smithi043 ceph-mon[36454]: osdmap e478: 8 total, 7 up, 8 in 2024-09-21T12:00:31.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:30 smithi175 ceph-mon[32613]: 3.0 scrub starts 2024-09-21T12:00:31.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:30 smithi175 ceph-mon[32613]: 3.0 scrub ok 2024-09-21T12:00:31.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:30 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-21T12:00:31.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:30 smithi175 ceph-mon[32613]: osdmap e478: 8 total, 7 up, 8 in 2024-09-21T12:00:31.635 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[36454]: pgmap v1008: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[36454]: 6.1 scrub starts 2024-09-21T12:00:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[36454]: 6.1 scrub ok 2024-09-21T12:00:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:00:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[36454]: 4.10 scrub starts 2024-09-21T12:00:31.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[36454]: 4.10 scrub ok 2024-09-21T12:00:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[29710]: pgmap v1008: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[29710]: 6.1 scrub starts 2024-09-21T12:00:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[29710]: 6.1 scrub ok 2024-09-21T12:00:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:00:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[29710]: 4.10 scrub starts 2024-09-21T12:00:31.742 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:31 smithi043 ceph-mon[29710]: 4.10 scrub ok 2024-09-21T12:00:31.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T12:00:31.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T12:00:31.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 bash[152328]: --> Failed to activate via raw: did not find any matching OSD to activate 2024-09-21T12:00:31.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 bash[152328]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T12:00:31.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T12:00:31.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 bash[152328]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-21T12:00:31.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T12:00:31.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 bash[152328]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T12:00:31.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-09-21T12:00:31.906 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 bash[152328]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-09-21T12:00:31.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-mon[32613]: pgmap v1008: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:31.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-mon[32613]: 6.1 scrub starts 2024-09-21T12:00:31.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-mon[32613]: 6.1 scrub ok 2024-09-21T12:00:31.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:00:31.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-mon[32613]: 4.10 scrub starts 2024-09-21T12:00:31.907 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-mon[32613]: 4.10 scrub ok 2024-09-21T12:00:32.174 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:32.174 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:32.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:00:32 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:00:32.248 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4/block 2024-09-21T12:00:32.248 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 bash[152328]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4/block 2024-09-21T12:00:32.249 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-09-21T12:00:32.249 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 bash[152328]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-09-21T12:00:32.249 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-21T12:00:32.249 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 bash[152328]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-21T12:00:32.249 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T12:00:32.249 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 bash[152328]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T12:00:32.249 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate[152343]: --> ceph-volume lvm activate successful for osd ID: 4 2024-09-21T12:00:32.249 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 bash[152328]: --> ceph-volume lvm activate successful for osd ID: 4 2024-09-21T12:00:32.249 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:31 smithi175 podman[152328]: 2024-09-21 12:00:31.933759279 +0000 UTC m=+3.872129189 container died 391df97c5c90802b75c24de983e88cb0e2e42835a75485984467a07abeff3d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T12:00:32.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-21T12:00:32.984 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:32 smithi175 ceph-mon[32613]: 2.1 scrub starts 2024-09-21T12:00:32.984 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:00:32.984 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:00:32.984 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:32 smithi175 ceph-mon[32613]: 2.1 scrub ok 2024-09-21T12:00:32.984 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:32 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:32.984 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:32.984 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:32 smithi175 ceph-mon[32613]: osdmap e479: 8 total, 7 up, 8 in 2024-09-21T12:00:32.985 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:32 smithi175 podman[152624]: 2024-09-21 12:00:32.647216407 +0000 UTC m=+0.702861356 container remove 391df97c5c90802b75c24de983e88cb0e2e42835a75485984467a07abeff3d01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-activate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T12:00:32.985 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:32 smithi175 podman[152681]: 2024-09-21 12:00:32.758282827 +0000 UTC m=+0.015889140 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:00:32.985 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:32 smithi175 podman[152681]: 2024-09-21 12:00:32.881497487 +0000 UTC m=+0.139103786 container create 7420c7dbcf221bd57281830ddbab98762859f033d1494c379c9285404067aff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T12:00:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[29710]: 2.1 scrub starts 2024-09-21T12:00:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:00:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:00:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[29710]: 2.1 scrub ok 2024-09-21T12:00:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:32.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[29710]: osdmap e479: 8 total, 7 up, 8 in 2024-09-21T12:00:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[36454]: 2.1 scrub starts 2024-09-21T12:00:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:00:32.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:00:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[36454]: 2.1 scrub ok 2024-09-21T12:00:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:32.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:32 smithi043 ceph-mon[36454]: osdmap e479: 8 total, 7 up, 8 in 2024-09-21T12:00:33.070 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:33.261 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:33 smithi175 podman[152681]: 2024-09-21 12:00:33.108853348 +0000 UTC m=+0.366459652 container init 7420c7dbcf221bd57281830ddbab98762859f033d1494c379c9285404067aff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:00:33.261 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:33 smithi175 podman[152681]: 2024-09-21 12:00:33.111996906 +0000 UTC m=+0.369603209 container start 7420c7dbcf221bd57281830ddbab98762859f033d1494c379c9285404067aff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:00:33.261 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:33 smithi175 ceph-osd[152700]: -- 172.21.15.175:0/1198545024 <== mon.1 v2:172.21.15.175:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e94249da40 con 0x55e94247bc00 2024-09-21T12:00:33.400 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T12:00:33.514 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:33.514 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:33.653 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:33.719 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:33 smithi175 bash[152681]: 7420c7dbcf221bd57281830ddbab98762859f033d1494c379c9285404067aff4 2024-09-21T12:00:33.719 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:33 smithi175 systemd[1]: Started Ceph osd.4 for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T12:00:33.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:33 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:33.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:33 smithi043 ceph-mon[29710]: pgmap v1009: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:33.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:33 smithi043 ceph-mon[29710]: 5.19 scrub starts 2024-09-21T12:00:33.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:33 smithi043 ceph-mon[29710]: 5.19 scrub ok 2024-09-21T12:00:33.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:33 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:33.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:33 smithi043 ceph-mon[36454]: pgmap v1009: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:33.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:33 smithi043 ceph-mon[36454]: 5.19 scrub starts 2024-09-21T12:00:33.929 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:33 smithi043 ceph-mon[36454]: 5.19 scrub ok 2024-09-21T12:00:33.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:33 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:33.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:33 smithi175 ceph-mon[32613]: pgmap v1009: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:33.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:33 smithi175 ceph-mon[32613]: 5.19 scrub starts 2024-09-21T12:00:33.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:33 smithi175 ceph-mon[32613]: 5.19 scrub ok 2024-09-21T12:00:34.093 INFO:teuthology.orchestra.run.smithi175.stderr:no valid command found; 10 closest matches: 2024-09-21T12:00:34.093 INFO:teuthology.orchestra.run.smithi175.stderr:0 2024-09-21T12:00:34.093 INFO:teuthology.orchestra.run.smithi175.stderr:1 2024-09-21T12:00:34.093 INFO:teuthology.orchestra.run.smithi175.stderr:2 2024-09-21T12:00:34.093 INFO:teuthology.orchestra.run.smithi175.stderr:abort 2024-09-21T12:00:34.093 INFO:teuthology.orchestra.run.smithi175.stderr:assert 2024-09-21T12:00:34.094 INFO:teuthology.orchestra.run.smithi175.stderr:config diff 2024-09-21T12:00:34.094 INFO:teuthology.orchestra.run.smithi175.stderr:config diff get 2024-09-21T12:00:34.094 INFO:teuthology.orchestra.run.smithi175.stderr:config get 2024-09-21T12:00:34.094 INFO:teuthology.orchestra.run.smithi175.stderr:config help [] 2024-09-21T12:00:34.094 INFO:teuthology.orchestra.run.smithi175.stderr:config set ... 2024-09-21T12:00:34.094 INFO:teuthology.orchestra.run.smithi175.stderr:admin_socket: invalid command 2024-09-21T12:00:34.189 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-21T12:00:34.449 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:34.761 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:34 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:00:34.496+0000 7faee555e640 -1 Falling back to public interface 2024-09-21T12:00:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:34 smithi175 ceph-mon[32613]: 3.12 scrub starts 2024-09-21T12:00:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:34 smithi175 ceph-mon[32613]: 3.12 scrub ok 2024-09-21T12:00:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:34 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T12:00:34.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:34 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T12:00:34.848 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T12:00:34.848 INFO:tasks.cephadm.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-21T12:00:34.894 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:34 smithi043 ceph-mon[29710]: 3.12 scrub starts 2024-09-21T12:00:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:34 smithi043 ceph-mon[29710]: 3.12 scrub ok 2024-09-21T12:00:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:34 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T12:00:34.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:34 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T12:00:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:34 smithi043 ceph-mon[36454]: 3.12 scrub starts 2024-09-21T12:00:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:34 smithi043 ceph-mon[36454]: 3.12 scrub ok 2024-09-21T12:00:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:34 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T12:00:34.895 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:34 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T12:00:35.002 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:35.003 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:35.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:00:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:00:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:35.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-21T12:00:35.924 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:35.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:35 smithi043 ceph-mon[36454]: pgmap v1011: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 767 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:35.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:35 smithi043 ceph-mon[29710]: pgmap v1011: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 767 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:36.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:35 smithi175 ceph-mon[32613]: pgmap v1011: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 767 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:36.397 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:36.397 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:36 smithi043 ceph-mon[36454]: 3.d scrub starts 2024-09-21T12:00:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:36 smithi043 ceph-mon[36454]: 3.d scrub ok 2024-09-21T12:00:36.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:36 smithi043 ceph-mon[29710]: 3.d scrub starts 2024-09-21T12:00:36.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:36 smithi043 ceph-mon[29710]: 3.d scrub ok 2024-09-21T12:00:37.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:36 smithi175 ceph-mon[32613]: 3.d scrub starts 2024-09-21T12:00:37.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:36 smithi175 ceph-mon[32613]: 3.d scrub ok 2024-09-21T12:00:37.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-21T12:00:37.340 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:37.840 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:37.840 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:37.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:37 smithi043 ceph-mon[36454]: pgmap v1012: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:37.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:37 smithi043 ceph-mon[36454]: 6.1c scrub starts 2024-09-21T12:00:37.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:37 smithi043 ceph-mon[36454]: 6.1c scrub ok 2024-09-21T12:00:37.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:37 smithi043 ceph-mon[29710]: pgmap v1012: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:37 smithi043 ceph-mon[29710]: 6.1c scrub starts 2024-09-21T12:00:37.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:37 smithi043 ceph-mon[29710]: 6.1c scrub ok 2024-09-21T12:00:38.010 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:37 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:00:37.834+0000 7faee555e640 -1 osd.4 458 log_to_monitors true 2024-09-21T12:00:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:37 smithi175 ceph-mon[32613]: pgmap v1012: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:37 smithi175 ceph-mon[32613]: 6.1c scrub starts 2024-09-21T12:00:38.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:37 smithi175 ceph-mon[32613]: 6.1c scrub ok 2024-09-21T12:00:38.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-21T12:00:38.909 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:38.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:38 smithi043 ceph-mon[36454]: from='osd.4 v2:172.21.15.175:6800/1102868557' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T12:00:38.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:38 smithi043 ceph-mon[36454]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T12:00:38.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:38 smithi043 ceph-mon[29710]: from='osd.4 v2:172.21.15.175:6800/1102868557' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T12:00:38.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:38 smithi043 ceph-mon[29710]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T12:00:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:38 smithi175 ceph-mon[32613]: from='osd.4 v2:172.21.15.175:6800/1102868557' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T12:00:39.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:38 smithi175 ceph-mon[32613]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T12:00:39.011 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:00:38 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:00:38.764+0000 7faedd308640 -1 osd.4 458 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T12:00:39.361 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:39.361 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:39.850 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T12:00:40.010 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:39 smithi175 ceph-mon[32613]: pgmap v1013: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:39 smithi175 ceph-mon[32613]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T12:00:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:39 smithi175 ceph-mon[32613]: osdmap e480: 8 total, 7 up, 8 in 2024-09-21T12:00:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:39 smithi175 ceph-mon[32613]: from='osd.4 v2:172.21.15.175:6800/1102868557' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T12:00:40.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:39 smithi175 ceph-mon[32613]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T12:00:40.103 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:40.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:39 smithi043 ceph-mon[36454]: pgmap v1013: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:40.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:39 smithi043 ceph-mon[36454]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T12:00:40.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:39 smithi043 ceph-mon[36454]: osdmap e480: 8 total, 7 up, 8 in 2024-09-21T12:00:40.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:39 smithi043 ceph-mon[36454]: from='osd.4 v2:172.21.15.175:6800/1102868557' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T12:00:40.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:39 smithi043 ceph-mon[36454]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T12:00:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:39 smithi043 ceph-mon[29710]: pgmap v1013: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:39 smithi043 ceph-mon[29710]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T12:00:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:39 smithi043 ceph-mon[29710]: osdmap e480: 8 total, 7 up, 8 in 2024-09-21T12:00:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:39 smithi043 ceph-mon[29710]: from='osd.4 v2:172.21.15.175:6800/1102868557' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T12:00:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:39 smithi043 ceph-mon[29710]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]} : dispatch 2024-09-21T12:00:40.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-21T12:00:40.530 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T12:00:40.530 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T12:00:40.530 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T12:00:40.530 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T12:00:40.598 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: 5.7 scrub starts 2024-09-21T12:00:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: 5.7 scrub ok 2024-09-21T12:00:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T12:00:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: osd.4 v2:172.21.15.175:6800/1102868557 boot 2024-09-21T12:00:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: osdmap e481: 8 total, 8 up, 8 in 2024-09-21T12:00:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T12:00:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: 1.9 scrub starts 2024-09-21T12:00:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: 1.9 scrub ok 2024-09-21T12:00:40.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: 6.11 scrub starts 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: 6.11 scrub ok 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: 6.a scrub starts 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: 6.a scrub ok 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[36454]: osdmap e482: 8 total, 8 up, 8 in 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: 5.7 scrub starts 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: 5.7 scrub ok 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: osd.4 v2:172.21.15.175:6800/1102868557 boot 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: osdmap e481: 8 total, 8 up, 8 in 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T12:00:40.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: 1.9 scrub starts 2024-09-21T12:00:40.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: 1.9 scrub ok 2024-09-21T12:00:40.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: 6.11 scrub starts 2024-09-21T12:00:40.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: 6.11 scrub ok 2024-09-21T12:00:40.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: 6.a scrub starts 2024-09-21T12:00:40.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: 6.a scrub ok 2024-09-21T12:00:40.993 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:40 smithi043 ceph-mon[29710]: osdmap e482: 8 total, 8 up, 8 in 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: 5.7 scrub starts 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: 5.7 scrub ok 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: osd.4 v2:172.21.15.175:6800/1102868557 boot 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: osdmap e481: 8 total, 8 up, 8 in 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: 1.9 scrub starts 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: 1.9 scrub ok 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: 6.11 scrub starts 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: 6.11 scrub ok 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: 6.a scrub starts 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: 6.a scrub ok 2024-09-21T12:00:41.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:40 smithi175 ceph-mon[32613]: osdmap e482: 8 total, 8 up, 8 in 2024-09-21T12:00:41.144 INFO:teuthology.orchestra.run.smithi043.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:41.144 INFO:teuthology.orchestra.run.smithi043.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:41.187 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-21T12:00:41.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd pool set .mgr pgp_num 11 2024-09-21T12:00:41.960 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-21T12:00:41.964 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:42.022 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:41 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:00:42.022 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:41 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:00:42.022 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:41 smithi175 ceph-mon[32613]: pgmap v1016: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 667 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:42.086 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:41 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:00:42.086 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:41 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:00:42.086 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:41 smithi043 ceph-mon[36454]: pgmap v1016: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 667 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:42.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:41 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:00:42.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:41 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:00:42.086 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:41 smithi043 ceph-mon[29710]: pgmap v1016: 142 pgs: 39 active+undersized, 22 active+undersized+degraded, 81 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 667 B/s rd, 0 op/s; 83/597 objects degraded (13.903%) 2024-09-21T12:00:42.211 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:42.332 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:00:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:00:42.696 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:42.696 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:42 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T12:00:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:42 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T12:00:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:42 smithi175 ceph-mon[32613]: 4.13 scrub starts 2024-09-21T12:00:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:42 smithi175 ceph-mon[32613]: 4.13 scrub ok 2024-09-21T12:00:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:42 smithi175 ceph-mon[32613]: osdmap e483: 8 total, 8 up, 8 in 2024-09-21T12:00:43.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/932663692' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"} : dispatch 2024-09-21T12:00:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T12:00:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T12:00:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[29710]: 4.13 scrub starts 2024-09-21T12:00:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[29710]: 4.13 scrub ok 2024-09-21T12:00:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[29710]: osdmap e483: 8 total, 8 up, 8 in 2024-09-21T12:00:43.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/932663692' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"} : dispatch 2024-09-21T12:00:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T12:00:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T12:00:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[36454]: 4.13 scrub starts 2024-09-21T12:00:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[36454]: 4.13 scrub ok 2024-09-21T12:00:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[36454]: osdmap e483: 8 total, 8 up, 8 in 2024-09-21T12:00:43.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/932663692' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"} : dispatch 2024-09-21T12:00:43.312 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-21T12:00:43.558 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:43.563 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 1 pgp_num to 11 2024-09-21T12:00:44.004 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:43 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:44.004 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:43 smithi175 ceph-mon[32613]: pgmap v1018: 142 pgs: 3 active+undersized, 3 active+undersized+degraded, 136 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 6/597 objects degraded (1.005%); 0 B/s, 9 objects/s recovering 2024-09-21T12:00:44.004 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:43 smithi175 ceph-mon[32613]: Health check update: Degraded data redundancy: 6/597 objects degraded (1.005%), 3 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:44.004 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:43 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/932663692' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-21T12:00:44.005 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:43 smithi175 ceph-mon[32613]: osdmap e484: 8 total, 8 up, 8 in 2024-09-21T12:00:44.103 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:44.103 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:44.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:43 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:44.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:43 smithi043 ceph-mon[29710]: pgmap v1018: 142 pgs: 3 active+undersized, 3 active+undersized+degraded, 136 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 6/597 objects degraded (1.005%); 0 B/s, 9 objects/s recovering 2024-09-21T12:00:44.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:43 smithi043 ceph-mon[29710]: Health check update: Degraded data redundancy: 6/597 objects degraded (1.005%), 3 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:44.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:43 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/932663692' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-21T12:00:44.141 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:43 smithi043 ceph-mon[29710]: osdmap e484: 8 total, 8 up, 8 in 2024-09-21T12:00:44.142 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:43 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:44.142 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:43 smithi043 ceph-mon[36454]: pgmap v1018: 142 pgs: 3 active+undersized, 3 active+undersized+degraded, 136 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 6/597 objects degraded (1.005%); 0 B/s, 9 objects/s recovering 2024-09-21T12:00:44.142 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:43 smithi043 ceph-mon[36454]: Health check update: Degraded data redundancy: 6/597 objects degraded (1.005%), 3 pgs degraded (PG_DEGRADED) 2024-09-21T12:00:44.142 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:43 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/932663692' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-21T12:00:44.142 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:43 smithi043 ceph-mon[36454]: osdmap e484: 8 total, 8 up, 8 in 2024-09-21T12:00:44.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-21T12:00:44.798 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-21T12:00:44.844 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:44 smithi175 ceph-mon[32613]: 4.9 scrub starts 2024-09-21T12:00:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:44 smithi175 ceph-mon[32613]: 4.9 scrub ok 2024-09-21T12:00:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:44 smithi175 ceph-mon[32613]: 5.9 scrub starts 2024-09-21T12:00:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:44 smithi175 ceph-mon[32613]: 5.9 scrub ok 2024-09-21T12:00:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:44 smithi175 ceph-mon[32613]: 3.1d scrub starts 2024-09-21T12:00:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:44 smithi175 ceph-mon[32613]: 3.1d scrub ok 2024-09-21T12:00:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:44 smithi175 ceph-mon[32613]: 4.c scrub starts 2024-09-21T12:00:45.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:44 smithi175 ceph-mon[32613]: 4.c scrub ok 2024-09-21T12:00:45.048 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:45.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[36454]: 4.9 scrub starts 2024-09-21T12:00:45.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[36454]: 4.9 scrub ok 2024-09-21T12:00:45.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[36454]: 5.9 scrub starts 2024-09-21T12:00:45.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[36454]: 5.9 scrub ok 2024-09-21T12:00:45.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[36454]: 3.1d scrub starts 2024-09-21T12:00:45.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[36454]: 3.1d scrub ok 2024-09-21T12:00:45.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[36454]: 4.c scrub starts 2024-09-21T12:00:45.116 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[36454]: 4.c scrub ok 2024-09-21T12:00:45.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[29710]: 4.9 scrub starts 2024-09-21T12:00:45.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[29710]: 4.9 scrub ok 2024-09-21T12:00:45.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[29710]: 5.9 scrub starts 2024-09-21T12:00:45.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[29710]: 5.9 scrub ok 2024-09-21T12:00:45.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[29710]: 3.1d scrub starts 2024-09-21T12:00:45.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[29710]: 3.1d scrub ok 2024-09-21T12:00:45.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[29710]: 4.c scrub starts 2024-09-21T12:00:45.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:44 smithi043 ceph-mon[29710]: 4.c scrub ok 2024-09-21T12:00:45.395 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:00:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:45.398 INFO:teuthology.orchestra.run.smithi043.stdout:mon.a: {} 2024-09-21T12:00:45.399 INFO:teuthology.orchestra.run.smithi043.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T12:00:45.407 INFO:teuthology.orchestra.run.smithi043.stdout:mon.b: {} 2024-09-21T12:00:45.408 INFO:teuthology.orchestra.run.smithi043.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T12:00:45.416 INFO:teuthology.orchestra.run.smithi043.stdout:mon.c: {} 2024-09-21T12:00:45.416 INFO:teuthology.orchestra.run.smithi043.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-21T12:00:45.529 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:45.529 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:46.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:45 smithi175 ceph-mon[32613]: pgmap v1021: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 13 objects/s recovering 2024-09-21T12:00:46.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:45 smithi175 ceph-mon[32613]: 6.1d scrub starts 2024-09-21T12:00:46.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:45 smithi175 ceph-mon[32613]: 6.1d scrub ok 2024-09-21T12:00:46.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:45 smithi175 ceph-mon[32613]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/597 objects degraded (1.005%), 3 pgs degraded) 2024-09-21T12:00:46.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:45 smithi175 ceph-mon[32613]: Cluster is now healthy 2024-09-21T12:00:46.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:45 smithi175 ceph-mon[32613]: 5.18 scrub starts 2024-09-21T12:00:46.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:45 smithi175 ceph-mon[32613]: 5.18 scrub ok 2024-09-21T12:00:46.150 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3645140365' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T12:00:46.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[29710]: pgmap v1021: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 13 objects/s recovering 2024-09-21T12:00:46.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[29710]: 6.1d scrub starts 2024-09-21T12:00:46.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[29710]: 6.1d scrub ok 2024-09-21T12:00:46.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[29710]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/597 objects degraded (1.005%), 3 pgs degraded) 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[29710]: Cluster is now healthy 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[29710]: 5.18 scrub starts 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[29710]: 5.18 scrub ok 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3645140365' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[36454]: pgmap v1021: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 13 objects/s recovering 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[36454]: 6.1d scrub starts 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[36454]: 6.1d scrub ok 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[36454]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/597 objects degraded (1.005%), 3 pgs degraded) 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[36454]: Cluster is now healthy 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[36454]: 5.18 scrub starts 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[36454]: 5.18 scrub ok 2024-09-21T12:00:46.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3645140365' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T12:00:46.179 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-21T12:00:46.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 0 1 2024-09-21T12:00:46.442 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:46.474 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:46.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[29710]: 4.f scrub starts 2024-09-21T12:00:46.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[29710]: 4.f scrub ok 2024-09-21T12:00:46.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[29710]: 5.5 scrub starts 2024-09-21T12:00:46.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[29710]: 5.5 scrub ok 2024-09-21T12:00:46.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[29710]: 5.e scrub starts 2024-09-21T12:00:46.912 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[29710]: 5.e scrub ok 2024-09-21T12:00:46.912 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[36454]: 4.f scrub starts 2024-09-21T12:00:46.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[36454]: 4.f scrub ok 2024-09-21T12:00:46.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[36454]: 5.5 scrub starts 2024-09-21T12:00:46.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[36454]: 5.5 scrub ok 2024-09-21T12:00:46.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[36454]: 5.e scrub starts 2024-09-21T12:00:46.913 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:46 smithi043 ceph-mon[36454]: 5.e scrub ok 2024-09-21T12:00:46.933 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:46.934 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:46 smithi175 ceph-mon[32613]: 4.f scrub starts 2024-09-21T12:00:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:46 smithi175 ceph-mon[32613]: 4.f scrub ok 2024-09-21T12:00:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:46 smithi175 ceph-mon[32613]: 5.5 scrub starts 2024-09-21T12:00:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:46 smithi175 ceph-mon[32613]: 5.5 scrub ok 2024-09-21T12:00:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:46 smithi175 ceph-mon[32613]: 5.e scrub starts 2024-09-21T12:00:47.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:46 smithi175 ceph-mon[32613]: 5.e scrub ok 2024-09-21T12:00:47.559 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.0 to 1 (10000) 2024-09-21T12:00:47.744 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: pgmap v1022: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 9 objects/s recovering 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: 5.1d scrub starts 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: 5.1d scrub ok 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: 3.15 scrub starts 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: 3.15 scrub ok 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: 6.7 scrub starts 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: 6.7 scrub ok 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1483404810' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:47.883 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: 3.a scrub starts 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: 3.a scrub ok 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1483404810' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[29710]: osdmap e485: 8 total, 8 up, 8 in 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: pgmap v1022: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 9 objects/s recovering 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: 5.1d scrub starts 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: 5.1d scrub ok 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: 3.15 scrub starts 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: 3.15 scrub ok 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: 6.7 scrub starts 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: 6.7 scrub ok 2024-09-21T12:00:47.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1483404810' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T12:00:47.885 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:47.885 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:47.885 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: 3.a scrub starts 2024-09-21T12:00:47.885 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: 3.a scrub ok 2024-09-21T12:00:47.885 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1483404810' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T12:00:47.885 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:47 smithi043 ceph-mon[36454]: osdmap e485: 8 total, 8 up, 8 in 2024-09-21T12:00:47.995 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: pgmap v1022: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 9 objects/s recovering 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: 5.1d scrub starts 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: 5.1d scrub ok 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: 3.15 scrub starts 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: 3.15 scrub ok 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: 6.7 scrub starts 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: 6.7 scrub ok 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1483404810' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: 3.a scrub starts 2024-09-21T12:00:48.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: 3.a scrub ok 2024-09-21T12:00:48.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1483404810' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-21T12:00:48.012 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:47 smithi175 ceph-mon[32613]: osdmap e485: 8 total, 8 up, 8 in 2024-09-21T12:00:48.333 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 0 1 2024-09-21T12:00:48.438 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:48.438 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:48.596 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:48.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:48 smithi043 ceph-mon[36454]: 4.1e scrub starts 2024-09-21T12:00:48.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:48 smithi043 ceph-mon[36454]: 4.1e scrub ok 2024-09-21T12:00:48.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:48 smithi043 ceph-mon[36454]: 4.1d scrub starts 2024-09-21T12:00:48.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:48 smithi043 ceph-mon[36454]: 4.1d scrub ok 2024-09-21T12:00:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:48 smithi043 ceph-mon[29710]: 4.1e scrub starts 2024-09-21T12:00:48.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:48 smithi043 ceph-mon[29710]: 4.1e scrub ok 2024-09-21T12:00:48.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:48 smithi043 ceph-mon[29710]: 4.1d scrub starts 2024-09-21T12:00:48.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:48 smithi043 ceph-mon[29710]: 4.1d scrub ok 2024-09-21T12:00:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:48 smithi175 ceph-mon[32613]: 4.1e scrub starts 2024-09-21T12:00:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:48 smithi175 ceph-mon[32613]: 4.1e scrub ok 2024-09-21T12:00:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:48 smithi175 ceph-mon[32613]: 4.1d scrub starts 2024-09-21T12:00:49.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:48 smithi175 ceph-mon[32613]: 4.1d scrub ok 2024-09-21T12:00:49.044 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-21T12:00:49.290 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:49.753 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:49.753 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:49.795 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-21T12:00:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:49 smithi175 ceph-mon[32613]: pgmap v1024: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T12:00:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:49 smithi175 ceph-mon[32613]: 4.2 scrub starts 2024-09-21T12:00:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:49 smithi175 ceph-mon[32613]: 4.2 scrub ok 2024-09-21T12:00:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:49 smithi175 ceph-mon[32613]: 3.6 scrub starts 2024-09-21T12:00:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:49 smithi175 ceph-mon[32613]: 3.6 scrub ok 2024-09-21T12:00:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:49 smithi175 ceph-mon[32613]: 5.0 scrub starts 2024-09-21T12:00:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:49 smithi175 ceph-mon[32613]: 5.0 scrub ok 2024-09-21T12:00:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:49 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1767857032' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-21T12:00:50.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:49 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-21T12:00:50.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[36454]: pgmap v1024: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T12:00:50.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[36454]: 4.2 scrub starts 2024-09-21T12:00:50.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[36454]: 4.2 scrub ok 2024-09-21T12:00:50.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[36454]: 3.6 scrub starts 2024-09-21T12:00:50.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[36454]: 3.6 scrub ok 2024-09-21T12:00:50.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[36454]: 5.0 scrub starts 2024-09-21T12:00:50.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[36454]: 5.0 scrub ok 2024-09-21T12:00:50.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1767857032' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-21T12:00:50.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-21T12:00:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[29710]: pgmap v1024: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T12:00:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[29710]: 4.2 scrub starts 2024-09-21T12:00:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[29710]: 4.2 scrub ok 2024-09-21T12:00:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[29710]: 3.6 scrub starts 2024-09-21T12:00:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[29710]: 3.6 scrub ok 2024-09-21T12:00:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[29710]: 5.0 scrub starts 2024-09-21T12:00:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[29710]: 5.0 scrub ok 2024-09-21T12:00:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1767857032' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-21T12:00:50.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:49 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-21T12:00:50.453 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-21T12:00:50.701 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:50.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 1 1 2024-09-21T12:00:51.079 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:51.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:50 smithi175 ceph-mon[32613]: 1.1 scrub starts 2024-09-21T12:00:51.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:50 smithi175 ceph-mon[32613]: 1.1 scrub ok 2024-09-21T12:00:51.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:50 smithi175 ceph-mon[32613]: 3.13 scrub starts 2024-09-21T12:00:51.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:50 smithi175 ceph-mon[32613]: 3.13 scrub ok 2024-09-21T12:00:51.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:50 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-21T12:00:51.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:50 smithi175 ceph-mon[32613]: osdmap e486: 8 total, 8 up, 8 in 2024-09-21T12:00:51.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:50 smithi175 ceph-mon[32613]: 4.b scrub starts 2024-09-21T12:00:51.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:50 smithi175 ceph-mon[32613]: 4.b scrub ok 2024-09-21T12:00:51.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[36454]: 1.1 scrub starts 2024-09-21T12:00:51.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[36454]: 1.1 scrub ok 2024-09-21T12:00:51.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[36454]: 3.13 scrub starts 2024-09-21T12:00:51.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[36454]: 3.13 scrub ok 2024-09-21T12:00:51.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-21T12:00:51.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[36454]: osdmap e486: 8 total, 8 up, 8 in 2024-09-21T12:00:51.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[36454]: 4.b scrub starts 2024-09-21T12:00:51.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[36454]: 4.b scrub ok 2024-09-21T12:00:51.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[29710]: 1.1 scrub starts 2024-09-21T12:00:51.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[29710]: 1.1 scrub ok 2024-09-21T12:00:51.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[29710]: 3.13 scrub starts 2024-09-21T12:00:51.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[29710]: 3.13 scrub ok 2024-09-21T12:00:51.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-21T12:00:51.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[29710]: osdmap e486: 8 total, 8 up, 8 in 2024-09-21T12:00:51.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[29710]: 4.b scrub starts 2024-09-21T12:00:51.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:50 smithi043 ceph-mon[29710]: 4.b scrub ok 2024-09-21T12:00:51.253 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:51.253 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:51.810 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.1 to 1 (10000) 2024-09-21T12:00:51.844 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:51 smithi175 ceph-mon[32613]: 1.a scrub starts 2024-09-21T12:00:51.844 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:51 smithi175 ceph-mon[32613]: 1.a scrub ok 2024-09-21T12:00:51.844 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:51 smithi175 ceph-mon[32613]: pgmap v1026: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:00:51.844 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:51 smithi175 ceph-mon[32613]: 3.19 scrub starts 2024-09-21T12:00:51.844 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:51 smithi175 ceph-mon[32613]: 3.19 scrub ok 2024-09-21T12:00:51.845 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:51 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:00:51.845 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:51 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:00:51.845 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:51 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2124835527' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:51.845 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:51 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:51.870 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-21T12:00:52.116 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:52.150 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[36454]: 1.a scrub starts 2024-09-21T12:00:52.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[36454]: 1.a scrub ok 2024-09-21T12:00:52.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[36454]: pgmap v1026: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:00:52.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[36454]: 3.19 scrub starts 2024-09-21T12:00:52.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[36454]: 3.19 scrub ok 2024-09-21T12:00:52.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:00:52.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:00:52.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2124835527' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:52.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:52.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[29710]: 1.a scrub starts 2024-09-21T12:00:52.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[29710]: 1.a scrub ok 2024-09-21T12:00:52.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[29710]: pgmap v1026: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:00:52.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[29710]: 3.19 scrub starts 2024-09-21T12:00:52.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[29710]: 3.19 scrub ok 2024-09-21T12:00:52.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:00:52.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:00:52.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2124835527' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:52.152 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:51 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:52.237 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:00:52 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:00:52.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 1 1 2024-09-21T12:00:52.620 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:52.620 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:53.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[29710]: 1.3 scrub starts 2024-09-21T12:00:53.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[29710]: 1.3 scrub ok 2024-09-21T12:00:53.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T12:00:53.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[29710]: osdmap e487: 8 total, 8 up, 8 in 2024-09-21T12:00:53.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[29710]: 3.1 scrub starts 2024-09-21T12:00:53.072 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[29710]: 3.1 scrub ok 2024-09-21T12:00:53.073 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[36454]: 1.3 scrub starts 2024-09-21T12:00:53.073 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[36454]: 1.3 scrub ok 2024-09-21T12:00:53.073 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T12:00:53.073 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[36454]: osdmap e487: 8 total, 8 up, 8 in 2024-09-21T12:00:53.073 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[36454]: 3.1 scrub starts 2024-09-21T12:00:53.073 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:52 smithi043 ceph-mon[36454]: 3.1 scrub ok 2024-09-21T12:00:53.215 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:52 smithi175 ceph-mon[32613]: 1.3 scrub starts 2024-09-21T12:00:53.215 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:52 smithi175 ceph-mon[32613]: 1.3 scrub ok 2024-09-21T12:00:53.216 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:52 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-21T12:00:53.216 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:52 smithi175 ceph-mon[32613]: osdmap e487: 8 total, 8 up, 8 in 2024-09-21T12:00:53.216 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:52 smithi175 ceph-mon[32613]: 3.1 scrub starts 2024-09-21T12:00:53.216 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:52 smithi175 ceph-mon[32613]: 3.1 scrub ok 2024-09-21T12:00:53.243 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-21T12:00:53.486 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-21T12:00:53.886 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:53 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:53.886 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:53 smithi175 ceph-mon[32613]: 1.8 scrub starts 2024-09-21T12:00:53.886 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:53 smithi175 ceph-mon[32613]: 1.8 scrub ok 2024-09-21T12:00:53.886 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:53 smithi175 ceph-mon[32613]: pgmap v1028: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:00:53.886 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:53 smithi175 ceph-mon[32613]: 5.3 scrub starts 2024-09-21T12:00:53.886 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:53 smithi175 ceph-mon[32613]: 5.3 scrub ok 2024-09-21T12:00:53.887 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:53 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4289122982' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:53.887 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:53 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:53.887 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:53 smithi175 ceph-mon[32613]: 4.e scrub starts 2024-09-21T12:00:53.887 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:53 smithi175 ceph-mon[32613]: 4.e scrub ok 2024-09-21T12:00:53.994 INFO:teuthology.orchestra.run.smithi175.stdout:ERROR: (22) Invalid argument 2024-09-21T12:00:53.994 INFO:teuthology.orchestra.run.smithi175.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-21T12:00:54.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:54.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[29710]: 1.8 scrub starts 2024-09-21T12:00:54.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[29710]: 1.8 scrub ok 2024-09-21T12:00:54.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[29710]: pgmap v1028: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:00:54.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[29710]: 5.3 scrub starts 2024-09-21T12:00:54.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[29710]: 5.3 scrub ok 2024-09-21T12:00:54.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4289122982' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:54.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:54.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[29710]: 4.e scrub starts 2024-09-21T12:00:54.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[29710]: 4.e scrub ok 2024-09-21T12:00:54.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:00:54.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[36454]: 1.8 scrub starts 2024-09-21T12:00:54.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[36454]: 1.8 scrub ok 2024-09-21T12:00:54.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[36454]: pgmap v1028: 142 pgs: 142 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:00:54.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[36454]: 5.3 scrub starts 2024-09-21T12:00:54.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[36454]: 5.3 scrub ok 2024-09-21T12:00:54.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4289122982' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:54.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-21T12:00:54.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[36454]: 4.e scrub starts 2024-09-21T12:00:54.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:53 smithi043 ceph-mon[36454]: 4.e scrub ok 2024-09-21T12:00:54.570 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-21T12:00:54.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 2 1 2024-09-21T12:00:54.819 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:54.889 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[36454]: 5.c scrub starts 2024-09-21T12:00:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[36454]: 5.c scrub ok 2024-09-21T12:00:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[36454]: 1.2 scrub starts 2024-09-21T12:00:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[36454]: 1.2 scrub ok 2024-09-21T12:00:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-21T12:00:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[36454]: osdmap e488: 8 total, 8 up, 8 in 2024-09-21T12:00:54.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[36454]: 4.3 scrub starts 2024-09-21T12:00:54.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[36454]: 4.3 scrub ok 2024-09-21T12:00:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[29710]: 5.c scrub starts 2024-09-21T12:00:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[29710]: 5.c scrub ok 2024-09-21T12:00:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[29710]: 1.2 scrub starts 2024-09-21T12:00:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[29710]: 1.2 scrub ok 2024-09-21T12:00:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-21T12:00:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[29710]: osdmap e488: 8 total, 8 up, 8 in 2024-09-21T12:00:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[29710]: 4.3 scrub starts 2024-09-21T12:00:54.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:54 smithi043 ceph-mon[29710]: 4.3 scrub ok 2024-09-21T12:00:55.085 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:54 smithi175 ceph-mon[32613]: 5.c scrub starts 2024-09-21T12:00:55.085 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:54 smithi175 ceph-mon[32613]: 5.c scrub ok 2024-09-21T12:00:55.085 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:54 smithi175 ceph-mon[32613]: 1.2 scrub starts 2024-09-21T12:00:55.085 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:54 smithi175 ceph-mon[32613]: 1.2 scrub ok 2024-09-21T12:00:55.085 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:54 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-21T12:00:55.085 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:54 smithi175 ceph-mon[32613]: osdmap e488: 8 total, 8 up, 8 in 2024-09-21T12:00:55.085 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:54 smithi175 ceph-mon[32613]: 4.3 scrub starts 2024-09-21T12:00:55.086 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:54 smithi175 ceph-mon[32613]: 4.3 scrub ok 2024-09-21T12:00:55.271 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:00:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:00:55.317 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T12:00:55.317 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T12:00:55.317 INFO:teuthology.orchestra.run.smithi175.stdout: "num_ops": 0 2024-09-21T12:00:55.317 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T12:00:55.827 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.2 to 1 (10000) 2024-09-21T12:00:55.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:55 smithi175 ceph-mon[32613]: pgmap v1030: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:00:55.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:55 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:00:55.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:55 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:00:55.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:55 smithi175 ceph-mon[32613]: 4.5 scrub starts 2024-09-21T12:00:55.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:55 smithi175 ceph-mon[32613]: 4.5 scrub ok 2024-09-21T12:00:55.998 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:55 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/66006439' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T12:00:56.027 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-21T12:00:56.205 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[36454]: pgmap v1030: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:00:56.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:00:56.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:00:56.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[36454]: 4.5 scrub starts 2024-09-21T12:00:56.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[36454]: 4.5 scrub ok 2024-09-21T12:00:56.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/66006439' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T12:00:56.206 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[29710]: pgmap v1030: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:00:56.206 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:00:56.206 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:00:56.206 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[29710]: 4.5 scrub starts 2024-09-21T12:00:56.206 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[29710]: 4.5 scrub ok 2024-09-21T12:00:56.207 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:55 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/66006439' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-21T12:00:56.272 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:56.593 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 2 1 2024-09-21T12:00:56.797 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T12:00:56.797 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [], 2024-09-21T12:00:56.797 INFO:teuthology.orchestra.run.smithi175.stdout: "complaint_time": 30, 2024-09-21T12:00:56.797 INFO:teuthology.orchestra.run.smithi175.stdout: "num_blocked_ops": 0 2024-09-21T12:00:56.797 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T12:00:56.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[29710]: 6.e scrub starts 2024-09-21T12:00:56.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[29710]: 6.e scrub ok 2024-09-21T12:00:56.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:00:56.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:00:56.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:00:56.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:00:56.832 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/66006439' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T12:00:56.866 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:56 smithi175 ceph-mon[32613]: 6.e scrub starts 2024-09-21T12:00:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:56 smithi175 ceph-mon[32613]: 6.e scrub ok 2024-09-21T12:00:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:56 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:00:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:56 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:00:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:56 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:00:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:56 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:00:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:56 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/66006439' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T12:00:57.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:56 smithi175 ceph-mon[32613]: osdmap e489: 8 total, 8 up, 8 in 2024-09-21T12:00:57.127 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[29710]: osdmap e489: 8 total, 8 up, 8 in 2024-09-21T12:00:57.127 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[36454]: 6.e scrub starts 2024-09-21T12:00:57.128 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[36454]: 6.e scrub ok 2024-09-21T12:00:57.128 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:00:57.128 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:00:57.128 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:00:57.128 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:00:57.128 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/66006439' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-21T12:00:57.128 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:56 smithi043 ceph-mon[36454]: osdmap e489: 8 total, 8 up, 8 in 2024-09-21T12:00:57.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-21T12:00:57.738 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.b/config 2024-09-21T12:00:58.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:57 smithi175 ceph-mon[32613]: pgmap v1032: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:00:58.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:57 smithi175 ceph-mon[32613]: 3.e scrub starts 2024-09-21T12:00:58.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:57 smithi175 ceph-mon[32613]: 3.e scrub ok 2024-09-21T12:00:58.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:57 smithi175 ceph-mon[32613]: 5.14 scrub starts 2024-09-21T12:00:58.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:57 smithi175 ceph-mon[32613]: 5.14 scrub ok 2024-09-21T12:00:58.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:57 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2459008236' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-21T12:00:58.218 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-21T12:00:58.218 INFO:teuthology.orchestra.run.smithi175.stdout: "size": 20, 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 600, 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: "ops": [ 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "pg_update_log_missing(5.e epoch 482/481 rep_tid 194725 entries 482'13 (0'0) error 5:7759931f:::notify.3:head by client.14445.0:4347 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.764043+0000", 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.451693661, 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0048813700000000003, 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.219 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764043+0000", 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764043+0000", 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764043+0000", 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.74e-07 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.220 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764050+0000", 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.6649999999999997e-06 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764063+0000", 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3458000000000001e-05 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764078+0000", 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5037e-05 2024-09-21T12:00:58.221 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764555+0000", 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00047632399999999997 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.768924+0000", 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043697120000000004 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.222 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "pg_update_log_missing(5.1f epoch 482/481 rep_tid 216806 entries 482'12 (0'0) error 5:f95f44c2:::notify.0:head by client.14445.0:4348 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.764736+0000", 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.451000517000001, 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0043912810000000004, 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764736+0000", 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.223 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764736+0000", 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764736+0000", 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.11e-07 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764742+0000", 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5140000000000001e-06 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.224 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764754+0000", 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2554999999999999e-05 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.764767+0000", 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2330000000000001e-05 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.765179+0000", 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00041258799999999997 2024-09-21T12:00:58.225 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.769127+0000", 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0039480829999999998 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.1 482/481 [PushOp(4:856b9b26:reshard::reshard.0000000002:head, version: 471'30, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:856b9b26:reshard::reshard.0000000002:head@471'30, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.766349+0000", 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.449387562999998, 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0012561479999999999, 2024-09-21T12:00:58.226 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.766349+0000", 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.766349+0000", 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.227 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.766350+0000", 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.8800000000000002e-07 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.766363+0000", 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3062e-05 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.766379+0000", 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5999999999999999e-05 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.228 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.766395+0000", 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5909e-05 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.767472+0000", 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001077017 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.767484+0000", 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1912999999999999e-05 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.229 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.767605+0000", 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012155900000000001 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "pg_update_log_missing(5.0 epoch 482/481 rep_tid 233543 entries 482'12 (0'0) error 5:05bf5b68:::notify.1:head by client.14445.0:4346 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.769677+0000", 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.446059762000001, 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0017631389999999999, 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.230 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.769677+0000", 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.769677+0000", 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.231 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.769677+0000", 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1299999999999999e-07 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.769682+0000", 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.9010000000000002e-06 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.769695+0000", 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3251000000000001e-05 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.232 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.769708+0000", 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2738999999999999e-05 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.769987+0000", 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000279564 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.771440+0000", 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0014524710000000001 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.233 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "pg_update_log_missing(5.9 epoch 482/481 rep_tid 138406 entries 482'10 (0'0) error 5:93e5b521:::notify.7:head by client.14445.0:4350 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.770960+0000", 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.444776348000001, 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0022834700000000001, 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.770960+0000", 2024-09-21T12:00:58.234 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.770960+0000", 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.770960+0000", 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6400000000000001e-07 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.770965+0000", 2024-09-21T12:00:58.235 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 4.8040000000000001e-06 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.770977+0000", 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.147e-05 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.770990+0000", 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3304e-05 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.236 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.771393+0000", 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00040281600000000001 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773244+0000", 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0018509119999999999 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.1 482/481 [PushOp(4:80b6fe23:reshard::reshard.0000000011:head, version: 471'32, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:80b6fe23:reshard::reshard.0000000011:head@471'32, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.771892+0000", 2024-09-21T12:00:58.237 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.443844978000001, 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001169205, 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.771892+0000", 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.771892+0000", 2024-09-21T12:00:58.238 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.771892+0000", 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.5899999999999996e-07 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.771904+0000", 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1943000000000001e-05 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.239 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.771919+0000", 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4596999999999999e-05 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.771940+0000", 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1168000000000001e-05 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772909+0000", 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00096886100000000001 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.240 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772921+0000", 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1885000000000001e-05 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773061+0000", 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000140192 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "pg_update_log_missing(4.5 epoch 482/481 rep_tid 216807 entries 482'42 (0'0) error 4:a60f9269:lc::lua_package_allowlist:head by client.14445.0:4349 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-21T12:00:58.241 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.772443+0000", 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.443293149999999, 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0025308209999999999, 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "reached pg", 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772443+0000", 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.242 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772443+0000", 2024-09-21T12:00:58.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.243 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.243 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772444+0000", 2024-09-21T12:00:58.243 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2599999999999998e-07 2024-09-21T12:00:58.243 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.243 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[29710]: pgmap v1032: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:00:58.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[29710]: 3.e scrub starts 2024-09-21T12:00:58.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[29710]: 3.e scrub ok 2024-09-21T12:00:58.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[29710]: 5.14 scrub starts 2024-09-21T12:00:58.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[29710]: 5.14 scrub ok 2024-09-21T12:00:58.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2459008236' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-21T12:00:58.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[36454]: pgmap v1032: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:00:58.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[36454]: 3.e scrub starts 2024-09-21T12:00:58.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[36454]: 3.e scrub ok 2024-09-21T12:00:58.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[36454]: 5.14 scrub starts 2024-09-21T12:00:58.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[36454]: 5.14 scrub ok 2024-09-21T12:00:58.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:57 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2459008236' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-21T12:00:58.245 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.245 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772450+0000", 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8680000000000004e-06 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772463+0000", 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3351e-05 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772476+0000", 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3203999999999999e-05 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.246 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773622+0000", 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.0011458460000000001 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.774974+0000", 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.001352226 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.9 482/481 [PushOp(4:9268b73e:::obj_delete_at_hint.0000000019:head, version: 471'20, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:9268b73e:::obj_delete_at_hint.0000000019:head@471'20, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.247 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.772874+0000", 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.442862539, 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00035498799999999998, 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772874+0000", 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772874+0000", 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.248 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772874+0000", 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.9600000000000001e-07 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772884+0000", 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.9420000000000007e-06 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772894+0000", 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0066e-05 2024-09-21T12:00:58.249 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.772907+0000", 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2394e-05 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773112+0000", 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000205548 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773124+0000", 2024-09-21T12:00:58.250 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2e-05 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773229+0000", 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000104842 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.18 482/481 [PushOp(4:1a734c59:::obj_delete_at_hint.0000000000:head, version: 471'17, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:1a734c59:::obj_delete_at_hint.0000000000:head@471'17, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.773844+0000", 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.441892873, 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00030621499999999998, 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.251 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773844+0000", 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773844+0000", 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773844+0000", 2024-09-21T12:00:58.252 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.5000000000000002e-07 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773857+0000", 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2204e-05 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773872+0000", 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5806000000000001e-05 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.773905+0000", 2024-09-21T12:00:58.253 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2295999999999997e-05 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.774036+0000", 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00013113600000000001 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.774048+0000", 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2435e-05 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.774150+0000", 2024-09-21T12:00:58.254 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010168800000000001 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.9 482/481 [PushOp(4:9195808b:::obj_delete_at_hint.0000000027:head, version: 471'22, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:9195808b:::obj_delete_at_hint.0000000027:head@471'22, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.777183+0000", 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.43855361, 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00034779500000000003, 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.255 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.777183+0000", 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.777183+0000", 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.777183+0000", 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.1299999999999999e-07 2024-09-21T12:00:58.256 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.777192+0000", 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.9870000000000005e-06 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.777204+0000", 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.167e-05 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.777219+0000", 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4786e-05 2024-09-21T12:00:58.257 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.777341+0000", 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012207 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.777354+0000", 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3210000000000001e-05 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.258 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.777531+0000", 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000176859 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.13 482/481 [PushOp(4:cd178ae0:reshard::reshard.0000000014:head, version: 471'21, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:cd178ae0:reshard::reshard.0000000014:head@471'21, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.778019+0000", 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.437717033999999, 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026275799999999998, 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.259 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778019+0000", 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778019+0000", 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778020+0000", 2024-09-21T12:00:58.260 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3000000000000001e-07 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778039+0000", 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8831e-05 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778051+0000", 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2123999999999999e-05 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.261 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778069+0000", 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7643e-05 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778145+0000", 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.6375e-05 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778158+0000", 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3489000000000001e-05 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.262 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778282+0000", 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00012376600000000001 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.1 482/481 [PushOp(4:80d23011:::obj_delete_at_hint.0000000085:head, version: 471'40, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:80d23011:::obj_delete_at_hint.0000000085:head@471'40, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.778020+0000", 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.437716430999998, 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025957200000000001, 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.263 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778020+0000", 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778020+0000", 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778021+0000", 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.1500000000000004e-07 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.264 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778030+0000", 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6250000000000002e-06 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778048+0000", 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.7725000000000001e-05 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778074+0000", 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.605e-05 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.265 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778168+0000", 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.3572000000000005e-05 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778179+0000", 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1001e-05 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778280+0000", 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010098399999999999 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.266 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.9 482/481 [PushOp(4:94441385:::obj_delete_at_hint.0000000039:head, version: 471'24, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:94441385:::obj_delete_at_hint.0000000039:head@471'24, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.778466+0000", 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.437270939000001, 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026917400000000001, 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778466+0000", 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.267 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778466+0000", 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778466+0000", 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.4700000000000001e-07 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778477+0000", 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.118e-05 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.268 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778494+0000", 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6711999999999999e-05 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778508+0000", 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4412e-05 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778610+0000", 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000101365 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778623+0000", 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3613e-05 2024-09-21T12:00:58.269 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.778735+0000", 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111345 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.13 482/481 [PushOp(4:cba150de:::obj_delete_at_hint.0000000051:head, version: 471'25, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:cba150de:::obj_delete_at_hint.0000000051:head@471'25, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.780105+0000", 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.43563164, 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00029104199999999998, 2024-09-21T12:00:58.270 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.780105+0000", 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.780105+0000", 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.271 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.780106+0000", 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.1099999999999995e-07 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.780116+0000", 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0522e-05 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.780134+0000", 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8033000000000002e-05 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.272 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.780149+0000", 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.4766000000000001e-05 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.780263+0000", 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000113646 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.780274+0000", 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1463e-05 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.780396+0000", 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000121901 2024-09-21T12:00:58.273 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.5 482/481 [PushOp(4:a4a56432:::obj_delete_at_hint.0000000012:head, version: 471'33, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:a4a56432:::obj_delete_at_hint.0000000012:head@471'33, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.782246+0000", 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.43349035, 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026206999999999999, 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.274 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.782246+0000", 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.782246+0000", 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.782247+0000", 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.2000000000000001e-07 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.782256+0000", 2024-09-21T12:00:58.275 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8689999999999999e-06 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.782275+0000", 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.8799000000000001e-05 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.782291+0000", 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.63e-05 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.782395+0000", 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010385100000000001 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.276 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.782407+0000", 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2092e-05 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.782508+0000", 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000100839 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.2 482/481 [PushOp(4:46cb1ec1:reshard::reshard.0000000003:head, version: 471'22, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:46cb1ec1:reshard::reshard.0000000003:head@471'22, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.783428+0000", 2024-09-21T12:00:58.277 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.432308278000001, 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026501900000000002, 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.783428+0000", 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.783428+0000", 2024-09-21T12:00:58.278 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.783429+0000", 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 7.0999999999999998e-07 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.783441+0000", 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1615e-05 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.279 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.783458+0000", 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.755e-05 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.783473+0000", 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5148e-05 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.783559+0000", 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.5591000000000001e-05 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.280 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.783572+0000", 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.3614e-05 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.783693+0000", 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000120791 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.2 482/481 [PushOp(4:47403372:::obj_delete_at_hint.0000000112:head, version: 471'30, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:47403372:::obj_delete_at_hint.0000000112:head@471'30, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.792815+0000", 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.422921671000001, 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00026952699999999998, 2024-09-21T12:00:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.792815+0000", 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.792815+0000", 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.282 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.792815+0000", 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.2799999999999996e-07 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.792826+0000", 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0823e-05 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.792843+0000", 2024-09-21T12:00:58.283 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.6946e-05 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.792870+0000", 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6650000000000001e-05 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.792962+0000", 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.2131000000000003e-05 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.792973+0000", 2024-09-21T12:00:58.284 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.1318000000000001e-05 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.793084+0000", 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000111131 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDPGPush(4.b 482/481 [PushOp(4:d201668e:::obj_delete_at_hint.0000000023:head, version: 471'17, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(4:d201668e:::obj_delete_at_hint.0000000023:head@471'17, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:40.793241+0000", 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 17.422495735999998, 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00025797400000000001, 2024-09-21T12:00:58.285 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.793241+0000", 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.793241+0000", 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.286 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.793241+0000", 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 6.06e-07 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.793251+0000", 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.6509999999999996e-06 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.793274+0000", 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.2940000000000001e-05 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.287 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.793300+0000", 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6241000000000002e-05 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.793382+0000", 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 8.2099000000000001e-05 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.793395+0000", 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2901000000000001e-05 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:40.793499+0000", 2024-09-21T12:00:58.288 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00010353600000000001 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDScrubReserve(3.6,QREQUEST,e:485,from:0,reservation_nonce:3)", 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:48.887432+0000", 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 9.328304266, 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000355368, 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.289 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:48.887432+0000", 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:48.887432+0000", 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:48.887434+0000", 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.846e-06 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.290 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:48.887444+0000", 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.0149999999999999e-05 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:48.887471+0000", 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.6576000000000001e-05 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:48.887484+0000", 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.2670000000000001e-05 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.291 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:48.887630+0000", 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.000146718 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:48.887689+0000", 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.8579000000000003e-05 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:48.887788+0000", 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 9.8828999999999999e-05 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.292 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "description": "MOSDScrubReserve(4.b,RELEASE,e:486,from:0,reservation_nonce:0)", 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "initiated_at": "2024-09-21T12:00:49.872974+0000", 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "age": 8.3427628649999992, 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00033733500000000002, 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "continuous": false, 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "type_data": { 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "flag_point": "started", 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "events": [ 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "initiated", 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:49.872974+0000", 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.293 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "header_read", 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:49.872974+0000", 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "throttled", 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:49.872974+0000", 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 3.5900000000000003e-07 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "all_read", 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:49.872976+0000", 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.3740000000000001e-06 2024-09-21T12:00:58.294 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "dispatched", 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:49.872982+0000", 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.3759999999999999e-06 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "queued_for_pg", 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:49.872987+0000", 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 5.6640000000000003e-06 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "reached_pg", 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:49.873276+0000", 2024-09-21T12:00:58.295 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 0.00028807799999999999 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "started", 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:49.873291+0000", 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 1.5452e-05 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: { 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: "event": "done", 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: "time": "2024-09-21T12:00:49.873311+0000", 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: "duration": 2.0032000000000001e-05 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.296 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-09-21T12:00:58.297 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-21T12:00:58.297 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-21T12:00:58.563 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-21T12:00:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[29710]: 5.1a scrub starts 2024-09-21T12:00:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[29710]: 5.1a scrub ok 2024-09-21T12:00:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[29710]: 3.17 scrub starts 2024-09-21T12:00:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[29710]: 3.17 scrub ok 2024-09-21T12:00:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[29710]: pgmap v1033: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 796 B/s rd, 0 op/s 2024-09-21T12:00:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[29710]: 4.a scrub starts 2024-09-21T12:00:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[29710]: 4.a scrub ok 2024-09-21T12:00:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2459008236' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-21T12:00:59.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[29710]: osdmap e490: 8 total, 8 up, 8 in 2024-09-21T12:00:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[36454]: 5.1a scrub starts 2024-09-21T12:00:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[36454]: 5.1a scrub ok 2024-09-21T12:00:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[36454]: 3.17 scrub starts 2024-09-21T12:00:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[36454]: 3.17 scrub ok 2024-09-21T12:00:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[36454]: pgmap v1033: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 796 B/s rd, 0 op/s 2024-09-21T12:00:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[36454]: 4.a scrub starts 2024-09-21T12:00:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[36454]: 4.a scrub ok 2024-09-21T12:00:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2459008236' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-21T12:00:59.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:58 smithi043 ceph-mon[36454]: osdmap e490: 8 total, 8 up, 8 in 2024-09-21T12:00:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:58 smithi175 ceph-mon[32613]: 5.1a scrub starts 2024-09-21T12:00:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:58 smithi175 ceph-mon[32613]: 5.1a scrub ok 2024-09-21T12:00:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:58 smithi175 ceph-mon[32613]: 3.17 scrub starts 2024-09-21T12:00:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:58 smithi175 ceph-mon[32613]: 3.17 scrub ok 2024-09-21T12:00:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:58 smithi175 ceph-mon[32613]: pgmap v1033: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 796 B/s rd, 0 op/s 2024-09-21T12:00:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:58 smithi175 ceph-mon[32613]: 4.a scrub starts 2024-09-21T12:00:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:58 smithi175 ceph-mon[32613]: 4.a scrub ok 2024-09-21T12:00:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:58 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2459008236' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-21T12:00:59.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:58 smithi175 ceph-mon[32613]: osdmap e490: 8 total, 8 up, 8 in 2024-09-21T12:00:59.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 3 1 2024-09-21T12:00:59.591 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:00:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[36454]: 5.12 scrub starts 2024-09-21T12:00:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[36454]: 5.12 scrub ok 2024-09-21T12:00:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[36454]: 5.10 scrub starts 2024-09-21T12:00:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[36454]: 5.10 scrub ok 2024-09-21T12:00:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[36454]: 5.11 scrub starts 2024-09-21T12:00:59.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[36454]: 5.11 scrub ok 2024-09-21T12:00:59.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[29710]: 5.12 scrub starts 2024-09-21T12:00:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[29710]: 5.12 scrub ok 2024-09-21T12:00:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[29710]: 5.10 scrub starts 2024-09-21T12:00:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[29710]: 5.10 scrub ok 2024-09-21T12:00:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[29710]: 5.11 scrub starts 2024-09-21T12:00:59.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:00:59 smithi043 ceph-mon[29710]: 5.11 scrub ok 2024-09-21T12:01:00.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:59 smithi175 ceph-mon[32613]: 5.12 scrub starts 2024-09-21T12:01:00.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:59 smithi175 ceph-mon[32613]: 5.12 scrub ok 2024-09-21T12:01:00.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:59 smithi175 ceph-mon[32613]: 5.10 scrub starts 2024-09-21T12:01:00.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:59 smithi175 ceph-mon[32613]: 5.10 scrub ok 2024-09-21T12:01:00.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:59 smithi175 ceph-mon[32613]: 5.11 scrub starts 2024-09-21T12:01:00.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:00:59 smithi175 ceph-mon[32613]: 5.11 scrub ok 2024-09-21T12:01:00.844 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.3 to 1 (10000) 2024-09-21T12:01:01.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:00 smithi043 ceph-mon[29710]: 3.3 scrub starts 2024-09-21T12:01:01.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:00 smithi043 ceph-mon[29710]: 3.3 scrub ok 2024-09-21T12:01:01.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:00 smithi043 ceph-mon[29710]: pgmap v1035: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 797 B/s rd, 0 op/s 2024-09-21T12:01:01.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:00 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2681148393' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T12:01:01.175 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:00 smithi043 ceph-mon[36454]: 3.3 scrub starts 2024-09-21T12:01:01.175 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:00 smithi043 ceph-mon[36454]: 3.3 scrub ok 2024-09-21T12:01:01.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:00 smithi043 ceph-mon[36454]: pgmap v1035: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 797 B/s rd, 0 op/s 2024-09-21T12:01:01.176 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:00 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2681148393' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T12:01:01.231 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:00 smithi175 ceph-mon[32613]: 3.3 scrub starts 2024-09-21T12:01:01.231 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:00 smithi175 ceph-mon[32613]: 3.3 scrub ok 2024-09-21T12:01:01.231 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:00 smithi175 ceph-mon[32613]: pgmap v1035: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 797 B/s rd, 0 op/s 2024-09-21T12:01:01.231 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:00 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2681148393' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-21T12:01:01.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 3 1 2024-09-21T12:01:01.848 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:02.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[36454]: 6.1a scrub starts 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[36454]: 6.1a scrub ok 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[36454]: 6.15 scrub starts 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[36454]: 6.15 scrub ok 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2681148393' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[36454]: osdmap e491: 8 total, 8 up, 8 in 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[36454]: 6.1e scrub starts 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[36454]: 6.1e scrub ok 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[29710]: 6.1a scrub starts 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[29710]: 6.1a scrub ok 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[29710]: 6.15 scrub starts 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[29710]: 6.15 scrub ok 2024-09-21T12:01:02.137 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2681148393' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T12:01:02.138 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[29710]: osdmap e491: 8 total, 8 up, 8 in 2024-09-21T12:01:02.138 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[29710]: 6.1e scrub starts 2024-09-21T12:01:02.138 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:01 smithi043 ceph-mon[29710]: 6.1e scrub ok 2024-09-21T12:01:02.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:01:02.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:01 smithi175 ceph-mon[32613]: 6.1a scrub starts 2024-09-21T12:01:02.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:01 smithi175 ceph-mon[32613]: 6.1a scrub ok 2024-09-21T12:01:02.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:01 smithi175 ceph-mon[32613]: 6.15 scrub starts 2024-09-21T12:01:02.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:01 smithi175 ceph-mon[32613]: 6.15 scrub ok 2024-09-21T12:01:02.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:01 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2681148393' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-21T12:01:02.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:01 smithi175 ceph-mon[32613]: osdmap e491: 8 total, 8 up, 8 in 2024-09-21T12:01:02.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:01 smithi175 ceph-mon[32613]: 6.1e scrub starts 2024-09-21T12:01:02.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:01 smithi175 ceph-mon[32613]: 6.1e scrub ok 2024-09-21T12:01:03.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: 6.14 scrub starts 2024-09-21T12:01:03.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: 6.14 scrub ok 2024-09-21T12:01:03.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: 3.4 scrub starts 2024-09-21T12:01:03.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: 3.4 scrub ok 2024-09-21T12:01:03.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: 6.b scrub starts 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: 6.b scrub ok 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: pgmap v1037: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/4173734168' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: 6.14 scrub starts 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: 6.14 scrub ok 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: 3.4 scrub starts 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: 3.4 scrub ok 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:03.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: 6.b scrub starts 2024-09-21T12:01:03.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: 6.b scrub ok 2024-09-21T12:01:03.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: pgmap v1037: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-21T12:01:03.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:03.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/4173734168' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T12:01:03.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:02 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: 6.14 scrub starts 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: 6.14 scrub ok 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: 3.4 scrub starts 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: 3.4 scrub ok 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: 6.b scrub starts 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: 6.b scrub ok 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: pgmap v1037: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/4173734168' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T12:01:03.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:02 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-21T12:01:03.570 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-21T12:01:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[36454]: 6.19 scrub starts 2024-09-21T12:01:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[36454]: 6.19 scrub ok 2024-09-21T12:01:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[36454]: 5.4 scrub starts 2024-09-21T12:01:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[36454]: 5.4 scrub ok 2024-09-21T12:01:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[36454]: 6.c scrub starts 2024-09-21T12:01:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[36454]: 6.c scrub ok 2024-09-21T12:01:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[36454]: 5.2 scrub starts 2024-09-21T12:01:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[36454]: 5.2 scrub ok 2024-09-21T12:01:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-21T12:01:04.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[36454]: osdmap e492: 8 total, 8 up, 8 in 2024-09-21T12:01:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[29710]: 6.19 scrub starts 2024-09-21T12:01:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[29710]: 6.19 scrub ok 2024-09-21T12:01:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[29710]: 5.4 scrub starts 2024-09-21T12:01:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[29710]: 5.4 scrub ok 2024-09-21T12:01:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[29710]: 6.c scrub starts 2024-09-21T12:01:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[29710]: 6.c scrub ok 2024-09-21T12:01:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[29710]: 5.2 scrub starts 2024-09-21T12:01:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[29710]: 5.2 scrub ok 2024-09-21T12:01:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-21T12:01:04.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:03 smithi043 ceph-mon[29710]: osdmap e492: 8 total, 8 up, 8 in 2024-09-21T12:01:04.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:03 smithi175 ceph-mon[32613]: 6.19 scrub starts 2024-09-21T12:01:04.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:03 smithi175 ceph-mon[32613]: 6.19 scrub ok 2024-09-21T12:01:04.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:03 smithi175 ceph-mon[32613]: 5.4 scrub starts 2024-09-21T12:01:04.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:03 smithi175 ceph-mon[32613]: 5.4 scrub ok 2024-09-21T12:01:04.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:03 smithi175 ceph-mon[32613]: 6.c scrub starts 2024-09-21T12:01:04.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:03 smithi175 ceph-mon[32613]: 6.c scrub ok 2024-09-21T12:01:04.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:03 smithi175 ceph-mon[32613]: 5.2 scrub starts 2024-09-21T12:01:04.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:03 smithi175 ceph-mon[32613]: 5.2 scrub ok 2024-09-21T12:01:04.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:03 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-21T12:01:04.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:03 smithi175 ceph-mon[32613]: osdmap e492: 8 total, 8 up, 8 in 2024-09-21T12:01:04.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 5 1 2024-09-21T12:01:04.721 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:05.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[29710]: 5.f scrub starts 2024-09-21T12:01:05.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[29710]: 5.f scrub ok 2024-09-21T12:01:05.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[29710]: 6.12 scrub starts 2024-09-21T12:01:05.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[29710]: 6.12 scrub ok 2024-09-21T12:01:05.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[29710]: 3.b scrub starts 2024-09-21T12:01:05.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[29710]: 3.b scrub ok 2024-09-21T12:01:05.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[29710]: pgmap v1039: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:05.121 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[29710]: 4.1f scrub starts 2024-09-21T12:01:05.122 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[29710]: 4.1f scrub ok 2024-09-21T12:01:05.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[36454]: 5.f scrub starts 2024-09-21T12:01:05.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[36454]: 5.f scrub ok 2024-09-21T12:01:05.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[36454]: 6.12 scrub starts 2024-09-21T12:01:05.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[36454]: 6.12 scrub ok 2024-09-21T12:01:05.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[36454]: 3.b scrub starts 2024-09-21T12:01:05.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[36454]: 3.b scrub ok 2024-09-21T12:01:05.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[36454]: pgmap v1039: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:05.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[36454]: 4.1f scrub starts 2024-09-21T12:01:05.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:04 smithi043 ceph-mon[36454]: 4.1f scrub ok 2024-09-21T12:01:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:04 smithi175 ceph-mon[32613]: 5.f scrub starts 2024-09-21T12:01:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:04 smithi175 ceph-mon[32613]: 5.f scrub ok 2024-09-21T12:01:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:04 smithi175 ceph-mon[32613]: 6.12 scrub starts 2024-09-21T12:01:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:04 smithi175 ceph-mon[32613]: 6.12 scrub ok 2024-09-21T12:01:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:04 smithi175 ceph-mon[32613]: 3.b scrub starts 2024-09-21T12:01:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:04 smithi175 ceph-mon[32613]: 3.b scrub ok 2024-09-21T12:01:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:04 smithi175 ceph-mon[32613]: pgmap v1039: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:04 smithi175 ceph-mon[32613]: 4.1f scrub starts 2024-09-21T12:01:05.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:04 smithi175 ceph-mon[32613]: 4.1f scrub ok 2024-09-21T12:01:05.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:01:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.5 to 1 (10000) 2024-09-21T12:01:06.217 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[29710]: 4.15 scrub starts 2024-09-21T12:01:06.217 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[29710]: 4.15 scrub ok 2024-09-21T12:01:06.217 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[29710]: 5.b scrub starts 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[29710]: 5.b scrub ok 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[29710]: 6.0 scrub starts 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[29710]: 6.0 scrub ok 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/187023057' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[36454]: 4.15 scrub starts 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[36454]: 4.15 scrub ok 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[36454]: 5.b scrub starts 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[36454]: 5.b scrub ok 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[36454]: 6.0 scrub starts 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[36454]: 6.0 scrub ok 2024-09-21T12:01:06.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:05 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/187023057' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T12:01:06.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:05 smithi175 ceph-mon[32613]: 4.15 scrub starts 2024-09-21T12:01:06.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:05 smithi175 ceph-mon[32613]: 4.15 scrub ok 2024-09-21T12:01:06.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:05 smithi175 ceph-mon[32613]: 5.b scrub starts 2024-09-21T12:01:06.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:05 smithi175 ceph-mon[32613]: 5.b scrub ok 2024-09-21T12:01:06.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:05 smithi175 ceph-mon[32613]: 6.0 scrub starts 2024-09-21T12:01:06.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:05 smithi175 ceph-mon[32613]: 6.0 scrub ok 2024-09-21T12:01:06.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:05 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/187023057' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-21T12:01:06.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 5 1 2024-09-21T12:01:06.947 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[29710]: 3.14 scrub starts 2024-09-21T12:01:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[29710]: 3.14 scrub ok 2024-09-21T12:01:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/187023057' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T12:01:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[29710]: osdmap e493: 8 total, 8 up, 8 in 2024-09-21T12:01:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[29710]: 5.1c scrub starts 2024-09-21T12:01:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[29710]: 5.1c scrub ok 2024-09-21T12:01:06.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[29710]: pgmap v1041: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[36454]: 3.14 scrub starts 2024-09-21T12:01:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[36454]: 3.14 scrub ok 2024-09-21T12:01:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/187023057' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T12:01:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[36454]: osdmap e493: 8 total, 8 up, 8 in 2024-09-21T12:01:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[36454]: 5.1c scrub starts 2024-09-21T12:01:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[36454]: 5.1c scrub ok 2024-09-21T12:01:06.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:06 smithi043 ceph-mon[36454]: pgmap v1041: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:07.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:06 smithi175 ceph-mon[32613]: 3.14 scrub starts 2024-09-21T12:01:07.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:06 smithi175 ceph-mon[32613]: 3.14 scrub ok 2024-09-21T12:01:07.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:06 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/187023057' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-21T12:01:07.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:06 smithi175 ceph-mon[32613]: osdmap e493: 8 total, 8 up, 8 in 2024-09-21T12:01:07.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:06 smithi175 ceph-mon[32613]: 5.1c scrub starts 2024-09-21T12:01:07.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:06 smithi175 ceph-mon[32613]: 5.1c scrub ok 2024-09-21T12:01:07.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:06 smithi175 ceph-mon[32613]: pgmap v1041: 142 pgs: 142 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-21T12:01:08.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[36454]: 4.19 scrub starts 2024-09-21T12:01:08.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[36454]: 4.19 scrub ok 2024-09-21T12:01:08.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[36454]: 3.10 scrub starts 2024-09-21T12:01:08.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[36454]: 3.10 scrub ok 2024-09-21T12:01:08.218 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/144623791' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-21T12:01:08.219 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-21T12:01:08.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[29710]: 4.19 scrub starts 2024-09-21T12:01:08.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[29710]: 4.19 scrub ok 2024-09-21T12:01:08.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[29710]: 3.10 scrub starts 2024-09-21T12:01:08.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[29710]: 3.10 scrub ok 2024-09-21T12:01:08.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/144623791' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-21T12:01:08.219 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:07 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-21T12:01:08.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:07 smithi175 ceph-mon[32613]: 4.19 scrub starts 2024-09-21T12:01:08.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:07 smithi175 ceph-mon[32613]: 4.19 scrub ok 2024-09-21T12:01:08.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:07 smithi175 ceph-mon[32613]: 3.10 scrub starts 2024-09-21T12:01:08.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:07 smithi175 ceph-mon[32613]: 3.10 scrub ok 2024-09-21T12:01:08.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:07 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/144623791' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-21T12:01:08.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:07 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-21T12:01:08.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 6 1 2024-09-21T12:01:08.884 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:09.119 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:08 smithi043 ceph-mon[29710]: 3.1f scrub starts 2024-09-21T12:01:09.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:08 smithi043 ceph-mon[29710]: 3.1f scrub ok 2024-09-21T12:01:09.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:08 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-21T12:01:09.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:08 smithi043 ceph-mon[29710]: osdmap e494: 8 total, 8 up, 8 in 2024-09-21T12:01:09.150 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:08 smithi043 ceph-mon[29710]: pgmap v1043: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:09.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:08 smithi043 ceph-mon[36454]: 3.1f scrub starts 2024-09-21T12:01:09.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:08 smithi043 ceph-mon[36454]: 3.1f scrub ok 2024-09-21T12:01:09.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:08 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-21T12:01:09.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:08 smithi043 ceph-mon[36454]: osdmap e494: 8 total, 8 up, 8 in 2024-09-21T12:01:09.151 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:08 smithi043 ceph-mon[36454]: pgmap v1043: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:09.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:08 smithi175 ceph-mon[32613]: 3.1f scrub starts 2024-09-21T12:01:09.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:08 smithi175 ceph-mon[32613]: 3.1f scrub ok 2024-09-21T12:01:09.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:08 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-21T12:01:09.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:08 smithi175 ceph-mon[32613]: osdmap e494: 8 total, 8 up, 8 in 2024-09-21T12:01:09.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:08 smithi175 ceph-mon[32613]: pgmap v1043: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:09.886 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.6 to 1 (10000) 2024-09-21T12:01:10.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:09 smithi043 ceph-mon[36454]: 3.1a scrub starts 2024-09-21T12:01:10.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:09 smithi043 ceph-mon[36454]: 3.1a scrub ok 2024-09-21T12:01:10.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:09 smithi043 ceph-mon[36454]: 6.5 scrub starts 2024-09-21T12:01:10.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:09 smithi043 ceph-mon[36454]: 6.5 scrub ok 2024-09-21T12:01:10.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:09 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1130648800' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T12:01:10.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:09 smithi043 ceph-mon[29710]: 3.1a scrub starts 2024-09-21T12:01:10.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:09 smithi043 ceph-mon[29710]: 3.1a scrub ok 2024-09-21T12:01:10.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:09 smithi043 ceph-mon[29710]: 6.5 scrub starts 2024-09-21T12:01:10.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:09 smithi043 ceph-mon[29710]: 6.5 scrub ok 2024-09-21T12:01:10.201 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:09 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1130648800' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T12:01:10.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:09 smithi175 ceph-mon[32613]: 3.1a scrub starts 2024-09-21T12:01:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:09 smithi175 ceph-mon[32613]: 3.1a scrub ok 2024-09-21T12:01:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:09 smithi175 ceph-mon[32613]: 6.5 scrub starts 2024-09-21T12:01:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:09 smithi175 ceph-mon[32613]: 6.5 scrub ok 2024-09-21T12:01:10.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:09 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1130648800' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-21T12:01:10.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 6 1 2024-09-21T12:01:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[29710]: 6.2 scrub starts 2024-09-21T12:01:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[29710]: 6.2 scrub ok 2024-09-21T12:01:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[29710]: 4.12 scrub starts 2024-09-21T12:01:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[29710]: 4.12 scrub ok 2024-09-21T12:01:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1130648800' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T12:01:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[29710]: osdmap e495: 8 total, 8 up, 8 in 2024-09-21T12:01:10.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[29710]: pgmap v1045: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[36454]: 6.2 scrub starts 2024-09-21T12:01:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[36454]: 6.2 scrub ok 2024-09-21T12:01:10.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[36454]: 4.12 scrub starts 2024-09-21T12:01:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[36454]: 4.12 scrub ok 2024-09-21T12:01:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1130648800' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T12:01:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[36454]: osdmap e495: 8 total, 8 up, 8 in 2024-09-21T12:01:10.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:10 smithi043 ceph-mon[36454]: pgmap v1045: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:11.011 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:01:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=cleanup t=2024-09-21T12:01:10.697529038Z level=info msg="Completed cleanup jobs" duration=10.164797ms 2024-09-21T12:01:11.011 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:01:10 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=plugins.update.checker t=2024-09-21T12:01:10.836725861Z level=info msg="Update check succeeded" duration=68.236619ms 2024-09-21T12:01:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:10 smithi175 ceph-mon[32613]: 6.2 scrub starts 2024-09-21T12:01:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:10 smithi175 ceph-mon[32613]: 6.2 scrub ok 2024-09-21T12:01:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:10 smithi175 ceph-mon[32613]: 4.12 scrub starts 2024-09-21T12:01:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:10 smithi175 ceph-mon[32613]: 4.12 scrub ok 2024-09-21T12:01:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:10 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1130648800' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-21T12:01:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:10 smithi175 ceph-mon[32613]: osdmap e495: 8 total, 8 up, 8 in 2024-09-21T12:01:11.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:10 smithi175 ceph-mon[32613]: pgmap v1045: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:11.903 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-21T12:01:12.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:11 smithi043 ceph-mon[36454]: 4.17 scrub starts 2024-09-21T12:01:12.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:11 smithi043 ceph-mon[36454]: 4.17 scrub ok 2024-09-21T12:01:12.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:11 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3839471061' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-21T12:01:12.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:11 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-21T12:01:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:11 smithi043 ceph-mon[29710]: 4.17 scrub starts 2024-09-21T12:01:12.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:11 smithi043 ceph-mon[29710]: 4.17 scrub ok 2024-09-21T12:01:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:11 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3839471061' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-21T12:01:12.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:11 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-21T12:01:12.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:11 smithi175 ceph-mon[32613]: 4.17 scrub starts 2024-09-21T12:01:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:11 smithi175 ceph-mon[32613]: 4.17 scrub ok 2024-09-21T12:01:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:11 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3839471061' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-21T12:01:12.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:11 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-21T12:01:12.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:01:12.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 7 1 2024-09-21T12:01:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:13.209 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: 3.16 scrub starts 2024-09-21T12:01:13.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: 3.16 scrub ok 2024-09-21T12:01:13.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: 4.1c scrub starts 2024-09-21T12:01:13.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: 4.1c scrub ok 2024-09-21T12:01:13.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: 6.3 scrub starts 2024-09-21T12:01:13.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: 6.3 scrub ok 2024-09-21T12:01:13.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-21T12:01:13.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: osdmap e496: 8 total, 8 up, 8 in 2024-09-21T12:01:13.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:13.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: pgmap v1047: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:01:13.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[29710]: osdmap e497: 8 total, 8 up, 8 in 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: 3.16 scrub starts 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: 3.16 scrub ok 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: 4.1c scrub starts 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: 4.1c scrub ok 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: 6.3 scrub starts 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: 6.3 scrub ok 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: osdmap e496: 8 total, 8 up, 8 in 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: pgmap v1047: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:01:13.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:12 smithi043 ceph-mon[36454]: osdmap e497: 8 total, 8 up, 8 in 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: 3.16 scrub starts 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: 3.16 scrub ok 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: 4.1c scrub starts 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: 4.1c scrub ok 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: 6.3 scrub starts 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: 6.3 scrub ok 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: osdmap e496: 8 total, 8 up, 8 in 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: pgmap v1047: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:01:13.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:12 smithi175 ceph-mon[32613]: osdmap e497: 8 total, 8 up, 8 in 2024-09-21T12:01:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.7 to 1 (10000) 2024-09-21T12:01:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[36454]: 5.15 scrub starts 2024-09-21T12:01:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[36454]: 5.15 scrub ok 2024-09-21T12:01:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[36454]: 4.1b scrub starts 2024-09-21T12:01:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[36454]: 4.1b scrub ok 2024-09-21T12:01:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[36454]: 4.18 scrub starts 2024-09-21T12:01:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[36454]: 4.18 scrub ok 2024-09-21T12:01:14.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1125425028' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T12:01:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[29710]: 5.15 scrub starts 2024-09-21T12:01:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[29710]: 5.15 scrub ok 2024-09-21T12:01:14.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[29710]: 4.1b scrub starts 2024-09-21T12:01:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[29710]: 4.1b scrub ok 2024-09-21T12:01:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[29710]: 4.18 scrub starts 2024-09-21T12:01:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[29710]: 4.18 scrub ok 2024-09-21T12:01:14.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:13 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1125425028' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T12:01:14.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:13 smithi175 ceph-mon[32613]: 5.15 scrub starts 2024-09-21T12:01:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:13 smithi175 ceph-mon[32613]: 5.15 scrub ok 2024-09-21T12:01:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:13 smithi175 ceph-mon[32613]: 4.1b scrub starts 2024-09-21T12:01:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:13 smithi175 ceph-mon[32613]: 4.1b scrub ok 2024-09-21T12:01:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:13 smithi175 ceph-mon[32613]: 4.18 scrub starts 2024-09-21T12:01:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:13 smithi175 ceph-mon[32613]: 4.18 scrub ok 2024-09-21T12:01:14.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:13 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1125425028' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-21T12:01:14.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 7 1 2024-09-21T12:01:15.132 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:15.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:14 smithi175 ceph-mon[32613]: 6.f scrub starts 2024-09-21T12:01:15.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:14 smithi175 ceph-mon[32613]: 6.f scrub ok 2024-09-21T12:01:15.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:14 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1125425028' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T12:01:15.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:14 smithi175 ceph-mon[32613]: osdmap e498: 8 total, 8 up, 8 in 2024-09-21T12:01:15.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:14 smithi175 ceph-mon[32613]: pgmap v1050: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:01:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:14 smithi043 ceph-mon[36454]: 6.f scrub starts 2024-09-21T12:01:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:14 smithi043 ceph-mon[36454]: 6.f scrub ok 2024-09-21T12:01:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:14 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1125425028' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T12:01:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:14 smithi043 ceph-mon[36454]: osdmap e498: 8 total, 8 up, 8 in 2024-09-21T12:01:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:14 smithi043 ceph-mon[36454]: pgmap v1050: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:01:15.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:14 smithi043 ceph-mon[29710]: 6.f scrub starts 2024-09-21T12:01:15.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:14 smithi043 ceph-mon[29710]: 6.f scrub ok 2024-09-21T12:01:15.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:14 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1125425028' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-21T12:01:15.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:14 smithi043 ceph-mon[29710]: osdmap e498: 8 total, 8 up, 8 in 2024-09-21T12:01:15.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:14 smithi043 ceph-mon[29710]: pgmap v1050: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-21T12:01:15.491 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:01:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:15.924 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-21T12:01:16.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-mon[36454]: 1.5 scrub starts 2024-09-21T12:01:16.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-mon[36454]: 1.5 scrub ok 2024-09-21T12:01:16.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-mon[36454]: 4.6 scrub starts 2024-09-21T12:01:16.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-mon[36454]: 4.6 scrub ok 2024-09-21T12:01:16.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2911068912' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-21T12:01:16.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-mon[29710]: 1.5 scrub starts 2024-09-21T12:01:16.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-mon[29710]: 1.5 scrub ok 2024-09-21T12:01:16.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-mon[29710]: 4.6 scrub starts 2024-09-21T12:01:16.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-mon[29710]: 4.6 scrub ok 2024-09-21T12:01:16.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:15 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2911068912' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-21T12:01:16.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:15 smithi175 ceph-mon[32613]: 1.5 scrub starts 2024-09-21T12:01:16.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:15 smithi175 ceph-mon[32613]: 1.5 scrub ok 2024-09-21T12:01:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:15 smithi175 ceph-mon[32613]: 4.6 scrub starts 2024-09-21T12:01:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:15 smithi175 ceph-mon[32613]: 4.6 scrub ok 2024-09-21T12:01:16.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:15 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2911068912' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-21T12:01:16.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd reweight 4 1 2024-09-21T12:01:16.932 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:17.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:16 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2911068912' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-21T12:01:17.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:16 smithi043 ceph-mon[36454]: osdmap e499: 8 total, 8 up, 8 in 2024-09-21T12:01:17.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:16 smithi043 ceph-mon[36454]: 3.7 scrub starts 2024-09-21T12:01:17.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:16 smithi043 ceph-mon[36454]: 3.7 scrub ok 2024-09-21T12:01:17.194 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:16 smithi043 ceph-mon[36454]: pgmap v1052: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail 2024-09-21T12:01:17.194 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:16 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2911068912' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-21T12:01:17.194 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:16 smithi043 ceph-mon[29710]: osdmap e499: 8 total, 8 up, 8 in 2024-09-21T12:01:17.194 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:16 smithi043 ceph-mon[29710]: 3.7 scrub starts 2024-09-21T12:01:17.194 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:16 smithi043 ceph-mon[29710]: 3.7 scrub ok 2024-09-21T12:01:17.194 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:16 smithi043 ceph-mon[29710]: pgmap v1052: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail 2024-09-21T12:01:17.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:16 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2911068912' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-21T12:01:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:16 smithi175 ceph-mon[32613]: osdmap e499: 8 total, 8 up, 8 in 2024-09-21T12:01:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:16 smithi175 ceph-mon[32613]: 3.7 scrub starts 2024-09-21T12:01:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:16 smithi175 ceph-mon[32613]: 3.7 scrub ok 2024-09-21T12:01:17.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:16 smithi175 ceph-mon[32613]: pgmap v1052: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail 2024-09-21T12:01:18.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[36454]: 4.1 scrub starts 2024-09-21T12:01:18.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[36454]: 4.1 scrub ok 2024-09-21T12:01:18.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[36454]: 2.0 scrub starts 2024-09-21T12:01:18.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[36454]: 2.0 scrub ok 2024-09-21T12:01:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:18.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3688983303' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T12:01:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[29710]: 4.1 scrub starts 2024-09-21T12:01:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[29710]: 4.1 scrub ok 2024-09-21T12:01:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[29710]: 2.0 scrub starts 2024-09-21T12:01:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[29710]: 2.0 scrub ok 2024-09-21T12:01:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:18.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:17 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3688983303' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T12:01:18.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:17 smithi175 ceph-mon[32613]: 4.1 scrub starts 2024-09-21T12:01:18.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:17 smithi175 ceph-mon[32613]: 4.1 scrub ok 2024-09-21T12:01:18.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:17 smithi175 ceph-mon[32613]: 2.0 scrub starts 2024-09-21T12:01:18.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:17 smithi175 ceph-mon[32613]: 2.0 scrub ok 2024-09-21T12:01:18.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:17 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:18.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:17 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3688983303' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-21T12:01:18.569 INFO:teuthology.orchestra.run.smithi043.stderr:reweighted osd.4 to 1 (10000) 2024-09-21T12:01:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[36454]: 3.1c scrub starts 2024-09-21T12:01:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[36454]: 3.1c scrub ok 2024-09-21T12:01:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[36454]: 4.d scrub starts 2024-09-21T12:01:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[36454]: 4.d scrub ok 2024-09-21T12:01:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[36454]: 3.f scrub starts 2024-09-21T12:01:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[36454]: 3.f scrub ok 2024-09-21T12:01:19.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[36454]: pgmap v1053: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3688983303' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[36454]: osdmap e500: 8 total, 8 up, 8 in 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[29710]: 3.1c scrub starts 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[29710]: 3.1c scrub ok 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[29710]: 4.d scrub starts 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[29710]: 4.d scrub ok 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[29710]: 3.f scrub starts 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[29710]: 3.f scrub ok 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[29710]: pgmap v1053: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3688983303' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T12:01:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:18 smithi043 ceph-mon[29710]: osdmap e500: 8 total, 8 up, 8 in 2024-09-21T12:01:19.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:18 smithi175 ceph-mon[32613]: 3.1c scrub starts 2024-09-21T12:01:19.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:18 smithi175 ceph-mon[32613]: 3.1c scrub ok 2024-09-21T12:01:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:18 smithi175 ceph-mon[32613]: 4.d scrub starts 2024-09-21T12:01:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:18 smithi175 ceph-mon[32613]: 4.d scrub ok 2024-09-21T12:01:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:18 smithi175 ceph-mon[32613]: 3.f scrub starts 2024-09-21T12:01:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:18 smithi175 ceph-mon[32613]: 3.f scrub ok 2024-09-21T12:01:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:18 smithi175 ceph-mon[32613]: pgmap v1053: 142 pgs: 142 active+clean; 583 KiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:18 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3688983303' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-21T12:01:19.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:18 smithi175 ceph-mon[32613]: osdmap e500: 8 total, 8 up, 8 in 2024-09-21T12:01:19.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd primary-affinity 4 1 2024-09-21T12:01:19.582 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:19.965 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:19 smithi043 ceph-mon[36454]: 5.1f scrub starts 2024-09-21T12:01:19.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:19 smithi043 ceph-mon[36454]: 5.1f scrub ok 2024-09-21T12:01:19.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:19 smithi043 ceph-mon[29710]: 5.1f scrub starts 2024-09-21T12:01:19.966 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:19 smithi043 ceph-mon[29710]: 5.1f scrub ok 2024-09-21T12:01:20.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:19 smithi175 ceph-mon[32613]: 5.1f scrub starts 2024-09-21T12:01:20.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:19 smithi175 ceph-mon[32613]: 5.1f scrub ok 2024-09-21T12:01:20.946 INFO:teuthology.orchestra.run.smithi043.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-21T12:01:21.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:20 smithi043 ceph-mon[29710]: 3.5 scrub starts 2024-09-21T12:01:21.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:20 smithi043 ceph-mon[29710]: 3.5 scrub ok 2024-09-21T12:01:21.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:20 smithi043 ceph-mon[29710]: pgmap v1055: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 809 B/s rd, 0 op/s 2024-09-21T12:01:21.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:20 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1396076807' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-21T12:01:21.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:20 smithi043 ceph-mon[36454]: 3.5 scrub starts 2024-09-21T12:01:21.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:20 smithi043 ceph-mon[36454]: 3.5 scrub ok 2024-09-21T12:01:21.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:20 smithi043 ceph-mon[36454]: pgmap v1055: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 809 B/s rd, 0 op/s 2024-09-21T12:01:21.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:20 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1396076807' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-21T12:01:21.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:20 smithi175 ceph-mon[32613]: 3.5 scrub starts 2024-09-21T12:01:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:20 smithi175 ceph-mon[32613]: 3.5 scrub ok 2024-09-21T12:01:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:20 smithi175 ceph-mon[32613]: pgmap v1055: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 809 B/s rd, 0 op/s 2024-09-21T12:01:21.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:20 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1396076807' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-21T12:01:21.766 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-21T12:01:21.766 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-21T12:01:21.766 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-21T12:01:21.767 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-21T12:01:21.767 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T12:01:21.767 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump --format=json 2024-09-21T12:01:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:21 smithi043 ceph-mon[29710]: 6.1f scrub starts 2024-09-21T12:01:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:21 smithi043 ceph-mon[29710]: 6.1f scrub ok 2024-09-21T12:01:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:21 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1396076807' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-21T12:01:21.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:21 smithi043 ceph-mon[29710]: osdmap e501: 8 total, 8 up, 8 in 2024-09-21T12:01:21.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:21 smithi043 ceph-mon[36454]: 6.1f scrub starts 2024-09-21T12:01:21.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:21 smithi043 ceph-mon[36454]: 6.1f scrub ok 2024-09-21T12:01:21.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:21 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1396076807' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-21T12:01:21.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:21 smithi043 ceph-mon[36454]: osdmap e501: 8 total, 8 up, 8 in 2024-09-21T12:01:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:21 smithi175 ceph-mon[32613]: 6.1f scrub starts 2024-09-21T12:01:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:21 smithi175 ceph-mon[32613]: 6.1f scrub ok 2024-09-21T12:01:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:21 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1396076807' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-21T12:01:22.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:21 smithi175 ceph-mon[32613]: osdmap e501: 8 total, 8 up, 8 in 2024-09-21T12:01:22.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:01:22.720 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T12:01:22.720 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":503,"fsid":"4f2c5252-780d-11ef-baf6-efdc52797490","created":"2024-09-21T11:34:07.249195+0000","modified":"2024-09-21T12:01:22.565437+0000","last_up_change":"2024-09-21T12:00:39.741407+0000","last_in_change":"2024-09-21T11:47:02.337266+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":19,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T11:37:26.023430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"484","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"414","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.1800000667572021,"score_stable":2.1800000667572021,"optimal_score":1,"raw_score_acting":2.1800000667572021,"raw_score_stable":2.1800000667572021,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-21T11:39:58.262982+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-21T11:39:59.229530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-21T11:40:00.380059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-21T11:40:02.330041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-21T11:40:04.395413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5229029d-4fd6-46b6-8325-a3e004df09b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":817279193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":817279193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":817279193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6803","nonce":817279193}]},"public_addr":"172.21.15.43:6801/817279193","cluster_addr":"172.21.15.43:6802/817279193","heartbeat_back_addr":"172.21.15.43:6804/817279193","heartbeat_front_addr":"172.21.15.43:6803/817279193","state":["exists","up"]},{"osd":1,"uuid":"a2a7b58b-ba99-4383-a6d3-3b19388b2607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":263756171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":263756171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":263756171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6807","nonce":263756171}]},"public_addr":"172.21.15.43:6805/263756171","cluster_addr":"172.21.15.43:6806/263756171","heartbeat_back_addr":"172.21.15.43:6808/263756171","heartbeat_front_addr":"172.21.15.43:6807/263756171","state":["exists","up"]},{"osd":2,"uuid":"c24a60b5-5713-492b-82ee-576ea7a743ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6809","nonce":687577863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":687577863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":687577863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":687577863}]},"public_addr":"172.21.15.43:6809/687577863","cluster_addr":"172.21.15.43:6810/687577863","heartbeat_back_addr":"172.21.15.43:6812/687577863","heartbeat_front_addr":"172.21.15.43:6811/687577863","state":["exists","up"]},{"osd":3,"uuid":"c5d43cf8-1c72-4d7d-ae41-e8136b47177d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":129493158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":129493158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":129493158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":129493158}]},"public_addr":"172.21.15.43:6813/129493158","cluster_addr":"172.21.15.43:6814/129493158","heartbeat_back_addr":"172.21.15.43:6816/129493158","heartbeat_front_addr":"172.21.15.43:6815/129493158","state":["exists","up"]},{"osd":4,"uuid":"79ee85c5-afdc-4d68-a253-89b7b0eb652c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":33,"last_clean_end":458,"up_from":481,"up_thru":501,"down_at":459,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1102868557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1102868557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1102868557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1102868557}]},"public_addr":"172.21.15.175:6800/1102868557","cluster_addr":"172.21.15.175:6801/1102868557","heartbeat_back_addr":"172.21.15.175:6803/1102868557","heartbeat_front_addr":"172.21.15.175:6802/1102868557","state":["exists","up"]},{"osd":5,"uuid":"4d647321-71a1-412b-bbf5-921441923a65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1163208458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":1163208458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":1163208458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1163208458}]},"public_addr":"172.21.15.175:6804/1163208458","cluster_addr":"172.21.15.175:6805/1163208458","heartbeat_back_addr":"172.21.15.175:6807/1163208458","heartbeat_front_addr":"172.21.15.175:6806/1163208458","state":["exists","up"]},{"osd":6,"uuid":"3c33bea7-94b5-46c1-9905-81b4612a2354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":932395625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":932395625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":932395625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":932395625}]},"public_addr":"172.21.15.175:6808/932395625","cluster_addr":"172.21.15.175:6809/932395625","heartbeat_back_addr":"172.21.15.175:6811/932395625","heartbeat_front_addr":"172.21.15.175:6810/932395625","state":["exists","up"]},{"osd":7,"uuid":"41bc9817-d9d1-4c11-94ff-7de332af6d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3676713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":3676713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":3676713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3676713275}]},"public_addr":"172.21.15.175:6812/3676713275","cluster_addr":"172.21.15.175:6813/3676713275","heartbeat_back_addr":"172.21.15.175:6815/3676713275","heartbeat_front_addr":"172.21.15.175:6814/3676713275","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:55.791104+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:01:22.148227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:01:21.466933+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:55.748325+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:38.841313+0000","dead_epoch":458},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:50.895990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:00.944660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:54.936095+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"6.2","osds":[2,4,5]},{"pgid":"6.1c","osds":[0,7,2]}],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":3,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]},{"pgid":"6.3","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2419845767":"2024-09-22T11:40:29.670859+0000","172.21.15.43:0/3249046048":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/2518306973":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/3115176321":"2024-09-22T11:35:03.554546+0000","172.21.15.43:6800/2583179759":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/4225358170":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/347827280":"2024-09-22T11:40:29.670859+0000","172.21.15.43:6800/1513342881":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/1113684867":"2024-09-22T11:40:29.670859+0000","172.21.15.43:0/3345448895":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/2454875216":"2024-09-22T11:40:29.670859+0000","172.21.15.43:0/911630798":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/983028036":"2024-09-22T11:40:29.670859+0000","172.21.15.43:6800/4145025121":"2024-09-22T11:40:29.670859+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T12:01:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: 4.11 scrub starts 2024-09-21T12:01:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: 4.11 scrub ok 2024-09-21T12:01:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: 5.8 scrub starts 2024-09-21T12:01:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: 5.8 scrub ok 2024-09-21T12:01:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:01:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:01:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: 6.9 scrub starts 2024-09-21T12:01:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: 6.9 scrub ok 2024-09-21T12:01:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: 3.11 scrub starts 2024-09-21T12:01:22.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: 3.11 scrub ok 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: osdmap e502: 8 total, 8 up, 8 in 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: 4.14 scrub starts 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: 4.14 scrub ok 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: pgmap v1058: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: osdmap e503: 8 total, 8 up, 8 in 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3248361890' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: 4.11 scrub starts 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: 4.11 scrub ok 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: 5.8 scrub starts 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: 5.8 scrub ok 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:01:22.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: 6.9 scrub starts 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: 6.9 scrub ok 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: 3.11 scrub starts 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: 3.11 scrub ok 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: osdmap e502: 8 total, 8 up, 8 in 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: 4.14 scrub starts 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: 4.14 scrub ok 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: pgmap v1058: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: osdmap e503: 8 total, 8 up, 8 in 2024-09-21T12:01:22.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:22 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3248361890' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: 4.11 scrub starts 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: 4.11 scrub ok 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: 5.8 scrub starts 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: 5.8 scrub ok 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: 6.9 scrub starts 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: 6.9 scrub ok 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: 3.11 scrub starts 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: 3.11 scrub ok 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: osdmap e502: 8 total, 8 up, 8 in 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: 4.14 scrub starts 2024-09-21T12:01:23.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: 4.14 scrub ok 2024-09-21T12:01:23.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: pgmap v1058: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:01:23.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: osdmap e503: 8 total, 8 up, 8 in 2024-09-21T12:01:23.262 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:22 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3248361890' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T12:01:23.500 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T12:01:23.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd dump --format=json 2024-09-21T12:01:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:24.199 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:01:24.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:01:24.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[29710]: 5.d scrub starts 2024-09-21T12:01:24.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[29710]: 5.d scrub ok 2024-09-21T12:01:24.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[29710]: 3.1b scrub starts 2024-09-21T12:01:24.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[29710]: 3.1b scrub ok 2024-09-21T12:01:24.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:01:24.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:01:24.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[36454]: 5.d scrub starts 2024-09-21T12:01:24.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[36454]: 5.d scrub ok 2024-09-21T12:01:24.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[36454]: 3.1b scrub starts 2024-09-21T12:01:24.201 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:23 smithi043 ceph-mon[36454]: 3.1b scrub ok 2024-09-21T12:01:24.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:23 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:01:24.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:23 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:01:24.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:23 smithi175 ceph-mon[32613]: 5.d scrub starts 2024-09-21T12:01:24.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:23 smithi175 ceph-mon[32613]: 5.d scrub ok 2024-09-21T12:01:24.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:23 smithi175 ceph-mon[32613]: 3.1b scrub starts 2024-09-21T12:01:24.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:23 smithi175 ceph-mon[32613]: 3.1b scrub ok 2024-09-21T12:01:24.400 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T12:01:24.400 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":503,"fsid":"4f2c5252-780d-11ef-baf6-efdc52797490","created":"2024-09-21T11:34:07.249195+0000","modified":"2024-09-21T12:01:22.565437+0000","last_up_change":"2024-09-21T12:00:39.741407+0000","last_in_change":"2024-09-21T11:47:02.337266+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":19,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T11:37:26.023430+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":11,"pg_placement_num":11,"pg_placement_num_target":11,"pg_num_target":11,"pg_num_pending":11,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"484","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"414","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.1800000667572021,"score_stable":2.1800000667572021,"optimal_score":1,"raw_score_acting":2.1800000667572021,"raw_score_stable":2.1800000667572021,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-21T11:39:58.262982+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-21T11:39:59.229530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-21T11:40:00.380059+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-21T11:40:02.330041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-21T11:40:04.395413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5229029d-4fd6-46b6-8325-a3e004df09b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":817279193}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":817279193}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":817279193}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6803","nonce":817279193}]},"public_addr":"172.21.15.43:6801/817279193","cluster_addr":"172.21.15.43:6802/817279193","heartbeat_back_addr":"172.21.15.43:6804/817279193","heartbeat_front_addr":"172.21.15.43:6803/817279193","state":["exists","up"]},{"osd":1,"uuid":"a2a7b58b-ba99-4383-a6d3-3b19388b2607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":263756171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":263756171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":263756171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6807","nonce":263756171}]},"public_addr":"172.21.15.43:6805/263756171","cluster_addr":"172.21.15.43:6806/263756171","heartbeat_back_addr":"172.21.15.43:6808/263756171","heartbeat_front_addr":"172.21.15.43:6807/263756171","state":["exists","up"]},{"osd":2,"uuid":"c24a60b5-5713-492b-82ee-576ea7a743ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6809","nonce":687577863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":687577863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":687577863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":687577863}]},"public_addr":"172.21.15.43:6809/687577863","cluster_addr":"172.21.15.43:6810/687577863","heartbeat_back_addr":"172.21.15.43:6812/687577863","heartbeat_front_addr":"172.21.15.43:6811/687577863","state":["exists","up"]},{"osd":3,"uuid":"c5d43cf8-1c72-4d7d-ae41-e8136b47177d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6813","nonce":129493158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":129493158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":129493158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6815","nonce":129493158}]},"public_addr":"172.21.15.43:6813/129493158","cluster_addr":"172.21.15.43:6814/129493158","heartbeat_back_addr":"172.21.15.43:6816/129493158","heartbeat_front_addr":"172.21.15.43:6815/129493158","state":["exists","up"]},{"osd":4,"uuid":"79ee85c5-afdc-4d68-a253-89b7b0eb652c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":33,"last_clean_end":458,"up_from":481,"up_thru":501,"down_at":459,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1102868557}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1102868557}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6803","nonce":1102868557}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1102868557}]},"public_addr":"172.21.15.175:6800/1102868557","cluster_addr":"172.21.15.175:6801/1102868557","heartbeat_back_addr":"172.21.15.175:6803/1102868557","heartbeat_front_addr":"172.21.15.175:6802/1102868557","state":["exists","up"]},{"osd":5,"uuid":"4d647321-71a1-412b-bbf5-921441923a65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1163208458}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6805","nonce":1163208458}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6807","nonce":1163208458}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1163208458}]},"public_addr":"172.21.15.175:6804/1163208458","cluster_addr":"172.21.15.175:6805/1163208458","heartbeat_back_addr":"172.21.15.175:6807/1163208458","heartbeat_front_addr":"172.21.15.175:6806/1163208458","state":["exists","up"]},{"osd":6,"uuid":"3c33bea7-94b5-46c1-9905-81b4612a2354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":932395625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6809","nonce":932395625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6811","nonce":932395625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":932395625}]},"public_addr":"172.21.15.175:6808/932395625","cluster_addr":"172.21.15.175:6809/932395625","heartbeat_back_addr":"172.21.15.175:6811/932395625","heartbeat_front_addr":"172.21.15.175:6810/932395625","state":["exists","up"]},{"osd":7,"uuid":"41bc9817-d9d1-4c11-94ff-7de332af6d22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":481,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":3676713275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6813","nonce":3676713275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6815","nonce":3676713275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":3676713275}]},"public_addr":"172.21.15.175:6812/3676713275","cluster_addr":"172.21.15.175:6813/3676713275","heartbeat_back_addr":"172.21.15.175:6815/3676713275","heartbeat_front_addr":"172.21.15.175:6814/3676713275","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:55.791104+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:01:22.148227+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:01:21.466933+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:55.748325+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:38.841313+0000","dead_epoch":458},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:50.895990+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:00.944660+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T12:00:54.936095+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"6.2","osds":[2,4,5]},{"pgid":"6.1c","osds":[0,7,2]}],"pg_upmap_items":[{"pgid":"4.4","mappings":[{"from":3,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]},{"pgid":"6.3","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2419845767":"2024-09-22T11:40:29.670859+0000","172.21.15.43:0/3249046048":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/2518306973":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/3115176321":"2024-09-22T11:35:03.554546+0000","172.21.15.43:6800/2583179759":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/4225358170":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/347827280":"2024-09-22T11:40:29.670859+0000","172.21.15.43:6800/1513342881":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/1113684867":"2024-09-22T11:40:29.670859+0000","172.21.15.43:0/3345448895":"2024-09-22T11:35:03.554546+0000","172.21.15.43:0/2454875216":"2024-09-22T11:40:29.670859+0000","172.21.15.43:0/911630798":"2024-09-22T11:34:37.015137+0000","172.21.15.43:0/983028036":"2024-09-22T11:40:29.670859+0000","172.21.15.43:6800/4145025121":"2024-09-22T11:40:29.670859+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T12:01:25.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T12:01:25.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T12:01:25.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T12:01:25.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T12:01:25.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T12:01:25.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T12:01:25.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T12:01:25.045 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T12:01:25.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:24 smithi043 ceph-mon[29710]: pgmap v1060: 142 pgs: 142 active+clean; 635 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 5.8 KiB/s rd, 26 KiB/s wr, 5 op/s; 9.2 KiB/s, 0 objects/s recovering 2024-09-21T12:01:25.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:24 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2419627884' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T12:01:25.081 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:24 smithi043 ceph-mon[36454]: pgmap v1060: 142 pgs: 142 active+clean; 635 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 5.8 KiB/s rd, 26 KiB/s wr, 5 op/s; 9.2 KiB/s, 0 objects/s recovering 2024-09-21T12:01:25.081 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:24 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2419627884' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T12:01:25.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:24 smithi175 ceph-mon[32613]: pgmap v1060: 142 pgs: 142 active+clean; 635 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 5.8 KiB/s rd, 26 KiB/s wr, 5 op/s; 9.2 KiB/s, 0 objects/s recovering 2024-09-21T12:01:25.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:24 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2419627884' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T12:01:25.488 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:25.490 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:01:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:25.545 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:26.178 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[36454]: 3.2 scrub starts 2024-09-21T12:01:26.178 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[36454]: 3.2 scrub ok 2024-09-21T12:01:26.178 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[36454]: 1.5 scrub starts 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[36454]: 1.5 scrub ok 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[36454]: 1.6 scrub starts 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[36454]: 1.6 scrub ok 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[36454]: 4.16 scrub starts 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[36454]: 4.16 scrub ok 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[29710]: 3.2 scrub starts 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[29710]: 3.2 scrub ok 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[29710]: 1.5 scrub starts 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[29710]: 1.5 scrub ok 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[29710]: 1.6 scrub starts 2024-09-21T12:01:26.179 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[29710]: 1.6 scrub ok 2024-09-21T12:01:26.180 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[29710]: 4.16 scrub starts 2024-09-21T12:01:26.180 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:25 smithi043 ceph-mon[29710]: 4.16 scrub ok 2024-09-21T12:01:26.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:25 smithi175 ceph-mon[32613]: 3.2 scrub starts 2024-09-21T12:01:26.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:25 smithi175 ceph-mon[32613]: 3.2 scrub ok 2024-09-21T12:01:26.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:25 smithi175 ceph-mon[32613]: 1.5 scrub starts 2024-09-21T12:01:26.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:25 smithi175 ceph-mon[32613]: 1.5 scrub ok 2024-09-21T12:01:26.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:25 smithi175 ceph-mon[32613]: 1.6 scrub starts 2024-09-21T12:01:26.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:25 smithi175 ceph-mon[32613]: 1.6 scrub ok 2024-09-21T12:01:26.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:25 smithi175 ceph-mon[32613]: 4.16 scrub starts 2024-09-21T12:01:26.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:25 smithi175 ceph-mon[32613]: 4.16 scrub ok 2024-09-21T12:01:27.003 INFO:teuthology.orchestra.run.smithi043.stdout:2065879269387 2024-09-21T12:01:27.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T12:01:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:27.027 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:27.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:26 smithi043 ceph-mon[29710]: purged_snaps scrub starts 2024-09-21T12:01:27.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:26 smithi043 ceph-mon[29710]: purged_snaps scrub ok 2024-09-21T12:01:27.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:26 smithi043 ceph-mon[29710]: pgmap v1061: 142 pgs: 142 active+clean; 635 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 5.5 KiB/s rd, 25 KiB/s wr, 5 op/s; 8.7 KiB/s, 0 objects/s recovering 2024-09-21T12:01:27.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:26 smithi043 ceph-mon[36454]: purged_snaps scrub starts 2024-09-21T12:01:27.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:26 smithi043 ceph-mon[36454]: purged_snaps scrub ok 2024-09-21T12:01:27.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:26 smithi043 ceph-mon[36454]: pgmap v1061: 142 pgs: 142 active+clean; 635 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 5.5 KiB/s rd, 25 KiB/s wr, 5 op/s; 8.7 KiB/s, 0 objects/s recovering 2024-09-21T12:01:27.253 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:27.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:26 smithi175 ceph-mon[32613]: purged_snaps scrub starts 2024-09-21T12:01:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:26 smithi175 ceph-mon[32613]: purged_snaps scrub ok 2024-09-21T12:01:27.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:26 smithi175 ceph-mon[32613]: pgmap v1061: 142 pgs: 142 active+clean; 635 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 5.5 KiB/s rd, 25 KiB/s wr, 5 op/s; 8.7 KiB/s, 0 objects/s recovering 2024-09-21T12:01:27.670 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:28.101 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[36454]: 5.a scrub starts 2024-09-21T12:01:28.101 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[36454]: 5.a scrub ok 2024-09-21T12:01:28.102 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[36454]: osdmap e504: 8 total, 8 up, 8 in 2024-09-21T12:01:28.102 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[36454]: 3.8 scrub starts 2024-09-21T12:01:28.102 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[36454]: 3.8 scrub ok 2024-09-21T12:01:28.102 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[36454]: osdmap e505: 8 total, 8 up, 8 in 2024-09-21T12:01:28.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[29710]: 5.a scrub starts 2024-09-21T12:01:28.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[29710]: 5.a scrub ok 2024-09-21T12:01:28.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[29710]: osdmap e504: 8 total, 8 up, 8 in 2024-09-21T12:01:28.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[29710]: 3.8 scrub starts 2024-09-21T12:01:28.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[29710]: 3.8 scrub ok 2024-09-21T12:01:28.102 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:27 smithi043 ceph-mon[29710]: osdmap e505: 8 total, 8 up, 8 in 2024-09-21T12:01:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:27 smithi175 ceph-mon[32613]: 5.a scrub starts 2024-09-21T12:01:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:27 smithi175 ceph-mon[32613]: 5.a scrub ok 2024-09-21T12:01:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:27 smithi175 ceph-mon[32613]: osdmap e504: 8 total, 8 up, 8 in 2024-09-21T12:01:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:27 smithi175 ceph-mon[32613]: 3.8 scrub starts 2024-09-21T12:01:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:27 smithi175 ceph-mon[32613]: 3.8 scrub ok 2024-09-21T12:01:28.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:27 smithi175 ceph-mon[32613]: osdmap e505: 8 total, 8 up, 8 in 2024-09-21T12:01:28.312 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[36454]: 6.d scrub starts 2024-09-21T12:01:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[36454]: 6.d scrub ok 2024-09-21T12:01:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[36454]: 4.4 scrub starts 2024-09-21T12:01:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[36454]: 4.4 scrub ok 2024-09-21T12:01:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[36454]: 3.18 scrub starts 2024-09-21T12:01:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[36454]: 3.18 scrub ok 2024-09-21T12:01:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[36454]: pgmap v1064: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-09-21T12:01:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[36454]: 4.1a scrub starts 2024-09-21T12:01:29.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[36454]: 4.1a scrub ok 2024-09-21T12:01:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[29710]: 6.d scrub starts 2024-09-21T12:01:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[29710]: 6.d scrub ok 2024-09-21T12:01:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[29710]: 4.4 scrub starts 2024-09-21T12:01:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[29710]: 4.4 scrub ok 2024-09-21T12:01:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[29710]: 3.18 scrub starts 2024-09-21T12:01:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[29710]: 3.18 scrub ok 2024-09-21T12:01:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[29710]: pgmap v1064: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-09-21T12:01:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[29710]: 4.1a scrub starts 2024-09-21T12:01:29.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:28 smithi043 ceph-mon[29710]: 4.1a scrub ok 2024-09-21T12:01:29.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:28 smithi175 ceph-mon[32613]: 6.d scrub starts 2024-09-21T12:01:29.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:28 smithi175 ceph-mon[32613]: 6.d scrub ok 2024-09-21T12:01:29.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:28 smithi175 ceph-mon[32613]: 4.4 scrub starts 2024-09-21T12:01:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:28 smithi175 ceph-mon[32613]: 4.4 scrub ok 2024-09-21T12:01:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:28 smithi175 ceph-mon[32613]: 3.18 scrub starts 2024-09-21T12:01:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:28 smithi175 ceph-mon[32613]: 3.18 scrub ok 2024-09-21T12:01:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:28 smithi175 ceph-mon[32613]: pgmap v1064: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-09-21T12:01:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:28 smithi175 ceph-mon[32613]: 4.1a scrub starts 2024-09-21T12:01:29.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:28 smithi175 ceph-mon[32613]: 4.1a scrub ok 2024-09-21T12:01:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:29 smithi043 ceph-mon[36454]: 6.18 scrub starts 2024-09-21T12:01:29.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:29 smithi043 ceph-mon[36454]: 6.18 scrub ok 2024-09-21T12:01:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:29 smithi043 ceph-mon[29710]: 6.18 scrub starts 2024-09-21T12:01:29.992 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:29 smithi043 ceph-mon[29710]: 6.18 scrub ok 2024-09-21T12:01:30.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:29 smithi175 ceph-mon[32613]: 6.18 scrub starts 2024-09-21T12:01:30.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:29 smithi175 ceph-mon[32613]: 6.18 scrub ok 2024-09-21T12:01:30.323 INFO:teuthology.orchestra.run.smithi043.stdout:231928234247 2024-09-21T12:01:30.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T12:01:31.043 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:30 smithi043 ceph-mon[36454]: 2.2 scrub starts 2024-09-21T12:01:31.043 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:30 smithi043 ceph-mon[36454]: 2.2 scrub ok 2024-09-21T12:01:31.043 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:30 smithi043 ceph-mon[36454]: 6.4 scrub starts 2024-09-21T12:01:31.043 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:30 smithi043 ceph-mon[36454]: 6.4 scrub ok 2024-09-21T12:01:31.043 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:30 smithi043 ceph-mon[36454]: pgmap v1065: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:01:31.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:30 smithi043 ceph-mon[29710]: 2.2 scrub starts 2024-09-21T12:01:31.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:30 smithi043 ceph-mon[29710]: 2.2 scrub ok 2024-09-21T12:01:31.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:30 smithi043 ceph-mon[29710]: 6.4 scrub starts 2024-09-21T12:01:31.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:30 smithi043 ceph-mon[29710]: 6.4 scrub ok 2024-09-21T12:01:31.044 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:30 smithi043 ceph-mon[29710]: pgmap v1065: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:01:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:30 smithi175 ceph-mon[32613]: 2.2 scrub starts 2024-09-21T12:01:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:30 smithi175 ceph-mon[32613]: 2.2 scrub ok 2024-09-21T12:01:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:30 smithi175 ceph-mon[32613]: 6.4 scrub starts 2024-09-21T12:01:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:30 smithi175 ceph-mon[32613]: 6.4 scrub ok 2024-09-21T12:01:31.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:30 smithi175 ceph-mon[32613]: pgmap v1065: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-21T12:01:31.845 INFO:teuthology.orchestra.run.smithi043.stdout:34359738672 2024-09-21T12:01:31.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T12:01:32.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:31 smithi043 ceph-mon[29710]: 6.13 scrub starts 2024-09-21T12:01:32.069 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:31 smithi043 ceph-mon[29710]: 6.13 scrub ok 2024-09-21T12:01:32.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:31 smithi043 ceph-mon[29710]: 5.1 scrub starts 2024-09-21T12:01:32.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:31 smithi043 ceph-mon[29710]: 5.1 scrub ok 2024-09-21T12:01:32.070 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:31 smithi043 ceph-mon[36454]: 6.13 scrub starts 2024-09-21T12:01:32.070 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:31 smithi043 ceph-mon[36454]: 6.13 scrub ok 2024-09-21T12:01:32.070 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:31 smithi043 ceph-mon[36454]: 5.1 scrub starts 2024-09-21T12:01:32.070 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:31 smithi043 ceph-mon[36454]: 5.1 scrub ok 2024-09-21T12:01:32.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:31 smithi175 ceph-mon[32613]: 6.13 scrub starts 2024-09-21T12:01:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:31 smithi175 ceph-mon[32613]: 6.13 scrub ok 2024-09-21T12:01:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:31 smithi175 ceph-mon[32613]: 5.1 scrub starts 2024-09-21T12:01:32.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:31 smithi175 ceph-mon[32613]: 5.1 scrub ok 2024-09-21T12:01:32.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:01:32.456 INFO:teuthology.orchestra.run.smithi043.stdout:201863463180 2024-09-21T12:01:32.457 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T12:01:32.705 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:33.074 INFO:teuthology.orchestra.run.smithi043.stdout:55834575146 2024-09-21T12:01:33.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: 4.8 scrub starts 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: 4.8 scrub ok 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: pgmap v1066: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: 6.17 scrub starts 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: 6.17 scrub ok 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:01:33.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: 4.8 scrub starts 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: 4.8 scrub ok 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: pgmap v1066: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: 6.17 scrub starts 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: 6.17 scrub ok 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:01:33.242 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:32 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: 4.8 scrub starts 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: 4.8 scrub ok 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: pgmap v1066: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: 6.17 scrub starts 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: 6.17 scrub ok 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:01:33.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:32 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:01:33.484 INFO:teuthology.orchestra.run.smithi043.stdout:77309411619 2024-09-21T12:01:33.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T12:01:34.080 INFO:teuthology.orchestra.run.smithi043.stdout:2065879269388 2024-09-21T12:01:34.080 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:34 smithi043 ceph-mon[29710]: 5.16 scrub starts 2024-09-21T12:01:34.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:34 smithi043 ceph-mon[29710]: 5.16 scrub ok 2024-09-21T12:01:34.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:34 smithi043 ceph-mon[29710]: 3.1e scrub starts 2024-09-21T12:01:34.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:34 smithi043 ceph-mon[29710]: 3.1e scrub ok 2024-09-21T12:01:34.081 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:34 smithi043 ceph-mon[29710]: 6.16 scrub starts 2024-09-21T12:01:34.081 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:34 smithi043 ceph-mon[36454]: 5.16 scrub starts 2024-09-21T12:01:34.081 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:34 smithi043 ceph-mon[36454]: 5.16 scrub ok 2024-09-21T12:01:34.081 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:34 smithi043 ceph-mon[36454]: 3.1e scrub starts 2024-09-21T12:01:34.081 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:34 smithi043 ceph-mon[36454]: 3.1e scrub ok 2024-09-21T12:01:34.081 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:34 smithi043 ceph-mon[36454]: 6.16 scrub starts 2024-09-21T12:01:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:34 smithi175 ceph-mon[32613]: 5.16 scrub starts 2024-09-21T12:01:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:34 smithi175 ceph-mon[32613]: 5.16 scrub ok 2024-09-21T12:01:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:34 smithi175 ceph-mon[32613]: 3.1e scrub starts 2024-09-21T12:01:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:34 smithi175 ceph-mon[32613]: 3.1e scrub ok 2024-09-21T12:01:34.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:34 smithi175 ceph-mon[32613]: 6.16 scrub starts 2024-09-21T12:01:34.426 INFO:teuthology.orchestra.run.smithi043.stdout:115964117277 2024-09-21T12:01:34.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T12:01:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:35.208 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:35.260 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:35 smithi175 ceph-mon[32613]: 6.16 scrub ok 2024-09-21T12:01:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:35 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2709698474' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T12:01:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:35 smithi175 ceph-mon[32613]: 1.4 scrub starts 2024-09-21T12:01:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:35 smithi175 ceph-mon[32613]: 1.4 scrub ok 2024-09-21T12:01:35.261 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:35 smithi175 ceph-mon[32613]: pgmap v1067: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail 2024-09-21T12:01:35.289 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:01:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:35.289 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-mon[36454]: 6.16 scrub ok 2024-09-21T12:01:35.289 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2709698474' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T12:01:35.289 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-mon[36454]: 1.4 scrub starts 2024-09-21T12:01:35.289 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-mon[36454]: 1.4 scrub ok 2024-09-21T12:01:35.289 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-mon[36454]: pgmap v1067: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail 2024-09-21T12:01:35.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-mon[29710]: 6.16 scrub ok 2024-09-21T12:01:35.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2709698474' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T12:01:35.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-mon[29710]: 1.4 scrub starts 2024-09-21T12:01:35.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-mon[29710]: 1.4 scrub ok 2024-09-21T12:01:35.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:35 smithi043 ceph-mon[29710]: pgmap v1067: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail 2024-09-21T12:01:35.721 INFO:tasks.cephadm.ceph_manager.ceph:need seq 2065879269387 got 2065879269388 for osd.4 2024-09-21T12:01:35.721 DEBUG:teuthology.parallel:result is None 2024-09-21T12:01:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:35.844 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:35.847 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:36.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:36 smithi043 ceph-mon[36454]: 2.1 scrub starts 2024-09-21T12:01:36.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:36 smithi043 ceph-mon[36454]: 2.1 scrub ok 2024-09-21T12:01:36.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:36 smithi043 ceph-mon[29710]: 2.1 scrub starts 2024-09-21T12:01:36.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:36 smithi043 ceph-mon[29710]: 2.1 scrub ok 2024-09-21T12:01:36.419 INFO:teuthology.orchestra.run.smithi043.stdout:167503724818 2024-09-21T12:01:36.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T12:01:36.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:36 smithi175 ceph-mon[32613]: 2.1 scrub starts 2024-09-21T12:01:36.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:36 smithi175 ceph-mon[32613]: 2.1 scrub ok 2024-09-21T12:01:37.294 INFO:teuthology.orchestra.run.smithi043.stdout:34359738673 2024-09-21T12:01:37.294 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[29710]: 5.1b scrub starts 2024-09-21T12:01:37.294 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[29710]: 5.1b scrub ok 2024-09-21T12:01:37.294 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[29710]: pgmap v1068: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 552 B/s rd, 0 op/s 2024-09-21T12:01:37.294 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[29710]: 3.12 scrub starts 2024-09-21T12:01:37.294 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[29710]: 3.12 scrub ok 2024-09-21T12:01:37.294 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[29710]: 1.0 scrub starts 2024-09-21T12:01:37.294 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[29710]: 1.0 scrub ok 2024-09-21T12:01:37.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[36454]: 5.1b scrub starts 2024-09-21T12:01:37.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[36454]: 5.1b scrub ok 2024-09-21T12:01:37.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[36454]: pgmap v1068: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 552 B/s rd, 0 op/s 2024-09-21T12:01:37.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[36454]: 3.12 scrub starts 2024-09-21T12:01:37.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[36454]: 3.12 scrub ok 2024-09-21T12:01:37.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[36454]: 1.0 scrub starts 2024-09-21T12:01:37.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:37 smithi043 ceph-mon[36454]: 1.0 scrub ok 2024-09-21T12:01:37.484 INFO:teuthology.orchestra.run.smithi043.stdout:201863463182 2024-09-21T12:01:37.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:37 smithi175 ceph-mon[32613]: 5.1b scrub starts 2024-09-21T12:01:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:37 smithi175 ceph-mon[32613]: 5.1b scrub ok 2024-09-21T12:01:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:37 smithi175 ceph-mon[32613]: pgmap v1068: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 552 B/s rd, 0 op/s 2024-09-21T12:01:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:37 smithi175 ceph-mon[32613]: 3.12 scrub starts 2024-09-21T12:01:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:37 smithi175 ceph-mon[32613]: 3.12 scrub ok 2024-09-21T12:01:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:37 smithi175 ceph-mon[32613]: 1.0 scrub starts 2024-09-21T12:01:37.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:37 smithi175 ceph-mon[32613]: 1.0 scrub ok 2024-09-21T12:01:38.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:38 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2066440738' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T12:01:38.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:38 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/155749399' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T12:01:38.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:38 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2066440738' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T12:01:38.184 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:38 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/155749399' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T12:01:38.214 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863463180 got 201863463182 for osd.6 2024-09-21T12:01:38.214 DEBUG:teuthology.parallel:result is None 2024-09-21T12:01:38.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:38 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2066440738' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T12:01:38.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:38 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/155749399' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T12:01:38.957 INFO:teuthology.orchestra.run.smithi043.stdout:77309411620 2024-09-21T12:01:39.485 INFO:teuthology.orchestra.run.smithi043.stdout:55834575148 2024-09-21T12:01:39.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[36454]: 6.6 scrub starts 2024-09-21T12:01:39.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[36454]: 6.6 scrub ok 2024-09-21T12:01:39.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[36454]: pgmap v1069: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 863 B/s rd, 0 op/s 2024-09-21T12:01:39.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[36454]: 1.7 scrub starts 2024-09-21T12:01:39.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[36454]: 1.7 scrub ok 2024-09-21T12:01:39.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3901304180' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T12:01:39.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[29710]: 6.6 scrub starts 2024-09-21T12:01:39.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[29710]: 6.6 scrub ok 2024-09-21T12:01:39.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[29710]: pgmap v1069: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 863 B/s rd, 0 op/s 2024-09-21T12:01:39.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[29710]: 1.7 scrub starts 2024-09-21T12:01:39.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[29710]: 1.7 scrub ok 2024-09-21T12:01:39.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:39 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3901304180' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T12:01:39.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:39 smithi175 ceph-mon[32613]: 6.6 scrub starts 2024-09-21T12:01:39.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:39 smithi175 ceph-mon[32613]: 6.6 scrub ok 2024-09-21T12:01:39.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:39 smithi175 ceph-mon[32613]: pgmap v1069: 142 pgs: 142 active+clean; 583 KiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 863 B/s rd, 0 op/s 2024-09-21T12:01:39.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:39 smithi175 ceph-mon[32613]: 1.7 scrub starts 2024-09-21T12:01:39.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:39 smithi175 ceph-mon[32613]: 1.7 scrub ok 2024-09-21T12:01:39.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:39 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3901304180' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T12:01:39.732 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:40.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:40 smithi043 ceph-mon[36454]: 5.13 scrub starts 2024-09-21T12:01:40.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:40 smithi043 ceph-mon[36454]: 5.13 scrub ok 2024-09-21T12:01:40.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:40 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2248280433' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T12:01:40.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:40 smithi043 ceph-mon[36454]: 6.8 scrub starts 2024-09-21T12:01:40.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:40 smithi043 ceph-mon[36454]: 6.8 scrub ok 2024-09-21T12:01:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:40 smithi043 ceph-mon[29710]: 5.13 scrub starts 2024-09-21T12:01:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:40 smithi043 ceph-mon[29710]: 5.13 scrub ok 2024-09-21T12:01:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:40 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2248280433' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T12:01:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:40 smithi043 ceph-mon[29710]: 6.8 scrub starts 2024-09-21T12:01:40.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:40 smithi043 ceph-mon[29710]: 6.8 scrub ok 2024-09-21T12:01:40.313 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411619 got 77309411620 for osd.2 2024-09-21T12:01:40.313 DEBUG:teuthology.parallel:result is None 2024-09-21T12:01:40.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:40 smithi175 ceph-mon[32613]: 5.13 scrub starts 2024-09-21T12:01:40.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:40 smithi175 ceph-mon[32613]: 5.13 scrub ok 2024-09-21T12:01:40.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:40 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2248280433' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T12:01:40.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:40 smithi175 ceph-mon[32613]: 6.8 scrub starts 2024-09-21T12:01:40.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:40 smithi175 ceph-mon[32613]: 6.8 scrub ok 2024-09-21T12:01:40.985 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834575146 got 55834575148 for osd.1 2024-09-21T12:01:40.986 DEBUG:teuthology.parallel:result is None 2024-09-21T12:01:41.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:41 smithi043 ceph-mon[36454]: pgmap v1070: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:01:41.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:41 smithi043 ceph-mon[36454]: 5.6 scrub starts 2024-09-21T12:01:41.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:41 smithi043 ceph-mon[36454]: 5.6 scrub ok 2024-09-21T12:01:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:41 smithi043 ceph-mon[29710]: pgmap v1070: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:01:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:41 smithi043 ceph-mon[29710]: 5.6 scrub starts 2024-09-21T12:01:41.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:41 smithi043 ceph-mon[29710]: 5.6 scrub ok 2024-09-21T12:01:41.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:41 smithi175 ceph-mon[32613]: pgmap v1070: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-21T12:01:41.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:41 smithi175 ceph-mon[32613]: 5.6 scrub starts 2024-09-21T12:01:41.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:41 smithi175 ceph-mon[32613]: 5.6 scrub ok 2024-09-21T12:01:41.555 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738672 got 34359738673 for osd.0 2024-09-21T12:01:41.555 DEBUG:teuthology.parallel:result is None 2024-09-21T12:01:41.954 INFO:teuthology.orchestra.run.smithi043.stdout:231928234249 2024-09-21T12:01:42.155 INFO:teuthology.orchestra.run.smithi043.stdout:115964117279 2024-09-21T12:01:42.475 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:42 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3233426651' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T12:01:42.475 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:42 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3233426651' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T12:01:42.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:42 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3233426651' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T12:01:42.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:01:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:01:42.888 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117277 got 115964117279 for osd.3 2024-09-21T12:01:42.888 DEBUG:teuthology.parallel:result is None 2024-09-21T12:01:43.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[36454]: 6.1b scrub starts 2024-09-21T12:01:43.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[36454]: 6.1b scrub ok 2024-09-21T12:01:43.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[36454]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:43.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/2974552480' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T12:01:43.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[36454]: pgmap v1071: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:01:43.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[36454]: 3.9 scrub starts 2024-09-21T12:01:43.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[36454]: 3.9 scrub ok 2024-09-21T12:01:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[29710]: 6.1b scrub starts 2024-09-21T12:01:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[29710]: 6.1b scrub ok 2024-09-21T12:01:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[29710]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/2974552480' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T12:01:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[29710]: pgmap v1071: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:01:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[29710]: 3.9 scrub starts 2024-09-21T12:01:43.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:43 smithi043 ceph-mon[29710]: 3.9 scrub ok 2024-09-21T12:01:43.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234247 got 231928234249 for osd.7 2024-09-21T12:01:43.379 DEBUG:teuthology.parallel:result is None 2024-09-21T12:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:43.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:43 smithi175 ceph-mon[32613]: 6.1b scrub starts 2024-09-21T12:01:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:43 smithi175 ceph-mon[32613]: 6.1b scrub ok 2024-09-21T12:01:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:43 smithi175 ceph-mon[32613]: from='client.24433 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-21T12:01:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:43 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/2974552480' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T12:01:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:43 smithi175 ceph-mon[32613]: pgmap v1071: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:01:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:43 smithi175 ceph-mon[32613]: 3.9 scrub starts 2024-09-21T12:01:43.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:43 smithi175 ceph-mon[32613]: 3.9 scrub ok 2024-09-21T12:01:44.139 INFO:teuthology.orchestra.run.smithi043.stdout:167503724820 2024-09-21T12:01:44.140 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:44 smithi043 ceph-mon[29710]: 6.a scrub starts 2024-09-21T12:01:44.140 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:44 smithi043 ceph-mon[29710]: 6.a scrub ok 2024-09-21T12:01:44.140 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:44 smithi043 ceph-mon[36454]: 6.a scrub starts 2024-09-21T12:01:44.140 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:44 smithi043 ceph-mon[36454]: 6.a scrub ok 2024-09-21T12:01:44.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:44 smithi175 ceph-mon[32613]: 6.a scrub starts 2024-09-21T12:01:44.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:44 smithi175 ceph-mon[32613]: 6.a scrub ok 2024-09-21T12:01:44.778 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724818 got 167503724820 for osd.5 2024-09-21T12:01:44.778 DEBUG:teuthology.parallel:result is None 2024-09-21T12:01:44.778 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-21T12:01:44.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T12:01:45.081 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:45.410 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:45 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: ::ffff:172.21.15.175 - - [21/Sep/2024:12:01:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-21T12:01:45.410 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:45 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/3403610698' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T12:01:45.411 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:45 smithi043 ceph-mon[36454]: pgmap v1072: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:45.411 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:45 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/3403610698' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T12:01:45.411 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:45 smithi043 ceph-mon[29710]: pgmap v1072: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:45.412 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:45 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/3403610698' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T12:01:45.412 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:45 smithi175 ceph-mon[32613]: pgmap v1072: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:45.886 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T12:01:45.888 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T12:01:46.414 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:46 smithi043 ceph-mon[36454]: 6.1 scrub starts 2024-09-21T12:01:46.414 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:46 smithi043 ceph-mon[36454]: 6.1 scrub ok 2024-09-21T12:01:46.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:46 smithi043 ceph-mon[29710]: 6.1 scrub starts 2024-09-21T12:01:46.415 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:46 smithi043 ceph-mon[29710]: 6.1 scrub ok 2024-09-21T12:01:46.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:46 smithi175 ceph-mon[32613]: 6.1 scrub starts 2024-09-21T12:01:46.510 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:46 smithi175 ceph-mon[32613]: 6.1 scrub ok 2024-09-21T12:01:46.637 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":1072,"stamp":"2024-09-21T12:01:44.243982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2849,"num_read_kb":2601,"num_write":1008,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1937,"ondisk_log_size":1937,"up":426,"acting":426,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":429,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":5450940,"kb_used_data":32316,"kb_used_omap":15,"kb_used_meta":5418480,"kb_avail":744510276,"statfs":{"total":767960285184,"available":762378522624,"internally_reserved":0,"allocated":33091584,"data_stored":27582897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15600,"internal_metadata":5548524304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003291"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":188,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612828+0000","last_change":"2024-09-21T12:01:07.788817+0000","last_active":"2024-09-21T12:01:27.612828+0000","last_peered":"2024-09-21T12:01:27.612828+0000","last_clean":"2024-09-21T12:01:27.612828+0000","last_became_active":"2024-09-21T11:57:58.529245+0000","last_became_peered":"2024-09-21T11:57:58.529245+0000","last_unstale":"2024-09-21T12:01:27.612828+0000","last_undegraded":"2024-09-21T12:01:27.612828+0000","last_fullsized":"2024-09-21T12:01:27.612828+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:07.788769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:07.788769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:18.906915+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"471'21","reported_seq":975,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571095+0000","last_change":"2024-09-21T12:01:21.965294+0000","last_active":"2024-09-21T12:01:27.571095+0000","last_peered":"2024-09-21T12:01:27.571095+0000","last_clean":"2024-09-21T12:01:27.571095+0000","last_became_active":"2024-09-21T12:01:21.964962+0000","last_became_peered":"2024-09-21T12:01:21.964962+0000","last_unstale":"2024-09-21T12:01:27.571095+0000","last_undegraded":"2024-09-21T12:01:27.571095+0000","last_fullsized":"2024-09-21T12:01:27.571095+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:01:13.523391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:13.523391+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:28.651463+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":670,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570409+0000","last_change":"2024-09-21T12:00:32.278040+0000","last_active":"2024-09-21T12:01:27.570409+0000","last_peered":"2024-09-21T12:01:27.570409+0000","last_clean":"2024-09-21T12:01:27.570409+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T12:01:27.570409+0000","last_undegraded":"2024-09-21T12:01:27.570409+0000","last_fullsized":"2024-09-21T12:01:27.570409+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:32.277990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:32.277990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:42.907481+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":730,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571189+0000","last_change":"2024-09-21T12:01:21.966845+0000","last_active":"2024-09-21T12:01:27.571189+0000","last_peered":"2024-09-21T12:01:27.571189+0000","last_clean":"2024-09-21T12:01:27.571189+0000","last_became_active":"2024-09-21T12:01:21.966286+0000","last_became_peered":"2024-09-21T12:01:21.966286+0000","last_unstale":"2024-09-21T12:01:27.571189+0000","last_undegraded":"2024-09-21T12:01:27.571189+0000","last_fullsized":"2024-09-21T12:01:27.571189+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:00.260916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:00.260916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.498477+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":964,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571083+0000","last_change":"2024-09-21T12:00:17.127676+0000","last_active":"2024-09-21T12:01:27.571083+0000","last_peered":"2024-09-21T12:01:27.571083+0000","last_clean":"2024-09-21T12:01:27.571083+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T12:01:27.571083+0000","last_undegraded":"2024-09-21T12:01:27.571083+0000","last_fullsized":"2024-09-21T12:01:27.571083+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:17.127621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:17.127621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:28.262753+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":954,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:33.118091+0000","last_change":"2024-09-21T12:01:33.118091+0000","last_active":"2024-09-21T12:01:33.118091+0000","last_peered":"2024-09-21T12:01:33.118091+0000","last_clean":"2024-09-21T12:01:33.118091+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T12:01:33.118091+0000","last_undegraded":"2024-09-21T12:01:33.118091+0000","last_fullsized":"2024-09-21T12:01:33.118091+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:33.118044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:33.118044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:53.081549+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"471'39","reported_seq":1017,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.572180+0000","last_change":"2024-09-21T12:01:07.150677+0000","last_active":"2024-09-21T12:01:27.572180+0000","last_peered":"2024-09-21T12:01:27.572180+0000","last_clean":"2024-09-21T12:01:27.572180+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T12:01:27.572180+0000","last_undegraded":"2024-09-21T12:01:27.572180+0000","last_fullsized":"2024-09-21T12:01:27.572180+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'39","last_scrub_stamp":"2024-09-21T12:01:07.150606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:07.150606+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:18.322620+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":780,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570976+0000","last_change":"2024-09-21T12:01:21.965378+0000","last_active":"2024-09-21T12:01:27.570976+0000","last_peered":"2024-09-21T12:01:27.570976+0000","last_clean":"2024-09-21T12:01:27.570976+0000","last_became_active":"2024-09-21T12:01:21.965021+0000","last_became_peered":"2024-09-21T12:01:21.965021+0000","last_unstale":"2024-09-21T12:01:27.570976+0000","last_undegraded":"2024-09-21T12:01:27.570976+0000","last_fullsized":"2024-09-21T12:01:27.570976+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:45.299024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:45.299024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:53.874908+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":954,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614227+0000","last_change":"2024-09-21T12:00:43.989346+0000","last_active":"2024-09-21T12:01:27.614227+0000","last_peered":"2024-09-21T12:01:27.614227+0000","last_clean":"2024-09-21T12:01:27.614227+0000","last_became_active":"2024-09-21T12:00:40.761723+0000","last_became_peered":"2024-09-21T12:00:40.761723+0000","last_unstale":"2024-09-21T12:01:27.614227+0000","last_undegraded":"2024-09-21T12:01:27.614227+0000","last_fullsized":"2024-09-21T12:01:27.614227+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:43.989297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:43.989297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:05.498788+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"471'21","reported_seq":975,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:28.449371+0000","last_change":"2024-09-21T12:01:28.449371+0000","last_active":"2024-09-21T12:01:28.449371+0000","last_peered":"2024-09-21T12:01:28.449371+0000","last_clean":"2024-09-21T12:01:28.449371+0000","last_became_active":"2024-09-21T12:01:21.964136+0000","last_became_peered":"2024-09-21T12:01:21.964136+0000","last_unstale":"2024-09-21T12:01:28.449371+0000","last_undegraded":"2024-09-21T12:01:28.449371+0000","last_fullsized":"2024-09-21T12:01:28.449371+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:01:28.449269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:28.449269+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:37.672700+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":959,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.222847+0000","last_change":"2024-09-21T12:01:35.222847+0000","last_active":"2024-09-21T12:01:35.222847+0000","last_peered":"2024-09-21T12:01:35.222847+0000","last_clean":"2024-09-21T12:01:35.222847+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T12:01:35.222847+0000","last_undegraded":"2024-09-21T12:01:35.222847+0000","last_fullsized":"2024-09-21T12:01:35.222847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:35.222788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:35.222788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:35.247504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":196,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:29.623778+0000","last_change":"2024-09-21T12:01:29.623778+0000","last_active":"2024-09-21T12:01:29.623778+0000","last_peered":"2024-09-21T12:01:29.623778+0000","last_clean":"2024-09-21T12:01:29.623778+0000","last_became_active":"2024-09-21T11:57:58.526820+0000","last_became_peered":"2024-09-21T11:57:58.526820+0000","last_unstale":"2024-09-21T12:01:29.623778+0000","last_undegraded":"2024-09-21T12:01:29.623778+0000","last_fullsized":"2024-09-21T12:01:29.623778+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:29.623712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:29.623712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:45.454344+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1008,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612449+0000","last_change":"2024-09-21T12:01:17.241924+0000","last_active":"2024-09-21T12:01:27.612449+0000","last_peered":"2024-09-21T12:01:27.612449+0000","last_clean":"2024-09-21T12:01:27.612449+0000","last_became_active":"2024-09-21T12:00:40.759034+0000","last_became_peered":"2024-09-21T12:00:40.759034+0000","last_unstale":"2024-09-21T12:01:27.612449+0000","last_undegraded":"2024-09-21T12:01:27.612449+0000","last_fullsized":"2024-09-21T12:01:27.612449+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:17.241870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:17.241870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:36.243892+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"471'9","reported_seq":959,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571388+0000","last_change":"2024-09-21T12:01:21.962922+0000","last_active":"2024-09-21T12:01:27.571388+0000","last_peered":"2024-09-21T12:01:27.571388+0000","last_clean":"2024-09-21T12:01:27.571388+0000","last_became_active":"2024-09-21T12:01:21.962563+0000","last_became_peered":"2024-09-21T12:01:21.962563+0000","last_unstale":"2024-09-21T12:01:27.571388+0000","last_undegraded":"2024-09-21T12:01:27.571388+0000","last_fullsized":"2024-09-21T12:01:27.571388+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'9","last_scrub_stamp":"2024-09-21T12:01:13.163707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:13.163707+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:38.244122+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":12,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":976,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613106+0000","last_change":"2024-09-21T12:00:57.756967+0000","last_active":"2024-09-21T12:01:27.613106+0000","last_peered":"2024-09-21T12:01:27.613106+0000","last_clean":"2024-09-21T12:01:27.613106+0000","last_became_active":"2024-09-21T12:00:40.770339+0000","last_became_peered":"2024-09-21T12:00:40.770339+0000","last_unstale":"2024-09-21T12:01:27.613106+0000","last_undegraded":"2024-09-21T12:01:27.613106+0000","last_fullsized":"2024-09-21T12:01:27.613106+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:57.756910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:57.756910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:00.222316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":961,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613222+0000","last_change":"2024-09-21T12:01:02.274827+0000","last_active":"2024-09-21T12:01:27.613222+0000","last_peered":"2024-09-21T12:01:27.613222+0000","last_clean":"2024-09-21T12:01:27.613222+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T12:01:27.613222+0000","last_undegraded":"2024-09-21T12:01:27.613222+0000","last_fullsized":"2024-09-21T12:01:27.613222+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:02.274749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:02.274749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:19.205320+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":529,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571970+0000","last_change":"2024-09-21T12:01:21.970447+0000","last_active":"2024-09-21T12:01:27.571970+0000","last_peered":"2024-09-21T12:01:27.571970+0000","last_clean":"2024-09-21T12:01:27.571970+0000","last_became_active":"2024-09-21T12:01:21.969324+0000","last_became_peered":"2024-09-21T12:01:21.969324+0000","last_unstale":"2024-09-21T12:01:27.571970+0000","last_undegraded":"2024-09-21T12:01:27.571970+0000","last_fullsized":"2024-09-21T12:01:27.571970+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:01.476028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:01.476028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:25.508196+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":207,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612312+0000","last_change":"2024-09-21T12:01:22.786474+0000","last_active":"2024-09-21T12:01:27.612312+0000","last_peered":"2024-09-21T12:01:27.612312+0000","last_clean":"2024-09-21T12:01:27.612312+0000","last_became_active":"2024-09-21T12:00:40.764140+0000","last_became_peered":"2024-09-21T12:00:40.764140+0000","last_unstale":"2024-09-21T12:01:27.612312+0000","last_undegraded":"2024-09-21T12:01:27.612312+0000","last_fullsized":"2024-09-21T12:01:27.612312+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:22.786418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:22.786418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:26.608584+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"471'39","reported_seq":1133,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612787+0000","last_change":"2024-09-21T12:01:11.392703+0000","last_active":"2024-09-21T12:01:27.612787+0000","last_peered":"2024-09-21T12:01:27.612787+0000","last_clean":"2024-09-21T12:01:27.612787+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T12:01:27.612787+0000","last_undegraded":"2024-09-21T12:01:27.612787+0000","last_fullsized":"2024-09-21T12:01:27.612787+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'39","last_scrub_stamp":"2024-09-21T12:01:11.392646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:11.392646+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:40.107122+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":686,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570947+0000","last_change":"2024-09-21T12:00:46.476017+0000","last_active":"2024-09-21T12:01:27.570947+0000","last_peered":"2024-09-21T12:01:27.570947+0000","last_clean":"2024-09-21T12:01:27.570947+0000","last_became_active":"2024-09-21T12:00:40.771901+0000","last_became_peered":"2024-09-21T12:00:40.771901+0000","last_unstale":"2024-09-21T12:01:27.570947+0000","last_undegraded":"2024-09-21T12:01:27.570947+0000","last_fullsized":"2024-09-21T12:01:27.570947+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:46.475967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:46.475967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:51.244834+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":958,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571559+0000","last_change":"2024-09-21T12:01:20.205243+0000","last_active":"2024-09-21T12:01:27.571559+0000","last_peered":"2024-09-21T12:01:27.571559+0000","last_clean":"2024-09-21T12:01:27.571559+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T12:01:27.571559+0000","last_undegraded":"2024-09-21T12:01:27.571559+0000","last_fullsized":"2024-09-21T12:01:27.571559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:20.205194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:20.205194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:44.483785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":750,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571378+0000","last_change":"2024-09-21T12:01:21.966317+0000","last_active":"2024-09-21T12:01:27.571378+0000","last_peered":"2024-09-21T12:01:27.571378+0000","last_clean":"2024-09-21T12:01:27.571378+0000","last_became_active":"2024-09-21T12:01:21.966023+0000","last_became_peered":"2024-09-21T12:01:21.966023+0000","last_unstale":"2024-09-21T12:01:27.571378+0000","last_undegraded":"2024-09-21T12:01:27.571378+0000","last_fullsized":"2024-09-21T12:01:27.571378+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:08.647539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:08.647539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:29.406037+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"471'24","reported_seq":1047,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571697+0000","last_change":"2024-09-21T12:00:48.124110+0000","last_active":"2024-09-21T12:01:27.571697+0000","last_peered":"2024-09-21T12:01:27.571697+0000","last_clean":"2024-09-21T12:01:27.571697+0000","last_became_active":"2024-09-21T12:00:40.776684+0000","last_became_peered":"2024-09-21T12:00:40.776684+0000","last_unstale":"2024-09-21T12:01:27.571697+0000","last_undegraded":"2024-09-21T12:01:27.571697+0000","last_fullsized":"2024-09-21T12:01:27.571697+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'24","last_scrub_stamp":"2024-09-21T12:00:48.124043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:48.124043+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:02.006286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":442,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571550+0000","last_change":"2024-09-21T12:01:21.967750+0000","last_active":"2024-09-21T12:01:27.571550+0000","last_peered":"2024-09-21T12:01:27.571550+0000","last_clean":"2024-09-21T12:01:27.571550+0000","last_became_active":"2024-09-21T12:01:21.967438+0000","last_became_peered":"2024-09-21T12:01:21.967438+0000","last_unstale":"2024-09-21T12:01:27.571550+0000","last_undegraded":"2024-09-21T12:01:27.571550+0000","last_fullsized":"2024-09-21T12:01:27.571550+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:06.190850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:06.190850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:31.275779+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":457,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612528+0000","last_change":"2024-09-21T12:00:36.790840+0000","last_active":"2024-09-21T12:01:27.612528+0000","last_peered":"2024-09-21T12:01:27.612528+0000","last_clean":"2024-09-21T12:01:27.612528+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T12:01:27.612528+0000","last_undegraded":"2024-09-21T12:01:27.612528+0000","last_fullsized":"2024-09-21T12:01:27.612528+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T12:00:36.790778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:36.790778+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:02.627806+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":707,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570550+0000","last_change":"2024-09-21T12:00:50.530029+0000","last_active":"2024-09-21T12:01:27.570550+0000","last_peered":"2024-09-21T12:01:27.570550+0000","last_clean":"2024-09-21T12:01:27.570550+0000","last_became_active":"2024-09-21T12:00:40.762620+0000","last_became_peered":"2024-09-21T12:00:40.762620+0000","last_unstale":"2024-09-21T12:01:27.570550+0000","last_undegraded":"2024-09-21T12:01:27.570550+0000","last_fullsized":"2024-09-21T12:01:27.570550+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T12:00:50.529956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:50.529956+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:05.653998+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"471'30","reported_seq":1001,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612808+0000","last_change":"2024-09-21T12:00:47.891826+0000","last_active":"2024-09-21T12:01:27.612808+0000","last_peered":"2024-09-21T12:01:27.612808+0000","last_clean":"2024-09-21T12:01:27.612808+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T12:01:27.612808+0000","last_undegraded":"2024-09-21T12:01:27.612808+0000","last_fullsized":"2024-09-21T12:01:27.612808+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'30","last_scrub_stamp":"2024-09-21T12:00:47.891763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:47.891763+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.873947+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"482'12","reported_seq":1294,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634288+0000","last_change":"2024-09-21T12:01:18.612421+0000","last_active":"2024-09-21T12:01:35.634288+0000","last_peered":"2024-09-21T12:01:35.634288+0000","last_clean":"2024-09-21T12:01:35.634288+0000","last_became_active":"2024-09-21T12:00:40.762782+0000","last_became_peered":"2024-09-21T12:00:40.762782+0000","last_unstale":"2024-09-21T12:01:35.634288+0000","last_undegraded":"2024-09-21T12:01:35.634288+0000","last_fullsized":"2024-09-21T12:01:35.634288+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'12","last_scrub_stamp":"2024-09-21T12:01:18.612327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:18.612327+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:45.592541+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":206,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614155+0000","last_change":"2024-09-21T12:00:50.137327+0000","last_active":"2024-09-21T12:01:27.614155+0000","last_peered":"2024-09-21T12:01:27.614155+0000","last_clean":"2024-09-21T12:01:27.614155+0000","last_became_active":"2024-09-21T11:57:45.504680+0000","last_became_peered":"2024-09-21T11:57:45.504680+0000","last_unstale":"2024-09-21T12:01:27.614155+0000","last_undegraded":"2024-09-21T12:01:27.614155+0000","last_fullsized":"2024-09-21T12:01:27.614155+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:50.137280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:50.137280+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:52.003733+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"471'39","reported_seq":740,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.572350+0000","last_change":"2024-09-21T12:00:45.439926+0000","last_active":"2024-09-21T12:01:27.572350+0000","last_peered":"2024-09-21T12:01:27.572350+0000","last_clean":"2024-09-21T12:01:27.572350+0000","last_became_active":"2024-09-21T12:00:40.771095+0000","last_became_peered":"2024-09-21T12:00:40.771095+0000","last_unstale":"2024-09-21T12:01:27.572350+0000","last_undegraded":"2024-09-21T12:01:27.572350+0000","last_fullsized":"2024-09-21T12:01:27.572350+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'39","last_scrub_stamp":"2024-09-21T12:00:45.439843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:45.439843+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.634394+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":984,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570783+0000","last_change":"2024-09-21T12:01:27.149114+0000","last_active":"2024-09-21T12:01:27.570783+0000","last_peered":"2024-09-21T12:01:27.570783+0000","last_clean":"2024-09-21T12:01:27.570783+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T12:01:27.570783+0000","last_undegraded":"2024-09-21T12:01:27.570783+0000","last_fullsized":"2024-09-21T12:01:27.570783+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:27.149054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:27.149054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:47.587603+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"502'14","reported_seq":592,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634307+0000","last_change":"2024-09-21T12:01:21.969048+0000","last_active":"2024-09-21T12:01:35.634307+0000","last_peered":"2024-09-21T12:01:35.634307+0000","last_clean":"2024-09-21T12:01:35.634307+0000","last_became_active":"2024-09-21T12:01:21.968662+0000","last_became_peered":"2024-09-21T12:01:21.968662+0000","last_unstale":"2024-09-21T12:01:35.634307+0000","last_undegraded":"2024-09-21T12:01:35.634307+0000","last_fullsized":"2024-09-21T12:01:35.634307+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'13","last_scrub_stamp":"2024-09-21T12:00:46.020472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:46.020472+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:46.467129+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1016,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613139+0000","last_change":"2024-09-21T12:01:27.310149+0000","last_active":"2024-09-21T12:01:27.613139+0000","last_peered":"2024-09-21T12:01:27.613139+0000","last_clean":"2024-09-21T12:01:27.613139+0000","last_became_active":"2024-09-21T11:57:58.526927+0000","last_became_peered":"2024-09-21T11:57:58.526927+0000","last_unstale":"2024-09-21T12:01:27.613139+0000","last_undegraded":"2024-09-21T12:01:27.613139+0000","last_fullsized":"2024-09-21T12:01:27.613139+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:27.310092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:27.310092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:41.158567+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":28,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571915+0000","last_change":"2024-09-21T12:01:25.542758+0000","last_active":"2024-09-21T12:01:27.571915+0000","last_peered":"2024-09-21T12:01:27.571915+0000","last_clean":"2024-09-21T12:01:27.571915+0000","last_became_active":"2024-09-21T12:01:21.967906+0000","last_became_peered":"2024-09-21T12:01:21.967906+0000","last_unstale":"2024-09-21T12:01:27.571915+0000","last_undegraded":"2024-09-21T12:01:27.571915+0000","last_fullsized":"2024-09-21T12:01:27.571915+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:01:25.542702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:01:25.542702+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:31.811384+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"471'42","reported_seq":1092,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570555+0000","last_change":"2024-09-21T12:00:18.086280+0000","last_active":"2024-09-21T12:01:27.570555+0000","last_peered":"2024-09-21T12:01:27.570555+0000","last_clean":"2024-09-21T12:01:27.570555+0000","last_became_active":"2024-09-21T11:57:58.537184+0000","last_became_peered":"2024-09-21T11:57:58.537184+0000","last_unstale":"2024-09-21T12:01:27.570555+0000","last_undegraded":"2024-09-21T12:01:27.570555+0000","last_fullsized":"2024-09-21T12:01:27.570555+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'42","last_scrub_stamp":"2024-09-21T12:00:18.086194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:18.086194+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:31.282861+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":978,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.572118+0000","last_change":"2024-09-21T12:01:16.118758+0000","last_active":"2024-09-21T12:01:27.572118+0000","last_peered":"2024-09-21T12:01:27.572118+0000","last_clean":"2024-09-21T12:01:27.572118+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T12:01:27.572118+0000","last_undegraded":"2024-09-21T12:01:27.572118+0000","last_fullsized":"2024-09-21T12:01:27.572118+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:16.118709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:16.118709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:17.543975+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":533,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:31.437280+0000","last_change":"2024-09-21T12:01:31.437280+0000","last_active":"2024-09-21T12:01:31.437280+0000","last_peered":"2024-09-21T12:01:31.437280+0000","last_clean":"2024-09-21T12:01:31.437280+0000","last_became_active":"2024-09-21T12:01:22.097383+0000","last_became_peered":"2024-09-21T12:01:22.097383+0000","last_unstale":"2024-09-21T12:01:31.437280+0000","last_undegraded":"2024-09-21T12:01:31.437280+0000","last_fullsized":"2024-09-21T12:01:31.437280+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:31.437219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:31.437219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:54.084667+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":484,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611796+0000","last_change":"2024-09-21T12:01:09.388467+0000","last_active":"2024-09-21T12:01:27.611796+0000","last_peered":"2024-09-21T12:01:27.611796+0000","last_clean":"2024-09-21T12:01:27.611796+0000","last_became_active":"2024-09-21T12:00:40.763206+0000","last_became_peered":"2024-09-21T12:00:40.763206+0000","last_unstale":"2024-09-21T12:01:27.611796+0000","last_undegraded":"2024-09-21T12:01:27.611796+0000","last_fullsized":"2024-09-21T12:01:27.611796+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:09.388420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:09.388420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:30.778544+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":211,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:34.127197+0000","last_change":"2024-09-21T12:01:34.127197+0000","last_active":"2024-09-21T12:01:34.127197+0000","last_peered":"2024-09-21T12:01:34.127197+0000","last_clean":"2024-09-21T12:01:34.127197+0000","last_became_active":"2024-09-21T11:57:45.514189+0000","last_became_peered":"2024-09-21T11:57:45.514189+0000","last_unstale":"2024-09-21T12:01:34.127197+0000","last_undegraded":"2024-09-21T12:01:34.127197+0000","last_fullsized":"2024-09-21T12:01:34.127197+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:01:34.127141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:01:34.127141+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:00.483585+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"471'42","reported_seq":1009,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571383+0000","last_change":"2024-09-21T12:01:21.966602+0000","last_active":"2024-09-21T12:01:27.571383+0000","last_peered":"2024-09-21T12:01:27.571383+0000","last_clean":"2024-09-21T12:01:27.571383+0000","last_became_active":"2024-09-21T12:01:21.966229+0000","last_became_peered":"2024-09-21T12:01:21.966229+0000","last_unstale":"2024-09-21T12:01:27.571383+0000","last_undegraded":"2024-09-21T12:01:27.571383+0000","last_fullsized":"2024-09-21T12:01:27.571383+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'42","last_scrub_stamp":"2024-09-21T12:01:16.214003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:16.214003+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:26.342431+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":203,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612940+0000","last_change":"2024-09-21T12:00:48.889353+0000","last_active":"2024-09-21T12:01:27.612940+0000","last_peered":"2024-09-21T12:01:27.612940+0000","last_clean":"2024-09-21T12:01:27.612940+0000","last_became_active":"2024-09-21T12:00:40.768342+0000","last_became_peered":"2024-09-21T12:00:40.768342+0000","last_unstale":"2024-09-21T12:01:27.612940+0000","last_undegraded":"2024-09-21T12:01:27.612940+0000","last_fullsized":"2024-09-21T12:01:27.612940+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T12:00:48.889300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:48.889300+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:13.137934+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"482'12","reported_seq":1259,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634150+0000","last_change":"2024-09-21T12:00:49.149552+0000","last_active":"2024-09-21T12:01:35.634150+0000","last_peered":"2024-09-21T12:01:35.634150+0000","last_clean":"2024-09-21T12:01:35.634150+0000","last_became_active":"2024-09-21T12:00:40.763011+0000","last_became_peered":"2024-09-21T12:00:40.763011+0000","last_unstale":"2024-09-21T12:01:35.634150+0000","last_undegraded":"2024-09-21T12:01:35.634150+0000","last_fullsized":"2024-09-21T12:01:35.634150+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'12","last_scrub_stamp":"2024-09-21T12:00:49.149500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:49.149500+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:52.037397+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":975,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613009+0000","last_change":"2024-09-21T12:01:11.872377+0000","last_active":"2024-09-21T12:01:27.613009+0000","last_peered":"2024-09-21T12:01:27.613009+0000","last_clean":"2024-09-21T12:01:27.613009+0000","last_became_active":"2024-09-21T11:56:34.581626+0000","last_became_peered":"2024-09-21T11:56:34.581626+0000","last_unstale":"2024-09-21T12:01:27.613009+0000","last_undegraded":"2024-09-21T12:01:27.613009+0000","last_fullsized":"2024-09-21T12:01:27.613009+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:11.872322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:11.872322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:37.849542+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":300,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:38.432838+0000","last_change":"2024-09-21T12:01:38.432838+0000","last_active":"2024-09-21T12:01:38.432838+0000","last_peered":"2024-09-21T12:01:38.432838+0000","last_clean":"2024-09-21T12:01:38.432838+0000","last_became_active":"2024-09-21T11:57:45.514310+0000","last_became_peered":"2024-09-21T11:57:45.514310+0000","last_unstale":"2024-09-21T12:01:38.432838+0000","last_undegraded":"2024-09-21T12:01:38.432838+0000","last_fullsized":"2024-09-21T12:01:38.432838+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:01:38.432773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:01:38.432773+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:03.257984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"471'30","reported_seq":767,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571850+0000","last_change":"2024-09-21T12:00:48.516078+0000","last_active":"2024-09-21T12:01:27.571850+0000","last_peered":"2024-09-21T12:01:27.571850+0000","last_clean":"2024-09-21T12:01:27.571850+0000","last_became_active":"2024-09-21T12:00:40.770823+0000","last_became_peered":"2024-09-21T12:00:40.770823+0000","last_unstale":"2024-09-21T12:01:27.571850+0000","last_undegraded":"2024-09-21T12:01:27.571850+0000","last_fullsized":"2024-09-21T12:01:27.571850+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'30","last_scrub_stamp":"2024-09-21T12:00:48.516016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:48.516016+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:58.232390+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1001,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614293+0000","last_change":"2024-09-21T12:01:19.230398+0000","last_active":"2024-09-21T12:01:27.614293+0000","last_peered":"2024-09-21T12:01:27.614293+0000","last_clean":"2024-09-21T12:01:27.614293+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T12:01:27.614293+0000","last_undegraded":"2024-09-21T12:01:27.614293+0000","last_fullsized":"2024-09-21T12:01:27.614293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:19.230332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:19.230332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:28.064213+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1232,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634158+0000","last_change":"2024-09-21T12:00:52.882768+0000","last_active":"2024-09-21T12:01:35.634158+0000","last_peered":"2024-09-21T12:01:35.634158+0000","last_clean":"2024-09-21T12:01:35.634158+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T12:01:35.634158+0000","last_undegraded":"2024-09-21T12:01:35.634158+0000","last_fullsized":"2024-09-21T12:01:35.634158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T12:00:52.882717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:52.882717+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:15.270057+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":964,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611327+0000","last_change":"2024-09-21T12:01:04.837641+0000","last_active":"2024-09-21T12:01:27.611327+0000","last_peered":"2024-09-21T12:01:27.611327+0000","last_clean":"2024-09-21T12:01:27.611327+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T12:01:27.611327+0000","last_undegraded":"2024-09-21T12:01:27.611327+0000","last_fullsized":"2024-09-21T12:01:27.611327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:04.837596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:04.837596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:29.766014+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":308,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612577+0000","last_change":"2024-09-21T12:01:25.549292+0000","last_active":"2024-09-21T12:01:27.612577+0000","last_peered":"2024-09-21T12:01:27.612577+0000","last_clean":"2024-09-21T12:01:27.612577+0000","last_became_active":"2024-09-21T11:57:45.518764+0000","last_became_peered":"2024-09-21T11:57:45.518764+0000","last_unstale":"2024-09-21T12:01:27.612577+0000","last_undegraded":"2024-09-21T12:01:27.612577+0000","last_fullsized":"2024-09-21T12:01:27.612577+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:01:25.549234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:01:25.549234+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:28.829769+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"471'39","reported_seq":1046,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613006+0000","last_change":"2024-09-21T12:00:53.851148+0000","last_active":"2024-09-21T12:01:27.613006+0000","last_peered":"2024-09-21T12:01:27.613006+0000","last_clean":"2024-09-21T12:01:27.613006+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T12:01:27.613006+0000","last_undegraded":"2024-09-21T12:01:27.613006+0000","last_fullsized":"2024-09-21T12:01:27.613006+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'39","last_scrub_stamp":"2024-09-21T12:00:53.851079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:53.851079+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:12.075446+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":671,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570992+0000","last_change":"2024-09-21T12:01:01.507916+0000","last_active":"2024-09-21T12:01:27.570992+0000","last_peered":"2024-09-21T12:01:27.570992+0000","last_clean":"2024-09-21T12:01:27.570992+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T12:01:27.570992+0000","last_undegraded":"2024-09-21T12:01:27.570992+0000","last_fullsized":"2024-09-21T12:01:27.570992+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:01.507853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:01.507853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:31.405641+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":953,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612076+0000","last_change":"2024-09-21T12:01:03.477305+0000","last_active":"2024-09-21T12:01:27.612076+0000","last_peered":"2024-09-21T12:01:27.612076+0000","last_clean":"2024-09-21T12:01:27.612076+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T12:01:27.612076+0000","last_undegraded":"2024-09-21T12:01:27.612076+0000","last_fullsized":"2024-09-21T12:01:27.612076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:03.477250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:03.477250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:21.505703+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":655,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570321+0000","last_change":"2024-09-21T12:00:30.282966+0000","last_active":"2024-09-21T12:01:27.570321+0000","last_peered":"2024-09-21T12:01:27.570321+0000","last_clean":"2024-09-21T12:01:27.570321+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T12:01:27.570321+0000","last_undegraded":"2024-09-21T12:01:27.570321+0000","last_fullsized":"2024-09-21T12:01:27.570321+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:30.282923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:30.282923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:42.684239+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":206,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613011+0000","last_change":"2024-09-21T12:00:49.113289+0000","last_active":"2024-09-21T12:01:27.613011+0000","last_peered":"2024-09-21T12:01:27.613011+0000","last_clean":"2024-09-21T12:01:27.613011+0000","last_became_active":"2024-09-21T11:57:45.515242+0000","last_became_peered":"2024-09-21T11:57:45.515242+0000","last_unstale":"2024-09-21T12:01:27.613011+0000","last_undegraded":"2024-09-21T12:01:27.613011+0000","last_fullsized":"2024-09-21T12:01:27.613011+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:49.113230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:49.113230+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.945088+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"471'55","reported_seq":873,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:40.634783+0000","last_change":"2024-09-21T12:01:27.518852+0000","last_active":"2024-09-21T12:01:40.634783+0000","last_peered":"2024-09-21T12:01:40.634783+0000","last_clean":"2024-09-21T12:01:40.634783+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T12:01:40.634783+0000","last_undegraded":"2024-09-21T12:01:40.634783+0000","last_fullsized":"2024-09-21T12:01:40.634783+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'55","last_scrub_stamp":"2024-09-21T12:01:27.518795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:27.518795+0000","objects_scrubbed":10,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:55.580498+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":515,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571305+0000","last_change":"2024-09-21T12:01:21.964098+0000","last_active":"2024-09-21T12:01:27.571305+0000","last_peered":"2024-09-21T12:01:27.571305+0000","last_clean":"2024-09-21T12:01:27.571305+0000","last_became_active":"2024-09-21T12:01:21.963623+0000","last_became_peered":"2024-09-21T12:01:21.963623+0000","last_unstale":"2024-09-21T12:01:27.571305+0000","last_undegraded":"2024-09-21T12:01:27.571305+0000","last_fullsized":"2024-09-21T12:01:27.571305+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T12:00:59.857482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:59.857482+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:25.350435+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":1001,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:29.262091+0000","last_change":"2024-09-21T12:01:29.262091+0000","last_active":"2024-09-21T12:01:29.262091+0000","last_peered":"2024-09-21T12:01:29.262091+0000","last_clean":"2024-09-21T12:01:29.262091+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T12:01:29.262091+0000","last_undegraded":"2024-09-21T12:01:29.262091+0000","last_fullsized":"2024-09-21T12:01:29.262091+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T12:01:29.262022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T12:01:29.262022+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:35.035827+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":206,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612512+0000","last_change":"2024-09-21T12:00:45.878432+0000","last_active":"2024-09-21T12:01:27.612512+0000","last_peered":"2024-09-21T12:01:27.612512+0000","last_clean":"2024-09-21T12:01:27.612512+0000","last_became_active":"2024-09-21T12:00:40.768563+0000","last_became_peered":"2024-09-21T12:00:40.768563+0000","last_unstale":"2024-09-21T12:01:27.612512+0000","last_undegraded":"2024-09-21T12:01:27.612512+0000","last_fullsized":"2024-09-21T12:01:27.612512+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:45.878382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:45.878382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:07.019792+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":952,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:38.064292+0000","last_change":"2024-09-21T12:01:38.064292+0000","last_active":"2024-09-21T12:01:38.064292+0000","last_peered":"2024-09-21T12:01:38.064292+0000","last_clean":"2024-09-21T12:01:38.064292+0000","last_became_active":"2024-09-21T12:00:40.774147+0000","last_became_peered":"2024-09-21T12:00:40.774147+0000","last_unstale":"2024-09-21T12:01:38.064292+0000","last_undegraded":"2024-09-21T12:01:38.064292+0000","last_fullsized":"2024-09-21T12:01:38.064292+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:38.064099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:38.064099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:51.144359+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":1097,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611666+0000","last_change":"2024-09-21T12:00:53.697047+0000","last_active":"2024-09-21T12:01:27.611666+0000","last_peered":"2024-09-21T12:01:27.611666+0000","last_clean":"2024-09-21T12:01:27.611666+0000","last_became_active":"2024-09-21T12:00:40.769976+0000","last_became_peered":"2024-09-21T12:00:40.769976+0000","last_unstale":"2024-09-21T12:01:27.611666+0000","last_undegraded":"2024-09-21T12:01:27.611666+0000","last_fullsized":"2024-09-21T12:01:27.611666+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:53.696991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:53.696991+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:06.952325+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"471'21","reported_seq":658,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611672+0000","last_change":"2024-09-21T12:00:21.629636+0000","last_active":"2024-09-21T12:01:27.611672+0000","last_peered":"2024-09-21T12:01:27.611672+0000","last_clean":"2024-09-21T12:01:27.611672+0000","last_became_active":"2024-09-21T11:57:58.530507+0000","last_became_peered":"2024-09-21T11:57:58.530507+0000","last_unstale":"2024-09-21T12:01:27.611672+0000","last_undegraded":"2024-09-21T12:01:27.611672+0000","last_fullsized":"2024-09-21T12:01:27.611672+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:00:21.629570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:21.629570+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:47.207499+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":667,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.569940+0000","last_change":"2024-09-21T12:00:29.254195+0000","last_active":"2024-09-21T12:01:27.569940+0000","last_peered":"2024-09-21T12:01:27.569940+0000","last_clean":"2024-09-21T12:01:27.569940+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T12:01:27.569940+0000","last_undegraded":"2024-09-21T12:01:27.569940+0000","last_fullsized":"2024-09-21T12:01:27.569940+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T12:00:29.254145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:29.254145+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:53.772710+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":995,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.347343+0000","last_change":"2024-09-21T12:01:35.347343+0000","last_active":"2024-09-21T12:01:35.347343+0000","last_peered":"2024-09-21T12:01:35.347343+0000","last_clean":"2024-09-21T12:01:35.347343+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T12:01:35.347343+0000","last_undegraded":"2024-09-21T12:01:35.347343+0000","last_fullsized":"2024-09-21T12:01:35.347343+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T12:01:35.347288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T12:01:35.347288+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:50.592627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":977,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:40.324176+0000","last_change":"2024-09-21T12:01:40.324176+0000","last_active":"2024-09-21T12:01:40.324176+0000","last_peered":"2024-09-21T12:01:40.324176+0000","last_clean":"2024-09-21T12:01:40.324176+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T12:01:40.324176+0000","last_undegraded":"2024-09-21T12:01:40.324176+0000","last_fullsized":"2024-09-21T12:01:40.324176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:40.324122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:40.324122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:00.588221+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":973,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613868+0000","last_change":"2024-09-21T12:01:08.802207+0000","last_active":"2024-09-21T12:01:27.613868+0000","last_peered":"2024-09-21T12:01:27.613868+0000","last_clean":"2024-09-21T12:01:27.613868+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T12:01:27.613868+0000","last_undegraded":"2024-09-21T12:01:27.613868+0000","last_fullsized":"2024-09-21T12:01:27.613868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:08.802156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:08.802156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:19.965570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":969,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570436+0000","last_change":"2024-09-21T12:00:51.525993+0000","last_active":"2024-09-21T12:01:27.570436+0000","last_peered":"2024-09-21T12:01:27.570436+0000","last_clean":"2024-09-21T12:01:27.570436+0000","last_became_active":"2024-09-21T12:00:40.759148+0000","last_became_peered":"2024-09-21T12:00:40.759148+0000","last_unstale":"2024-09-21T12:01:27.570436+0000","last_undegraded":"2024-09-21T12:01:27.570436+0000","last_fullsized":"2024-09-21T12:01:27.570436+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:51.525919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:51.525919+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:18.249569+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"471'36","reported_seq":220,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611339+0000","last_change":"2024-09-21T12:01:14.750890+0000","last_active":"2024-09-21T12:01:27.611339+0000","last_peered":"2024-09-21T12:01:27.611339+0000","last_clean":"2024-09-21T12:01:27.611339+0000","last_became_active":"2024-09-21T11:57:58.527381+0000","last_became_peered":"2024-09-21T11:57:58.527381+0000","last_unstale":"2024-09-21T12:01:27.611339+0000","last_undegraded":"2024-09-21T12:01:27.611339+0000","last_fullsized":"2024-09-21T12:01:27.611339+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'36","last_scrub_stamp":"2024-09-21T12:01:14.750830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:14.750830+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:24.597446+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1009,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611965+0000","last_change":"2024-09-21T12:00:51.836919+0000","last_active":"2024-09-21T12:01:27.611965+0000","last_peered":"2024-09-21T12:01:27.611965+0000","last_clean":"2024-09-21T12:01:27.611965+0000","last_became_active":"2024-09-21T12:00:40.768771+0000","last_became_peered":"2024-09-21T12:00:40.768771+0000","last_unstale":"2024-09-21T12:01:27.611965+0000","last_undegraded":"2024-09-21T12:01:27.611965+0000","last_fullsized":"2024-09-21T12:01:27.611965+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:51.836873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:51.836873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:19.042519+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":2311,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:39.928624+0000","last_change":"2024-09-21T12:01:16.865531+0000","last_active":"2024-09-21T12:01:39.928624+0000","last_peered":"2024-09-21T12:01:39.928624+0000","last_clean":"2024-09-21T12:01:39.928624+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T12:01:39.928624+0000","last_undegraded":"2024-09-21T12:01:39.928624+0000","last_fullsized":"2024-09-21T12:01:39.928624+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T12:01:16.865481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T12:01:16.865481+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:36.557499+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1291,"num_read_kb":1286,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":982,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613483+0000","last_change":"2024-09-21T12:00:39.121692+0000","last_active":"2024-09-21T12:01:27.613483+0000","last_peered":"2024-09-21T12:01:27.613483+0000","last_clean":"2024-09-21T12:01:27.613483+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T12:01:27.613483+0000","last_undegraded":"2024-09-21T12:01:27.613483+0000","last_fullsized":"2024-09-21T12:01:27.613483+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:39.121641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:39.121641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:56.458399+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":655,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:29.511642+0000","last_change":"2024-09-21T12:01:29.511597+0000","last_active":"2024-09-21T12:01:29.511642+0000","last_peered":"2024-09-21T12:01:29.511642+0000","last_clean":"2024-09-21T12:01:29.511642+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T12:01:29.511642+0000","last_undegraded":"2024-09-21T12:01:29.511642+0000","last_fullsized":"2024-09-21T12:01:29.511642+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:29.511547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:29.511547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:42.697292+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"482'42","reported_seq":1319,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634392+0000","last_change":"2024-09-21T12:00:55.389722+0000","last_active":"2024-09-21T12:01:35.634392+0000","last_peered":"2024-09-21T12:01:35.634392+0000","last_clean":"2024-09-21T12:01:35.634392+0000","last_became_active":"2024-09-21T12:00:40.763382+0000","last_became_peered":"2024-09-21T12:00:40.763382+0000","last_unstale":"2024-09-21T12:01:35.634392+0000","last_undegraded":"2024-09-21T12:01:35.634392+0000","last_fullsized":"2024-09-21T12:01:35.634392+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'42","last_scrub_stamp":"2024-09-21T12:00:55.389641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:55.389641+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:58.671006+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":970,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571310+0000","last_change":"2024-09-21T12:01:25.155304+0000","last_active":"2024-09-21T12:01:27.571310+0000","last_peered":"2024-09-21T12:01:27.571310+0000","last_clean":"2024-09-21T12:01:27.571310+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T12:01:27.571310+0000","last_undegraded":"2024-09-21T12:01:27.571310+0000","last_fullsized":"2024-09-21T12:01:27.571310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:25.155211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:25.155211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:45.135538+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":1097,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:36.839540+0000","last_change":"2024-09-21T12:01:36.839540+0000","last_active":"2024-09-21T12:01:36.839540+0000","last_peered":"2024-09-21T12:01:36.839540+0000","last_clean":"2024-09-21T12:01:36.839540+0000","last_became_active":"2024-09-21T11:57:58.528825+0000","last_became_peered":"2024-09-21T11:57:58.528825+0000","last_unstale":"2024-09-21T12:01:36.839540+0000","last_undegraded":"2024-09-21T12:01:36.839540+0000","last_fullsized":"2024-09-21T12:01:36.839540+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:01:36.839482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:01:36.839482+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:00.643537+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":961,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613812+0000","last_change":"2024-09-21T12:01:02.717098+0000","last_active":"2024-09-21T12:01:27.613812+0000","last_peered":"2024-09-21T12:01:27.613812+0000","last_clean":"2024-09-21T12:01:27.613812+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T12:01:27.613812+0000","last_undegraded":"2024-09-21T12:01:27.613812+0000","last_fullsized":"2024-09-21T12:01:27.613812+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:02.717040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:02.717040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:09.319774+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":983,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613779+0000","last_change":"2024-09-21T12:00:47.031989+0000","last_active":"2024-09-21T12:01:27.613779+0000","last_peered":"2024-09-21T12:01:27.613779+0000","last_clean":"2024-09-21T12:01:27.613779+0000","last_became_active":"2024-09-21T12:00:40.762491+0000","last_became_peered":"2024-09-21T12:00:40.762491+0000","last_unstale":"2024-09-21T12:01:27.613779+0000","last_undegraded":"2024-09-21T12:01:27.613779+0000","last_fullsized":"2024-09-21T12:01:27.613779+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:47.031940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:47.031940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:06.826727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"471'27","reported_seq":972,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571796+0000","last_change":"2024-09-21T12:01:21.967462+0000","last_active":"2024-09-21T12:01:27.571796+0000","last_peered":"2024-09-21T12:01:27.571796+0000","last_clean":"2024-09-21T12:01:27.571796+0000","last_became_active":"2024-09-21T12:01:21.966594+0000","last_became_peered":"2024-09-21T12:01:21.966594+0000","last_unstale":"2024-09-21T12:01:27.571796+0000","last_undegraded":"2024-09-21T12:01:27.571796+0000","last_fullsized":"2024-09-21T12:01:27.571796+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'27","last_scrub_stamp":"2024-09-21T12:00:53.421947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:53.421947+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:56.088796+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":451,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:42.427291+0000","last_change":"2024-09-21T12:01:42.427291+0000","last_active":"2024-09-21T12:01:42.427291+0000","last_peered":"2024-09-21T12:01:42.427291+0000","last_clean":"2024-09-21T12:01:42.427291+0000","last_became_active":"2024-09-21T12:01:22.098130+0000","last_became_peered":"2024-09-21T12:01:22.098130+0000","last_unstale":"2024-09-21T12:01:42.427291+0000","last_undegraded":"2024-09-21T12:01:42.427291+0000","last_fullsized":"2024-09-21T12:01:42.427291+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:42.427211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:42.427211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:01.322673+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":985,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613637+0000","last_change":"2024-09-21T12:01:03.276226+0000","last_active":"2024-09-21T12:01:27.613637+0000","last_peered":"2024-09-21T12:01:27.613637+0000","last_clean":"2024-09-21T12:01:27.613637+0000","last_became_active":"2024-09-21T12:00:40.761624+0000","last_became_peered":"2024-09-21T12:00:40.761624+0000","last_unstale":"2024-09-21T12:01:27.613637+0000","last_undegraded":"2024-09-21T12:01:27.613637+0000","last_fullsized":"2024-09-21T12:01:27.613637+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:03.276171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:03.276171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:30.681124+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":963,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571231+0000","last_change":"2024-09-21T12:01:03.155086+0000","last_active":"2024-09-21T12:01:27.571231+0000","last_peered":"2024-09-21T12:01:27.571231+0000","last_clean":"2024-09-21T12:01:27.571231+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T12:01:27.571231+0000","last_undegraded":"2024-09-21T12:01:27.571231+0000","last_fullsized":"2024-09-21T12:01:27.571231+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:03.155032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:03.155032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:21.014530+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":213,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611950+0000","last_change":"2024-09-21T12:00:52.210298+0000","last_active":"2024-09-21T12:01:27.611950+0000","last_peered":"2024-09-21T12:01:27.611950+0000","last_clean":"2024-09-21T12:01:27.611950+0000","last_became_active":"2024-09-21T11:57:45.534494+0000","last_became_peered":"2024-09-21T11:57:45.534494+0000","last_unstale":"2024-09-21T12:01:27.611950+0000","last_undegraded":"2024-09-21T12:01:27.611950+0000","last_fullsized":"2024-09-21T12:01:27.611950+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:52.210248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:52.210248+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:52.407043+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"471'45","reported_seq":1086,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.572266+0000","last_change":"2024-09-21T12:01:21.970735+0000","last_active":"2024-09-21T12:01:27.572266+0000","last_peered":"2024-09-21T12:01:27.572266+0000","last_clean":"2024-09-21T12:01:27.572266+0000","last_became_active":"2024-09-21T12:01:21.970150+0000","last_became_peered":"2024-09-21T12:01:21.970150+0000","last_unstale":"2024-09-21T12:01:27.572266+0000","last_undegraded":"2024-09-21T12:01:27.572266+0000","last_fullsized":"2024-09-21T12:01:27.572266+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'45","last_scrub_stamp":"2024-09-21T12:01:17.257005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:17.257005+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:20.434969+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1009,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612347+0000","last_change":"2024-09-21T12:00:47.384892+0000","last_active":"2024-09-21T12:01:27.612347+0000","last_peered":"2024-09-21T12:01:27.612347+0000","last_clean":"2024-09-21T12:01:27.612347+0000","last_became_active":"2024-09-21T12:00:40.764915+0000","last_became_peered":"2024-09-21T12:00:40.764915+0000","last_unstale":"2024-09-21T12:01:27.612347+0000","last_undegraded":"2024-09-21T12:01:27.612347+0000","last_fullsized":"2024-09-21T12:01:27.612347+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:47.384844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:47.384844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.952453+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":673,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570770+0000","last_change":"2024-09-21T12:00:53.526047+0000","last_active":"2024-09-21T12:01:27.570770+0000","last_peered":"2024-09-21T12:01:27.570770+0000","last_clean":"2024-09-21T12:01:27.570770+0000","last_became_active":"2024-09-21T12:00:40.760044+0000","last_became_peered":"2024-09-21T12:00:40.760044+0000","last_unstale":"2024-09-21T12:01:27.570770+0000","last_undegraded":"2024-09-21T12:01:27.570770+0000","last_fullsized":"2024-09-21T12:01:27.570770+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:53.525998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:53.525998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:54.986795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":984,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611792+0000","last_change":"2024-09-21T12:01:13.298392+0000","last_active":"2024-09-21T12:01:27.611792+0000","last_peered":"2024-09-21T12:01:27.611792+0000","last_clean":"2024-09-21T12:01:27.611792+0000","last_became_active":"2024-09-21T12:00:40.762281+0000","last_became_peered":"2024-09-21T12:00:40.762281+0000","last_unstale":"2024-09-21T12:01:27.611792+0000","last_undegraded":"2024-09-21T12:01:27.611792+0000","last_fullsized":"2024-09-21T12:01:27.611792+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:13.298338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:13.298338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:21.475928+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":1087,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612028+0000","last_change":"2024-09-21T12:00:39.762851+0000","last_active":"2024-09-21T12:01:27.612028+0000","last_peered":"2024-09-21T12:01:27.612028+0000","last_clean":"2024-09-21T12:01:27.612028+0000","last_became_active":"2024-09-21T11:57:45.516610+0000","last_became_peered":"2024-09-21T11:57:45.516610+0000","last_unstale":"2024-09-21T12:01:27.612028+0000","last_undegraded":"2024-09-21T12:01:27.612028+0000","last_fullsized":"2024-09-21T12:01:27.612028+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:39.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:39.762779+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:45.629829+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"471'30","reported_seq":928,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571808+0000","last_change":"2024-09-21T12:01:21.965260+0000","last_active":"2024-09-21T12:01:27.571808+0000","last_peered":"2024-09-21T12:01:27.571808+0000","last_clean":"2024-09-21T12:01:27.571808+0000","last_became_active":"2024-09-21T12:01:21.964870+0000","last_became_peered":"2024-09-21T12:01:21.964870+0000","last_unstale":"2024-09-21T12:01:27.571808+0000","last_undegraded":"2024-09-21T12:01:27.571808+0000","last_fullsized":"2024-09-21T12:01:27.571808+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'30","last_scrub_stamp":"2024-09-21T12:00:44.093780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:44.093780+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:46.496540+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":978,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571633+0000","last_change":"2024-09-21T12:01:04.133660+0000","last_active":"2024-09-21T12:01:27.571633+0000","last_peered":"2024-09-21T12:01:27.571633+0000","last_clean":"2024-09-21T12:01:27.571633+0000","last_became_active":"2024-09-21T12:00:40.775319+0000","last_became_peered":"2024-09-21T12:00:40.775319+0000","last_unstale":"2024-09-21T12:01:27.571633+0000","last_undegraded":"2024-09-21T12:01:27.571633+0000","last_fullsized":"2024-09-21T12:01:27.571633+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:04.133595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:04.133595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:30.659592+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1246,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634173+0000","last_change":"2024-09-21T12:01:22.230453+0000","last_active":"2024-09-21T12:01:35.634173+0000","last_peered":"2024-09-21T12:01:35.634173+0000","last_clean":"2024-09-21T12:01:35.634173+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T12:01:35.634173+0000","last_undegraded":"2024-09-21T12:01:35.634173+0000","last_fullsized":"2024-09-21T12:01:35.634173+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T12:01:22.230403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:22.230403+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:28.579500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":761,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.572172+0000","last_change":"2024-09-21T12:01:21.970040+0000","last_active":"2024-09-21T12:01:27.572172+0000","last_peered":"2024-09-21T12:01:27.572172+0000","last_clean":"2024-09-21T12:01:27.572172+0000","last_became_active":"2024-09-21T12:01:21.968677+0000","last_became_peered":"2024-09-21T12:01:21.968677+0000","last_unstale":"2024-09-21T12:01:27.572172+0000","last_undegraded":"2024-09-21T12:01:27.572172+0000","last_fullsized":"2024-09-21T12:01:27.572172+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:55.484970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:55.484970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:04.871190+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"471'21","reported_seq":1035,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612873+0000","last_change":"2024-09-21T12:00:49.872758+0000","last_active":"2024-09-21T12:01:27.612873+0000","last_peered":"2024-09-21T12:01:27.612873+0000","last_clean":"2024-09-21T12:01:27.612873+0000","last_became_active":"2024-09-21T12:00:40.764620+0000","last_became_peered":"2024-09-21T12:00:40.764620+0000","last_unstale":"2024-09-21T12:01:27.612873+0000","last_undegraded":"2024-09-21T12:01:27.612873+0000","last_fullsized":"2024-09-21T12:01:27.612873+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:00:49.872707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:49.872707+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:01.339809+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1000,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612618+0000","last_change":"2024-09-21T12:00:34.050812+0000","last_active":"2024-09-21T12:01:27.612618+0000","last_peered":"2024-09-21T12:01:27.612618+0000","last_clean":"2024-09-21T12:01:27.612618+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T12:01:27.612618+0000","last_undegraded":"2024-09-21T12:01:27.612618+0000","last_fullsized":"2024-09-21T12:01:27.612618+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:34.050765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:34.050765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:46.281564+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":986,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611935+0000","last_change":"2024-09-21T12:01:26.269162+0000","last_active":"2024-09-21T12:01:27.611935+0000","last_peered":"2024-09-21T12:01:27.611935+0000","last_clean":"2024-09-21T12:01:27.611935+0000","last_became_active":"2024-09-21T12:00:40.772986+0000","last_became_peered":"2024-09-21T12:00:40.772986+0000","last_unstale":"2024-09-21T12:01:27.611935+0000","last_undegraded":"2024-09-21T12:01:27.611935+0000","last_fullsized":"2024-09-21T12:01:27.611935+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:26.269116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:26.269116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:40.794995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":196,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611859+0000","last_change":"2024-09-21T12:01:21.793451+0000","last_active":"2024-09-21T12:01:27.611859+0000","last_peered":"2024-09-21T12:01:27.611859+0000","last_clean":"2024-09-21T12:01:27.611859+0000","last_became_active":"2024-09-21T11:57:58.529278+0000","last_became_peered":"2024-09-21T11:57:58.529278+0000","last_unstale":"2024-09-21T12:01:27.611859+0000","last_undegraded":"2024-09-21T12:01:27.611859+0000","last_fullsized":"2024-09-21T12:01:27.611859+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:21.793391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:21.793391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:27.944567+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"471'51","reported_seq":1097,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611454+0000","last_change":"2024-09-21T12:00:58.443531+0000","last_active":"2024-09-21T12:01:27.611454+0000","last_peered":"2024-09-21T12:01:27.611454+0000","last_clean":"2024-09-21T12:01:27.611454+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T12:01:27.611454+0000","last_undegraded":"2024-09-21T12:01:27.611454+0000","last_fullsized":"2024-09-21T12:01:27.611454+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'51","last_scrub_stamp":"2024-09-21T12:00:58.443478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:58.443478+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:20.410139+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":968,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614060+0000","last_change":"2024-09-21T12:00:35.799954+0000","last_active":"2024-09-21T12:01:27.614060+0000","last_peered":"2024-09-21T12:01:27.614060+0000","last_clean":"2024-09-21T12:01:27.614060+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T12:01:27.614060+0000","last_undegraded":"2024-09-21T12:01:27.614060+0000","last_fullsized":"2024-09-21T12:01:27.614060+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:35.799871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:35.799871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:47.164347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":969,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611577+0000","last_change":"2024-09-21T12:01:04.373627+0000","last_active":"2024-09-21T12:01:27.611577+0000","last_peered":"2024-09-21T12:01:27.611577+0000","last_clean":"2024-09-21T12:01:27.611577+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T12:01:27.611577+0000","last_undegraded":"2024-09-21T12:01:27.611577+0000","last_fullsized":"2024-09-21T12:01:27.611577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:04.373572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:04.373572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:09.392332+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":980,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:39.899206+0000","last_change":"2024-09-21T12:01:39.899206+0000","last_active":"2024-09-21T12:01:39.899206+0000","last_peered":"2024-09-21T12:01:39.899206+0000","last_clean":"2024-09-21T12:01:39.899206+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T12:01:39.899206+0000","last_undegraded":"2024-09-21T12:01:39.899206+0000","last_fullsized":"2024-09-21T12:01:39.899206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:39.899146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:39.899146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:06.239649+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"471'24","reported_seq":963,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571199+0000","last_change":"2024-09-21T12:01:21.963208+0000","last_active":"2024-09-21T12:01:27.571199+0000","last_peered":"2024-09-21T12:01:27.571199+0000","last_clean":"2024-09-21T12:01:27.571199+0000","last_became_active":"2024-09-21T12:01:21.962746+0000","last_became_peered":"2024-09-21T12:01:21.962746+0000","last_unstale":"2024-09-21T12:01:27.571199+0000","last_undegraded":"2024-09-21T12:01:27.571199+0000","last_fullsized":"2024-09-21T12:01:27.571199+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'24","last_scrub_stamp":"2024-09-21T12:00:43.435914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:43.435914+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.607209+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1038,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613736+0000","last_change":"2024-09-21T12:00:56.708295+0000","last_active":"2024-09-21T12:01:27.613736+0000","last_peered":"2024-09-21T12:01:27.613736+0000","last_clean":"2024-09-21T12:01:27.613736+0000","last_became_active":"2024-09-21T12:00:40.770111+0000","last_became_peered":"2024-09-21T12:00:40.770111+0000","last_unstale":"2024-09-21T12:01:27.613736+0000","last_undegraded":"2024-09-21T12:01:27.613736+0000","last_fullsized":"2024-09-21T12:01:27.613736+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:56.708253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:56.708253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:01.065742+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":983,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611127+0000","last_change":"2024-09-21T12:01:21.242722+0000","last_active":"2024-09-21T12:01:27.611127+0000","last_peered":"2024-09-21T12:01:27.611127+0000","last_clean":"2024-09-21T12:01:27.611127+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T12:01:27.611127+0000","last_undegraded":"2024-09-21T12:01:27.611127+0000","last_fullsized":"2024-09-21T12:01:27.611127+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:21.242668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:21.242668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:37.681983+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":963,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571191+0000","last_change":"2024-09-21T12:01:02.191279+0000","last_active":"2024-09-21T12:01:27.571191+0000","last_peered":"2024-09-21T12:01:27.571191+0000","last_clean":"2024-09-21T12:01:27.571191+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T12:01:27.571191+0000","last_undegraded":"2024-09-21T12:01:27.571191+0000","last_fullsized":"2024-09-21T12:01:27.571191+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:02.191218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:02.191218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:02.649439+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"471'39","reported_seq":1041,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:31.252581+0000","last_change":"2024-09-21T12:01:31.252581+0000","last_active":"2024-09-21T12:01:31.252581+0000","last_peered":"2024-09-21T12:01:31.252581+0000","last_clean":"2024-09-21T12:01:31.252581+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T12:01:31.252581+0000","last_undegraded":"2024-09-21T12:01:31.252581+0000","last_fullsized":"2024-09-21T12:01:31.252581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'39","last_scrub_stamp":"2024-09-21T12:01:31.252491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:31.252491+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:34.644925+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":1041,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613677+0000","last_change":"2024-09-21T12:01:17.880609+0000","last_active":"2024-09-21T12:01:27.613677+0000","last_peered":"2024-09-21T12:01:27.613677+0000","last_clean":"2024-09-21T12:01:27.613677+0000","last_became_active":"2024-09-21T12:00:40.770271+0000","last_became_peered":"2024-09-21T12:00:40.770271+0000","last_unstale":"2024-09-21T12:01:27.613677+0000","last_undegraded":"2024-09-21T12:01:27.613677+0000","last_fullsized":"2024-09-21T12:01:27.613677+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T12:01:17.880528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:17.880528+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:28.339143+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"482'10","reported_seq":1250,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634301+0000","last_change":"2024-09-21T12:00:43.723921+0000","last_active":"2024-09-21T12:01:35.634301+0000","last_peered":"2024-09-21T12:01:35.634301+0000","last_clean":"2024-09-21T12:01:35.634301+0000","last_became_active":"2024-09-21T12:00:40.767189+0000","last_became_peered":"2024-09-21T12:00:40.767189+0000","last_unstale":"2024-09-21T12:01:35.634301+0000","last_undegraded":"2024-09-21T12:01:35.634301+0000","last_fullsized":"2024-09-21T12:01:35.634301+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'10","last_scrub_stamp":"2024-09-21T12:00:43.723879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:43.723879+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:58.537685+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":956,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614352+0000","last_change":"2024-09-21T12:00:40.084762+0000","last_active":"2024-09-21T12:01:27.614352+0000","last_peered":"2024-09-21T12:01:27.614352+0000","last_clean":"2024-09-21T12:01:27.614352+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T12:01:27.614352+0000","last_undegraded":"2024-09-21T12:01:27.614352+0000","last_fullsized":"2024-09-21T12:01:27.614352+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:40.084714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:40.084714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:40.121785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1006,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612877+0000","last_change":"2024-09-21T12:01:07.492251+0000","last_active":"2024-09-21T12:01:27.612877+0000","last_peered":"2024-09-21T12:01:27.612877+0000","last_clean":"2024-09-21T12:01:27.612877+0000","last_became_active":"2024-09-21T11:57:58.527156+0000","last_became_peered":"2024-09-21T11:57:58.527156+0000","last_unstale":"2024-09-21T12:01:27.612877+0000","last_undegraded":"2024-09-21T12:01:27.612877+0000","last_fullsized":"2024-09-21T12:01:27.612877+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:07.492190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:07.492190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:22.590948+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"471'18","reported_seq":1005,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571893+0000","last_change":"2024-09-21T12:01:11.150673+0000","last_active":"2024-09-21T12:01:27.571893+0000","last_peered":"2024-09-21T12:01:27.571893+0000","last_clean":"2024-09-21T12:01:27.571893+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T12:01:27.571893+0000","last_undegraded":"2024-09-21T12:01:27.571893+0000","last_fullsized":"2024-09-21T12:01:27.571893+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'18","last_scrub_stamp":"2024-09-21T12:01:11.150621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:11.150621+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:32.972824+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1013,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:32.229743+0000","last_change":"2024-09-21T12:01:32.229743+0000","last_active":"2024-09-21T12:01:32.229743+0000","last_peered":"2024-09-21T12:01:32.229743+0000","last_clean":"2024-09-21T12:01:32.229743+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T12:01:32.229743+0000","last_undegraded":"2024-09-21T12:01:32.229743+0000","last_fullsized":"2024-09-21T12:01:32.229743+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:32.229687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:32.229687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:45.121554+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1004,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613811+0000","last_change":"2024-09-21T12:01:00.765910+0000","last_active":"2024-09-21T12:01:27.613811+0000","last_peered":"2024-09-21T12:01:27.613811+0000","last_clean":"2024-09-21T12:01:27.613811+0000","last_became_active":"2024-09-21T12:00:40.771109+0000","last_became_peered":"2024-09-21T12:00:40.771109+0000","last_unstale":"2024-09-21T12:01:27.613811+0000","last_undegraded":"2024-09-21T12:01:27.613811+0000","last_fullsized":"2024-09-21T12:01:27.613811+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:00.765857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:00.765857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:26.858419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"471'21","reported_seq":1005,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612941+0000","last_change":"2024-09-21T12:01:25.670129+0000","last_active":"2024-09-21T12:01:27.612941+0000","last_peered":"2024-09-21T12:01:27.612941+0000","last_clean":"2024-09-21T12:01:27.612941+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T12:01:27.612941+0000","last_undegraded":"2024-09-21T12:01:27.612941+0000","last_fullsized":"2024-09-21T12:01:27.612941+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:01:25.670072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:25.670072+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:26.697884+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":971,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612241+0000","last_change":"2024-09-21T12:01:21.902276+0000","last_active":"2024-09-21T12:01:27.612241+0000","last_peered":"2024-09-21T12:01:27.612241+0000","last_clean":"2024-09-21T12:01:27.612241+0000","last_became_active":"2024-09-21T12:00:40.768605+0000","last_became_peered":"2024-09-21T12:00:40.768605+0000","last_unstale":"2024-09-21T12:01:27.612241+0000","last_undegraded":"2024-09-21T12:01:27.612241+0000","last_fullsized":"2024-09-21T12:01:27.612241+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:21.902221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:21.902221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:33.292148+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":982,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571671+0000","last_change":"2024-09-21T12:00:11.248254+0000","last_active":"2024-09-21T12:01:27.571671+0000","last_peered":"2024-09-21T12:01:27.571671+0000","last_clean":"2024-09-21T12:01:27.571671+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T12:01:27.571671+0000","last_undegraded":"2024-09-21T12:01:27.571671+0000","last_fullsized":"2024-09-21T12:01:27.571671+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:11.248185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:11.248185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:38.996521+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":968,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612519+0000","last_change":"2024-09-21T12:01:01.307806+0000","last_active":"2024-09-21T12:01:27.612519+0000","last_peered":"2024-09-21T12:01:27.612519+0000","last_clean":"2024-09-21T12:01:27.612519+0000","last_became_active":"2024-09-21T12:00:40.759897+0000","last_became_peered":"2024-09-21T12:00:40.759897+0000","last_unstale":"2024-09-21T12:01:27.612519+0000","last_undegraded":"2024-09-21T12:01:27.612519+0000","last_fullsized":"2024-09-21T12:01:27.612519+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:01.307744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:01.307744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:05.151218+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"471'21","reported_seq":1005,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614228+0000","last_change":"2024-09-21T12:01:04.245620+0000","last_active":"2024-09-21T12:01:27.614228+0000","last_peered":"2024-09-21T12:01:27.614228+0000","last_clean":"2024-09-21T12:01:27.614228+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T12:01:27.614228+0000","last_undegraded":"2024-09-21T12:01:27.614228+0000","last_fullsized":"2024-09-21T12:01:27.614228+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:01:04.245503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:04.245503+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:31.561136+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":991,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:36.696204+0000","last_change":"2024-09-21T12:01:36.696204+0000","last_active":"2024-09-21T12:01:36.696204+0000","last_peered":"2024-09-21T12:01:36.696204+0000","last_clean":"2024-09-21T12:01:36.696204+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T12:01:36.696204+0000","last_undegraded":"2024-09-21T12:01:36.696204+0000","last_fullsized":"2024-09-21T12:01:36.696204+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:36.696155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:36.696155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:41.654052+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1206,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634451+0000","last_change":"2024-09-21T12:00:57.169707+0000","last_active":"2024-09-21T12:01:35.634451+0000","last_peered":"2024-09-21T12:01:35.634451+0000","last_clean":"2024-09-21T12:01:35.634451+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T12:01:35.634451+0000","last_undegraded":"2024-09-21T12:01:35.634451+0000","last_fullsized":"2024-09-21T12:01:35.634451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T12:00:57.169657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:57.169657+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:21.434751+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":536,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:32.449161+0000","last_change":"2024-09-21T12:01:32.449161+0000","last_active":"2024-09-21T12:01:32.449161+0000","last_peered":"2024-09-21T12:01:32.449161+0000","last_clean":"2024-09-21T12:01:32.449161+0000","last_became_active":"2024-09-21T12:01:21.970735+0000","last_became_peered":"2024-09-21T12:01:21.970735+0000","last_unstale":"2024-09-21T12:01:32.449161+0000","last_undegraded":"2024-09-21T12:01:32.449161+0000","last_fullsized":"2024-09-21T12:01:32.449161+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:32.449113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:32.449113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:43.974374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"471'30","reported_seq":1032,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571641+0000","last_change":"2024-09-21T12:01:22.179206+0000","last_active":"2024-09-21T12:01:27.571641+0000","last_peered":"2024-09-21T12:01:27.571641+0000","last_clean":"2024-09-21T12:01:27.571641+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T12:01:27.571641+0000","last_undegraded":"2024-09-21T12:01:27.571641+0000","last_fullsized":"2024-09-21T12:01:27.571641+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'30","last_scrub_stamp":"2024-09-21T12:01:22.179153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:22.179153+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:48.533988+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":979,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613441+0000","last_change":"2024-09-21T12:00:49.745941+0000","last_active":"2024-09-21T12:01:27.613441+0000","last_peered":"2024-09-21T12:01:27.613441+0000","last_clean":"2024-09-21T12:01:27.613441+0000","last_became_active":"2024-09-21T12:00:40.758596+0000","last_became_peered":"2024-09-21T12:00:40.758596+0000","last_unstale":"2024-09-21T12:01:27.613441+0000","last_undegraded":"2024-09-21T12:01:27.613441+0000","last_fullsized":"2024-09-21T12:01:27.613441+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:49.745886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:49.745886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.678659+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":1267,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634300+0000","last_change":"2024-09-21T12:01:12.210642+0000","last_active":"2024-09-21T12:01:35.634300+0000","last_peered":"2024-09-21T12:01:35.634300+0000","last_clean":"2024-09-21T12:01:35.634300+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T12:01:35.634300+0000","last_undegraded":"2024-09-21T12:01:35.634300+0000","last_fullsized":"2024-09-21T12:01:35.634300+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T12:01:12.210581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:12.210581+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:16.606737+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":196,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:33.705254+0000","last_change":"2024-09-21T12:01:33.705254+0000","last_active":"2024-09-21T12:01:33.705254+0000","last_peered":"2024-09-21T12:01:33.705254+0000","last_clean":"2024-09-21T12:01:33.705254+0000","last_became_active":"2024-09-21T11:57:58.533810+0000","last_became_peered":"2024-09-21T11:57:58.533810+0000","last_unstale":"2024-09-21T12:01:33.705254+0000","last_undegraded":"2024-09-21T12:01:33.705254+0000","last_fullsized":"2024-09-21T12:01:33.705254+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:33.705191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:33.705191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:45.161643+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"471'27","reported_seq":983,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571502+0000","last_change":"2024-09-21T12:01:21.967060+0000","last_active":"2024-09-21T12:01:27.571502+0000","last_peered":"2024-09-21T12:01:27.571502+0000","last_clean":"2024-09-21T12:01:27.571502+0000","last_became_active":"2024-09-21T12:01:21.966662+0000","last_became_peered":"2024-09-21T12:01:21.966662+0000","last_unstale":"2024-09-21T12:01:27.571502+0000","last_undegraded":"2024-09-21T12:01:27.571502+0000","last_fullsized":"2024-09-21T12:01:27.571502+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'27","last_scrub_stamp":"2024-09-21T12:00:42.330951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:42.330951+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:48.090414+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":516,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571910+0000","last_change":"2024-09-21T12:01:22.098173+0000","last_active":"2024-09-21T12:01:27.571910+0000","last_peered":"2024-09-21T12:01:27.571910+0000","last_clean":"2024-09-21T12:01:27.571910+0000","last_became_active":"2024-09-21T12:01:22.097597+0000","last_became_peered":"2024-09-21T12:01:22.097597+0000","last_unstale":"2024-09-21T12:01:27.571910+0000","last_undegraded":"2024-09-21T12:01:27.571910+0000","last_fullsized":"2024-09-21T12:01:27.571910+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:05.752333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:05.752333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:24.854306+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":670,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571727+0000","last_change":"2024-09-21T12:00:58.509822+0000","last_active":"2024-09-21T12:01:27.571727+0000","last_peered":"2024-09-21T12:01:27.571727+0000","last_clean":"2024-09-21T12:01:27.571727+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T12:01:27.571727+0000","last_undegraded":"2024-09-21T12:01:27.571727+0000","last_fullsized":"2024-09-21T12:01:27.571727+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:58.509775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:58.509775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:12.553557+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":955,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570353+0000","last_change":"2024-09-21T12:00:40.049914+0000","last_active":"2024-09-21T12:01:27.570353+0000","last_peered":"2024-09-21T12:01:27.570353+0000","last_clean":"2024-09-21T12:01:27.570353+0000","last_became_active":"2024-09-21T11:57:58.531213+0000","last_became_peered":"2024-09-21T11:57:58.531213+0000","last_unstale":"2024-09-21T12:01:27.570353+0000","last_undegraded":"2024-09-21T12:01:27.570353+0000","last_fullsized":"2024-09-21T12:01:27.570353+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:40.049860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:40.049860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:56.937093+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"471'21","reported_seq":690,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571973+0000","last_change":"2024-09-21T12:01:09.639759+0000","last_active":"2024-09-21T12:01:27.571973+0000","last_peered":"2024-09-21T12:01:27.571973+0000","last_clean":"2024-09-21T12:01:27.571973+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T12:01:27.571973+0000","last_undegraded":"2024-09-21T12:01:27.571973+0000","last_fullsized":"2024-09-21T12:01:27.571973+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:01:09.639704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:09.639704+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:38.484984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":970,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614034+0000","last_change":"2024-09-21T12:00:46.805427+0000","last_active":"2024-09-21T12:01:27.614034+0000","last_peered":"2024-09-21T12:01:27.614034+0000","last_clean":"2024-09-21T12:01:27.614034+0000","last_became_active":"2024-09-21T12:00:40.768603+0000","last_became_peered":"2024-09-21T12:00:40.768603+0000","last_unstale":"2024-09-21T12:01:27.614034+0000","last_undegraded":"2024-09-21T12:01:27.614034+0000","last_fullsized":"2024-09-21T12:01:27.614034+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:46.805382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:46.805382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:12.354004+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":967,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:39.026421+0000","last_change":"2024-09-21T12:01:39.026421+0000","last_active":"2024-09-21T12:01:39.026421+0000","last_peered":"2024-09-21T12:01:39.026421+0000","last_clean":"2024-09-21T12:01:39.026421+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T12:01:39.026421+0000","last_undegraded":"2024-09-21T12:01:39.026421+0000","last_fullsized":"2024-09-21T12:01:39.026421+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:39.026373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:39.026373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:53.324449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":994,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611446+0000","last_change":"2024-09-21T12:00:27.764240+0000","last_active":"2024-09-21T12:01:27.611446+0000","last_peered":"2024-09-21T12:01:27.611446+0000","last_clean":"2024-09-21T12:01:27.611446+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T12:01:27.611446+0000","last_undegraded":"2024-09-21T12:01:27.611446+0000","last_fullsized":"2024-09-21T12:01:27.611446+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T12:00:27.764155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:27.764155+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:49.090379+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"471'27","reported_seq":1008,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570943+0000","last_change":"2024-09-21T12:01:21.219311+0000","last_active":"2024-09-21T12:01:27.570943+0000","last_peered":"2024-09-21T12:01:27.570943+0000","last_clean":"2024-09-21T12:01:27.570943+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T12:01:27.570943+0000","last_undegraded":"2024-09-21T12:01:27.570943+0000","last_fullsized":"2024-09-21T12:01:27.570943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'27","last_scrub_stamp":"2024-09-21T12:01:21.219229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:21.219229+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:30.105512+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":989,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612890+0000","last_change":"2024-09-21T12:01:11.173623+0000","last_active":"2024-09-21T12:01:27.612890+0000","last_peered":"2024-09-21T12:01:27.612890+0000","last_clean":"2024-09-21T12:01:27.612890+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T12:01:27.612890+0000","last_undegraded":"2024-09-21T12:01:27.612890+0000","last_fullsized":"2024-09-21T12:01:27.612890+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:11.173564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:11.173564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:27.386819+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":955,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612347+0000","last_change":"2024-09-21T12:00:58.765038+0000","last_active":"2024-09-21T12:01:27.612347+0000","last_peered":"2024-09-21T12:01:27.612347+0000","last_clean":"2024-09-21T12:01:27.612347+0000","last_became_active":"2024-09-21T12:00:40.759290+0000","last_became_peered":"2024-09-21T12:00:40.759290+0000","last_unstale":"2024-09-21T12:01:27.612347+0000","last_undegraded":"2024-09-21T12:01:27.612347+0000","last_fullsized":"2024-09-21T12:01:27.612347+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:58.764974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:58.764974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:27.673279+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":971,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:31.111709+0000","last_change":"2024-09-21T12:01:31.111709+0000","last_active":"2024-09-21T12:01:31.111709+0000","last_peered":"2024-09-21T12:01:31.111709+0000","last_clean":"2024-09-21T12:01:31.111709+0000","last_became_active":"2024-09-21T11:57:58.537121+0000","last_became_peered":"2024-09-21T11:57:58.537121+0000","last_unstale":"2024-09-21T12:01:31.111709+0000","last_undegraded":"2024-09-21T12:01:31.111709+0000","last_fullsized":"2024-09-21T12:01:31.111709+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:31.111660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:31.111660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:57.578005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"471'12","reported_seq":1006,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570469+0000","last_change":"2024-09-21T12:00:31.069099+0000","last_active":"2024-09-21T12:01:27.570469+0000","last_peered":"2024-09-21T12:01:27.570469+0000","last_clean":"2024-09-21T12:01:27.570469+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T12:01:27.570469+0000","last_undegraded":"2024-09-21T12:01:27.570469+0000","last_fullsized":"2024-09-21T12:01:27.570469+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'12","last_scrub_stamp":"2024-09-21T12:00:31.069038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:31.069038+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:46.062255+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":950,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612961+0000","last_change":"2024-09-21T12:00:57.855315+0000","last_active":"2024-09-21T12:01:27.612961+0000","last_peered":"2024-09-21T12:01:27.612961+0000","last_clean":"2024-09-21T12:01:27.612961+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T12:01:27.612961+0000","last_undegraded":"2024-09-21T12:01:27.612961+0000","last_fullsized":"2024-09-21T12:01:27.612961+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:57.855253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:57.855253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:06.707025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":993,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612077+0000","last_change":"2024-09-21T12:00:59.490747+0000","last_active":"2024-09-21T12:01:27.612077+0000","last_peered":"2024-09-21T12:01:27.612077+0000","last_clean":"2024-09-21T12:01:27.612077+0000","last_became_active":"2024-09-21T12:00:40.759826+0000","last_became_peered":"2024-09-21T12:00:40.759826+0000","last_unstale":"2024-09-21T12:01:27.612077+0000","last_undegraded":"2024-09-21T12:01:27.612077+0000","last_fullsized":"2024-09-21T12:01:27.612077+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:59.490692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:59.490692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:05.615698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":983,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611792+0000","last_change":"2024-09-21T12:01:03.761526+0000","last_active":"2024-09-21T12:01:27.611792+0000","last_peered":"2024-09-21T12:01:27.611792+0000","last_clean":"2024-09-21T12:01:27.611792+0000","last_became_active":"2024-09-21T12:00:40.769332+0000","last_became_peered":"2024-09-21T12:00:40.769332+0000","last_unstale":"2024-09-21T12:01:27.611792+0000","last_undegraded":"2024-09-21T12:01:27.611792+0000","last_fullsized":"2024-09-21T12:01:27.611792+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:03.761470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:03.761470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:24.060257+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":678,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570783+0000","last_change":"2024-09-21T12:00:44.397658+0000","last_active":"2024-09-21T12:01:27.570783+0000","last_peered":"2024-09-21T12:01:27.570783+0000","last_clean":"2024-09-21T12:01:27.570783+0000","last_became_active":"2024-09-21T12:00:40.771228+0000","last_became_peered":"2024-09-21T12:00:40.771228+0000","last_unstale":"2024-09-21T12:01:27.570783+0000","last_undegraded":"2024-09-21T12:01:27.570783+0000","last_fullsized":"2024-09-21T12:01:27.570783+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:44.397597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:44.397597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:09.751270+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":996,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:28.103594+0000","last_change":"2024-09-21T12:01:28.103594+0000","last_active":"2024-09-21T12:01:28.103594+0000","last_peered":"2024-09-21T12:01:28.103594+0000","last_clean":"2024-09-21T12:01:28.103594+0000","last_became_active":"2024-09-21T11:57:58.535015+0000","last_became_peered":"2024-09-21T11:57:58.535015+0000","last_unstale":"2024-09-21T12:01:28.103594+0000","last_undegraded":"2024-09-21T12:01:28.103594+0000","last_fullsized":"2024-09-21T12:01:28.103594+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:28.103523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:28.103523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:58.043872+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"471'27","reported_seq":1086,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611566+0000","last_change":"2024-09-21T12:01:04.453705+0000","last_active":"2024-09-21T12:01:27.611566+0000","last_peered":"2024-09-21T12:01:27.611566+0000","last_clean":"2024-09-21T12:01:27.611566+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T12:01:27.611566+0000","last_undegraded":"2024-09-21T12:01:27.611566+0000","last_fullsized":"2024-09-21T12:01:27.611566+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'27","last_scrub_stamp":"2024-09-21T12:01:04.453628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:04.453628+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:33.956167+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":960,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612798+0000","last_change":"2024-09-21T12:00:28.776761+0000","last_active":"2024-09-21T12:01:27.612798+0000","last_peered":"2024-09-21T12:01:27.612798+0000","last_clean":"2024-09-21T12:01:27.612798+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T12:01:27.612798+0000","last_undegraded":"2024-09-21T12:01:27.612798+0000","last_fullsized":"2024-09-21T12:01:27.612798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:28.776682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:28.776682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:45.995150+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1541,"num_read_kb":1298,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":6904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":970,"ondisk_log_size":970,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1291,"num_read_kb":1286,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234250,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":790976,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":789182,"kb_avail":92954176,"statfs":{"total":95995035648,"available":95185076224,"internally_reserved":0,"allocated":1818624,"data_stored":1137350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":808122826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463183,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":672628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":671166,"kb_avail":93072524,"statfs":{"total":95995035648,"available":95306264576,"internally_reserved":0,"allocated":1478656,"data_stored":810014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":687274438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724820,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":691276,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":689214,"kb_avail":93053876,"statfs":{"total":95995035648,"available":95287169024,"internally_reserved":0,"allocated":2093056,"data_stored":1399500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":705755590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":481,"seq":2065879269391,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":96460,"kb_used_data":20348,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":93648692,"statfs":{"total":95995035648,"available":95896260608,"internally_reserved":0,"allocated":20836352,"data_stored":20143303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":77917859},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117280,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":805120,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":803070,"kb_avail":92940032,"statfs":{"total":95995035648,"available":95170592768,"internally_reserved":0,"allocated":2080768,"data_stored":1399376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":822344135},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411622,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":799236,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":797502,"kb_avail":92945916,"statfs":{"total":95995035648,"available":95176617984,"internally_reserved":0,"allocated":1757184,"data_stored":1072407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":816642501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575149,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":786524,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":785022,"kb_avail":92958628,"statfs":{"total":95995035648,"available":95189635072,"internally_reserved":0,"allocated":1519616,"data_stored":810159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":803862982},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738675,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":808720,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":807230,"kb_avail":92936432,"statfs":{"total":95995035648,"available":95166906368,"internally_reserved":0,"allocated":1507328,"data_stored":810788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":826603973},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T12:01:46.642 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph pg dump --format=json 2024-09-21T12:01:46.906 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:47.384 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[36454]: from='client.26927 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T12:01:47.384 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[36454]: 4.0 scrub starts 2024-09-21T12:01:47.384 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[36454]: 4.0 scrub ok 2024-09-21T12:01:47.384 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[36454]: pgmap v1073: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:47.384 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[36454]: 1.9 scrub starts 2024-09-21T12:01:47.384 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[36454]: 1.9 scrub ok 2024-09-21T12:01:47.384 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[29710]: from='client.26927 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T12:01:47.384 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[29710]: 4.0 scrub starts 2024-09-21T12:01:47.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[29710]: 4.0 scrub ok 2024-09-21T12:01:47.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[29710]: pgmap v1073: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:47.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[29710]: 1.9 scrub starts 2024-09-21T12:01:47.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:47 smithi043 ceph-mon[29710]: 1.9 scrub ok 2024-09-21T12:01:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:47 smithi175 ceph-mon[32613]: from='client.26927 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T12:01:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:47 smithi175 ceph-mon[32613]: 4.0 scrub starts 2024-09-21T12:01:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:47 smithi175 ceph-mon[32613]: 4.0 scrub ok 2024-09-21T12:01:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:47 smithi175 ceph-mon[32613]: pgmap v1073: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-21T12:01:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:47 smithi175 ceph-mon[32613]: 1.9 scrub starts 2024-09-21T12:01:47.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:47 smithi175 ceph-mon[32613]: 1.9 scrub ok 2024-09-21T12:01:47.667 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-21T12:01:47.669 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-09-21T12:01:48.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[29710]: 3.c scrub starts 2024-09-21T12:01:48.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[29710]: 3.c scrub ok 2024-09-21T12:01:48.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[29710]: 5.17 scrub starts 2024-09-21T12:01:48.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[29710]: 5.17 scrub ok 2024-09-21T12:01:48.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:48.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[29710]: 5.19 scrub starts 2024-09-21T12:01:48.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[29710]: 5.19 scrub ok 2024-09-21T12:01:48.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[29710]: from='client.17130 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T12:01:48.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[29710]: 3.d scrub starts 2024-09-21T12:01:48.406 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[29710]: 3.d scrub ok 2024-09-21T12:01:48.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[36454]: 3.c scrub starts 2024-09-21T12:01:48.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[36454]: 3.c scrub ok 2024-09-21T12:01:48.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[36454]: 5.17 scrub starts 2024-09-21T12:01:48.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[36454]: 5.17 scrub ok 2024-09-21T12:01:48.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:48.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[36454]: 5.19 scrub starts 2024-09-21T12:01:48.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[36454]: 5.19 scrub ok 2024-09-21T12:01:48.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[36454]: from='client.17130 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T12:01:48.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[36454]: 3.d scrub starts 2024-09-21T12:01:48.407 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:48 smithi043 ceph-mon[36454]: 3.d scrub ok 2024-09-21T12:01:48.439 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":1073,"stamp":"2024-09-21T12:01:46.244433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2849,"num_read_kb":2601,"num_write":1008,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1937,"ondisk_log_size":1937,"up":426,"acting":426,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":429,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":5450940,"kb_used_data":32316,"kb_used_omap":15,"kb_used_meta":5418480,"kb_avail":744510276,"statfs":{"total":767960285184,"available":762378522624,"internally_reserved":0,"allocated":33091584,"data_stored":27582897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15600,"internal_metadata":5548524304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003103"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":188,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612828+0000","last_change":"2024-09-21T12:01:07.788817+0000","last_active":"2024-09-21T12:01:27.612828+0000","last_peered":"2024-09-21T12:01:27.612828+0000","last_clean":"2024-09-21T12:01:27.612828+0000","last_became_active":"2024-09-21T11:57:58.529245+0000","last_became_peered":"2024-09-21T11:57:58.529245+0000","last_unstale":"2024-09-21T12:01:27.612828+0000","last_undegraded":"2024-09-21T12:01:27.612828+0000","last_fullsized":"2024-09-21T12:01:27.612828+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:07.788769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:07.788769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:18.906915+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"471'21","reported_seq":975,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571095+0000","last_change":"2024-09-21T12:01:21.965294+0000","last_active":"2024-09-21T12:01:27.571095+0000","last_peered":"2024-09-21T12:01:27.571095+0000","last_clean":"2024-09-21T12:01:27.571095+0000","last_became_active":"2024-09-21T12:01:21.964962+0000","last_became_peered":"2024-09-21T12:01:21.964962+0000","last_unstale":"2024-09-21T12:01:27.571095+0000","last_undegraded":"2024-09-21T12:01:27.571095+0000","last_fullsized":"2024-09-21T12:01:27.571095+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:01:13.523391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:13.523391+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:28.651463+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":670,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570409+0000","last_change":"2024-09-21T12:00:32.278040+0000","last_active":"2024-09-21T12:01:27.570409+0000","last_peered":"2024-09-21T12:01:27.570409+0000","last_clean":"2024-09-21T12:01:27.570409+0000","last_became_active":"2024-09-21T11:47:03.372392+0000","last_became_peered":"2024-09-21T11:47:03.372392+0000","last_unstale":"2024-09-21T12:01:27.570409+0000","last_undegraded":"2024-09-21T12:01:27.570409+0000","last_fullsized":"2024-09-21T12:01:27.570409+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:32.277990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:32.277990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:42.907481+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":730,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571189+0000","last_change":"2024-09-21T12:01:21.966845+0000","last_active":"2024-09-21T12:01:27.571189+0000","last_peered":"2024-09-21T12:01:27.571189+0000","last_clean":"2024-09-21T12:01:27.571189+0000","last_became_active":"2024-09-21T12:01:21.966286+0000","last_became_peered":"2024-09-21T12:01:21.966286+0000","last_unstale":"2024-09-21T12:01:27.571189+0000","last_undegraded":"2024-09-21T12:01:27.571189+0000","last_fullsized":"2024-09-21T12:01:27.571189+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:00.260916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:00.260916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.498477+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":972,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:42.003596+0000","last_change":"2024-09-21T12:01:42.003596+0000","last_active":"2024-09-21T12:01:42.003596+0000","last_peered":"2024-09-21T12:01:42.003596+0000","last_clean":"2024-09-21T12:01:42.003596+0000","last_became_active":"2024-09-21T11:40:06.489518+0000","last_became_peered":"2024-09-21T11:40:06.489518+0000","last_unstale":"2024-09-21T12:01:42.003596+0000","last_undegraded":"2024-09-21T12:01:42.003596+0000","last_fullsized":"2024-09-21T12:01:42.003596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:42.003552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:42.003552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:53.070290+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":954,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:33.118091+0000","last_change":"2024-09-21T12:01:33.118091+0000","last_active":"2024-09-21T12:01:33.118091+0000","last_peered":"2024-09-21T12:01:33.118091+0000","last_clean":"2024-09-21T12:01:33.118091+0000","last_became_active":"2024-09-21T11:40:00.306924+0000","last_became_peered":"2024-09-21T11:40:00.306924+0000","last_unstale":"2024-09-21T12:01:33.118091+0000","last_undegraded":"2024-09-21T12:01:33.118091+0000","last_fullsized":"2024-09-21T12:01:33.118091+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:33.118044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:33.118044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:53.081549+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"471'39","reported_seq":1017,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.572180+0000","last_change":"2024-09-21T12:01:07.150677+0000","last_active":"2024-09-21T12:01:27.572180+0000","last_peered":"2024-09-21T12:01:27.572180+0000","last_clean":"2024-09-21T12:01:27.572180+0000","last_became_active":"2024-09-21T11:40:02.314223+0000","last_became_peered":"2024-09-21T11:40:02.314223+0000","last_unstale":"2024-09-21T12:01:27.572180+0000","last_undegraded":"2024-09-21T12:01:27.572180+0000","last_fullsized":"2024-09-21T12:01:27.572180+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'39","last_scrub_stamp":"2024-09-21T12:01:07.150606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:07.150606+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:18.322620+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":780,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570976+0000","last_change":"2024-09-21T12:01:21.965378+0000","last_active":"2024-09-21T12:01:27.570976+0000","last_peered":"2024-09-21T12:01:27.570976+0000","last_clean":"2024-09-21T12:01:27.570976+0000","last_became_active":"2024-09-21T12:01:21.965021+0000","last_became_peered":"2024-09-21T12:01:21.965021+0000","last_unstale":"2024-09-21T12:01:27.570976+0000","last_undegraded":"2024-09-21T12:01:27.570976+0000","last_fullsized":"2024-09-21T12:01:27.570976+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:45.299024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:45.299024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:53.874908+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":954,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614227+0000","last_change":"2024-09-21T12:00:43.989346+0000","last_active":"2024-09-21T12:01:27.614227+0000","last_peered":"2024-09-21T12:01:27.614227+0000","last_clean":"2024-09-21T12:01:27.614227+0000","last_became_active":"2024-09-21T12:00:40.761723+0000","last_became_peered":"2024-09-21T12:00:40.761723+0000","last_unstale":"2024-09-21T12:01:27.614227+0000","last_undegraded":"2024-09-21T12:01:27.614227+0000","last_fullsized":"2024-09-21T12:01:27.614227+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:43.989297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:43.989297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:05.498788+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"471'21","reported_seq":975,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:28.449371+0000","last_change":"2024-09-21T12:01:28.449371+0000","last_active":"2024-09-21T12:01:28.449371+0000","last_peered":"2024-09-21T12:01:28.449371+0000","last_clean":"2024-09-21T12:01:28.449371+0000","last_became_active":"2024-09-21T12:01:21.964136+0000","last_became_peered":"2024-09-21T12:01:21.964136+0000","last_unstale":"2024-09-21T12:01:28.449371+0000","last_undegraded":"2024-09-21T12:01:28.449371+0000","last_fullsized":"2024-09-21T12:01:28.449371+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:01:28.449269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:28.449269+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:37.672700+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":959,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.222847+0000","last_change":"2024-09-21T12:01:35.222847+0000","last_active":"2024-09-21T12:01:35.222847+0000","last_peered":"2024-09-21T12:01:35.222847+0000","last_clean":"2024-09-21T12:01:35.222847+0000","last_became_active":"2024-09-21T11:40:04.339319+0000","last_became_peered":"2024-09-21T11:40:04.339319+0000","last_unstale":"2024-09-21T12:01:35.222847+0000","last_undegraded":"2024-09-21T12:01:35.222847+0000","last_fullsized":"2024-09-21T12:01:35.222847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:35.222788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:35.222788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:35.247504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":196,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:29.623778+0000","last_change":"2024-09-21T12:01:29.623778+0000","last_active":"2024-09-21T12:01:29.623778+0000","last_peered":"2024-09-21T12:01:29.623778+0000","last_clean":"2024-09-21T12:01:29.623778+0000","last_became_active":"2024-09-21T11:57:58.526820+0000","last_became_peered":"2024-09-21T11:57:58.526820+0000","last_unstale":"2024-09-21T12:01:29.623778+0000","last_undegraded":"2024-09-21T12:01:29.623778+0000","last_fullsized":"2024-09-21T12:01:29.623778+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:29.623712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:29.623712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:45.454344+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1008,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612449+0000","last_change":"2024-09-21T12:01:17.241924+0000","last_active":"2024-09-21T12:01:27.612449+0000","last_peered":"2024-09-21T12:01:27.612449+0000","last_clean":"2024-09-21T12:01:27.612449+0000","last_became_active":"2024-09-21T12:00:40.759034+0000","last_became_peered":"2024-09-21T12:00:40.759034+0000","last_unstale":"2024-09-21T12:01:27.612449+0000","last_undegraded":"2024-09-21T12:01:27.612449+0000","last_fullsized":"2024-09-21T12:01:27.612449+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:17.241870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:17.241870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:36.243892+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"471'9","reported_seq":959,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571388+0000","last_change":"2024-09-21T12:01:21.962922+0000","last_active":"2024-09-21T12:01:27.571388+0000","last_peered":"2024-09-21T12:01:27.571388+0000","last_clean":"2024-09-21T12:01:27.571388+0000","last_became_active":"2024-09-21T12:01:21.962563+0000","last_became_peered":"2024-09-21T12:01:21.962563+0000","last_unstale":"2024-09-21T12:01:27.571388+0000","last_undegraded":"2024-09-21T12:01:27.571388+0000","last_fullsized":"2024-09-21T12:01:27.571388+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'9","last_scrub_stamp":"2024-09-21T12:01:13.163707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:13.163707+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:38.244122+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":12,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":976,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613106+0000","last_change":"2024-09-21T12:00:57.756967+0000","last_active":"2024-09-21T12:01:27.613106+0000","last_peered":"2024-09-21T12:01:27.613106+0000","last_clean":"2024-09-21T12:01:27.613106+0000","last_became_active":"2024-09-21T12:00:40.770339+0000","last_became_peered":"2024-09-21T12:00:40.770339+0000","last_unstale":"2024-09-21T12:01:27.613106+0000","last_undegraded":"2024-09-21T12:01:27.613106+0000","last_fullsized":"2024-09-21T12:01:27.613106+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:57.756910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:57.756910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:00.222316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":961,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613222+0000","last_change":"2024-09-21T12:01:02.274827+0000","last_active":"2024-09-21T12:01:27.613222+0000","last_peered":"2024-09-21T12:01:27.613222+0000","last_clean":"2024-09-21T12:01:27.613222+0000","last_became_active":"2024-09-21T11:47:03.380590+0000","last_became_peered":"2024-09-21T11:47:03.380590+0000","last_unstale":"2024-09-21T12:01:27.613222+0000","last_undegraded":"2024-09-21T12:01:27.613222+0000","last_fullsized":"2024-09-21T12:01:27.613222+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:02.274749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:02.274749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:19.205320+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":529,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571970+0000","last_change":"2024-09-21T12:01:21.970447+0000","last_active":"2024-09-21T12:01:27.571970+0000","last_peered":"2024-09-21T12:01:27.571970+0000","last_clean":"2024-09-21T12:01:27.571970+0000","last_became_active":"2024-09-21T12:01:21.969324+0000","last_became_peered":"2024-09-21T12:01:21.969324+0000","last_unstale":"2024-09-21T12:01:27.571970+0000","last_undegraded":"2024-09-21T12:01:27.571970+0000","last_fullsized":"2024-09-21T12:01:27.571970+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:01.476028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:01.476028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:25.508196+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":207,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612312+0000","last_change":"2024-09-21T12:01:22.786474+0000","last_active":"2024-09-21T12:01:27.612312+0000","last_peered":"2024-09-21T12:01:27.612312+0000","last_clean":"2024-09-21T12:01:27.612312+0000","last_became_active":"2024-09-21T12:00:40.764140+0000","last_became_peered":"2024-09-21T12:00:40.764140+0000","last_unstale":"2024-09-21T12:01:27.612312+0000","last_undegraded":"2024-09-21T12:01:27.612312+0000","last_fullsized":"2024-09-21T12:01:27.612312+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:22.786418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:22.786418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:26.608584+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"471'39","reported_seq":1133,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612787+0000","last_change":"2024-09-21T12:01:11.392703+0000","last_active":"2024-09-21T12:01:27.612787+0000","last_peered":"2024-09-21T12:01:27.612787+0000","last_clean":"2024-09-21T12:01:27.612787+0000","last_became_active":"2024-09-21T11:55:33.737341+0000","last_became_peered":"2024-09-21T11:55:33.737341+0000","last_unstale":"2024-09-21T12:01:27.612787+0000","last_undegraded":"2024-09-21T12:01:27.612787+0000","last_fullsized":"2024-09-21T12:01:27.612787+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'39","last_scrub_stamp":"2024-09-21T12:01:11.392646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:11.392646+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:40.107122+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":686,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570947+0000","last_change":"2024-09-21T12:00:46.476017+0000","last_active":"2024-09-21T12:01:27.570947+0000","last_peered":"2024-09-21T12:01:27.570947+0000","last_clean":"2024-09-21T12:01:27.570947+0000","last_became_active":"2024-09-21T12:00:40.771901+0000","last_became_peered":"2024-09-21T12:00:40.771901+0000","last_unstale":"2024-09-21T12:01:27.570947+0000","last_undegraded":"2024-09-21T12:01:27.570947+0000","last_fullsized":"2024-09-21T12:01:27.570947+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:46.475967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:46.475967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:51.244834+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":958,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571559+0000","last_change":"2024-09-21T12:01:20.205243+0000","last_active":"2024-09-21T12:01:27.571559+0000","last_peered":"2024-09-21T12:01:27.571559+0000","last_clean":"2024-09-21T12:01:27.571559+0000","last_became_active":"2024-09-21T11:40:06.487577+0000","last_became_peered":"2024-09-21T11:40:06.487577+0000","last_unstale":"2024-09-21T12:01:27.571559+0000","last_undegraded":"2024-09-21T12:01:27.571559+0000","last_fullsized":"2024-09-21T12:01:27.571559+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:20.205194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:20.205194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:44.483785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":750,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571378+0000","last_change":"2024-09-21T12:01:21.966317+0000","last_active":"2024-09-21T12:01:27.571378+0000","last_peered":"2024-09-21T12:01:27.571378+0000","last_clean":"2024-09-21T12:01:27.571378+0000","last_became_active":"2024-09-21T12:01:21.966023+0000","last_became_peered":"2024-09-21T12:01:21.966023+0000","last_unstale":"2024-09-21T12:01:27.571378+0000","last_undegraded":"2024-09-21T12:01:27.571378+0000","last_fullsized":"2024-09-21T12:01:27.571378+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:08.647539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:08.647539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:29.406037+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"471'24","reported_seq":1047,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571697+0000","last_change":"2024-09-21T12:00:48.124110+0000","last_active":"2024-09-21T12:01:27.571697+0000","last_peered":"2024-09-21T12:01:27.571697+0000","last_clean":"2024-09-21T12:01:27.571697+0000","last_became_active":"2024-09-21T12:00:40.776684+0000","last_became_peered":"2024-09-21T12:00:40.776684+0000","last_unstale":"2024-09-21T12:01:27.571697+0000","last_undegraded":"2024-09-21T12:01:27.571697+0000","last_fullsized":"2024-09-21T12:01:27.571697+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'24","last_scrub_stamp":"2024-09-21T12:00:48.124043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:48.124043+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:02.006286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":442,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571550+0000","last_change":"2024-09-21T12:01:21.967750+0000","last_active":"2024-09-21T12:01:27.571550+0000","last_peered":"2024-09-21T12:01:27.571550+0000","last_clean":"2024-09-21T12:01:27.571550+0000","last_became_active":"2024-09-21T12:01:21.967438+0000","last_became_peered":"2024-09-21T12:01:21.967438+0000","last_unstale":"2024-09-21T12:01:27.571550+0000","last_undegraded":"2024-09-21T12:01:27.571550+0000","last_fullsized":"2024-09-21T12:01:27.571550+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:06.190850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:06.190850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:31.275779+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":457,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612528+0000","last_change":"2024-09-21T12:00:36.790840+0000","last_active":"2024-09-21T12:01:27.612528+0000","last_peered":"2024-09-21T12:01:27.612528+0000","last_clean":"2024-09-21T12:01:27.612528+0000","last_became_active":"2024-09-21T11:51:31.472865+0000","last_became_peered":"2024-09-21T11:51:31.472865+0000","last_unstale":"2024-09-21T12:01:27.612528+0000","last_undegraded":"2024-09-21T12:01:27.612528+0000","last_fullsized":"2024-09-21T12:01:27.612528+0000","mapping_epoch":289,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":290,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T12:00:36.790778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:36.790778+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:02.627806+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":707,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570550+0000","last_change":"2024-09-21T12:00:50.530029+0000","last_active":"2024-09-21T12:01:27.570550+0000","last_peered":"2024-09-21T12:01:27.570550+0000","last_clean":"2024-09-21T12:01:27.570550+0000","last_became_active":"2024-09-21T12:00:40.762620+0000","last_became_peered":"2024-09-21T12:00:40.762620+0000","last_unstale":"2024-09-21T12:01:27.570550+0000","last_undegraded":"2024-09-21T12:01:27.570550+0000","last_fullsized":"2024-09-21T12:01:27.570550+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T12:00:50.529956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:50.529956+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:05.653998+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"471'30","reported_seq":1001,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612808+0000","last_change":"2024-09-21T12:00:47.891826+0000","last_active":"2024-09-21T12:01:27.612808+0000","last_peered":"2024-09-21T12:01:27.612808+0000","last_clean":"2024-09-21T12:01:27.612808+0000","last_became_active":"2024-09-21T11:40:02.399944+0000","last_became_peered":"2024-09-21T11:40:02.399944+0000","last_unstale":"2024-09-21T12:01:27.612808+0000","last_undegraded":"2024-09-21T12:01:27.612808+0000","last_fullsized":"2024-09-21T12:01:27.612808+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'30","last_scrub_stamp":"2024-09-21T12:00:47.891763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:47.891763+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.873947+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"482'12","reported_seq":1295,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:40.635147+0000","last_change":"2024-09-21T12:01:18.612421+0000","last_active":"2024-09-21T12:01:40.635147+0000","last_peered":"2024-09-21T12:01:40.635147+0000","last_clean":"2024-09-21T12:01:40.635147+0000","last_became_active":"2024-09-21T12:00:40.762782+0000","last_became_peered":"2024-09-21T12:00:40.762782+0000","last_unstale":"2024-09-21T12:01:40.635147+0000","last_undegraded":"2024-09-21T12:01:40.635147+0000","last_fullsized":"2024-09-21T12:01:40.635147+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'12","last_scrub_stamp":"2024-09-21T12:01:18.612327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:18.612327+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:45.592541+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"74'79","reported_seq":206,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614155+0000","last_change":"2024-09-21T12:00:50.137327+0000","last_active":"2024-09-21T12:01:27.614155+0000","last_peered":"2024-09-21T12:01:27.614155+0000","last_clean":"2024-09-21T12:01:27.614155+0000","last_became_active":"2024-09-21T11:57:45.504680+0000","last_became_peered":"2024-09-21T11:57:45.504680+0000","last_unstale":"2024-09-21T12:01:27.614155+0000","last_undegraded":"2024-09-21T12:01:27.614155+0000","last_fullsized":"2024-09-21T12:01:27.614155+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:50.137280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:50.137280+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:52.003733+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"471'39","reported_seq":740,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.572350+0000","last_change":"2024-09-21T12:00:45.439926+0000","last_active":"2024-09-21T12:01:27.572350+0000","last_peered":"2024-09-21T12:01:27.572350+0000","last_clean":"2024-09-21T12:01:27.572350+0000","last_became_active":"2024-09-21T12:00:40.771095+0000","last_became_peered":"2024-09-21T12:00:40.771095+0000","last_unstale":"2024-09-21T12:01:27.572350+0000","last_undegraded":"2024-09-21T12:01:27.572350+0000","last_fullsized":"2024-09-21T12:01:27.572350+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'39","last_scrub_stamp":"2024-09-21T12:00:45.439843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:45.439843+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.634394+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":984,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570783+0000","last_change":"2024-09-21T12:01:27.149114+0000","last_active":"2024-09-21T12:01:27.570783+0000","last_peered":"2024-09-21T12:01:27.570783+0000","last_clean":"2024-09-21T12:01:27.570783+0000","last_became_active":"2024-09-21T11:47:03.391212+0000","last_became_peered":"2024-09-21T11:47:03.391212+0000","last_unstale":"2024-09-21T12:01:27.570783+0000","last_undegraded":"2024-09-21T12:01:27.570783+0000","last_fullsized":"2024-09-21T12:01:27.570783+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:27.149054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:27.149054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:47.587603+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"502'14","reported_seq":592,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634307+0000","last_change":"2024-09-21T12:01:21.969048+0000","last_active":"2024-09-21T12:01:35.634307+0000","last_peered":"2024-09-21T12:01:35.634307+0000","last_clean":"2024-09-21T12:01:35.634307+0000","last_became_active":"2024-09-21T12:01:21.968662+0000","last_became_peered":"2024-09-21T12:01:21.968662+0000","last_unstale":"2024-09-21T12:01:35.634307+0000","last_undegraded":"2024-09-21T12:01:35.634307+0000","last_fullsized":"2024-09-21T12:01:35.634307+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'13","last_scrub_stamp":"2024-09-21T12:00:46.020472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:46.020472+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:46.467129+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1016,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613139+0000","last_change":"2024-09-21T12:01:27.310149+0000","last_active":"2024-09-21T12:01:27.613139+0000","last_peered":"2024-09-21T12:01:27.613139+0000","last_clean":"2024-09-21T12:01:27.613139+0000","last_became_active":"2024-09-21T11:57:58.526927+0000","last_became_peered":"2024-09-21T11:57:58.526927+0000","last_unstale":"2024-09-21T12:01:27.613139+0000","last_undegraded":"2024-09-21T12:01:27.613139+0000","last_fullsized":"2024-09-21T12:01:27.613139+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:27.310092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:27.310092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:41.158567+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"74'79","reported_seq":28,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571915+0000","last_change":"2024-09-21T12:01:25.542758+0000","last_active":"2024-09-21T12:01:27.571915+0000","last_peered":"2024-09-21T12:01:27.571915+0000","last_clean":"2024-09-21T12:01:27.571915+0000","last_became_active":"2024-09-21T12:01:21.967906+0000","last_became_peered":"2024-09-21T12:01:21.967906+0000","last_unstale":"2024-09-21T12:01:27.571915+0000","last_undegraded":"2024-09-21T12:01:27.571915+0000","last_fullsized":"2024-09-21T12:01:27.571915+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:01:25.542702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:01:25.542702+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:31.811384+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"471'42","reported_seq":1100,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:46.076400+0000","last_change":"2024-09-21T12:01:46.076400+0000","last_active":"2024-09-21T12:01:46.076400+0000","last_peered":"2024-09-21T12:01:46.076400+0000","last_clean":"2024-09-21T12:01:46.076400+0000","last_became_active":"2024-09-21T11:57:58.537184+0000","last_became_peered":"2024-09-21T11:57:58.537184+0000","last_unstale":"2024-09-21T12:01:46.076400+0000","last_undegraded":"2024-09-21T12:01:46.076400+0000","last_fullsized":"2024-09-21T12:01:46.076400+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'42","last_scrub_stamp":"2024-09-21T12:01:46.076340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:46.076340+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:56.078273+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":978,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.572118+0000","last_change":"2024-09-21T12:01:16.118758+0000","last_active":"2024-09-21T12:01:27.572118+0000","last_peered":"2024-09-21T12:01:27.572118+0000","last_clean":"2024-09-21T12:01:27.572118+0000","last_became_active":"2024-09-21T11:40:00.310079+0000","last_became_peered":"2024-09-21T11:40:00.310079+0000","last_unstale":"2024-09-21T12:01:27.572118+0000","last_undegraded":"2024-09-21T12:01:27.572118+0000","last_fullsized":"2024-09-21T12:01:27.572118+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:16.118709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:16.118709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:17.543975+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":533,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:31.437280+0000","last_change":"2024-09-21T12:01:31.437280+0000","last_active":"2024-09-21T12:01:31.437280+0000","last_peered":"2024-09-21T12:01:31.437280+0000","last_clean":"2024-09-21T12:01:31.437280+0000","last_became_active":"2024-09-21T12:01:22.097383+0000","last_became_peered":"2024-09-21T12:01:22.097383+0000","last_unstale":"2024-09-21T12:01:31.437280+0000","last_undegraded":"2024-09-21T12:01:31.437280+0000","last_fullsized":"2024-09-21T12:01:31.437280+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:31.437219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:31.437219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:54.084667+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":484,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611796+0000","last_change":"2024-09-21T12:01:09.388467+0000","last_active":"2024-09-21T12:01:27.611796+0000","last_peered":"2024-09-21T12:01:27.611796+0000","last_clean":"2024-09-21T12:01:27.611796+0000","last_became_active":"2024-09-21T12:00:40.763206+0000","last_became_peered":"2024-09-21T12:00:40.763206+0000","last_unstale":"2024-09-21T12:01:27.611796+0000","last_undegraded":"2024-09-21T12:01:27.611796+0000","last_fullsized":"2024-09-21T12:01:27.611796+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:09.388420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:09.388420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:30.778544+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,5],"acting":[2,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"74'79","reported_seq":211,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:34.127197+0000","last_change":"2024-09-21T12:01:34.127197+0000","last_active":"2024-09-21T12:01:34.127197+0000","last_peered":"2024-09-21T12:01:34.127197+0000","last_clean":"2024-09-21T12:01:34.127197+0000","last_became_active":"2024-09-21T11:57:45.514189+0000","last_became_peered":"2024-09-21T11:57:45.514189+0000","last_unstale":"2024-09-21T12:01:34.127197+0000","last_undegraded":"2024-09-21T12:01:34.127197+0000","last_fullsized":"2024-09-21T12:01:34.127197+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:01:34.127141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:01:34.127141+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:00.483585+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"471'42","reported_seq":1009,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571383+0000","last_change":"2024-09-21T12:01:21.966602+0000","last_active":"2024-09-21T12:01:27.571383+0000","last_peered":"2024-09-21T12:01:27.571383+0000","last_clean":"2024-09-21T12:01:27.571383+0000","last_became_active":"2024-09-21T12:01:21.966229+0000","last_became_peered":"2024-09-21T12:01:21.966229+0000","last_unstale":"2024-09-21T12:01:27.571383+0000","last_undegraded":"2024-09-21T12:01:27.571383+0000","last_fullsized":"2024-09-21T12:01:27.571383+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'42","last_scrub_stamp":"2024-09-21T12:01:16.214003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:16.214003+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:26.342431+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":203,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612940+0000","last_change":"2024-09-21T12:00:48.889353+0000","last_active":"2024-09-21T12:01:27.612940+0000","last_peered":"2024-09-21T12:01:27.612940+0000","last_clean":"2024-09-21T12:01:27.612940+0000","last_became_active":"2024-09-21T12:00:40.768342+0000","last_became_peered":"2024-09-21T12:00:40.768342+0000","last_unstale":"2024-09-21T12:01:27.612940+0000","last_undegraded":"2024-09-21T12:01:27.612940+0000","last_fullsized":"2024-09-21T12:01:27.612940+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T12:00:48.889300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:48.889300+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:13.137934+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":138,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"482'12","reported_seq":1260,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:40.634775+0000","last_change":"2024-09-21T12:00:49.149552+0000","last_active":"2024-09-21T12:01:40.634775+0000","last_peered":"2024-09-21T12:01:40.634775+0000","last_clean":"2024-09-21T12:01:40.634775+0000","last_became_active":"2024-09-21T12:00:40.763011+0000","last_became_peered":"2024-09-21T12:00:40.763011+0000","last_unstale":"2024-09-21T12:01:40.634775+0000","last_undegraded":"2024-09-21T12:01:40.634775+0000","last_fullsized":"2024-09-21T12:01:40.634775+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'12","last_scrub_stamp":"2024-09-21T12:00:49.149500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:49.149500+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:52.037397+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":975,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613009+0000","last_change":"2024-09-21T12:01:11.872377+0000","last_active":"2024-09-21T12:01:27.613009+0000","last_peered":"2024-09-21T12:01:27.613009+0000","last_clean":"2024-09-21T12:01:27.613009+0000","last_became_active":"2024-09-21T11:56:34.581626+0000","last_became_peered":"2024-09-21T11:56:34.581626+0000","last_unstale":"2024-09-21T12:01:27.613009+0000","last_undegraded":"2024-09-21T12:01:27.613009+0000","last_fullsized":"2024-09-21T12:01:27.613009+0000","mapping_epoch":390,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":391,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:11.872322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:11.872322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:37.849542+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"74'79","reported_seq":300,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:38.432838+0000","last_change":"2024-09-21T12:01:38.432838+0000","last_active":"2024-09-21T12:01:38.432838+0000","last_peered":"2024-09-21T12:01:38.432838+0000","last_clean":"2024-09-21T12:01:38.432838+0000","last_became_active":"2024-09-21T11:57:45.514310+0000","last_became_peered":"2024-09-21T11:57:45.514310+0000","last_unstale":"2024-09-21T12:01:38.432838+0000","last_undegraded":"2024-09-21T12:01:38.432838+0000","last_fullsized":"2024-09-21T12:01:38.432838+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:01:38.432773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:01:38.432773+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:03.257984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"471'30","reported_seq":767,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571850+0000","last_change":"2024-09-21T12:00:48.516078+0000","last_active":"2024-09-21T12:01:27.571850+0000","last_peered":"2024-09-21T12:01:27.571850+0000","last_clean":"2024-09-21T12:01:27.571850+0000","last_became_active":"2024-09-21T12:00:40.770823+0000","last_became_peered":"2024-09-21T12:00:40.770823+0000","last_unstale":"2024-09-21T12:01:27.571850+0000","last_undegraded":"2024-09-21T12:01:27.571850+0000","last_fullsized":"2024-09-21T12:01:27.571850+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'30","last_scrub_stamp":"2024-09-21T12:00:48.516016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:48.516016+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:58.232390+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1001,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614293+0000","last_change":"2024-09-21T12:01:19.230398+0000","last_active":"2024-09-21T12:01:27.614293+0000","last_peered":"2024-09-21T12:01:27.614293+0000","last_clean":"2024-09-21T12:01:27.614293+0000","last_became_active":"2024-09-21T11:40:00.318519+0000","last_became_peered":"2024-09-21T11:40:00.318519+0000","last_unstale":"2024-09-21T12:01:27.614293+0000","last_undegraded":"2024-09-21T12:01:27.614293+0000","last_fullsized":"2024-09-21T12:01:27.614293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:19.230332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:19.230332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:28.064213+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1232,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634158+0000","last_change":"2024-09-21T12:00:52.882768+0000","last_active":"2024-09-21T12:01:35.634158+0000","last_peered":"2024-09-21T12:01:35.634158+0000","last_clean":"2024-09-21T12:01:35.634158+0000","last_became_active":"2024-09-21T11:40:04.335271+0000","last_became_peered":"2024-09-21T11:40:04.335271+0000","last_unstale":"2024-09-21T12:01:35.634158+0000","last_undegraded":"2024-09-21T12:01:35.634158+0000","last_fullsized":"2024-09-21T12:01:35.634158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T12:00:52.882717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:52.882717+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:15.270057+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":964,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611327+0000","last_change":"2024-09-21T12:01:04.837641+0000","last_active":"2024-09-21T12:01:27.611327+0000","last_peered":"2024-09-21T12:01:27.611327+0000","last_clean":"2024-09-21T12:01:27.611327+0000","last_became_active":"2024-09-21T11:40:06.351373+0000","last_became_peered":"2024-09-21T11:40:06.351373+0000","last_unstale":"2024-09-21T12:01:27.611327+0000","last_undegraded":"2024-09-21T12:01:27.611327+0000","last_fullsized":"2024-09-21T12:01:27.611327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:04.837596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:04.837596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:29.766014+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"74'79","reported_seq":308,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612577+0000","last_change":"2024-09-21T12:01:25.549292+0000","last_active":"2024-09-21T12:01:27.612577+0000","last_peered":"2024-09-21T12:01:27.612577+0000","last_clean":"2024-09-21T12:01:27.612577+0000","last_became_active":"2024-09-21T11:57:45.518764+0000","last_became_peered":"2024-09-21T11:57:45.518764+0000","last_unstale":"2024-09-21T12:01:27.612577+0000","last_undegraded":"2024-09-21T12:01:27.612577+0000","last_fullsized":"2024-09-21T12:01:27.612577+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:01:25.549234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:01:25.549234+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:28.829769+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"471'39","reported_seq":1046,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613006+0000","last_change":"2024-09-21T12:00:53.851148+0000","last_active":"2024-09-21T12:01:27.613006+0000","last_peered":"2024-09-21T12:01:27.613006+0000","last_clean":"2024-09-21T12:01:27.613006+0000","last_became_active":"2024-09-21T11:40:02.398810+0000","last_became_peered":"2024-09-21T11:40:02.398810+0000","last_unstale":"2024-09-21T12:01:27.613006+0000","last_undegraded":"2024-09-21T12:01:27.613006+0000","last_fullsized":"2024-09-21T12:01:27.613006+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'39","last_scrub_stamp":"2024-09-21T12:00:53.851079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:53.851079+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:12.075446+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":671,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570992+0000","last_change":"2024-09-21T12:01:01.507916+0000","last_active":"2024-09-21T12:01:27.570992+0000","last_peered":"2024-09-21T12:01:27.570992+0000","last_clean":"2024-09-21T12:01:27.570992+0000","last_became_active":"2024-09-21T11:47:03.381003+0000","last_became_peered":"2024-09-21T11:47:03.381003+0000","last_unstale":"2024-09-21T12:01:27.570992+0000","last_undegraded":"2024-09-21T12:01:27.570992+0000","last_fullsized":"2024-09-21T12:01:27.570992+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:01.507853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:01.507853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:31.405641+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":953,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612076+0000","last_change":"2024-09-21T12:01:03.477305+0000","last_active":"2024-09-21T12:01:27.612076+0000","last_peered":"2024-09-21T12:01:27.612076+0000","last_clean":"2024-09-21T12:01:27.612076+0000","last_became_active":"2024-09-21T11:40:04.338855+0000","last_became_peered":"2024-09-21T11:40:04.338855+0000","last_unstale":"2024-09-21T12:01:27.612076+0000","last_undegraded":"2024-09-21T12:01:27.612076+0000","last_fullsized":"2024-09-21T12:01:27.612076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:03.477250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:03.477250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:21.505703+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":655,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570321+0000","last_change":"2024-09-21T12:00:30.282966+0000","last_active":"2024-09-21T12:01:27.570321+0000","last_peered":"2024-09-21T12:01:27.570321+0000","last_clean":"2024-09-21T12:01:27.570321+0000","last_became_active":"2024-09-21T11:47:03.378824+0000","last_became_peered":"2024-09-21T11:47:03.378824+0000","last_unstale":"2024-09-21T12:01:27.570321+0000","last_undegraded":"2024-09-21T12:01:27.570321+0000","last_fullsized":"2024-09-21T12:01:27.570321+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:30.282923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:30.282923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:42.684239+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"74'79","reported_seq":206,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613011+0000","last_change":"2024-09-21T12:00:49.113289+0000","last_active":"2024-09-21T12:01:27.613011+0000","last_peered":"2024-09-21T12:01:27.613011+0000","last_clean":"2024-09-21T12:01:27.613011+0000","last_became_active":"2024-09-21T11:57:45.515242+0000","last_became_peered":"2024-09-21T11:57:45.515242+0000","last_unstale":"2024-09-21T12:01:27.613011+0000","last_undegraded":"2024-09-21T12:01:27.613011+0000","last_fullsized":"2024-09-21T12:01:27.613011+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:49.113230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:49.113230+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.945088+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"471'55","reported_seq":873,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:40.634783+0000","last_change":"2024-09-21T12:01:27.518852+0000","last_active":"2024-09-21T12:01:40.634783+0000","last_peered":"2024-09-21T12:01:40.634783+0000","last_clean":"2024-09-21T12:01:40.634783+0000","last_became_active":"2024-09-21T11:47:03.381602+0000","last_became_peered":"2024-09-21T11:47:03.381602+0000","last_unstale":"2024-09-21T12:01:40.634783+0000","last_undegraded":"2024-09-21T12:01:40.634783+0000","last_fullsized":"2024-09-21T12:01:40.634783+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'55","last_scrub_stamp":"2024-09-21T12:01:27.518795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:27.518795+0000","objects_scrubbed":10,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:55.580498+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":83,"num_read_kb":68,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":1074,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"59'1","reported_seq":515,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571305+0000","last_change":"2024-09-21T12:01:21.964098+0000","last_active":"2024-09-21T12:01:27.571305+0000","last_peered":"2024-09-21T12:01:27.571305+0000","last_clean":"2024-09-21T12:01:27.571305+0000","last_became_active":"2024-09-21T12:01:21.963623+0000","last_became_peered":"2024-09-21T12:01:21.963623+0000","last_unstale":"2024-09-21T12:01:27.571305+0000","last_undegraded":"2024-09-21T12:01:27.571305+0000","last_fullsized":"2024-09-21T12:01:27.571305+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T12:00:59.857482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:59.857482+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:25.350435+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":1001,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:29.262091+0000","last_change":"2024-09-21T12:01:29.262091+0000","last_active":"2024-09-21T12:01:29.262091+0000","last_peered":"2024-09-21T12:01:29.262091+0000","last_clean":"2024-09-21T12:01:29.262091+0000","last_became_active":"2024-09-21T11:47:03.384807+0000","last_became_peered":"2024-09-21T11:47:03.384807+0000","last_unstale":"2024-09-21T12:01:29.262091+0000","last_undegraded":"2024-09-21T12:01:29.262091+0000","last_fullsized":"2024-09-21T12:01:29.262091+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-21T12:01:29.262022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T12:01:29.262022+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:35.035827+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0015015370000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":206,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612512+0000","last_change":"2024-09-21T12:00:45.878432+0000","last_active":"2024-09-21T12:01:27.612512+0000","last_peered":"2024-09-21T12:01:27.612512+0000","last_clean":"2024-09-21T12:01:27.612512+0000","last_became_active":"2024-09-21T12:00:40.768563+0000","last_became_peered":"2024-09-21T12:00:40.768563+0000","last_unstale":"2024-09-21T12:01:27.612512+0000","last_undegraded":"2024-09-21T12:01:27.612512+0000","last_fullsized":"2024-09-21T12:01:27.612512+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:45.878382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:45.878382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:07.019792+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":953,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:38.064336+0000","last_change":"2024-09-21T12:01:38.064292+0000","last_active":"2024-09-21T12:01:38.064336+0000","last_peered":"2024-09-21T12:01:38.064336+0000","last_clean":"2024-09-21T12:01:38.064336+0000","last_became_active":"2024-09-21T12:00:40.774147+0000","last_became_peered":"2024-09-21T12:00:40.774147+0000","last_unstale":"2024-09-21T12:01:38.064336+0000","last_undegraded":"2024-09-21T12:01:38.064336+0000","last_fullsized":"2024-09-21T12:01:38.064336+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:38.064099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:38.064099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:51.144359+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"74'79","reported_seq":1097,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611666+0000","last_change":"2024-09-21T12:00:53.697047+0000","last_active":"2024-09-21T12:01:27.611666+0000","last_peered":"2024-09-21T12:01:27.611666+0000","last_clean":"2024-09-21T12:01:27.611666+0000","last_became_active":"2024-09-21T12:00:40.769976+0000","last_became_peered":"2024-09-21T12:00:40.769976+0000","last_unstale":"2024-09-21T12:01:27.611666+0000","last_undegraded":"2024-09-21T12:01:27.611666+0000","last_fullsized":"2024-09-21T12:01:27.611666+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:53.696991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:53.696991+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:06.952325+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"471'21","reported_seq":658,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611672+0000","last_change":"2024-09-21T12:00:21.629636+0000","last_active":"2024-09-21T12:01:27.611672+0000","last_peered":"2024-09-21T12:01:27.611672+0000","last_clean":"2024-09-21T12:01:27.611672+0000","last_became_active":"2024-09-21T11:57:58.530507+0000","last_became_peered":"2024-09-21T11:57:58.530507+0000","last_unstale":"2024-09-21T12:01:27.611672+0000","last_undegraded":"2024-09-21T12:01:27.611672+0000","last_fullsized":"2024-09-21T12:01:27.611672+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:00:21.629570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:21.629570+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:47.207499+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":30,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"59'1","reported_seq":667,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.569940+0000","last_change":"2024-09-21T12:00:29.254195+0000","last_active":"2024-09-21T12:01:27.569940+0000","last_peered":"2024-09-21T12:01:27.569940+0000","last_clean":"2024-09-21T12:01:27.569940+0000","last_became_active":"2024-09-21T11:47:03.365078+0000","last_became_peered":"2024-09-21T11:47:03.365078+0000","last_unstale":"2024-09-21T12:01:27.569940+0000","last_undegraded":"2024-09-21T12:01:27.569940+0000","last_fullsized":"2024-09-21T12:01:27.569940+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T12:00:29.254145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:29.254145+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:53.772710+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":995,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.347343+0000","last_change":"2024-09-21T12:01:35.347343+0000","last_active":"2024-09-21T12:01:35.347343+0000","last_peered":"2024-09-21T12:01:35.347343+0000","last_clean":"2024-09-21T12:01:35.347343+0000","last_became_active":"2024-09-21T11:39:59.307437+0000","last_became_peered":"2024-09-21T11:39:59.307437+0000","last_unstale":"2024-09-21T12:01:35.347343+0000","last_undegraded":"2024-09-21T12:01:35.347343+0000","last_fullsized":"2024-09-21T12:01:35.347343+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-21T12:01:35.347288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T12:01:35.347288+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:50.592627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704139,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":977,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:40.324176+0000","last_change":"2024-09-21T12:01:40.324176+0000","last_active":"2024-09-21T12:01:40.324176+0000","last_peered":"2024-09-21T12:01:40.324176+0000","last_clean":"2024-09-21T12:01:40.324176+0000","last_became_active":"2024-09-21T11:40:04.331785+0000","last_became_peered":"2024-09-21T11:40:04.331785+0000","last_unstale":"2024-09-21T12:01:40.324176+0000","last_undegraded":"2024-09-21T12:01:40.324176+0000","last_fullsized":"2024-09-21T12:01:40.324176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:40.324122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:40.324122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:00.588221+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":973,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613868+0000","last_change":"2024-09-21T12:01:08.802207+0000","last_active":"2024-09-21T12:01:27.613868+0000","last_peered":"2024-09-21T12:01:27.613868+0000","last_clean":"2024-09-21T12:01:27.613868+0000","last_became_active":"2024-09-21T11:40:06.487419+0000","last_became_peered":"2024-09-21T11:40:06.487419+0000","last_unstale":"2024-09-21T12:01:27.613868+0000","last_undegraded":"2024-09-21T12:01:27.613868+0000","last_fullsized":"2024-09-21T12:01:27.613868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:08.802156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:08.802156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:19.965570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"74'79","reported_seq":969,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570436+0000","last_change":"2024-09-21T12:00:51.525993+0000","last_active":"2024-09-21T12:01:27.570436+0000","last_peered":"2024-09-21T12:01:27.570436+0000","last_clean":"2024-09-21T12:01:27.570436+0000","last_became_active":"2024-09-21T12:00:40.759148+0000","last_became_peered":"2024-09-21T12:00:40.759148+0000","last_unstale":"2024-09-21T12:01:27.570436+0000","last_undegraded":"2024-09-21T12:01:27.570436+0000","last_fullsized":"2024-09-21T12:01:27.570436+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":3,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:51.525919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:51.525919+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:18.249569+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"471'36","reported_seq":220,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611339+0000","last_change":"2024-09-21T12:01:14.750890+0000","last_active":"2024-09-21T12:01:27.611339+0000","last_peered":"2024-09-21T12:01:27.611339+0000","last_clean":"2024-09-21T12:01:27.611339+0000","last_became_active":"2024-09-21T11:57:58.527381+0000","last_became_peered":"2024-09-21T11:57:58.527381+0000","last_unstale":"2024-09-21T12:01:27.611339+0000","last_undegraded":"2024-09-21T12:01:27.611339+0000","last_fullsized":"2024-09-21T12:01:27.611339+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'36","last_scrub_stamp":"2024-09-21T12:01:14.750830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:14.750830+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:24.597446+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1009,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611965+0000","last_change":"2024-09-21T12:00:51.836919+0000","last_active":"2024-09-21T12:01:27.611965+0000","last_peered":"2024-09-21T12:01:27.611965+0000","last_clean":"2024-09-21T12:01:27.611965+0000","last_became_active":"2024-09-21T12:00:40.768771+0000","last_became_peered":"2024-09-21T12:00:40.768771+0000","last_unstale":"2024-09-21T12:01:27.611965+0000","last_undegraded":"2024-09-21T12:01:27.611965+0000","last_fullsized":"2024-09-21T12:01:27.611965+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:51.836873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:51.836873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:19.042519+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"66'5","reported_seq":2311,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:39.928624+0000","last_change":"2024-09-21T12:01:16.865531+0000","last_active":"2024-09-21T12:01:39.928624+0000","last_peered":"2024-09-21T12:01:39.928624+0000","last_clean":"2024-09-21T12:01:39.928624+0000","last_became_active":"2024-09-21T11:47:03.368356+0000","last_became_peered":"2024-09-21T11:47:03.368356+0000","last_unstale":"2024-09-21T12:01:39.928624+0000","last_undegraded":"2024-09-21T12:01:39.928624+0000","last_fullsized":"2024-09-21T12:01:39.928624+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-21T12:01:16.865481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:58.289765+0000","last_clean_scrub_stamp":"2024-09-21T12:01:16.865481+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:36.557499+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00049402699999999996,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1291,"num_read_kb":1286,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":982,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613483+0000","last_change":"2024-09-21T12:00:39.121692+0000","last_active":"2024-09-21T12:01:27.613483+0000","last_peered":"2024-09-21T12:01:27.613483+0000","last_clean":"2024-09-21T12:01:27.613483+0000","last_became_active":"2024-09-21T11:47:03.379927+0000","last_became_peered":"2024-09-21T11:47:03.379927+0000","last_unstale":"2024-09-21T12:01:27.613483+0000","last_undegraded":"2024-09-21T12:01:27.613483+0000","last_fullsized":"2024-09-21T12:01:27.613483+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:39.121641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:39.121641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:56.458399+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":655,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:29.511642+0000","last_change":"2024-09-21T12:01:29.511597+0000","last_active":"2024-09-21T12:01:29.511642+0000","last_peered":"2024-09-21T12:01:29.511642+0000","last_clean":"2024-09-21T12:01:29.511642+0000","last_became_active":"2024-09-21T11:47:03.374155+0000","last_became_peered":"2024-09-21T11:47:03.374155+0000","last_unstale":"2024-09-21T12:01:29.511642+0000","last_undegraded":"2024-09-21T12:01:29.511642+0000","last_fullsized":"2024-09-21T12:01:29.511642+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:29.511547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:29.511547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:42.697292+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"482'42","reported_seq":1320,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:40.635417+0000","last_change":"2024-09-21T12:00:55.389722+0000","last_active":"2024-09-21T12:01:40.635417+0000","last_peered":"2024-09-21T12:01:40.635417+0000","last_clean":"2024-09-21T12:01:40.635417+0000","last_became_active":"2024-09-21T12:00:40.763382+0000","last_became_peered":"2024-09-21T12:00:40.763382+0000","last_unstale":"2024-09-21T12:01:40.635417+0000","last_undegraded":"2024-09-21T12:01:40.635417+0000","last_fullsized":"2024-09-21T12:01:40.635417+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'42","last_scrub_stamp":"2024-09-21T12:00:55.389641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:55.389641+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:58.671006+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":970,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571310+0000","last_change":"2024-09-21T12:01:25.155304+0000","last_active":"2024-09-21T12:01:27.571310+0000","last_peered":"2024-09-21T12:01:27.571310+0000","last_clean":"2024-09-21T12:01:27.571310+0000","last_became_active":"2024-09-21T11:40:00.309182+0000","last_became_peered":"2024-09-21T11:40:00.309182+0000","last_unstale":"2024-09-21T12:01:27.571310+0000","last_undegraded":"2024-09-21T12:01:27.571310+0000","last_fullsized":"2024-09-21T12:01:27.571310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:25.155211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:25.155211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:45.135538+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":1097,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:36.839540+0000","last_change":"2024-09-21T12:01:36.839540+0000","last_active":"2024-09-21T12:01:36.839540+0000","last_peered":"2024-09-21T12:01:36.839540+0000","last_clean":"2024-09-21T12:01:36.839540+0000","last_became_active":"2024-09-21T11:57:58.528825+0000","last_became_peered":"2024-09-21T11:57:58.528825+0000","last_unstale":"2024-09-21T12:01:36.839540+0000","last_undegraded":"2024-09-21T12:01:36.839540+0000","last_fullsized":"2024-09-21T12:01:36.839540+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:01:36.839482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:01:36.839482+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:00.643537+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":961,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613812+0000","last_change":"2024-09-21T12:01:02.717098+0000","last_active":"2024-09-21T12:01:27.613812+0000","last_peered":"2024-09-21T12:01:27.613812+0000","last_clean":"2024-09-21T12:01:27.613812+0000","last_became_active":"2024-09-21T11:40:04.332925+0000","last_became_peered":"2024-09-21T11:40:04.332925+0000","last_unstale":"2024-09-21T12:01:27.613812+0000","last_undegraded":"2024-09-21T12:01:27.613812+0000","last_fullsized":"2024-09-21T12:01:27.613812+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:02.717040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:02.717040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:09.319774+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":983,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613779+0000","last_change":"2024-09-21T12:00:47.031989+0000","last_active":"2024-09-21T12:01:27.613779+0000","last_peered":"2024-09-21T12:01:27.613779+0000","last_clean":"2024-09-21T12:01:27.613779+0000","last_became_active":"2024-09-21T12:00:40.762491+0000","last_became_peered":"2024-09-21T12:00:40.762491+0000","last_unstale":"2024-09-21T12:01:27.613779+0000","last_undegraded":"2024-09-21T12:01:27.613779+0000","last_fullsized":"2024-09-21T12:01:27.613779+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:47.031940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:47.031940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:06.826727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"471'27","reported_seq":972,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571796+0000","last_change":"2024-09-21T12:01:21.967462+0000","last_active":"2024-09-21T12:01:27.571796+0000","last_peered":"2024-09-21T12:01:27.571796+0000","last_clean":"2024-09-21T12:01:27.571796+0000","last_became_active":"2024-09-21T12:01:21.966594+0000","last_became_peered":"2024-09-21T12:01:21.966594+0000","last_unstale":"2024-09-21T12:01:27.571796+0000","last_undegraded":"2024-09-21T12:01:27.571796+0000","last_fullsized":"2024-09-21T12:01:27.571796+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'27","last_scrub_stamp":"2024-09-21T12:00:53.421947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:53.421947+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:56.088796+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":451,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:42.427291+0000","last_change":"2024-09-21T12:01:42.427291+0000","last_active":"2024-09-21T12:01:42.427291+0000","last_peered":"2024-09-21T12:01:42.427291+0000","last_clean":"2024-09-21T12:01:42.427291+0000","last_became_active":"2024-09-21T12:01:22.098130+0000","last_became_peered":"2024-09-21T12:01:22.098130+0000","last_unstale":"2024-09-21T12:01:42.427291+0000","last_undegraded":"2024-09-21T12:01:42.427291+0000","last_fullsized":"2024-09-21T12:01:42.427291+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:42.427211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:42.427211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:01.322673+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":985,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613637+0000","last_change":"2024-09-21T12:01:03.276226+0000","last_active":"2024-09-21T12:01:27.613637+0000","last_peered":"2024-09-21T12:01:27.613637+0000","last_clean":"2024-09-21T12:01:27.613637+0000","last_became_active":"2024-09-21T12:00:40.761624+0000","last_became_peered":"2024-09-21T12:00:40.761624+0000","last_unstale":"2024-09-21T12:01:27.613637+0000","last_undegraded":"2024-09-21T12:01:27.613637+0000","last_fullsized":"2024-09-21T12:01:27.613637+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:03.276171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:03.276171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:30.681124+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":963,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571231+0000","last_change":"2024-09-21T12:01:03.155086+0000","last_active":"2024-09-21T12:01:27.571231+0000","last_peered":"2024-09-21T12:01:27.571231+0000","last_clean":"2024-09-21T12:01:27.571231+0000","last_became_active":"2024-09-21T11:53:36.649160+0000","last_became_peered":"2024-09-21T11:53:36.649160+0000","last_unstale":"2024-09-21T12:01:27.571231+0000","last_undegraded":"2024-09-21T12:01:27.571231+0000","last_fullsized":"2024-09-21T12:01:27.571231+0000","mapping_epoch":333,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":334,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:03.155032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:03.155032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:21.014530+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"74'79","reported_seq":213,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611950+0000","last_change":"2024-09-21T12:00:52.210298+0000","last_active":"2024-09-21T12:01:27.611950+0000","last_peered":"2024-09-21T12:01:27.611950+0000","last_clean":"2024-09-21T12:01:27.611950+0000","last_became_active":"2024-09-21T11:57:45.534494+0000","last_became_peered":"2024-09-21T11:57:45.534494+0000","last_unstale":"2024-09-21T12:01:27.611950+0000","last_undegraded":"2024-09-21T12:01:27.611950+0000","last_fullsized":"2024-09-21T12:01:27.611950+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:52.210248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:52.210248+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:52.407043+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"471'45","reported_seq":1086,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.572266+0000","last_change":"2024-09-21T12:01:21.970735+0000","last_active":"2024-09-21T12:01:27.572266+0000","last_peered":"2024-09-21T12:01:27.572266+0000","last_clean":"2024-09-21T12:01:27.572266+0000","last_became_active":"2024-09-21T12:01:21.970150+0000","last_became_peered":"2024-09-21T12:01:21.970150+0000","last_unstale":"2024-09-21T12:01:27.572266+0000","last_undegraded":"2024-09-21T12:01:27.572266+0000","last_fullsized":"2024-09-21T12:01:27.572266+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'45","last_scrub_stamp":"2024-09-21T12:01:17.257005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:17.257005+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:20.434969+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1010,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612390+0000","last_change":"2024-09-21T12:00:47.384892+0000","last_active":"2024-09-21T12:01:27.612390+0000","last_peered":"2024-09-21T12:01:27.612390+0000","last_clean":"2024-09-21T12:01:27.612390+0000","last_became_active":"2024-09-21T12:00:40.764915+0000","last_became_peered":"2024-09-21T12:00:40.764915+0000","last_unstale":"2024-09-21T12:01:27.612390+0000","last_undegraded":"2024-09-21T12:01:27.612390+0000","last_fullsized":"2024-09-21T12:01:27.612390+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:47.384844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:47.384844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.952453+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":673,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570770+0000","last_change":"2024-09-21T12:00:53.526047+0000","last_active":"2024-09-21T12:01:27.570770+0000","last_peered":"2024-09-21T12:01:27.570770+0000","last_clean":"2024-09-21T12:01:27.570770+0000","last_became_active":"2024-09-21T12:00:40.760044+0000","last_became_peered":"2024-09-21T12:00:40.760044+0000","last_unstale":"2024-09-21T12:01:27.570770+0000","last_undegraded":"2024-09-21T12:01:27.570770+0000","last_fullsized":"2024-09-21T12:01:27.570770+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:53.525998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:53.525998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:54.986795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":984,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611792+0000","last_change":"2024-09-21T12:01:13.298392+0000","last_active":"2024-09-21T12:01:27.611792+0000","last_peered":"2024-09-21T12:01:27.611792+0000","last_clean":"2024-09-21T12:01:27.611792+0000","last_became_active":"2024-09-21T12:00:40.762281+0000","last_became_peered":"2024-09-21T12:00:40.762281+0000","last_unstale":"2024-09-21T12:01:27.611792+0000","last_undegraded":"2024-09-21T12:01:27.611792+0000","last_fullsized":"2024-09-21T12:01:27.611792+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:13.298338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:13.298338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:21.475928+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"74'79","reported_seq":1087,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612028+0000","last_change":"2024-09-21T12:00:39.762851+0000","last_active":"2024-09-21T12:01:27.612028+0000","last_peered":"2024-09-21T12:01:27.612028+0000","last_clean":"2024-09-21T12:01:27.612028+0000","last_became_active":"2024-09-21T11:57:45.516610+0000","last_became_peered":"2024-09-21T11:57:45.516610+0000","last_unstale":"2024-09-21T12:01:27.612028+0000","last_undegraded":"2024-09-21T12:01:27.612028+0000","last_fullsized":"2024-09-21T12:01:27.612028+0000","mapping_epoch":416,"log_start":"0'0","ondisk_log_start":"0'0","created":414,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":4,"last_scrub":"74'79","last_scrub_stamp":"2024-09-21T12:00:39.762779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:37:26.614095+0000","last_clean_scrub_stamp":"2024-09-21T12:00:39.762779+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:45.629829+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"471'30","reported_seq":928,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571808+0000","last_change":"2024-09-21T12:01:21.965260+0000","last_active":"2024-09-21T12:01:27.571808+0000","last_peered":"2024-09-21T12:01:27.571808+0000","last_clean":"2024-09-21T12:01:27.571808+0000","last_became_active":"2024-09-21T12:01:21.964870+0000","last_became_peered":"2024-09-21T12:01:21.964870+0000","last_unstale":"2024-09-21T12:01:27.571808+0000","last_undegraded":"2024-09-21T12:01:27.571808+0000","last_fullsized":"2024-09-21T12:01:27.571808+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'30","last_scrub_stamp":"2024-09-21T12:00:44.093780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:44.093780+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:46.496540+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":978,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571633+0000","last_change":"2024-09-21T12:01:04.133660+0000","last_active":"2024-09-21T12:01:27.571633+0000","last_peered":"2024-09-21T12:01:27.571633+0000","last_clean":"2024-09-21T12:01:27.571633+0000","last_became_active":"2024-09-21T12:00:40.775319+0000","last_became_peered":"2024-09-21T12:00:40.775319+0000","last_unstale":"2024-09-21T12:01:27.571633+0000","last_undegraded":"2024-09-21T12:01:27.571633+0000","last_fullsized":"2024-09-21T12:01:27.571633+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:04.133595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:04.133595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:30.659592+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":1246,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634173+0000","last_change":"2024-09-21T12:01:22.230453+0000","last_active":"2024-09-21T12:01:35.634173+0000","last_peered":"2024-09-21T12:01:35.634173+0000","last_clean":"2024-09-21T12:01:35.634173+0000","last_became_active":"2024-09-21T11:40:04.332638+0000","last_became_peered":"2024-09-21T11:40:04.332638+0000","last_unstale":"2024-09-21T12:01:35.634173+0000","last_undegraded":"2024-09-21T12:01:35.634173+0000","last_fullsized":"2024-09-21T12:01:35.634173+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T12:01:22.230403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:22.230403+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:28.579500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":761,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.572172+0000","last_change":"2024-09-21T12:01:21.970040+0000","last_active":"2024-09-21T12:01:27.572172+0000","last_peered":"2024-09-21T12:01:27.572172+0000","last_clean":"2024-09-21T12:01:27.572172+0000","last_became_active":"2024-09-21T12:01:21.968677+0000","last_became_peered":"2024-09-21T12:01:21.968677+0000","last_unstale":"2024-09-21T12:01:27.572172+0000","last_undegraded":"2024-09-21T12:01:27.572172+0000","last_fullsized":"2024-09-21T12:01:27.572172+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:55.484970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:55.484970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:04.871190+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"471'21","reported_seq":1035,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612873+0000","last_change":"2024-09-21T12:00:49.872758+0000","last_active":"2024-09-21T12:01:27.612873+0000","last_peered":"2024-09-21T12:01:27.612873+0000","last_clean":"2024-09-21T12:01:27.612873+0000","last_became_active":"2024-09-21T12:00:40.764620+0000","last_became_peered":"2024-09-21T12:00:40.764620+0000","last_unstale":"2024-09-21T12:01:27.612873+0000","last_undegraded":"2024-09-21T12:01:27.612873+0000","last_fullsized":"2024-09-21T12:01:27.612873+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:00:49.872707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:49.872707+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:01.339809+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1000,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612618+0000","last_change":"2024-09-21T12:00:34.050812+0000","last_active":"2024-09-21T12:01:27.612618+0000","last_peered":"2024-09-21T12:01:27.612618+0000","last_clean":"2024-09-21T12:01:27.612618+0000","last_became_active":"2024-09-21T11:40:00.317759+0000","last_became_peered":"2024-09-21T11:40:00.317759+0000","last_unstale":"2024-09-21T12:01:27.612618+0000","last_undegraded":"2024-09-21T12:01:27.612618+0000","last_fullsized":"2024-09-21T12:01:27.612618+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:34.050765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:34.050765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:46.281564+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":986,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611935+0000","last_change":"2024-09-21T12:01:26.269162+0000","last_active":"2024-09-21T12:01:27.611935+0000","last_peered":"2024-09-21T12:01:27.611935+0000","last_clean":"2024-09-21T12:01:27.611935+0000","last_became_active":"2024-09-21T12:00:40.772986+0000","last_became_peered":"2024-09-21T12:00:40.772986+0000","last_unstale":"2024-09-21T12:01:27.611935+0000","last_undegraded":"2024-09-21T12:01:27.611935+0000","last_fullsized":"2024-09-21T12:01:27.611935+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:26.269116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:26.269116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:40.794995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":196,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611859+0000","last_change":"2024-09-21T12:01:21.793451+0000","last_active":"2024-09-21T12:01:27.611859+0000","last_peered":"2024-09-21T12:01:27.611859+0000","last_clean":"2024-09-21T12:01:27.611859+0000","last_became_active":"2024-09-21T11:57:58.529278+0000","last_became_peered":"2024-09-21T11:57:58.529278+0000","last_unstale":"2024-09-21T12:01:27.611859+0000","last_undegraded":"2024-09-21T12:01:27.611859+0000","last_fullsized":"2024-09-21T12:01:27.611859+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:21.793391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:21.793391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:27.944567+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"471'51","reported_seq":1097,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611454+0000","last_change":"2024-09-21T12:00:58.443531+0000","last_active":"2024-09-21T12:01:27.611454+0000","last_peered":"2024-09-21T12:01:27.611454+0000","last_clean":"2024-09-21T12:01:27.611454+0000","last_became_active":"2024-09-21T11:47:03.364639+0000","last_became_peered":"2024-09-21T11:47:03.364639+0000","last_unstale":"2024-09-21T12:01:27.611454+0000","last_undegraded":"2024-09-21T12:01:27.611454+0000","last_fullsized":"2024-09-21T12:01:27.611454+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'51","last_scrub_stamp":"2024-09-21T12:00:58.443478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:58.443478+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:20.410139+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":968,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614060+0000","last_change":"2024-09-21T12:00:35.799954+0000","last_active":"2024-09-21T12:01:27.614060+0000","last_peered":"2024-09-21T12:01:27.614060+0000","last_clean":"2024-09-21T12:01:27.614060+0000","last_became_active":"2024-09-21T11:40:00.313011+0000","last_became_peered":"2024-09-21T11:40:00.313011+0000","last_unstale":"2024-09-21T12:01:27.614060+0000","last_undegraded":"2024-09-21T12:01:27.614060+0000","last_fullsized":"2024-09-21T12:01:27.614060+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:35.799871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:35.799871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:47.164347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":969,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611577+0000","last_change":"2024-09-21T12:01:04.373627+0000","last_active":"2024-09-21T12:01:27.611577+0000","last_peered":"2024-09-21T12:01:27.611577+0000","last_clean":"2024-09-21T12:01:27.611577+0000","last_became_active":"2024-09-21T11:40:04.341419+0000","last_became_peered":"2024-09-21T11:40:04.341419+0000","last_unstale":"2024-09-21T12:01:27.611577+0000","last_undegraded":"2024-09-21T12:01:27.611577+0000","last_fullsized":"2024-09-21T12:01:27.611577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:04.373572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:04.373572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:09.392332+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":980,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:39.899206+0000","last_change":"2024-09-21T12:01:39.899206+0000","last_active":"2024-09-21T12:01:39.899206+0000","last_peered":"2024-09-21T12:01:39.899206+0000","last_clean":"2024-09-21T12:01:39.899206+0000","last_became_active":"2024-09-21T11:40:06.348039+0000","last_became_peered":"2024-09-21T11:40:06.348039+0000","last_unstale":"2024-09-21T12:01:39.899206+0000","last_undegraded":"2024-09-21T12:01:39.899206+0000","last_fullsized":"2024-09-21T12:01:39.899206+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:39.899146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:39.899146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:03:06.239649+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"471'24","reported_seq":963,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571199+0000","last_change":"2024-09-21T12:01:21.963208+0000","last_active":"2024-09-21T12:01:27.571199+0000","last_peered":"2024-09-21T12:01:27.571199+0000","last_clean":"2024-09-21T12:01:27.571199+0000","last_became_active":"2024-09-21T12:01:21.962746+0000","last_became_peered":"2024-09-21T12:01:21.962746+0000","last_unstale":"2024-09-21T12:01:27.571199+0000","last_undegraded":"2024-09-21T12:01:27.571199+0000","last_fullsized":"2024-09-21T12:01:27.571199+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'24","last_scrub_stamp":"2024-09-21T12:00:43.435914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:43.435914+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.607209+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1038,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613736+0000","last_change":"2024-09-21T12:00:56.708295+0000","last_active":"2024-09-21T12:01:27.613736+0000","last_peered":"2024-09-21T12:01:27.613736+0000","last_clean":"2024-09-21T12:01:27.613736+0000","last_became_active":"2024-09-21T12:00:40.770111+0000","last_became_peered":"2024-09-21T12:00:40.770111+0000","last_unstale":"2024-09-21T12:01:27.613736+0000","last_undegraded":"2024-09-21T12:01:27.613736+0000","last_fullsized":"2024-09-21T12:01:27.613736+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:56.708253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:56.708253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:01.065742+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":983,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611127+0000","last_change":"2024-09-21T12:01:21.242722+0000","last_active":"2024-09-21T12:01:27.611127+0000","last_peered":"2024-09-21T12:01:27.611127+0000","last_clean":"2024-09-21T12:01:27.611127+0000","last_became_active":"2024-09-21T11:47:03.385067+0000","last_became_peered":"2024-09-21T11:47:03.385067+0000","last_unstale":"2024-09-21T12:01:27.611127+0000","last_undegraded":"2024-09-21T12:01:27.611127+0000","last_fullsized":"2024-09-21T12:01:27.611127+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:21.242668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:21.242668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:37.681983+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":963,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571191+0000","last_change":"2024-09-21T12:01:02.191279+0000","last_active":"2024-09-21T12:01:27.571191+0000","last_peered":"2024-09-21T12:01:27.571191+0000","last_clean":"2024-09-21T12:01:27.571191+0000","last_became_active":"2024-09-21T11:47:03.375689+0000","last_became_peered":"2024-09-21T11:47:03.375689+0000","last_unstale":"2024-09-21T12:01:27.571191+0000","last_undegraded":"2024-09-21T12:01:27.571191+0000","last_fullsized":"2024-09-21T12:01:27.571191+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:02.191218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:02.191218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:02.649439+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"471'39","reported_seq":1041,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:31.252581+0000","last_change":"2024-09-21T12:01:31.252581+0000","last_active":"2024-09-21T12:01:31.252581+0000","last_peered":"2024-09-21T12:01:31.252581+0000","last_clean":"2024-09-21T12:01:31.252581+0000","last_became_active":"2024-09-21T11:40:02.399208+0000","last_became_peered":"2024-09-21T11:40:02.399208+0000","last_unstale":"2024-09-21T12:01:31.252581+0000","last_undegraded":"2024-09-21T12:01:31.252581+0000","last_fullsized":"2024-09-21T12:01:31.252581+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'39","last_scrub_stamp":"2024-09-21T12:01:31.252491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:31.252491+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:34.644925+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":1041,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613677+0000","last_change":"2024-09-21T12:01:17.880609+0000","last_active":"2024-09-21T12:01:27.613677+0000","last_peered":"2024-09-21T12:01:27.613677+0000","last_clean":"2024-09-21T12:01:27.613677+0000","last_became_active":"2024-09-21T12:00:40.770271+0000","last_became_peered":"2024-09-21T12:00:40.770271+0000","last_unstale":"2024-09-21T12:01:27.613677+0000","last_undegraded":"2024-09-21T12:01:27.613677+0000","last_fullsized":"2024-09-21T12:01:27.613677+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-21T12:01:17.880528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:17.880528+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:28.339143+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"482'10","reported_seq":1250,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634301+0000","last_change":"2024-09-21T12:00:43.723921+0000","last_active":"2024-09-21T12:01:35.634301+0000","last_peered":"2024-09-21T12:01:35.634301+0000","last_clean":"2024-09-21T12:01:35.634301+0000","last_became_active":"2024-09-21T12:00:40.767189+0000","last_became_peered":"2024-09-21T12:00:40.767189+0000","last_unstale":"2024-09-21T12:01:35.634301+0000","last_undegraded":"2024-09-21T12:01:35.634301+0000","last_fullsized":"2024-09-21T12:01:35.634301+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"482'10","last_scrub_stamp":"2024-09-21T12:00:43.723879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:43.723879+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:58.537685+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":956,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614352+0000","last_change":"2024-09-21T12:00:40.084762+0000","last_active":"2024-09-21T12:01:27.614352+0000","last_peered":"2024-09-21T12:01:27.614352+0000","last_clean":"2024-09-21T12:01:27.614352+0000","last_became_active":"2024-09-21T11:40:06.487327+0000","last_became_peered":"2024-09-21T11:40:06.487327+0000","last_unstale":"2024-09-21T12:01:27.614352+0000","last_undegraded":"2024-09-21T12:01:27.614352+0000","last_fullsized":"2024-09-21T12:01:27.614352+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:40.084714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:40.084714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:40.121785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1006,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612877+0000","last_change":"2024-09-21T12:01:07.492251+0000","last_active":"2024-09-21T12:01:27.612877+0000","last_peered":"2024-09-21T12:01:27.612877+0000","last_clean":"2024-09-21T12:01:27.612877+0000","last_became_active":"2024-09-21T11:57:58.527156+0000","last_became_peered":"2024-09-21T11:57:58.527156+0000","last_unstale":"2024-09-21T12:01:27.612877+0000","last_undegraded":"2024-09-21T12:01:27.612877+0000","last_fullsized":"2024-09-21T12:01:27.612877+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:07.492190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:07.492190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:22.590948+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"471'18","reported_seq":1005,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571893+0000","last_change":"2024-09-21T12:01:11.150673+0000","last_active":"2024-09-21T12:01:27.571893+0000","last_peered":"2024-09-21T12:01:27.571893+0000","last_clean":"2024-09-21T12:01:27.571893+0000","last_became_active":"2024-09-21T11:47:03.391996+0000","last_became_peered":"2024-09-21T11:47:03.391996+0000","last_unstale":"2024-09-21T12:01:27.571893+0000","last_undegraded":"2024-09-21T12:01:27.571893+0000","last_fullsized":"2024-09-21T12:01:27.571893+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'18","last_scrub_stamp":"2024-09-21T12:01:11.150621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:11.150621+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:32.972824+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1013,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:32.229743+0000","last_change":"2024-09-21T12:01:32.229743+0000","last_active":"2024-09-21T12:01:32.229743+0000","last_peered":"2024-09-21T12:01:32.229743+0000","last_clean":"2024-09-21T12:01:32.229743+0000","last_became_active":"2024-09-21T11:47:03.381128+0000","last_became_peered":"2024-09-21T11:47:03.381128+0000","last_unstale":"2024-09-21T12:01:32.229743+0000","last_undegraded":"2024-09-21T12:01:32.229743+0000","last_fullsized":"2024-09-21T12:01:32.229743+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:32.229687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:32.229687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:45.121554+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1004,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613811+0000","last_change":"2024-09-21T12:01:00.765910+0000","last_active":"2024-09-21T12:01:27.613811+0000","last_peered":"2024-09-21T12:01:27.613811+0000","last_clean":"2024-09-21T12:01:27.613811+0000","last_became_active":"2024-09-21T12:00:40.771109+0000","last_became_peered":"2024-09-21T12:00:40.771109+0000","last_unstale":"2024-09-21T12:01:27.613811+0000","last_undegraded":"2024-09-21T12:01:27.613811+0000","last_fullsized":"2024-09-21T12:01:27.613811+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:00.765857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:00.765857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:26.858419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"471'21","reported_seq":1005,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612941+0000","last_change":"2024-09-21T12:01:25.670129+0000","last_active":"2024-09-21T12:01:27.612941+0000","last_peered":"2024-09-21T12:01:27.612941+0000","last_clean":"2024-09-21T12:01:27.612941+0000","last_became_active":"2024-09-21T11:43:32.034146+0000","last_became_peered":"2024-09-21T11:43:32.034146+0000","last_unstale":"2024-09-21T12:01:27.612941+0000","last_undegraded":"2024-09-21T12:01:27.612941+0000","last_fullsized":"2024-09-21T12:01:27.612941+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:01:25.670072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:25.670072+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:26.697884+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":971,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612241+0000","last_change":"2024-09-21T12:01:21.902276+0000","last_active":"2024-09-21T12:01:27.612241+0000","last_peered":"2024-09-21T12:01:27.612241+0000","last_clean":"2024-09-21T12:01:27.612241+0000","last_became_active":"2024-09-21T12:00:40.768605+0000","last_became_peered":"2024-09-21T12:00:40.768605+0000","last_unstale":"2024-09-21T12:01:27.612241+0000","last_undegraded":"2024-09-21T12:01:27.612241+0000","last_fullsized":"2024-09-21T12:01:27.612241+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:21.902221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:21.902221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:33.292148+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":982,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571671+0000","last_change":"2024-09-21T12:00:11.248254+0000","last_active":"2024-09-21T12:01:27.571671+0000","last_peered":"2024-09-21T12:01:27.571671+0000","last_clean":"2024-09-21T12:01:27.571671+0000","last_became_active":"2024-09-21T11:47:03.378781+0000","last_became_peered":"2024-09-21T11:47:03.378781+0000","last_unstale":"2024-09-21T12:01:27.571671+0000","last_undegraded":"2024-09-21T12:01:27.571671+0000","last_fullsized":"2024-09-21T12:01:27.571671+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:11.248185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:11.248185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:38.996521+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":968,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612519+0000","last_change":"2024-09-21T12:01:01.307806+0000","last_active":"2024-09-21T12:01:27.612519+0000","last_peered":"2024-09-21T12:01:27.612519+0000","last_clean":"2024-09-21T12:01:27.612519+0000","last_became_active":"2024-09-21T12:00:40.759897+0000","last_became_peered":"2024-09-21T12:00:40.759897+0000","last_unstale":"2024-09-21T12:01:27.612519+0000","last_undegraded":"2024-09-21T12:01:27.612519+0000","last_fullsized":"2024-09-21T12:01:27.612519+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:01.307744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:01.307744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:05.151218+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"471'21","reported_seq":1005,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614228+0000","last_change":"2024-09-21T12:01:04.245620+0000","last_active":"2024-09-21T12:01:27.614228+0000","last_peered":"2024-09-21T12:01:27.614228+0000","last_clean":"2024-09-21T12:01:27.614228+0000","last_became_active":"2024-09-21T11:40:02.400402+0000","last_became_peered":"2024-09-21T11:40:02.400402+0000","last_unstale":"2024-09-21T12:01:27.614228+0000","last_undegraded":"2024-09-21T12:01:27.614228+0000","last_fullsized":"2024-09-21T12:01:27.614228+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:01:04.245503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:04.245503+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:31.561136+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":991,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:36.696204+0000","last_change":"2024-09-21T12:01:36.696204+0000","last_active":"2024-09-21T12:01:36.696204+0000","last_peered":"2024-09-21T12:01:36.696204+0000","last_clean":"2024-09-21T12:01:36.696204+0000","last_became_active":"2024-09-21T11:40:00.311059+0000","last_became_peered":"2024-09-21T11:40:00.311059+0000","last_unstale":"2024-09-21T12:01:36.696204+0000","last_undegraded":"2024-09-21T12:01:36.696204+0000","last_fullsized":"2024-09-21T12:01:36.696204+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:36.696155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:36.696155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:41.654052+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":1207,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:40.634780+0000","last_change":"2024-09-21T12:00:57.169707+0000","last_active":"2024-09-21T12:01:40.634780+0000","last_peered":"2024-09-21T12:01:40.634780+0000","last_clean":"2024-09-21T12:01:40.634780+0000","last_became_active":"2024-09-21T11:40:04.327982+0000","last_became_peered":"2024-09-21T11:40:04.327982+0000","last_unstale":"2024-09-21T12:01:40.634780+0000","last_undegraded":"2024-09-21T12:01:40.634780+0000","last_fullsized":"2024-09-21T12:01:40.634780+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-21T12:00:57.169657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:57.169657+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:21.434751+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":536,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:32.449161+0000","last_change":"2024-09-21T12:01:32.449161+0000","last_active":"2024-09-21T12:01:32.449161+0000","last_peered":"2024-09-21T12:01:32.449161+0000","last_clean":"2024-09-21T12:01:32.449161+0000","last_became_active":"2024-09-21T12:01:21.970735+0000","last_became_peered":"2024-09-21T12:01:21.970735+0000","last_unstale":"2024-09-21T12:01:32.449161+0000","last_undegraded":"2024-09-21T12:01:32.449161+0000","last_fullsized":"2024-09-21T12:01:32.449161+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:32.449113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:32.449113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:43.974374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"471'30","reported_seq":1032,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571641+0000","last_change":"2024-09-21T12:01:22.179206+0000","last_active":"2024-09-21T12:01:27.571641+0000","last_peered":"2024-09-21T12:01:27.571641+0000","last_clean":"2024-09-21T12:01:27.571641+0000","last_became_active":"2024-09-21T11:47:03.392060+0000","last_became_peered":"2024-09-21T11:47:03.392060+0000","last_unstale":"2024-09-21T12:01:27.571641+0000","last_undegraded":"2024-09-21T12:01:27.571641+0000","last_fullsized":"2024-09-21T12:01:27.571641+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'30","last_scrub_stamp":"2024-09-21T12:01:22.179153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:22.179153+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:48.533988+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":979,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.613441+0000","last_change":"2024-09-21T12:00:49.745941+0000","last_active":"2024-09-21T12:01:27.613441+0000","last_peered":"2024-09-21T12:01:27.613441+0000","last_clean":"2024-09-21T12:01:27.613441+0000","last_became_active":"2024-09-21T12:00:40.758596+0000","last_became_peered":"2024-09-21T12:00:40.758596+0000","last_unstale":"2024-09-21T12:01:27.613441+0000","last_undegraded":"2024-09-21T12:01:27.613441+0000","last_fullsized":"2024-09-21T12:01:27.613441+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:49.745886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:49.745886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:08.678659+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"205'10","reported_seq":1267,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:35.634300+0000","last_change":"2024-09-21T12:01:12.210642+0000","last_active":"2024-09-21T12:01:35.634300+0000","last_peered":"2024-09-21T12:01:35.634300+0000","last_clean":"2024-09-21T12:01:35.634300+0000","last_became_active":"2024-09-21T11:47:03.384646+0000","last_became_peered":"2024-09-21T11:47:03.384646+0000","last_unstale":"2024-09-21T12:01:35.634300+0000","last_undegraded":"2024-09-21T12:01:35.634300+0000","last_fullsized":"2024-09-21T12:01:35.634300+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'10","last_scrub_stamp":"2024-09-21T12:01:12.210581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:12.210581+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:16.606737+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":196,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:33.705254+0000","last_change":"2024-09-21T12:01:33.705254+0000","last_active":"2024-09-21T12:01:33.705254+0000","last_peered":"2024-09-21T12:01:33.705254+0000","last_clean":"2024-09-21T12:01:33.705254+0000","last_became_active":"2024-09-21T11:57:58.533810+0000","last_became_peered":"2024-09-21T11:57:58.533810+0000","last_unstale":"2024-09-21T12:01:33.705254+0000","last_undegraded":"2024-09-21T12:01:33.705254+0000","last_fullsized":"2024-09-21T12:01:33.705254+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:33.705191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:33.705191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:45.161643+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"471'27","reported_seq":983,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571502+0000","last_change":"2024-09-21T12:01:21.967060+0000","last_active":"2024-09-21T12:01:27.571502+0000","last_peered":"2024-09-21T12:01:27.571502+0000","last_clean":"2024-09-21T12:01:27.571502+0000","last_became_active":"2024-09-21T12:01:21.966662+0000","last_became_peered":"2024-09-21T12:01:21.966662+0000","last_unstale":"2024-09-21T12:01:27.571502+0000","last_undegraded":"2024-09-21T12:01:27.571502+0000","last_fullsized":"2024-09-21T12:01:27.571502+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'27","last_scrub_stamp":"2024-09-21T12:00:42.330951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:42.330951+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:48.090414+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":516,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571910+0000","last_change":"2024-09-21T12:01:22.098173+0000","last_active":"2024-09-21T12:01:27.571910+0000","last_peered":"2024-09-21T12:01:27.571910+0000","last_clean":"2024-09-21T12:01:27.571910+0000","last_became_active":"2024-09-21T12:01:22.097597+0000","last_became_peered":"2024-09-21T12:01:22.097597+0000","last_unstale":"2024-09-21T12:01:27.571910+0000","last_undegraded":"2024-09-21T12:01:27.571910+0000","last_fullsized":"2024-09-21T12:01:27.571910+0000","mapping_epoch":501,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":502,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:05.752333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:05.752333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:24.854306+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":670,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571727+0000","last_change":"2024-09-21T12:00:58.509822+0000","last_active":"2024-09-21T12:01:27.571727+0000","last_peered":"2024-09-21T12:01:27.571727+0000","last_clean":"2024-09-21T12:01:27.571727+0000","last_became_active":"2024-09-21T11:47:03.375601+0000","last_became_peered":"2024-09-21T11:47:03.375601+0000","last_unstale":"2024-09-21T12:01:27.571727+0000","last_undegraded":"2024-09-21T12:01:27.571727+0000","last_fullsized":"2024-09-21T12:01:27.571727+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:58.509775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:58.509775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:12.553557+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":955,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570353+0000","last_change":"2024-09-21T12:00:40.049914+0000","last_active":"2024-09-21T12:01:27.570353+0000","last_peered":"2024-09-21T12:01:27.570353+0000","last_clean":"2024-09-21T12:01:27.570353+0000","last_became_active":"2024-09-21T11:57:58.531213+0000","last_became_peered":"2024-09-21T11:57:58.531213+0000","last_unstale":"2024-09-21T12:01:27.570353+0000","last_undegraded":"2024-09-21T12:01:27.570353+0000","last_fullsized":"2024-09-21T12:01:27.570353+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:40.049860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:40.049860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:56.937093+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"471'21","reported_seq":690,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.571973+0000","last_change":"2024-09-21T12:01:09.639759+0000","last_active":"2024-09-21T12:01:27.571973+0000","last_peered":"2024-09-21T12:01:27.571973+0000","last_clean":"2024-09-21T12:01:27.571973+0000","last_became_active":"2024-09-21T11:47:03.379907+0000","last_became_peered":"2024-09-21T11:47:03.379907+0000","last_unstale":"2024-09-21T12:01:27.571973+0000","last_undegraded":"2024-09-21T12:01:27.571973+0000","last_fullsized":"2024-09-21T12:01:27.571973+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'21","last_scrub_stamp":"2024-09-21T12:01:09.639704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:09.639704+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:38.484984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":970,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.614034+0000","last_change":"2024-09-21T12:00:46.805427+0000","last_active":"2024-09-21T12:01:27.614034+0000","last_peered":"2024-09-21T12:01:27.614034+0000","last_clean":"2024-09-21T12:01:27.614034+0000","last_became_active":"2024-09-21T12:00:40.768603+0000","last_became_peered":"2024-09-21T12:00:40.768603+0000","last_unstale":"2024-09-21T12:01:27.614034+0000","last_undegraded":"2024-09-21T12:01:27.614034+0000","last_fullsized":"2024-09-21T12:01:27.614034+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:46.805382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:46.805382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:12.354004+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":967,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:39.026421+0000","last_change":"2024-09-21T12:01:39.026421+0000","last_active":"2024-09-21T12:01:39.026421+0000","last_peered":"2024-09-21T12:01:39.026421+0000","last_clean":"2024-09-21T12:01:39.026421+0000","last_became_active":"2024-09-21T11:47:03.377529+0000","last_became_peered":"2024-09-21T11:47:03.377529+0000","last_unstale":"2024-09-21T12:01:39.026421+0000","last_undegraded":"2024-09-21T12:01:39.026421+0000","last_fullsized":"2024-09-21T12:01:39.026421+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:39.026373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:01:39.026373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:53.324449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"66'1","reported_seq":994,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611446+0000","last_change":"2024-09-21T12:00:27.764240+0000","last_active":"2024-09-21T12:01:27.611446+0000","last_peered":"2024-09-21T12:01:27.611446+0000","last_clean":"2024-09-21T12:01:27.611446+0000","last_became_active":"2024-09-21T11:47:03.385021+0000","last_became_peered":"2024-09-21T11:47:03.385021+0000","last_unstale":"2024-09-21T12:01:27.611446+0000","last_undegraded":"2024-09-21T12:01:27.611446+0000","last_fullsized":"2024-09-21T12:01:27.611446+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-21T12:00:27.764155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:27.764155+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:49.090379+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"471'27","reported_seq":1008,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570943+0000","last_change":"2024-09-21T12:01:21.219311+0000","last_active":"2024-09-21T12:01:27.570943+0000","last_peered":"2024-09-21T12:01:27.570943+0000","last_clean":"2024-09-21T12:01:27.570943+0000","last_became_active":"2024-09-21T11:40:02.399384+0000","last_became_peered":"2024-09-21T11:40:02.399384+0000","last_unstale":"2024-09-21T12:01:27.570943+0000","last_undegraded":"2024-09-21T12:01:27.570943+0000","last_fullsized":"2024-09-21T12:01:27.570943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'27","last_scrub_stamp":"2024-09-21T12:01:21.219229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:21.219229+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:30.105512+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":989,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612890+0000","last_change":"2024-09-21T12:01:11.173623+0000","last_active":"2024-09-21T12:01:27.612890+0000","last_peered":"2024-09-21T12:01:27.612890+0000","last_clean":"2024-09-21T12:01:27.612890+0000","last_became_active":"2024-09-21T11:47:03.380538+0000","last_became_peered":"2024-09-21T11:47:03.380538+0000","last_unstale":"2024-09-21T12:01:27.612890+0000","last_undegraded":"2024-09-21T12:01:27.612890+0000","last_fullsized":"2024-09-21T12:01:27.612890+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:11.173564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:11.173564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:27.386819+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":955,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612347+0000","last_change":"2024-09-21T12:00:58.765038+0000","last_active":"2024-09-21T12:01:27.612347+0000","last_peered":"2024-09-21T12:01:27.612347+0000","last_clean":"2024-09-21T12:01:27.612347+0000","last_became_active":"2024-09-21T12:00:40.759290+0000","last_became_peered":"2024-09-21T12:00:40.759290+0000","last_unstale":"2024-09-21T12:01:27.612347+0000","last_undegraded":"2024-09-21T12:01:27.612347+0000","last_fullsized":"2024-09-21T12:01:27.612347+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:58.764974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:58.764974+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:27.673279+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":971,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:31.111709+0000","last_change":"2024-09-21T12:01:31.111709+0000","last_active":"2024-09-21T12:01:31.111709+0000","last_peered":"2024-09-21T12:01:31.111709+0000","last_clean":"2024-09-21T12:01:31.111709+0000","last_became_active":"2024-09-21T11:57:58.537121+0000","last_became_peered":"2024-09-21T11:57:58.537121+0000","last_unstale":"2024-09-21T12:01:31.111709+0000","last_undegraded":"2024-09-21T12:01:31.111709+0000","last_fullsized":"2024-09-21T12:01:31.111709+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:31.111660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:31.111660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:57.578005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"471'12","reported_seq":1006,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570469+0000","last_change":"2024-09-21T12:00:31.069099+0000","last_active":"2024-09-21T12:01:27.570469+0000","last_peered":"2024-09-21T12:01:27.570469+0000","last_clean":"2024-09-21T12:01:27.570469+0000","last_became_active":"2024-09-21T11:47:03.378841+0000","last_became_peered":"2024-09-21T11:47:03.378841+0000","last_unstale":"2024-09-21T12:01:27.570469+0000","last_undegraded":"2024-09-21T12:01:27.570469+0000","last_fullsized":"2024-09-21T12:01:27.570469+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'12","last_scrub_stamp":"2024-09-21T12:00:31.069038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:00:31.069038+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:46.062255+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":950,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612961+0000","last_change":"2024-09-21T12:00:57.855315+0000","last_active":"2024-09-21T12:01:27.612961+0000","last_peered":"2024-09-21T12:01:27.612961+0000","last_clean":"2024-09-21T12:01:27.612961+0000","last_became_active":"2024-09-21T11:40:00.313760+0000","last_became_peered":"2024-09-21T11:40:00.313760+0000","last_unstale":"2024-09-21T12:01:27.612961+0000","last_undegraded":"2024-09-21T12:01:27.612961+0000","last_fullsized":"2024-09-21T12:01:27.612961+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:57.855253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:00:57.855253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:06.707025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":993,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612077+0000","last_change":"2024-09-21T12:00:59.490747+0000","last_active":"2024-09-21T12:01:27.612077+0000","last_peered":"2024-09-21T12:01:27.612077+0000","last_clean":"2024-09-21T12:01:27.612077+0000","last_became_active":"2024-09-21T12:00:40.759826+0000","last_became_peered":"2024-09-21T12:00:40.759826+0000","last_unstale":"2024-09-21T12:01:27.612077+0000","last_undegraded":"2024-09-21T12:01:27.612077+0000","last_fullsized":"2024-09-21T12:01:27.612077+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:59.490692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:59.490692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:05.615698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":983,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611792+0000","last_change":"2024-09-21T12:01:03.761526+0000","last_active":"2024-09-21T12:01:27.611792+0000","last_peered":"2024-09-21T12:01:27.611792+0000","last_clean":"2024-09-21T12:01:27.611792+0000","last_became_active":"2024-09-21T12:00:40.769332+0000","last_became_peered":"2024-09-21T12:00:40.769332+0000","last_unstale":"2024-09-21T12:01:27.611792+0000","last_undegraded":"2024-09-21T12:01:27.611792+0000","last_fullsized":"2024-09-21T12:01:27.611792+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:03.761470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:01:03.761470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:24.060257+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":678,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.570783+0000","last_change":"2024-09-21T12:00:44.397658+0000","last_active":"2024-09-21T12:01:27.570783+0000","last_peered":"2024-09-21T12:01:27.570783+0000","last_clean":"2024-09-21T12:01:27.570783+0000","last_became_active":"2024-09-21T12:00:40.771228+0000","last_became_peered":"2024-09-21T12:00:40.771228+0000","last_unstale":"2024-09-21T12:01:27.570783+0000","last_undegraded":"2024-09-21T12:01:27.570783+0000","last_fullsized":"2024-09-21T12:01:27.570783+0000","mapping_epoch":481,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":482,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:44.397597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:05.327009+0000","last_clean_scrub_stamp":"2024-09-21T12:00:44.397597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:09.751270+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":996,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:28.103594+0000","last_change":"2024-09-21T12:01:28.103594+0000","last_active":"2024-09-21T12:01:28.103594+0000","last_peered":"2024-09-21T12:01:28.103594+0000","last_clean":"2024-09-21T12:01:28.103594+0000","last_became_active":"2024-09-21T11:57:58.535015+0000","last_became_peered":"2024-09-21T11:57:58.535015+0000","last_unstale":"2024-09-21T12:01:28.103594+0000","last_undegraded":"2024-09-21T12:01:28.103594+0000","last_fullsized":"2024-09-21T12:01:28.103594+0000","mapping_epoch":420,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:01:28.103523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:39:59.294953+0000","last_clean_scrub_stamp":"2024-09-21T12:01:28.103523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:58.043872+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"471'27","reported_seq":1086,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.611566+0000","last_change":"2024-09-21T12:01:04.453705+0000","last_active":"2024-09-21T12:01:27.611566+0000","last_peered":"2024-09-21T12:01:27.611566+0000","last_clean":"2024-09-21T12:01:27.611566+0000","last_became_active":"2024-09-21T11:47:03.366825+0000","last_became_peered":"2024-09-21T11:47:03.366825+0000","last_unstale":"2024-09-21T12:01:27.611566+0000","last_undegraded":"2024-09-21T12:01:27.611566+0000","last_fullsized":"2024-09-21T12:01:27.611566+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":0,"last_scrub":"471'27","last_scrub_stamp":"2024-09-21T12:01:04.453628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:01.298098+0000","last_clean_scrub_stamp":"2024-09-21T12:01:04.453628+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:02:33.956167+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":960,"reported_epoch":505,"state":"active+clean","last_fresh":"2024-09-21T12:01:27.612798+0000","last_change":"2024-09-21T12:00:28.776761+0000","last_active":"2024-09-21T12:01:27.612798+0000","last_peered":"2024-09-21T12:01:27.612798+0000","last_clean":"2024-09-21T12:01:27.612798+0000","last_became_active":"2024-09-21T11:40:04.336982+0000","last_became_peered":"2024-09-21T11:40:04.336982+0000","last_unstale":"2024-09-21T12:01:27.612798+0000","last_undegraded":"2024-09-21T12:01:27.612798+0000","last_fullsized":"2024-09-21T12:01:27.612798+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T12:00:28.776682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T11:40:03.313144+0000","last_clean_scrub_stamp":"2024-09-21T12:00:28.776682+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-21T12:01:45.995150+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":429,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1541,"num_read_kb":1298,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":6904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":970,"ondisk_log_size":970,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":942,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1291,"num_read_kb":1286,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234250,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":790976,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":789182,"kb_avail":92954176,"statfs":{"total":95995035648,"available":95185076224,"internally_reserved":0,"allocated":1818624,"data_stored":1137350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":808122826},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463184,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":672628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":671166,"kb_avail":93072524,"statfs":{"total":95995035648,"available":95306264576,"internally_reserved":0,"allocated":1478656,"data_stored":810014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":687274438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724820,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":691276,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":689214,"kb_avail":93053876,"statfs":{"total":95995035648,"available":95287169024,"internally_reserved":0,"allocated":2093056,"data_stored":1399500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":705755590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":481,"seq":2065879269391,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":96460,"kb_used_data":20348,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":93648692,"statfs":{"total":95995035648,"available":95896260608,"internally_reserved":0,"allocated":20836352,"data_stored":20143303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":77917859},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117281,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":805120,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":803070,"kb_avail":92940032,"statfs":{"total":95995035648,"available":95170592768,"internally_reserved":0,"allocated":2080768,"data_stored":1399376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":822344135},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411622,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":799236,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":797502,"kb_avail":92945916,"statfs":{"total":95995035648,"available":95176617984,"internally_reserved":0,"allocated":1757184,"data_stored":1072407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":816642501},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575149,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":786524,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":785022,"kb_avail":92958628,"statfs":{"total":95995035648,"available":95189635072,"internally_reserved":0,"allocated":1519616,"data_stored":810159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":803862982},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738675,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":808720,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":807230,"kb_avail":92936432,"statfs":{"total":95995035648,"available":95166906368,"internally_reserved":0,"allocated":1507328,"data_stored":810788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":826603973},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T12:01:48.444 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-21T12:01:48.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301 shell --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-21T12:01:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:48 smithi175 ceph-mon[32613]: 3.c scrub starts 2024-09-21T12:01:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:48 smithi175 ceph-mon[32613]: 3.c scrub ok 2024-09-21T12:01:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:48 smithi175 ceph-mon[32613]: 5.17 scrub starts 2024-09-21T12:01:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:48 smithi175 ceph-mon[32613]: 5.17 scrub ok 2024-09-21T12:01:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:48 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-21T12:01:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:48 smithi175 ceph-mon[32613]: 5.19 scrub starts 2024-09-21T12:01:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:48 smithi175 ceph-mon[32613]: 5.19 scrub ok 2024-09-21T12:01:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:48 smithi175 ceph-mon[32613]: from='client.17130 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T12:01:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:48 smithi175 ceph-mon[32613]: 3.d scrub starts 2024-09-21T12:01:48.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:48 smithi175 ceph-mon[32613]: 3.d scrub ok 2024-09-21T12:01:48.707 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/mon.a/config 2024-09-21T12:01:49.444 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-mon[29710]: 4.7 scrub starts 2024-09-21T12:01:49.445 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-mon[29710]: 4.7 scrub ok 2024-09-21T12:01:49.445 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-mon[29710]: pgmap v1074: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:01:49.445 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-mon[29710]: 5.1e scrub starts 2024-09-21T12:01:49.445 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-mon[29710]: 5.1e scrub ok 2024-09-21T12:01:49.445 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-mon[36454]: 4.7 scrub starts 2024-09-21T12:01:49.445 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-mon[36454]: 4.7 scrub ok 2024-09-21T12:01:49.445 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-mon[36454]: pgmap v1074: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:01:49.445 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-mon[36454]: 5.1e scrub starts 2024-09-21T12:01:49.445 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-mon[36454]: 5.1e scrub ok 2024-09-21T12:01:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:49 smithi175 ceph-mon[32613]: 4.7 scrub starts 2024-09-21T12:01:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:49 smithi175 ceph-mon[32613]: 4.7 scrub ok 2024-09-21T12:01:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:49 smithi175 ceph-mon[32613]: pgmap v1074: 142 pgs: 142 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T12:01:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:49 smithi175 ceph-mon[32613]: 5.1e scrub starts 2024-09-21T12:01:49.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:49 smithi175 ceph-mon[32613]: 5.1e scrub ok 2024-09-21T12:01:49.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:49] ENGINE Bus STOPPING 2024-09-21T12:01:50.114 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:49] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T12:01:50.114 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:49] ENGINE Bus STOPPED 2024-09-21T12:01:50.114 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:49] ENGINE Bus STARTING 2024-09-21T12:01:50.114 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:49] ENGINE Serving on http://:::9283 2024-09-21T12:01:50.114 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:49 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:49] ENGINE Bus STARTED 2024-09-21T12:01:50.186 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T12:01:50.195 INFO:tasks.cephadm:Teardown begin 2024-09-21T12:01:50.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T12:01:50.237 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T12:01:50.290 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T12:01:50.290 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T12:01:50.318 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T12:01:50.339 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T12:01:50.339 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T12:01:50.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.a 2024-09-21T12:01:50.380 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1802250030' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-21T12:01:50.380 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[29710]: from='client.? 172.21.15.43:0/1802250030' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-21T12:01:50.380 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:01:50.380 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[29710]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[29710]: 6.10 scrub starts 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[29710]: 6.10 scrub ok 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[29710]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[29710]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1802250030' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[36454]: from='client.? 172.21.15.43:0/1802250030' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[36454]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:01:50.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[36454]: 6.10 scrub starts 2024-09-21T12:01:50.382 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[36454]: 6.10 scrub ok 2024-09-21T12:01:50.382 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[36454]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T12:01:50.382 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-mon[36454]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T12:01:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1802250030' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-21T12:01:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:50 smithi175 ceph-mon[32613]: from='client.? 172.21.15.43:0/1802250030' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-21T12:01:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:50 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T12:01:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:50 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T12:01:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:50 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T12:01:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:50 smithi175 ceph-mon[32613]: from='mgr.14586 ' entity='mgr.y' 2024-09-21T12:01:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:50 smithi175 ceph-mon[32613]: 6.10 scrub starts 2024-09-21T12:01:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:50 smithi175 ceph-mon[32613]: 6.10 scrub ok 2024-09-21T12:01:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:50 smithi175 ceph-mon[32613]: from='mgr.14586 172.21.15.43:0/3786067687' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-21T12:01:50.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:50 smithi175 ceph-mon[32613]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-21T12:01:50.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 systemd[1]: Stopping Ceph mon.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:01:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a[29706]: 2024-09-21T12:01:50.667+0000 7f5b5a0e7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:01:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a[29706]: 2024-09-21T12:01:50.667+0000 7f5b5a0e7640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T12:01:50.991 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:50 smithi043 podman[207529]: 2024-09-21 12:01:50.71892297 +0000 UTC m=+0.163194809 container died 4062dd7fcef735b53b9051ca2cfd3d11de41564243d23bcdfd68f51637eee38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T12:01:51.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:51 smithi043 podman[207529]: 2024-09-21 12:01:51.117788693 +0000 UTC m=+0.562060531 container cleanup 4062dd7fcef735b53b9051ca2cfd3d11de41564243d23bcdfd68f51637eee38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:01:51.387 INFO:journalctl@ceph.mon.a.smithi043.stdout:Sep 21 12:01:51 smithi043 bash[207529]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-a 2024-09-21T12:01:51.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.a.service' 2024-09-21T12:01:51.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:01:51.639 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T12:01:51.639 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-21T12:01:51.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.c 2024-09-21T12:01:51.940 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:51 smithi043 systemd[1]: Stopping Ceph mon.c for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:01:52.241 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:51 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:51] ENGINE Bus STOPPING 2024-09-21T12:01:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:51 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-c[36450]: 2024-09-21T12:01:51.938+0000 7f94d1696640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:01:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:51 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-c[36450]: 2024-09-21T12:01:51.938+0000 7f94d1696640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T12:01:52.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:52 smithi043 podman[207651]: 2024-09-21 12:01:52.061577856 +0000 UTC m=+0.230630789 container died 56e77e4ebe6d8eb3b74d4cba1581d99ed0f2df68af9875b27161f0511b460730 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:01:52.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:01:52 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug there is no tcmu-runner data available 2024-09-21T12:01:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:52 smithi043 podman[207651]: 2024-09-21 12:01:52.581429798 +0000 UTC m=+0.750482772 container cleanup 56e77e4ebe6d8eb3b74d4cba1581d99ed0f2df68af9875b27161f0511b460730 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-c, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T12:01:52.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Sep 21 12:01:52 smithi043 bash[207651]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-c 2024-09-21T12:01:52.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:52 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:52] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-21T12:01:52.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:52 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:52] ENGINE Bus STOPPED 2024-09-21T12:01:52.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:52 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:52] ENGINE Bus STARTING 2024-09-21T12:01:52.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:52 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:52] ENGINE Serving on http://:::9283 2024-09-21T12:01:52.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:52 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: [21/Sep/2024:12:01:52] ENGINE Bus STARTED 2024-09-21T12:01:53.018 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.c.service' 2024-09-21T12:01:53.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:01:53.053 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-21T12:01:53.053 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-21T12:01:53.054 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.b 2024-09-21T12:01:53.511 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:53 smithi175 systemd[1]: Stopping Ceph mon.b for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:01:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:53 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-b[32609]: 2024-09-21T12:01:53.648+0000 7f2ee0193640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:01:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:53 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-b[32609]: 2024-09-21T12:01:53.648+0000 7f2ee0193640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T12:01:54.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:53 smithi175 podman[154736]: 2024-09-21 12:01:53.863363976 +0000 UTC m=+0.616698315 container died 5423a374aab414a26f46d41f2f79717b99f63eb4affea51b609c37901fd59f3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-b, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T12:01:54.501 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:54 smithi175 podman[154736]: 2024-09-21 12:01:54.299073461 +0000 UTC m=+1.052407878 container cleanup 5423a374aab414a26f46d41f2f79717b99f63eb4affea51b609c37901fd59f3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-b, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T12:01:54.501 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 21 12:01:54 smithi175 bash[154736]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mon-b 2024-09-21T12:01:54.723 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mon.b.service' 2024-09-21T12:01:54.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:01:54.761 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-21T12:01:54.761 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T12:01:54.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.y 2024-09-21T12:01:55.071 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:54 smithi043 systemd[1]: Stopping Ceph mgr.y for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:01:55.447 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: 2024-09-21T12:01:55.069+0000 7fe24d8af640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:01:55.447 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:55 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y[29937]: 2024-09-21T12:01:55.069+0000 7fe24d8af640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T12:01:55.447 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:55 smithi043 podman[207784]: 2024-09-21 12:01:55.100723792 +0000 UTC m=+0.139677663 container died 4ca0cb7bf57c6af598cb451e44cfb4ade33d19c1a72c59c587c25b56f6a773e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:01:55.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:55 smithi043 podman[207784]: 2024-09-21 12:01:55.5701085 +0000 UTC m=+0.609062366 container cleanup 4ca0cb7bf57c6af598cb451e44cfb4ade33d19c1a72c59c587c25b56f6a773e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:01:55.741 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:55 smithi043 bash[207784]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y 2024-09-21T12:01:56.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.y.service' 2024-09-21T12:01:56.032 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:55 smithi043 podman[207797]: 2024-09-21 12:01:55.763726633 +0000 UTC m=+0.657409251 container remove 4ca0cb7bf57c6af598cb451e44cfb4ade33d19c1a72c59c587c25b56f6a773e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5c1cdc3c470dae78e91a0ed2bcf13509a046d301, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-y, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS) 2024-09-21T12:01:56.032 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:56 smithi043 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.y.service: Deactivated successfully. 2024-09-21T12:01:56.032 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:56 smithi043 systemd[1]: Stopped Ceph mgr.y for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T12:01:56.032 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Sep 21 12:01:56 smithi043 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.y.service: Consumed 1min 11.625s CPU time. 2024-09-21T12:01:56.046 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:01:56.046 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T12:01:56.046 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T12:01:56.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.x 2024-09-21T12:01:56.370 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 12:01:56 smithi175 systemd[1]: Stopping Ceph mgr.x for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:01:56.655 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 12:01:56 smithi175 podman[154858]: 2024-09-21 12:01:56.368809225 +0000 UTC m=+0.128741068 container died fec51c1f8ac423528b1d77933b2baf80ef741bb7b97bd044f117a4070c8c0596 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:01:57.011 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 12:01:56 smithi175 podman[154858]: 2024-09-21 12:01:56.778994539 +0000 UTC m=+0.538926373 container cleanup fec51c1f8ac423528b1d77933b2baf80ef741bb7b97bd044f117a4070c8c0596 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T12:01:57.011 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 21 12:01:56 smithi175 bash[154858]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-mgr-x 2024-09-21T12:01:57.207 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@mgr.x.service' 2024-09-21T12:01:57.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:01:57.242 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T12:01:57.242 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T12:01:57.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.0 2024-09-21T12:01:57.568 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:01:57 smithi043 systemd[1]: Stopping Ceph osd.0 for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:01:57.991 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:01:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0[39801]: 2024-09-21T12:01:57.565+0000 7fdea7887640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:01:57.991 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:01:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0[39801]: 2024-09-21T12:01:57.565+0000 7fdea7887640 -1 osd.0 505 *** Got signal Terminated *** 2024-09-21T12:01:57.991 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:01:57 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0[39801]: 2024-09-21T12:01:57.565+0000 7fdea7887640 -1 osd.0 505 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:02:03.328 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:02:03 smithi043 podman[207907]: 2024-09-21 12:02:03.024039209 +0000 UTC m=+5.583993038 container died 0dbf844943b65868868adc3cb8e7e6bcdcb4f073583e5dd045bc541aa57a141a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T12:02:03.741 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:02:03 smithi043 podman[207907]: 2024-09-21 12:02:03.483505768 +0000 UTC m=+6.043459636 container cleanup 0dbf844943b65868868adc3cb8e7e6bcdcb4f073583e5dd045bc541aa57a141a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T12:02:03.741 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:02:03 smithi043 bash[207907]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0 2024-09-21T12:02:04.115 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:02:03 smithi043 podman[207926]: 2024-09-21 12:02:03.781481498 +0000 UTC m=+0.750492582 container remove 0dbf844943b65868868adc3cb8e7e6bcdcb4f073583e5dd045bc541aa57a141a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T12:02:04.408 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:02:04 smithi043 podman[208012]: 2024-09-21 12:02:04.016701386 +0000 UTC m=+0.016556848 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:02:04.409 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:02:04 smithi043 podman[208012]: 2024-09-21 12:02:04.127793852 +0000 UTC m=+0.127649339 container create be4ee6343810d6aaa633e88f0d74ab76db8cdfa86fd8c35308adf3a81118e5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T12:02:04.409 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:02:04 smithi043 podman[208012]: 2024-09-21 12:02:04.323856523 +0000 UTC m=+0.323711977 container init be4ee6343810d6aaa633e88f0d74ab76db8cdfa86fd8c35308adf3a81118e5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:02:04.409 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:02:04 smithi043 podman[208012]: 2024-09-21 12:02:04.327237106 +0000 UTC m=+0.327092549 container start be4ee6343810d6aaa633e88f0d74ab76db8cdfa86fd8c35308adf3a81118e5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:02:04.741 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:02:04 smithi043 podman[208012]: 2024-09-21 12:02:04.407142423 +0000 UTC m=+0.406997865 container attach be4ee6343810d6aaa633e88f0d74ab76db8cdfa86fd8c35308adf3a81118e5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0-deactivate, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T12:02:04.741 INFO:journalctl@ceph.osd.0.smithi043.stdout:Sep 21 12:02:04 smithi043 podman[208012]: 2024-09-21 12:02:04.58287955 +0000 UTC m=+0.582735005 container died be4ee6343810d6aaa633e88f0d74ab76db8cdfa86fd8c35308adf3a81118e5f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T12:02:05.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.0.service' 2024-09-21T12:02:05.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:02:05.413 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T12:02:05.413 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T12:02:05.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.1 2024-09-21T12:02:05.767 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:05 smithi043 systemd[1]: Stopping Ceph osd.1 for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:02:06.241 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1[45207]: 2024-09-21T12:02:05.764+0000 7fc7219da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:02:06.241 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1[45207]: 2024-09-21T12:02:05.764+0000 7fc7219da640 -1 osd.1 505 *** Got signal Terminated *** 2024-09-21T12:02:06.241 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:05 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1[45207]: 2024-09-21T12:02:05.764+0000 7fc7219da640 -1 osd.1 505 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:02:11.658 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:11 smithi043 podman[208117]: 2024-09-21 12:02:11.309913912 +0000 UTC m=+5.667122809 container died 79f6ba01d42eeb69a08a3c8b8725d8425ed170b3f64c55c86c2770bc9f870e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T12:02:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:02:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:02:11.415Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:02:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:02:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:02:11.407Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:02:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:02:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:02:11.415Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:02:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:02:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:02:11.415Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:02:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:02:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:02:11.415Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:02:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:02:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:02:11.415Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:02:11.762 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:02:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:02:11.415Z caller=refresh.go:90 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:02:11.991 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:11 smithi043 podman[208117]: 2024-09-21 12:02:11.848235467 +0000 UTC m=+6.205444357 container cleanup 79f6ba01d42eeb69a08a3c8b8725d8425ed170b3f64c55c86c2770bc9f870e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2) 2024-09-21T12:02:11.991 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:11 smithi043 bash[208117]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1 2024-09-21T12:02:12.490 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:12 smithi043 podman[208143]: 2024-09-21 12:02:12.083243291 +0000 UTC m=+0.772788265 container remove 79f6ba01d42eeb69a08a3c8b8725d8425ed170b3f64c55c86c2770bc9f870e60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T12:02:12.854 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:12 smithi043 podman[208235]: 2024-09-21 12:02:12.436523407 +0000 UTC m=+0.017040440 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:02:12.854 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:12 smithi043 podman[208235]: 2024-09-21 12:02:12.533908968 +0000 UTC m=+0.114425992 container create c68acd56b2019a859a5eb078b3a5b2b0724bc188a76047fa27c9e6d2aea95f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:02:12.854 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:12 smithi043 podman[208235]: 2024-09-21 12:02:12.761566255 +0000 UTC m=+0.342083286 container init c68acd56b2019a859a5eb078b3a5b2b0724bc188a76047fa27c9e6d2aea95f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:02:12.854 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:12 smithi043 podman[208235]: 2024-09-21 12:02:12.765378736 +0000 UTC m=+0.345895758 container start c68acd56b2019a859a5eb078b3a5b2b0724bc188a76047fa27c9e6d2aea95f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-21T12:02:12.854 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:12 smithi043 podman[208235]: 2024-09-21 12:02:12.852984553 +0000 UTC m=+0.433501591 container attach c68acd56b2019a859a5eb078b3a5b2b0724bc188a76047fa27c9e6d2aea95f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T12:02:13.241 INFO:journalctl@ceph.osd.1.smithi043.stdout:Sep 21 12:02:12 smithi043 podman[208235]: 2024-09-21 12:02:12.966809942 +0000 UTC m=+0.547326963 container died c68acd56b2019a859a5eb078b3a5b2b0724bc188a76047fa27c9e6d2aea95f60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T12:02:13.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.1.service' 2024-09-21T12:02:13.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:02:13.656 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T12:02:13.656 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T12:02:13.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.2 2024-09-21T12:02:13.991 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:13 smithi043 systemd[1]: Stopping Ceph osd.2 for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:02:14.491 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:14 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2[50799]: 2024-09-21T12:02:14.011+0000 7f52b0833640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:02:14.491 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:14 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2[50799]: 2024-09-21T12:02:14.011+0000 7f52b0833640 -1 osd.2 505 *** Got signal Terminated *** 2024-09-21T12:02:14.491 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:14 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2[50799]: 2024-09-21T12:02:14.011+0000 7f52b0833640 -1 osd.2 505 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:02:19.812 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:19 smithi043 podman[208339]: 2024-09-21 12:02:19.53268893 +0000 UTC m=+5.651441081 container died faa4ebb8dad667a40024adbc536b1708f13cb5d6925b6e19bcd157450bd41fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T12:02:20.241 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:20 smithi043 podman[208339]: 2024-09-21 12:02:20.001156885 +0000 UTC m=+6.119909035 container cleanup faa4ebb8dad667a40024adbc536b1708f13cb5d6925b6e19bcd157450bd41fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T12:02:20.241 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:20 smithi043 bash[208339]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2 2024-09-21T12:02:20.679 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:20 smithi043 podman[208356]: 2024-09-21 12:02:20.361403551 +0000 UTC m=+0.825398152 container remove faa4ebb8dad667a40024adbc536b1708f13cb5d6925b6e19bcd157450bd41fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:02:20.933 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:20 smithi043 podman[208444]: 2024-09-21 12:02:20.581356997 +0000 UTC m=+0.016838564 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:02:20.933 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:20 smithi043 podman[208444]: 2024-09-21 12:02:20.75438802 +0000 UTC m=+0.189869563 container create 7aeee036357b8012f46e88aacc0f36937e758eea789c8245a9d2e2713132db62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:02:20.933 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:20 smithi043 podman[208444]: 2024-09-21 12:02:20.931614235 +0000 UTC m=+0.367095788 container init 7aeee036357b8012f46e88aacc0f36937e758eea789c8245a9d2e2713132db62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:02:21.241 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:20 smithi043 podman[208444]: 2024-09-21 12:02:20.935219011 +0000 UTC m=+0.370700554 container start 7aeee036357b8012f46e88aacc0f36937e758eea789c8245a9d2e2713132db62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T12:02:21.241 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:21 smithi043 podman[208444]: 2024-09-21 12:02:21.048328309 +0000 UTC m=+0.483809853 container attach 7aeee036357b8012f46e88aacc0f36937e758eea789c8245a9d2e2713132db62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2-deactivate, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:02:21.241 INFO:journalctl@ceph.osd.2.smithi043.stdout:Sep 21 12:02:21 smithi043 podman[208444]: 2024-09-21 12:02:21.128606439 +0000 UTC m=+0.564088000 container died 7aeee036357b8012f46e88aacc0f36937e758eea789c8245a9d2e2713132db62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T12:02:21.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.2.service' 2024-09-21T12:02:21.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:02:21.754 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T12:02:21.754 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T12:02:21.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.3 2024-09-21T12:02:21.991 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:21 smithi043 systemd[1]: Stopping Ceph osd.3 for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:02:22.493 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:22 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3[56403]: 2024-09-21T12:02:22.104+0000 7f28cef06640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:02:22.494 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:22 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3[56403]: 2024-09-21T12:02:22.104+0000 7f28cef06640 -1 osd.3 505 *** Got signal Terminated *** 2024-09-21T12:02:22.494 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:22 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3[56403]: 2024-09-21T12:02:22.104+0000 7f28cef06640 -1 osd.3 505 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:02:24.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:23 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:23.756+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:24.734 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:24 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:24.481+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:25.010 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:24 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:24.732+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:25.510 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:25 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:25.451+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:25.741 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:25 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3[56403]: 2024-09-21T12:02:25.270+0000 7f28c7513640 -1 osd.3 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:01.361797+0000 front 2024-09-21T12:02:01.361913+0000 (oldest deadline 2024-09-21T12:02:24.861987+0000) 2024-09-21T12:02:26.010 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:25 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:25.704+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:26.511 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:26 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:26.412+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:26.511 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:26 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:26.169+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:59.248455+0000 front 2024-09-21T12:01:59.248391+0000 (oldest deadline 2024-09-21T12:02:25.148355+0000) 2024-09-21T12:02:26.741 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:26 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3[56403]: 2024-09-21T12:02:26.244+0000 7f28c7513640 -1 osd.3 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:01.361797+0000 front 2024-09-21T12:02:01.361913+0000 (oldest deadline 2024-09-21T12:02:24.861987+0000) 2024-09-21T12:02:27.010 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:26 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:26.704+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:27.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:26 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:26.623+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:27.423 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:27 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:27.126+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:59.248455+0000 front 2024-09-21T12:01:59.248391+0000 (oldest deadline 2024-09-21T12:02:25.148355+0000) 2024-09-21T12:02:27.760 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:27 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:27.421+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:27.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:27 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:27.647+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:27.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:27 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:27.657+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:27.949 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:27 smithi043 podman[208546]: 2024-09-21 12:02:27.677625252 +0000 UTC m=+5.681555702 container died d52a91576f37cf2ac16c450650c975b9b84217622e363cf412b63c477957ce96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:02:28.240 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:28 smithi043 podman[208546]: 2024-09-21 12:02:28.138230304 +0000 UTC m=+6.142160752 container cleanup d52a91576f37cf2ac16c450650c975b9b84217622e363cf412b63c477957ce96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:02:28.241 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:28 smithi043 bash[208546]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3 2024-09-21T12:02:28.457 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:28 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:28.081+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:59.248455+0000 front 2024-09-21T12:01:59.248391+0000 (oldest deadline 2024-09-21T12:02:25.148355+0000) 2024-09-21T12:02:28.741 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:28 smithi043 podman[208562]: 2024-09-21 12:02:28.46478176 +0000 UTC m=+0.789250920 container remove d52a91576f37cf2ac16c450650c975b9b84217622e363cf412b63c477957ce96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T12:02:28.760 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:28 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:28.456+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:28.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:28 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:28.687+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:28.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:28 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:28.610+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:29.127 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:28 smithi043 podman[208651]: 2024-09-21 12:02:28.717697355 +0000 UTC m=+0.016773736 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:02:29.127 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:28 smithi043 podman[208651]: 2024-09-21 12:02:28.848887293 +0000 UTC m=+0.147963657 container create 2df17c99e835a9c0efd29dddb361c69e794ee5c5da1700e088e642a174b00e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T12:02:29.127 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:29 smithi043 podman[208651]: 2024-09-21 12:02:29.04279918 +0000 UTC m=+0.341875555 container init 2df17c99e835a9c0efd29dddb361c69e794ee5c5da1700e088e642a174b00e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T12:02:29.127 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:29 smithi043 podman[208651]: 2024-09-21 12:02:29.046099625 +0000 UTC m=+0.345175996 container start 2df17c99e835a9c0efd29dddb361c69e794ee5c5da1700e088e642a174b00e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:02:29.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:29 smithi043 podman[208651]: 2024-09-21 12:02:29.126041463 +0000 UTC m=+0.425117836 container attach 2df17c99e835a9c0efd29dddb361c69e794ee5c5da1700e088e642a174b00e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T12:02:29.477 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:29 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:29.084+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:59.248455+0000 front 2024-09-21T12:01:59.248391+0000 (oldest deadline 2024-09-21T12:02:25.148355+0000) 2024-09-21T12:02:29.491 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 21 12:02:29 smithi043 podman[208651]: 2024-09-21 12:02:29.239048534 +0000 UTC m=+0.538124902 container died 2df17c99e835a9c0efd29dddb361c69e794ee5c5da1700e088e642a174b00e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T12:02:29.760 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:29 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:29.652+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:29.761 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:29 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:29.475+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:29.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:29 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:29.600+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:29.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.3.service' 2024-09-21T12:02:29.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:02:29.910 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T12:02:29.910 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T12:02:29.910 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.4 2024-09-21T12:02:30.215 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:29 smithi175 systemd[1]: Stopping Ceph osd.4 for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:02:30.216 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:30.065+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:59.248455+0000 front 2024-09-21T12:01:59.248391+0000 (oldest deadline 2024-09-21T12:02:25.148355+0000) 2024-09-21T12:02:30.510 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:30.213+0000 7faee24f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:02:30.511 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:30.213+0000 7faee24f6640 -1 osd.4 505 *** Got signal Terminated *** 2024-09-21T12:02:30.511 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:30.213+0000 7faee24f6640 -1 osd.4 505 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:02:31.011 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:30.516+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:31.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:30.688+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:31.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:30 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:30.552+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:31.496 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:31.073+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:59.248455+0000 front 2024-09-21T12:01:59.248391+0000 (oldest deadline 2024-09-21T12:02:25.148355+0000) 2024-09-21T12:02:31.760 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:31.557+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:31.761 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:31.494+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:31.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:31 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:31.684+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:32.510 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:32 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:32.086+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:59.248455+0000 front 2024-09-21T12:01:59.248391+0000 (oldest deadline 2024-09-21T12:02:25.148355+0000) 2024-09-21T12:02:33.010 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:32 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:32.511+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:33.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:32 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:32.535+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:33.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:32 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:32.662+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:33.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:32 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:32.662+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:33.470 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:33 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:33.070+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:59.248455+0000 front 2024-09-21T12:01:59.248391+0000 (oldest deadline 2024-09-21T12:02:25.148355+0000) 2024-09-21T12:02:33.761 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:33 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:33.468+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:33.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:33 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:33.625+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:33.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:33 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:33.625+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:33.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:33 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:33.557+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:34.508 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:34 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:34.084+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:59.248455+0000 front 2024-09-21T12:01:59.248391+0000 (oldest deadline 2024-09-21T12:02:25.148355+0000) 2024-09-21T12:02:34.761 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:34 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:34.506+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:34.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:34 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:34.617+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:34.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:34 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:34.617+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:34.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:34 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:34.574+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:34.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:34 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:34.574+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:35.307 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:35 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:35.044+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:59.248455+0000 front 2024-09-21T12:01:59.248391+0000 (oldest deadline 2024-09-21T12:02:25.148355+0000) 2024-09-21T12:02:35.307 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:35 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4[152696]: 2024-09-21T12:02:35.044+0000 7faedeb0b640 -1 osd.4 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.349776+0000 front 2024-09-21T12:02:10.349855+0000 (oldest deadline 2024-09-21T12:02:34.449773+0000) 2024-09-21T12:02:35.594 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:35 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:35.507+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:35.594 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:35 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:35.507+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.679262+0000 front 2024-09-21T12:02:10.679319+0000 (oldest deadline 2024-09-21T12:02:34.779274+0000) 2024-09-21T12:02:35.594 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:35 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:35.550+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:35.594 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:35 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:35.550+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:36.000 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:35 smithi175 podman[154996]: 2024-09-21 12:02:35.725059812 +0000 UTC m=+5.623120885 container died 7420c7dbcf221bd57281830ddbab98762859f033d1494c379c9285404067aff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:02:36.000 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:35 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:35.592+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:36.000 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:35 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:35.592+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:36.260 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:36 smithi175 podman[154996]: 2024-09-21 12:02:36.096874102 +0000 UTC m=+5.994935173 container cleanup 7420c7dbcf221bd57281830ddbab98762859f033d1494c379c9285404067aff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=) 2024-09-21T12:02:36.261 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:36 smithi175 bash[154996]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4 2024-09-21T12:02:36.616 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:36 smithi175 podman[155014]: 2024-09-21 12:02:36.339703006 +0000 UTC m=+0.611302299 container remove 7420c7dbcf221bd57281830ddbab98762859f033d1494c379c9285404067aff4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:02:36.616 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:36 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:36.547+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:36.616 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:36 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:36.547+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.679262+0000 front 2024-09-21T12:02:10.679319+0000 (oldest deadline 2024-09-21T12:02:34.779274+0000) 2024-09-21T12:02:36.616 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:36 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:36.504+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:36.616 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:36 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:36.505+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:36.936 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:36 smithi175 podman[155104]: 2024-09-21 12:02:36.575115637 +0000 UTC m=+0.016377846 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:02:36.936 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:36 smithi175 podman[155104]: 2024-09-21 12:02:36.707735898 +0000 UTC m=+0.148998106 container create afa80595daaa7253433c841dd8c85203736e4d94a95b97e7db3c9747c4e4c376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T12:02:36.937 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:36 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:36.614+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:36.937 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:36 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:36.614+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:37.261 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:36 smithi175 podman[155104]: 2024-09-21 12:02:36.935372334 +0000 UTC m=+0.376634547 container init afa80595daaa7253433c841dd8c85203736e4d94a95b97e7db3c9747c4e4c376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:02:37.261 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:36 smithi175 podman[155104]: 2024-09-21 12:02:36.938912241 +0000 UTC m=+0.380174454 container start afa80595daaa7253433c841dd8c85203736e4d94a95b97e7db3c9747c4e4c376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-21T12:02:37.261 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:37 smithi175 podman[155104]: 2024-09-21 12:02:37.035312866 +0000 UTC m=+0.476575084 container attach afa80595daaa7253433c841dd8c85203736e4d94a95b97e7db3c9747c4e4c376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:02:37.261 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 21 12:02:37 smithi175 podman[155104]: 2024-09-21 12:02:37.12468597 +0000 UTC m=+0.565948179 container died afa80595daaa7253433c841dd8c85203736e4d94a95b97e7db3c9747c4e4c376 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-4-deactivate, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:02:37.707 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:37 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:37.565+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:37.707 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:37 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:37.565+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.679262+0000 front 2024-09-21T12:02:10.679319+0000 (oldest deadline 2024-09-21T12:02:34.779274+0000) 2024-09-21T12:02:37.707 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:37 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:37.574+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:37.708 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:37 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:37.574+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:37.708 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:37 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:37.472+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:37.708 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:37 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:37.472+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:37.722 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.4.service' 2024-09-21T12:02:37.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:02:37.762 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T12:02:37.763 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T12:02:37.763 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.5 2024-09-21T12:02:38.010 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:37 smithi175 systemd[1]: Stopping Ceph osd.5 for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:02:38.371 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:38 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:38.100+0000 7fd298ff6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:02:38.371 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:38 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:38.100+0000 7fd298ff6640 -1 osd.5 505 *** Got signal Terminated *** 2024-09-21T12:02:38.371 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:38 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:38.100+0000 7fd298ff6640 -1 osd.5 505 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:02:38.760 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:38 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:38.565+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:38.761 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:38 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:38.565+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.679262+0000 front 2024-09-21T12:02:10.679319+0000 (oldest deadline 2024-09-21T12:02:34.779274+0000) 2024-09-21T12:02:38.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:38 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:38.610+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:38.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:38 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:38.610+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:38.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:38 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:38.501+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:38.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:38 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:38.501+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:40.011 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:39 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:39.559+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:40.011 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:39 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:39.559+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.679262+0000 front 2024-09-21T12:02:10.679319+0000 (oldest deadline 2024-09-21T12:02:34.779274+0000) 2024-09-21T12:02:40.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:39 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:39.604+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:40.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:39 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:39.604+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:40.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:39 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:39.545+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:40.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:39 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:39.545+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:41.010 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:40.561+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:41.011 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:40.561+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.679262+0000 front 2024-09-21T12:02:10.679319+0000 (oldest deadline 2024-09-21T12:02:34.779274+0000) 2024-09-21T12:02:41.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:40.607+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:41.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:40.607+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:41.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:40.607+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:41.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:40.554+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:41.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:40.554+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:41.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:40 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:40.554+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:42.011 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:41.545+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:42.011 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:41.545+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.679262+0000 front 2024-09-21T12:02:10.679319+0000 (oldest deadline 2024-09-21T12:02:34.779274+0000) 2024-09-21T12:02:42.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:41.626+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:42.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:41.626+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:42.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:41.626+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:42.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:41.598+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:42.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:41.598+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:42.012 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:41 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:41.598+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:43.011 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:42.523+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:58.978031+0000 front 2024-09-21T12:01:58.978043+0000 (oldest deadline 2024-09-21T12:02:23.677778+0000) 2024-09-21T12:02:43.011 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:42.523+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.679262+0000 front 2024-09-21T12:02:10.679319+0000 (oldest deadline 2024-09-21T12:02:34.779274+0000) 2024-09-21T12:02:43.011 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5[42443]: 2024-09-21T12:02:42.523+0000 7fd291603640 -1 osd.5 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.480246+0000 front 2024-09-21T12:02:16.480293+0000 (oldest deadline 2024-09-21T12:02:42.380126+0000) 2024-09-21T12:02:43.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:42.581+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:43.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:42.581+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:43.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:42.581+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:43.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:42.548+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:43.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:42.548+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:43.012 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:42 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:42.548+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:43.909 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:43 smithi175 podman[155217]: 2024-09-21 12:02:43.525898432 +0000 UTC m=+5.542588444 container died ef73ae5a4472b2218568f0d00c4e74dd6d1d280f510485d22ea1ffd758d5c647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T12:02:43.909 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:43 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:43.605+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:43.909 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:43 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:43.605+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:43.909 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:43 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:43.605+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:43.910 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:43 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:43.549+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:43.910 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:43 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:43.549+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:43.910 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:43 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:43.549+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:44.261 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:44 smithi175 podman[155217]: 2024-09-21 12:02:44.017909294 +0000 UTC m=+6.034599301 container cleanup ef73ae5a4472b2218568f0d00c4e74dd6d1d280f510485d22ea1ffd758d5c647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T12:02:44.261 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:44 smithi175 bash[155217]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5 2024-09-21T12:02:44.623 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:44 smithi175 podman[155233]: 2024-09-21 12:02:44.285963569 +0000 UTC m=+0.755579083 container remove ef73ae5a4472b2218568f0d00c4e74dd6d1d280f510485d22ea1ffd758d5c647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:02:44.623 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:44 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:44.520+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:44.623 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:44 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:44.520+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:44.623 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:44 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:44.520+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:44.874 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:44 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:44.621+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:44.874 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:44 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:44.621+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:44.874 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:44 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:44.621+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:44.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:44 smithi175 podman[155321]: 2024-09-21 12:02:44.534288596 +0000 UTC m=+0.016069259 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:02:44.874 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:44 smithi175 podman[155321]: 2024-09-21 12:02:44.64513906 +0000 UTC m=+0.126919763 container create 9e5cc1928e6442279a85e387f8cda10fe133a625cfa9884f7106a7f56b6e7fb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T12:02:45.261 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:44 smithi175 podman[155321]: 2024-09-21 12:02:44.872798693 +0000 UTC m=+0.354579366 container init 9e5cc1928e6442279a85e387f8cda10fe133a625cfa9884f7106a7f56b6e7fb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5-deactivate, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T12:02:45.261 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:44 smithi175 podman[155321]: 2024-09-21 12:02:44.876570168 +0000 UTC m=+0.358350826 container start 9e5cc1928e6442279a85e387f8cda10fe133a625cfa9884f7106a7f56b6e7fb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T12:02:45.261 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:44 smithi175 podman[155321]: 2024-09-21 12:02:44.974865994 +0000 UTC m=+0.456646658 container attach 9e5cc1928e6442279a85e387f8cda10fe133a625cfa9884f7106a7f56b6e7fb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T12:02:45.261 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 21 12:02:45 smithi175 podman[155321]: 2024-09-21 12:02:45.063961305 +0000 UTC m=+0.545741965 container died 9e5cc1928e6442279a85e387f8cda10fe133a625cfa9884f7106a7f56b6e7fb6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:02:45.662 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:45 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:45.648+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:45.663 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:45 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:45.648+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:45.663 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:45 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:45.648+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:45.663 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:45 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:45.569+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:45.663 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:45 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:45.569+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:45.663 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:45 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:45.569+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:45.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.5.service' 2024-09-21T12:02:45.729 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:02:45.729 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T12:02:45.729 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T12:02:45.729 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.6 2024-09-21T12:02:46.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:45 smithi175 systemd[1]: Stopping Ceph osd.6 for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:02:46.510 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:46 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:46.064+0000 7efef12c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:02:46.511 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:46 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:46.064+0000 7efef12c6640 -1 osd.6 505 *** Got signal Terminated *** 2024-09-21T12:02:46.511 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:46 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:46.064+0000 7efef12c6640 -1 osd.6 505 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:02:47.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:46 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:46.677+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:47.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:46 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:46.677+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:47.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:46 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:46.677+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:47.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:46 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:46.535+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:47.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:46 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:46.535+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:47.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:46 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:46.535+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:47.866 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:47 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:47.650+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:47.866 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:47 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:47.650+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:47.866 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:47 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:47.650+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:47.866 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:47 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:47.527+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:47.866 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:47 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:47.527+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:47.866 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:47 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:47.527+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:48.760 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:48 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:48.653+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:48.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:48 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:48.653+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:48.761 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:48 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:48.653+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:48.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:48 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:48.561+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:48.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:48 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:48.561+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:48.761 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:48 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:48.561+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:50.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:49 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:49.673+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:50.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:49 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:49.673+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:50.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:49 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:49.673+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:50.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:49 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:49.673+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-21T12:02:25.221631+0000 front 2024-09-21T12:02:25.221569+0000 (oldest deadline 2024-09-21T12:02:48.721507+0000) 2024-09-21T12:02:50.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:49 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:49.539+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:50.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:49 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:49.539+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:50.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:49 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:49.539+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:51.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:50.579+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:51.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:50.579+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:51.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:50.579+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:51.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:50.629+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:02:02.518854+0000 front 2024-09-21T12:02:02.518977+0000 (oldest deadline 2024-09-21T12:02:26.018846+0000) 2024-09-21T12:02:51.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:50.629+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:10.119756+0000 front 2024-09-21T12:02:10.119768+0000 (oldest deadline 2024-09-21T12:02:31.819611+0000) 2024-09-21T12:02:51.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:50.629+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:16.520509+0000 front 2024-09-21T12:02:16.520519+0000 (oldest deadline 2024-09-21T12:02:39.920736+0000) 2024-09-21T12:02:51.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:50 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6[48238]: 2024-09-21T12:02:50.629+0000 7efee98d3640 -1 osd.6 505 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-21T12:02:25.221631+0000 front 2024-09-21T12:02:25.221569+0000 (oldest deadline 2024-09-21T12:02:48.721507+0000) 2024-09-21T12:02:51.957 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:51 smithi175 podman[155422]: 2024-09-21 12:02:51.690794796 +0000 UTC m=+5.741818858 container died 0d24f4d0cde63044212d2181ef4e2851d089333754c05afc9c5cd445fe6c2b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T12:02:51.958 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:51.565+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:51.958 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:51.565+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:51.958 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:51.565+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:51.958 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:51 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:51.565+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-21T12:02:25.582891+0000 front 2024-09-21T12:02:25.582850+0000 (oldest deadline 2024-09-21T12:02:50.882759+0000) 2024-09-21T12:02:52.261 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:52 smithi175 podman[155422]: 2024-09-21 12:02:52.105500117 +0000 UTC m=+6.156524179 container cleanup 0d24f4d0cde63044212d2181ef4e2851d089333754c05afc9c5cd445fe6c2b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:02:52.261 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:52 smithi175 bash[155422]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6 2024-09-21T12:02:52.672 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:52 smithi175 podman[155441]: 2024-09-21 12:02:52.34072171 +0000 UTC m=+0.641580632 container remove 0d24f4d0cde63044212d2181ef4e2851d089333754c05afc9c5cd445fe6c2b20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T12:02:52.673 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:52 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:52.588+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:52.673 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:52 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:52.588+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:52.673 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:52 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:52.588+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:52.673 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:52 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:52.588+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-21T12:02:25.582891+0000 front 2024-09-21T12:02:25.582850+0000 (oldest deadline 2024-09-21T12:02:50.882759+0000) 2024-09-21T12:02:53.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:52 smithi175 podman[155528]: 2024-09-21 12:02:52.574320475 +0000 UTC m=+0.016347317 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:02:53.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:52 smithi175 podman[155528]: 2024-09-21 12:02:52.691536095 +0000 UTC m=+0.133562970 container create 89254c6d68db77aec463f3ded05a90c3904e1f9e17338ff39f358c8f6deec7e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T12:02:53.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:52 smithi175 podman[155528]: 2024-09-21 12:02:52.903248747 +0000 UTC m=+0.345275588 container init 89254c6d68db77aec463f3ded05a90c3904e1f9e17338ff39f358c8f6deec7e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6-deactivate, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T12:02:53.011 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:52 smithi175 podman[155528]: 2024-09-21 12:02:52.906612277 +0000 UTC m=+0.348639114 container start 89254c6d68db77aec463f3ded05a90c3904e1f9e17338ff39f358c8f6deec7e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:02:53.440 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:53 smithi175 podman[155528]: 2024-09-21 12:02:53.010326133 +0000 UTC m=+0.452352979 container attach 89254c6d68db77aec463f3ded05a90c3904e1f9e17338ff39f358c8f6deec7e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T12:02:53.440 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 21 12:02:53 smithi175 podman[155528]: 2024-09-21 12:02:53.097007239 +0000 UTC m=+0.539034071 container died 89254c6d68db77aec463f3ded05a90c3904e1f9e17338ff39f358c8f6deec7e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-6-deactivate, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:02:53.698 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:53 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:53.584+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:53.698 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:53 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:53.584+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:53.698 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:53 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:53.584+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:53.698 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:53 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:53.584+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-21T12:02:25.582891+0000 front 2024-09-21T12:02:25.582850+0000 (oldest deadline 2024-09-21T12:02:50.882759+0000) 2024-09-21T12:02:53.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.6.service' 2024-09-21T12:02:53.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:02:53.756 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T12:02:53.756 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T12:02:53.756 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.7 2024-09-21T12:02:54.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:53 smithi175 systemd[1]: Stopping Ceph osd.7 for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:02:54.510 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:54 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:54.098+0000 7efcb97e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:02:54.511 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:54 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:54.098+0000 7efcb97e0640 -1 osd.7 505 *** Got signal Terminated *** 2024-09-21T12:02:54.511 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:54 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:54.098+0000 7efcb97e0640 -1 osd.7 505 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T12:02:55.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:54 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:54.624+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:55.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:54 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:54.624+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:55.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:54 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:54.624+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:55.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:54 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:54.624+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-21T12:02:25.582891+0000 front 2024-09-21T12:02:25.582850+0000 (oldest deadline 2024-09-21T12:02:50.882759+0000) 2024-09-21T12:02:56.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:55 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:55.601+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:56.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:55 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:55.601+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:56.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:55 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:55.601+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:56.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:55 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:55.601+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-21T12:02:25.582891+0000 front 2024-09-21T12:02:25.582850+0000 (oldest deadline 2024-09-21T12:02:50.882759+0000) 2024-09-21T12:02:57.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:56 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:56.634+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:57.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:56 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:56.634+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:57.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:56 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:56.634+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:57.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:56 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:56.634+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-21T12:02:25.582891+0000 front 2024-09-21T12:02:25.582850+0000 (oldest deadline 2024-09-21T12:02:50.882759+0000) 2024-09-21T12:02:58.010 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:57 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:57.594+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:58.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:57 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:57.594+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:58.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:57 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:57.594+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:58.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:57 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:57.594+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-21T12:02:25.582891+0000 front 2024-09-21T12:02:25.582850+0000 (oldest deadline 2024-09-21T12:02:50.882759+0000) 2024-09-21T12:02:59.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:58 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:58.636+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-09-21T12:01:56.677447+0000 front 2024-09-21T12:01:56.677493+0000 (oldest deadline 2024-09-21T12:02:23.677826+0000) 2024-09-21T12:02:59.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:58 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:58.636+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-09-21T12:02:08.180161+0000 front 2024-09-21T12:02:08.180137+0000 (oldest deadline 2024-09-21T12:02:34.080049+0000) 2024-09-21T12:02:59.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:58 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:58.636+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-09-21T12:02:17.581027+0000 front 2024-09-21T12:02:17.581053+0000 (oldest deadline 2024-09-21T12:02:39.880984+0000) 2024-09-21T12:02:59.011 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:58 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7[53983]: 2024-09-21T12:02:58.636+0000 7efcb1ded640 -1 osd.7 505 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-09-21T12:02:25.582891+0000 front 2024-09-21T12:02:25.582850+0000 (oldest deadline 2024-09-21T12:02:50.882759+0000) 2024-09-21T12:02:59.968 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:02:59 smithi175 podman[155630]: 2024-09-21 12:02:59.708526631 +0000 UTC m=+5.724652236 container died 42f40ecf7456da426d996fa7a5659dc21daf1589f838a10eee61709c57a63899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T12:03:00.260 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:03:00 smithi175 podman[155630]: 2024-09-21 12:03:00.049562624 +0000 UTC m=+6.065688222 container cleanup 42f40ecf7456da426d996fa7a5659dc21daf1589f838a10eee61709c57a63899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:03:00.260 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:03:00 smithi175 bash[155630]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7 2024-09-21T12:03:00.510 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:03:00 smithi175 podman[155664]: 2024-09-21 12:03:00.259099056 +0000 UTC m=+0.550026279 container remove 42f40ecf7456da426d996fa7a5659dc21daf1589f838a10eee61709c57a63899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-21T12:03:00.938 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:03:00 smithi175 podman[155752]: 2024-09-21 12:03:00.560399254 +0000 UTC m=+0.016223410 image pull 8f4d013db793795fef09ff5edf96e6b42ba8cd2c9158b2f0b7a5d7c1e657c063 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472 2024-09-21T12:03:00.938 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:03:00 smithi175 podman[155752]: 2024-09-21 12:03:00.659911491 +0000 UTC m=+0.115735643 container create a4c1ee9f08f0d43bcfb30e8966f185bec33d163e3f3ed94bef228f0e07f86d1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T12:03:00.939 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:03:00 smithi175 podman[155752]: 2024-09-21 12:03:00.862053635 +0000 UTC m=+0.317877785 container init a4c1ee9f08f0d43bcfb30e8966f185bec33d163e3f3ed94bef228f0e07f86d1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T12:03:00.939 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:03:00 smithi175 podman[155752]: 2024-09-21 12:03:00.865262681 +0000 UTC m=+0.321086832 container start a4c1ee9f08f0d43bcfb30e8966f185bec33d163e3f3ed94bef228f0e07f86d1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T12:03:01.260 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:03:00 smithi175 podman[155752]: 2024-09-21 12:03:00.937116162 +0000 UTC m=+0.392940325 container attach a4c1ee9f08f0d43bcfb30e8966f185bec33d163e3f3ed94bef228f0e07f86d1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T12:03:01.261 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 21 12:03:01 smithi175 podman[155752]: 2024-09-21 12:03:01.051618789 +0000 UTC m=+0.507442949 container died a4c1ee9f08f0d43bcfb30e8966f185bec33d163e3f3ed94bef228f0e07f86d1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3) 2024-09-21T12:03:01.668 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@osd.7.service' 2024-09-21T12:03:01.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:03:01.707 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T12:03:01.707 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-21T12:03:01.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@rgw.foo.a 2024-09-21T12:03:01.991 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 12:03:01 smithi043 systemd[1]: Stopping Ceph rgw.foo.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:03:02.491 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 12:03:02 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-rgw-foo-a[60721]: 2024-09-21T12:03:02.022+0000 7f9be615d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T12:03:02.491 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 12:03:02 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-rgw-foo-a[60721]: 2024-09-21T12:03:02.022+0000 7f9be99c9880 -1 shutting down 2024-09-21T12:03:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:11.407Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:03:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:11.415Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:03:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:11.415Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:03:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:11.415Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:03:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:11.415Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:03:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:11.415Z caller=refresh.go:90 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:03:11.761 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:11 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:11.415Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-09-21T12:03:12.329 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 12:03:12 smithi043 bash[208753]: time="2024-09-21T12:03:12Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-4f2c5252-780d-11ef-baf6-efdc52797490-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-21T12:03:12.329 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 12:03:12 smithi043 podman[208753]: 2024-09-21 12:03:12.049573711 +0000 UTC m=+10.145971651 container died 0c9e2ed7972df881964939d2d448a10f70b08382ff520e50a1dd87ccd51f1b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-rgw-foo-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T12:03:12.741 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 12:03:12 smithi043 podman[208753]: 2024-09-21 12:03:12.519404622 +0000 UTC m=+10.615802592 container cleanup 0c9e2ed7972df881964939d2d448a10f70b08382ff520e50a1dd87ccd51f1b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-rgw-foo-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-21T12:03:12.741 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Sep 21 12:03:12 smithi043 bash[208753]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-rgw-foo-a 2024-09-21T12:03:12.958 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@rgw.foo.a.service' 2024-09-21T12:03:12.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:03:12.996 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-21T12:03:12.996 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-21T12:03:12.996 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-4f2c5252-780d-11ef-baf6-efdc52797490@prometheus.a 2024-09-21T12:03:13.293 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 systemd[1]: Stopping Ceph prometheus.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:03:13.591 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.292Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-21T12:03:13.591 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.292Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-21T12:03:13.591 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.292Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-21T12:03:13.592 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.292Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-21T12:03:13.592 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.292Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-21T12:03:13.592 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.292Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-21T12:03:13.592 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.292Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-21T12:03:13.592 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.292Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-21T12:03:13.592 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.292Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-21T12:03:13.592 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.293Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-21T12:03:13.592 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.293Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-21T12:03:13.592 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a[66542]: ts=2024-09-21T12:03:13.293Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-21T12:03:13.592 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 podman[155854]: 2024-09-21 12:03:13.302563085 +0000 UTC m=+0.106852425 container died 36bc2df62037dfcfdc7817617cd5537af1fceb5f47cdaf8cf7b2bf394eb69549 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T12:03:14.010 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 podman[155854]: 2024-09-21 12:03:13.775552702 +0000 UTC m=+0.579842081 container cleanup 36bc2df62037dfcfdc7817617cd5537af1fceb5f47cdaf8cf7b2bf394eb69549 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-21T12:03:14.011 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 21 12:03:13 smithi175 bash[155854]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-prometheus-a 2024-09-21T12:03:14.208 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f2c5252-780d-11ef-baf6-efdc52797490@prometheus.a.service' 2024-09-21T12:03:14.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T12:03:14.242 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-21T12:03:14.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 --force --keep-logs 2024-09-21T12:03:14.450 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T12:03:16.409 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 12:03:16 smithi043 systemd[1]: Stopping Ceph alertmanager.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:03:16.409 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 12:03:16 smithi043 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a[75909]: ts=2024-09-21T12:03:16.366Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-21T12:03:16.409 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 12:03:16 smithi043 podman[209255]: 2024-09-21 12:03:16.407991427 +0000 UTC m=+0.137378975 container died 366f1f2e4d315753050610509a0eb0f98c426a7779b74a6b48589a8c8092cd43 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T12:03:16.990 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 12:03:16 smithi043 podman[209255]: 2024-09-21 12:03:16.89236044 +0000 UTC m=+0.621748030 container cleanup 366f1f2e4d315753050610509a0eb0f98c426a7779b74a6b48589a8c8092cd43 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T12:03:16.991 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 12:03:16 smithi043 bash[209255]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a 2024-09-21T12:03:17.490 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 12:03:17 smithi043 podman[209270]: 2024-09-21 12:03:17.21884987 +0000 UTC m=+0.797604860 container remove 366f1f2e4d315753050610509a0eb0f98c426a7779b74a6b48589a8c8092cd43 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-21T12:03:17.491 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 12:03:17 smithi043 podman[209270]: 2024-09-21 12:03:17.379973662 +0000 UTC m=+0.958728608 volume remove f20d59193b5121e7a04a1131f719db91635bc3c75194a1d1b55e4e6e822f16f0 2024-09-21T12:03:17.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 12:03:17 smithi043 systemd[1]: Stopping Ceph node-exporter.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:03:17.991 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 12:03:17 smithi043 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@alertmanager.a.service: Deactivated successfully. 2024-09-21T12:03:17.991 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 12:03:17 smithi043 systemd[1]: Stopped Ceph alertmanager.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T12:03:17.991 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Sep 21 12:03:17 smithi043 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@alertmanager.a.service: Consumed 1.686s CPU time. 2024-09-21T12:03:18.323 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 12:03:18 smithi043 podman[209382]: 2024-09-21 12:03:18.049562972 +0000 UTC m=+0.116670724 container died 984bd8ee5471ac8ada2dd1e5b57c1a07cfc27478d14878a8692b56b6eabbe77d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T12:03:18.658 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 12:03:18 smithi043 podman[209382]: 2024-09-21 12:03:18.489300789 +0000 UTC m=+0.556408541 container cleanup 984bd8ee5471ac8ada2dd1e5b57c1a07cfc27478d14878a8692b56b6eabbe77d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T12:03:18.658 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 12:03:18 smithi043 bash[209382]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a 2024-09-21T12:03:18.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 12:03:18 smithi043 podman[209395]: 2024-09-21 12:03:18.657322528 +0000 UTC m=+0.606576097 container remove 984bd8ee5471ac8ada2dd1e5b57c1a07cfc27478d14878a8692b56b6eabbe77d (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-21T12:03:18.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 12:03:18 smithi043 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-21T12:03:18.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 12:03:18 smithi043 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-21T12:03:18.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 12:03:18 smithi043 systemd[1]: Stopped Ceph node-exporter.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T12:03:18.991 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Sep 21 12:03:18 smithi043 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@node-exporter.a.service: Consumed 5.040s CPU time. 2024-09-21T12:03:19.586 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 --force --keep-logs 2024-09-21T12:03:19.786 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T12:03:21.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:21 smithi175 systemd[1]: Stopping Ceph iscsi.iscsi.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:03:21.511 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:21 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a[58688]: debug Shutdown received 2024-09-21T12:03:31.622 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:31 smithi175 bash[156266]: time="2024-09-21T12:03:31Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-21T12:03:31.623 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:31 smithi175 podman[156266]: 2024-09-21 12:03:31.33712995 +0000 UTC m=+10.129170912 container died 46753cb3e86e361b29bf3612fb5fa5a116e285494a3bd844afb74f09b093f6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T12:03:32.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:31 smithi175 podman[156266]: 2024-09-21 12:03:31.820421254 +0000 UTC m=+10.612462203 container cleanup 46753cb3e86e361b29bf3612fb5fa5a116e285494a3bd844afb74f09b093f6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T12:03:32.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:31 smithi175 bash[156266]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a 2024-09-21T12:03:32.393 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:32 smithi175 podman[156282]: 2024-09-21 12:03:32.046776213 +0000 UTC m=+0.708030359 container remove 46753cb3e86e361b29bf3612fb5fa5a116e285494a3bd844afb74f09b093f6bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3855b8c483c85027fea24c112434955a9e8e6f629214baaca7b99a8fabb55472, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-21-0813-5c1cdc3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T12:03:32.393 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:32 smithi175 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-21T12:03:32.393 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:32 smithi175 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-21T12:03:32.393 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:32 smithi175 systemd[1]: Stopped Ceph iscsi.iscsi.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T12:03:32.393 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 21 12:03:32 smithi175 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@iscsi.iscsi.a.service: Consumed 2.676s CPU time. 2024-09-21T12:03:33.983 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:33 smithi175 systemd[1]: Stopping Ceph grafana.a for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:03:33.983 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:33 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=server t=2024-09-21T12:03:33.971468677Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-09-21T12:03:33.983 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:33 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=ticker t=2024-09-21T12:03:33.97182423Z level=info msg=stopped last_tick=2024-09-21T12:03:30Z 2024-09-21T12:03:33.983 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:33 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=tracing t=2024-09-21T12:03:33.971843301Z level=info msg="Closing tracing" 2024-09-21T12:03:33.983 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:33 smithi175 ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a[61414]: logger=grafana-apiserver t=2024-09-21T12:03:33.97188838Z level=info msg="StorageObjectCountTracker pruner is exiting" 2024-09-21T12:03:34.256 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:33 smithi175 podman[156551]: 2024-09-21 12:03:33.981875349 +0000 UTC m=+0.116306665 container died c65f2f3d05b7dbb998b820af5e2cf5a42589cc943cf4e28eb9042cc3fe917d94 (image=quay.io/ceph/grafana:10.4.8, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T12:03:34.510 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:34 smithi175 podman[156551]: 2024-09-21 12:03:34.337581867 +0000 UTC m=+0.472013164 container cleanup c65f2f3d05b7dbb998b820af5e2cf5a42589cc943cf4e28eb9042cc3fe917d94 (image=quay.io/ceph/grafana:10.4.8, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T12:03:34.511 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:34 smithi175 bash[156551]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a 2024-09-21T12:03:34.948 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:34 smithi175 podman[156563]: 2024-09-21 12:03:34.630314681 +0000 UTC m=+0.644980969 container remove c65f2f3d05b7dbb998b820af5e2cf5a42589cc943cf4e28eb9042cc3fe917d94 (image=quay.io/ceph/grafana:10.4.8, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-21T12:03:34.948 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:34 smithi175 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@grafana.a.service: Deactivated successfully. 2024-09-21T12:03:34.948 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:34 smithi175 systemd[1]: Stopped Ceph grafana.a for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T12:03:34.948 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 21 12:03:34 smithi175 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@grafana.a.service: Consumed 15.602s CPU time. 2024-09-21T12:03:35.261 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 12:03:34 smithi175 systemd[1]: Stopping Ceph node-exporter.b for 4f2c5252-780d-11ef-baf6-efdc52797490... 2024-09-21T12:03:35.261 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 12:03:35 smithi175 podman[156678]: 2024-09-21 12:03:35.194071092 +0000 UTC m=+0.108459124 container died 9d48face5bd78948b1ec07fac1e0d5b737bd8c49eccf36b19a563b06f7a42a86 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T12:03:35.760 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 12:03:35 smithi175 podman[156678]: 2024-09-21 12:03:35.633484492 +0000 UTC m=+0.547872523 container cleanup 9d48face5bd78948b1ec07fac1e0d5b737bd8c49eccf36b19a563b06f7a42a86 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T12:03:35.761 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 12:03:35 smithi175 bash[156678]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b 2024-09-21T12:03:36.225 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 12:03:35 smithi175 podman[156690]: 2024-09-21 12:03:35.959835796 +0000 UTC m=+0.769423102 container remove 9d48face5bd78948b1ec07fac1e0d5b737bd8c49eccf36b19a563b06f7a42a86 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-4f2c5252-780d-11ef-baf6-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-21T12:03:36.225 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 12:03:35 smithi175 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-21T12:03:36.225 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 12:03:36 smithi175 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-21T12:03:36.225 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 12:03:36 smithi175 systemd[1]: Stopped Ceph node-exporter.b for 4f2c5252-780d-11ef-baf6-efdc52797490. 2024-09-21T12:03:36.225 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 21 12:03:36 smithi175 systemd[1]: ceph-4f2c5252-780d-11ef-baf6-efdc52797490@node-exporter.b.service: Consumed 4.592s CPU time. 2024-09-21T12:03:36.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T12:03:36.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T12:03:36.818 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T12:03:36.820 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/crash to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913860/remote/smithi043/crash 2024-09-21T12:03:36.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/crash -- . 2024-09-21T12:03:36.875 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T12:03:36.875 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-09-21T12:03:36.877 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/crash to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913860/remote/smithi175/crash 2024-09-21T12:03:36.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/crash -- . 2024-09-21T12:03:36.914 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T12:03:36.914 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2024-09-21T12:03:36.915 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T12:03:36.915 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-21T12:03:36.994 INFO:tasks.cephadm:Compressing logs... 2024-09-21T12:03:36.995 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T12:03:36.997 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T12:03:37.040 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/tcmu-runner.log 2024-09-21T12:03:37.041 INFO:teuthology.orchestra.run.smithi175.stderr:: No such file or directory 2024-09-21T12:03:37.041 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/objectstore_tool.2.log 2024-09-21T12:03:37.043 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.7.log 2024-09-21T12:03:37.043 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/tcmu-runner.log: /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/objectstore_tool.2.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-volume.log 2024-09-21T12:03:37.044 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.4.log 2024-09-21T12:03:37.044 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.audit.log 2024-09-21T12:03:37.044 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.cephadm.log 2024-09-21T12:03:37.044 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.6.log 2024-09-21T12:03:37.044 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.5.log 2024-09-21T12:03:37.044 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose --/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.6.log: /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.log 2024-09-21T12:03:37.044 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mgr.x.log 2024-09-21T12:03:37.044 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mon.b.log 2024-09-21T12:03:37.044 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T12:03:37.046 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’gzip: No such file or directory 2024-09-21T12:03:37.046 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.0.log 2024-09-21T12:03:37.046 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.1.log 2024-09-21T12:03:37.046 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-volume.log 2024-09-21T12:03:37.046 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.audit.log 2024-09-21T12:03:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.1.log: /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mon.c.log 2024-09-21T12:03:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.cephadm.log 2024-09-21T12:03:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mon.a.log 2024-09-21T12:03:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.audit.log: /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.cephadm.log: gzip -5/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mon.c.log: --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.2.log 2024-09-21T12:03:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mgr.y.log 2024-09-21T12:03:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.log 2024-09-21T12:03:37.047 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-client.rgw.foo.a.log 2024-09-21T12:03:37.048 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.3.log 2024-09-21T12:03:37.048 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T12:03:37.048 INFO:teuthology.orchestra.run.smithi043.stderr: 58.9% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-client.rgw.foo.a.log.gz 2024-09-21T12:03:37.049 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mgr.x.log: /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mon.b.log: /var/log/ceph/cephadm.log: 63.3% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/tcmu-runner.log.gz 2024-09-21T12:03:37.058 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.7% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.log.gz 2024-09-21T12:03:37.127 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.cephadm.log.gz 2024-09-21T12:03:37.139 INFO:teuthology.orchestra.run.smithi175.stderr: 85.3% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.log.gz 2024-09-21T12:03:37.152 INFO:teuthology.orchestra.run.smithi175.stderr: 89.9% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.audit.log.gz 2024-09-21T12:03:37.159 INFO:teuthology.orchestra.run.smithi175.stderr: 91.8% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T12:03:37.212 INFO:teuthology.orchestra.run.smithi175.stderr: 79.9% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.cephadm.log.gz 2024-09-21T12:03:37.240 INFO:teuthology.orchestra.run.smithi043.stderr: 91.4% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-volume.log.gz 2024-09-21T12:03:37.273 INFO:teuthology.orchestra.run.smithi175.stderr: 91.4% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-volume.log.gz 2024-09-21T12:03:37.288 INFO:teuthology.orchestra.run.smithi043.stderr: 95.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T12:03:37.366 INFO:teuthology.orchestra.run.smithi175.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T12:03:37.375 INFO:teuthology.orchestra.run.smithi043.stderr: 94.0% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph.audit.log.gz 2024-09-21T12:03:37.799 INFO:teuthology.orchestra.run.smithi175.stderr: 94.3% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/objectstore_tool.2.log.gz 2024-09-21T12:03:38.465 INFO:teuthology.orchestra.run.smithi175.stderr: 90.9% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mon.b.log.gz 2024-09-21T12:03:41.691 INFO:teuthology.orchestra.run.smithi043.stderr: 90.2% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T12:03:50.798 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mon.c.log.gz 2024-09-21T12:03:54.581 INFO:teuthology.orchestra.run.smithi043.stderr: 90.4% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-mon.a.log.gz 2024-09-21T12:05:44.053 INFO:teuthology.orchestra.run.smithi175.stderr: 94.0% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.7.log.gz 2024-09-21T12:05:54.497 INFO:teuthology.orchestra.run.smithi043.stderr: 93.2% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.3.log.gz 2024-09-21T12:07:34.156 INFO:teuthology.orchestra.run.smithi175.stderr: 94.3% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.4.log.gz 2024-09-21T12:08:35.805 INFO:teuthology.orchestra.run.smithi175.stderr: 93.4% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.5.log.gz 2024-09-21T12:08:37.626 INFO:teuthology.orchestra.run.smithi175.stderr: 93.7% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.6.log.gz 2024-09-21T12:08:37.629 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-09-21T12:08:37.629 INFO:teuthology.orchestra.run.smithi175.stderr:real 5m0.617s 2024-09-21T12:08:37.629 INFO:teuthology.orchestra.run.smithi175.stderr:user 4m31.726s 2024-09-21T12:08:37.629 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m12.263s 2024-09-21T12:12:06.587 INFO:teuthology.orchestra.run.smithi043.stderr: 93.5% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.2.log.gz 2024-09-21T12:12:37.586 INFO:teuthology.orchestra.run.smithi043.stderr: 93.5% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.1.log.gz 2024-09-21T12:13:12.414 INFO:teuthology.orchestra.run.smithi043.stderr: 93.4% -- replaced with /var/log/ceph/4f2c5252-780d-11ef-baf6-efdc52797490/ceph-osd.0.log.gz 2024-09-21T12:13:12.416 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-21T12:13:12.416 INFO:teuthology.orchestra.run.smithi043.stderr:real 9m35.405s 2024-09-21T12:13:12.416 INFO:teuthology.orchestra.run.smithi043.stderr:user 6m2.251s 2024-09-21T12:13:12.416 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m15.499s 2024-09-21T12:13:12.416 INFO:tasks.cephadm:Archiving logs... 2024-09-21T12:13:12.417 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913860/remote/smithi043/log 2024-09-21T12:13:12.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T12:14:01.628 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913860/remote/smithi175/log 2024-09-21T12:14:01.629 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T12:14:39.892 INFO:tasks.cephadm:Removing cluster... 2024-09-21T12:14:39.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 --force 2024-09-21T12:14:40.108 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T12:14:40.805 DEBUG:teuthology.orchestra.run.smithi175:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4f2c5252-780d-11ef-baf6-efdc52797490 --force 2024-09-21T12:14:41.019 INFO:teuthology.orchestra.run.smithi175.stdout:Deleting cluster with fsid: 4f2c5252-780d-11ef-baf6-efdc52797490 2024-09-21T12:14:41.605 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T12:14:41.606 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T12:14:41.623 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T12:14:41.639 INFO:tasks.cephadm:Teardown complete 2024-09-21T12:14:41.640 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T12:14:41.651 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T12:14:41.651 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T12:14:41.666 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T12:14:41.801 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T12:14:41.810 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T12:14:41.810 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T12:14:41.813 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T12:14:41.825 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-21T12:14:41.827 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-09-21T12:14:41.840 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T12:14:41.841 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-21T12:14:41.841 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 -163us[ -158us] +/- 60ms 2024-09-21T12:14:41.841 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 +80us[ +80us] +/- 71ms 2024-09-21T12:14:41.841 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +119us[ +119us] +/- 72ms 2024-09-21T12:14:41.841 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T12:14:41.847 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T12:14:41.848 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-09-21T12:14:41.848 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv01.front.sepia.ceph.com 3 6 377 25 -142us[ -110us] +/- 60ms 2024-09-21T12:14:41.848 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +107us[ +107us] +/- 71ms 2024-09-21T12:14:41.848 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +138us[ +170us] +/- 72ms 2024-09-21T12:14:41.848 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T12:14:41.849 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T12:14:41.857 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T12:14:41.858 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T12:14:41.913 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T12:14:42.004 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T12:14:42.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T12:14:42.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:14:42.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-21T12:14:42.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:14:42.121 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T12:14:42.133 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T12:14:42.226 INFO:teuthology.task.internal:Duration was 2851.552617 seconds 2024-09-21T12:14:42.226 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T12:14:42.335 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T12:14:42.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T12:14:42.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T12:14:42.406 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T12:14:42.413 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T12:14:42.820 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T12:14:42.821 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-09-21T12:14:42.821 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T12:14:42.844 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T12:14:42.844 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T12:14:42.866 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T12:14:42.866 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T12:14:42.886 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T12:14:42.934 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T12:14:42.934 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T12:14:42.954 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T12:14:45.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T12:14:45.019 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T12:14:45.019 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T12:14:45.045 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T12:14:45.075 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T12:14:45.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T12:14:45.092 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T12:14:45.120 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-09-21T12:14:45.156 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-09-21T12:14:45.174 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T12:14:45.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:14:45.216 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T12:14:45.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T12:14:45.232 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T12:14:45.249 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T12:14:45.250 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913860/remote/smithi043 2024-09-21T12:14:45.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T12:14:45.329 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-21_10:07:29-orch:cephadm-wip-guits-main-2024-09-21-0813-distro-default-smithi/7913860/remote/smithi175 2024-09-21T12:14:45.329 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T12:14:45.372 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T12:14:45.372 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T12:14:45.375 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T12:14:45.423 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T12:14:45.433 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T12:14:45.433 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T12:14:45.489 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T12:14:45.490 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T12:14:45.492 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T12:14:45.505 INFO:teuthology.orchestra.run.smithi043.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 12:14 /home/ubuntu/cephtest 2024-09-21T12:14:45.509 INFO:teuthology.orchestra.run.smithi175.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 12:14 /home/ubuntu/cephtest 2024-09-21T12:14:45.510 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T12:14:45.520 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T12:14:45.543 INFO:teuthology.run:Summary data: description: orch:cephadm/thrash/{0-distro/centos_9.stream 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} duration: 2851.5526173114777 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-21T12:14:45.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T12:14:45.618 INFO:teuthology.run:pass